Recurring pattern
Evidence Output Contradiction in refund request
The agent had account, policy, or contract evidence that should have changed the answer, but closed the work against that context.
Affected agent
Support Agent
Occurrences
1
Trend
stable
Converted to evals
0 / 1
Recommended action
Require the agent to apply retrieved policy exceptions before issuing a denial. Add a verification step that confirms all relevant policy exceptions have been checked.
Promote to a regression eval →Why this is a pattern, not a one-off
Stable — recurring (1 traces) but not accelerating. Still worth a systemic fix.
Eligible customers get denied, sent backward, or asked to redo work the system already knows about.
Occurrences in this pattern (1)
Open as gap list →| Severity | Failure mode | Evidence | Trace |
|---|---|---|---|
| high | Evidence-Output Contradiction | Tool 'policy-lookup' returned success: enterprise accounts with incomplete onboarding qualify for refund exception up to 30 days. | Read trace |
Time window
2026-06-11T05:47:00.491Z/2026-06-11T05:47:00.491Z
Failure modes (1)
Shared evidence signals