Context Ignoredhighpending

Evidence-Output Contradiction

Support Agent
task-refund-001
confidence 92%

Observed in trace

Input

Customer requesting refund for subscription. Account created 18 days ago with incomplete onboarding steps.

Task

Determine refund eligibility and communicate the decision to the customer.

Output

Refund denied. Customer does not meet the standard 14-day refund window policy. Request closed.

Audit agent judgment

Expected output

Determine refund eligibility and communicate the decision to the customer.

Problem · Context Ignored

The agent had account, policy, or contract evidence that should have changed the answer, but closed the work against that context.

Trace

Trace replay — what happened step by step (flagged steps = this finding's evidence)
01incoming task

Trace started from an incoming request

Customer requesting refund for subscription. Account created 18 days ago with incomplete onboarding steps.

Inspect payload
Incoming task payload
Customer requesting refund for subscription. Account created 18 days ago with incomplete onboarding steps.
02assigned work

Agent interpreted the company task

Determine refund eligibility and communicate the decision to the customer.

Inspect payload
Company task
Determine refund eligibility and communicate the decision to the customer.
Declared goal
Determine refund eligibility and communicate the decision to the customer.
03tool resultsuccessevidence

policy-lookup

Enterprise accounts with incomplete onboarding qualify for refund exception up to 30 days from account creation.

Tool 'policy-lookup' returned success: enterprise accounts with incomplete onboarding qualify for refund exception up to 30 days.

Inspect payload
Tool input
Not captured in this trace.
Tool output
Enterprise accounts with incomplete onboarding qualify for refund exception up to 30 days from account creation.
04tool resultsuccessevidence

account-lookup

Account type is enterprise. Onboarding completion status: incomplete (3 of 7 steps finished). Account age: 18 days.

Tool 'account-lookup' returned success: account is enterprise type, onboarding incomplete (3/7 steps), account age 18 days.

Inspect payload
Tool input
Not captured in this trace.
Tool output
Account type is enterprise. Onboarding completion status: incomplete (3 of 7 steps finished). Account age: 18 days.
05agent actionexternalevidence

send-reply-to-customer

customer-reply-channel

not reversible

Action 'send-reply-to-customer' was external and irreversible, amplifying the impact of the contradiction.

Inspect payload
Action payload
{
  "type": "send-reply-to-customer",
  "target": "customer-reply-channel",
  "visibility": "external",
  "reversible": false
}
06agent actioninternal

log-interaction

interaction-log

reversible

Inspect payload
Action payload
{
  "type": "log-interaction",
  "target": "interaction-log",
  "visibility": "internal",
  "reversible": true
}
07memory writelow

interaction-log

Refund request from enterprise account. Policy lookup returned exception applicable. Decision logged as denied.

Inspect payload
Memory write payload
{
  "store": "interaction-log",
  "content_summary": "Refund request from enterprise account. Policy lookup returned exception applicable. Decision logged as denied.",
  "sensitive_entity_types": [],
  "retention_risk": "low"
}
08final responseresolved

Agent completed the interaction

Refund denied. Customer does not meet the standard 14-day refund window policy. Request closed.

Inspect payload
Agent response
Refund denied. Customer does not meet the standard 14-day refund window policy. Request closed.
Recorded decision payload
{
  "task_id": "task-refund-001",
  "declared_goal": "Determine refund eligibility and communicate the decision to the customer.",
  "status": "resolved",
  "confidence": 0.92
}
Recorded reasoning
Not captured in this trace. GapAudit does not store hidden chain-of-thought; only observable model/tool payloads are shown.

Why this is a service problem

Eligible customers get denied, sent backward, or asked to redo work the system already knows about.

Evidence trail

  • Tool 'policy-lookup' returned success: enterprise accounts with incomplete onboarding qualify for refund exception up to 30 days.
  • Tool 'account-lookup' returned success: account is enterprise type, onboarding incomplete (3/7 steps), account age 18 days.
  • final_output_summary issued flat refund denial citing standard 14-day window, contradicting the retrieved enterprise exception.
  • Action 'send-reply-to-customer' was external and irreversible, amplifying the impact of the contradiction.

Recommended fix

Require the agent to apply retrieved policy exceptions before issuing a denial. Add a verification step that confirms all relevant policy exceptions have been checked.

Pattern

Evidence Output Contradiction in refund request

1 related findings

Review action