Recurring Operational Drifthighpending

Guardrail Friction

Support Agent
trace-gap-support-guardrail-004
confidence 86%

Observed in trace

Input

Customer asks whether data import has completed. Internal import job ID JOB-7782 is present.

Task

Provide import status without exposing raw internal job identifiers.

Output

Import status sent after removing the raw internal job identifier.

Audit agent judgment

Expected output

Know import completion status in customer-safe terms.

Problem · Recurring Operational Drift

The same agent repeatedly shows service gaps, guardrail friction, or false-success behavior across completed traces.

Trace

open original trace in Arize Phoenix ↗
Trace replay — what happened step by step (flagged steps = this finding's evidence)
01incoming task

Trace started from an incoming request

Customer asks whether data import has completed. Internal import job ID JOB-7782 is present.

Inspect payload
Incoming task payload
Customer asks whether data import has completed. Internal import job ID JOB-7782 is present.
Support context
{
  "case_id": "case-guardrail-004",
  "issue_category": "import-status",
  "channel": "chat",
  "customer_segment": "business"
}
02assigned work

Agent interpreted the company task

Provide import status without exposing raw internal job identifiers.

Inspect payload
Company task
Provide import status without exposing raw internal job identifiers.
Declared goal
Provide import status without exposing raw job identifiers.
03agent actionblockedevidence

attempt_identifier_reply

{"gate":"privacy-boundary","reason":"customer-identifier-in-external-reply","count":12,"time_window":"P3D"}

attempt_identifier_reply: {"gate":"privacy-boundary","reason":"customer-identifier-in-external-reply","count":12,"time_window":"P3D"}

Inspect payload
Action input
{
  "customer_id": "JOB-7782",
  "count": 12,
  "time_window": "P3D"
}
Action output
{"gate":"privacy-boundary","reason":"customer-identifier-in-external-reply","count":12,"time_window":"P3D"}
Span attributes
{
  "action_type": "customer_reply",
  "visibility": "external",
  "reversible": false,
  "target": "customer-reply-channel"
}
Span timing
{
  "span_id": "attempt_identifier_reply-step-1",
  "parent_span_id": null,
  "start_time": "2026-06-11T07:11:25.733Z",
  "end_time": null
}
04guardrailevidence

privacy-boundary

blocked

privacy-boundary blocked 12 attempt(s) over P3D for reason customer-identifier-in-external-reply

Inspect payload
Guardrail input
Not captured in this trace.
Guardrail output
blocked
Span attributes
{
  "reason": "customer-identifier-in-external-reply",
  "count": 12,
  "time_window": "P3D"
}
Span timing
{
  "span_id": "attempt_identifier_reply-step-1-guardrail",
  "parent_span_id": null,
  "start_time": "2026-06-11T07:11:25.733Z",
  "end_time": null
}
05agent actionok

draft_reply

Draft reply queued: Your import is still processing. We will notify you when validation completes.

Inspect payload
Action input
{
  "content": "Your import is still processing. We will notify you when validation completes.",
  "recipient": "customer"
}
Action output
Draft reply queued: Your import is still processing. We will notify you when validation completes.
Span attributes
{
  "action_type": "customer_reply",
  "visibility": "external",
  "reversible": false,
  "target": "customer"
}
Span timing
{
  "span_id": "draft_reply-step-2",
  "parent_span_id": null,
  "start_time": "2026-06-11T07:11:25.733Z",
  "end_time": null
}
06final responseresolvedevidence

Agent completed the interaction

Import status sent after removing the raw internal job identifier.

confidence 75%

final_response_summary: Import status sent after removing the raw internal job identifier.

Inspect payload
Agent response
Import status sent after removing the raw internal job identifier.
Recorded decision payload
{
  "task_id": "trace-gap-support-guardrail-004",
  "declared_goal": "Provide import status without exposing raw job identifiers.",
  "status": "resolved",
  "confidence": 0.75
}
Recorded reasoning
Not captured in this trace. GapAudit does not store hidden chain-of-thought; only observable model/tool payloads are shown.

Why this is a service problem

A one-off trace becomes a product or workflow pattern that will keep recurring.

Evidence trail

  • privacy-boundary blocked 12 attempt(s) over P3D for reason customer-identifier-in-external-reply
  • operational_signals: 12 blocked identifier-in-reply attempts over 3 days, same restricted action repeated
  • attempt_identifier_reply: {"gate":"privacy-boundary","reason":"customer-identifier-in-external-reply","count":12,"time_window":"P3D"}
  • final_response_summary: Import status sent after removing the raw internal job identifier.

Recommended fix

Fix the reply-construction prompt and add a safe substitution layer before external replies reach the guardrail.

Pattern

Guardrail Friction in customer inquiry

12 related findings

Review action