Home/ Part VIII — Retrieval, Grounding, and "Don't Make Stuff Up" Engineering/26. Guardrails for Grounded Systems/26.4 Refusal and escalation flows

26.4 Refusal and escalation flows

Overview and links for this section of the guide.

On this page

Goal: handle “can’t answer” states safely

A grounded system must behave well when it can’t answer:

  • because evidence is missing (not found),
  • because sources conflict (conflict),
  • because the user lacks permission (restricted),
  • because policy or safety filters refuse the request (refused),
  • because the question is ambiguous (needs clarification).

Refusal and escalation flows are what prevent users from learning to distrust your product.

Types of “can’t answer” outcomes

  • Not found: corpus doesn’t contain the answer.
  • Needs clarification: multiple interpretations; must ask a follow-up.
  • Conflict: sources disagree.
  • Restricted: user isn’t allowed to see relevant sources.
  • Refused: request violates policies or safety constraints.

Each outcome needs a different UX and next action.

Good systems “route” instead of “answer”

Sometimes the best outcome is: create a ticket, notify an owner, or point to the correct human process.

UX: refusal-aware and escalation-aware behavior

Design goals:

  • Be transparent: say why you can’t answer (as much as policy allows).
  • Be helpful: provide the next best action.
  • Don’t leak: don’t reveal restricted content via explanations.

Example next actions:

  • Not found: ask user for the missing doc or suggest where to search.
  • Clarification: ask one targeted question.
  • Conflict: show conflicting quotes and ask which policy/version to use.
  • Restricted: instruct user to request access or contact an owner.
  • Refused: explain refusal and offer a safe alternative request.

A practical escalation workflow

Escalation should be a pipeline, not an ad-hoc suggestion:

  1. Classify the failure state: not found vs conflict vs restricted vs refused.
  2. Collect evidence: store question, retrieved chunk ids, and why it failed.
  3. Route: choose the right owner/team based on doc type or topic tags.
  4. Create artifact: open a ticket with the evidence and a suggested resolution.
  5. Close the loop: when resolved, update corpus/docs and add an eval case.

Schema for refusal/escalation

{
  "status": "refused" | "restricted" | "not_found" | "needs_clarification" | "conflict",
  "user_message": string,
  "safe_alternative": string|null,
  "escalation": {
    "recommended": boolean,
    "target_team": string|null,
    "ticket_summary": string|null,
    "evidence": { "retrieved_chunk_ids": string[] }
  }
}

Copy-paste prompts

Prompt: generate an escalation ticket (evidence-only)

Create an escalation ticket draft based on this failed Q&A attempt.

Rules:
- Do not invent answers.
- Include the user question, failure status, and retrieved chunk ids.
- Suggest what info is missing or what conflict must be resolved.

Inputs:
- Question: ...
- Status: not_found | conflict | restricted | refused | needs_clarification
- Retrieved chunks: [ids + titles + versions]

Output:
- Ticket title
- Ticket description (context, evidence, requested decision)

Where to go next