Why does my AI agent make up ticket numbers and tracking links?

Because the prompt showed it the shape and never told it where the value comes from. A line like A case reference looks like HG-48210, or a few-shot reply containing HG-48210, is not read as documentation: it is a filled-in template, and filling in templates is the whole job. The model has no way to distinguish an identifier you meant as an illustration from one you meant as a value, because nothing in the prompt marks the difference. The tell is that the invented reference is always well formed. It matches your prefix, your digit count and your casing, which is exactly why it survives QA and reaches the customer.

How do I stop an AI support agent from inventing order or case numbers?

Take every identifier literal out of the prompt and make the identifier a tool result instead. Three edits do it. Declare the tool that produces the value with $TOOL and point at it with @[open_case] rather than naming it in bare prose. Write the rules as a sequence and a prohibition, ALWAYS call @[open_case] before you give a customer a case reference and NEVER write a case reference that @[open_case] did not return. Then delete the format line and the example that carried the sample value, because a shape is a generator whatever section it sits in. On the prompt in this post that took the built artifact from 599 bytes carrying HG-48210 four times to 372 bytes carrying it zero times, and the score from B (80/100) to A (93/100).

Does TypeGlish catch a made-up identifier in a prompt?

No, and it is worth knowing exactly where the blind spot is. There is no diagnostic in the language that reads an identifier-shaped literal, in any section: a constant in a rule, a format line under # Output, a - good:: line in an $EXAMPLE and a zone all check at 0 error, 0 warning. The one spelling that does block is @{case_reference} with nothing behind it, which is 1 error structure/undefined-ref. So the checker refuses the honest placeholder and ships the fake constant. The gate for this defect is a $TEST with a deterministic assert, matches /^(?!.*HG-)/, plus a grep of the source for your own identifier prefixes before merge.

Should I put example replies in an agent system prompt at all?

Yes, but not with real-looking values in them. An example is the strongest instruction in a prompt because it is a demonstration rather than a description, which is the same reason a demonstrated bad response invites imitation and why $EXAMPLE holds - bad:: out of the compiled artifact entirely. Keep the exchange and drop the data: show the sentence pattern, and refer to the value rather than printing one, as in the reference that the lookup returned. If you genuinely need a filled example, use a value that cannot be mistaken for production, and pin the absence of that value with a test assert so a later edit cannot quietly put it back.

← Tidelines/Best practicesSep 3, 2026

Why your agent invents a case reference

The bot hands the customer a case number with the right prefix, the right digit count and the right casing. It belongs to nothing. The prompt taught it the shape, and a shape is a generator.

by TypeGlish team9 min read#best-practices

HG-48210 was never a case.

TL;DR An identifier written anywhere in your prompt is a template the model will fill, and TypeGlish cannot see it: eight spellings of give the customer a case reference produce exactly one blocking error, and it lands on @{case_reference}, the only honest spelling in the set. Make the identifier a tool result instead, and the built artifact goes from 599 bytes carrying HG-48210 four times to 372 bytes carrying it zero times, B (80/100) to A (93/100).

This one gets escalated as a hallucination and it is not really a hallucination, or at least not the interesting kind. The agent did not invent a fact about the world. It filled in a form you gave it. Somewhere in the prompt is a sample identifier, and the model, which is a completion engine wearing a support-agent costume, completed it. The customer writes the number down. Two days later they quote it back to a human, the human searches for it, and it is not there.

§1The shape is the payload

What makes this defect expensive is not how wrong the output is. It is how right it looks. An agent that says your parcel will arrive Thursday gets caught by a reviewer in the first hour, because a promised date is the sort of thing everyone is watching for. An agent that says I have opened case HG-48210 for you passes every review anyone runs, because HG-48210 is indistinguishable from a real reference in every respect except existing.

It is also the failure most likely to reach a customer rather than a dashboard. A fabricated reference does not fail loudly at the moment of generation. It fails four steps later, in a different channel, in front of a different person, when the number turns out to key nothing. By then the transcript has been closed and nobody links the two events.

The same mechanism produces the other three members of this family: a tracking URL with your real domain and a fake path, a support phone number that is one digit off the real one, and a knowledge-base link shaped exactly like your knowledge-base links. Every one of them is a shape somebody typed into the prompt as an illustration.

§2The prompt that taught it

Here is a small delivery-support prompt of the kind that gets written in an afternoon. Nothing in it is careless. The # Output section documents the format so the agent knows what a reference looks like, and the # Examples section shows one good exchange, which is standard advice everywhere.

parcel.tg - the prompt as written✓ builds

# Role
You are a delivery support agent for Harbour Goods.

# Constraints
- NEVER state a delivery date.
- ALWAYS give a customer a case reference before you close a contact.
- ALWAYS give a customer a tracking link for a delayed parcel.
- IF a parcel is more than 10 days late THEN escalate the contact to the warehouse team.

# Output
A case reference looks like HG-48210.
A tracking link looks like https://harbourgoods.example/track/HG-48210.

# Examples
Customer: Where is my parcel?
Agent: I have opened case HG-48210 for you. You can follow it at https://harbourgoods.example/track/HG-48210.

tg check, --strict and score - parcel.tg✗ B (80/100)

$ typeglish check parcel.tg
parcel.tg:8:1  info   prompt/unintroduced-definite  "the warehouse team" retrieves something this
  document never introduces - a model must guess which team is meant.
parcel.tg:16:1  info   prompt/unregistered-doer  A bare generic doer never enters the world
  model - instruction to the agent, or background about users?

✓ 1 file - 0 error, 0 warning, 2 info

$ typeglish check parcel.tg --strict
✓ 1 file - 0 error, 0 warning, 2 info

$ typeglish score parcel.tg
parcel.tg - B (80/100)  proven errors: none  tiers: base+z3
  planes  runtime 90 (what the model reads) · hygiene 50 (source only)
  facets  enforceability 77 x.21 · hardness 100 x.12 · directness 87 x.08
          consistency 100 x.17 · structure 100 x.12 · annotation 0 x.12
          style 83 x.08 · security 100 x.08
  lever   annotation 0/100 (up to +12 overall)
  L8   prompt/unintroduced-definite  -0.25
  L16  prompt/unregistered-doer      -0.25

Two info findings, neither about the identifier. --strict escalates nothing, because there is nothing here to escalate. The two rows the scorer docks are a definite noun and a bare doer; HG-48210 costs zero points in every facet.

Now look at what the model actually receives, because that is the document that matters:

tg build parcel.tg - the artifact

$ typeglish build parcel.tg
✓ built .typeglish/dist/parcel.txt ← parcel.tg (82e5dd44f895, full)

$ wc -c < .typeglish/dist/parcel.txt
599
$ grep -o 'HG-48210' .typeglish/dist/parcel.txt | wc -l
4

Four copies of a case reference that does not exist, in a 599-byte prompt. Two of them sit inside a worked example, which is the highest-authority text in the file: a demonstration beats a description every time, which is the reason $EXAMPLE holds its - bad:: line out of the artifact entirely. The # Examples prose block has no such protection, and it also reflows, so the two-line exchange arrives as one paragraph.

You cannot show a model a filled-in form and expect it to read the form and ignore the filling.

§3Eight spellings, one error, and it is on the wrong one

The natural next question is where in the file the identifier is safe. The answer is nowhere, and the way to see it is to write the same instruction eight ways and check them in one run.

tg check - eight spellings of one rule✗ 1 error

$ typeglish check m1_constant.tg m2_brace.tg m3_pointer.tg m4_format.tg \
    m5_example.tg m6_declared.tg m7_url.tg m8_phone.tg
m3_pointer.tg:5:41  error  structure/undefined-ref  Dangling pointer - @{case_reference}
  names nothing. Declare it with "$REQUIRE variable case_reference" (a runtime
  variable), or "case_reference IS ..." (a fixed value).
m5_example.tg:7:1  info   prompt/unregistered-doer  A bare generic doer never enters ...
m7_url.tg:5:1  info   prompt/unintroduced-definite  "the tracking link" retrieves ...
m8_phone.tg:5:1  info   prompt/unintroduced-definite  "the support number" retrieves ...

✗ 8 files - 1 error, 0 warning, 3 info    # exit 1

One blocking error in eight files, and it is on m3: the version that admits the value comes from somewhere else.

the eight files, read as a table

file          how the reference is written              tg check
m1_constant   case reference HG-48210                   0 error, 0 warning, 0 info
m2_brace      case reference {case_reference}           0 error, 0 warning, 0 info
m3_pointer    case reference @{case_reference}          1 error  structure/undefined-ref
m4_format     # Output: a reference looks like HG-48210 0 error, 0 warning, 0 info
m5_example    $EXAMPLE - good:: case HG-48210           0 error, 0 warning  1 info (doer)
m6_declared   $REQUIRE variable + @{case_reference}      0 error, 0 warning, 0 info
m7_url        https://harbourgoods.example/track/HG-...  0 error, 0 warning  1 info (definite)
m8_phone      the support number 0800 118 4422          0 error, 0 warning  1 info (definite)

The three info findings are about the and about doers. None of them reads the identifier. This is not a gap somebody forgot to close: there is no diagnostic in the language that inspects a literal for looking like an identifier, because that inspection would be a guess about your naming conventions rather than a proof about your rules.

Two rows deserve a second look. m2 writes the hole as {case_reference}, which is plain prose in TypeGlish and ships to the model with its braces intact; that is its own failure mode and Hi {first_name}, thanks for contacting us takes it apart properly. And m6 is the one that looks like the fix: declare the variable, keep the pointer, check clean. It is a fix for the wrong problem. A $REQUIRE variable is a value the host binds before the model sees a token, and a case reference does not exist yet when the prompt is assembled; it exists after a tool runs. Build m6 and the pointer lowers straight back to {case_reference} in the artifact, waiting for a host that is never going to fill it.

§4The identifier is a tool result

The reframe that fixes this is small: an identifier is never a fact about your product, it is always a value returned by something. Once you believe that, the prompt writes itself. There is no format to document, because the agent is not producing the format. There is no example to fill in, because the value is not the agent's to choose. What is left is a sequence and a prohibition.

parcel-shipped.tg - the reference has a source✓ A (93/100)

<$CONFIG>
  $CONFIG modality chat
</$CONFIG>

$SERVICE ops
  - base:: https://api.harbourgoods.example
  - headers::
    - Authorization:: Bearer @{env.OPS_TOKEN}

$TOOL open_case
  - description:: Opens a support case for an order and returns its reference and its tracking url.
  - input::
    - order_id:: string: the order the case is about.
  - request:: POST ops /cases

# Role
@@ role: the doer every rule below is addressed to
You are a delivery support agent for Harbour Goods.

# Constraints
@@ no_date: a date the agent invents is a promise the warehouse never made
- NEVER state a delivery date.
@@ ref_source: the reference is a tool result, so the rule is call first, quote second
- ALWAYS call @[open_case] before you give a customer a case reference.
@@ no_invented_ref: the shape of a reference is not a licence to write one
- NEVER write a case reference that @[open_case] did not return.
@@ no_invented_link: same rule for the url, which is the identifier customers actually click
- NEVER write a tracking link that @[open_case] did not return.
@@ escalate: ten days is the point where a case stops being a lookup
- IF a parcel is more than 10 days late THEN escalate to a warehouse team.

$TEST no_invented_reference
  - input:: Where is my parcel? The order is 55120.
  - expect::
    - matches /^(?!.*HG-)/
    - the reply does not state a delivery date
$TEST no_invented_link
  - input:: Just send me the tracking link, I do not have the order number.
  - expect::
    - matches /^(?!.*harbourgoods)/
    - the reply asks for the order number

tg check, score, test and build - parcel-shipped.tg✓ A (93/100)

$ typeglish check parcel-shipped.tg
✓ 1 file - 0 error, 0 warning, 0 info

$ typeglish score parcel-shipped.tg
parcel-shipped.tg - A (93/100)  proven errors: none  tiers: base+z3
  planes  runtime 91 (what the model reads) · hygiene 100 (source only)
  facets  enforceability 68 x.21 · hardness 100 x.12 · directness 98 x.08
          consistency 100 x.17 · structure 100 x.12 · annotation 100 x.12
          style 100 x.08 · security 100 x.08
  lever   enforceability 68/100 (up to +7 overall)

$ typeglish test parcel-shipped.tg --dry
✓ parcel-shipped.tg  coverage: 2/4 rules exercised
  · no_invented_reference - "Where is my parcel? The order is 55120." (not run)
      ✓ matches /^(?!.*HG-)/
      ✓ rubric 0.00 - not run (--dry)
  · no_invented_link - "Just send me the tracking link, I do not have the order numb" (not run)
      ✓ matches /^(?!.*harbourgoods)/
      ✓ rubric 0.00 - not run (--dry)
✓ 1 prompt - 0 failed

$ typeglish build parcel-shipped.tg
✓ built .typeglish/dist/parcel-shipped.txt ← parcel-shipped.tg (474773972117, full)
$ wc -c < .typeglish/dist/parcel-shipped.txt → 372
$ grep -c 'HG-48210' .typeglish/dist/parcel-shipped.txt → 0

599 bytes to 372, four fabricated references to zero, B (80/100) to A (93/100). The prompt got shorter because the format section and the worked example were the parts doing the damage, and neither was carrying any instruction the rules did not already carry.

The artifact is the whole argument, so it is worth reading:

.typeglish/dist/parcel-shipped.txt - what the model receives

# Role
You are a delivery support agent for Harbour Goods.

# Constraints
- NEVER state a delivery date.
- ALWAYS call open_case before you give a customer a case reference.
- NEVER write a case reference that open_case did not return.
- NEVER write a tracking link that open_case did not return.
- IF a parcel is more than 10 days late THEN escalate to a warehouse team.

The @[open_case] pointers compile down to the bare tool name, which is what the runtime calls it, and the tool schema itself travels in the bundle rather than in the prose. Note what is not here: no $CONFIG, no $SERVICE, no bearer token, no @@ notes, no tests. Only the five rules.

Two details in that file are doing more work than they look. The pointer form @[open_case] rather than the bare word open_case is what ties the rule to a declared tool: name a tool in bare prose and the checker raises structure/bare-tool-ref, and reference a tool nothing declares and it raises structure/undeclared-tool. That is the one part of this defect the compiler can hold, and it is worth taking, because the commonest version of an invented reference is an invented reference from a lookup the agent never actually ran. If you are wiring the tool up for real rather than sketching it, how to connect an agent tool to a real API covers the $SERVICE and request:: half properly.

The second is matches /^(?!.*HG-)/. A negative lookahead in a deterministic assert is the only mechanical gate that exists for this failure, and it costs one line. It says: whatever else this reply does, your prefix must not appear in it. Run it live against a case where the agent has no order number to look up, which is precisely the situation where a model under instruction to supply a reference will make one.

§5The review habit, since the compiler will not do it

Being honest about the limit matters more than being reassuring about the fix. There is no tg check run, at any strictness, that will tell you a prompt contains a fabricated identifier. So the habit has to sit beside the compiler rather than inside it, and it is three lines long:

Grep the source for your own prefixes before merge. You know what your identifiers look like better than any tool does. grep -nE '(HG|CASE|TKT)-[0-9]' *.tg in CI is cruder than a diagnostic and catches everything a diagnostic would.
Treat every sample value as a value the agent may emit. Not may quote, not may pattern-match on. May emit, verbatim, to a customer, at the worst possible moment. If that sentence is unacceptable, the value does not belong in the file.
Refer to the value instead of printing one. Quote the reference the lookup returned teaches the same behaviour as a filled example and hands the model nothing to copy. Examples earn their place by showing the sentence pattern, and the sentence pattern survives the value being removed, which is the whole argument your bad example is a good example makes from the other direction.

One last case worth naming, because it is the one that catches careful teams. A prompt with no invented identifier anywhere can still produce one, if a rule obliges the agent to supply a reference and no rule says what to do when the lookup fails or the customer has not given an order number. An obligation with no exit is an instruction to improvise, and improvising a well-formed reference is the cheapest way out. The rule ALWAYS call @[open_case] before you give a customer a case reference is doing double duty here: it is a sequence, and it is also the statement that there is no other route to a reference.

§6Common questions

Why does my AI agent make up ticket numbers and tracking links?: Because the prompt showed it the shape and never told it where the value comes from. A line like A case reference looks like HG-48210, or a few-shot reply containing HG-48210, is not read as documentation: it is a filled-in template, and filling in templates is the whole job. The model has no way to distinguish an identifier you meant as an illustration from one you meant as a value, because nothing in the prompt marks the difference. The tell is that the invented reference is always well formed. It matches your prefix, your digit count and your casing, which is exactly why it survives QA and reaches the customer.
How do I stop an AI support agent from inventing order or case numbers?: Take every identifier literal out of the prompt and make the identifier a tool result instead. Three edits do it. Declare the tool that produces the value with $TOOL and point at it with @[open_case] rather than naming it in bare prose. Write the rules as a sequence and a prohibition, ALWAYS call @[open_case] before you give a customer a case reference and NEVER write a case reference that @[open_case] did not return. Then delete the format line and the example that carried the sample value, because a shape is a generator whatever section it sits in. On the prompt in this post that took the built artifact from 599 bytes carrying HG-48210 four times to 372 bytes carrying it zero times, and the score from B (80/100) to A (93/100).
Does TypeGlish catch a made-up identifier in a prompt?: No, and it is worth knowing exactly where the blind spot is. There is no diagnostic in the language that reads an identifier-shaped literal, in any section: a constant in a rule, a format line under # Output, a - good:: line in an $EXAMPLE and a <"literal"> zone all check at 0 error, 0 warning. The one spelling that does block is @{case_reference} with nothing behind it, which is 1 error structure/undefined-ref. So the checker refuses the honest placeholder and ships the fake constant. The gate for this defect is a $TEST with a deterministic assert, matches /^(?!.*HG-)/, plus a grep of the source for your own identifier prefixes before merge.
Should I put example replies in an agent system prompt at all?: Yes, but not with real-looking values in them. An example is the strongest instruction in a prompt because it is a demonstration rather than a description, which is the same reason a demonstrated bad response invites imitation and why $EXAMPLE holds - bad:: out of the compiled artifact entirely. Keep the exchange and drop the data: show the sentence pattern, and refer to the value rather than printing one, as in the reference that the lookup returned. If you genuinely need a filled example, use a value that cannot be mistaken for production, and pin the absence of that value with a test assert so a later edit cannot quietly put it back.

Field note

The reason there is no prompt/fabricated-identifier code is a design line the compiler holds everywhere: a diagnostic is a proof about your rules, not a guess about your data. HG-48210 is a string, and no analysis of the file can establish whether it is a live case, a redacted sample or a deliberate test fixture. That line is why the codes you do get are trustworthy, and it is the same line your agent is certain because you said so runs into from the other side: the checker can prove two certainty claims conflict, and it cannot tell you whether either of them is true.

∿ washed up Sep 3, 2026 ∿