Building a grounded help-centre agent, prompt-first
Six sentences from a support lead, compiled into a spec that checks. The hard part is not the retrieval wiring, it is that the one rule everybody writes first cannot be enforced by anything, and the checker will happily pass all seven ways of writing it.
TL;DR A retrieval-backed support agent has exactly one enforceable grounding rule, and it is not answer only from the help centre: that sentence and five paraphrases of it all check at 0 error because provenance is invisible in the output. Write You MUST end every factual sentence with an article id instead, declare the id shape so a regex can find it, and make the retrieval miss a compiler branch on @{article_count} rather than a judgement call, so the agent that found nothing is running a prompt with no answering rule in it. The brief goes from C (70/100) to A (94/100) and ends with two deterministic asserts.
This is the ninth prompt-first build in the series and the first one where the interesting failure is epistemic. A refund agent has authority, a triage agent has a return type, a booking agent has a tool. A help-centre agent has a corpus, and its whole job is to stay inside it, which turns out to be the one property a prompt cannot state and a compiler cannot check. So the build is mostly the work of trading that property for one that can.
§1The brief, typed straight in
What arrived, verbatim, from a support lead who has been running a help centre for four years and has a retrieval pipeline already stood up.
You are our help centre assistant. You should only answer using information from the help centre. Try to always include the article number. If you are not sure, or the docs do not cover it, it is best to be upfront and hand the customer over to a human. Keep it brief and friendly. Do not make up policies or prices. Answers should be grounded in the retrieved content.
$ npx typeglish check kb.tg kb.tg:3:1 info prompt/hedging Hedging — "Try to" turns this instruction into a suggestion the model may skip. Delete the hedge, or commit to a modal (MUST / NEVER / SHOULD). kb.tg:3:1 info prompt/unintroduced-definite "the article number" retrieves something this document never introduces - a model must guess which number is meant. Introduce it on another line ("You manage a number.") or name it outright. kb.tg:3:1 info prompt/unregistered-doer A bare generic doer never enters the world model - instruction to the agent, or background about users? Use the imperative if the agent acts, or a definite party ("The user should ...") to register the doer. kb.tg:4:1 info typeglish/if-then IF needs a THEN — write IF <condition> THEN <action>. kb.tg:7:1 info prompt/agentless-passive The doer is deleted (agentless passive) - who performs this? Name the actor or use the imperative: "Verify the order." instead of "The order is verified." ✓ 1 file — 0 error, 0 warning, 5 info $ npx typeglish score kb.tg kb.tg — C (70/100) proven errors: none tiers: base+z3 planes runtime 78 (what the model reads) · hygiene 45 (source only) facets enforceability 70 x.21 · hardness 68 x.12 · directness 69 x.08 · consistency 100 x.17 · structure 89 x.12 (hygiene) · annotation 0 x.12 (hygiene) · style 57 x.08 · security 100 x.08 lever annotation 0/100 (up to +12 overall) — Put a "@@ why" note directly above each statement ("@@ name: why" also names it).
THEN. Line 7 is grounding, written as a passive with no doer. Lines 2 and 6, the two sentences the lead cares most about, draw nothing at all.§2The rule you cannot check
That silence on lines 2 and 6 is not an oversight, and it does not go away if you write them better. Seven phrasings of the same grounding intent, ranging from the loose to the strict, in one file.
<role> You are a help centre assistant for a broadband provider. </role> <constraints> - You MUST base every answer on the help centre. - You MUST NOT answer from your own knowledge. - You MUST be accurate. - You MUST only use retrieved content. - You MUST ground every claim in a source. - You MUST cite an article in every answer. - Every factual sentence MUST carry an article id. </constraints>
$ npx typeglish check grounding.tg grounding.tg:8:1 warn prompt/unmeasurable Unmeasurable rule — no observable action or bound, so nothing can check compliance. Name a concrete action ("cite the source", "at most 3 sentences") or move it to a prose block. ✓ 1 file — 0 error, 1 warning, 0 info
You MUST be accurate. The other six pass, including MUST NOT answer from your own knowledge, which is unfalsifiable, and MUST ground every claim in a source, which describes a mental process. They pass because they are well-formed directives with real verbs, and form is what a form check reads.This is worth being precise about, because the wrong lesson here is that the checker is soft on grounding. It is not soft: it is looking at the file, and grounding is not a property of the file. Only answer from the help centre is a claim about where an answer came from, and provenance leaves no trace in the output. Two identical sentences, one lifted from an article and one invented, are the same bytes. Nothing downstream, no assert, no judge, no reviewer, no regex, can separate them.
Grounded is a fact about the past. Cited is a fact about the text.
So you trade. The last line of the seven, Every factual sentence MUST carry an article id, is the same intent expressed as a property of the output, and it is enforceable end to end: a missing id is visible in the reply, a wrong id is a lie somebody can look up, and both are things a deterministic assert can reach. It does not make hallucination impossible. It makes it attributable, which is the difference between a defect you can find in a QA sample and one you cannot. The same move shows up in what your agent claims to know: the rule that survives is always the one whose compliance shows up in the transcript.
One clean-up before it goes in the file. the article number drew prompt/unintroduced-definite in §1, and the fix is not to delete the definite article but to introduce the object, which is also where the id format belongs.
An article id IS a string like HC-1234. - You MUST end every factual sentence with an article id.
§5 is written against: pin the shape once, assert on it forever.§3The miss is a different prompt, not a different mood
Line 4 of the brief bundles two situations that behave nothing alike: you are not sure, and the docs do not cover it. The first is a state of the model and this blog has already taken that one apart. The second is a number, and the host has it before the model sees a token, because retrieval either returned articles or it did not.
That makes it a compile-time branch rather than a runtime judgement, which is the single most useful structural decision in this build.
$REQUIRE variable article_count: integer $IMPORT tool search_help_centre, open_ticket // the retrieval result is a number the host knows and the model does not $DEFINE nothing_found AS @{article_count} is equal to 0
$IMPORT tool is the right declaration here because the host runs the search: the prompt is not defining an HTTP binding, it is saying which capabilities exist so @[search_help_centre] resolves to something.A small trap on the way past. The // above the $DEFINE is a line comment and not a @@ annotation, and swapping them is a blocking error rather than a style preference.
$ npx typeglish check bad-ann.tg bad-ann.tg:6:1 error structure/unattached-annotation A `@@` annotation documents the statement DIRECTLY below it — here the next line is a compiler command. Move the note to the line immediately above its statement. ✗ 1 file — 1 error, 0 warning, 0 info
@@ belongs to the rules and // to the machine plane. The distinction pays off in the annotation facet, which counts notes attached to statements and would otherwise be rewarding a note attached to nothing.§4The whole spec
Assembled: the corpus declared, the retrieval rule flat because it applies every turn, and the two situations as arms of one chain.
<$CONFIG> $REQUIRE variable article_count: integer $IMPORT tool search_help_centre, open_ticket </$CONFIG> // the retrieval result is a number the host knows and the model does not $DEFINE nothing_found AS @{article_count} is equal to 0 <role> @@ persona: the assistant answers from one corpus and nothing else You are a help centre assistant for a broadband provider. @@ corpus: naming the id shape is what makes a citation checkable An article id IS a string like HC-1234. </role> <retrieval> @@ search: retrieve before you answer, every turn, no exceptions - You MUST call @[search_help_centre] before you answer a question. </retrieval> <answering> $IF @{nothing_found}: @@ miss_say: an empty result set is a hand off, not a prompt to improvise - You MUST tell a customer that the help centre does not cover a question. @@ miss_ticket: the customer leaves with a reference, not an apology - You MUST call @[open_ticket]. @@ miss_stop: the one rule the retrieval miss exists to enforce - You MUST NOT answer a question. $ELSE: @@ cite: a claim with no id is the failure this prompt exists to stop - You MUST end every factual sentence with an article id. @@ scope: the corpus is the ceiling on what may be said - You MUST NOT state a fact that a retrieved article does not carry. @@ brevity: four sentences fits a chat pane without scrolling - You MUST keep every answer to at most 4 sentences. </answering>
@@ note saying why it exists. Keep it brief and friendly from the brief became a bound on one side and nothing on the other, deliberately: an agent handing a customer to a ticket has no length target worth stating.$ npx typeglish check helpdesk.tg
✓ 1 file — 0 error, 0 warning, 0 info
$ npx typeglish score helpdesk.tg
helpdesk.tg — A (94/100) proven errors: none tiers: base+z3
planes runtime 95 (what the model reads) · hygiene 91 (source only)
facets enforceability 83 x.21 · hardness 100 x.12 · directness 93 x.08 · consistency 100 x.17 · structure 100 x.12 (hygiene) · annotation 82 x.12 (hygiene) · style 100 x.08 · security 100 x.08
lever enforceability 83/100 (up to +4 overall) — Write rules as MUST / NEVER <verb> … with concrete bounds ("at most 3 sentences"), not vague qualities — a rule the checker can parse is a rule it can defend.
C (70/100) to A (94/100), hygiene 45 to 91. The remaining lever is enforceability 83, and the rule holding it there is MUST NOT state a fact that a retrieved article does not carry, which is the one grounding sentence that stayed in as intent. Fair enough: the scorecard is telling the truth about it.Now build both arms and read what actually reaches the model, because that is where the branch stops being a diagram.
$ npx typeglish build helpdesk.tg --out-dir dist --vars '{"article_count":3}'
✓ built dist/helpdesk.txt ← helpdesk.tg (cc021c723665, full)
<answering>
- You MUST end every factual sentence with an article id.
- You MUST NOT state a fact that a retrieved article does not carry.
- You MUST keep every answer to at most 4 sentences.
</answering>
$ npx typeglish build helpdesk.tg --out-dir dist --vars '{"article_count":0}'
✓ built dist/helpdesk.txt ← helpdesk.tg (c0d77f54dace, full)
<answering>
- You MUST tell a customer that the help centre does not cover a question.
- You MUST call open_ticket.
- You MUST NOT answer a question.
</answering>
<answering> section that differs. The miss artifact does not contain the citation rule, the length bound, or the word answer as anything but a prohibition. An agent that found nothing is not an agent being careful; it is running a document with no answering instruction in it.Leave --vars off and you get the template build, which is the wrong thing to deploy and a useful thing to look at once.
$ npx typeglish build helpdesk.tg --out-dir dist
✓ built dist/helpdesk.txt ← helpdesk.tg (e9301a6fc11c, full)
<answering>
If nothing_found:
- You MUST tell a customer that the help centre does not cover a question.
- You MUST call open_ticket.
- You MUST NOT answer a question.
Otherwise:
- You MUST end every factual sentence with an article id.
- You MUST NOT state a fact that a retrieved article does not carry.
- You MUST keep every answer to at most 4 sentences.
</answering>
<answering> section; the rest of the artifact is identical to the bound builds. Unbound, the chain ships as prose and the model decides for itself which branch it is in, on a question it has no information about. That is the version you get by writing the brief's line 4 as an ordinary sentence, which is exactly what most help-centre prompts do.§5Pinning it with two cases
The citation rule was chosen because it is testable, so test it. Both cases use deterministic asserts, so nothing here needs a judge or a rubric.
$TEST cited_answer - input:: How long does a broadband transfer take? - expect:: - matches /HC-\d{4}/ - at most 4 sentences $TEST miss_hands_off - input:: Can I pay my bill in Norwegian kroner? - expect:: - matches /^(?!.*HC-\d{4})/ - contains "ticket"
does not contain assert, so a negative is written as a lookahead: matches /^(?!.*HC-\d{4})/ passes only when no article id appears anywhere in the reply. An agent that invents a plausible HC-0412 to look grounded fails on the assert that is checking it did not.$ npx typeglish test helpdesk.tg --dry ✓ helpdesk.tg coverage: 1/8 rules exercised · cited_answer — "How long does a broadband transfer take?" (not run) ✓ matches /HC-\d{4}/ ✓ at most 4 sentences · miss_hands_off — "Can I pay my bill in Norwegian kroner?" (not run) ✓ matches /^(?!.*HC-\d{4})/ ✓ contains "ticket" ✓ 1 prompt — 0 failed $ npx typeglish score helpdesk.tg --min A helpdesk.tg — A (94/100) proven errors: none tiers: base+z3
coverage: 1/8 is honest and low, and the denominator is every rule in the file including both arms of a chain no single conversation can be in at once. Read it as a worklist, not a grade. --dry validates the suite and the asserts fully offline; running the cases needs a model.Three rules from this file are worth writing a case against next, in order: the retrieval rule, because an agent that answers without searching is the whole failure mode and a tool-call assert is the only thing that sees it; MUST NOT answer a question in the miss arm, because a helpful model will try; and the four-sentence bound, because bounds drift. That is a coverage of 4/8 for about twenty minutes of work, which is a better afternoon than most prompt reviews.
§6Common questions
- How do I write a system prompt that makes an AI agent answer only from my help centre?
- You cannot write that rule, so write its observable twin instead. Only answer from the help centre is a claim about where an answer came from, and nothing in the output reveals that, which is why seven differently worded grounding rules check at
0 errorand only the vaguest of them draws a finding. The rule that has teeth isYou MUST end every factual sentence with an article id, because a missing id is visible in the text and a fabricated one is a lie a reviewer can look up. Keep the intent sentence if you like the reminder, but do not mistake it for the control. - Should the no results case be a rule in the prompt or a branch the compiler resolves?
- A branch, when the host already knows the answer. Retrieval returns a count before the model sees anything, so declare it as
$REQUIRE variable article_count: integer, definenothing_foundas@{article_count} is equal to 0, and put the two behaviours in a$IFchain. The compiler then builds two different prompts: the miss artifact carries you MUST tell a customer that the help centre does not cover a question and you MUST NOT answer a question, and it does not contain the citation rule at all. Written as prose the model would weigh both sets of instructions on every turn and decide for itself which situation it is in. - What is the right way to test that an AI support agent cited a source?
- Pin the id shape in the prompt and assert on it. Declare the format as a fact the model reads (
An article id IS a string like HC-1234), then write two$TESTcases: a hit case assertingmatches /HC-\d{4}/andat most 4 sentences, and a miss case assertingmatches /^(?!.*HC-\d{4})/andcontains "ticket". Both are deterministic asserts, so nothing is judged and nothing is subjective.typeglish test --dryvalidates the suite offline and reportscoverage: 1/8 rules exercised; the cases themselves need a model to actually run. - Why does TypeGlish reject a
@@note above a$DEFINEline? - Because
@@annotates the statement directly below it, and a$DEFINEis a compiler command rather than a statement. The checker returns a blockingstructure/unattached-annotationreading A `@@` annotation documents the statement DIRECTLY below it - here the next line is a compiler command. Use a//line comment for the machine plane and keep@@for the rules, which is also what theannotationfacet scores: the finished prompt in this post readsannotation 82with a@@note on each of its eight rules and a//comment on its one$DEFINE.
The line that did not survive the build is the one the lead will ask about, so it is worth having the answer ready: do not make up policies or prices is not in the finished prompt as a rule, and removing it did not remove the protection. It moved. In the miss arm the agent is forbidden from answering at all, so there is nothing to invent; in the hit arm every factual sentence has to end with an id, so an invented price has to arrive wearing a citation that a reviewer can open. A prohibition that nothing can observe was traded for a structure in which the failure has to announce itself. That is usually what a prompt-first build is doing when it looks like it is deleting your rules, and it is the same trade keeping an agent on topic makes when a blocklist becomes a scope.