Every WHEN is a classifier
Your checker proves what happens after THEN. Everything before it is the model reading a customer’s message and deciding which label fits, with no training data, no precision number, and no rule for the message that fits two.
TL;DR Every IF and WHEN in a system prompt is a classification the model performs on the customer’s message, and a checker proves what follows from each label but never whether the label fits: move the conditions your platform already knows into a typed input the compiler decides, and write the rest as observable evidence pinned by $TEST cases, including one that should not fire.
Most prompt review happens after the THEN. Is the action clear, is it one verb and one object, does it collide with another rule. The part before the THEN gets read once, nodded at, and left alone, because if the customer seems confused sounds like plain English and plain English sounds finished. It is the least finished part of the file.
§1Two rules that cannot collide, until a customer writes in
Brindle Mobile sells pay-monthly phone plans, and its chat agent has four conditional rules that have been in the prompt since launch. Each was written by someone who had watched real conversations go wrong, and each reads as obviously correct.
# Role You are the chat support agent for Brindle Mobile, a pay-monthly mobile network. # Rules - IF the issue is urgent THEN escalate to a human agent. - WHEN the customer wants to cancel THEN NEVER escalate to a human agent. - IF the customer seems confused THEN MUST explain the bill. - IF the customer is out of contract THEN MAY offer a SIM-only plan.
$ typeglish check brindle.tg brindle.tg:5:1 info prompt/unintroduced-definite "the issue" retrieves something this document never introduces - a model must guess which issue is meant. Mention an issue on an earlier line, or anchor it here ("their issue", "the issue you ..."). brindle.tg:7:1 info prompt/unintroduced-definite "the bill" retrieves something this document never introduces - a model must guess which bill is meant. Mention a bill on an earlier line, or anchor it here ("their bill", "the bill you ..."). ✓ 1 file - 0 error, 0 warning, 2 info $ typeglish score brindle.tg brindle.tg - C (79/100) proven errors: none tiers: base+z3 planes runtime 88 (what the model reads) · hygiene 50 (source only) facets enforceability 100 x.21 · hardness 75 x.12 · directness 63 x.08 · consistency 100 x.17 · structure 100 x.12 (hygiene) · annotation 0 x.12 (hygiene) · style 70 x.08 · security 100 x.08 lever annotation 0/100 (up to +12 overall) - 5 statements without a @@ note (L2 L5 L6 L7 L8) - put a "@@ why" line directly above each rules 4 of 4 rule-shaped lines read
enforceability 100: every rule reads. consistency 100: nothing collides. Both numbers are true, and neither is about the part of the file that decides what happens.Now the customer types: my phone was stolen last night and I want to cancel the line. Is the issue urgent? A stolen handset with a live SIM is about as urgent as mobile support gets. Does the customer want to cancel? They said so. Line 5 says escalate, line 6 says never escalate, and which one wins is decided by the model, on this message, with no rule telling it how.
The checker is not wrong to pass the file. It proved what it can prove: that the two rules have different conditions. What it cannot know is whether both conditions can be true of one message, because deciding whether a message is urgent is not logic. It is classification, and the model is the classifier.
§2The prover reads the words, not the situation
Here is the same pair with the conditions spelled identically. Nothing else changes.
# Role You are the chat support agent for Brindle Mobile, a pay-monthly mobile network. # Rules - IF the customer reports a stolen phone THEN escalate to a human agent. - IF the customer reports a stolen phone THEN NEVER escalate to a human agent.
$ typeglish check same-guard.tg same-guard.tg:5:1 error logic/contradiction Conflicts with line 6. Logical conflict - "escalate to a human agent" vs "escalate to a human agent." is both required and forbidden. Keep one, or scope the two rules so they cannot both apply (IF <condition> THEN ...). ↳ line 6: - IF the customer reports a stolen phone THEN NEVER escalate to a human agent. same-guard.tg:6:1 error logic/contradiction Conflicts with line 5. Logical conflict - "escalate to a human agent" vs "escalate to a human agent." is both required and forbidden. Keep one, or scope the two rules so they cannot both apply (IF <condition> THEN ...). ↳ line 5: - IF the customer reports a stolen phone THEN escalate to a human agent. ✗ 1 file - 2 error, 0 warning, 0 info
brindle.tg it compared them and found two.That is the whole mental model. A guard is a label the model assigns to the customer’s message, and the checker proves what follows from each label, never whether a label fits. Write the same words twice and the checker can reason about them. Write urgent once and wants to cancel once and you have two independent classifiers, both run by the model on every turn, with no stated rule for the message that trips both.
Every IF and every WHEN in a support prompt is one of these. Seems confused is a sentiment classifier. Is urgent is a priority classifier. Wants to cancel is an intent classifier. Nobody labelled training data for any of them, nobody measured their precision, and each is re-run from scratch on every message. It is the same move the duty of care nothing can check makes with the IF...THEN frame: the condition is where the prompt stops being checkable and starts being judgment.
The checker proves the consequence. The model decides the condition. Nobody tested the condition.
§3Two kinds of condition
Once guards read as classifiers, they sort into two piles, and the piles get opposite treatment.
Conditions the host already knows. Is out of contract is not a judgment; it is a field in the billing system. The topic a customer picked from the chat menu is a field in the widget. Asking the model to infer either from the conversation is asking it to guess at a fact your platform is holding. These belong in a typed input the host fills, read by a $SWITCH ON block, so the compiler makes the decision and the losing arms never reach the model at all. The typed domain also buys a proof the prose version cannot have: every member gets an arm, or the build fails.
$REQUIRE variable topic: one of stolen, fault, bill, cancel # Role You are the chat support agent for Brindle Mobile, a pay-monthly mobile network. # By topic $SWITCH ON @{topic} - stolen:: You MUST call @[bar_sim]. - fault:: You MUST call @[check_outage]. - bill:: You MUST explain each charge. $TOOL bar_sim - description:: Bar the SIM and the handset on the customer's account. $TOOL check_outage - description:: Check for a network outage at the customer's postcode.
$ typeglish check menu.tg menu.tg:7:1 error structure/non-exhaustive-switch @{topic} can be cancel, but no arm handles it - add a "- <member>::" row for each (or a deliberate "- otherwise::" fallback). ✗ 1 file - 1 error, 0 warning, 0 info
IF the customer wants to cancel has no such property: a missing case is just a message the model handles however it likes.One caution about that pile: a typed input holds exactly one value. The customer who picked cancel from the menu and then mentions the stolen phone is still one value in the widget, which is the trap your router assumes one thing is wrong takes apart. The menu tells you where the conversation started, not everything that is in it.
Conditions only the model can judge. What the customer is saying, in their own words, mid-conversation, is irreducibly the model’s call. Keep those guards, but write them as the classifier you would actually want: observable rather than inferred. The customer says their phone was stolen is evidence on the screen. The issue is urgent is a conclusion the model has to reach first. The same instinct is behind nothing in your prompt ends the conversation: a trigger you can point at in the transcript is a trigger you can test.
§4What ships
$REQUIRE variable topic: one of stolen, fault, bill, cancel # Role @@ role: the desk this file speaks for, and the owner of every rule below You are the chat support agent for Brindle Mobile, a pay-monthly mobile network. # By topic @@ topic: the customer picks a topic from the chat menu, so the host knows it and the compiler decides it $SWITCH ON @{topic} - stolen:: You MUST call @[bar_sim]. - fault:: You MUST call @[check_outage]. - bill:: You MUST explain each charge. - cancel:: You MUST call @[offer_retention]. # Any topic @@ theft_mid_chat: the menu is one label at the start; a theft mentioned later is the model's call, so it gets its own rule and its own tests - WHEN the customer says their phone was stolen THEN MUST call @[bar_sim]. @@ human: asking for a person is words on the screen, so the trigger is one the model cannot misread - WHEN the customer asks for a human THEN escalate to a human agent. $TOOL bar_sim - description:: Bar the SIM and the handset on the customer's account. $TOOL check_outage - description:: Check for a network outage at the customer's postcode. - input:: - postcode:: the customer's postcode. $TOOL offer_retention - description:: Fetch the retention offers this account is eligible for. $TEST theft_plain - input:: My phone was stolen last night. - expect:: - calls bar_sim $TEST theft_slang - input:: Some lad nicked my phone off the table in the pub. - expect:: - calls bar_sim $TEST theft_inside_cancel - input:: I want to cancel. Someone took my phone out of my bag yesterday and I am done with all of it. - expect:: - calls bar_sim $TEST lost_not_stolen - input:: I think I left my phone on the train. - expect:: - does not call bar_sim - asks whether the phone was stolen $TEST asks_for_person - input:: Can I speak to an actual person, please? - expect:: - escalates to a human agent
$ typeglish check brindle-fixed.tg ✓ 1 file - 0 error, 0 warning, 0 info $ typeglish score brindle-fixed.tg brindle-fixed.tg - A (93/100) proven errors: none tiers: base+z3 planes runtime 90 (what the model reads) · hygiene 100 (source only) facets enforceability 67 x.21 · hardness 100 x.12 · directness 94 x.08 · consistency 100 x.17 · structure 100 x.12 (hygiene) · annotation 100 x.12 (hygiene) · style 100 x.08 · security 100 x.08 lever enforceability 67/100 (up to +7 overall) - Lead every rule with a modal - MUST / NEVER / SHOULD - or put it after THEN; one action and one object per line, a verb the checker knows or a bound ("at most 3 sentences"). Descriptive detail with no bound does not move this facet. `typeglish score <file>` prints the rules table with the edit per line; `typeglish --explain enforceability <file>` prints just that table. rules 2 of 6 rule-shaped lines read · 2/2 covered by $TEST L10 · arm stolen:: You MUST call @[bar_sim]. a $SWITCH / $IF arm body - selected at compile time; the checker does not read arm bodies as rules L11 · arm fault:: You MUST call @[check_outage]. a $SWITCH / $IF arm body - selected at compile time; the checker does not read arm bodies as rules L12 · arm bill:: You MUST explain each charge. a $SWITCH / $IF arm body - selected at compile time; the checker does not read arm bodies as rules L13 · arm cancel:: You MUST call @[offer_retention]. a $SWITCH / $IF arm body - selected at compile time; the checker does not read arm bodies as rules $ typeglish test brindle-fixed.tg --dry ✓ brindle-fixed.tg coverage: 2/2 rules exercised · theft_plain - "My phone was stolen last night." (not run) · rubric - 1 prose expectation, judged on a live run (needs ANTHROPIC_API_KEY) · theft_slang - "Some lad nicked my phone off the table in the pub." (not run) · rubric - 1 prose expectation, judged on a live run (needs ANTHROPIC_API_KEY) · theft_inside_cancel - "I want to cancel. Someone took my phone out of my bag yester" (not run) · rubric - 1 prose expectation, judged on a live run (needs ANTHROPIC_API_KEY) · lost_not_stolen - "I think I left my phone on the train." (not run) · rubric - 2 prose expectations, judged on a live run (needs ANTHROPIC_API_KEY) · asks_for_person - "Can I speak to an actual person, please?" (not run) · rubric - 1 prose expectation, judged on a live run (needs ANTHROPIC_API_KEY) ✓ 1 prompt - 0 failed
enforceability 67 and 2 of 6 rule-shaped lines read look like a regression and are the opposite. The four arm bodies are not scored as rules because they are not decisions the model makes: the compiler picks one. The two rules left for the model to weigh are the two whose conditions are words on the screen.Bind the topic the way the chat widget would, and the model’s copy of the prompt carries one arm and two classifiers.
$ typeglish build brindle-fixed.tg --vars '{"topic":"cancel"}' ✓ built .typeglish/dist/brindle-fixed.txt ← brindle-fixed.tg (279d7fb93af0, full) $ cat .typeglish/dist/brindle-fixed.txt # Role You are the chat support agent for Brindle Mobile, a pay-monthly mobile network. # By topic You MUST call offer_retention. # Any topic - WHEN the customer says their phone was stolen THEN MUST call bar_sim. - WHEN the customer asks for a human THEN escalate to a human agent.
§5Test the classifier, not the rule
A classifier you did not test is a classifier you are guessing about, and the fixed file carries four cases for one rule. They are not four copies of the happy path. They are the shape of a classifier test:
- The plain case. My phone was stolen last night. The words in the guard, nearly verbatim. If this fails, nothing else matters.
- The paraphrase. Some lad nicked my phone off the table in the pub. Real customers do not use your vocabulary; this is where a rule written in your vocabulary drifts.
- The buried case. I want to cancel. Someone took my phone out of my bag yesterday and I am done with all of it. The trigger arrives inside a different intent, which is exactly the message that broke the original pair.
- The near miss. I think I left my phone on the train. Lost is not stolen, and barring a SIM the customer might get back is its own bad outcome. A classifier is defined by where it stops as much as by where it fires.
test --dry reports coverage: 2/2 rules exercised offline, and it is honest about the rest: the prose expectations such as calls bar_sim and does not call bar_sim are judged on a live run, which needs a model and an API key. That split is the point of this whole post in miniature. The compiler can prove the file is well formed, that the switch is total, and that each rule has a case. Whether the model’s classifier fires on the pub message is a question only running it can answer, and the four cases are what make that run worth having.
§6Common questions
- Why does my AI agent follow a rule in testing and ignore it in production?
- Often the rule is fine and its condition is not. Every IF or WHEN is a classification the model performs on the customer’s message, and test messages tend to use the same words as the guard, so the classifier fires. Production messages paraphrase, bury the trigger inside another request, or trip two guards at once. Write guards as observable evidence rather than inferred states, and test each one with a plain case, a paraphrase, a buried case, and a near miss.
- Can typeglish check tell me when two IF conditions overlap?
- Only when they are written the same way. Two rules with the identical guard and opposite modals are a proven
logic/contradiction. Two rules guarded by the issue is urgent and the customer wants to cancel check clean, because deciding whether one message satisfies both is classification, not logic, and the checker does not claim to know. Decide the precedence yourself, or remove one of the guards. - What should go in a typed input instead of a condition in the prompt?
- Anything your platform already knows: contract status, plan, channel, the topic picked from a chat menu, whether the customer is verified. Declare it with
$REQUIRE variableand a closed domain, read it with$SWITCH ON, and the compiler selects the arm, so the model never has to infer a fact from the conversation. The switch must cover every member of the domain or the build fails, which a prose condition can never guarantee. - How do I write a guard the model will apply consistently?
- Name what is on the screen, not what you conclude from it. WHEN the customer says their phone was stolen is evidence the model can point to; IF the issue is urgent asks it to reach a judgment first, differently each time. Then pin the guard with
$TESTcases whose expectations share the rule’s wording, sotest --drycounts the rule as covered, and include at least one case that should not fire.
The conditions in a support prompt are usually the oldest lines in it, and the most confident. If the customer is upset. If it is urgent. If they are a VIP. They were written by people who could tell those things at a glance, because they had spent years on the phones, and they wrote them down the way you would brief a new hire who shares your instincts. The model does not share them. It has its own, re-derived on every message, and nobody in the building has seen them written down. The useful thing a checker does here is narrow and honest: it proves the half of each rule it can see, and by passing the half it cannot, it shows you exactly where the untested judgment lives.