Rewrite: one conversation is not 70 percent
Six bullets in a broadband agent's Constraints section, three of them lifted straight off the quarterly deck: 70 percent containment, four-minute handle time, 4.5 CSAT. They are precise, they are numeric, they are unhedged, and the scorer cannot tell them apart from the rules that work.
TL;DR A containment or handle-time target in a system prompt is graded exactly like a rule and cannot be followed by any single reply: You MUST resolve at least 70 percent of conversations without escalating scores B (87/100) with enforceability 100, hardness 100, directness 100, byte-identical to a real escalation rule. The compiler will prove that target against another percentage (6 blocking errors against at most 50 percent) and never against a rule, so You MUST NOT resolve a conversation without escalating sitting beside it is 0 error, 0 warning, 0 info. Replace the target with the per-turn rules that produce it and measure the rate on transcripts: B (83/100) to A (98/100), and the rule count goes from 6 to 7.
The target gets into the prompt for a good reason, which is that somebody is accountable for it. The quarterly commitment is 70 percent containment, the agent is the thing that has to deliver it, and the prompt is the only place anybody writes down what the agent is for. So it goes in, near the top, phrased exactly as it appears on the deck. It reviews well: it is specific, it has a number in it, and it is impossible to argue with. Then it sits there for a year being read, one conversation at a time, by a system that has never seen the other nine thousand nine hundred and ninety nine.
§1Before: six rules, three of them about ten thousand conversations
Ines is first line for Meridian Broadband, a regional ISP. Six bullets, and three of them came off a slide.
# Role You are Ines, the first-line support agent for Meridian Broadband. # Constraints - You MUST resolve at least 70 percent of conversations without escalating. - You MUST keep the average handle time under 4 minutes. - You MUST achieve a customer satisfaction score of at least 4.5 out of 5. - You MUST resolve most billing questions yourself. - You MUST escalate every complaint about an engineer visit to a human agent. - You MUST NOT promise a credit over 25 pounds.
$ typeglish check ines.tg ines.tg:6:1 info prompt/unintroduced-definite "the average handle time" retrieves something this document never introduces - a model must guess which time is meant. Introduce it on another line ("You manage a time.") or name it outright. ✓ 1 file — 0 error, 0 warning, 1 info $ typeglish score ines.tg ines.tg — B (83/100) proven errors: none tiers: base+z3 planes runtime 94 (what the model reads) · hygiene 50 (source only) facets enforceability 90 x.21 · hardness 92 x.12 · directness 92 x.08 · consistency 100 x.17 · structure 100 x.12 (hygiene) · annotation 0 x.12 (hygiene) · style 89 x.08 · security 100 x.08 lever annotation 0/100 (up to +12 overall) — Put a "@@ why" note directly above each statement ("@@ name: why" also names it).
5a4035171d4f.Enforceability 90 on a file where half the rules are properties of a population. That is not a bug in the facet, and it is worth being precise about why: enforceability measures the share of candidate rule lines that lower into checkable IR, how many hard rules parse proof-grade, and how many are measurable. At least 70 percent is measurable. It parses. It lowers. Everything the facet is asking about is true of it. What the facet does not ask, and cannot, is whether the thing being measured is available to the reader of the prompt.
§2Line by line, the target and the rule are the same object
Split the six bullets into six files, one rule each, and score them separately. This is the cleanest way to see what the scorer thinks each line is worth on its own merits.
$ tail -1 k1.tg - You MUST resolve at least 70 percent of conversations without escalating. k1.tg — B (87/100) proven errors: none tiers: base+z3 planes runtime 100 (what the model reads) · hygiene 50 (source only) facets enforceability 100 x.21 · hardness 100 x.12 · directness 100 x.08 · consistency 100 x.17 · structure 100 x.12 (hygiene) · annotation 0 x.12 (hygiene) · style 100 x.08 · security 100 x.08 $ tail -1 k5.tg - You MUST escalate every complaint about an engineer visit to a human agent. k5.tg — B (87/100) proven errors: none tiers: base+z3 planes runtime 100 (what the model reads) · hygiene 50 (source only) facets enforceability 100 x.21 · hardness 100 x.12 · directness 100 x.08 · consistency 100 x.17 · structure 100 x.12 (hygiene) · annotation 0 x.12 (hygiene) · style 100 x.08 · security 100 x.08 $ tail -1 k6.tg - You MUST NOT promise a credit over 25 pounds. k6.tg — B (87/100) proven errors: none tiers: base+z3 planes runtime 100 (what the model reads) · hygiene 50 (source only) facets enforceability 100 x.21 · hardness 100 x.12 · directness 100 x.08 · consistency 100 x.17 · structure 100 x.12 (hygiene) · annotation 0 x.12 (hygiene) · style 100 x.08 · security 100 x.08
The other three land where you would not guess. The CSAT line, MUST achieve a customer satisfaction score of at least 4.5 out of 5, is B (81/100) with enforceability 70: it is the most obviously unactionable line in the file and it is only marked down a little, because achieve is not a verb the ontology reads, so it half-lowers. The handle-time line is B (80/100) with directness 50 and style 63, and the marks come off for the average handle time, which is the definite article correctly reporting that the average is not in this document and could not be. And the line the scorer actually punishes is the softest one in the set.
$ tail -1 k4.tg - You MUST resolve most billing questions yourself. k4.tg — C (75/100) proven errors: none tiers: base+z3 planes runtime 83 (what the model reads) · hygiene 50 (source only) facets enforceability 70 x.21 · hardness 50 x.12 · directness 100 x.08 · consistency 100 x.17 · structure 100 x.12 (hygiene) · annotation 0 x.12 (hygiene) · style 100 x.08 · security 100 x.08
hardness 50. Most is the middle of the quantifier square, and the middle wavers, so the whole statement is capped at half.Line up the two verdicts and the ordering is upside down. Resolve most billing questions yourself is the one line in the file an experienced agent could act on immediately, and it is the only one the grade objects to. Resolve at least 70 percent of conversations without escalating is the one line no conversation can act on, and it is a perfect score on every facet the scorer has. This is the same shape as a permission being priced at nothing, run in reverse: there, a real piece of policy scored zero because of the modal it wore; here an impossible instruction scores full because of the number it wears.
A percentage is not a rule with a number in it. It is a rule about a set the reader is not standing in.
§3What the compiler will prove, and what it will not
The target is not invisible. It lands in the counting layer, with a real count in it, and the solver will happily do arithmetic on it. The question is what it gets compared with. Four files, each one the target plus one more line.
# every file is line 5: You MUST resolve at least 70 percent of conversations without escalating. $ tail -1 m1.tg - You MUST resolve at most 50 percent of conversations without escalating. m1.tg:5:1 error logic/contradiction Conflicts with line 5. Counted rule — "resolve at least 70 percent of conversations without escalating." vs "resolve at most 50 percent of conversations without escalating." is bound to at least 70 but at most 50. The counts cannot both hold. m1.tg:5:1 error logic/action-count Conflicts with line 5. One action, two counts — "resolve at least 70 percent" but "resolve at most 50 percent". A directive carries ONE count for "resolve percents of conversations without escalating" — state how many once. m1.tg:5:1 error logic/numeric Conflicts with line 5. Numeric conflict — "at least 70 percent" and "at most 50 percent" can't both hold. ✗ 1 file — 6 error, 0 warning, 0 info $ tail -1 m2.tg - You MUST resolve at least 90 percent of conversations without escalating. m2.tg:5:1 error logic/quantifier-strength Conflicts with line 5. One rule, two counts — "resolve at least 90 percent of conversations without escalating." already sits inside "resolve at least 70 percent of conversations without escalating."; the looser bound is dead weight. Keep the tighter line. ✗ 1 file — 4 error, 0 warning, 0 info $ tail -1 m4.tg - You MUST escalate every conversation to a human agent. ✓ 1 file — 0 error, 0 warning, 0 info $ tail -1 m5.tg - You MUST NOT resolve a conversation without escalating. ✓ 1 file — 0 error, 0 warning, 0 info
check --strict either.Look at m5.tg for a second longer, because it is the one that should fire and does not. You MUST NOT resolve a conversation without escalating is a flat prohibition on the precise action the target obliges, in the same file, four lines apart. A human reads those two lines and sees a policy that cannot be satisfied. The prover reads a count over conversations and a prohibition over a conversation, which are different slots, and correctly reports that it has proved nothing. It is not wrong. The two statements are about different things, and that is the whole problem: the target is about the population and every real rule is about the turn, so they never end up in the same argument.
Which means the dashboard is checked against the dashboard. Put two quarterly numbers in one prompt and the compiler will referee them for you, down to the case where one target merely entails the other and is therefore dead weight. Put a target next to the behaviour that decides it and the file is silent forever. The arithmetic in your prompt is real arithmetic; it just does not reach across that line.
§4After: the target becomes the rules that produce it
Deleting the three targets and stopping there would be worse than leaving them, for the same reason deleting a discretion clause is: they were carrying something. Somebody has to answer the question the target was standing in for, which is what should the agent do differently to make this number move. For containment, the answer came out of forty transcripts and it was specific: the escalations Meridian was losing were handed off before anybody looked at the account.
<$CONFIG> $IMPORT tool lookup_account $IMPORT route human_agent </$CONFIG> # Role @@ one addressee, so every rule below is a rule about this turn You are Ines, the first-line support agent for Meridian Broadband. # Constraints @@ containment 1: a billing answer given without the account open is the avoidable escalation - WHEN a customer asks about a bill THEN you MUST call @[lookup_account] before you answer. @@ containment 2: a premature handoff is the only escalation a prompt can actually prevent - You MUST NOT escalate a conversation before you have called @[lookup_account]. @@ the escalation list IS the containment rate; anything not on it stays with you - WHEN a customer complains about an engineer visit THEN you MUST escalate to human_agent. @@ CX-604: retention owns cancellations, not first line - WHEN a customer asks to cancel THEN you MUST escalate to human_agent. @@ this is what "average handle time under 4 minutes" was asking of a single reply - You MUST keep every reply to at most 3 sentences. @@ two questions in one reply is the pattern that doubles the turn count - You MUST ask at most 1 question in a reply. @@ finance sets the cap; above it a human decides - You MUST NOT promise a credit over 25 pounds. $TEST engineer_complaint - input:: The engineer who visited yesterday left mud all over my hallway. - expect:: - contains "human agent"
0 error, 0 warning, 0 info at A (98/100): enforceability 91, hardness 100, annotation 100. Builds to 48a0dec70669. Not one number from the deck survived, and every number that replaced them is a bound a single reply can hold.Three trades happened, and it is worth naming which one each target got.
- 70 percent containment became two rules and a list. The prohibition on escalating before the lookup is the direct lever, and it is the only escalation a prompt can genuinely prevent. The two
WHENrules are the escalations Meridian wants, written out, because an explicit escalation list is the containment rate: whatever is not on it stays with the agent. The target was a summary of that list all along. - Four-minute handle time became a reply-length bound and a one-question rule. Neither is handle time. Both are the part of handle time a single reply controls, and the second one came from reading transcripts: a reply that asks two questions gets one answered and costs a round trip. This is the same move as replacing an unobservable trigger with a named condition, applied to the object of a rule instead of its guard.
- 4.5 CSAT became nothing, and that is the honest outcome. Nobody at Meridian could name a behaviour that would move it which was not already covered by the length bound or the escalation list. A target with no behaviour behind it is not a rule that needs rewriting; it is a line that was never doing anything.
before after B (83/100) A (98/100) runtime 94 · hygiene 50 runtime 97 · hygiene 100 enforceability 90 enforceability 91 hardness 92 hardness 100 directness 92 directness 96 annotation 0 annotation 100 style 89 style 100 0 error, 0 warning, 1 info 0 error, 0 warning, 0 info
Enforceability 90 to 91 is the receipt for everything above. The rewrite deleted three lines that cannot be followed and added four that can, and the facet that measures whether rules are well formed barely noticed, because the deleted lines were extremely well formed. Most of the fifteen points are the @@ notes buying the hygiene plane and the average handle time leaving with its style penalty. If you want a number that isolates what changed, it is not in the grade at all.
§5The denominator says six, then seven
There is one measurement that gets closest, and it is the same one that catches a discretion clause: bolt the identical single-case suite onto both files and ask the runner how many rules it thinks it has.
$ typeglish test inesB.tg --dry # before ✓ inesB.tg coverage: 1/6 rules exercised · engineer_complaint — "The engineer who visited yesterday left mud all over my hall" (not run) ✓ contains "human agent" ✓ 1 prompt — 0 failed $ typeglish test ines4.tg --dry # after ✓ ines4.tg coverage: 2/7 rules exercised · engineer_complaint — "The engineer who visited yesterday left mud all over my hall" (not run) ✓ contains "human agent" ✓ 1 prompt — 0 failed
This is the number to take away, and it cuts both ways. On the one hand it is the honest count: the before file really does contain six lines shaped like rules, and the runner is right to say so. On the other hand a coverage denominator is exactly the metric a team reaches for when it wants to know how well tested a prompt is, and three of those six can never be covered, so the ceiling on that file is 3/6 and nothing in the toolchain will tell you that 3/6 is full marks. Both runs end in 0 failed, which is a fair report of a suite with one case in it and no report at all about the file.
The rate itself has not gone away, and it should not. It moved to the place it can be counted: the transcript pipeline, after the fact, over a week of conversations, where a percentage is a percentage. What the prompt now contains is the set of decisions that produce that percentage, one per line, each one about the conversation the agent is actually in. When containment comes in at 61 percent next quarter, the argument is about which line to change, and there is a line to change.
§6Common questions
- Should I put our containment or deflection target in the agent's system prompt?
- No, because the agent reads one conversation and the target is a property of ten thousand of them.
You MUST resolve at least 70 percent of conversations without escalatingscores B (87/100) with enforceability 100, hardness 100 and directness 100, which is byte-identical toYou MUST escalate every complaint about an engineer visit to a human agent, so nothing in the toolchain will tell you the two lines are different in kind. Write the rules that produce the rate instead: the escalation list, a prohibition on handing off before the lookup, and a reply-length bound. The rate is then something you measure on transcripts, where you can actually count it. - Why does my agent escalate more than the target in the prompt says?
- Because the target and the escalation rules never meet. A percentage lands in the counting layer, where the only thing it can be compared with is another count about the same action, and every rule that actually decides an escalation is a different action.
You MUST resolve at least 70 percent of conversations without escalatingbesideYou MUST escalate every conversation to a human agentis0 error, 0 warning, 0 info, and so is the same target beside the flat prohibitionYou MUST NOT resolve a conversation without escalating. The escalation rules win every time, because they are the only lines that describe a decision the agent can make in the turn it is in. - Does typeglish check catch a KPI written as a rule?
- It catches it against another KPI and never against a rule. Two percentage targets on one action are blocking:
at least 70 percentbesideat most 50 percentis 6 errors acrosslogic/contradiction,logic/action-countandlogic/numeric, andat least 70 percentbesideat least 90 percentis 4 errors acrosslogic/quantifier-strengthandlogic/action-count, because a looser bound the tighter one already entails is dead weight. Put the same target beside any behavioural rule and the file is0 error, 0 warning, 0 info, unchanged undercheck --strict. The percentage is in the numeric plane, so the compiler proves the dashboard against itself. - What replaces average handle time in an agent prompt?
- The two things a single reply can control: its length and how many questions it asks.
You MUST keep every reply to at most 3 sentencesandYou MUST ask at most 1 question in a replyare both bounds Z3 can hold, and between them they are most of what an AHT target was reaching for.You MUST keep the average handle time under 4 minutesis not, and it is the one line in the set the checker has a complaint about: an info reading the average handle time retrieves something this document never introduces, which is the definite article noticing that the average is not in the file and could not be. - Does the test runner know a KPI is not a rule?
- No.
typeglish test --drycounts rules, not whether a rule is about a turn: the six-line KPI prompt reportscoverage: 1/6 rules exercised, so all six lines are in the denominator, including the containment target, the handle-time target and the CSAT target. The rewrite reportscoverage: 2/7. Both runs end in0 failed. A coverage denominator is a count of things the suite could exercise, and it will happily include three lines no conversation can exercise at all.
The KPI line is harder to remove than a hedge or a contradiction, and not for technical reasons. It is the only line in the prompt that connects the file to somebody's objectives, so taking it out reads as taking the agent out of scope for the number, and the person who put it there is usually the person the number belongs to. The conversation that works is the same one that works for a discretion clause: not this line is wrong, but show me the last five conversations where this line would have changed what the agent did. There will not be five. There will be a story about handoffs that happened before anybody opened the account, and that story is a rule with a tool pointer in it. The target then leaves the file the way it should, replaced rather than deleted, and it goes back to the dashboard where it can be measured by something that can see more than one conversation at a time. A QA scorecard fails in the prompt for a related reason, but not the same one: a rubric is written in the wrong grammar and half of it never reaches the rule plane, while a KPI is written in perfect rule grammar and reaches the plane at full strength. Wrong grammar is a cheap defect. Perfect grammar about the wrong subject is the expensive one.