Somebody wants the model bill smaller
The swap itself is one line. What makes it risky is that it is only one line: the prompt you spent a year tuning is about to be read by something with less patience for the parts you never finished.
TL;DR Nine checks before an agent prompt moves to a cheaper model, each with a command behind it. No checker can tell you how a model will behave, but it can enumerate every place your prompt delegated a decision to the reader's judgment, and judgment is exactly the budget a smaller model has less of. Start by proving the swap changes nothing the model reads: two builds differing only in the model id come back with the same artifact hash.
It arrives as a spreadsheet. Inference is the second line on the infra bill, the cheap tier is a fifth of the price, and somebody has already worked out what the saving is annualised. Nobody is being unreasonable. The candidate model is genuinely good, the traffic is genuinely repetitive, and most of what your agent does all day is look up an order and say a sentence about it.
This list is for the two weeks between that spreadsheet and the cutover. It is not the launch checklist, which assumed nothing was live yet, and it is not the one-more-rule checklist, which assumed the reader was staying the same. It is the case where the text is fine and the audience changed.
<$CONFIG> $CONFIG model claude-opus-4-8 - via:: anthropic - max_tokens:: 400 $REQUIRE variable tier: one of standard, premium $IMPORT tool get_order </$CONFIG> $SERVICE anthropic - base:: https://api.anthropic.com - headers:: - x-api-key:: @{env.ANTHROPIC_API_KEY} # Role You are a support agent for Cobalt Mobile. # Constraints - You SHOULD probably confirm an account before you discuss a bill. - You MUST be professional. - You MUST handle a complaint appropriately. - You MUST call @[get_order] before you state a delivery date. - You MUST answer a delivery question in at most 3 sentences.
§1The swap, and what it does not change
1. Put the model id in the file, not in the deploy script. A model swap is a change to the system's behaviour and it deserves a diff with a date and an author on it, not a dropdown in a dashboard. $CONFIG model takes the id on the header and its settings as indented rows, and it is checked rather than trusted. Three ways to get it wrong, and they do not fail alike.
$ npx typeglish check kind-typo.tg # $CONFIG modl claude-haiku-4-5 kind-typo.tg:2:3 error config/unknown-kind $CONFIG modl isn't a known kind - the line was stripped and configured nothing. Did you mean "$CONFIG model"? Known kinds come from the built-ins + your glish.tgc. $ npx typeglish check via-typo.tg # - via:: anthropi via-typo.tg:3:5 error config/dangling-service $CONFIG model → via names "anthropi", but no "$SERVICE anthropi" is defined. via-typo.tg:9:1 warn structure/unused-service $SERVICE anthropic is never used by any tool's request or via:: transport. $ npx typeglish check param-typo.tg # - maxtokens:: 400 param-typo.tg:4:5 info config/unknown-param "maxtokens" isn't a known param of $CONFIG model. ✓ 1 file — 0 error, 3 warning, 3 info
- maxtokens:: 400 is an info finding, the file still exits 0, and your token cap is silently not set. Worth knowing before you go looking for why the cheap model is truncating.2. Build both and diff the artifact. Do this before anything else, because it reframes the whole job. Copy the file, change the model id, and build the pair.
$ npx typeglish build support.tg support-cheap.tg ✓ built .typeglish/dist/support.txt ← support.tg (237e0fbf0c32, full) ✓ built .typeglish/dist/support-cheap.txt ← support-cheap.tg (237e0fbf0c32, full) ✓ built 2/2 $ diff .typeglish/dist/support.txt .typeglish/dist/support-cheap.txt $ # no output
sourceSha256 values against one artifactSha256, because $CONFIG is compiler scaffolding and never reaches the model.That is the shape of the risk in one command. You are not migrating a prompt. You are handing an unchanged document to a reader with a different amount of slack, and every instruction that worked because the old model filled a gap is still exactly as unfilled as it was. The rest of this list is finding the gaps.
A cheaper model is not worse at following instructions. It is worse at guessing which one you meant.
§2The delegation inventory
3. Run check and read every advisory as a delegation. The blocking errors were fixed a year ago. The advisories are still there, because advisories do not fail builds and everybody learns to scroll past them. On a model swap they are the single most useful artefact you have, because each one names a decision your prompt handed to the reader.
support.tg:5:3 warn structure/unused-import Required variable "tier" is never used. support.tg:5:21 warn clarity/unused-variable $REQUIRE variable "tier" is never used — no @{tier} reference fills it. Remove it, or reference it in the prompt. support.tg:18:7 info prompt/hedging Hedging — "SHOULD probably" turns this instruction into a suggestion the model may skip. Delete the hedge, or commit to a modal (MUST / NEVER / SHOULD). support.tg:19:1 warn prompt/unmeasurable Unmeasurable rule — no observable action or bound, so nothing can check compliance. Name a concrete action ("cite the source", "at most 3 sentences") or move it to a prose block. support.tg:20:31 info prompt/vague Vague — "appropriately" names a judgment call without the criterion to judge by, so nothing can check it. Replace it with a measurable form ("at most 3 sentences", "within 24 hours", "cite the source"). ✓ 1 file — 0 error, 3 warning, 2 info
4. Read hardness on the scorecard, and commit to a modal. Hardness measures how binding the language is, and a statement takes the minimum over its force words, so one hedge softens the entire rule. It is the facet that predicts a model swap better than any other, because a hedged rule is not an instruction with a caveat, it is an instruction plus explicit permission to skip it.
support.tg — D (57/100) proven errors: none tiers: base+z3
planes runtime 68 (what the model reads) · hygiene 25 (source only)
facets enforceability 74 x.21 · hardness 66 x.12 · directness 100 x.08 · consistency 50 x.17
structure 50 x.12 (hygiene) · annotation 0 x.12 (hygiene) · style 25 x.08 · security 100 x.08
lever annotation 0/100 (up to +12 overall) — Put a "@@ why" note directly above each
statement ("@@ name: why" also names it).
hardness 66 on five rules, four of which are a bare MUST. One SHOULD probably is doing all of that. enforceability 74 is the companion number: it asks whether a rule names something checkable, and two of these do not.5. Run check --strict and decide which advisories are now blockers. Strict escalates every correctness advisory to an error, and it is a different exit code on the same file. This is the moment to promote it in CI, because the class of finding you have been tolerating is precisely the class the new reader is worse at absorbing.
$ npx typeglish check support.tg ✓ 1 file — 0 error, 3 warning, 2 info $ npx typeglish check support.tg --strict ✗ 1 file — 2 error, 1 warning, 2 info
§3Take the decisions back
6. Resolve at compile time whatever the compiler can resolve. The cheapest way to stop a model getting a decision wrong is to not give it the decision. A prose conditional is a statement the model weighs at runtime; a $SWITCH ON over a typed input is resolved by the compiler, and the losing arm never reaches the model at all. Same source file, two builds.
$ npx typeglish build support-final.tg ✓ built .typeglish/dist/support-final.txt ← support-final.tg (6c663e579a51, full) ... If tier is standard: You MUST offer a callback within 2 working days. If tier is premium: You MUST offer a callback within 4 hours. $ npx typeglish build support-final.tg --vars '{"tier":"standard"}' ✓ built .typeglish/dist/support-final.txt ← support-final.tg (397966ae7b4e, full) ... You MUST offer a callback within 2 working days.
Anything the host already knows belongs on that plane: channel, tier, locale, region, whether an id is in hand. The technique is the same one in turning parts of your system prompt on and off, and its value goes up when the reader gets cheaper, because a resolved conditional costs the new model nothing to get right.
7. Name the judgment calls you cannot remove. Some of what the advisories flagged is real policy that arrived as an adjective. Professional and appropriately both stood for something a person could describe, and neither survived being written down. Here is the file after the inventory, on the cheap model.
<$CONFIG> $CONFIG model claude-haiku-4-5 - via:: anthropic - max_tokens:: 400 $REQUIRE variable tier: one of standard, premium $IMPORT tool get_order </$CONFIG> $SERVICE anthropic - base:: https://api.anthropic.com - headers:: - x-api-key:: @{env.ANTHROPIC_API_KEY} # Role @@ role: delivery and billing for a mobile network You are a support agent for Cobalt Mobile. # Constraints @@ identity: a bill is account data, so it never moves before identity does - You MUST confirm a date of birth before you discuss a bill. @@ brevity: "professional" was a tone note, and this is the part of it a reply can fail - You MUST answer a delivery question in at most 3 sentences. @@ complaint: "appropriately" was a judgment call, so name the move it stood for - WHEN a customer complains THEN you MUST apologize in at most 1 sentence. @@ lookup_first: a delivery date lives in one system, and it is not this file - You MUST call @[get_order] before you state a delivery date. $SWITCH ON @{tier} - standard:: You MUST offer a callback within 2 working days. - premium:: You MUST offer a callback within 4 hours. $TEST delivery - input:: When will my phone arrive? - expect:: - at most 3 sentences $TEST complaint - input:: This is the third time I have called about this and nobody calls me back. - expect:: - contains "callback"
tier variable got a job instead of a deletion, which is why the two warnings on line 5 are gone.Note what did not happen. Nobody wrote a longer prompt for the smaller model, and nobody added be careful or think step by step. Four rules became four rules. Two of them stopped being adjectives.
§4Ship it with a gate and an exit
8. Re-pin the behaviour with $TEST, offline first and then against the candidate. The suite is the only instrument in this list that touches the actual model, and it has two modes for a reason. --dry validates every case and reports rule coverage with zero model calls, so it runs in CI on every commit regardless of who is paying for inference. Then point the same suite at the model you are considering.
$ npx typeglish test support-final.tg --dry ✓ support-final.tg coverage: 2/4 rules exercised · delivery — "When will my phone arrive?" (not run) ✓ at most 3 sentences · complaint — "This is the third time I have called about this and nobody c" (not run) ✓ contains "callback" ✓ 1 prompt — 0 failed $ npx typeglish test support-final.tg --model claude-haiku-4-5 ✗ support-final.tg coverage: 2/4 rules exercised ANTHROPIC_API_KEY is not set — live runs send each case to a model. Export the key, or use --dry for offline suite validation + coverage.
2/4 rules exercised says half the rules in this file have no case behind them, so a live comparison between two models says nothing about those two rules either way.Prefer deterministic asserts over judged prose here specifically, because a judge is itself a model call and you are in the middle of changing which model you trust. at most 3 sentences and contains "callback" give the same verdict on every run and on every tier; we put the two instruments against each other and the asserts are the ones that settle in CI.
9. Set the ratchet at the number, and keep the exit. A grade floor is too coarse to catch a regression: this file passes --min B at 88 and would still pass it at 80.
$ npx typeglish score support-final.tg --min B && echo pass pass # B (88/100), exit 0 $ npx typeglish score support-final.tg --min 85 && echo pass pass # exit 0 $ npx typeglish score support-final.tg --min 90 || echo blocked blocked # exit 1
And keep the way back. The build manifest records a sourceSha256 and an artifactSha256 per build, so the artifact that was running before the swap is a file you already have, addressable by hash. Rolling back a model change is redeploying a text file, which takes about as long as it sounds, and it is worth having established that before you need it at 4pm on a Friday.
§5Common questions
- Will my system prompt still work on a smaller or cheaper model?
- No checker can answer that, and any tool that claims to is guessing. What you can do before you find out in production is enumerate every place the prompt left a decision to the reader's judgment, because judgment is the budget a smaller model has less of. Run
typeglish checkand read each advisory as one delegation:prompt/hedgingis a rule you made optional,prompt/unmeasurableis a rule with nothing to comply with,prompt/vagueis a judgment call with no criterion. On a live support prompt that list was five findings long and the score was D (57/100) withhardness 66. - Why did my AI agent get worse after I changed models?
- Usually because nothing about the prompt changed. Put the model id in the
.tgwith$CONFIG modeland build two versions that differ only in that id, and both artifacts come back with the same hash:237e0fbf0c32for a support prompt onclaude-opus-4-8and onclaude-haiku-4-5.$CONFIGnever reaches the model, so a swap changes the reader and leaves the text alone. Every instruction that used to work because the old model filled a gap is still exactly as unfilled as it was. - How do I test a prompt against a new model before I switch?
- Write the behaviour you are afraid of losing as
$TESTcases and run the suite twice.typeglish test --dryis fully offline: it validates every case and reports rule coverage with zero model calls, so it belongs in CI whatever model you deploy. Thentypeglish test --model claude-haiku-4-5answers each case with the candidate model, which needsANTHROPIC_API_KEY(without it the run exits non-zero and says so). Read the coverage number as well as the pass:2/4 rules exercisedmeans half your rules have no case. - Should the model id live in the prompt file or in the deploy config?
- In the prompt file, so the swap is a reviewable diff instead of a dashboard setting nobody can date.
$CONFIG model claude-haiku-4-5with a- via::naming a$SERVICEis checked: a via that names no service is a blockingconfig/dangling-service, and a misspelled kind is a blockingconfig/unknown-kindwhose message says the line was stripped and configured nothing. One gap to know about: a misspelled param is onlyconfig/unknown-paramat info level, so- maxtokens:: 400configures nothing andcheckstill exits 0.
The through-line of this list is that a cheaper model punishes exactly the prompt defects that were already defects. Nothing here is advice you would refuse on a frontier model; the swap just removes the cushion that let you skip it. The same is true of the sentences that are not rules at all: a capability claim like you have access to the order history costs you more when the reader is less able to notice it was never true, which is the certainty post from the same morning as this one. Do the inventory before the cutover, not after, because after the cutover every finding looks like it was caused by the model.