> Markdown rendition of https://typeglish.dev/blog/prompt-cache-order ("Sort your prompt by how often it changes - Tidelines"). Canonical page: https://typeglish.dev/blog/prompt-cache-order · All pages: https://typeglish.dev/llms.txt [← Tidelines](https://typeglish.dev/blog)/ Deep dives Oct 18, 2026 # Sort your prompt by how often it changes Your rules are the same for every customer. Your prompt introduces the customer on line two. A prefix cache reads that as a different prompt every time, and nothing in review will tell you. by **TypeGlish team** 8 min read #deep-dives Slowest clock first. **TL;DR** Prompt caching only reuses the identical front of a prompt, so order an agent's system prompt by how often each part changes: rules every conversation shares first, per-plan or per-segment rules next, and the customer's name, account details and today's date last. Most agent prompts introduce the customer in the second paragraph, because that is where a person would. It reads well and checks clean. It also means that, for caching purposes, nearly every token after that line is new for every conversation, and nothing in your review process will ever mention it. ## §1 Every line in your prompt has a clock Look at a system prompt the way your provider's cache does. Some of it is the same for every conversation the agent will ever have: who the agent is, the rules, the tools. Some of it changes by segment: the plan the customer is on, the region, the channel. And some of it changes every conversation: the customer's name, their account number, today's date. Prompt caching, where a provider offers it, works on the front of the prompt. If the leading run of tokens is identical to one it processed recently, that work is reused instead of redone, which is cheaper and gets the first word back sooner. The reuse stops at the first token that differs. Everything after it is processed fresh, however static it is. That gives a support prompt three clocks: lines that never change between conversations, lines that change per segment, and lines that change per conversation. The mental model is that a prompt should be sorted by its clocks, slowest first. Position is no longer only a question of what the model attends to. It decides how much of the prompt every conversation has to pay for in full. ## §2 The customer is on line two Here is a billing chat prompt for Pellow Mobile, written the natural way. The role comes first, then the facts about the conversation, then the rules: billing.tg - the customer introduced up top ✓ A (99/100) ```typeglish $REQUIRE variable customer_name $REQUIRE variable plan_name $REQUIRE variable current_date $TOOL check_usage - description:: Look up the customer's data usage for this month. $TOOL transfer_to_human - description:: Hand the chat to the billing team. # Role @@ role: the chat desk for a mobile network; every later "you" is the agent You are Ada, the billing assistant for Pellow Mobile. @@ date: the model has no clock, so the platform passes the date The date today is @{current_date}. @@ greeting: the customer is signed in, so the agent knows who they are You are speaking with @{customer_name}, who is on the @{plan_name} plan. # Constraints @@ brevity: a chat bubble, not an email - MUST keep every reply to at most 3 sentences. @@ usage_needs_tool: never guess a usage figure - WHEN the customer asks about data usage THEN MUST call @[check_usage]. @@ no_waivers: credits are a billing team decision - NEVER promise a credit. @@ disputes_route: a disputed charge goes to a person - WHEN the customer disputes a charge THEN MUST call @[transfer_to_human]. @@ no_internal: notes are for staff - NEVER share internal notes. @@ roaming_rule: roaming questions need the usage tool too - WHEN the customer asks about roaming charges THEN MUST call @[check_usage]. ``` tg check, score and analyze - billing.tg ✗ 18% potentially cacheable ``` $ typeglish check billing.tg ✓ 1 file — 0 error, 0 warning, 0 info $ typeglish score billing.tg | head -1 billing.tg — A (99/100) proven errors: none tiers: base+z3 $ typeglish analyze billing.tg | head -2 TG A 99 · 274 estimated source tokens 112 estimated compiled tokens · 18% potentially cacheable ``` Clean check, an A, and less than a fifth of the compiled prompt ahead of the first value that changes. The build shows why. The two prose lines in `# Role` flow into one paragraph, and the date and the customer's name land in the first sentence after the agent's own: the artifact - .typeglish/dist/billing.txt ✗ first placeholder in line 2 ``` $ head -2 .typeglish/dist/billing.txt # Role You are Ada, the billing assistant for Pellow Mobile. The date today is {current_date}. You are speaking with {customer_name}, who is on the {plan_name} plan. ``` Everything from `{current_date}` on is different in every conversation's prompt, as far as a prefix cache can tell, including the six rules that are word for word the same for every customer Pellow has. On a real prompt, with a policy section and a dozen tool descriptions below the role, that is thousands of tokens re-read on every new chat to save one sentence of narrative order. Two things are worth noticing. The grade does not move for this, and it should not: the score measures whether the rules can be read, proven and kept consistent, and none of that depends on where the customer's name sits. And this is not a mistake anyone would catch in review, because the prompt reads exactly the way a person would brief a new hire. That is the trap. A prompt is read by a person once and by a cache on every request. ## §3 Move the fast clock to the bottom The fix is a cut and a paste. The two lines that change per conversation move out of `# Role` into a `# Customer` section at the very end, and nothing else in the file changes: billing-sorted.tg - the same lines, sorted by clock ✓ A (99/100) ```typeglish $REQUIRE variable customer_name $REQUIRE variable plan_name $REQUIRE variable current_date $TOOL check_usage - description:: Look up the customer's data usage for this month. $TOOL transfer_to_human - description:: Hand the chat to the billing team. # Role @@ role: the chat desk for a mobile network; every later "you" is the agent You are Ada, the billing assistant for Pellow Mobile. # Constraints @@ brevity: a chat bubble, not an email - MUST keep every reply to at most 3 sentences. @@ usage_needs_tool: never guess a usage figure - WHEN the customer asks about data usage THEN MUST call @[check_usage]. @@ no_waivers: credits are a billing team decision - NEVER promise a credit. @@ disputes_route: a disputed charge goes to a person - WHEN the customer disputes a charge THEN MUST call @[transfer_to_human]. @@ no_internal: notes are for staff - NEVER share internal notes. @@ roaming_rule: roaming questions need the usage tool too - WHEN the customer asks about roaming charges THEN MUST call @[check_usage]. # Customer @@ date: the model has no clock, so the platform passes the date The date today is @{current_date}. @@ customer: per-conversation facts go last, after everything every conversation shares You are speaking with @{customer_name}, who is on the @{plan_name} plan. ``` tg check, score and analyze - billing-sorted.tg ✓ 86% potentially cacheable ``` $ typeglish check billing-sorted.tg ✓ 1 file — 0 error, 0 warning, 0 info $ typeglish score billing-sorted.tg | head -1 billing-sorted.tg — A (99/100) proven errors: none tiers: base+z3 $ typeglish analyze billing-sorted.tg | head -2 TG A 99 · 273 estimated source tokens 114 estimated compiled tokens · 86% potentially cacheable $ tail -2 .typeglish/dist/billing-sorted.txt # Customer The date today is {current_date}. You are speaking with {customer_name}, who is on the {plan_name} plan. ``` Same grade, same rules, same words. The shared run goes from 18% to 86% of the compiled prompt, and the only part left to process per conversation is the sentence that is actually about that conversation. Inside the fast section, the same logic applies at a finer grain. The date changes once a day and the customer changes every chat, so the date goes first. It is a small thing on a prompt this size and a habit worth having on a large one. Within a single conversation, by the way, the original prompt was never the problem. Every turn resends the same system prompt with that customer's name already filled in, so a provider can reuse it across turns either way. What the sorted version adds is reuse across customers: the rules are processed once and shared by every chat that starts while the cache is warm. ## §4 The middle clock: rules that fork by plan Real prompts have a middle speed. Pellow sells two plans, and only pay-as-you-go customers can buy a data add-on. That rule is the same for every pay-as-you-go customer and different for everyone else, so it belongs to the plan's clock, not the conversation's. The tier gets a typed input and a `$IF`, the way [the Premium upsell rewrite](https://typeglish.dev/blog/upsell-line-rewrite) handled a plan-gated offer. Put that chain where it sits logically, next to the other offer rule in `# Constraints`, and the analyzer reads it as a fork: tg analyze - the plan rule mid-list vs. in its own section ✓ 45% to 75% ``` $ typeglish analyze plan-mid.tg | head -2 TG A 97 · 317 estimated source tokens 122 estimated compiled tokens · 45% potentially cacheable $ typeglish analyze plan-last.tg | head -2 TG A 97 · 319 estimated source tokens 125 estimated compiled tokens · 75% potentially cacheable ``` An unbound `$IF` on a runtime input marks the point where the prompt differs between plans, so `analyze` ends the shared run there. Moved into its own section below the shared rules, it gives back 30 percentage points. Here is the sorted file. Three sections, three clocks, slowest first: plan-last.tg - shared, then per plan, then per conversation ✓ A (97/100) ```typeglish $REQUIRE variable customer_name $REQUIRE variable plan: one of payg, unlimited $REQUIRE variable current_date $TOOL check_usage - description:: Look up the customer's data usage for this month. $TOOL transfer_to_human - description:: Hand the chat to the billing team. # Role @@ role: the chat desk for a mobile network; every later "you" is the agent You are Ada, the billing assistant for Pellow Mobile. # Constraints @@ brevity: a chat bubble, not an email - MUST keep every reply to at most 3 sentences. @@ usage_needs_tool: never guess a usage figure - WHEN the customer asks about data usage THEN MUST call @[check_usage]. @@ no_waivers: credits are a billing team decision - NEVER promise a credit. @@ disputes_route: a disputed charge goes to a person - WHEN the customer disputes a charge THEN MUST call @[transfer_to_human]. @@ no_internal: notes are for staff - NEVER share internal notes. @@ roaming_rule: roaming questions need the usage tool too - WHEN the customer asks about roaming charges THEN MUST call @[check_usage]. # Plan @@ addon_offer: only pay-as-you-go customers can buy a data add-on $IF @{plan} is equal to payg: - MAY offer a data add-on. $ELSE: - NEVER offer a data add-on. # Customer @@ date: the model has no clock, so the platform passes the date The date today is @{current_date}. @@ customer: per-conversation facts go last, after everything every conversation shares You are speaking with @{customer_name}, who is on the @{plan} plan. ``` Build it once per plan and you can see where the two prompts part company: tg build - one artifact per plan ✓ first difference in line 13 ``` $ typeglish build plan-last.tg --vars '{"plan":"payg"}' --out-dir payg ✓ built payg/plan-last.txt ← plan-last.tg (135f0c32f2ac, full) $ typeglish build plan-last.tg --vars '{"plan":"unlimited"}' --out-dir unlimited ✓ built unlimited/plan-last.txt ← plan-last.tg (129f140194a4, full) $ cmp payg/plan-last.txt unlimited/plan-last.txt payg/plan-last.txt unlimited/plan-last.txt differ: byte 408, line 13 ``` Twelve identical lines, through the `# Plan` heading. Every rule above it is one shared prefix for both plans; each plan's arm is a shared prefix for everyone on that plan. Sorting has a cost worth naming. Moving lines changes the prompt the model receives, and [the rule-order ablation](https://typeglish.dev/blog/rule-order-ablation) found that position-only edits ship different bytes at the same grade. Here that is the point, but it means the move is a real change: read the artifact, and run the same test cases against the sorted prompt before it goes live. The checker can tell you the rules still agree with each other. It cannot tell you the model weighs a customer fact at the bottom the way it weighed it at the top. And keep the number in proportion. `analyze` estimates the shape of one prompt. Whether a cache actually hits depends on your provider: some cache automatically and some need a marked breakpoint, most have a minimum prompt length below which nothing is cached, and entries expire after a few idle minutes. A quiet overnight queue may never see a warm cache. A busy Monday morning will. ## §5 Common questions **Why is my AI agent's system prompt not getting cached?** Prompt caching reuses an identical leading run of tokens. If a value that changes per conversation, such as the customer's name, the date or an account number, sits near the top of the system prompt, the shared run ends there and everything after it is new work for every conversation. Move per-conversation values below the rules every conversation shares. **Where should the customer's name go in a system prompt?** At the end, in its own section after the rules every conversation shares. The model still reads it before the first customer turn, and the rules above it stay identical byte for byte from one conversation to the next, which is what a prefix cache needs. **Does moving variables to the end change how my agent behaves?** It can. Moving a line ships a different prompt, and the checker grading both versions the same only says the rules still agree with each other. Read the built artifact after the move and run your test cases against it before you deploy. **How do I measure how much of my prompt can be cached?** Run typeglish analyze on the .tg file. Its second line estimates the compiled tokens and the share potentially cacheable, the run before the first value that changes. It describes the prompt's shape, not your bill: providers set their own minimum lengths, cache lifetimes and breakpoint rules. Field note List every `@{...}` pointer and every `$IF` in your prompt and write its clock next to it: never, per plan, per day, per conversation. Then check that the file is sorted by that column, slowest first, and run `typeglish analyze` before and after the move. The runtime fields you list are the same ones [your prompt's other authors](https://typeglish.dev/blog/prompt-has-other-authors) write, so the bottom of the prompt is also where most of the untrusted text ends up, fenced and in one place. ∿ washed up Oct 18, 2026 ∿