> Markdown rendition of https://typeglish.dev/blog/tests-inline-vs-sibling ("Bake-off: tests in the prompt vs. a file beside it - Tidelines"). Canonical page: https://typeglish.dev/blog/tests-inline-vs-sibling · All pages: https://typeglish.dev/llms.txt [← Tidelines](https://typeglish.dev/blog)/ Best practices Aug 18, 2026 # Bake-off: tests in the prompt vs. a file beside it A prompt and its evals can live in one file or two. Every number the toolchain prints says the choice does not matter. Then a typo, a rename and a directory build say otherwise. by **TypeGlish team** 8 min read #best-practices One suite. Two places to keep it. **TL;DR** A `.tg` file and a sibling `.test.tg` merge into one suite, so the two layouts tie on `check`, `score`, coverage and the artifact hash, and the difference only appears when the suite breaks: a malformed case inline is a blocking `structure/bad-test` that refuses to build, while the same case in the sibling leaves `check`, `score` and `build` on the prompt file green. The question arrives about a week after a team starts keeping prompts in git. The rules are forty lines and the eval suite is two hundred, the person who owns the rules is not the person who owns the suite, and somebody asks whether the cases should really be sitting at the bottom of the prompt. TypeGlish supports both answers: `$TEST` blocks inline, or a sibling file named `.test.tg` that the toolchain picks up automatically. This is the bake-off, refereed the usual way, by running both. ## §1 The two contenders A billing agent for a mobile network. Three rules, two cases, and nothing exotic in either: one case checks that the agent asks for an account number, and one checks that it never says a number back when asked about price. A - billing.tg, one file ✓ compiles ```typeglish # Role You are a billing support agent for Cobalt Mobile. # Constraints - MUST keep every reply to at most 3 sentences. - ALWAYS ask for an account number before you discuss a bill. - NEVER state a specific price. $TEST bill_query - input:: Why is my bill higher this month? - expect:: - contains "account number" $TEST no_price - input:: How much is the unlimited plan? - expect:: - matches /^(?!.*[0-9]).*$/ ``` Contender B is the same text, cut along the dotted line. The rules stay in `billing.tg`; the cases move to `billing.test.tg` beside it. Nothing declares the relationship, and nothing needs to: the sibling is found by name. B - billing.tg + billing.test.tg, two files ✓ compiles ```typeglish # Role You are a billing support agent for Cobalt Mobile. # Constraints - MUST keep every reply to at most 3 sentences. - ALWAYS ask for an account number before you discuss a bill. - NEVER state a specific price. ``` billing.test.tg - the sibling suite ✓ compiles ```typeglish $TEST bill_query - input:: Why is my bill higher this month? - expect:: - contains "account number" $TEST no_price - input:: How much is the unlimited plan? - expect:: - matches /^(?!.*[0-9]).*$/ ``` A `.test.tg` is an ordinary `.tg` file with an extension the tools recognise. Check it on its own and it is a prompt with no rules in it, which is worth remembering in §4. ## §2 Round 1: nothing can tell them apart Run everything. Both layouts check clean, both report the same rule coverage, both score identically down to the facet, and both build to the same artifact. tg check, test and build - both contenders ``` $ typeglish check billing.tg # A, one file ✓ 1 file - 0 error, 0 warning, 0 info $ typeglish test billing.tg --dry ✓ billing.tg coverage: 1/3 rules exercised · bill_query - "Why is my bill higher this month?" (not run) ✓ contains "account number" · no_price - "How much is the unlimited plan?" (not run) ✓ matches /^(?!.*[0-9]).*$/ ✓ 1 prompt - 0 failed $ typeglish build billing.tg ✓ built .typeglish/dist/billing.txt ← billing.tg (415e35be65a8, full) $ typeglish check billing.tg # B, two files ✓ 1 file - 0 error, 0 warning, 0 info $ typeglish test billing.tg --dry ✓ billing.tg (+ /tmp/split/billing.test.tg) coverage: 1/3 rules exercised · bill_query - "Why is my bill higher this month?" (not run) ✓ contains "account number" · no_price - "How much is the unlimited plan?" (not run) ✓ matches /^(?!.*[0-9]).*$/ ✓ 1 prompt - 0 failed $ typeglish build billing.tg ✓ built .typeglish/dist/billing.txt ← billing.tg (415e35be65a8, full) ``` The same artifact hash, the same coverage, the same case order. One parenthesis, `(+ .../billing.test.tg)`, is the only place in the whole toolchain that mentions where the suite came from. tg score - both contenders ``` billing.tg - B (85/100) proven errors: none tiers: base+z3 # A planes runtime 97 (what the model reads) · hygiene 50 (source only) facets enforceability 90 x.21 · hardness 100 x.12 · directness 100 x.08 · consistency 100 x.17 structure 100 x.12 (hygiene) · annotation 0 x.12 (hygiene) · style 100 x.08 · security 100 x.08 billing.tg - B (85/100) proven errors: none tiers: base+z3 # B planes runtime 97 (what the model reads) · hygiene 50 (source only) facets enforceability 90 x.21 · hardness 100 x.12 · directness 100 x.08 · consistency 100 x.17 structure 100 x.12 (hygiene) · annotation 0 x.12 (hygiene) · style 100 x.08 · security 100 x.08 ``` Eight facets, no daylight. Tests are control plane, which [delete every test, the score does not move](https://typeglish.dev/blog/test-suite-ablation) measured the hard way: the suite has no inbound edges, so moving it to another file cannot change a grade it never touched. Score the sibling on its own and you get the joke the layout makes possible: `billing.test.tg - A (100/100)`, four facets, all of them full marks. A file with no rules in it has nothing to mark down. Do not put that number on a slide. > Round 1 is a tie, and a tie on every instrument is the reason people pick by taste. ## §3 Round 2: somebody mistypes an expectation Here is the edit. An engineer adds an expectation the way it reads in their head, on one line, instead of as a sub-bullet under `- expect::`. It is the single most common malformed field in a suite, and it is the same three characters wrong in both layouts. A - billing.tg with the typo ✗ blocked ```typeglish # Role You are a billing support agent for Cobalt Mobile. # Constraints - MUST keep every reply to at most 3 sentences. - ALWAYS ask for an account number before you discuss a bill. - NEVER state a specific price. $TEST bill_query - input:: Why is my bill higher this month? - expect:: contains "account number" $TEST no_price - input:: How much is the unlimited plan? - expect:: - matches /^(?!.*[0-9]).*$/ ``` A - the whole toolchain, one typo ``` $ typeglish check billing.tg billing.tg:9:1 error structure/bad-test "- expect::" is a list - put each expectation on its own "- " sub-bullet beneath it. ✗ 1 file - 1 error, 0 warning, 0 info # exit 1 $ typeglish build billing.tg typeglish build: billing.tg refused - nothing written $ typeglish score billing.tg billing.tg - F (73/100) proven errors - grade capped at F tiers: base+z3 ✖ L7 structure/bad-test: "- expect::" is a list - put each expectation on its own "- " sub-bullet beneath it. planes runtime 97 (what the model reads) · hygiene 0 (source only) facets enforceability 90 x.21 · hardness 100 x.12 · directness 100 x.08 · consistency 100 x.17 structure 0 x.12 (hygiene) · annotation 0 x.12 (hygiene) · style 100 x.08 · security 100 x.08 lever structure 0/100 (up to +12 overall) - Fix the ledger rows with structure/ codes; most carry a one-click fix. L9 structure/bad-test −2 "- expect::" is a list - put each expectation on its own "- " sub-bullet benea ``` Exit 1, no artifact, and an F on a file whose three rules did not change. `runtime 97` is untouched, because the defect is in the source plane and the model was never going to read it. Now the identical typo in contender B, in `billing.test.tg`, with the prompt file byte-for-byte the same as it was in §1. B - the same typo, in the sibling ``` $ typeglish check billing.tg ✓ 1 file - 0 error, 0 warning, 0 info # exit 0 $ typeglish score billing.tg billing.tg - B (85/100) proven errors: none tiers: base+z3 $ typeglish build billing.tg ✓ built .typeglish/dist/billing.txt ← billing.tg (415e35be65a8, full) $ typeglish test billing.tg --dry ✗ billing.tg (+ /tmp/r2b/billing.test.tg) does not compile - structure/bad-test: "- expect::" is a list - put each expectation on its own "- " sub-bullet beneath it. ✗ 1 prompt - 1 failed # exit 1 $ typeglish check . billing.test.tg:1:1 error structure/bad-test "- expect::" is a list - put each expectation on its own "- " sub-bullet beneath it. ✗ 2 files - 1 error, 0 warning, 0 info # exit 1 program: 2 independent files - no $IMPORT compositions ``` Three of the five commands are green and one of them shipped an artifact. The defect is real and reachable, but only from a command that names the test file: `test`, or a directory `check`. Which behaviour you want here is a genuine argument, and it is the argument. Contender A couples the two: your prompt cannot build while its suite is malformed, which is right if you believe an eval suite is part of the spec, and irritating at 6pm when a test typo is holding a rules change. Contender B decouples them: the prompt is judged on the prompt. The cost is that `check` and `score` on the file you actually deploy will tell you everything is fine while the suite beside it has not compiled since March. > A suite that fails loudly is a suite. A suite nothing runs is a directory. ## §4 Round 3: the rename, and the directory The sibling relationship is by filename, and filenames move. Split the prompt for a second market, `git mv billing.tg billing-uk.tg`, and the suite stays where it was. B - after a rename ``` $ typeglish test billing-uk.tg --dry ✓ billing-uk.tg no $TEST cases ✓ 1 prompt - 0 failed # exit 0 $ typeglish check . ✓ 2 files - 0 error, 0 warning, 0 info # exit 0 program: 2 independent files - no $IMPORT compositions $ typeglish test *.tg --dry ✓ billing-uk.tg no $TEST cases ✗ billing.test.tg (+ /tmp/r3/billing.test.tg) testfile/missing-prompt: billing.test.tg has no sibling "billing.tg" to test - a .test.tg tests the .tg beside it. ✗ 2 prompts - 1 failed # exit 1 $ typeglish test . --dry typeglish: EISDIR: illegal operation on a directory, read ``` `no $TEST cases` with a tick and exit 0 is what a detached suite looks like from the prompt side. `testfile/missing-prompt` exists and is exactly right, but it only fires when the test file is named as a target, and `test` does not take a directory the way `check` does. Two more findings live in the same corner, and both come from `test` rather than `check`. Keep a case inline and add one with the same id to the sibling, and the merged suite reports `testfile/duplicate-case: case "bill_query" is defined both inline in billing.tg and in billing.test.tg - ids must be unique across the merged suite` at exit 1, while `typeglish check .` on that same pair prints `0 error, 0 warning, 0 info`. The two files are not co-checked, they are co-run. And then the directory build, which is the one that actually costs something. A `.test.tg` is a `.tg`, so `typeglish build` on a folder builds it. tg build . - what lands in dist ``` $ typeglish build . ✓ built .typeglish/dist/billing.test.txt ← billing.test.tg (e3b0c44298fc, full) ✓ built .typeglish/dist/billing.txt ← billing.tg (415e35be65a8, full) ✓ built 2/2 $ cat .typeglish/dist/billing.test.txt $ ``` `e3b0c44298fc` is the SHA-256 of the empty string. A suite-only file compiles to nothing, gets its own row in `.typeglish/build-manifest.json`, and lands in the directory a deploy script globs. The sharper version of the same fact: a `.test.tg` will hold anything a prompt holds. Write a rule in it, the way a QA team reasonably might when the rule is theirs, and it counts as a rule of the prompt for coverage while reaching no model at all. a rule in the test file ``` # appended to billing.test.tg @@ qa_survey: the QA team's own rule, kept with the suite - MUST close every reply with a survey link. $ typeglish test billing.tg --dry ✓ billing.tg (+ /tmp/r4/billing.test.tg) coverage: 1/4 rules exercised $ typeglish build . ✓ built .typeglish/dist/billing.test.txt ← billing.test.tg (094e31fe192f, full) ✓ built .typeglish/dist/billing.txt ← billing.tg (415e35be65a8, full) $ cat .typeglish/dist/billing.test.txt - MUST close every reply with a survey link. ``` Coverage goes `1/3` to `1/4`: the denominator counts a rule that is not in the prompt, and the agent never sees the survey line. The merge is at file scope, not at `$TEST` scope. ## §5 The verdict, and the gate either way **Inline wins the default.** Not on any measurement in §2, since there are none, but on the two failure modes: the suite cannot silently detach from the rules it tests, and a broken case is a blocking error on the file you deploy rather than a green check with a dead suite behind it. One file also means one diff, and a rules change that arrives without a case change is visible in the same review. **The sibling wins two specific arguments.** When the suite is much larger than the prompt and you want the prompt readable end to end, and when a different team owns the cases and you want their commits out of the file that ships. Both are real. Take the deal knowingly: you are trading a coupling for a filename convention, and filenames are not checked by anything. Either way the gate has to name the tests, because no default command reaches them. This is the minimum that fails on all four defects in this post. ci.sh - the four lines ``` #!/usr/bin/env bash set -euo pipefail typeglish check prompts # the directory, so .test.tg files are checked too typeglish score prompts/billing.tg --min B typeglish test prompts/*.tg --dry # the glob, so an orphaned suite reports typeglish build prompts/billing.tg # name the prompts; never build the folder into a dist you deploy ``` Line 3 is the one nobody has. It is also the only line that catches `testfile/missing-prompt` and `testfile/duplicate-case`, neither of which any `check` will ever report. One thing worth saying plainly, since a bake-off invites a scoreboard: the number that decides whether your suite is worth anything is not in this post. It is `coverage`, and it is a rate over the rules, not over the conversations you worried about. [Deterministic asserts vs. an LLM judge](https://typeglish.dev/blog/test-asserts-vs-judge) is the argument about what a case should assert; this one is only about which file it sits in. ## §6 Common questions **Where should my agent prompt's tests live?** Inline, in the `.tg` file, unless something concrete pushes them out. A `.tg` and its sibling `.test.tg` merge into one suite, so the layouts tie on `check`, `score`, rule coverage and the artifact hash: the same three rules and two cases give `0 error, 0 warning, 0 info`, `B (85/100)` with byte-identical facets, `coverage 1/3`, and the artifact `415e35be65a8` either way. What differs is failure. A malformed test inline is a blocking `structure/bad-test` that refuses to build, so the suite cannot rot unnoticed, and the suite cannot be detached from the rules by a rename. Move to a sibling file when the suite is large or another team owns it, and then add the test files to your CI command by name. **Does a .test.tg file change what my agent receives?** No. Tests are control plane in both layouts: the same prompt built with its cases inline and with its cases in a sibling produces the same artifact and the same short hash, `415e35be65a8`. Two cautions. A `.test.tg` is still an ordinary `.tg` file, so `typeglish build` on a directory writes an artifact for it too, and a suite-only file builds to an empty artifact at `e3b0c44298fc`, which is the SHA-256 of nothing. And any rule you write in the test file counts in the prompt's coverage denominator, taking three rules to `1/4` rather than `1/3`, while reaching no model at all. **Why does a broken test block my prompt from building?** Because inline cases are part of the source, and a `$TEST` block with a malformed field is a blocking `structure/bad-test` error like any other structural defect. Writing an expectation as `- expect:: contains "account number"` instead of a list of sub-bullets takes the file to 1 error, exit 1, `typeglish build: billing.tg refused - nothing written`, and `score` to `F (73/100)` with the grade capped at F. The rules did not change. If you would rather a test typo never stopped a deploy, that is the argument for the sibling layout, where the prompt file still checks at `0 error, 0 warning, 0 info` and builds. **Can typeglish test run over a directory of prompts?** No. `typeglish check` takes a directory and expands it, but `typeglish test` does not: pointing it at one prints `typeglish: EISDIR: illegal operation on a directory, read`. Use a glob so the test files are named as targets, `typeglish test prompts/*.tg --dry`, which is also the only way an orphaned suite reports. Rename a prompt and leave its sibling behind, and `typeglish test` on the renamed prompt prints a tick and `no $TEST cases` at exit 0 while the directory check stays clean; the glob turns the same tree into `testfile/missing-prompt` at exit 1. Field note Everything above runs offline. `test --dry` validates the merged suite, attributes coverage and reports `testfile/` defects with no model call and no API key, which is what makes it safe to put in the same CI job as `check`. The moment you drop `--dry`, every case becomes a request, and the file layout question turns into a bill. Worth deciding the layout while it is still free. ∿ washed up Aug 18, 2026 ∿