Where should my agent prompt's tests live?

Inline, in the .tg file, unless something concrete pushes them out. A .tg and its sibling .test.tg merge into one suite, so the layouts tie on check, score, rule coverage and the artifact hash: the same three rules and two cases give 0 error, 0 warning, 0 info, B (85/100) with byte-identical facets, coverage 1/3, and the artifact 415e35be65a8 either way. What differs is failure. A malformed test inline is a blocking structure/bad-test that refuses to build, so the suite cannot rot unnoticed, and the suite cannot be detached from the rules by a rename. Move to a sibling file when the suite is large or another team owns it, and then add the test files to your CI command by name.

Does a .test.tg file change what my agent receives?

No. Tests are control plane in both layouts: the same prompt built with its cases inline and with its cases in a sibling produces the same artifact and the same short hash, 415e35be65a8. Two cautions. A .test.tg is still an ordinary .tg file, so typeglish build on a directory writes an artifact for it too, and a suite-only file builds to an empty artifact at e3b0c44298fc, which is the SHA-256 of nothing. And any rule you write in the test file counts in the prompt's coverage denominator, taking three rules to 1/4 rather than 1/3, while reaching no model at all.

Why does a broken test block my prompt from building?

Because inline cases are part of the source, and a $TEST block with a malformed field is a blocking structure/bad-test error like any other structural defect. Writing an expectation as - expect:: contains "account number" instead of a list of sub-bullets takes the file to 1 error, exit 1, typeglish build: billing.tg refused - nothing written, and score to F (73/100) with the grade capped at F. The rules did not change. If you would rather a test typo never stopped a deploy, that is the argument for the sibling layout, where the prompt file still checks at 0 error, 0 warning, 0 info and builds.

Can typeglish test run over a directory of prompts?

No. typeglish check takes a directory and expands it, but typeglish test does not: pointing it at one prints typeglish: EISDIR: illegal operation on a directory, read. Use a glob so the test files are named as targets, typeglish test prompts/*.tg --dry, which is also the only way an orphaned suite reports. Rename a prompt and leave its sibling behind, and typeglish test on the renamed prompt prints a tick and no $TEST cases at exit 0 while the directory check stays clean; the glob turns the same tree into testfile/missing-prompt at exit 1.

← Tidelines/Best practicesAug 18, 2026

Bake-off: tests in the prompt vs. a file beside it

A prompt and its evals can live in one file or two. Every number the toolchain prints says the choice does not matter. Then a typo, a rename and a directory build say otherwise.

by TypeGlish team8 min read#best-practices

One suite. Two places to keep it.

TL;DR A .tg file and a sibling .test.tg merge into one suite, so the two layouts tie on check, score, coverage and the artifact hash, and the difference only appears when the suite breaks: a malformed case inline is a blocking structure/bad-test that refuses to build, while the same case in the sibling leaves check, score and build on the prompt file green.

The question arrives about a week after a team starts keeping prompts in git. The rules are forty lines and the eval suite is two hundred, the person who owns the rules is not the person who owns the suite, and somebody asks whether the cases should really be sitting at the bottom of the prompt. TypeGlish supports both answers: $TEST blocks inline, or a sibling file named <prompt>.test.tg that the toolchain picks up automatically. This is the bake-off, refereed the usual way, by running both.

§1The two contenders

A billing agent for a mobile network. Three rules, two cases, and nothing exotic in either: one case checks that the agent asks for an account number, and one checks that it never says a number back when asked about price.

A - billing.tg, one file✓ compiles

# Role
You are a billing support agent for Cobalt Mobile.

# Constraints
- MUST keep every reply to at most 3 sentences.
- ALWAYS ask for an account number before you discuss a bill.
- NEVER state a specific price.

$TEST bill_query
  - input:: Why is my bill higher this month?
  - expect::
    - contains "account number"
$TEST no_price
  - input:: How much is the unlimited plan?
  - expect::
    - matches /^(?!.*[0-9]).*$/

Contender B is the same text, cut along the dotted line. The rules stay in billing.tg; the cases move to billing.test.tg beside it. Nothing declares the relationship, and nothing needs to: the sibling is found by name.

B - billing.tg + billing.test.tg, two files✓ compiles

# Role
You are a billing support agent for Cobalt Mobile.

# Constraints
- MUST keep every reply to at most 3 sentences.
- ALWAYS ask for an account number before you discuss a bill.
- NEVER state a specific price.

billing.test.tg - the sibling suite✓ compiles

$TEST bill_query
  - input:: Why is my bill higher this month?
  - expect::
    - contains "account number"
$TEST no_price
  - input:: How much is the unlimited plan?
  - expect::
    - matches /^(?!.*[0-9]).*$/

A .test.tg is an ordinary .tg file with an extension the tools recognise. Check it on its own and it is a prompt with no rules in it, which is worth remembering in §4.

§2Round 1: nothing can tell them apart

Run everything. Both layouts check clean, both report the same rule coverage, both score identically down to the facet, and both build to the same artifact.

tg check, test and build - both contenders

$ typeglish check billing.tg          # A, one file
✓ 1 file - 0 error, 0 warning, 0 info

$ typeglish test billing.tg --dry
✓ billing.tg  coverage: 1/3 rules exercised
  · bill_query - "Why is my bill higher this month?" (not run)
      ✓ contains "account number"
  · no_price - "How much is the unlimited plan?" (not run)
      ✓ matches /^(?!.*[0-9]).*$/
✓ 1 prompt - 0 failed

$ typeglish build billing.tg
✓ built .typeglish/dist/billing.txt ← billing.tg (415e35be65a8, full)

$ typeglish check billing.tg          # B, two files
✓ 1 file - 0 error, 0 warning, 0 info

$ typeglish test billing.tg --dry
✓ billing.tg (+ /tmp/split/billing.test.tg)  coverage: 1/3 rules exercised
  · bill_query - "Why is my bill higher this month?" (not run)
      ✓ contains "account number"
  · no_price - "How much is the unlimited plan?" (not run)
      ✓ matches /^(?!.*[0-9]).*$/
✓ 1 prompt - 0 failed

$ typeglish build billing.tg
✓ built .typeglish/dist/billing.txt ← billing.tg (415e35be65a8, full)

The same artifact hash, the same coverage, the same case order. One parenthesis, (+ .../billing.test.tg), is the only place in the whole toolchain that mentions where the suite came from.

tg score - both contenders

billing.tg - B (85/100)  proven errors: none  tiers: base+z3        # A
  planes  runtime 97 (what the model reads) · hygiene 50 (source only)
  facets  enforceability 90 x.21 · hardness 100 x.12 · directness 100 x.08 · consistency 100 x.17
          structure 100 x.12 (hygiene) · annotation 0 x.12 (hygiene) · style 100 x.08 · security 100 x.08

billing.tg - B (85/100)  proven errors: none  tiers: base+z3        # B
  planes  runtime 97 (what the model reads) · hygiene 50 (source only)
  facets  enforceability 90 x.21 · hardness 100 x.12 · directness 100 x.08 · consistency 100 x.17
          structure 100 x.12 (hygiene) · annotation 0 x.12 (hygiene) · style 100 x.08 · security 100 x.08

Eight facets, no daylight. Tests are control plane, which delete every test, the score does not move measured the hard way: the suite has no inbound edges, so moving it to another file cannot change a grade it never touched.

Score the sibling on its own and you get the joke the layout makes possible: billing.test.tg - A (100/100), four facets, all of them full marks. A file with no rules in it has nothing to mark down. Do not put that number on a slide.

Round 1 is a tie, and a tie on every instrument is the reason people pick by taste.

§3Round 2: somebody mistypes an expectation

Here is the edit. An engineer adds an expectation the way it reads in their head, on one line, instead of as a sub-bullet under - expect::. It is the single most common malformed field in a suite, and it is the same three characters wrong in both layouts.

A - billing.tg with the typo✗ blocked

# Role
You are a billing support agent for Cobalt Mobile.

# Constraints
- MUST keep every reply to at most 3 sentences.
- ALWAYS ask for an account number before you discuss a bill.
- NEVER state a specific price.

$TEST bill_query
  - input:: Why is my bill higher this month?
  - expect:: contains "account number"
$TEST no_price
  - input:: How much is the unlimited plan?
  - expect::
    - matches /^(?!.*[0-9]).*$/

A - the whole toolchain, one typo

$ typeglish check billing.tg
billing.tg:9:1  error  structure/bad-test  "- expect::" is a list - put each expectation on its
  own "- " sub-bullet beneath it.

✗ 1 file - 1 error, 0 warning, 0 info                                      # exit 1

$ typeglish build billing.tg
typeglish build: billing.tg refused - nothing written

$ typeglish score billing.tg
billing.tg - F (73/100)  proven errors - grade capped at F  tiers: base+z3
  ✖ L7 structure/bad-test: "- expect::" is a list - put each expectation on its own "- "
    sub-bullet beneath it.
  planes  runtime 97 (what the model reads) · hygiene 0 (source only)
  facets  enforceability 90 x.21 · hardness 100 x.12 · directness 100 x.08 · consistency 100 x.17
          structure 0 x.12 (hygiene) · annotation 0 x.12 (hygiene) · style 100 x.08 · security 100 x.08
  lever   structure 0/100 (up to +12 overall) - Fix the ledger rows with structure/ codes; most
          carry a one-click fix.
  L9  structure/bad-test  −2  "- expect::" is a list - put each expectation on its own "- "
      sub-bullet benea

Exit 1, no artifact, and an F on a file whose three rules did not change. runtime 97 is untouched, because the defect is in the source plane and the model was never going to read it.

Now the identical typo in contender B, in billing.test.tg, with the prompt file byte-for-byte the same as it was in §1.

B - the same typo, in the sibling

$ typeglish check billing.tg
✓ 1 file - 0 error, 0 warning, 0 info                                      # exit 0

$ typeglish score billing.tg
billing.tg - B (85/100)  proven errors: none  tiers: base+z3

$ typeglish build billing.tg
✓ built .typeglish/dist/billing.txt ← billing.tg (415e35be65a8, full)

$ typeglish test billing.tg --dry
✗ billing.tg (+ /tmp/r2b/billing.test.tg)
  does not compile - structure/bad-test: "- expect::" is a list - put each expectation on its
  own "- " sub-bullet beneath it.
✗ 1 prompt - 1 failed                                                      # exit 1

$ typeglish check .
billing.test.tg:1:1  error  structure/bad-test  "- expect::" is a list - put each expectation on
  its own "- " sub-bullet beneath it.

✗ 2 files - 1 error, 0 warning, 0 info                                     # exit 1
program: 2 independent files - no $IMPORT compositions

Three of the five commands are green and one of them shipped an artifact. The defect is real and reachable, but only from a command that names the test file: test, or a directory check.

Which behaviour you want here is a genuine argument, and it is the argument. Contender A couples the two: your prompt cannot build while its suite is malformed, which is right if you believe an eval suite is part of the spec, and irritating at 6pm when a test typo is holding a rules change. Contender B decouples them: the prompt is judged on the prompt. The cost is that check and score on the file you actually deploy will tell you everything is fine while the suite beside it has not compiled since March.

A suite that fails loudly is a suite. A suite nothing runs is a directory.

§4Round 3: the rename, and the directory

The sibling relationship is by filename, and filenames move. Split the prompt for a second market, git mv billing.tg billing-uk.tg, and the suite stays where it was.

B - after a rename

$ typeglish test billing-uk.tg --dry
✓ billing-uk.tg
  no $TEST cases
✓ 1 prompt - 0 failed                                                      # exit 0

$ typeglish check .
✓ 2 files - 0 error, 0 warning, 0 info                                     # exit 0
program: 2 independent files - no $IMPORT compositions

$ typeglish test *.tg --dry
✓ billing-uk.tg
  no $TEST cases
✗ billing.test.tg (+ /tmp/r3/billing.test.tg)
  testfile/missing-prompt: billing.test.tg has no sibling "billing.tg" to test - a .test.tg
  tests the .tg beside it.
✗ 2 prompts - 1 failed                                                     # exit 1

$ typeglish test . --dry
typeglish: EISDIR: illegal operation on a directory, read

no $TEST cases with a tick and exit 0 is what a detached suite looks like from the prompt side. testfile/missing-prompt exists and is exactly right, but it only fires when the test file is named as a target, and test does not take a directory the way check does.

Two more findings live in the same corner, and both come from test rather than check. Keep a case inline and add one with the same id to the sibling, and the merged suite reports testfile/duplicate-case: case "bill_query" is defined both inline in billing.tg and in billing.test.tg - ids must be unique across the merged suite at exit 1, while typeglish check . on that same pair prints 0 error, 0 warning, 0 info. The two files are not co-checked, they are co-run.

And then the directory build, which is the one that actually costs something. A .test.tg is a .tg, so typeglish build on a folder builds it.

tg build . - what lands in dist

$ typeglish build .
✓ built .typeglish/dist/billing.test.txt ← billing.test.tg (e3b0c44298fc, full)
✓ built .typeglish/dist/billing.txt      ← billing.tg      (415e35be65a8, full)
✓ built 2/2

$ cat .typeglish/dist/billing.test.txt
$

e3b0c44298fc is the SHA-256 of the empty string. A suite-only file compiles to nothing, gets its own row in .typeglish/build-manifest.json, and lands in the directory a deploy script globs.

The sharper version of the same fact: a .test.tg will hold anything a prompt holds. Write a rule in it, the way a QA team reasonably might when the rule is theirs, and it counts as a rule of the prompt for coverage while reaching no model at all.

a rule in the test file

# appended to billing.test.tg
@@ qa_survey: the QA team's own rule, kept with the suite
- MUST close every reply with a survey link.

$ typeglish test billing.tg --dry
✓ billing.tg (+ /tmp/r4/billing.test.tg)  coverage: 1/4 rules exercised

$ typeglish build .
✓ built .typeglish/dist/billing.test.txt ← billing.test.tg (094e31fe192f, full)
✓ built .typeglish/dist/billing.txt      ← billing.tg      (415e35be65a8, full)

$ cat .typeglish/dist/billing.test.txt
- MUST close every reply with a survey link.

Coverage goes 1/3 to 1/4: the denominator counts a rule that is not in the prompt, and the agent never sees the survey line. The merge is at file scope, not at $TEST scope.

§5The verdict, and the gate either way

Inline wins the default. Not on any measurement in §2, since there are none, but on the two failure modes: the suite cannot silently detach from the rules it tests, and a broken case is a blocking error on the file you deploy rather than a green check with a dead suite behind it. One file also means one diff, and a rules change that arrives without a case change is visible in the same review.

The sibling wins two specific arguments. When the suite is much larger than the prompt and you want the prompt readable end to end, and when a different team owns the cases and you want their commits out of the file that ships. Both are real. Take the deal knowingly: you are trading a coupling for a filename convention, and filenames are not checked by anything.

Either way the gate has to name the tests, because no default command reaches them. This is the minimum that fails on all four defects in this post.

ci.sh - the four lines

#!/usr/bin/env bash
set -euo pipefail

typeglish check prompts                # the directory, so .test.tg files are checked too
typeglish score prompts/billing.tg --min B
typeglish test prompts/*.tg --dry      # the glob, so an orphaned suite reports
typeglish build prompts/billing.tg     # name the prompts; never build the folder into a dist you deploy

Line 3 is the one nobody has. It is also the only line that catches testfile/missing-prompt and testfile/duplicate-case, neither of which any check will ever report.

One thing worth saying plainly, since a bake-off invites a scoreboard: the number that decides whether your suite is worth anything is not in this post. It is coverage, and it is a rate over the rules, not over the conversations you worried about. Deterministic asserts vs. an LLM judge is the argument about what a case should assert; this one is only about which file it sits in.

§6Common questions

Where should my agent prompt's tests live?: Inline, in the .tg file, unless something concrete pushes them out. A .tg and its sibling .test.tg merge into one suite, so the layouts tie on check, score, rule coverage and the artifact hash: the same three rules and two cases give 0 error, 0 warning, 0 info, B (85/100) with byte-identical facets, coverage 1/3, and the artifact 415e35be65a8 either way. What differs is failure. A malformed test inline is a blocking structure/bad-test that refuses to build, so the suite cannot rot unnoticed, and the suite cannot be detached from the rules by a rename. Move to a sibling file when the suite is large or another team owns it, and then add the test files to your CI command by name.
Does a .test.tg file change what my agent receives?: No. Tests are control plane in both layouts: the same prompt built with its cases inline and with its cases in a sibling produces the same artifact and the same short hash, 415e35be65a8. Two cautions. A .test.tg is still an ordinary .tg file, so typeglish build on a directory writes an artifact for it too, and a suite-only file builds to an empty artifact at e3b0c44298fc, which is the SHA-256 of nothing. And any rule you write in the test file counts in the prompt's coverage denominator, taking three rules to 1/4 rather than 1/3, while reaching no model at all.
Why does a broken test block my prompt from building?: Because inline cases are part of the source, and a $TEST block with a malformed field is a blocking structure/bad-test error like any other structural defect. Writing an expectation as - expect:: contains "account number" instead of a list of sub-bullets takes the file to 1 error, exit 1, typeglish build: billing.tg refused - nothing written, and score to F (73/100) with the grade capped at F. The rules did not change. If you would rather a test typo never stopped a deploy, that is the argument for the sibling layout, where the prompt file still checks at 0 error, 0 warning, 0 info and builds.
Can typeglish test run over a directory of prompts?: No. typeglish check takes a directory and expands it, but typeglish test does not: pointing it at one prints typeglish: EISDIR: illegal operation on a directory, read. Use a glob so the test files are named as targets, typeglish test prompts/*.tg --dry, which is also the only way an orphaned suite reports. Rename a prompt and leave its sibling behind, and typeglish test on the renamed prompt prints a tick and no $TEST cases at exit 0 while the directory check stays clean; the glob turns the same tree into testfile/missing-prompt at exit 1.

Field note

Everything above runs offline. test --dry validates the merged suite, attributes coverage and reports testfile/ defects with no model call and no API key, which is what makes it safe to put in the same CI job as check. The moment you drop --dry, every case becomes a request, and the file layout question turns into a bill. Worth deciding the layout while it is still free.

∿ washed up Aug 18, 2026 ∿