How to write the rules for a support agent that gets sent photos
The moment your chat agent can see, the highest-volume thing customers do is send it a picture of a broken thing and ask what happens now. The four lines most prompts grow to cover that are all clean, and the vaguest one scores highest.
TL;DR Declare the attachment as a typed input and branch on it, write one rule for what the photo is allowed to prove and one for what must never come back out of it, and give the retry a floor. The shipped file is 0 error, 0 warning, 0 info, unchanged under --strict, at B (89/100), and the rule it does not contain is the one about deciding the warranty claim.
Meridian Appliances sells fridges and washing machines through a website and a chat agent. Since the agent got vision, the single highest-volume thing a customer does is send a photograph of a dented door with three words under it: this arrived today. The prompt grew a section to cover that, in one sprint, and it is the section everybody writes. It is four lines long, it checks clean, and it has decided nothing at all.
§1The four lines everybody writes
# Role You are a chat support agent for Meridian Appliances, an appliance retailer. # Photos - MUST look at the photo the customer sends. - MUST decide whether the damage is covered by the warranty. - IF the photo is unclear THEN ask for another one. - MUST NOT share any personal information you can see in the photo.
$ typeglish check photos.tg photos.tg:6:1 info prompt/unintroduced-definite "the damage" retrieves something this document never introduces - a model must guess which damage is meant. photos.tg:6:1 info prompt/agentless-passive The subject is not the doer (passive). If the by-phrase names the doer, put the doer first. ✓ 1 file - 0 error, 0 warning, 2 info $ typeglish score photos.tg photos.tg - C (75/100) proven errors: none tiers: base+z3+spell planes runtime 83 (what the model reads) · hygiene 50 (source only) facets enforceability 57 x.21 · hardness 100 x.12 · directness 83 x.08 consistency 100 x.17 · structure 100 x.12 · annotation 0 x.12 style 70 x.08 · security 100 x.08 lever annotation 0/100 (up to +12 overall)
§2Decide what the photo is allowed to prove
Before any of the mechanics, the question that actually needs answering: a photograph is evidence of how something looked at one moment. It is not evidence of what caused the fault, when it happened, who owns the appliance, or whether the warranty covers it. Every one of those is a fact the photo is silent about and your prompt is about to assert.
That boundary belongs in the file as a declaration rather than in everyone's head, because a declaration both binds a name and renders as a sentence the model reads:
# Role You are a chat support agent for Meridian Appliances, an appliance retailer. You handle a fault report about one appliance, and the customer may send a photo of it. A photo IS evidence of how an appliance looks, and not evidence of what caused a fault.
Now the same policy in seven spellings, one rule per file, checked and scored. This is the part worth staring at.
the rule check score MUST look at the photo the customer sends. 0 error, 0 info B (81/100) MUST check the photo for damage. 0 error, 0 info B (87/100) MUST decide whether the damage is covered by 0 error, 2 info C (73/100) the warranty. MUST describe what the photo shows before you say 0 error, 0 info C (79/100) what it proves. NEVER state what caused a fault that appears in 0 error, 0 info C (79/100) the photo. IF the photo is unclear THEN ask for another one. 0 error, 0 info C (75/100) MUST ask for at most 1 more photo when the photo 0 error, 0 info B (87/100) is too dark to describe.
MUST check the photo for damage is the shortest, loosest rule in the table and it ties for the top score, level with the one line that carries a real bound. The two rules that actually encode the boundary, describe before you conclude and never state a cause, sit six points below it at C (79/100). The score is measuring how cleanly a rule is phrased, not whether it is possible to obey.A photo is an input. Everything your prompt says it proves is an inference you wrote.
§3Declare the input before you point at it
The four-line version assumes the photo is there. In production it very often is not: the customer describes the dent in words, or sends a PDF of the delivery note, or sends the picture three turns after the agent has already committed to an answer. A rule that begins the photo has quietly asserted that a photo exists.
Pointers are the one place TypeGlish will not let that slide. Reference an attachment you never declared and the file refuses to build:
$ typeglish check photos.tg # - MUST describe what @{attachment} shows before ... photos.tg:6:22 error structure/undefined-ref Dangling pointer - @{attachment} names nothing. Declare it with "$REQUIRE variable attachment" (a runtime variable), or "attachment IS ..." (a fixed value). ✗ 1 file - 1 error, 0 warning, 0 info
@{} is a variable, @<> a section, @[] a tool. A pointer whose target does not exist is a compile error in all three families, which is what makes the declaration compulsory rather than polite.Declare it with a typed domain and the branch becomes a member table the compiler can count. Three things can arrive with a message: an image, some other file, or nothing but text. Write two arms and the third one is a blocking error that names the member you dropped.
<$CONFIG> $REQUIRE variable attachment: one of image, document, text </$CONFIG> # Role You are a chat support agent for Meridian Appliances, an appliance retailer. You handle a fault report about one appliance, and the customer may send a photo of it. # What arrived with this message $SWITCH ON @{attachment} - image:: Read it alongside the order record before you answer. - document:: Ask the customer for a photo instead.
$ typeglish check photos.tg # attachment: one of image, document, text photos.tg:10:1 error structure/non-exhaustive-switch @{attachment} can be text, but no arm handles it - add a "- <member>::" row for each (or a deliberate "- otherwise::" fallback). ✗ 1 file - 1 error, 0 warning, 1 info $ typeglish check photos.tg # $REQUIRE variable attachment (no domain) photos.tg:10:1 error structure/opaque-switch @{attachment} has an open domain (untyped) - a $SWITCH on it needs a "- otherwise::" row (members can't cover an open domain). ✗ 1 file - 1 error, 0 warning, 1 info
§4What must never come back out of the image
A photograph of a dented fridge door contains a dented fridge door and whatever else was in the kitchen. Customers photograph their appliance next to the delivery note, screenshot their banking app with the transaction and the balance in the same frame, and send pictures with a child in the reflection. None of that was offered to you deliberately, and all of it is now in a transcript that an agent may quote back.
Write the rule. Then be clear with yourself about what writing it buys:
- It documents the boundary for the next person who edits the file, in one line, next to the rules it constrains.
- It gives a test something to assert against, which is the only mechanism here that can actually fail.
- It does not make the checker enforce anything.
NEVER read out a name, an address or a card number that appears in the photois0 error, 0 warning, 0 info, and a file with no such rule at all also reportssecurity 100. The facet grades the source text, not the pixels. Redaction is a runtime step, and the prompt is the place you write down that it is expected. The same gap, from the other direction, is why your agent will read out your API key.
§5What ships
Six rules, one branch, one worked example, one test. Every line that points at a photo sits under a declaration that says what a photo is, and the retry rule has a floor underneath it so send me another one cannot run forever.
<$CONFIG> $IMPORT tool log_photo_note $REQUIRE variable attachment: one of image, document, text </$CONFIG> # Role @@ role: the desk this file speaks for You are a chat support agent for Meridian Appliances, an appliance retailer. @@ world: the objects the rules below point at, introduced before they are used You handle a fault report about one appliance, and the customer may send a photo of it. @@ scope: what a photo is allowed to establish, written once so every rule can lean on it A photo IS evidence of how an appliance looks, and not evidence of what caused a fault. # What arrived with this message $SWITCH ON @{attachment} - image:: Read it alongside the order record before you answer. - document:: Ask the customer for a photo instead. - text:: Ask the customer to describe the fault in at most 2 sentences. # Photo rules @@ describe: the description is the durable record - the image is not in the ticket - MUST describe what the photo shows in at most 2 sentences before you say what it proves. @@ log: the next person to open this case reads the note, not the attachment - MUST record that description with @[log_photo_note]. @@ cause: a cause is a warranty decision, and a warranty decision is not a thing you can see - NEVER state what caused a fault that appears in the photo. @@ pii: a photo carries more than the customer meant to send - NEVER read out a name, an address or a card number that appears in the photo. @@ retry: one more photo, then a human - the loop needs a floor - MUST ask for at most 1 more photo when the photo is too dark to describe. @@ floor: the escalation is what makes the retry rule terminate - MUST escalate the fault report to a human agent after 2 photos you cannot describe. $EXAMPLE damaged_door - input:: [photo of a dented fridge door] - good:: The photo shows a dent on the lower left of the door. I have logged that and booked an engineer. - bad:: That dent was caused in transit, so it is covered. $TEST describes_before_concluding - input:: [photo of a cracked oven glass] - expect:: - at most 3 sentences
$ typeglish check photos.tg ✓ 1 file - 0 error, 0 warning, 0 info $ typeglish check photos.tg --strict ✓ 1 file - 0 error, 0 warning, 0 info $ typeglish score photos.tg photos.tg - B (89/100) proven errors: none tiers: base+z3+spell planes runtime 89 (what the model reads) · hygiene 91 (source only) facets enforceability 63 x.21 · hardness 100 x.12 · directness 90 x.08 consistency 100 x.17 · structure 100 x.12 · annotation 82 x.12 style 100 x.08 · security 100 x.08 lever enforceability 63/100 (up to +8 overall) $ typeglish test photos.tg --dry ✓ photos.tg coverage: 1/6 rules exercised · describes_before_concluding - "[photo of a cracked oven glass]" (not run) ✓ at most 3 sentences ✓ 1 prompt - 0 failed $ typeglish build photos.tg ✓ built .typeglish/dist/photos.txt ← photos.tg (b48a31fe9c2a, full)
coverage: 1/6 rules exercised is the honest number and it is the one to chase next. One test, six rules, and the five untested ones include both prohibitions. A test suite that exercises a prohibition needs a case designed to tempt it, which is harder than it sounds and is exactly the work.The branch is still open in that artifact, because nothing has told the compiler which member is live. Bind it and the losing arms are gone before the model sees anything:
$ typeglish build photos.tg # template build # What arrived with this message If attachment is image: Read it alongside the order record before you answer. If attachment is document: Ask the customer for a photo instead. If attachment is text: Ask the customer to describe the fault in at most 2 sentences. $ typeglish build photos.tg --vars '{"attachment":"image"}' # What arrived with this message Read it alongside the order record before you answer.
b48a31fe9c2a unbound and e6e378cf02cd with the image arm resolved. If your runtime knows what arrived before it assembles the prompt, the agent never reads the two branches that do not apply, and never has to decide which of them it is in.§6The example you cannot write
One limit worth naming, because it is the first thing a careful person reaches for. An $EXAMPLE block takes a text turn, so there is no way to put an actual image in the prompt as a demonstration. What you can write is a stand-in, - input:: [photo of a dented fridge door], and it compiles clean and reaches the model as a User line reading exactly those words.
That is less useless than it looks. The value of the block was never the picture; it was the shape of the reply. The - good:: half pins the order of operations (describe what you see, then act on it) and the - bad:: half is held out of the compiled prompt entirely, so That dent was caused in transit, so it is covered lives in the source as documentation of the failure mode without ever being demonstrated to the model. Real image coverage belongs in an eval set with real files, and that is a different artifact from the prompt. The general version of this argument, for the case where the failure is a lookup rather than an inference, is how to write what your agent says when the lookup fails.
§7Common questions
- How do I handle image attachments in an AI support agent prompt?
- Declare what arrived as a typed input and branch on it, rather than writing rules that assume a photo is there.
$REQUIRE variable attachment: one of image, document, text, then a$SWITCH ON @{attachment}with one arm per member. Leave a member out and the file is a blockingstructure/non-exhaustive-switcherror naming it; leave the variable untyped and it is a blockingstructure/opaque-switcherror instead. Both refuse to build, which is the point: the no-photo path is the one teams forget, and this is the only version of the prompt that cannot forget it. - Can an AI agent decide a warranty claim from a photo?
- It can produce a decision. It cannot see the thing the decision depends on. A photo is evidence of how an appliance looks at one moment, and coverage turns on what caused the fault and when, which is not in the frame. Write the boundary down as a declaration,
A photo IS evidence of how an appliance looks, and not evidence of what caused a fault, and pair it withNEVER state what caused a fault that appears in the photo.MUST decide whether the damage is covered by the warrantychecks at0 error, 0 warning, 2 infoatC (73/100), and neither info is about the impossible inference. - How do I stop my agent reading personal information out of a screenshot?
- Write the prohibition as a rule naming what may not be read out, and then stop expecting the checker to enforce it.
NEVER read out a name, an address or a card number that appears in the photois0 error, 0 warning, 0 info, and a file with no such rule at all also scoressecurity 100. The facet grades the source text, not the pixels, so the rule is documentation plus a hook for a test, and the enforcement has to be a redaction step in the runtime. - Can I write a few-shot example for an image in a prompt?
- Not as an image. An
$EXAMPLEblock takes a text turn, so the closest you get is a stand-in like- input:: [photo of a dented fridge door], which compiles clean and ships to the model as aUserline reading exactly that. It is still worth writing, because the- good::half pins the shape of the answer (describe, then act) and the- bad::half is held out of the compiled prompt, so That dent was caused in transit, so it is covered documents the failure without demonstrating it. Real image coverage belongs in an eval set, not in the prompt.
The first draft of the shipped file used the members photo, document, none and came out at 0 error, 0 warning, 2 info, with a prompt/unregistered-doer on the photo arm and another on the none arm. Nothing was wrong with the arm bodies: the finding was reading the keys. A bare generic noun standing alone never enters the world model, and photo and none both read as one. Renaming the domain to image, document, text took the file to zero findings with every rule byte-identical. Worth knowing before you spend twenty minutes rewriting arm bodies that were fine, and worth remembering as a general shape: in a language where the member names are English, the member names are parsed.