Opening

Friday, five in the afternoon. Someone on the product team opens the settings panel and edits the ticket classifier prompt.

They swap the word urgent for high priority, to sound a bit more polite. Nobody runs a test. After all, it's just text.

On Monday, the urgent queue is empty. The code that read the model's output was looking for the word urgent. Half of the critical tickets spent the whole weekend in the regular queue.

The prompt was an interface. Nobody treated it like one. In this lesson, you'll see what it means to treat it that way, and the four questions that turn a prompt from a draft into a contract.

The idea

A production prompt is an API contract, between your code and the model. It has the same parts as any interface you already version and test. You'd never let someone rename a field in a public API from a settings panel on a Friday afternoon. The prompt deserves the same respect.

How it works

Let's go through the contract. The calling code knows exactly what it sends, and what it expects back.

The input has a defined shape. Which variables, what types, and where they come from.

The prompt has a name, a version and an owner. Changing the prompt is changing the interface.

The output follows a schema, with fixed fields, not free text that someone has to interpret.

The code validates that output before using it, just like it would with any external API response.

And a set of test cases, the eval, runs against every new version of the prompt.

That Friday afternoon broke steps four and six at the same time. The word in the output was part of the contract. And nobody had a test that failed when it changed.

The four questions

Before any prompt goes to production, answer four questions.

Input. What goes in, and from where? For the classifier: the ticket title and body, as plain text, with personal data already removed.

Output. What format comes out? A priority field that can only be urgent or normal, and a reason field with a short explanation.

Version. How do I know which one ran? Ticket classifier, version seven, recorded on every trace.

Test. How do I know it didn't get worse? Two hundred labeled tickets, run on every change.

If any answer is I don't know, the prompt is still a draft. Even if it's already in production.

And on input, also write down what must not go in. Support tickets at a Brazilian bank often carry a national ID number, a payment key, or pieces of a receipt. That's personal data under Brazil's data protection law. The contract says your code masks it before the call. And the eval has a case proving sensitive data doesn't leak to the provider.

That goes for the test cases too. A private repository doesn't cancel the law. If a real ID number sits in a test case, it goes to the provider on every eval run. Mask the fixture as well.

A challenge

Here's one for you. On Monday, the urgent queue is empty. Friday's trace says ticket classifier, version seven. But the settings panel doesn't record any version. What does that already tell you?

It tells you the change went around the contract. The trace says version seven, but the text that actually ran isn't version seven from the repository. You can't match what ran to the versioned file. So the trace can't help you investigate.

And an eval alone wouldn't save you either. It would test the file in the repository, while production runs something else. The next step is to take the prompt out of the panel, version the file, and reject any output the schema doesn't know.

The common mistakes

The most common trap is exactly that settings panel. It looks harmless, because the text is right there, visible. In practice, it's a deploy with no diff, no review, and no eval.

And the person who edited it on Friday wasn't being reckless. The path the product offered was the wrong one. Fix the path, not the person.

The other extreme breaks too. Three copies of the same prompt, in three services, each one almost the same. A bug gets fixed in one, and the other two stay stale. That smell is called scattered prompt strings. The cure is one versioned template, reused by every service.

And there's a quieter one. Say the schema asks for priority and reason. In four percent of cases, the model returns only the priority. And the code quietly fills in a default reason and moves on. That's the same fragility as searching the text for the word urgent. Incomplete output should fail validation, and go to a retry or a review queue. If the field is really optional, say so in the schema.

Try it

Now take one of your own prompts and write its contract, in five minutes, in a file right next to the prompt. A name, a version, an owner.

What goes in, and what must stay out.

If you couldn't fill in the output without hesitating, your code probably parses free text. That's a smell called free text handoff.

And if there's nothing to put next to eval, that's untested prompt.

Then compare the contract with the code that consumes the output. If the code looks for a Portuguese word, and the contract promises an English value, you just found Friday's bug before it happens. And the fix isn't translating the schema. It's reading only the validated field, and deleting the word search.

In production

So what changes in practice? The prompt lives in the repository, in its own file. Not scattered as strings across the code.

Prompt changes go through review, like any other code change.

Output is never read by searching the text for a word. It's read from a field.

And every call records the prompt version on the trace, so you know what ran when something goes wrong.

And a small change still follows the ritual. Say product wants a summary of five sentences instead of three. You bump the version, run the eval, and ship only if the length check still passes. The goal isn't a prompt that never changes. It's a prompt you can change with a test, and a way back.

Wrap-up

So here's the idea to take with you. Treating the prompt as an interface doesn't make the model deterministic. It makes the rest of your system ready for when it isn't. And it means your team stops being afraid to touch it.

In the next lesson, we'll see what happens when that interface grows and nobody prunes it: prompt bloat, where every incident left a rule behind. See you there.