Opening

A logistics company's contract team asked the model: list the main data from this file. And the answer came back as a friendly little paragraph.

The contract is with a carrier called Aurora. Total amount, twelve thousand four hundred dollars. Starts in January, automatic renewal, twelve months. A regular expression pulled out the total. And it worked for weeks.

Then one day the model wrote the amount in words. Twelve thousand four hundred dollars, spelled out.

And for the start date, just January.

The system stored the total as empty, and the start date as empty. Nobody noticed until finance reconciled the next month.

And the parser also let through things that only looked right. A contract ending in one year and starting in the next went in, because the text had both dates, and the regular expression grabbed both. In this lesson, you'll do the refactoring that ends this, safely, with a measurement before and after.

The idea

The solution has three moves.

Define a schema for what this step outputs.

Ask the model to answer in that schema, using the API's structured output feature. Not by asking in the prompt.

And validate the result in code, before it moves on.

Before and after

Before, the format is a convention nobody wrote down. The model writes text, a regular expression guesses, and whatever it finds goes into the system. Including nothing.

After, the format is a contract, checked at both ends. Same extraction call, now with a schema attached.

The API returns an object that follows the schema. Right types, required fields present.

Code checks the rules a schema can't express, like dates that make sense together.

And only valid data reaches the destination. Failures go to review, with the reason attached.

Here's the same contract, now as an object. The tax ID, digits only, because the schema asked for that. The total as an integer, in cents. Dates in a fixed format. Automatic renewal, true or false.

No twelve thousand in words. And if the file has no end date, that field comes back empty, explicitly. Not as a phrase like open ended term that the old parser never expected.

Why it works

With free text, the format is an implicit promise. The model usually writes it one way, and your code counts on that. With structured output, the format becomes part of the call. The API constrains what the model can generate, token by token, so the answer always closes into valid data with the required fields.

The schema also improves extraction. Clear field names, types and descriptions work as instructions. Total in cents, as an integer, says much more than the total amount. The model doesn't have to decide whether to write dollars, cents, or the number in words.

And the error moves. Instead of an empty field that shows up weeks later, the failure appears right away, in the step that failed. Reviewers get the object and the reason. Not a paragraph to reread.

A challenge

Here's one for you. Your schema has a required penalty field, an integer. But half the contracts in the folder have no penalty clause at all. The eval shows a penalty filled in on those cases anyway, almost always zero, or a number copied from another field. What do you do?

Make it optional, and validate in code whatever comes back filled in. A required field on a document that sometimes lacks the data pushes the model to invent it.

Telling the prompt to use zero when there's no penalty makes zero mean two things. A zero penalty, and a missing penalty. The next step can't tell them apart. And dropping the field throws away the half of the contracts where legal really needs it. Absence is modeled with optional, not by deleting the field.

Safe mechanics

Now the mechanics, in an order that keeps you safe.

Step zero is measuring. Build an eval from real documents, with the expected value for each field. Run the current version, and record accuracy per field and the empty field rate. A field that already comes back empty twenty percent of the time, because the regular expression failed, is the first one to benefit.

Then list what the next step really uses. If the system only stores the tax ID, the total, the dates and the renewal, those are your fields. Summaries, adjectives and main obligations go away.

Write the schema with strong types. Integers for cents, real dates, closed lists. And a short description on each field. That description is the field's prompt.

And decide honestly what's optional. Don't let the model invent an end date for a contract with no end.

Then swap the call for the structured version, and delete the old parser. While both exist, someone will just fix the regular expression one more time.

Add business validation in code. Line items that add up to the total. An end date after the start date. Tax ID check digits.

Send failures to review, with the object and the reason, instead of saving partial data.

And measure again. Accuracy per field should hold or rise. Empty fields caused by parsing should drop to zero. Reading errors still exist, like the model reading twelve thousand four hundred as one thousand two hundred forty. But now they show up in validation, not weeks later.

Risks

A few risks to watch. A schema that forces invention. We just saw it with the penalty.

A schema that's too big. Dozens of nested fields, and quality drops. The model spreads its attention and starts filling in by guesswork, usually in the fields in the middle. If the document has two parts, like the cover and the annexes, split it into two calls, each with what the next step actually uses.

Trusting only the schema. Valid data can still be wrong. A total of one cent is a perfectly legal integer, and an absurd contract. Sum the line items in code, and send it to review when the numbers don't match.

And some rules the provider's structured mode won't accept, like certain patterns. That's fine. Ask for a plain string, and validate the fourteen digits in code, right after the call.

And the most common mistake. Swapping the regular expression for structured output, and leaving the total typed as a string. The model is still free to return the amount with a currency symbol, as a plain number, or in words. You just moved the parser into a field with a nice name. A strong type is what closes the door.

Beyond extraction

This refactoring isn't only for files. Picture two agents sharing a block of text, a kind of process scratchpad. One writes: customer ok, proceed. The other reads that as a credit approval.

Same cure. The scratchpad becomes an object, with a field like credit status, from a closed list of values, validated before the next agent reads it. Prose in the middle becomes a contract, and the second agent stops interpreting okay.

Wrap-up

So here's the idea to take with you. The parser that guesses is not a bug you fix one more time. It's a contract nobody wrote. Measure, write the contract as a schema with strong types, validate what a schema can't express, and delete the parser for good.

In the next lesson, we look at the same extraction from another angle: the model doing math and rules that code would get right every time. LLM doing deterministic work. See you there.