Structured Output

Intent

Require the model to answer in a defined schema and validate the answer in code before using it. It applies to any output another program will read: extraction, classification, routing, API calls, data headed for a database.

Problem

Bianca automated reading supplier invoices. The examples on this page use Brazilian invoices (NF-e), where every company is identified by a CNPJ, a 14-digit tax ID with two check digits. The prompt asked: “return JSON with the issuer’s CNPJ, the issue date, and the total amount”. In the first week it worked on almost everything.

Then the odd cases started. One invoice came back with the JSON wrapped in triple backticks and the word json. Another returned the total as "R$ 1.234,56", a string, when the system expected a number. A third named the field issuer_tax_id instead of issuer_cnpj. And in one of them the model replied “Sure! Here is the JSON:” before the braces.

Each case became a try, a replace, and a regular expression in the code. The parser grew larger than the business logic, and it still failed a few times a day.

The question this pattern answers: how do you get the model to deliver data your code can use without guessing?

Solution

Stop asking for a format in the prompt text and start declaring the format as part of the call. Current APIs accept a JSON schema (or a Pydantic class, or a Zod object) and constrain generation so the output follows that schema: field names, types, and allowed values.

The schema guarantees shape, not truth. A CNPJ can have 14 digits and still be wrong. That is why the second half of the pattern is validating business rules in code: check digits, a date in the past, a positive amount. Shape guaranteed by the API, content checked by you.

The refactoring that lands here is Replace Free Text with Structured Output. Use it when the JSON parser has become the product.

Structure

flowchart LR
  IN(["1. Invoice text"]):::io --> M(["2. Model with schema"]):::llm
  M --> P["3. Typed parse"]:::tool
  P --> V{"4. Business rules OK?"}
  V -->|"yes"| DB[("5. Accounting system")]:::store
  V -->|"no"| Q((("6. Review queue"))):::human

1. The input is the text that needs to become data.
2. The call carries the schema. The model can only generate outputs in that shape.
3. The SDK turns the response into a typed object, no regular expressions.
4. Code checks the rules the schema cannot express.
5. Valid data goes straight to the system.
6. Data with the right shape and suspicious content goes to a person for review.

Worked example

Bianca receives this invoice (excerpt). Brazilian invoices (NF-e) identify every company by a CNPJ, a 14-digit tax ID with two check digits:

ISSUER: Padaria São Jorge Ltda
CNPJ: 11.222.333/0001-81
Issue date: 14/08/2026
Total: R$ 1.234,56
Items: French rolls 800un  R$ 800,00 | Coffee 20kg  R$ 434,56

With “return JSON” in the prompt, a realistic reply was a short paragraph (“Sure! Here is the JSON:”), triple backticks, the field cnpj_emissor instead of issuer_cnpj, and the total as the string "R$ 1.234,56".

Three shape failures in one reply: surrounding text, the wrong field name, a number as a string with a currency symbol. Bianca’s parser needed all three fixes, and the next week a fourth showed up.

With a native schema, the API can only emit the fields issuer_cnpj, issue_date, total, items. The SDK returns an object. Then Bianca’s code still has work: the CNPJ 11222333000181 has to pass the check digits, the item sum has to match the total, the date cannot be in the future.

The Padaria São Jorge CNPJ in this example is illustrative. In production, use the check-digit validator; fourteen identical digits are already invalid under the rule.

Pseudocode

schema Invoice {
    issuer_cnpj: string        # 14 digits
    issue_date: date
    total: float
    items: list of { description: string, amount: float }
}

function read_invoice(text):
    # 2 and 3. shape guaranteed by the API
    invoice = llm(EXTRACT_PROMPT, text) -> Invoice

    # 4. content checked in code
    errors = []
    if not valid_cnpj(invoice.issuer_cnpj): errors.add("cnpj")
    if invoice.issue_date > today(): errors.add("future date")
    if abs(sum(invoice.items.amount) - invoice.total) > 0.01: errors.add("sum")

    if errors: return human.review(invoice, errors)   # 6
    return accounting.save(invoice)                    # 5

Real-world scenarios

• Invoice and contract extraction. Every document becomes an object with tax fields. The schema ends the fragile parser, and the CNPJ and sum checks catch what the model misread.
• Automatic ticket triage. Queue, priority, and product come back as enums. The code never receives a “super urgent” priority that is not on the list.
• Resume screening with bias control. Each evaluation criterion becomes a field with a score and a justification. The structure makes the decision auditable field by field and lets you check bias by group.

Applicability

• Another program will read the model’s output.
• The answer has fields you need to access by name.
• The possible values of a field form a closed list.
• You will store the result in a database or send it to an API.
• Steps in a chain pass data to each other.

When not to use

• The output is text for a person to read, like a chat reply. Forcing a schema there only gets in the way of the writing.
• The output is a short free-form text and the code only displays it.
• The schema would be so loose (a single answer: string field) that it guarantees nothing. Then it only adds ceremony.

Common mistake: a required field the source does not have

Every NF-e has an issuer. Not every service contract has a penalty. If penalty is required, the model invents “10% standard” so it does not break the schema. In the eval, that shows up as the invention rate: documents without the field should return null.

Mark as optional whatever can be missing. Use an enum for priority, product, invoice type. An open string is the loose schema that lets "super urgent" through.

Structured output is also not for the chat reply a person reads. Forcing JSON there stiffens the writing and does not help the code, because nobody is going to parse it.

How to implement

1. Write the schema before the prompt. Use the narrowest types possible: enum instead of string, number instead of text.
2. Mark as optional whatever can really be missing from the document. A required field that does not exist in the source pushes the model to invent it.
3. Use the SDK’s native structured output (messages.parse with Pydantic or Zod).
4. Keep in the prompt only the explanation of each field, with examples of tricky cases. The format already lives in the schema.
5. Write the business validations the schema cannot cover in code.
6. Decide what happens to invalid data: automatic repair, human review, or discard with a log entry.
7. Build an eval with expected values per field.

Do not evaluate “valid JSON”. With a native schema that almost always passes and hides content errors. Evaluate issuer_cnpj, total, issue_date against the label, after normalization (digits only in the CNPJ, ISO date).

Variations

• Native structured output. The API constrains generation to the schema. It is the default option today.
• Tool as schema. You define a tool with strict: true and the model “calls” it with the data. Useful when the same call also uses other tools.
• Validate and repair. Without native support, you validate the output and, on failure, send the error back to the model to fix. It has its own page, Output Validation and Repair.

Version the schema the way you version the prompt. A new required field pushes invention on old documents in the eval. A removed field breaks code that still reads the old name. Store schema_version on the span next to the prompt version, so a spike in “invalid tax ID” points at the right deploy.

When the provider still does not constrain generation, do not pretend that asking for JSON in the text is enough. Validate, and on failure, enter repair with the parser’s error message. That is a different pattern, and its retry cap is the same cap as any loop.

Cost, latency, and failure modes

How to evaluate

Evaluate field by field. “The JSON was valid” is no longer a metric: with native structured output it almost always is.

• Per-field accuracy: does the value match the label? Normalize before comparing (dates, case).
• Invention rate: on documents where the field does not exist, does the model return null or invent something?
• Validation stop rate: how much goes to human review, and how many of those were real errors.

An eval case:

The label needs the canonical value, not the invoice text. R$ 1.234,56 on the PDF becomes 1234.56 in expected. If you compare string to string, the eval punishes formatting and lets a wrong CNPJ with the right punctuation through.

Include bad documents on purpose: crooked scans, missing fields, two similar amounts on the same page. The easy case does not teach the schema. The case with two totals (the cover and the last page) teaches whether the model guesses the larger one.

{ "input": "invoices/supplier-118.txt", "expected": { "issuer_cnpj": "11222333000181", "issue_date": "2026-08-14", "total": 1234.56 } }

In production

• Log the schema version along with the prompt version on every span.
• Track how many documents fail each validation rule. A rule that suddenly fires a lot usually means a supplier with a new layout.
• Monitor the size of the human review queue and the time until someone looks.
• Periodically sample automatically approved documents to find errors that passed every rule.

Security

Structured output removes one class of risk: your code stops interpreting free text that could contain anything. But field contents still come from a model that read an external document. Treat the values as untrusted input: escape before displaying, use parameters in SQL queries, and never execute a field as code.

The OWASP Top 10 for LLM Applications covers this as insecure output handling: the model is not a trusted source, even when the shape is right. A description field copied into HTML is XSS. A query field interpolated into SQL is injection. A schema does not sanitize.

Pros and cons

Code

The tabs extract the same invoice with a schema and validate the CNPJ and the sum in code. Python and TypeScript use the Anthropic SDK. The OpenAI tab is the same call on another provider. LangChain hides the envelope for both.

from datetime import date

import anthropic
from pydantic import BaseModel

client = anthropic.Anthropic()


class Item(BaseModel):
    description: str
    amount: float


class Invoice(BaseModel):
    issuer_cnpj: str
    issue_date: date
    total: float
    items: list[Item]


def valid_cnpj(cnpj: str) -> bool:
    digits = [int(c) for c in cnpj if c.isdigit()]
    if len(digits) != 14 or len(set(digits)) == 1:
        return False
    for size in (12, 13):
        weights = [6, 5, 4, 3, 2, 9, 8, 7, 6, 5, 4, 3, 2][-size:]
        check = sum(d * w for d, w in zip(digits, weights)) % 11
        if digits[size] != (0 if check < 2 else 11 - check):
            return False
    return True


def read_invoice(text: str) -> Invoice | list[str]:
    # 2 and 3. shape guaranteed by the API
    invoice = client.messages.parse(
        model="claude-opus-5-5",
        max_tokens=4096,
        system="Extract the invoice data. CNPJ as digits only. Amounts in BRL, as numbers.",
        messages=[{"role": "user", "content": text}],
        output_format=Invoice,
    ).parsed_output

    # 4. content checked in code
    errors = []
    if not valid_cnpj(invoice.issuer_cnpj):
        errors.append("cnpj")
    if invoice.issue_date > date.today():
        errors.append("future date")
    if abs(sum(i.amount for i in invoice.items) - invoice.total) > 0.01:
        errors.append("sum")
    return errors or invoice

import Anthropic from "@anthropic-ai/sdk";
import { zodOutputFormat } from "@anthropic-ai/sdk/helpers/zod";
import { z } from "zod";

const client = new Anthropic();

const Invoice = z.object({
  issuer_cnpj: z.string(),
  issue_date: z.string(), // YYYY-MM-DD
  total: z.number(),
  items: z.array(z.object({ description: z.string(), amount: z.number() })),
});
type Invoice = z.infer<typeof Invoice>;

function validCnpj(cnpj: string): boolean {
  const d = cnpj.replace(/\D/g, "").split("").map(Number);
  if (d.length !== 14 || new Set(d).size === 1) return false;
  for (const size of [12, 13]) {
    const weights = [6, 5, 4, 3, 2, 9, 8, 7, 6, 5, 4, 3, 2].slice(-size);
    const check = weights.reduce((acc, w, i) => acc + w * d[i], 0) % 11;
    if (d[size] !== (check < 2 ? 0 : 11 - check)) return false;
  }
  return true;
}

export async function readInvoice(text: string): Promise<Invoice | string[]> {
  // 2 and 3. shape guaranteed by the API
  const { parsed_output: invoice } = await client.messages.parse({
    model: "claude-opus-5-5",
    max_tokens: 4096,
    system: "Extract the invoice data. CNPJ as digits only. Amounts in BRL, as numbers. Date as YYYY-MM-DD.",
    messages: [{ role: "user", content: text }],
    output_config: { format: zodOutputFormat(Invoice) },
  });
  if (!invoice) return ["schema"];

  // 4. content checked in code
  const errors: string[] = [];
  if (!validCnpj(invoice.issuer_cnpj)) errors.push("cnpj");
  if (new Date(invoice.issue_date) > new Date()) errors.push("future date");
  const sum = invoice.items.reduce((acc, i) => acc + i.amount, 0);
  if (Math.abs(sum - invoice.total) > 0.01) errors.push("sum");
  return errors.length ? errors : invoice;
}

from datetime import date

from openai import OpenAI

client = OpenAI()  # reads OPENAI_API_KEY from the environment


def read_invoice(text: str) -> Invoice | list[str]:
    # 2 and 3. shape guaranteed by the API. Invoice is the class from the Python tab.
    invoice = client.chat.completions.parse(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "Extract the invoice data. CNPJ as digits only. Amounts in BRL, as numbers."},
            {"role": "user", "content": text},
        ],
        response_format=Invoice,
    ).choices[0].message.parsed
    if invoice is None:
        return ["schema"]

    # 4. content checked in code
    errors = []
    if not valid_cnpj(invoice.issuer_cnpj):
        errors.append("cnpj")
    if invoice.issue_date > date.today():
        errors.append("future date")
    if abs(sum(i.amount for i in invoice.items) - invoice.total) > 0.01:
        errors.append("sum")
    return errors or invoice

from datetime import date

from langchain.chat_models import init_chat_model

MODEL = "openai:gpt-4o-mini"  # or "anthropic:claude-opus-5-5"
reader = init_chat_model(MODEL, temperature=0).with_structured_output(Invoice)


def read_invoice(text: str) -> Invoice | list[str]:
    # 2 and 3. the schema is the same Pydantic object as the Python tab
    invoice = reader.invoke([
        ("system", "Extract the invoice data. CNPJ as digits only. Amounts in BRL, as numbers."),
        ("human", text),
    ])

    # 4. content checked in code
    errors = []
    if not valid_cnpj(invoice.issuer_cnpj):
        errors.append("cnpj")
    if invoice.issue_date > date.today():
        errors.append("future date")
    if abs(sum(i.amount for i in invoice.items) - invoice.total) > 0.01:
        errors.append("sum")
    return errors or invoice

Framework support

Checked against the OpenAI and LangChain 1.x docs in October 2026.

Check your understanding