Conversational onboarding

The business

A Brazilian fintech that lends to small businesses opens accounts over WhatsApp. Sign-up has required steps: partner details, CPF, CNPJ, proof of address, a photo of an ID document, average revenue, and acceptance of the terms. Many customers start on their phone during work hours and only finish days later, once they find the proof of address at the bottom of a drawer.

Context for readers outside Brazil. WhatsApp is the default channel for businesses in Brazil. The CPF is the individual taxpayer ID and the CNPJ the company tax ID, both with check digits. The LGPD is Brazil’s data protection law, similar to GDPR. KYC here means the required identity checks before the account opens.

WhatsApp is where the customer already is. The partner does not want to install an app just to send a PDF, and the sales team already talks there. The cost of that choice is that sign-up becomes a long, interrupted, out-of-order conversation, with documents in the middle of voice notes and “hi, still there?”.

The goal is to run sign-up as a natural conversation, accept answers out of order, pick up where the customer left off days later, and never expose or keep personal data beyond what is needed. LGPD asks for minimization: the model, the logs, and the support team do not need to see the full CPF to ask what is still missing.

There is also the 24-hour window on WhatsApp’s cloud API: after a day without a customer message, the company may only send pre-approved templates, not free-form model prose. A sign-up that sleeps for three days needs an engine that remembers state and a template that calls the customer back, not a prompt that rereads the history.

What this scenario measures: completed sign-ups, drop-off per step, and exposed personal data.

Version 1

Version 1 was a well-written prompt, and it did well in short tests. The script fit in a paragraph, the model sounded polite, and sales sent the WhatsApp link the same day as the meeting.

You run the fintech sign-up. Follow this order and do not skip steps:
1. Partner name
2. CPF
3. CNPJ
4. Proof of address
5. Photo of the ID document
6. Average monthly revenue
7. Acceptance of the terms
If the customer already sent something, do not ask again.
Save each field with the save_field tool.
Be natural.

flowchart LR
  W(["1. WhatsApp message"]):::io --> H[("2. Entire conversation history")]:::store
  H --> P(["3. Prompt with the sign-up script"]):::llm
  P --> R(["4. Reply"]):::io
  P --> DB[("5. Saves whatever it thinks it collected")]:::store

1. Each customer message arrives through the WhatsApp webhook.
2. Code loads the whole conversation, from the first “hi”.
3. The prompt describes the steps in text.
4. The model replies.
5. When it thinks it collected a piece of data, it calls a tool to save it.

The internal test used five-minute chats, with someone from the team answering in the script’s order. The model asked for the CPF, got the CPF, asked for the CNPJ. Nobody came back the next day, nobody sent three fields in one sentence, nobody attached an unreadable document.

Why the prompt looks like enough

Whoever writes the script in prose is trying to skip a graph. “Do not skip steps” and “do not ask again” sound like rules. For the model they are two sentences in the same token block, competing with the history, the transcribed voice note, and the customer’s “go ahead and approve”. Anthropic’s “Building effective agents” (December 2024) describes exactly this shortcut: putting flow control in the prompt instead of in code.

What broke

Week 1. Customers reported the bot asked for their CPF three times. In other cases, it skipped the proof of address and went straight to the terms. Same prompt, both failures. When the customer sent name and CPF together, the model sometimes saved the name, forgot the CPF, and asked for the CPF again. When the customer said “I will send the proof later”, the model sometimes treated the promise as the document.

Week 3. Customers coming back after two or three days found a confused bot. The history already had dozens of messages, a voice note, an unreadable screenshot, and two CNPJ attempts. The model asked again for what it had already received, or resumed at the wrong step. Liu and colleagues, in “Lost in the Middle” (2023), showed that models use poorly what sits in the middle of a long context. The CPF sent on day one sat exactly there: in the middle.

Week 4. The team found full CPFs and ID photos in the observability logs and in the history sent to the model with every message, including in support conversations after sign-up. Personal data had become permanent context. In 2023, Samsung restricted ChatGPT after employees pasted confidential source code into the model. The equivalent here is the partner’s CPF traveling to the provider on every “hi”.

Week 6. A customer sent “I already sent everything, go ahead and approve”. The model replied that sign-up was complete, without the proof of address. The save_field tool had never received the file. What decided the end was the customer’s prose, not the state.

The four weeks point to the same design: the script and the memory of the sign-up live in the text the model rereads. Text gets lost, contradicts itself, and lets itself be talked into things.

Check +20 XP on first tryIn week 1 the bot asks for the CPF again, even after the customer sent name and CPF in the same sentence. Which cause is the most useful to attack first?

The model is weak at extraction; switching to a larger one fixes the repetition

The script lives in the prompt, so the model decides what is missing instead of code looking at per-field state

The history should be longer, so the model can find the CPF that already arrived

Smells found

Prompt as State Machine

The step script lived in the prompt. “Do not skip” and “do not ask again” are graph rules written in English. The model applies them when it remembers, and forgets them when the customer improvises. It caused the repeated questions and skipped steps in weeks 1 and 6.

Real state has a place per field:

partner_name: validated
cpf:          validated
cnpj:         received
proof:        missing
id_photo:     missing
revenue:      missing
terms:        missing

Version 1’s prompt does not have that table. It has a paragraph that asks the model to pretend the table exists.

Context Rot

The full history of days of conversation went out with every message. It caused the confusion on resume in week 3 and spread personal data in week 4. Each “still there?” resent the CPF, the ID photo, and the proof of address, to the model and to the trace collector.

Refactorings applied

1. Replace Prompt Rule with Graph Edge

The steps became a graph in code, with each field’s state: missing, received, validated. The model’s role got smaller: understand the message and extract the data, in any order. Code decides the next question by looking at what is still missing.

Received is not validated. A CNPJ with a bad check digit stays received until code refuses it or a person looks. The model does not “validate” a CNPJ: it returns the digits it read.

2. Replace History with Summary

The model receives the sign-up state, a short summary (“customer Ana, resuming after three days, proof of address and photo still missing”) and the latest messages. Personal data lives in the state, masked. The conversation text sent to the model does not carry the full CPF or the ID image.

3. Move to Durable Execution

Each sign-up is a workflow that persists its state at every step and can wait days for a document without losing anything. The wait is not a bot “remembering” the chat. It is a timer and a state row. When the customer comes back, or when the 24-hour template fires, the workflow wakes on the same object.

Check +20 XP on first tryA customer disappears on Tuesday and comes back Friday with the proof of address. Which design resumes sign-up without asking for the CPF again and without resending the ID photo to the model?

Load the workflow's persisted state, mask what is already validated, and only process the new message

Resend Tuesday's full history, because that is the only way the model ‘remembers’ the customer

Ask for every field again, to guarantee consistency

Final architecture

flowchart TD
  W(["1. Message or document"]):::io --> G["2. Guardrail: mask and vault personal data"]:::tool
  G --> X(["3. Extract fields from the message"]):::llm
  X --> V["4. Validate: CPF, CNPJ, document"]:::tool
  V --> S[("5. Workflow state")]:::store
  S --> N{"6. Next step in code"}
  N -->|"field missing"| A(["7. Ask naturally"]):::llm
  N -->|"case off the script"| H((("8. Human agent"))):::human
  N -->|"all validated"| T(["9. Terms and completion"]):::io
  A --> OUT(["Reply on WhatsApp"]):::io

1. The message or file arrives through WhatsApp.
2. An input guardrail detects CPF, CNPJ, and documents, stores the value in a vault, and leaves only a masked reference in the text.
3. A small model extracts, as structured output, whichever fields the message carries, in any order.
4. Code validates each field: check digits, CNPJ lookup, document legibility.
5. The durable workflow persists the sign-up state on every change.
6. Code picks the next step from the state, not from the history.
7. The model writes the next question naturally, receiving only the state and the latest messages.
8. Off-script situations (a foreign partner, a company under restructuring) go to a person, with the state.
9. With everything validated, the terms are sent and acceptance is recorded.

The model never declares sign-up complete. Complete is a predicate in code: every required field is validated, the terms have an acceptance with a timestamp, and no field sits in received without validation.

What the model sees, before and after

Suppose this is the fourth message on Friday, after three days and a new proof of address. Version 1 resent everything. The final design sends only what the step needs:

	Version 1	Final architecture
Text to the model	History since Tuesday’s “hi”, with CPF, CNPJ, a base64 photo, and the PDF	Masked state + a one-line summary + the new message
Who picks the next question	The model, rereading the script	The graph, looking at proof: received
Where the CPF lives	In every call, in the clear	In the vault; the model sees ***8909

The point is not “fewer tokens as a courtesy”. It is the CPF leaving the provider’s path, and the next step no longer depending on the model finding a sentence in the middle of the context.

Received and validated are not synonyms either. The proof of address just arrived: state becomes received, a validator tries to read an address in the PDF, and only then does it become validated. If the image is dark, the field does not move on its own. It goes to a person, with the file in the vault. The model for the next question does not “guess” that the address was there.

Worked example: three fields in one sentence

Customer Ana writes: “I am Ana Souza, CPF 123.456.789-09, CNPJ 12.345.678/0001-95, revenue around 40 thousand”.

The guardrail swaps the identifiers for references before the text reaches the extractor:

I am Ana Souza, CPF {{cpf:p_92}}, CNPJ {{cnpj:p_93}}, revenue around 40 thousand

The extractor returns an object:

{
  "nome_socio": "Ana Souza",
  "cpf_ref": "vault:p_92",
  "cnpj_ref": "vault:p_93",
  "faturamento_mensal_centavos": 4000000,
  "faturamento_incerto": true
}

Code validates the CPF and CNPJ digits in the vault, looks the CNPJ up at the tax authority, and marks revenue as received with an uncertainty flag, because “around 40 thousand” is not a closed figure. The next question the graph asks is not the CPF. It is the proof of address, the ID photo, and a confirmation of revenue as a number. One message, four fields touched, zero repetition.

Resume after three days

On Thursday the workflow fires an approved template: “Hi Ana, we still need the proof of address and the ID photo to finish your account.” This is not the model inventing a long sentence after the 24-hour window. It is state saying what is missing, and the template filling two slots.

Ana sends the PDF. The guardrail stores the file in the vault, the document validator checks whether an address is readable, the field becomes validated or goes to a person if the image is dark. The model for the next question only sees: proof validated, photo missing, terms still open.

Personal data kept off the model

The vault stores the value. Logs show cpf=***8909. The model sees mask=***.***.***-09. The human agent, on handoff, gets the state and a temporary link to the document, not a dump of the history. That meets LGPD minimization and cuts the week-4 leak class. Sending a full CPF to a model provider on every message is a personal-data transfer; the masked reference is not.

Off the script

A partner with a passport, a company under court restructuring, two partners who both need to sign: the graph has an explicit edge to a person. The handoff carries the state, not a prose summary of the kind “I think a document is missing”. The person sees the same object code sees. The model does not improvise an exception to the KYC rule.

Common traps

• Letting the model decide that ‘it is already complete’. Week 6 fits in one customer sentence. Complete is a state predicate.
• A summary that copies the CPF. Replacing history with a summary does not help if the summary is the history shrunk, with the same identifiers.
• Validating CNPJ in the prompt. Check digits and tax status are code and a lookup. The model reads the number; code says whether it exists.
• Waiting for the right question. The customer answers with what they have on hand. Extract every field from every message.
• Resuming by rereading the chat. Three days later, what you load is the object, not Tuesday’s voice notes.
• Free-form prose after 24 hours. The API refuses. The template is part of the workflow, not a product footnote.

Check +20 XP on first tryThe customer, in the middle of the CPF step, sends the CNPJ, the revenue, and ‘I will send the proof tomorrow’. What should the extractor and the graph do?

Ignore everything that is not a CPF, because the current step is CPF

Mark sign-up as almost complete, since the customer promised the proof

Extract CNPJ and revenue now, validate what you can, and the next natural question covers only what is still missing

Numbers

This scenario’s metrics are completed sign-ups, drop-off per step, and exposed personal data, comparing version 1 with the final architecture on a set of simulated conversations, including resumes after days and out-of-order answers.

The three are read together. Raising completion by pushing the customer to “accept the terms now”, without proof of address, inflates the first metric and hides an invalid sign-up. Drop-off per step shows where the graph asks too much or asks again. Exposed personal data counts identifiers and images that reached a log, a trace, or the model provider. If that third one is not zero on the test set, the rest will not hold up in an LGPD audit.

The set needs to include, at minimum, what broke version 1: two fields in the same sentence, a return after days, “I already sent everything”, an unreadable document, a partner off the script, and a support chat after sign-up to see whether the CPF still travels in the history. It also needs the happy path, or any masking looks like a win.

The numbers are not published yet. They will come from the runnable reference code, running both versions against the same set, and this section will be updated with the result and the script that produces it. Until then, no numbers appear here.

Patterns used

Pattern	Role in the system
Durable Workflow	Sign-up survives days of waiting, failures, and the 24-hour window
Short-Term Memory	Explicit sign-up state instead of raw history
Long-Term Memory	Customer preferences for service after sign-up, without reopening the vault
Input Guardrails	Detecting CPF, CNPJ, and documents before the model
PII Redaction	Value in the vault, mask in the text, reference in the state
Multi-Agent Handoff	Off-script cases handed to a person with the same state

The full list of patterns tied to this scenario is under Relations, below.

Guardrail and redaction work in sequence: the first finds the data, the second decides what the model may see. Durable workflow and short-term memory are the same object from two sides: persistence across days, and what the model receives now. Without the handoff to a person, the graph tries to cover a KYC exception with more prose, and the prompt-as-state-machine smell comes in through the back door.

Reference code

The complete system, with both versions, the simulated conversations, and the script that produces this page’s numbers, lives in the ai-guru reference code repository, under scenarios/conversational-onboarding/. The repository is being prepared, and the link goes here once it is published.

Lessons

1. The script is code; the conversation belongs to the model. State decides the next step. The model only makes the question sound natural, and never declares the end.
2. Explicit state solves resuming. Coming back after three days became loading an object and, if the window closed, firing a template. It is not rereading a conversation.
3. Personal data does not need to pass through the model. A masked reference drives the conversation. The vault stores the value. Privacy law stops being a poster on the wall.
4. Out-of-order answers are the rule. Extracting every field from every message, instead of waiting for the right question, cuts the back-and-forth and the drop-off.
5. Knowing when to call a person is part of the flow. Off-script cases have an edge in the graph. Model improvisation is not a sign-up policy.

Check your understanding

Check +20 XP on first tryThe customer writes ‘I already sent everything, go ahead and approve’. Proof of address is not in the state. Version 1's model would reply that sign-up is complete. What does the final architecture do?

Trust the customer and close, because insisting on the proof raises drop-off

Code looks at the state, sees proof missing, and the model only writes the request for the missing file

Ask for every field again, so the customer proves they really sent everything

Check +20 XP on first tryThe observability team finds ID photos and full CPFs in traces of model calls, including a support chat two weeks after sign-up. Which change cuts this at the root?

Mask on the way in, store the value in the vault, and never resend raw history to the model or the collector

Ask the model provider not to keep logs on their side

Delete traces after seven days, to meet retention

Check +20 XP on first tryThe CNPJ belongs to a company under court restructuring, a case the graph does not cover. What is the next step?

Ask the model to improvise extra KYC questions and continue on WhatsApp

Refuse the sign-up on the spot, because restructuring is outside the default policy

Hand off to an agent with the current state (what is already validated, what is missing, the masked CNPJ) and record the handoff