Move to Durable Execution

Problem

A process that lasts hours or days (waiting for documents, approvals, other systems’ responses) lives in a process’s memory or in a conversation history. A crash, a deploy, or a long wait loses the state. This is the Prompt as State Machine smell, in its long-running form.

Solution

Run the process on a durable execution engine: each step is recorded, state is persisted between steps, waits become events that resume execution, and a failure resumes from the last completed step without repeating what was already done.

Before and after

-def onboarding(customer_id):
-    data = ask_for_documents(customer_id)            # may take days; process in memory
-    if not validate(data):
-        ask_again(customer_id)
-    approval = wait_for_compliance(customer_id)      # if the server restarts, it is gone
-    create_account(customer_id)
+@workflow
+def onboarding(customer_id):
+    docs = workflow.wait_for_signal("documents_received", timeout=days(7))
+    result = workflow.step(validate_documents, docs)          # result recorded
+    while not result.ok:
+        workflow.step(send_whatsapp, customer_id, result.missing)
+        docs = workflow.wait_for_signal("documents_received", timeout=days(7))
+        result = workflow.step(validate_documents, docs)
+    approval = workflow.wait_for_signal("compliance_decision", timeout=days(3))
+    if approval.approved:
+        workflow.step(create_account, customer_id, idempotency_key=workflow.id)

Before: state lives in memory.

flowchart LR
  S(["1. Start"]):::io --> W["2. Waits days for a document"]:::tool
  W --> X{"3. Server restarted?"}
  X -->|yes| L(["4. State lost"]):::io

After: every step recorded, waits as events.

flowchart LR
  S(["1. Start"]):::io --> P1["2. Recorded step"]:::tool
  P1 --> J[("3. Workflow history")]:::store
  J --> W{"4. Wait for a signal"}
  W -->|"document arrived"| P2["5. Next step"]:::tool
  P2 --> J
  J -->|"crash or deploy"| R["6. Resume from the last step"]:::tool

1. The process starts as a workflow with an identifier.
2. Each side-effect step runs and its result is recorded.
3. The engine keeps the step history and the state.
4. Waits (document, approval, deadline) are events; no process sits idle waiting.
5. When the event arrives, execution continues where it was.
6. If something crashes, the workflow resumes with the recorded results, without repeating completed steps.

Worked example

A fintech opens a business account over WhatsApp. The customer sends the articles of association, a proof of address, and a selfie. Compliance takes up to three business days. In the old design, a Node process sits in await. A Friday deploy wipes whoever was in the middle. On Monday the customer hears “please send the documents again”.

In the durable design, the workflow id is the customer_id. The validate_documents step records a result with ok set to false and the list of missing files, for example the proof of address. The send_whatsapp step sends that list, keyed as wf.id + ":ask:" + n. The wait becomes a documents_received signal with a 7-day deadline.

On Tuesday the proof of address arrives. The engine wakes the workflow, reads the already recorded validation result, and runs validation again. Nobody asks for the tax ID a second time. If the server dies while creating the account, the idempotency key workflow.id prevents a duplicate account.

The model stays in the right place: read the PDF, say what is missing, write the message in the fintech’s tone. It does not store “we are on step 4”. That lives in the workflow history.

Why it works

A long process has two enemies: time and failures. In-memory processes survive neither. A durable execution engine records each step’s result and rebuilds the state from that record, so a crash or a deploy becomes a pause, not a loss.

The model keeps its role: understanding messages, extracting data, writing replies. The flow (what comes next, what waits for what) lives in the workflow’s code, testable and visible. And the workflow’s identifier is a natural idempotency key for side-effect actions. The target patterns are Durable Workflow and Checkpoint and Resume.

Replay: why the model cannot live in the workflow body

Engines such as Temporal rerun the workflow code from the top to rebuild state. They do not “continue from line 47”. They replay recorded steps and only then run what has not run yet.

If you call the model in the workflow body, every replay fires a new call. The answer changes, the if branch changes, and the recorded history no longer matches the code. The engine reports nondeterminism.

The rule is simple. Anything nondeterministic (model, clock, randomness, HTTP) goes in a workflow.step. The workflow body only reads recorded results and decides the next step.

Safe mechanics

0. Measure. Count how many processes get lost today (crashes, deploys, timeouts) and how long each stage waits.
1. Draw the flow as explicit steps and waits.
2. Choose the mechanism: a durable workflow engine, a queue with state in a database, or your platform’s solution.
3. Turn each effect into a recorded step, with an idempotency key.
4. Turn each wait into an event (signal, timer), with a deadline and what to do when it expires.
5. Keep model calls inside steps, so their result is recorded and not redone on resume.
6. Test resuming: kill the process midway and confirm it continues without repeating effects.
7. Measure again: completed processes, successful resumes, duplicated effects (zero).

When a database checkpoint is enough

A workflow engine is one more moving part: flow versions, replay, visibility. For a 20-minute job with three steps and no human wait, a database row with status, step, and idempotency_key can be enough.

The line that asks for the engine is the long wait (hours or days), the volume of parked processes, and the need to see each one. If you have a thousand open onboardings waiting for a document, sleep in a thousand processes does not scale. One signal per process does.

Risks

• Nondeterministic code in the workflow. Durable execution engines often replay the workflow’s code to rebuild state. Model calls, timestamps, and randomness must live inside recorded steps.
• Flow changes with processes in flight. A new workflow version may not match the history of older runs. Version the workflow.
• Complexity. A workflow engine is one more moving part. For minute-long processes, a simple checkpoint in a database may be enough.

Common mistake: the prompt as a state machine

The team describes the steps in the system prompt (“do not skip step 3”) and treats the conversation history as the checkpoint. That is the Prompt as State Machine smell. The customer sends a tax ID and a proof of address in the same message, the model loses count, and the flow skips.

Durable execution does not replace the model. It takes away the job of remembering which step the process is on. Code knows. The model only fills in the current step.

Challenge

A collections process sends a reminder, waits 3 days, sends another, waits 3 more days, and, if there is no payment, opens a ticket for the finance team. Today it is a script with sleep. What can go wrong, and what does the durable version look like?

workflow collections(invoice_id):
    step send_reminder(invoice_id, n=1, key=wf.id+":1")
    paid = wait_for_signal("paid", timeout=days(3))
    if paid: return
    step send_reminder(invoice_id, n=2, key=wf.id+":2")
    paid = wait_for_signal("paid", timeout=days(3))
    if not paid: step open_finance_ticket(invoice_id, key=wf.id+":ticket")

Check your understanding