Inline Trivial Step

Problem

A pipeline step does not change the result measurably, but it costs a whole call. The most common case is the Redundant Self-Critique smell: the model writes a text and, in the next call, reviews its own text without receiving any new information.

Imagine a news site that generates 10,000 summaries a day. The pipeline has two calls: summarize and review. When the team compares the outputs, they see the review returns the draft unchanged, or with one comma moved, in the vast majority of cases. The second call has become an expensive rubber stamp.

Solution

Remove the call that does not change the result and move whatever was useful in it (a criterion, a format, a check) into the neighboring step or into code.

Before and after

 def write_summary(document):
-    draft = llm("Summarize the document in at most 150 words.", document)
-    reviewed = llm("Review the summary below: fix mistakes, make it clearer, "
-                   "and keep it to at most 150 words.", draft)
-    return reviewed
+    summary = llm("Summarize the document in at most 150 words. Keep names, "
+                  "numbers, and dates exactly as in the original.", document)
+    if word_count(summary) > 150:
+        summary = truncate_to_sentence(summary, max_words=150)   # rule in code
+    return summary

Before: two calls, and the second almost always returns the first.

flowchart LR
  D(["1. Document"]):::io --> G(["2. Summarize"]):::llm
  G --> R(["3. Review its own summary"]):::llm
  R --> OUT(["4. Summary"]):::io

After: one call with the useful criterion built in, and the objective rule in code.

flowchart LR
  D(["1. Document"]):::io --> G(["2. Summarize with the criterion"]):::llm
  G --> C["3. Check length in code"]:::tool
  C --> OUT(["4. Summary"]):::io

1. The same document comes in.
2. The generation prompt receives the criterion the review was trying to apply.
3. The objective part (length) becomes code, which counts words without mistakes.
4. The result goes out with one fewer call.

Why it works

A review step with no new information reads the same thing with the same model and tends to agree with it. Its value is usually an instruction that could live in the previous step (“keep the numbers exact”) or an objective rule that code handles better (“at most 150 words”).

Look at one detail of the old code: the review receives only the draft, not the document. It has no way to know whether a number is right. The most it can do is rewrite, and every rewrite is one more chance to turn “$1.2 million” into “$12 million”.

Research points the same way. The paper “Large Language Models Cannot Self-Correct Reasoning Yet” (Huang et al., 2023) showed that, without an external signal, asking a model to review its own answer does not improve reasoning and sometimes makes it worse.

Building the criterion into the generating step prevents the error instead of trying to fix it later. The generator is the one that sees the original document, so it is the one that needs to hear “keep the numbers”. Moving the objective rule to code swaps a probabilistic call for a deterministic check.

The math of a trivial step

An illustrative calculation shows where the cost lives. Suppose a 3,000-token document and a 200-token summary.

	Input	Output	Wait
Summarize	3,000 tokens	200 tokens	one generation
Review	about 250 tokens	200 tokens	another full generation

The review’s input is cheap. What weighs is the output: the model writes the summary again, token by token. Output tokens usually cost several times more than input tokens, and they are what decides how long the user waits. At 10,000 summaries a day, that is 2 million extra output tokens to return nearly the same text.

Check +20 XP on first tryYour pipeline writes a summary and then calls the same model with 'review the summary below', passing only the draft. What can this review guarantee about the numbers in the text?

That they are correct, because the model rereads the text more carefully

Nothing: it never sees the original document, so it cannot check

That they are correct, as long as the review runs at temperature zero

Safe mechanics

0. Measure. Run the eval on the current pipeline and record quality, cost, and latency. Also keep the output before and after the suspect step, for every case.
1. Compare the step’s input and output. Count in how many cases the output differs meaningfully. A simple metric is enough to start: the share of characters changed. If 95% of cases change by less than 2%, the step is barely working.
2. Read the cases that changed. Did the change improve things, make them worse, or just swap words? Sometimes the little that changes is exactly where the step does damage.
3. Run the eval without the step. If quality does not drop, it is a candidate for removal.
4. List what the step was trying to guarantee. Separate criteria (they go into the previous prompt) from objective rules (they go into code).
5. Move the criterion and the rule, and remove the call.
6. Measure again. Equal or better quality, one fewer call, lower latency. Check per category too, not only the average.

Risks

• The step helped on rare cases. The average hides it. Check the eval per category and the hard cases.
• The previous prompt bloats. If the moved criterion is long, consider whether the step deserved to become a check with an external signal, as in Reflection.
• Truncating in code can cut badly. Cut at a sentence boundary and record when a cut happens. If cuts are frequent, the generation prompt needs tuning.

Common mistake: throwing away a review that had a signal

Not every review is trivial. What separates them is the information the step receives.

Signal	Trivial step	Step worth keeping
What it sees	Only the previous output	The output plus something new: the document, a test, a tool result
Who judges	The same model, with the same vague request	A clear rubric, another model, or code
What it produces	Almost always the same text	Concrete problems that change the result

A step that compares the summary with the document and lists mismatched numbers has a signal. So does a step that runs the tests of generated code. Those become an Evaluator-Optimizer, not a step to inline.

Check +20 XP on first tryYou are auditing a pipeline with several review calls. Which one should you NOT inline into the previous step?

A call that asks to 'improve the text' and receives only the draft

A call that checks whether the text stayed under 150 words

A call that compares the summary with the original document and flags mismatched numbers

Challenge

This ticket classification pipeline has a step that may be trivial. Which one, and what do you do with it?

1. classify(ticket) -> category
2. confirm("Is category X correct for this ticket? Answer yes or no.")
3. if "no": classify again

Show solution

Step 2 asks the model to confirm its own answer, with no new information. It almost always says “yes”. Remove steps 2 and 3. Instead, ask for a confidence along with the category (structured output with confidence, or token probabilities when the API offers them) and calibrate the threshold on the eval, because the confidence a model states does not always match its real hit rate. Low-confidence cases go to human review or to a larger model, as in a Model Cascade.

Check your understanding

Check +20 XP on first tryYou removed the review and the eval's average accuracy stayed the same: 92% before and 92% after. What is left to check before deploying?

The results per category and on the hard cases

Nothing, an equal average proves the step was useless

Whether the old pipeline is still cheaper at peak hours

Check +20 XP on first tryThe old review guaranteed 'at most 150 words'. Where does that rule go after the refactoring?

Only into the generation prompt, which now asks for the limit

Into a new, smaller call dedicated only to counting words

Into code that counts the words and cuts at a sentence boundary

Check +20 XP on first tryWhy does building 'keep names, numbers, and dates exactly as in the original' into the generating step work better than asking for that fix afterwards?

Because longer prompts always produce better answers

Because the generator sees the document and can avoid the error, while the reviewer only saw the draft

Because the model pays more attention to the first call in a pipeline

Check +20 XP on first tryAfter every classification, your ticket classifier asks the model 'is category X correct? yes or no'. Which change fixes this best?

Ask for a confidence in the structured output, calibrate the threshold on the eval, and send uncertain cases to a person or a larger model

Ask the confirmation question three times and keep the majority

Change the question to 'are you absolutely sure?' to pressure the model