Routing System Prompt

Signs and symptoms

A telecom carrier’s virtual assistant handles sales, tech support, and billing in the same prompt. An excerpt (anonymized):

You are the Carrier's virtual assistant.

IF the customer asks about plans or offers:
  be enthusiastic, highlight benefits, offer an upgrade
  [800 tokens of sales rules]

IF the customer has a technical problem:
  ask for the device model, suggest a restart
  [1,200 tokens of tech script]

IF the customer asks about their bill:
  be objective, do not offer products, explain each charge
  [900 tokens of billing rules]

A customer asks why the bill came in higher. The answer explains the charge and ends with “take the chance to check out our plan with 50 GB more!”. The sales tone leaked into billing.

Signs of a Routing System Prompt:

• The prompt has “IF the topic is X, do Y” blocks for several topics.
• One topic’s instructions show up in answers about another (tone, offers, questions).
• Every request carries the instructions for every topic.
• Different teams edit different blocks of the same prompt.
• A change to the sales rules alters support answers.

flowchart LR
  IN(["1. Billing question"]):::io --> P(["2. Prompt with sales, support, and billing"]):::llm
  P --> OUT(["3. Billing answer in a sales tone"]):::io

1. A request about a single topic arrives.
2. The model receives the instructions for all three topics and has to decide which apply.
3. Parts of the wrong instructions contaminate the answer.

Check +20 XP on first tryA customer asks only about the bill. The answer explains the charge and offers an upgrade. The prompt has three IF blocks, one per topic. What is happening first?

The sales model is the only one available, so the tone leaks for lack of choice

The model received all three topics' rules and mixed the sales block into a billing answer

You need a NEVER offer products in billing rule at the top of the prompt

Reasons

• The product grew by topic. The bot was born for sales; support and billing came later, each as a new block.
• One prompt looks simpler than a router and three prompts.
• The model is right almost always. The leak is rare enough not to feel urgent.
• Each team only looks at its own block. Nobody sees the whole prompt as a system.

Anthropic’s “Building effective agents” (Dec 2024) describes routing as a workflow: a step classifies, code picks the path. Leaving that decision in prompt text asks the model to emulate the if.

Cost of ignoring

• Quality. Answers with the wrong topic’s tone or rules, intermittently and hard to reproduce.
• Cost. Every call pays for every topic’s instructions.
• Coupling. One team’s change breaks another team’s behavior.
• Evaluation. You cannot measure each topic separately, because everything goes through the same prompt.

A back-of-the-envelope leak

The numbers below are illustrative. The three blocks add up to 2,900 tokens of rules. A billing question needs only the 900 billing tokens.

Call	Instruction tokens sent	What should go
Bill	2,900	900 (billing)
Bad signal	2,900	1,200 (tech)
Offer	2,900	800 (sales)

At 10,000 chats per day, you pay almost 29 million rule tokens. With a router, the average would drop to about 1,000 per call if the topic mix is even. The extra tokens are the smaller problem. The sales block sits next to the bill, waiting to leak.

The same design shows up in a Brazilian bank’s WhatsApp: Pix (instant payments), boleto (bank slips), and cards in one prompt. The customer asks for the status of a Pix. The answer ends by suggesting a loan. Collections and product should not share the same rule block.

Treatment

Refactoring	Choose it when
Introduce Router	Whenever the prompt has blocks per topic. A short step classifies the request and code picks the specialized prompt. Targets: Routing and Tool Router.

How the flow looks:

1. A small classifier returns the topic (and a confidence).
2. Code picks the prompt. Low confidence or topic “other” goes to a human.
3. Each owning team edits only its prompt, with its own eval.

The classifier can be wrong. That is visible and measurable. Leakage in a single prompt is invisible: the answer looks on-topic, with a gift from the wrong topic.

Another trap: classifying and answering in the same call, with the same long prompt. The model labels the topic “billing” and still uses the sales tone, because all three blocks remain in the input. Classification only counts when code, on a later call, sends only that route’s prompt.

Before and after, on the billing question:

	Before	After
Text sent	sales + support + billing	billing only
Offer at the end	common	out of the prompt, so out of the answer
Owner of a change	any team in the same file	the billing team in its prompt

Check +20 XP on first tryThe team added a router with three prompts. Each specialized prompt still ends with: if the customer changes topic, follow the new topic's rules. The leak came back. What remains?

The same smell at a smaller scale: topic changes are still decided in the prompt

Nothing, because the router already exists and the rest is wording

Prompt Bloat, because the specialists got long

Payoff

• Each topic with a short prompt, tested and maintained by its owning team.
• No tone or rule leaks between topics.
• Fewer tokens per call.
• Per-route metrics that show where the problem is.

When to ignore

• Two similar topics with the same rules. Splitting brings no gain.
• Low volume and low risk. An internal assistant with few users can live with one prompt for a while.
• Prototype. A single prompt helps discover which topics exist; split once they settle.

Is it this smell?

Check +20 XP on first tryAn HR bot's prompt has blocks for vacation, benefits, and payroll, but the tone and rules are the same for all three, and the model never mixes the answers. Is this a Routing System Prompt?

Yes, there are blocks per topic

No, or not yet

Yes, because three topics always need three prompts

Check +20 XP on first tryA bot has a router that chooses between three specialized prompts, but each specialized prompt still has an 'if the customer changes topic, do...' block. Is this a Routing System Prompt?

Yes, on a smaller scale

No, there is already a router

No, this is Rule Stacking

Check +20 XP on first tryThe prompt is long because it holds the price table for every plan, but it only handles sales. Which smell is it?

Routing System Prompt

Tool Sprawl

Prompt Bloat

Check +20 XP on first tryThe billing team changed one sentence in the bill block. The next day the sales eval dropped: the bot started refusing upgrades in a billing tone. What does the incident show?

That the sales eval is unstable and needs more examples

That sales needs an ALWAYS be enthusiastic rule

That sales and billing share the same prompt, so an edit leaks