Classifier Tiers in Agent Pipelines: Code, Model, or Fallback?

Updated 9 November 2026 · Standard formula, shown below

In an agent pipeline, use code for clear, stable classification rules; use a model when the decision depends on context or ambiguity; and use a fallback when neither can return a sufficiently reliable answer. A tiered design routes easy cases cheaply, reserves model judgment for harder ones, and handles uncertainty explicitly rather than pretending every input fits.

What classifier tiers mean

A classifier assigns an input to one or more categories: for example, routing a support request to billing, technical help, or account access. In an agent pipeline, classification may determine which tool runs, which instructions apply, or whether a request should be answered, clarified, or escalated.

“Code, model, and fallback” describes a practical routing pattern, not a universal standard with fixed tiers. A pipeline can begin with deterministic rules, send unresolved cases to a language model, then apply a safe fallback if the model’s result is missing, invalid, or too uncertain. Some systems reverse the first two stages or use several specialized classifiers; the right order depends on the task.

When code should classify

Use ordinary code when category boundaries can be expressed as explicit, stable conditions. Code is particularly suitable when the input has a known structure or the rule is an exact match, range check, or required-field check.

Structured values: Route a request with order_status set to “delayed” to an order-status workflow.
Exact commands or identifiers: Recognize a supported command, file type, or product code.
Hard constraints: Reject an unknown category, enforce an allowed value list, or check that required fields are present.
High-impact, unambiguous rules: Apply a specifically defined routing rule rather than asking a model to infer it.

Code is usually fast, repeatable, and straightforward to test. Its limitation is that it only captures the cases its authors anticipated. A keyword rule for “refund,” for instance, may miss “I was charged twice,” while a broad rule can incorrectly route an unrelated mention of a refund.

When a model is worth using

Use a model when classification depends on meaning across the whole message, paraphrase, or context that is difficult to represent as a manageable set of rules. “I can’t get in after changing phones” could concern account recovery, device setup, or both; a useful decision may require interpreting the surrounding details.

Models can generalize across varied wording, but they are probabilistic. They may misunderstand a request, return a category outside the permitted set, or sound certain despite ambiguity. Treat a model’s label as a proposal that must be checked, not as proof that the decision is correct.

Constrain the output to a defined schema, such as a category from an allowed list plus a short rationale or an uncertainty signal. Validate the response in code. If a model returns malformed data or an unknown label, do not quietly pass it to a tool as if it were valid.

Designing the fallback

A fallback is an intentional outcome for cases that cannot be classified safely or reliably. It should preserve control of the workflow, not merely hide a classifier failure. Depending on the task, the fallback can ask the user to clarify, route the case to a human, choose a low-risk general response, or stop before taking an action.

A typical sequence is:

Normalize and validate the input, then apply clear deterministic rules.
Send unresolved cases to a model with the available categories and decision criteria.
Validate the model’s output and check any confidence or uncertainty signal the system can meaningfully support.
Accept a valid result only when it meets the task’s reliability threshold; otherwise use the fallback.
Record the route and outcome so errors and frequent fallback cases can be reviewed.

Do not assume that a model’s self-reported confidence is a calibrated probability. A threshold is useful only when it has been evaluated against representative examples. For consequential actions, a safe fallback may be preferable even when the model appears confident.

Choosing the right tier

Choose based on the cost of mistakes, the variety of inputs, and the cost of delay or review—not just on which approach is easiest to implement. A useful starting point is to define categories, acceptable error types, and the consequences of each possible misroute.

Prefer code when rules are explicit, inputs are structured, and the categories change infrequently.
Prefer a model when natural-language variation or context makes rule coverage brittle.
Prefer a fallback when the input is incomplete, categories overlap, the output fails validation, or a wrong route could cause harm.

For example, a message-routing system might use code to detect a clearly formatted password-reset command, a model to distinguish nuanced billing questions, and a clarification prompt when a message says only “help with my account.” The pipeline should not force a precise label when the user has not supplied enough information.

Common problems and trade-offs

A common misconception is that adding a model automatically makes classification more accurate. It can improve coverage of varied language, but it also adds variability, operational complexity, and possible latency or usage costs. A small, well-maintained rule set can outperform a model on narrow, exact tasks.

Another problem is allowing the model to choose from vague or overlapping labels. Define categories with boundaries and examples, and include an “unclear” or “other” route when appropriate. If two categories routinely compete, revise the taxonomy or permit multiple labels rather than relying on increasingly elaborate prompts.

Evaluate each tier on examples that resemble real inputs, including misspellings, short messages, unusual phrasing, and cases near category boundaries. Track not only overall accuracy but also which errors matter most, how often cases fall back, and whether results vary across categories. A high average score can conceal poor performance on a rare but important class.

Keep classification separate from authorization. A label such as “account access” should not itself grant access, disclose private data, or authorize an external action. Downstream tools should enforce their own permissions and validate the information they need.

Frequently Asked Questions

Should code or a model run first?
Run code first when it can confidently handle clear, common cases and cheaply reject invalid inputs. If most inputs are ambiguous natural language and rule checks add little value, a model-first design may be simpler, with code validating its output afterward.

Can a model’s confidence score decide when to fall back?
It can be one signal, but do not treat a score as a reliable probability without evaluation. Compare scores with actual outcomes on representative data and choose thresholds according to the cost of false classifications.

What should happen when the classifier is unsure?
Use an explicit outcome: ask a targeted clarifying question, send the case for human review, or stop before a consequential action. The best choice depends on the task and the harm a wrong route could cause.

Is a fallback the same as an error handler?
No. An error handler responds to failures such as timeouts or invalid output. A fallback is a deliberate path for uncertainty or unsupported cases, although a robust pipeline needs both.

How many categories should a classifier have?
Use enough categories to support distinct downstream decisions, but avoid splitting labels that cannot be distinguished reliably or do not change what happens next. Review real examples to find confusing boundaries.

Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.