Turning Evals Into Guardrails That Run at Inference Time

There's a category error hiding inside a lot of eval infrastructure that otherwise looks mature: a comprehensive offline test suite, a well-tuned LLM judge, a golden dataset that's grown for a year through the eval-driven discipline described in the previous piece in this series — and none of it touches a single request in production. It all runs in CI, before deploy, against a fixed dataset. The moment a new failure mode shows up in the wild that the dataset doesn't cover, the eval suite has nothing to say about it until someone notices, writes a new test case, and the fix ships in the next deploy cycle. In the meantime, every user who hit that failure mode got the bad response, in real time, with nothing standing between the agent's output and their screen.

Runtime guardrails close that gap by moving a subset of evaluation logic from "runs offline, before deploy" to "runs inline, before the response is returned." The 2026 consensus that evals should be "embedded directly into the agent runtime as guardrails," not just an offline test suite, is a direct response to exactly this gap — and it's worth being precise about what changes when an eval becomes a guardrail, because the two are related but not identical.

The difference between measuring and blocking

An offline eval, and even the async, sampled production evaluation described in "Cheap Heuristics on 100% of Traffic, Expensive Judges on 10%," both share a property: they observe and report, but they don't intervene. A trajectory gets graded after the fact, a score gets logged, a dashboard gets updated, and if the score is bad, someone finds out — eventually, through a dashboard or an alert — and fixes it in the next iteration. The bad output already reached the user by the time any of that happened.

A guardrail is the same underlying logic, applied before the response leaves the system rather than after. It runs inline, in the critical path of the request, and it has the authority to block, rewrite, or redirect a response that fails its check — not just log that the failure happened. This is a meaningfully different piece of infrastructure, not just an eval running earlier: it needs to be fast enough not to blow the agent's latency budget, reliable enough not to false-positive on legitimate responses at a rate that damages the product, and narrow enough in scope that it can make a real-time pass/fail call rather than the nuanced multi-dimensional scoring a full judge produces.

What actually belongs in a guardrail

Not every eval check is a candidate for inline guardrail duty, and trying to force all of them into that role is the fastest way to build a guardrail layer that's either too slow to survive in production or too blunt to be trustworthy. The checks that translate well into guardrails share a specific shape: they're fast to compute, they have a clear binary or near-binary pass/fail criterion, and a false positive is cheap enough to tolerate.

Schema and format validation on structured outputs is close to the canonical example — cheap to check, unambiguous, and a failure here means the response would have broken the calling system anyway, so blocking it and triggering a retry is strictly better than letting it through. Policy and safety checks belong here too: a response that would leak a system prompt, commit to a claim the agent has no basis for, or cross an explicit content boundary is exactly the kind of thing worth blocking inline rather than just logging for later review. Tool-use guardrails — refusing to execute a write action the agent's own trajectory reasoning doesn't actually support, or requiring a second confirmation pass before an irreversible action — are the sharpest version of this pattern, because they intervene at the point of highest consequence rather than only at the point of final output.

What doesn't belong in a guardrail is the heavier, more nuanced evaluation covered elsewhere in this series — the full chain-of-thought, multi-dimensional LLM-as-judge scoring from "The Biases Baked Into Every AI Judge," or a full agent-as-judge trajectory review from "Agent-as-Judge: Letting an Agent Grade Another Agent's Work." Both are too slow and too expensive to run in the critical path of every request, and both produce a nuanced score rather than a fast binary decision — exactly the properties that make them unsuitable for blocking a response in real time, and exactly why they belong in the sampled, asynchronous tier instead.

The layering, made concrete

Put together, a mature setup ends up with three distinct layers operating at three different points in a request's lifecycle, each doing a job the others can't. Inline guardrails run on every request, in the critical path, with narrow fast checks that can block or rewrite a response before it reaches the user. The cheap heuristic tier from the sampling architecture runs on every request too, but just after the response goes out, logging signal without blocking anything, catching patterns too broad or too slow for a hard block. The expensive judge tier runs on a sample, asynchronously, off the critical path, doing the deep multi-dimensional and trajectory-level analysis that neither of the faster layers can afford to do at full volume.

None of these layers is a substitute for the others — a guardrail that blocks bad schema output doesn't tell you anything about whether a syntactically valid response gave bad advice, and a sampled deep judge running once every ten requests can't stop the one bad response that happened to fall in the other nine. The layers are complementary precisely because they trade off speed, cost, and depth differently, and a system that only has one of the three has a real gap that will eventually cost something in production.

The trap: guardrails that quietly become the whole eval strategy

The risk with runtime guardrails, once a team has them working, is the same risk that shows up with LLM-as-judge in the very first piece of this series: a genuinely useful tool becomes the entire strategy by default, because it's the part that's visibly working and stopping bad things in real time. A guardrail catching a schema violation feels like a solved problem in a way an async sampled judge score sitting in a dashboard doesn't, and that visibility bias can quietly starve the deeper evaluation layers of attention over time.

Guardrails are necessarily narrow — fast binary checks by design — which means they will never catch the subtler failures that trajectory evaluation and multi-dimensional judging exist to catch: a technically valid response that took a wasteful or risky path to get there, a tone that's technically fine but drifting from what users actually want, a factual claim that's plausible but wrong in a way no schema check would ever flag. A guardrail layer stops the failures that are cheap enough to catch instantly. It was never going to be the layer that catches everything else, and treating it as if it were is how a well-built inline defense ends up masking exactly the kind of drift the rest of this series has been about catching.

Part of the "Evals for Agents" series on aiskill.market.