Cost Control for Agents on a Shared Compute Budget

The failure mode is almost always the same shape. A team ships an agent that does something legitimately useful — answers questions, processes requests, runs a workflow — and it works, so it gets more traffic. Nobody sets an explicit budget for it because nobody thought it needed one; it was one feature among many, running on shared infrastructure alongside everything else the organization operates. Then one day the shared account gets throttled, paused, or hit with a bill nobody budgeted for, and the postmortem reveals that one agent, doing an expensive thing on every single invocation including ones that didn't need it, was responsible for the overwhelming majority of the consumption. Nobody decided this was acceptable. Nobody decided anything — the absence of a limit was the decision, by default.

This is a different kind of production risk than the ones that get most of the attention in agent safety discussions. It's not about the agent doing something harmful or wrong. It's about the agent doing something correct, over and over, at a cost nobody was tracking against a budget nobody had defined — until the budget was a shared one, and the overage became everyone's problem at once.

Why shared infrastructure makes this worse, not just bigger

On dedicated infrastructure, an expensive agent is your problem alone — annoying, costly, but contained. On shared infrastructure — a shared cloud account, a shared compute budget, a shared rate-limited API tier — one agent's runaway cost pattern becomes every other project's outage. The mechanism is usually indirect: it's not that the expensive agent directly breaks another team's service, it's that both are drawing from the same finite resource pool, and one consuming disproportionately trips a limit that pauses or degrades everything sharing that account, including work that had nothing to do with the actual cause.

This is precisely the dynamic that makes cost control a governance problem, not just a FinOps line item. The team running the expensive agent often doesn't feel the pain first — the pain shows up as a mysterious outage report from an unrelated team, who then has to trace it back through shared billing and usage data to find the actual source. By the time that tracing happens, the cost has usually been accumulating for a while, because nothing was watching it in real time from the perspective of "is this reasonable," only in the eventual, blunt form of "did we hit the account-wide limit."

The pattern that burns budget fastest: doing expensive work on every request

The single most common root cause, across very different kinds of agents, is the same: an expensive operation — a large-context model call, a search across a big index, a synchronous render, a complex multi-step reasoning chain — running on every single invocation, including the ones where a cheap path would have worked fine. Traffic composition matters enormously here and is almost always underestimated: a meaningful share of real-world traffic to any public-facing agent endpoint is automated — crawlers, bots, scrapers, health checks, retries — and if the expensive path runs indiscriminately regardless of who or what is asking, a huge fraction of your cost is being spent on traffic that will never convert into anything valuable at all.

The fix, in the cases where this shows up, is rarely "the model is too expensive" — it's "the same expensive thing is running every time when a cached or cheaper path would serve most requests correctly." That distinction matters because it changes where you look for savings: not necessarily a smaller model or fewer tokens, but a smarter decision about when the expensive path needs to run at all.

Building the equivalent of a blast-radius gate, but for spend

The same design principle behind blast-radius gates on agent actions — cap the damage a single action or a single run can do, enforced in infrastructure rather than trusted to the agent's own judgment — applies directly to cost. A per-request cost ceiling, enforced before the expensive call happens rather than measured after the bill arrives, catches the pathological cases (an unexpectedly huge context, a request that triggers an expensive retry loop) before they become a pattern. A per-agent or per-workflow budget, tracked in near-real-time rather than reconciled monthly, means the first sign of a cost anomaly is an alert, not a bill.

Concretely: cache aggressively wherever the underlying data doesn't change on every request — a listing page, a summary, a lookup — so that a single expensive computation serves many requests instead of running fresh every time. Filter out automated and low-value traffic before it reaches the expensive path at all, at the edge if possible, so bots and crawlers get served a cheap static response rather than triggering the full pipeline. And apply the ceiling at the level of the individual call, not just the aggregate account — because an aggregate monthly ceiling that's only checked after the fact is a smoke detector that goes off after the building has already burned; it tells you what happened, not how to stop it happening again tomorrow.

Observability for cost, not just for correctness

The parallel to why agents need observability that looks nothing like conventional monitoring holds here almost exactly. A dashboard that shows total spend for the month is the cost equivalent of a status-code check — it tells you the aggregate outcome without telling you which decisions produced it. What you actually need is per-request-type cost breakdown: which category of task is expensive, is that expense growing as a share of total traffic or shrinking, and is a recent change — a new feature, a prompt update, a new class of request being handled — correlated with a shift in the trend.

Without that breakdown, diagnosing a cost spike after the fact means digging through logs under time pressure to figure out what changed, exactly the position teams find themselves in during an incident-response process when the trace wasn't captured in advance. Cost anomalies deserve the same proactive instrumentation as behavioral anomalies — not because a cost spike is as dangerous as a compliance violation, but because on shared infrastructure, it can be just as disruptive to everyone sharing the account, and it's just as invisible until someone builds the dashboard that makes it visible.

Who owns the budget, explicitly

The organizational fix underneath all of this is less technical than it sounds: someone has to own an explicit budget for each agent, in writing, before it ships — not "we'll keep an eye on it," an actual number with an actual owner who gets alerted when consumption trends toward it. That sounds bureaucratic for something that feels like an engineering decision, but the absence of explicit ownership is exactly how these incidents happen — everyone assumed someone else was watching the trend line, and the shared account got paused before anyone with the authority to act had reason to look.

The same logic that applies to when to kill an agent run rather than let it keep going applies to cost: a kill switch that only exists conceptually, with no defined threshold and no defined owner, isn't a kill switch. It's a plan to figure out a kill switch after the damage is already visible to everyone sharing the bill.

Part of the "Running Agents in Production" series on aiskill.market.