Why Eval-Driven Development Is Replacing Vibe-Checking Outputs

There's a specific moment every team building with LLMs goes through, usually somewhere in the first few months: someone tweaks a system prompt, pastes in a couple of test inputs, reads the outputs, decides they look better than before, and ships. It works, in the sense that it feels productive and the outputs genuinely do look better in that moment. It also has no memory. Three weeks later, someone else tweaks the same prompt for a different reason, and nobody notices that the earlier fix silently regressed, because the only record of the earlier problem was a feeling someone had while reading three examples in a terminal.

This is vibe-checking, and it's not a strawman — it's the default workflow for almost every team before they've been burned by it, because it's fast and it requires zero setup. The alternative that teams converge on after the first real regression is eval-driven development: writing the test case before you touch the prompt, code, or skill definition, the same way test-driven development asks you to write a failing test before you write the implementation that makes it pass.

The shape of the shift

Eval-driven development isn't a new invention layered on top of everything covered earlier in this series — it's what happens when you take LLM-as-judge, trajectory grading, and golden datasets and change when in the workflow they get used. Vibe-checking uses eval-like judgment reactively, after a change, informally, and only in the moment. Eval-driven development uses the same judgment proactively, as the first step, formally enough to be re-run automatically forever after.

Concretely: when a user reports that the agent handled a refund request badly, the vibe-checking response is to read the transcript, form an opinion about what went wrong, adjust the prompt, and manually try a few similar inputs until they look right. The eval-driven response is to first turn that exact failing case into a permanent test case in the golden dataset — the same kind of dataset covered in "Building a Golden Dataset When You Don't Have Real Traffic Yet" — confirm it currently fails, then make the change, and confirm the new case now passes without re-checking by eye whether it "feels" fixed. The fix isn't done when it looks right. It's done when the test that didn't exist yesterday passes today, and every test that existed before still passes too.

Why "it looks better" isn't actually evidence

The deeper problem with vibe-checking isn't laziness — it's that a human reading three or five example outputs and forming an impression is a genuinely weak instrument for detecting the kind of failures that matter in an agent system. A prompt change that fixes the exact case someone is looking at can simultaneously make ten other, less visible cases worse, and nobody manually re-checks those ten cases, because nobody thought to look. The entire reason a golden dataset and an automated judge exist is to replace "I looked at a few examples and it seems fine" with "I ran eighty cases and the pass rate went from 91% to 94%, and here are the two cases that newly regressed." One of these is a feeling. The other is a number with a diff attached to it.

This is exactly the trap described in the first piece of this series, "LLM-as-Judge Was the Start, Not the Finish": even teams that have adopted automated evals can still vibe-check within that system, by only running the eval suite occasionally, or by treating a passing eval run as confirmation rather than as one input alongside actually reading a sample of the outputs. Eval-driven development is as much a discipline about when you check as it is about what you check with.

What actually changes day to day

The practical shift is smaller than it sounds and doesn't require an elaborate testing framework to start. Before changing a prompt, a skill's instructions, or an agent's tool configuration, the eval-driven habit is: write down the specific case you're trying to fix as a test input plus an expected outcome, add it to your dataset, run your existing eval suite once to confirm it currently fails in the way you think it does, make the change, and run the suite again. If the new case passes and nothing else in the suite regressed, you're done. If something else regressed, you now know that before a user does, instead of finding out the same way you found out about the original problem — from a complaint.

The habit compounds. A team's first eval suite might be eight cases written down after the first three user complaints. A year of running this loop consistently produces a dataset with real coverage — every fix leaves behind a permanent test, so the dataset grows exactly in proportion to the number of real problems the team has actually encountered and fixed, which is a far more efficient way to build coverage than trying to anticipate every case up front.

The friction that makes teams skip it

Nobody skips eval-driven development because they think vibe-checking is better. They skip it because writing a proper test case, deciding what "pass" means for a free-form agent output, and running a full suite takes real minutes when the alternative — read three outputs, ship — takes seconds. This friction is legitimate, and the honest answer isn't to pretend it doesn't exist. It's to make the loop as cheap as possible: a small, fast eval suite that runs in seconds rather than minutes for the common case, with the heavier LLM-as-judge and trajectory evaluation reserved for a slower, less frequent pass — which is the same tiering logic covered in "Cheap Heuristics on 100% of Traffic, Expensive Judges on 10%," applied to development-time checks instead of production traffic.

Where the discipline actually pays off

The value of eval-driven development doesn't show up on day one — a single prompt tweak checked by eye and checked by an eval suite will often produce the same correct-looking answer, so the discipline can feel like overhead with no payoff early on. It shows up six months later, when a team has made two hundred small changes to an agent's prompts and tools, and the question isn't "does this one change look right" but "has cumulative drift across two hundred changes quietly broken something none of the individual changes seemed to touch." A vibe-checking team has no way to answer that question except starting over and re-reading everything. A team with an eval suite that's grown alongside every real fix can just run it, and get an answer in minutes instead of days. That's not a marginal productivity gain — it's the difference between an agent a team can keep iterating on confidently for years, and one where every change becomes progressively scarier because nobody's sure what they might silently be breaking.

Part of the "Evals for Agents" series on aiskill.market.