Build the eval before you build the feature

Almost every stalled AI project we are called into shares a shape. There is a prototype. The prototype demos beautifully. Someone senior has seen it and is now asking when it ships. And nobody in the room can answer a simple question: is it right?

Not right in the abstract — right on the thousand inputs that matter to this business, measured in a way two people would agree on. Without that, every subsequent decision is taken on vibes, and vibes do not survive contact with production traffic.

What an eval actually is

An evaluation harness is not a benchmark score. It is a dataset drawn from your real work, a scoring function that reflects what your users care about, and a way to run both on every change. That is the whole idea. The hard part is not the tooling; it is deciding what good means and writing it down.

• A dataset sampled from real inputs, including the ugly ones people currently handle manually.
• Labels produced by whoever owns the decision today — the underwriter, the clinician, the support lead.
• A scoring function that distinguishes wrong from unhelpful, because they need different fixes.
• A held-out slice nobody optimises against, so you can tell improvement from overfitting.

The abstain path is a feature

Systems that cannot decline are systems that guess. On the Atlas underwriting project, the single largest jump in user trust came not from better retrieval but from allowing the assistant to say it did not know — and then scoring it on choosing that correctly. Underwriters stopped auditing every answer once they believed the confident ones were confident for a reason.

“They spent the first three weeks building the thing that told us whether we were wrong. It turned out to be the whole project.”

What this buys you

Once the harness exists, arguments get shorter. Model upgrades become a measurement rather than a leap of faith. Cost reduction stops being scary, because you can drop a cheaper model into the routine 80% of traffic and prove nothing regressed. And when someone senior asks whether it is ready, there is an answer with a number attached.

It costs a week or two up front. We have never once regretted spending it, and we have repeatedly been called in to clean up projects that did not.