From Model Orchestration to Harness Engineering: What Makes AI Production-Ready?
The demo works perfectly. The same system falls over in production. The model didn't get worse — it just lost the layer of engineering standing between raw intelligence and a system you can trust.
A demo is a single, rehearsed path through a system. Production is every path a real user, a flaky network, and an ambiguous request can find. The model doesn't get worse between the two — the engineering around it just stops existing.
01 — the real problem
Why the Demo Works and Production Doesn't
Every AI demo has the same shape: a clean input, a model that has seen something close to it before, and a presenter who — consciously or not — steers around the cases that break it. Production has no presenter. It has thousands of inputs you never tried, tools that occasionally time out, and users who phrase the same request nine different ways. The model is identical in both rooms. What's missing in the second one is everything that was quietly compensating for it in the first.
02 — layer one
Model Orchestration: Coordinating Intelligence
The first layer most teams reach for is orchestration — the logic that decides which model runs, in what order, calling which tools. It has a handful of recurring shapes: choosing a model for the task, chaining prompts into steps, letting an agent loop between observing and acting, calling tools with structured arguments, and routing different requests down different paths.
03 — layer two — the real work
Harness Engineering: The Reliability Layer
A harness is everything standing between raw model output and an action a real system is willing to take. It's not one technique — it's a set of concerns that all have to be answered before an AI system is production-ready: what goes into the model, what happens when it gets something wrong, and what it costs to keep running. None of these live inside the model. All of them live in the engineering around it.
None of these are exotic. Most engineers already build equivalents for traditional systems — input validation, access control, retries, logging, budgets, regression tests. Harness engineering is that same discipline, applied to a component whose failure mode is a confident, plausible-sounding wrong answer instead of a stack trace.
04 — seeing it happen
One Request, Two Outcomes
Take the simplest possible production shape: User → AI Agent → Database/API → Action. Walk the same request — a cancellation — through it twice: once with nothing but the model and the API in between, and once with a harness around them.
This isn't just a diagram — it's real, runnable TypeScript. examples/harness-engineering-demo runs these same five requests twice each, unguarded and harnessed, and prints exactly where the harness catches what the unguarded path doesn't.
05 — the whole story in one line
Intelligence, Coordination, Reliability
Strip away the frameworks and the vendor names, and production-ready AI systems all break down into the same three layers, each answering a question the others can't:
The model provides intelligence. Orchestration provides coordination. Harness engineering provides reliability. A system is only as production-ready as its weakest layer — and for most AI systems that fail in the wild, the model was never the weak layer. The harness was missing.
06 — apply it
Workshop: From Client Requirement to Running Code
Theory earns its keep the first time you apply it to a request nobody wrote for the article. Here's one worked end to end — a client brief, the decisions it forces, and the two runnable projects that came out of them.
A mid-size e-commerce company wants their support chatbot to cancel orders on request, without a human in the loop for the routine cases. Four hard requirements: it must never cancel the wrong customer's order; it must never let the agent loop or spend unsupervised API cost; every cancellation must be explainable after the fact; and an ambiguous or malformed request must fail safely, not silently succeed on a guess.
The brief doesn't come with an architecture. Getting from one to the other is two steps: pull a plain feature list out of the brief, then decide, feature by feature, which layer actually owns it.
That table is the architecture. Two implementations of it exist, at two different levels of realism:
The LangGraph + Gemini version lives alongside the simulated one in the same workspace — same decision doc, same nine files, a real model behind the same graph.