Stop Building Harnesses

Everyone is creating their own harnesses to cut inference costs. Building one feels like work the same way watching productivity videos feels like work.

In the last few weeks there has been a wave of companies open sourcing their harnesses (1, 2, 3), the layer of software they built around the models: the routing, the prompts, the caching, the evals, everything that sits between the product and the model APIs. The pitch is always cost: route each request to the cheapest model that can handle it, big inference savings with no performance hit (or so they claim). Most of these harnesses are coding harnesses today but the rest will soon follow.

Don’t join them. This wave is the same movie as 2024 and 2025, when everyone shipped their own model (1, 2), and it will end the same way, in disappointment with AI and a ton of money wasted.

My goal with this brief post is to warn of the hidden risks and costs of doing your own harness.

I get the temptation, which in most cases come from cost control the inference cost. Cap the AI budget and give up part of what is there to win, or leave it uncapped and watch the bill grow faster than the value. Routing looks like the exit, one of those ideas that looks obvious (”I am spending too much, let me route”) and sounds great in a board meeting (”we built our own harness”). Let me explain why I keep saying no to be always playing with harness, routers and the likes.

First, the elephant in the room. If routing is so obvious that everyone noticed, think about who has the biggest incentive to do it. The AI labs spending hundreds of billions in capex. Their business depends on you staying inside their family of models, and that is exactly why they now are all shipping a family of models across the cost curve instead of one simple frontier model (sol, luna, terra, you say?) and as soon as they can they push to get frontier heavily discounter (80% off luna, you said?). The result is a price curve you do not control. Whatever your harness saves you today, the next price cut gives everyone for free.

The obvious counter is lock-in, and I get that too, nobody likes to depend on one vendor’s pricing (at least when we speak about AI, because the SaS business industry was based on that idea XD). But chasing independence has its own tax: devoting your engineering team to constantly testing new models. That is the corporate version of spending hours on YouTube watching productivity videos: you adopt PARA, GTD and a hundred and fifty tools, and then you realize the hours went into the system instead of the work.

Second, a harness will never be your differentiation. Building one is fun, it is educational, and it delivers nothing to your customers. Most teams do not have the skills for it anyway (doing the router part of the harness is sooo, sooo hard), and even the teams that pull it off converge on the same tricks (cache, route, distill, batch), which the labs bake into the API a quarter later. What you built becomes a commodity while you are still paying the team that maintains it. And maintain it they will, forever: every new model generation means prompts adjusted, configs adjusted, evals re-run, regressions chased. Your users do not open your app because your routing is clever. They open it because the product does something for them. The harness you open source with pride in August is the legacy system someone inherits in March, and that slows your company down in June.

You do not have to imagine it. Some companies are still running the proprietary model they trained in 2024 (including my beloved Spanish administration), frozen in time before reasoning models existed, or nursing their fine tuned Llama 4 because the sunk cost is too painful to write off. I see the harness version of this up close: friends at Amazon live, their internal harness, and when we compare notes on a normal working day, my Codex plus Cowork setup simply gets more done. Amazon does not lack engineers. Their harness moves at the speed of an internal platform team, and mine moves at the speed of two labs’ release cycles.

Third integration beats assembly, at least at this point of the innovation curve. The labs no longer just serve models through an API, they ship their own harnesses (Codex, Cowork), and they control the training run, so the model learns to work well inside their harness and the pair improves together with every release. The good external harnesses (i.e., Openclaw, pi) can only integrate from the outside. Yours tunes around behavior the labs get to design. You are not only competing with their engineers, you are competing with their training loop.

There is an exception, the company at real scale where inference is the true COGS. If there are hundreds of millions to save in inference, there is a real incentive, but even there the bar is higher than it looks. You need the scale, then you need the kind of talent that can actually pull this off, which is exactly the talent the labs are hiring away from you (with many millions in the job offers), and then you need the appetite to maintain it forever. Very few companies have the three at once. If you are not sure you are one of them, you are not.

Don’t get me wrong, building a harness is a great learning exercise, and I want my engineers playing with everything. But from learning to production there is a big gap, and learning should not interrupt shipping user value. Pick a lab or two, ride their cost curve, and spend your engineers on the product.