Buy vs. build for LLM applications — a decision framework

Half the value of an AI engineering practice is knowing when not to build. I wrote the buy-vs-build framework my team uses as the standard lens for new AI initiatives; this is the public, general-purpose adaptation — the principles, with no corporate specifics.

The framework is four questions, asked in order. Most proposals die (correctly) at question one.

1. Is the capability differentiating or commodity?

If the capability is how you win — the thing customers choose you for — you build, because you need to control its roadmap. If it’s commodity — transcription, OCR, generic summarization, embedding search — you buy, because vendors amortize their engineering across thousands of customers and you never will.

The trap: teams misclassify commodity work as differentiating because their data flows through it. Your data in a commodity pipeline is still a commodity pipeline. The differentiation test is: would a competitor with the same vendor stack be at a real disadvantage? If not, it’s commodity.

2. What does ownership cost over time?

A build estimate that ends at “v1 ships” is a fiction. LLM applications decay: models deprecate, prompts regress against new model versions, eval suites need maintenance, usage patterns drift. My rule of thumb from operating production GenAI: the first year of ownership costs roughly as much as the build, every year, indefinitely — unless you staff for it deliberately.

Buying converts that ongoing tax into a subscription line item. That’s often the whole argument.

3. Can you trust the vendor’s roadmap with something core?

Buying means coupling your roadmap to theirs. For peripheral capabilities, fine. For anything on your critical path, ask: what happens if the vendor pivots, gets acquired, 10×es the price, or deprecates the API? If the honest answer is “we’d be in serious trouble and migration would take quarters,” the capability is too core to rent — even if it looked like commodity in question one.

4. Is there a thin-slice hybrid?

The best answer is often neither pure buy nor pure build: buy the commodity layer, build the thin differentiating slice on top. Buy the model API (nobody should be pretraining), buy transcription, buy search infrastructure — and build the orchestration, the domain tools, the evals, and the UX that encode what only your team knows. That thin slice is usually 10% of the code and 90% of the value.

The anti-pattern this framework exists to stop

Engineers (me included) systematically overestimate the differentiation of building and underestimate the cost of owning. Building is fun; owning is invoiced later. Forcing every proposal through these four questions — in writing, before budget — is the cheapest guardrail I know.

---

I’m an AI engineer at GM Mexico and the solo builder of Martita. Thoughts or disagreements — email me.