Almost every organisation now has an AI prototype that impressed a room. Far fewer have one running in production. The distance between those two states is consistently underestimated, and it is almost never closed by choosing a better model.
What actually breaks
Three things, in our experience.
Retrieval quality. A demo runs against ten curated documents. Production runs against ten thousand, half of them outdated, some contradicting each other, many restricted to particular roles. Answer quality collapses long before the model is the constraint.
No definition of correct. Prototypes are judged by whoever is watching. Production needs a scored evaluation set, built from real questions with known good answers, so a change can be shown to be an improvement rather than assumed to be one.
The surrounding system. Permissions, audit trails, rate limits, cost controls, fallback behaviour when the provider is slow. None of it is interesting to demo, and all of it decides whether the thing survives contact with real users.
A more reliable sequence
- Write down the decision the system is meant to improve, and how you will know it did.
- Build the evaluation set before building the feature. Fifty real questions is usually enough to start.
- Fix retrieval before touching prompts. Most quality problems live here.
- Ship to a small group with a feedback path, and watch what they actually ask.
- Only then optimise cost and latency.
The uncomfortable question
Ask early whether the problem needs a model at all. A meaningful share of the work labelled AI is better solved with a query, a rule or a redesigned form — faster, cheaper and easier to defend. Knowing where the boundary sits is most of the skill.