Why enterprise AI pilots never reach production

Your AI pilot is not stuck because the model is not good enough. It is stuck because you called it a pilot, and everyone in the room heard permission to skip the parts that make software real.

I have been running enterprise delivery for twenty-five years, most of it out of Gujarat serving clients across three continents, and I have watched this exact failure repeat since long before anyone said "generative". We used to do it with data lakes. Then with chatbots. The word changes; the shape does not.

The pilot that works is the one that never ships

Here is the uncomfortable pattern. The pilot succeeds. The demo lands. Somebody screenshots it for a board deck. And then it sits, for eleven months, until a reorg quietly ends it.

It did not fail on accuracy. It failed because a pilot is permitted to fake the four things production cannot fake: who the user is, where the data lives, what happens when it is wrong, and who gets woken up. A pilot borrows a CSV export. Production needs a live connection to a system of record that somebody else owns and is not motivated to give you.

Why "start small and prove value" quietly guarantees failure

This is the advice in every consulting deck, including, I will admit, several of mine from around 2019. It sounds like risk management. In practice it selects for the wrong projects.

Small scope hides the expensive part

"Start small" almost always means "start where integration is easy" — which means starting somewhere that does not touch the ERP, the CRM, or anything under audit. You have then deliberately chosen the one corner of the business where your result does not generalise. When you try to move it somewhere that matters, you are not scaling a pilot. You are starting over, with a deadline and a reputation.

Proving value on a proxy metric proves nothing

Pilots get measured on things like time saved per query. Production gets measured on things like whether the number in the quarterly filing is defensible. Those are not the same test, and passing the first tells you very little about the second.

In March 2023 we ran a document-extraction pilot for a mid-market insurer. Accuracy on the sample set was excellent, the client was delighted, and we were three weeks from a signed scale-up. Then their compliance lead asked one question: for any given extracted field, could we show which page and which line it came from? We could not. Not because it was hard, but because nobody had asked during the pilot, so we had built a system that produced answers without provenance. That single missing requirement cost more to retrofit than the original build, and it is the reason I now treat "how will you defend this output" as a scoping question rather than a testing one.

The three questions we now ask before writing any code

We replaced our pilot checklist with three questions. They are deliberately awkward, and roughly a third of proposed engagements do not survive them. That is the point.

One: what system of record does this write to? Not read from — write to. Reading is easy and proves nothing. If the answer is "a spreadsheet the team maintains", the value ceiling is already set.

Two: who gets paged when it is wrong at 2am? Name the person. If that person is not in the kickoff, you do not have a production project, and you will discover this at the worst moment.

Three: what would make us stop? Written down, before we start. Kill criteria are far more useful than success criteria, because success criteria get quietly renegotiated and kill criteria do not.

What this looked like on a live delivery

A logistics client came to us wanting an assistant over their shipment exception data. Under the old approach we would have built a chat interface over an export and demoed it in five weeks.

Instead we spent the first eleven days answering question one, and the answer was ugly: the exception data lived in three systems that disagreed with each other, and nobody owned reconciliation. So the first deliverable was not AI at all. It was a reconciliation job and an owner for it.

That was an unpopular conversation. It was also the reason the assistant we shipped four months later is still running, because it sits on data somebody is accountable for. Had we demoed in five weeks, we would have had a very good screenshot and nothing in production.

What I would do differently

I would stop using the word pilot. We now say "phase one of production" even when the scope is tiny, because the words set the standard. A phase one has an owner, an on-call rota, and kill criteria. A pilot has a sponsor and a deadline for a demo. The engineering is identical; the outcomes are not.

If you want the external view, the published work on adoption gaps from McKinsey, Gartner and MIT Sloan Management Review is worth reading, as is Harvard Business Review on operating-model change. On the engineering side, the evaluation guidance from Anthropic and the deployment material from AWS and Databricks are more practical than most vendor content. But be careful: none of them can tell you who owns your reconciliation job.

Frequently asked questions

How do I know if my AI pilot will reach production?

Ask which named production system it writes to and who gets paged when it fails. If either answer is missing at kickoff, it will not reach production, regardless of how well the model performs on your sample data.

Should we stop running pilots entirely?

No, but stop calling them pilots. Scope the smallest possible slice of real production, with a real owner and real access, rather than a larger slice of pretend production. Small and real beats big and simulated.

What is the most underestimated cost in enterprise AI delivery?

In our experience it is provenance — being able to show where an output came from. It is rarely scoped at pilot stage and is consistently the most expensive thing to retrofit afterwards.

Here is your challenge for the next 24 hours. Open your current AI initiative and write down two names: the production system it will write to, and the person who gets paged when it is wrong. If you cannot fill in both by tomorrow, you do not have a delivery problem yet — you have a scoping problem, and it is far cheaper to fix today than in month eleven.