The One Word in Our Quality Gate That Cost 200k Tokens
Ten of our thirteen automated checks were advisory. All thirteen returned the same verdict word — and a machine reading it rewrote nine articles that were never broken.
Nine articles, and every one of them got rewritten at least twice.
The pipeline was doing exactly what we built it to do. An automated scorer read each finished draft, counted thirteen different things about it, and returned a verdict. The verdict came back as the word revise. So the writer revised — rewrote the whole piece, top to bottom, then submitted it again. Sometimes three times.
Not one of those rewrites was required. I went back through the logs afterward and checked every single one.
What does a quality gate actually owe you?
A gate owes you a decision you can act on. Ours was returning a mood.
Here is the part that took me embarrassingly long to see. Of the thirteen checks, exactly three could stop a publish: whether the HTML would survive the platform's ingest, whether the article had the structural pieces it needed, and whether it opened with a summary block. That is 23% of the set. The other 77% were suggestions — sentence rhythm, citation density, how uniform the paragraphs were. We had marked them advisory in the code. We had been explicit about it.
And then we handed every one of them the same word.
The scorer said revise whether the article was genuinely unpublishable or whether it merely had two attributed claims where the target was three. To a machine reading that output, those are the same instruction. It is an imperative. So it obeyed.
Myth one: the checks were too strict
This was my first theory and it was wrong. I spent a good while arguing we should loosen the thresholds — drop the citation requirement, relax the rhythm target. The team pushed back hard on that, and they were right to. Every one of those thresholds exists because something bad shipped without it.
The thresholds were fine. The reporting was not.
Myth two: it was a prompt problem
The second theory was that the writer needed better instructions about when to ignore a suggestion. We tried it. It helped a little and it papered over the actual defect, which is that no amount of instruction reliably beats a tool whose own output contradicts it. If the tool says "revise" and the prompt says "but only sometimes", the tool wins more often than you would like.
Myth three: nobody was counting the cost
This one is the expensive one, because it was true and invisible at the same time. A single unnecessary rewrite meant resending about 5k tokens of HTML, regenerating another 5k, and re-scoring for 1.5k more. Call it 12k per round. At two rounds per article across nine articles, that is a bit over 200k tokens spent producing versions nobody needed. Nothing in the system reported that number, because from the inside every round looked like diligence.
The two principles that actually fixed it
Once we stopped trying to change the rules and started looking at how the rules were being communicated, the fix got small.
| Name a state, not a demand. "Revise" tells the reader what to do. "Advisory" tells them what is true, and lets them decide. Only one of those survives contact with a system that follows instructions literally. |
So the verdict became three states instead of two: blocked, advisory, and pass. Alongside it we added a plain boolean called publishable, which is false only when something genuinely blocking failed, and a one-sentence summary that leads with the decision rather than burying it under thirteen check results.
The second principle is older and I keep relearning it. One rule, one definition.
The two lists that quietly disagreed
Our writing instructions carried a list of words to avoid. Our scorer carried its own list of words it graded. They were written months apart by the same people, and of course they drifted. Research on this failure mode is not exactly thin — every style guide that has ever been maintained in two places has done the same thing — but knowing that in the abstract did not stop us doing it.
A writer could follow our instructions to the letter and still get marked down for a stock phrase that sat on the grader’s list and appeared nowhere in the instructions. Worse, two of the scored targets — sentence rhythm and citation density — appeared nowhere in the blog-mode instructions at all. Data from our own revision logs put those two at the top of the list of things that triggered a rewrite. We were grading people on a syllabus we had not handed out.
The fix was to delete the second copy. The instructions now print the grader's own list, from the same array the check uses, and the numeric targets are computed for the specific article length rather than left as a ratio the writer has to do arithmetic on. If you are asked for eight pieces of evidence in a 1,400-word piece, the brief says eight, not "one per 250 words".
How do you find this in your own pipeline?
Four steps, in the order that actually works.
Count what can truly block. Go through your checks and separate the ones that prevent shipping from the ones that improve quality. If the ratio surprises you — ours was 3 to 10 — you have probably been treating the whole set as mandatory.
Read your tool's output as a machine would. Not as the author who wrote it. A verdict, a status string, an error message: each one is an instruction to something downstream. Ask what a literal reader would do with it, then check whether that is what you want.
Look for the second copy. Any standard stated in two places has already drifted or is about to. Generate one from the other, or delete one.
Measure the rework, not the failures. Failed runs are loud and visible. Successful-but-unnecessary runs look identical to real work from every angle except cost. Ours reported nothing at all until someone added up the rounds by hand.
One more trap worth naming
A client rollout last month produced a report that an image provider had gone dead — the health check said the model was reachable, generation failed anyway, and that combination sounds exactly like a broken upstream API. We spent real time on it. The live API turned out to be perfectly healthy. The machine running the tool had version 1.6.1 installed globally while the repository sat at 1.7.1, and the fix everyone was looking at had simply never been installed. Before you probe a vendor, confirm which build of your own code is actually running.
What this looks like from inside the industry
Automated quality gates are spreading fast into content and code pipelines, and most of the attention goes to what they measure. Almost none goes to how they report. That is backwards, and I think it is going to bite a lot of teams over the next couple of years, because the consumer of these verdicts is increasingly not a person who can apply judgment.
Protocols like the Model Context Protocol have made it genuinely easy to hand a model a set of tools and let it work. What that changes is the audience for your error messages. A person reads "revise" and weighs it. A model reads "revise" and revises. The tools published in the open server collection are only as good as the words they return, because those words are the interface now.
Our test suite grew by 5% while we did this, from 947 passing to 993, and not one existing test was deleted to make a change fit. That matters more than it sounds: the temptation when you rename a verdict is to quietly retire the tests that asserted the old one. Two of the failures we hit were genuine regressions we had introduced, and one of them — a check that reported "not evaluated" being silently collapsed into "passed" — would have shipped a system claiming it had verified citations it had never looked at.
The word cost us about 200k tokens. Finding it cost an afternoon. I would take that trade again, but I would rather have read our own output properly the first time.
Frequently asked questions
How do I know if my quality gate has this problem?
Look at whether a passing-but-imperfect result and a genuinely failing result produce different words in your output. If both return something imperative like "fix", "revise" or "retry", anything reading it literally will treat them identically. Separate the vocabulary before you touch the thresholds.
Should advisory checks just be removed instead?
No, and this is the trap. Those checks caught real quality problems and removing them would have traded rework for worse output. What they needed was an honest label. A suggestion reported as a suggestion still gets applied when it is cheap, and stops triggering a rewrite when it is not.
Does this only affect AI-driven pipelines?
It shows up hardest there, because a model follows the literal instruction every time. But the same defect wastes human effort too — a reviewer who cannot tell a blocker from a nitpick will either over-correct or start ignoring the whole report, and both outcomes cost you the gate.
