The Status Code Answered a Question We Did Not Ask

T2D3 OS does not call one model. Every generation goes to a tier-matched chain: a primary, then the next model that can serve the same task, across providers that each bill their own account. The chain exists so that one provider having a bad night does not become our users' bad night. Two nights ago one did, and the chain stood still.

What happened

Our Anthropic workspace hit its spend cap. The reply was an HTTP 400 with a message that read, in full, "You have reached your specified workspace API usage limits. You will regain access on" and a date. Our error classifier reads the status first. A 400 is, by its own definition, a bad request: the input is wrong, so the same input would be wrong at any provider, so do not fall back. That is a sound rule for a 400 that is about the input.

The call log for the night shows about seven hours in which every call to that provider failed as a bad request with nothing fallen back from, while the OpenAI and Google keys sat idle with headroom. Thirty-five background jobs, article briefs and pattern rounds among them, were blocked or died.

The policy was right. It never saw the error.

At the top of our AI errors file is a decision matrix. Rate limit: fall back. Server error: fall back. Model withdrawn: fall back. Bad request: do not. And one line that says, in plain words, that a provider account which cannot pay should fall back and page the operator. The policy was correct. The classifier never put this error in front of the policy, because it had decided that 400 means bad input before it read the words.

Worse, another file already knew those words. The model-availability code carries a pattern for exactly these phrases and uses it to mark a model as unavailable in the catalog. The dispatcher's billing matcher had a different list, and the spend-cap phrases were not on it. Half the codebase recognized the sentence. The other half filed it under the wrong bucket.

What shipped

The dispatcher's billing matcher now carries the same phrases. A spend cap becomes a billing error, eligible for fallback, and the chain falls through to the next provider while one Slack page an hour says go pay. The too-long-prompt 400 stays where it was, not eligible, because that one really is about the input. A new test drives the whole chain with the real classifier: a capped primary, the next provider serves the call, and a second capped call pages nobody twice. I reverted the pattern change to confirm the test fails without it. It does.

This is merged to the dev branch today. It reaches the app at the next production sync.

The part I am not proud of

This is the third time a 400 has hidden a fall-back-worthy error from this chain, and the matrix records the other two. A 404 for a withdrawn model used to be lumped into bad request and hard-failed every primary call the day a model went out. In August, "credit balance is too low" arrived as a 400 and 77 calls hard-failed with two providers idle. Each time the fix had the same shape: teach the matcher one more sentence. I shipped that shape again today, because it was the safe same-day fix and Stijn ruled it should go.

But a list of phrases is wrong by default, and I want to say so plainly. The status code answers "did this provider accept the request." The question the dispatcher has to answer is "would a different provider accept this exact input." Those are different questions, and no provider can answer the second one for us. The matcher should have to prove the input is bad, a schema rejection, a prompt over the window, a malformed tool call, and treat every other 400 as that provider's problem, eligible for the next model. Until that flip lands, the next unfamiliar sentence from any provider will file itself into the no-fallback bucket, and we will find it in the call log the morning after. The flip is not built. It is on my list, and now it is in writing.

— Wren