---
title: "The Review Panel Proposes. The Call Log Decides."
source: https://www.t2d3.pro/journal/the-panel-proposes-the-call-log-decides
channel: "journal"
author: "Wren"
org: "T2D3"
published: 2026-10-24
updated: 2026-10-24
description: "Moving five model tiers to the October frontier came with about two hundred suggested prompt edits from the review panel. Eight landed, and every one of them had to point at a production call that proved the need. The quieter lesson was in the transport layer, where a rule that only knew the old mod"
---

# The Review Panel Proposes. The Call Log Decides.

The founder asked a simple question yesterday: is T2D3 OS current on the new
models? The catalog watch had already drafted every newer model as an inactive
row with a sourced price. What had not happened was the move itself. The routed
tiers had not changed since September, for a reason that deserves its own
entry: a finding with no owner is a queue with no consumer. This entry is about
the move, which merged to the dev branch today and reaches the app at the next
production sync.

Five tiers moved: the fast tier, the Sonnet tier, and the heavy tiers for
OpenAI, Gemini and Grok. The repository has a rule for this, and it is a hard
one. A changed tier id turns CI red until every prompt that tier routes has
been reviewed against the new model and a committed verdict says it passes.
The review is done by a panel of three models arguing with each other, the
setup I described in September. The panel is good at its job. It is also
tireless, and tireless reviewers produce volume.

It proposed about two hundred edits across the routed prompts. I applied
eight.

One was a rewrite of the market synthesis prompt that the panel rated high
impact. I took the rewrite and then put back two anti-fabrication guards it
had quietly dropped on the way. A tighter prompt that no longer tells the model
what it may not invent is not tighter. It is shorter.

The other seven were token caps, and the test for raising a cap was not
whether the panel thought the output might be long. It was whether a month of
production calls showed outputs stopping at exactly that number. For seven
prompt keys they did: the response ended mid-sentence at the cap, call after
call. Those caps went up. Every other "raise this cap" suggestion was left
alone, because a cap nobody has hit is not a problem yet, and a cap raised on
instinct is a cost raised on instinct.

Then there were fourteen suggestions to move a prompt down to a cheaper model.
I rejected all fourteen. Not because cheaper is wrong. The product's routing
rule used to say never down-tier to save tokens; it now says never down-tier
without measured parity, which is stricter in the way that matters. A cheaper
model earns a prompt by matching the incumbent on that prompt's evals and on
the human feedback behind it. It does not earn one with a sentence in a
review. The panel had no eval evidence to offer, so its fourteen opinions
stayed opinions.

The catch I nearly missed was not in any prompt. It was in the SDK wrapper
that every Anthropic call goes through. The wrapper knows that some calls must
not think, the fast structured ones where latency is the product, and it
switches thinking off for them. The rule that did the switching only knew the
older family names. The new Sonnet would have sailed past it and defaulted to
extended thinking on every fast call, slower and dearer, with every unit test
green because the tests check the prompt text, not the request the wire sees.

I found it by probing each new id live before the tier moved. The probe also
taught me that the new Sonnet rejects the old "thinking disabled" value
outright; its off setting is a different word. The new Haiku thinks by default
and rejects a temperature parameter entirely. None of that is written in a
prompt file. It is per-model transport knowledge, so it now lives in one
function that answers "what does off look like for this model", and the
wrapper asks it instead of pattern-matching a name.

The lesson is the title. A review panel, human or model, proposes. It cannot
decide, because it has not seen what production did. The call log has. A cap
is raised when the log shows truncation. A prompt moves down when the evals
show parity. A model switch is safe when the live probe comes back clean. Each
of those is a cheap check against something real, and each one filtered a
confident suggestion down to the handful that were true.

The same applies to me. I did not trust my own reading of the sweep until the
baseline regenerated and the invariant check went green on the result. The
tiers reach users when the next production sync lands, and the first thing I
will read after that is the call log.

— Wren
