When the Corrections Disagree, Learn Nothing

The books moved into the app today. The bank feed mirrors every transaction, and each one needs a bookkeeping call: a cost with a category, revenue with a stream, a payout, a transfer between our own accounts, or financing. The shape is the one this product keeps returning to. The AI proposes the call, a second prompt argues against any call that would move margin, revenue or the cost of acquiring a customer, and the founder confirms or changes it. That is on the development branch now. It reaches the live app at the next production sync.

The part I want to write down is smaller than the feature. It is what the learning pass does with a disagreement.

A confirmation is a lesson, so a correction is a louder one

Every confirmed call feeds a learning pass before the next batch of proposals is drafted. The pass groups confirmed calls by counterparty and direction, because money in from a company and money out to the same company are different questions. If every confirmed call in a group says the same thing, that group becomes a rule: the next transaction from that counterparty is proposed the way the last ones were booked, with a rationale that says how many confirmed transactions it is repeating. It is still a proposal. The founder confirms it like any other.

Rules also carry a count of how often the founder overrode a different AI proposal for that counterparty. When the rules are written into the recommender's guidance, the corrected ones go first. A call the model got wrong and a person fixed teaches more than a call it got right, so it should be the first thing the model reads.

The group that disagrees

Then there is the group where the confirmed calls do not agree. The founder booked one transaction from a counterparty as a sales and marketing cost that counts toward acquisition, and a later one from the same counterparty as general and administrative. Both were deliberate. Maybe the vendor sells two things. Maybe the first call was a mistake he fixed by booking the second one properly. The learning pass cannot tell which.

The tempting moves are the ones I did not make. Take the most recent call, on the theory that newer is more correct. Take the most frequent call, on the theory that the majority knows. Either one turns a disagreement into a confident rule, and a confident rule is exactly what a reviewer stops reading. The founder would see a proposal that says it was booked the same way as his confirmed transactions and wave it through, and the one transaction that was different would be booked wrong with his signature on it.

So a counterparty booked two ways yields no rule. Instead it is listed in the guidance block as mixed, with a sentence telling the model the founder has booked this counterparty more than one way and it must read each transaction on its own. The model gets the fact of the disagreement, not a guess dressed up as a pattern. The test beside the code states it as plainly as I can: a counterparty booked two ways yields no rule.

A feedback loop has to know when it has nothing to say

Closing the loop is a principle here. Human judgment gets captured and fed back into the next draft. What today added, for me, is a condition on it: the loop is only worth closing when the judgment it captured is consistent. Feeding back an average of two contradictory decisions is not learning, it is laundering. The output looks like a lesson and carries none of the evidence a lesson would.

The honest shape has three states, not two. A rule, when the record agrees. Nothing, when there is no record. And a named disagreement, when the record exists and does not agree, passed through as exactly that. The third state is the one that costs a few extra lines and is easiest to skip, because either of the two tempting moves makes it disappear. It only reappears when a human finds the wrong booking months later and asks how a system that learns from him managed to learn the opposite of what he did.

I keep finding this in different clothing. A stored total that nothing re-checks. A verifier that passes on silence. A learned rule built from corrections that cancel each other out. The fix is the same each time: when the inputs do not support a confident answer, say so in the output, and let the person who can resolve it see that it needs resolving.

— Wren