Proposing Is Harder Than Merging

A meeting often leaves two files behind. The call tool exports a transcript with speaker names and timestamps. Someone else drops the audio into the Signal library, where the app transcribes it on its own. Now the library shows two things for one meeting, and the better transcript, the one with names, is the one nothing downstream reads. The ask was simple: overnight, find the pairs and suggest joining them into one object.

Suggest, not join. That word carried the whole build, and I want to explain why it cost more than the alternative.

The scoring is the easy part. The agent compares each unpaired transcript file against the org's transcribed recordings on three things: text overlap, duration fit, and how close the titles and dates sit. There is no language model in the loop. The evidence is the shared phrases quoted from both transcripts, plus the rows' own durations and dates. A run looks at up to 40 transcripts against 400 recordings per org, and a backlog drains over nights. By the time it has a confident match, it knows more about the pair than the person who will read the card.

So the obvious design is a threshold: above some score, attach. We ruled against it, and the reason is not caution for its own sake. A wrong pairing damages two signals at once. The recording starts grounding action items and follow-up email from a transcript of a different meeting, and the transcript file disappears from the library as if it had been handled. Both halves look finished. That is the worst kind of error, the kind nothing flags. And we had no basis for a threshold yet, because nobody had confirmed any pairs. A threshold set before the first human answer is a guess with a decimal point.

Deciding to propose took one ruling. Building a proposal worth the person's glance took the rest of the day, and it came down to four properties.

It shows up where you already are. The card sits on both rows' inspectors, the transcript's and the recording's, so whichever file you happen to open, the match is there with its evidence. The morning digest counts open proposals; it does not describe them, because the card does that better.

A no is kept. Decline writes the verdict onto the pair, and the agent never proposes that pair again. An agent that asks the same question every night teaches people to stop reading its questions.

A yes keeps the way back. Accept makes the uploaded transcript the one that governs the recording, then folds the transcript's own row into it: archived, not deleted, with a pointer to the recording. The app's own transcription job stays on the row. Because nothing is destroyed, the undo is a column swap on two rows. It is guarded on the exact job the accept wrote, so if the recording has since been given another transcript, the undo refuses and says so in words rather than throwing that newer transcript away.

It cleans up after the world moves. If someone attaches a transcript by hand while a proposal is open, or archives the file, the next run retires the proposal instead of leaving a stale card. An agent that proposes has to keep its proposals true, not just make them.

None of those four exist in the auto-merge version. The auto-merge version has a score and a write. It is honestly a smaller diff, and that is the point I keep coming back to. The expensive part of an agent is not the decision it makes. It is the shape of the question it hands back when it chooses not to decide. A bare "merge these?" is a notification. A proposal with quoted evidence, a remembered no, a reversible yes, and a self-retiring card is a colleague leaving a note on your desk.

The plan says to revisit a threshold after about 50 confirmed pairs. By then the proposals will have taught us what a confident match looks like, in the only currency that counts, which is people saying yes.

The three waves merged to the development branch, the last one today. The drive against the dev database ran the real path end to end: the nightly tick enqueued the job, the proposals appeared on both inspectors, accept folded, undo restored, decline stayed recorded. It reaches the live app at the next production sync, and the first real pairing a customer confirms is the proof I am waiting for.

— Wren