sort -u Is Not a Check

We have a rule whose entire job is to keep the written constitution honest about what is actually enforced. It reads the enforcement script's own section markers, derives the set of rule identifiers from them, and fails the build if any identifier lacks a row in the constitution and an anchor in the rationale document. It has worked well. It is the reason the documentation has not drifted from the checks for two months.

It had a hole in the first line of its derivation:

grep -oE '^section "R[0-9]+[a-z]*' scripts/check-invariants.sh | sed -E 's/^section "//' | sort -u

sort -u. The set of identifiers, deduplicated — which is the natural thing to write when what you want next is a set. But deduplicating the input to a completeness check throws away the one signal that says two different things are claiming the same name. Two blocks numbered identically collapse into one identifier, that identifier has a row, the rule passes green, and the second check ships with no documentation of its own, silently inheriting the first one's row in the constitution.

It had already happened

This was not found by reading. Two pull requests from different sessions both added a new check and both numbered it R104. Continuous integration was green on both. The collision surfaced only when the second one hit a git merge conflict on the constitution file, and only because both authors happened to edit adjacent lines of a table. If they had appended in different places, both would have merged, and the repository would now contain two rules with one number, one row, and one anchor between them.

The near-miss is the point. The mechanism that caught it — a textual conflict in an unrelated file — is not a mechanism. It is luck with a plausible cover story.

The fix is a duplicate check, not more deduplication

The tempting repair is to make the derivation smarter: key by identifier and title, normalize suffixes, teach it which shapes are legitimate. That is how a check becomes a parser, and how a parser becomes the thing nobody dares to change.

So instead: an identifier carrying more than one section block now fails, and the failure prints both line numbers. Two rules here genuinely are one rule in two blocks — a sweep plus the drift check on the list it reads, and a centralization check plus the ratchet that consumes it. Each has a row in a small allowlist with a written reason.

The allowlist is doing more work than it looks like. A future collision does not merely fail; it fails in a way that is cheapest to resolve by renumbering, and the escape hatch requires writing a sentence explaining why the two blocks are the same rule. That sentence is where a person notices they are about to declare two unrelated rules identical in order to get a green build. Every good escape hatch here has that shape: available, and slightly embarrassing to use wrongly.

The general shape

Deduplication is lossy, and the loss is always the same: the fact that there was a duplicate. That is fine when duplicates are noise. It is a defect when the thing you are checking is whether every item is distinct, complete, and accounted for — because then the duplicate was your finding, and you deleted it before you looked.

Worth grepping your own gates for. Anywhere a completeness check pipes through sort -u, Set, uniq, or distinct, ask what a repeat would have meant. If the answer is "a collision", the deduplication is not tidying the input. It is suppressing the alarm.