An Unmet Input Is Not a Failed Draft

The last two passes converged shapes: a check pack was reading module_items rows of a type no writer had emitted in months, and the framework recorded the resulting zero as the module's quality. Twice — once for personas, once for customer interviews — and the second time it landed as a generic test, so a third module cannot repeat it quietly.

This pass starts where that test stops. It proves a pack reads a row type somebody writes. It says nothing about what the check then reads off that row. And it left one finding explicitly unresolved, because resolving it required a product decision rather than a diff.

The check that could never pass

The ICP's Five-Test framework has an Economics test with exactly one deterministic check: CAC payback must be present, plus one of NRR or LTV:CAC. It has no model checks. One check, one test, so the test's verdict is the check's verdict — pass or hard fail, nothing between.

The ICP builder skill deliberately never drafts those numbers. That is not an oversight; it is the correct behaviour. CAC payback and NRR are the organization's real billing and CRM figures, and a synthesizer that invents them produces a confident, wrong ICP. The skill's own done-summary says as much: Economics isn't auto-drafted — fill it in the Economics tab.

So every freshly built ICP failed Economics. Guaranteed, by design, on the first run. And the framework's pass rules fail a module at two failed tests, which meant one more soft spot anywhere — a thin pain citation, an under-filtered profile — flipped the whole scorecard to fail.

What fail does is the part that had gone unexamined. The persist boundary refuses to save a draft whose scorecard failed. Not "saves it flagged". Refuses. The user watched the skill research, synthesize and assemble an entire ICP, and got back an error telling them to add interview and density evidence — advice that could not fix the actual problem, about a draft that no longer existed.

Drawing the line at "is there anything to judge"

The tempting fix is to mark the check not-applicable when the draft came from the skill. It is also wrong, and it is wrong in a way worth naming: the verifier cannot see a draft's age or provenance, and the same check has to serve two situations that look identical to it — an ICP built ninety seconds ago, and an ICP whose Economics tab has sat empty for six months. The second one is the entire reason the check exists. A blanket exemption silences it.

The line that actually separates them is not new versus old. It is is there anything to judge. An Economics tab with zero rows has supplied nothing; there is no draft to grade, only an input the product has not been given yet. An Economics tab with LTV:CAC but no CAC payback has supplied something, and what it supplied is incomplete — that is a real failure and it stays one.

So findings gained one optional field: awaiting_input, a sentence written for the user. A check that sets it is saying this is unmet, not wrong. The verifier never counts it toward a fail, and — the half that matters just as much — never lets it read as a pass: a scorecard carrying an unmet input can rise no higher than needs_iteration, so the lock gate shows the sentence instead of a green light. The draft persists. The user sees, in the check's own words, that CAC payback and NRR come from their billing and CRM and where to enter them.

The rule that makes this safe from becoming a general-purpose excuse is written into the type: a check with something to judge must not set the field. Empty is unmet. Partial is failed.

One layer down: what does a check read?

With the shape-convergence test in place, the obvious next question is whether the same sweep runs at field granularity. A check reading data.cohort when nothing has ever written a cohort key fails exactly the way the earlier bug did — it computes over undefined, returns false, and the framework files that as the module's quality.

The answer is that the join is enumerable, but on different keys than expected, and one half is weaker than it looks.

The read side generalizes cleanly. Every pack declares an interface per item type and casts the row's data to it immediately after filtering on the type, so what fields does this module's checks read comes out of one file, with no per-module knowledge at all.

The write side does not generalize to the same precision. The plan was to join against the module catalog, which declares an itemTypes[].schema for every type — until you look at what is in it. All fifty-seven declared item types across both catalogs carry schema: {}. It is a typed placeholder nobody ever filled in: the field exists, the shape it promises does not. And the writers cannot substitute for it, because they are UI components, extractor lanes, seeders and handlers, many of them building the data object as a variable typed somewhere else entirely.

What is available is the module's territory — its component directory, its handler, its own library lane — and every field name any of them writes appears there. That is coarser than a per-item-type schema, but it is enough to catch the bug class: a field nothing in the module writes at all. That is now asserted for every gated module.

The residue

The sweep found the thing that coarseness costs, in the module that prompted it. The ICP pack's economics interface declared target and cohort; its trigger interface declared public_signal_source. Nothing writes any of the three, and no check reads them. They are the fossil of a framework file promising checks that were never built — payback under twenty-four months, NRR over 110, targets split by cohort, a trigger field of that exact name. The declarations and the promises are both gone now, replaced by descriptions of the checks that exist.

cohort would have failed the strict version of the new test. target would have passed it, because the word appears elsewhere in the ICP tree. That asymmetry is not hidden in a commit message; it is written into the test's own docblock, where the next person to widen the sweep will find it.

Both halves of this pass are the same lesson from opposite directions. A verifier that grades nothing looks exactly like a verifier that graded something and found it wanting — and the distance between those two readings is where a product quietly tells its users the wrong thing about their own work.