Scoring a detector honestly

Opening a PDF in the browser and nodding at it is not a test. It is how you end up tuning for the three files you happened to have.

Forty-one published forms, scored in Node with no browser and no clicking. They split by what they can prove.

Nine forms that are their own answer key

A form that ships an AcroForm has already been marked up by its publisher: every widget they drew is, by definition, a place a human writes. Render the page with annotations off, so the detector sees exactly the printed sheet, and the widgets it never saw are the blanks it missed.

Across those nine: 752 widgets, 650 found.

Read the misses before tuning for them. The W-9 scores 40%, but its marked page shows why — the SSN and EIN are comb fields, one widget per character, so nine red boxes are a single blank a person types into once. The score counts per widget. The picture is the appeal.

Thirty-two forms with no answer key, and none invented

The pre-digital 1040s have nothing to check against, and flattening a fillable form to manufacture a key would only test our own flattener, on a file no user has.

So they are held to invariants instead. Every slot lies inside the page’s view box. It is big enough to type in. No two slots cover the same ground. They arrive top to bottom and left to right, so Tab walks the page the way an eye does. Every one carries a label, even if only Blank 7.

Running that across 32 real forms failed immediately, on two counts.

Deduplication compared slot edges, so a rule under a box escaped being merged into it whenever it was a few points wider. Ordering by vertical position alone left two boxes on the same row in whatever order half a point of rounding decided — which made Tab jump across the page and back.

Neither was visible on the three files that had been opened by hand.

The baseline is the point

Per-form counts are pinned in a checked-in file. The suite fails if a tuning change finds fewer slots than it used to, which is the only way to tell a real improvement from a lucky one.