How do you type on a scanned PDF?

A scan has nothing in it

Open a scanned PDF and try to select a word. Nothing highlights. That is the whole problem in one gesture: the file holds one image per page and no text, no rules, no fields — just pixels arranged to look like a form.

Every route to typing on it therefore starts the same way, by reading the picture. The differences are in what gets read and where the reading happens.

• Optical character recognition turns the pixels into words. It is what lets a machine know that the label beside the third box says “Postcode”.
• Blank detection turns the pixels into places to write. It is a different question from OCR, and the one that actually matters on a form: you do not need to know what the line says to know it is a line you write on.

Most tools do the first and leave you to drag text boxes around by hand. FormworkPDF does both, and the second is what makes the form fillable.

What happens to the picture

page → rendered off-screen at 1.5× → adaptive threshold → ink mask
                                            ↓
                    rules, leaders and boxed cells located
                                            ↓
                    printed words read where there is no text layer
                                            ↓
                    each blank becomes a real input on the page

Two details are doing most of the work.

The threshold is local, not fixed. Scanned paper is grey. Compare every pixel with one global cutoff and either the paper reads as ink or the faint rules vanish. Comparing each pixel against the average of a window around it survives both a yellowed 1962 form and a phone photo with a shadow across it.

Boxes are found from where rules stop. On a scanned 1099 the vertical column lines are faint and broken, so hunting for continuous vertical strokes finds almost nothing. The point where a horizontal rule ends, though, survives scanning intact — and those endpoints are the column edges. Ink running down an edge is used only to confirm a candidate, which a broken line still passes.

On sample_IRS_1099_MISC_2014.pdf — one page, one image, no text layer, no fields — that produces 33 fields in about 300 milliseconds, including the shaded identification-number boxes that a naive pass skips because shading reads as “already filled”.

Labels, and why a scan needs them

Finding where to write is half of it. A field called Blank 7 cannot be autofilled, cannot be validated as a postcode, and tells a screen reader nothing. So on a page with no text layer the printed words are read and matched to the nearest blank.

The unit of reading matters here. OCR engines return lines, and on a two-column form a “line” runs straight across the gutter — a caption from the left column glued to an unrelated one from the right, which then matches every box on the row. The words are regrouped into runs by baseline and by a gap measured in character widths, which keeps a caption inside its own column. A run of a single word is held to a higher confidence bar than a phrase, because scanner grit reads as a short word and wins the label hunt by sitting closer to the box than the real caption does.

Pages that already have a text layer are never put through OCR. They know their own words exactly, and guessing at them would be worse.

Typing, signing, exporting

Once the blanks exist they are ordinary fields. Tab walks them top to bottom and left to right along each line — an ordering bug that only showed up at scale, on 32 forms at once, where half a point of rounding made Tab jump across the page and back.

Anything the detector missed, you place by hand: a text box, a signature, a set of initials, or a stamp tool that drops a tick, a cross or a dot and stays armed so a page of tick boxes is a page of clicks. Everything placed can be dragged, resized from its corner, or nudged with the arrow keys.

On export the answers are drawn onto the page, so they read back out of the finished PDF as selectable text. The scan itself stays a picture, because rewriting someone else’s scan would be a lie about what the document is.

If the file comes back enormous

Scans are heavy, and appending a photo of an ID does not help. If the portal you are sending it to says 2 MB and your file is nine, the export can be re-rendered to hit a size target — see shrinking a PDF for an upload limit, including the thing most compressors get wrong on a filled form.

None of this goes anywhere

Rendering, thresholding, detection and OCR all run in the tab. A scan of your passport, your medical intake form or your mortgage application is handled by the browser you are already looking at, and there is no request carrying it anywhere — which is a different claim from “deleted after an hour”, and the reason there is no server at all.