Fill in college deadlines from the Common App grid

[DanialBeg]

46 of 6,273 colleges had a real application deadline. Everything else fell back to a typical date for the round and was shown to the student as an estimate — which is the one thing a deadline tool should not do.

Common App publishes its members' deadlines as a PDF and keeps it current through the cycle, so this is a re-runnable ingest rather than a one-off import.

1,127 schools parsed → 955 matched → 935 rows to fill
  agrees with a value we already had:  30
  differs (left alone):                13
  unmatched:                          169

Real deadlines go from 46 → ~980 colleges.

The bug worth knowing about

Fixed column offsets read Yale's restrictive-early date as its regular-decision deadline.

The grid repeats its column header on all 55 pages and puts the columns in a different place on each: ED ranges over columns 36–39, and the drift compounds rightward until two pages sit 28 characters apart at RD. On Yale's page, REA is where other pages keep RD.

Every round is now read against its own page's header. parse.test.mjs pins that case and the four others that broke during development (wrapped names, the Coordinate school type, Rolling, rows before the first header).

The 30 values that agree exactly with the hand-curated rows are the main evidence the parse is sound.

Two deliberate limits

It never overwrites an existing value. The 13 disagreements have no consistent winner — the grid is right about Stanford (Jan 5, where we held a wrong Jan 2) and wrong about Georgetown (Jan 1, where Jan 10 is real). The 46 curated rows were checked by hand and a bulk import doesn't get to undo that quietly, so disagreements print as differs: lines for someone to settle.

It won't guess between campuses. Arizona State, Kent State and Colorado State have six or more campus rows each, so they stay unmatched rather than risk putting a main campus's deadline on a branch. Most of the other 169 unmatched are foreign universities absent from the Scorecard dataset (Aberystwyth, Dublin City, Esade, IE).

It also skips schools listed more than once (a few appear once per program with different deadlines under one name), and doesn't store EDII/EAII — 185 schools publish them, but AppDeadlineType has no round for them and a column nothing renders is data nobody reads.

Failing loudly

PDF parsing is brittle by nature, so the guards matter more than the parser:

• throws if fewer than 900 schools parse (layout-change detector)
• refuses to let two grid rows claim the same college
• writing requires --apply; it used to be the no-flag default, which puts one mistyped invocation straight into production

Testing

cd scripts/ingest-deadlines
node --test parse.test.mjs   # 8 passing
node ingest.mjs              # dry run, writes nothing

Not yet applied to the database — this PR is the script only.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
webapp	Ready	Preview	Sep 25, 2026 6:14am UTC