385 portals and 49 US states: how coverage is built, and what quietly breaks

What coverage actually means

Coverage is 385 enabled portals worldwide, including 49 of the 54 US state and territory systems. The remaining five are blocked at the portal, not broken in our code — a 403, a bot interstitial, a refused connection, and two that say so outright. We leave them unwired rather than listing a source that cannot deliver, because a config entry for an unreachable portal just relocates the silence.

The measurement that was wrong

For a long time we judged a source by when it last wrote a new row. It is the obvious metric and it is wrong. A portal that keeps listing the same open tenders writes nothing new, and scores as dead.

The fix was to record every run — how many notices came back, and how many of those were new — and read that instead. A run returning zero is broken. A run returning thousands of already-known notices is healthy but quiet. Correcting this changed the daily summary from 193 healthy and 155 stale to 352 healthy and 6 stale. No source had actually changed.

The failure nobody sees

The worst class of failure is the one with no error message. A source needs two halves: a client that knows how to fetch it, and a config entry that enables it. Wire only one and the source never runs, and nothing anywhere says so. Sixty-nine sources sat in that state until an audit compared the two lists directly. That equality is now enforced automatically.

The same shape appears in the fetch window. A one-day window returned literally nothing from sources that filter server-side by publication date — one portal gave 0 notices at one day and 1,123 at seven. The pipeline had been fetching zero from it on every run, successfully.

Frequently asked questions