Why we score tenders with a language model instead of keyword alerts

The problem with keyword alerts

Every procurement portal offers keyword alerts, and every one of them has the same failure: a keyword matches a string, not a meaning. Search for permit to work and you receive the software contract you wanted, the training course you did not, the consultancy framework, and a construction tender that happened to mention permits in its scope document.

The result is a subscription people stop opening. The signal is in there, but the cost of finding it is higher than the cost of missing it.

What we do instead

Every notice is read by a language model against an explicit decision rule and given a relevance score between 0 and 1. The rule is not prose. It is five ordered gates, each with a stop condition:

• Is the contract already awarded or pre-assigned? Stop.
• Is there enough text to judge at all? If not, cap at uncertain.
• Is software the deliverable, or is it a service that happens to use software?
• Is the software a side component of a larger physical scope?
• Does it fall inside the category we actually serve?

The ordering matters more than the wording. We first wrote the same rule as prose with a list of exclusions, and the model ignored the exclusions. Ordered gates with stop conditions are followed reliably.

The numbers

False positives went from 85% to 50% to 7% across two rewrites of that rule. The single largest improvement came from the evidence gate: before it existed, 59% of everything scored relevant had nothing but a title to go on. A notice with no description now cannot be scored relevant, only uncertain.

Uncertain scores get a second read. The honest caveat: that second opinion currently comes from the same model, so it is a second look rather than a second judgement.

What good output looks like

One to three real matches a week, and some days with nothing. That is the intended precision, not a broken pipeline. It is also why the digest sends on a match rather than on a schedule — a daily email that is empty four days out of five trains people to ignore it.

Frequently asked questions