Which user journeys could this change have broken?

An agent finishes a change and has to decide whether it is done. Running the whole suite is the safe answer and the slow one; running nothing is fast and dishonest. The tools built for that gap — SeaLights, Datadog’s test impact analysis, pytest-testmon, Ekstazi — all narrow the suite by watching which tests touched which lines on a previous run.

That mechanism is also their prerequisite. Coverage is something you have only after you have run the thing, which means none of them can say anything at all about a repository on its first day, or about the parts of a repository nobody has exercised yet.

And they answer the wrong unit. test_resolve_backfill failing tells you a test failed. It does not tell you that a claimant can no longer submit a signed form.

Confidence is the strongest path that reached the journey: the maximum, over every path, of the weakest edge on that path. A chain is only as trustworthy as its worst hop, but one solid route through is enough. A journey reached only through weak or synthesized edges is labelled VERIFY MANUALLY — it is never silently dropped from the list.

The failure modes are not symmetric. Flagging a journey that turns out to be fine costs a few minutes. Silently dropping a journey a change really did break costs a production incident and the user’s trust in the tool at the same time. So testgraph over-selects on purpose: edit a shared config file and it fans out to almost everything, because that is the truth about a shared config file.

The same asymmetry drives the integrity guard. Before answering, testgraph refuses to run off a stale or corrupted index — a wrong graph produces a confidently wrong “you don’t need to test that”, which is the one output a test selector must never produce. Seeds come from line ranges, so an index built before the code moved resolves the diff against spans that no longer mean anything. If a changed file’s bytes still disagree with the indexed copy after a re-sync, the answer degrades to RECALL DEGRADED and names the file.

The obvious pitch is suite reduction, and I am not making it. The regression-test-selection literature is directionally unkind to static selection: measured against a dynamic baseline, the static tool STARTS reduced suites 68.3% against Ekstazi’s 84.1%, and carried a 5.9% safety-violation rate doing it. Dynamic beat static on both size and safety.

testgraph has selectivity numbers of its own across four repositories, and they are not published here, because it has no measured missed-fault rate at those scales to put beside them. Recall 1.00 was established on exactly one target — five hand-labelled real commits and roughly twenty seeded mutation sites — and a recall figure from one repository is not a property of the tool.

What is defensible is narrower and still decisive: it answers cold. Zero coverage, zero CI, zero prior runs. That is a positioning advantage rather than a moat — an incumbent could collect coverage on build one, or bolt static analysis on — and it is not free either, because a maintained journey registry is real hand-authored judgement that nothing derives for you.

The other half of that ledger is what running a journey then found. Joined on repository and commit, the two halves produce the one number the project has so far asserted rather than measured: a journey that failed on a commit whose selection never named it.