The model writes the code. Something else has to say whether it worked.
A solo harness cannot beat a lab on agent count or model quality. What it can have is a
real verification oracle for the repositories it actually runs against — so the judge here
is mechanical, deterministic, and has no language model in it anywhere.
checks in the verdict4
model calls in the verdict0
auto-commits0
runtime dependenciesstdlib
The invariant everything is arranged around
An agent that writes code is model-driven by definition. The interesting question is what
decides whether what it wrote is acceptable — and the common answer, a second model
reviewing the first, inherits every failure mode of the first. It is confident on a bad
diff, it is not reproducible, and it cannot be re-run to the same answer tomorrow.
So: the language model is never the judge. The verdict is four stateless
checks, each one a program with an exit code. Run it twice on the same tree and you get the
same answer, on any machine, with no network and nothing to re-roll.
Check
Fails when
tests
the project’s own suite does not pass
ir_contract
a change leaves its declared scope, or removes a symbol without opting in
secrets
a credential is found in the staged change
cve
a reachable vulnerability is found — not merely a listed one
The symbol contract is the interesting one. It is the same deterministic index that powers
RunEcho: a change declares the scope it intends to touch,
and the gate compares the actual structural diff against that declaration. An agent that
quietly deletes a function on its way past is caught by a hash comparison, not by someone
noticing in review.
A pass is not a merge
When all four checks pass, the harness leaves the branch sitting there for a human to
commit. There is no auto-commit. That is a deliberate guardrail rather
than an unfinished feature, and it is the boundary that production agent tooling has
converged on after the fully-autonomous-to-merge pitch was walked back — Devin repositioned
from “builds anything on its own” to “human on the loop required”
once real unattended completion rates landed at 14–15%.
“Autonomous” here means the plan, produce and verify cycle runs unattended. It
does not mean nobody sees the result. The gate’s value is highest exactly when nobody
is watching each step — in an interactive tool the human is already judging every diff,
which is precisely where a mechanical oracle earns least.
Roles, not models
The second half is the runtime. A role is a job — consult, scout, builder,
reviewer, scribe — and a gateway resolves each role to whatever model currently does
that job well. Swapping a model is a gateway configuration change; it is never a code
change here.
Roles carry tools, and the runtime runs the full request, tool call, result, request
loop against them. The two tool sets are a code-intelligence set and a retrieval set,
and every tool in them is read-only. It can find, trace, fetch and
reason. It cannot yet edit a file or run a command — that gap is the honest distance
between what exists and a self-driving loop.
the four checks — a model, not a capture
tests the suite passes
ir_contract scope held, no silent symbol removal
secrets nothing credential-shaped is staged
cve no reachable vulnerability
any failure → exit 1, nothing proceeds
all four → the branch waits for a human
The code-intelligence set picks its index by the question being asked: one index answers
where a symbol is, live and never stale; another answers how things
connect, stored and able to lag uncommitted edits. A structural answer costs a few
hundred tokens where the file it was derived from costs thousands.
Where the loop actually lives
The orchestration that ties plan, produce and verify together does not yet run inside this
project. It runs in a foreign agent runtime, with each phase as an agent call, so today the
harness supplies the judge and the tools while somebody else supplies the loop.
The sharpest edge of that is embarrassing enough to be worth naming: even the harness’s
own event emissions are currently written by asking a model to run a shell command. That
violates the project’s own invariant on the write side, and it is the first thing
being fixed — before the event bus grows a read side, and well before any write-capable
tool set exists.
Runtime state lives outside the repository entirely; code lives here, state never does.
Every step of a run is appended to a local event log as newline-delimited JSON.
Why this and not a better chat interface
The alternative direction was live and got rejected: make this a model-agnostic interactive
tool you sit and drive. Matching an interactive coding assistant means matching its write
tools, session loop, approval layer and context management — pure catch-up against teams
orders of magnitude larger, with no edge anywhere in it for one person.
The mechanical verdict has no equivalent in any of those tools. That is the thing worth
building, so it is what the roadmap points at.
Agent infrastructureVerificationPythonLiteLLM
Private repository. The oracle it verifies against is public: RunEcho.
