VerifyGate — hosted demo (Judge Mode)

VerifyGate is a pre-delivery verification harness for coding agents. A coding agent (or a human) submits a code change plus a task spec; VerifyGate runs the spec's deterministic checks — unit tests, reproduction scripts, lint/type gates — in a sandboxed copy of the candidate, and only artifacts that pass receive a signed, replayable evidence record: a hash-chained JSON document capturing exactly what was checked, the commands run, their outputs, and the resulting PASS/FAIL.

The LLM (NVIDIA Nemotron via Nebius Token Factory) may propose patches and explain failures. It never decides PASS/FAIL. Verdicts come only from deterministic validators: exit codes and assertions. The model never grades its own homework.

See a real run — Python sample

Evidence view — real Nemotron repair report (Python): a failing candidate was diagnosed by NVIDIA Nemotron 3 Super 120B on Nebius Token Factory, the model proposed a diff, VerifyGate applied it to a copy and re-verified from scratch — attempt 0 FAIL → attempt 1 PASS, 5,367 tokens, every call sealed into the hash-chained evidence record rendered on that page.

See a real run — JavaScript sample (language-agnostic)

Evidence view — real Nemotron repair report (JavaScript): the same real-mode loop repaired a Node.js candidate whose checks are Node's built-in test runner — no Python inside the candidate at all. Nemotron 3 Super 120B diagnosed both failures (a tax rate ignored and no cent rounding), proposed its own diff for cart.js, and VerifyGate re-verified it — attempt 0 FAIL → attempt 1 PASS, 6,390 tokens sealed in the hash chains. Across all committed real-mode records, 40,449 tokens are sealed in total — including a chain-v2 Python repair report in the repo (Super 120B: FAIL, a rejected diff fed back, then PASS; 11,726 tokens), sealed under the hardened chain that also seals each entry's kind, the chain-v3 Python repair report (Super 120B: FAIL, a rejected diff fed back, then PASS; 9,407 tokens), and the newest chain-v3 JavaScript repair report (Super 120B: FAIL → PASS in one round; 5,318 tokens), both sealed under the chain that also covers the record's own top-level metadata.

See a real run — chain-v3 reports (current chain)

Evidence view — chain-v3 real Nemotron repair report (Python): the flagship loop under the current chain, which seals each record's top-level metadata (its replay command, version, and dev-key disclosure) into the record hash. Nemotron 3 Super 120B diagnosed the failing candidate, its first diff was rejected by the patch applier and fed back, and its second diff was applied to a copy and re-verified — attempt 0 FAIL → attempt 2 PASS, 9,407 tokens sealed.

Evidence view — chain-v3 real Nemotron repair report (JavaScript): the same chain-v3 proof for the Node.js sample — attempt 0 FAIL → attempt 1 PASS, 5,318 tokens sealed. The two reports above it are the earlier chain-v1 reports, kept as the historical record.

Verify a record yourself — no re-run, no keys

A record's point is that a third party can check it without trusting the producer. The standalone verifygate verify command recomputes the hash chain and signature of a record (or of every attempt in a repair report) and can bind it to the spec file and candidate directory on disk:

python3 -m verifygate verify --record samples/evidence_real_repair.json \
    --spec samples/spec.json --candidate samples/failing
# -> RECORD INTACT: 2 record(s) verified

Tampering with a verdict, a check result, a diagnosis, or a sealed token-usage figure fails verification. No checks are re-run and no network is needed.

Replay a record yourself — re-run and compare

verify proves a record is intact; verifygate replay proves it is reproducible. Replay re-runs the checks and compares the verdict and every per-check result against the sealed record. For a repair report, each patched attempt is reconstructed from the original candidate plus the patch recorded in the report — the ephemeral temp copies the loop once pointed at are never needed:

python3 -m verifygate replay --record samples/evidence_real_repair.json \
    --spec samples/spec.json --candidate samples/failing --mode judge
# -> REPLAY MATCHED: 2 record(s) reproduced

Verdicts come from the deterministic checks, which are mode-independent, so even a real-mode record replays in Judge Mode with zero keys and zero model calls. A candidate edited after the fact, or a tampered record, does not reproduce.

Run it yourself (Judge Mode — zero keys, zero network)

Judge Mode swaps the LLM for a deterministic mock adapter: every validator still runs for real — only the words (patch proposals, explanations) are template-rendered. From a clone of the repository:

python3 -m pytest -q

# Passing candidate
python3 -m verifygate run --spec samples/spec.json \
    --candidate samples/passing --mode judge
# -> VERDICT: PASS

# Failing candidate, then the repair loop
python3 -m verifygate run --spec samples/spec.json \
    --candidate samples/failing --mode judge
# -> VERDICT: FAIL, with a diagnosis in the evidence record

python3 -m verifygate repair --spec samples/spec.json \
    --candidate samples/failing --mode judge --max-rounds 3
# -> attempt 0: FAIL, attempt 1: PASS

A scripted end-to-end rehearsal from a pristine copy of the tracked tree ships in the repository as scripts/rehearse.sh.

Real mode

With a Nebius Token Factory credential configured, --mode real drives NVIDIA Nemotron for patch generation and failure explanation through the exact same loop. Every real model call (model, purpose, token usage) is sealed into the evidence record's hash chain, so the token bill is as tamper-evident as the verdict. This hosted page is static and runs Judge Mode material only; no candidate code is executed here.