research-stack · research.devclusterai.com
MVP · records to 2026-09-23 · every number is from the project's records, listed at the end
build log21 August → 18 September 2026ten entries · newest first

The POC, entry by entry.

The half before the MVP: eight questions answered by the tool this replaces, a kernel designed to be checkable, twenty-eight packages built in five days, and a gate of eight checks run three times in one day. Each entry is dated, says what was built or measured, gives the number it produced and names the record that holds it. One of the ten is a correction of a number this page itself carried until today.

2026-09-18BUILD-1docs/PACKAGES.json BUILD-1issue #981PR #1028

Fifty-nine test binaries, and the wait between a change and its verdict.

what was wrong

Every file under crates/kernel/tests/ is its own integration-test binary linking the whole kernel. Fifty-nine of them. Invisible in a diff, paid on every change.

what we built

The fifty-nine become one binary, tests/it/main.rs, a module per former file. Nextest filters move with them (binary(x)binary(it) & test(/^x::/)) and every insta snapshot is renamed rather than regenerated, so an unreferenced snapshot errors instead of quietly passing. Dev and test profiles dropped to line-tables-only debuginfo.

test build
226.7 → 82.1 s
cargo test --workspace --no-run --timings · same pod, before and after
MEASURED
unit time
856 → 197 s
total compile time across the test units
MEASURED
kernel test binaries
59 → 1
7.15 GB of linked test executables → 263 MB
MEASURED
workspace executables
8.4 → 1.5 GB
recorded in the PR as the done_when asks
MEASURED

Measured on the same pod, before and after — the number is worthless otherwise. Same test count as main at the rebase point, every snapshot referenced. BUILD-1 is an MVP-stage ops package; it sits here because it merged the day before the gate.docs/PACKAGES.json BUILD-1 · issue #981 · PR #1028 · commits 54d76881, ae0a1793

2026-09-17 → 18S1 · S3 · E2 · B6 · S2agit log --mergesdocs/PACKAGES.jsonSPEC [C-210], [C-216]

The layer the side-by-side said to replace.

E1's finding was not the tally; it was that the kernel's reasoning was ahead of the old engine and its sourcing was behind, because R1 — the interim retrieval built to get the POC measured at all — was a single search API and a plain fetch. Between the gate and the MVP campaigns, that layer was replaced.

  • search and fetchReal search and fetch through the substrate, and the first of the fallback ladder. S1 #682 · S1a #693 · S3 #748 · S2a #857 · S2b #869
  • the kernelThe seams those units needed: K6 #739, K7 #740, K9 #741, K13 #812, K8 #858 — and FIX-686 #813 behind them.
  • evaluationE2 #809 — the synthesist numbers its claims by content, so a re-ask is a memo hit on every unit — and E7 #859, the evaluation doctrine E1's method was the seed of.
  • governanceG2 #810: every fresh fetch and search row carries the id of the Policy event it ran under.

Two of those merges changed what a ledger is: B6 (#811) landed Campaign-memory as the twelfth, S2a (#857) Knowledge as the thirteenth. The POC's kernel admitted eleven. What it was worth is the MVP gate, three days later.git log --merges main, 2026-09-16 → 18 · SPEC [C-210], [C-216] · docs/PACKAGES.json

what that layer then measured: the MVP build log, 19 September

2026-09-16 · 22:49X1docs/GATE-POC.mdcommit 89e22a6

Eight of eight — and the seven this page published instead.

the gate as it stands

The eight checks of the spec, each one a number the Eval ledger holds, the verdict recomputed from value and target when the document renders and never read from a stored flag. The document is written by rsk gate doc over the ledger; nothing in it is typed by hand.

check 1 · gold-8 headless
8 / 8
runs at the baseline's terminal state · 0 deviations · 72 calls invoked
PASS
check 2 · kill −9 → resume
0 repeats
resumed report hash equal to the uninterrupted one
PASS
check 3 · replay
79 / 79 · $0
0 calls, 0 reserves · 29 tables rebuild-equal
PASS
check 4 · anchor re-read
490 / 490
of 498 anchors, every resolved one re-read byte-for-byte · 0 mismatches
PASS
check 5 · reject → repair
5 pairs
7 unit outputs rejected on an invariant, 5 repaired, live
PASS
check 6 · wall p95
19.4 s
3 live runs at standard depth · max $0.031 · 1 paid call each, 24 cached
PASS
check 7 · the bench line
42 · 0 FAIL
gated rows inside the gate's own ledger
PASS
check 8 · replay = rebuild
14 / 14
172 claim rows equal · 6,130 event refs, 0 dangling
PASS

One clause of check 8 is recorded and not gated, and the document says so: the idempotency refutation reproduces on no real run (outcome=no_charter_line) — only the hand-built fixture proves the shape. FIX-2-3 below is what made the document admit it.docs/GATE-POC.md — measured 2026-09-16T22:49:15Z at 89e22a6, release · "Failures: None"

the retraction · this page published the pass before it

The gate ran three times on 16 September, and this site has been quoting the middle pass — the 16:12 run at e41f64a, where check 8 failed. Its numbers were real; they were simply superseded ten hours later by the pass taken after FIX-590 merged, and nobody moved the page. Every figure below is the last pass.

what movedthis page saidthe record says
the eight checks7 PASS / 1 FAIL8 PASS
anchors · resolved · confirmed466 · 451 · 451498 · 490 · 490
reject → repair pairs45
live wall p9536.5 s19.4 s
most expensive live run$0.057$0.031
replay served from the ledger73 / 7379 / 79
event refs · dangling6,485 · 06,130 · 0

Nothing moved in our favour by accident — the anchor count rose because the later pass covers more runs, and the p95 improved because the Mac was quiet. The point is that a public page carried a superseded measurement for days while the file it cites said otherwise.superseded: docs/GATE-POC.md at be6d0d4b (16:12:59Z, tree e41f64a) · current at a627ff4f, tree 89e22a6

2026-09-16FIX-590issue #590PR #673SPEC §4, §7

One live run would not replay.

what the gate found

The 16:12 pass is the one drawn below: seven checks pass and check 8 fails on a single run. Of thirteen replays over 182 claim rows, one came back unequal — live run d7f8f21a…, the third of the E-4 live trio.

1 PASS 8/8 · 67 calls gold-8 headless 2 PASS 0 repeats kill -9 → resume 3 PASS 73/73 · equal replay · rebuild 4 PASS 0 mismatches anchor re-read 5 PASS 4 pairs · live reject → repair 6 PASS p95 36.5 s wall p95 ≤ 180 s 7 PASS 0 FAIL of 42 bench line 8 FAIL 1 of 13 ≠ replay == rebuild #590 · a live run at concurrency 6 cited a page a sibling task had fetched replayed at concurrency 1 the same output is rejected: “A1: url not fetched in run” FIX-590 packaged: anchors resolve against the task's own retrievals · re-run at $0

The pass that found the miss, exactly as it rendered. Check 7 shows PASS here after two earlier failures on one K-1 row; check 8 is the red one. Ten hours later, with FIX-590 merged, the same eight checks all read PASS — the entry above.docs/GATE-POC.md at be6d0d4b — check 8, the named unequal run, "Failures"

what it actually was

Above concurrency one, invariant A1 resolved a claim's anchor against any task.retrieved row of the run rather than the task's own — so admission depended on which sibling committed first. At concurrency 6 the live run admitted a claim whose page a sibling had fetched (docs.stripe.com/error-low-level); replayed at concurrency 1 the same output was rejected invariant.A1 url not fetched in run. The spec already forbade it; A1 was simply not covered.

The fix is the issue's own option (a): A1 resolves against the task's own task.retrieved rows. Found by the second gate pass, confirmed by the independent audit, merged the same day as PR #673.

This is the failure the POC was run to find. A gate whose checks all pass on the first attempt has told you about your gate, not about your system.issue #590 — the run, the task, the sibling and the two options · docs/PACKAGES.json FIX-590 · PR #673 · docs/SPEC-kernel.md §4, §7

2026-09-16PERF-1 · T-BENCHdocs/BENCH-POC.mdcommit 79c3370issue #506

Forty-seven gated rows, one unmeasured, and a laptop that had to be quiet.

the line

Eleven micro-benchmarks against the targets written down in the test plan before any of them ran. Every value is a bench.result event carrying the doctrine and plan hashes of the tree that produced it; rsk bench report recomputes each verdict from value and target. Eighty lines after the latest-per-row rule, 47 gated, 0 FAIL.

benchPOC targetmeasuredverdict
K-1 admit latencyp99 ≤ 2 ms per kind, n = 10 k0.342 – 1.703 msPASS
K-2 sustained admit≥ 500 events/s5,375 – 5,649 /sPASS
K-3 rebuildhash-equal, < 60 s @ 100 k eventsequal · 4.166 sPASS
K-4 anchor resolve≤ 5 ms @ 315 KB (exact ≤ 100 µs)0.089 / 0.315 / 1.781 msPASS
K-5 footprint≤ 60 MB peak, headless @ 837.3 MBPASS
K-6 guardsp50 ≤ 10 ms per guard0.002 – 0.009 msPASS
A-6 the fold≤ 50 ms @ 1 k claims11.829 msPASS
U-4 replay100 % served from the ledger78 / 78 · 0 callsPASS
O-1 kill −920 random kills: 0 repeats, equal hash0 · equalPASS
G-2 budget0 over-cap admits, ≥ 1 refusal, cap reached0 of 400 · 30 refusalsPASS
U-2 first-try≥ 90 % (live, ≤ $2)not attemptedunmeasured

U-2 is the honest gap: the live row was never attempted — the OpenRouter balance was below zero, and the previous pass's one attempt halted on payment after $0.019. The document says unmeasured rather than carrying the old figure forward.docs/BENCH-POC.md — bench by bench; 2026-09-16T11:33–11:35Z at 79c3370, release, mock model server

the box it ran on

The line is timing-sensitive and ran on a laptop, so the laptop is part of the measurement: a waiter polled every 20 s for a one-minute load ≤ 8 with no rustc, cargo, ld or test process alive. It opened at load 6.74 and ended at 12.16, and every K-1 row carries the host's load at its own measurement.

Two earlier passes failed gate 7 on task.retrieved — 2.214 ms then 2.789 ms against a 2 ms target, both under the load ceiling. The target was not loosened. D-K6 (#506) is still open: how K-1's number should be stated is the spec owner's call.docs/BENCH-POC.md — "Load while measuring", K-1 per kind · issue #506 (open)

2026-09-16E1docs/E1-2026-09-16.mdbench/sets/gold-8doctrine/eval/v1

Five of eight against the engine it replaces. The bar was six.

the judgement

The old browser-driving engine's report beside the kernel's on each of the eight questions, judged by a method written down first: a verdict with flip conditions, a mechanical source audit, quote fidelity against the live page, then the draft attacked from four angles. Better 0, not-worse 5, worse 3. The self-check moved two questions down — they ask for authoritative sources and the kernel had two among twenty vendor blogs.

browser agents at scale deep $0.72 $0.20 92 → 12 calls NOT-WORSE liveness checks standard $0.87 $0.19 117 → 11 calls WORSE RLS as a backstop scout $0.11 $0.08 16 → 4 calls NOT-WORSE RLS super- user, practice standard $0.94 $0.17 122 → 10 calls NOT-WORSE idempotency tutorial standard $0.71 $0.15 94 → 9 calls WORSE retries and backoff guide standard $0.76 $0.16 93 → 10 calls WORSE observability and alerting standard $0.72 $0.16 99 → 9 calls no report NOT-WORSE same question re-run standard $0.14 $0.04 23 → 2 calls no report NOT-WORSE old tool · its own estimate at batch rates (real ≈ 2×) kernel · settled in the ledger, live better 0 · not-worse 5 · worse 3 → 5 of 8 · the bar was 6 · recorded as eight Decision.feedback events, actor owner

656 paid calls and $4.97 against 67 calls and $1.15 — the kernel's as the eight reports E1 judged recorded them, which is why they do not sum to the 72 the gate counted over the re-recorded tree six hours later. Ours is settled in the Budget ledger; the old engine's is its own estimate at batch rates.docs/E1-2026-09-16.md — method, verdicts, tally · bench/sets/gold-8/questions.json

the finding that mattered more than the tally

Every loss had one shape. The kernel was structurally ahead — adversarial checks, could-not-establish, honest confidence, verbatim quotes — and behind on where it read: its primary-source share roughly halved on every question where the old engine found primary sources.

primary-source share per question · old (Q1–Q6) vs kernel (Q1–Q8) Q1 47 % 36 % Q2 76 % 38 % Q3 100 % 75 % Q4 100 % 20 % Q5 92 % 11 % Q6 85 % 42 % Q7 5 % Q8 5 % of 179 fetch.done rows in the gate ledger stackoverflow.com 7 / 7 failed · 403 challenge reddit.com 3 / 3 failed · robots_disallowed yoti.com 3 not ok browser-use.com/pricing ok · 200 — the $0.02 miss is extraction, not fetching ahead: adversarial checks · could-not-establish · honest confidence · verbatim quotes behind: primary sources · locked out of the sites practitioners write on ten defects filed → the evaluation doctrine (E7)

Of 179 fetch.done rows, stackoverflow.com failed 7 of 7 on a 403 challenge and reddit.com 3 of 3 on robots. But the pricing page the first question needed was fetched ok/200 and the rate on it never reached the answer — that miss is extraction, not fetching, the same sentence this project is still writing about itself a week later.docs/E1-2026-09-16.md — the ledger check, the ten defects · doctrine/eval/v1 (E7)

the same eight questions, re-judged inside the MVP campaigns: better 3, not-worse 1, worse 4 — the bar is still six

2026-09-15 → 16FIX-2-1…7docs/PACKAGES.jsonissues #494–#500PRs #562–#568

A critic read the POC against its own test plan.

Before the gate could close, a completeness pass read the built POC against the spec, the test plan and the package catalogue, looking only for places where the record claims something the code does not do. Seven, each becoming a package with an issue, a branch and a PR.

packagewhat the record claimedwhat was true
FIX-2-1gate 7 PASSesone K-1 kind at 2.214 ms against a 2 ms target, measured under the load ceiling — re-run, not loosened
FIX-2-2check 5 is gatedthe target attached only to a live-recorded row, and nobody had run the live gate — check 5 read “recorded, not gated”
FIX-2-3B3's refutation reproducesunmet on every real run: only the hand-built fixture proves the shape — the doc now says so
FIX-2-4live rows run on a scheduleone test id with no evidence of ever having run, and no scheduled workflow existed
FIX-2-5snapshots are checked in CIcargo insta --unreferenced=reject named in three documents, absent from the CI job
FIX-2-6gate 1 expects seven done, one gapfour test-plan lines contradicted the baseline file, which records eight done
FIX-2-7every test id is greppable20 test-plan ids cited by no test function's doc comment

Two of the seven were resolved by changing the document instead of the code — FIX-2-3 and FIX-2-4 — recorded as the outcome, with the reason: do not leave the done_when claiming what the record contradicts.docs/PACKAGES.json FIX-2-1 … FIX-2-7 · issues #494–#500 · PRs #562–#568

2026-09-14 → 15K0 … F1docs/PACKAGES.json stage pocPRs #100–#267

Twenty-eight packages, and the harness before any of them.

what was built

Twenty-eight packages carry stage poc. The order is the part worth saying: the test harness merged first — mock model server, fixture recorder, fault-injection runner, bench harness, adversarial fixtures (PRs #100–#103) — and only then the doctrine, the units, the statechart and ledger, retrieval, the budget stub and MCP. Sixteen merged on 14 September, each with CI green.

  • the harnessT-MOCK #100 · T-FAULT #101 · T-BENCH #102 · T-FIXTURES #103 — the mock server's gold-8 seed recordings cost 8 live calls and $0.0121.
  • the brainB1 #104 doctrine · B2 #105 six unit manifests · B3 #225 the argument ledger · B4 #266 the report renderer.
  • the kernelK2 #176 invariants · K3 #177 statechart · K4 #193 unit runner · K5 #224 run ledger and resume. K0 and K1 landed on main on the 11th and 12th, before the rule that every merge goes through a PR.
  • the edgesR2 #194 fetch ledger · R1 #196 interim retrieval · G0 #195 budget stub · F1 #267 MCP server.

The other twelve are the seven completeness fixes, FIX-590, and E1 and X1 — which produce documents, not code, so have no merge commit. Each package carries a done_when written before it was built.docs/PACKAGES.json — the 28 at stage poc · git log --merges main 2026-09-14 → 16 · commits f3ff025d, f0b9d5f4

2026-09-11 → 12R0 · D0 · D1 · D2 · T0docs/DESIGN-kernel.mddocs/SPEC-kernel.mddoctrine/v1

Three candidate designs, one judged, then a spec you can cite by line.

the design

One research synthesis, then three candidate kernels written in full and judged against each other — testability-replay-first, simplest-correct, libsql-perf-first — then one design, two rounds of spec fixes, and a test plan with two coverage rounds. All before the first line of kernel code. What came out: one writer, an append-only event log, eleven ledgers.

Every rule that can refuse an event has a stable code and a path into the payload that caused it — a refusal is a row, not a log line. That is what makes the gate's checks statable: a claim marked observed carries an anchor whose quote is the exact bytes of a page the run fetched, or the event does not go in.

the doctrine

The doctrine was not rewritten for the kernel. B1 lifted it verbatim from the old engine's prompts, and its done_when is that the generated skill file comes back byte-identical — so any later change to how the kernel reads is on the record, not drift. Six laws, four of which need a reader:

  • L1Audit before you search. Never issue a query on the asker's framing alone. the ledger's rule
  • L2Evidence outranks priors — including when it contradicts the asker, or your own earlier conclusion. judged
  • L3Every claim carries a source you actually opened. A URL you did not visit is a fabrication. the ledger's rule
  • L4Label your epistemics: observed, inferred, assumed. Never let assumed wear the costume of observed. judged
  • L5Gaps get reported, not smoothed. Absence of evidence is never quietly upgraded. judged
  • L6The reframe is offered, not imposed: answer the question asked, then say what the better question is. judged

L1 and L3 are process facts the ledger records, so the kernel decides them per finished run. The other four need a reader — which is the whole of the MVP's evaluation problem, and why six of 53 reports pass all four.doctrine/v1/DOCTRINE.md · doctrine/eval/v1/JUDGES.md · docs/SPEC-kernel.md §1–§3 · docs/PACKAGES.json B1, B2

2026-08-21 → 09-10gold-8bench/sets/gold-8/questions.jsonweb-research/runs

Eight questions, answered first by the tool this replaces.

The set was not invented for the kernel. Eight real questions had been put to the old browser-driving engine across August and September and its runs kept — question, terminal state, wall clock, calls, cost. On 11 September those eight became gold-8.

the set
8 questions
1 scout · 6 standard · 1 deep · built 2026-09-11 from web-research/runs
RECORDED
the old engine
656 calls
LLM calls across the eight runs · 10.6 M prompt tokens
RECORDED
its cost
$4.97
its own estimate at batch rates · the figure a live re-run doubles
RECORDED
its wall clock
60.7 min
3,644 s over the eight · 738.8 s on the slowest
RECORDED

Two of the eight never finished: the observability question and its cached re-run ended in error on a provider 402, which is why the side-by-side's last two verdicts compare a kernel report against nothing. A baseline with two holes is still a baseline, as long as the holes are in the file.bench/sets/gold-8/questions.json — the eight, each with terminal state, elapsed, calls and estimated cost

21 Aug old engine 8 questions 11 Sep the design 11 ledgers 14 Sep the build 28 packages 16 Sep measured bench · gate · E1 16 Sep FIX-590 merged #673 18 Sep build time 227 → 82 s

The whole half in one line. What happens after the last stop is the MVP: twenty campaigns, fifty-three questions, and every report read.the entries above, each with its own record

next: the MVP build log — 53 questions for $8.27, 16.42 over 672 expert-written rubric items, and the four numbers we had to take back