research-stack · research.devclusterai.com
MVP · records to 2026-09-23 · every number is from the project's records, listed at the end
seq 1Questioncharter.locked

It is all on the ledger.

Thirteen append-only, content-addressed ledgers in three consistency classes, one writer, one gate — the whole system, not a Python pipeline. The direction changed on 11 September.

seq 2Questioncharter.locked

Composed ledgers, as rails.

sequenced · one writer · dense seq per stream Question Plan Doctrine Policy Decision Run Unit Argument Memory B6 mergeable · order by (hlc, node) Fetch Eval Knowledge S2a scarce · Σ live + settled ≤ cap Budget one transaction admit() · schema · refs · invariants ledger.sqlite3 append-only derived views triggers · rebuilt hash-equal report.md a pure function of the ledger rejected: nothing written · admission.rejected + the attempt, kept
sequenced · one writer · dense seq Question Plan Doctrine Policy Memory Decision Run Unit Argument mergeable · (hlc, node) order Fetch Eval Knowledge scarce · Σ live + settled ≤ cap Budget admit() · schema · refs · invariants · class ledger.sqlite3 append-only derived views rebuilt hash-equal report.md pure function of the ledger rejected: nothing written admission.rejected + the attempt, kept

Every event's id is a hash of its content and every ref must resolve; views and report are re-derived from the events alone, and a rejection writes only the rejection. The POC’s kernel admitted eleven ledgers; Memory (B6) and Knowledge (S2a) arrived on 17–18 September.crates/kernel-events/src/event.rs — Ledger::ALL [13], Ledger::POC [11] · docs/SPEC-kernel.md §1–§2, [C-210] Memory, [C-216] Knowledge · docs/DESIGN-kernel.md §0 · BENCH K-3: rebuild hash-equal on 29 tables

seq 3Argumentclaim.asserted · seq 2

Append-only, addressed by content.

Argument · stream = run id · seq dense · one writer refs — ids of earlier events, hashed into this id hypothesisseq 1id = blake3(content) claim.assertedseq 2id = blake3(content) verdictseq 3id = blake3(content) edge.attacksseq 4id = blake3(content) synthesisseq 5id = blake3(content) claim.asserted′same seq 2 · new bytes rejected · no UPDATE exists: the only write is an appendnew bytes → a different id · seq 2 is takenUNIQUE (ledger, stream, seq) · the attempt is kept

There is no update. A rewrite would be a different id under a seq already taken, so the batch is rejected and the attempt kept as evidence. Each id is blake3 over the content, the ids it references included, so replay re-derives the same ids from the same bytes.docs/SPEC-kernel.md §1 event shape, id definition · docs/DESIGN-kernel.md P6–P7, §13.3

seq 4Argumentclaim.asserted · anchors[0]

The quote has to be in the bytes.

report.md · Findings Stripe caches status codes and response bodies for keyed requests once endpoint execution begins, replaying even 500 server errors on subsequent retries. high confidence · docs.stripe.com “Stripe’s idempotency works by saving the resulting status code and body of the first request made for any given idempotency key, regardless of whether it succeeds or fails. Subsequent requests with the same key return the same result, including 500 errors.” A1 fetch.done · status ok · docs.stripe.com bodies/<sha256_text>.zst · extracted text, bytes Stripe’s idempotency works by saving the resulting status code and body of the first request made for any given idempotency key, regardless of whether it succeeds or fails. Subsequent requests with the same key return the same result, including 500 errors. start end the model cites url + quote · the kernel resolves fetch_ref and [start, end) · rungs exact → nfc → folded GATE-4 · every resolved anchor decoded from the store and compared byte-for-byte 490 / 490 · 0 mismatches

A quote that is not in the bytes is a rejection, not a footnote. Three rungs only — exact, NFC-normalised, then whitespace runs and typographic punctuation folded; none for case folding or edit distance. The gate re-read every resolved anchor: 490 of 490, 0 mismatches.docs/SPEC-kernel.md §2 anchor resolution · docs/GATE-POC.md check 4 · BENCH K-4: exact / nfc / folded 0.09 / 0.32 / 1.78 ms at 315 KB

seq 5Runadmission.rejected · attempts

Mechanical checks reject on the spot. Judged ones come back with a reason.

admission mechanical · program schema · number.unsafe refs resolve · refs.unresolved A1 anchor bytes == quote A2 a label needs evidence B2 Σ live + settled ≤ cap class · seq · hlc judged · reasoned L1 audit first L2 evidence > priors L3 source opened L4 epistemics labelled L5 gaps reported L6 reframe offered claim.assertedobserved · quote not found rejected on the spot · invariant.A1 · /anchors/0/quoteattempts row kept · the model gets attempt 2 with the reason claim.assertedobserved · 1 anchor resolved COMMITseq n · hlc judge unitor the owner judge.scored · label, reason, calibration → Eval ledger

The guarantee is no unlabeled error. A refusal is a row with a code and a path into the payload that caused it, never a log line: reason_codes_closed asserts the set the kernel can emit equals the spec’s exactly, and every code maps to one stage.docs/DESIGN-kernel.md §7.2 — the invariant registry and the closed reason-code list · docs/SPEC-kernel.md §1 reason codes, §7 judged.L1–L6 · doctrine/v1/DOCTRINE.md

seq 6Runrun.lease · resume

Survives kill -9 without paying twice.

live run process 1 audit orient recharter investigate + verify synthesise done kill -9process dies resume process 2 served from the ledger: checkpoint (task_ref, attempt) → no call repeated model calls: 0 · report hash equal O-1 · 20 random kills · 0 repeated model calls · report hash equal · gate check 2 · resume_repeats = 0 the one sanctioned re-issue: a call lost between the wire and its output commit

Twenty random kills: zero repeated calls, identical report. Each model call has three durable points — recorded before the wire, raw body committed on arrival, then admission — so a new process re-issues only what was never committed.docs/BENCH-POC.md O-1 · docs/GATE-POC.md check 2 · docs/SPEC-kernel.md §2 Unit row, §8 check 2

seq 7Runresearch replay

Replays for $0.

live run job 20260910- 225300-83c859 OpenRouter · Brave · the pages 10 calls · $0.1625 · 19 sources call 1 call 2 call 3 call 4 call 5 call 6 call 7 call 8 call 9 call 10 replay rsk --offline research replay ledger ledger ledger ledger ledger ledger ledger ledger ledger ledger no network client is constructed · every call answered by its checkpoint (task_ref, attempt) · every search and fetch answered by the seeded Fetch events 0 calls · 0 reserves · $0 · same event ids · report blake3 equal U-4 · 78 of 78 calls served from the ledger · 0 calls · 0 reserves gate check 3 · 79 / 79 · report identical · rebuild equal on 29 tables

79 of 79 calls served from the ledger; the same event ids; the same report bytes. No network client is built at all. The bench line’s U-4 row is 78 of 78 over a different ledger — two measurements, not one, so the drawing carries both.docs/GATE-POC.md check 3 — replay_cache_hit hits 79, total 79, 0 calls, 0 reserves · docs/BENCH-POC.md U-4 — 78 of 78 · docs/DESIGN-kernel.md §13.2 · job id and cost: Gold-8 Verdicts, Q6

seq 8Budgetreserve · settle · cap

Money is reserved before it is spent.

cap · run:<id> · $0.40 for a standard run reserve reserve reserve reserve reserve refused · class.scarce_cap settled actuals reserve before the call Σ live + Σ settled + amount ≤ cap checked by the one writer settle after the actual replaces the estimate integer micro-dollars refused a rejection with a reason never an overrun, never a crash G-2 · 8 clients × 5 reserves of 10 against a cap of 100 · 100 admitted · 30 refused · 0 over-cap · p99 1.35 ms caps are events · expiry is an event · a settle that arrives after expiry is accepted and marked late

Eight clients demanding 400 against a cap of 100: 100 admitted, 30 refused, none over. No unit is invoked without a reservation; the writer checks live plus settled plus the new amount against every cap it names, and settle replaces the estimate with the actual.docs/SPEC-kernel.md §2 Budget row · docs/GATE-POC.md G-2 · bench/sets/gold-8/caps.toml (standard cap $0.40)

seq 9Runrun.started … run.finished

One run, on the record.

rsk research start · or deep_research_start over MCP plan and doctrine hashed into run.started audit audited orient oriented recharter chartered investigate investigation_done synthesise synthesised done investigator × lines · verifier × claims, together state.entered task.created unit.invoked unit.output unit.admitted premise ×n search.done fetch.done task.retrieved hypothesis ×n charter.locked lines + claims to verify claim.asserted + anchors verdict → edge.attacks claim_status (the fold) unit.rejected → repaired synthesis A5: cites observed claims, or gap run.finished report.md report.md Decision.feedback · actor owner refs the run and the claim it judges every arrow above is an event; every column is a stream you can query

The plan is the old method as a hashed statechart. Every state writes its events, the report is rendered from them, and the owner's verdict is itself an event that refs the claim it judges.docs/SPEC-kernel.md §3 plan v1, §7 report and Decision appends · docs/E1-2026-09-16.md

The walk-through, state by state

  • run.startedrsk research start, or deep_research_start over MCP. The plan and the doctrine are hashed into the first event, so the run names exactly the method it will follow.
  • auditThe auditor unit is invoked under a reservation. state.entered · task.created · unit.invoked · unit.output · unit.admitted · premise ×n
  • orientSearches and fetches are events; every page the task read is a task.retrieved ref. search.done · fetch.done · task.retrieved · hypothesis ×n
  • recharterThe charter is locked: the lines to pursue and the claims to verify. charter.locked
  • investigateInvestigators work the lines and verifiers the claims, together: a claim is asserted with its anchors, a verdict becomes an attacking edge, claim_status is the fold over them, and a rejected output comes back repaired. claim.asserted + anchors · verdict → edge.attacks · claim_status · unit.rejected → repaired
  • synthesiseThe synthesis may cite only observed claims, or declare a gap (A5). synthesis
  • donerun.finished; report.md is rendered from the ledger — a pure function of it, with no timestamps. run.finished · report.md
  • Decision.feedbackThe owner's verdict on the report, actor owner, refs the run and the claim it judges. The eight side-by-side verdicts on the POC page were recorded exactly this way.

Plan v2: a cover loop, an article, and a source a run may not read

Plan v1 above is still the default. Plan v2 (plans/audit-first.v2.toml) adds two states and reads twice as much: cover writes a charter addendum for every coverage cell still short and sends the run back to investigate — twice at deep, three times at exhaustive, ending when a pass adds no counting claim — and write renders the article. The charter is never edited; the addendum folds after the locked lines.

BLOCK-1 blocks a source at the fetch lane by work identity, not by URL string: a listed /abs/ arXiv id refuses its /pdf/ twin, a DOI refuses its mirror, and rsk eval ext refuses to score a run whose body store holds one. It is what makes an external benchmark number mean anything.

Reading roughly doubled, information recall roughly doubled, and analysis fell by about half — finding #1328, still open. The first explanation, a fixed output budget on the synthesist, was measured and is false: 39 of 39 calls ended stop at no more than 3,670 of 8,000 tokens. The article is claim-starved instead. Promotion between plans is E4's, by score, never by hand.docs/DEPTH-1-NOTES.md · docs/SPEC-kernel.md [C-360]…[C-364], §3 plan v2 · docs/GOLD-EXT-CLEAN.md — the three refusals BLOCK-1 made live · issues #1283, #1284, #1355, #1328

seq 10Evalbench.result · gold-ext/v3

How a number on this site is made.

No figure here is written by hand. BENCH-POC.md and GATE-POC.md are rsk bench report and rsk gate doc; GATE-MVP.md is rsk gate mvp report; GOLD-EXT-CLEAN.md is assembled from the bench.result rows rsk eval ext wrote. A verdict is recomputed from value and target at render time, and a row the ledger cannot give prints not measured with its reason.

the external number, end to end

16.42 is the one number here that is not ours to define. An external benchmark score is the easiest number in this field to quote dishonestly, so: exactly how it is produced.

  • the setTen DeepResearch Bench II tasks. The question, the rubric and all 672 binary items were written by the benchmark's authors; an item is satisfied or it is not.docs/GOLD-EXT-CLEAN.md — the ten, per task
  • the methodrsk eval ext under gold-ext/v3: one bench.result row per rubric item — the item text, its dimension, the benchmark's score, our credit, and the judge's own fact_present / as_asked with reason and evidence. 672 rows, countable and joinable; a partial set is refused by name, never averaged.docs/JUDGE-1-NOTES.md · the partial-row gate, repair round 1
  • the judgeGemini-3.7-flash through OpenRouter, under our own prompts, one seat for every run on both sides of any comparison we draw.docs/GOLD-EXT-CLEAN.md — “How to read it against a leaderboard”
  • the conditionNo run counted read the source its own rubric came from — BLOCK-1 at the fetch lane, and rsk eval ext separately refusing to score a run whose body store holds one. The guard and its auditor are not the same code.docs/GOLD-EXT-CLEAN.md · issue #1355
  • the ceiling52 of the 500 information-recall items exist only inside the source each task forbids, so a compliant run's ceiling on recall is about 66.7 of the 74.4 points it is worth, not 74.4.docs/GOLD-EXT-CLEAN.md — the last section

and why the leaderboard is a direction, not a scoreboard

The published table (arXiv:2601.08536v3) gives 45.40 overall for the best system, and that row comes from a different judge: its evaluator was Gemini-2.5-Pro, later GPT-5.5, against our Gemini-3.7-flash under prompts we wrote. Comparable in shape, not in calibration — so 45.40 is a direction of travel. Our own before-and-after comparisons hold, because both sides are scored by the same seat.

It has already bitten us in our own favour: 10.67 was published as the tool's score and was our own judge misreading its own rubric — 212 of 366 refused recall items refused for the report's shape, the fact itself quoted in the evidence field. Naming each item's dimension moved the number to 16.58 with no new research. A benchmark number measures the judge as much as the tool, which is why the seat, the method version and the date are attached to it here.docs/GOLD-EXT-CLEAN.md · docs/JUDGE-1-NOTES.md · docs/GOLD-EXT-V2.md · the MVP entry that took 10.67 back

next: the POC — what all of this measured to, including the check that failed — and the MVP, fifty-three questions later.