Agent-Effectiveness Benchmarks¶
This page publishes results from the agent-effectiveness benchmark
(bench/, issue #930): does an agent connected to this platform answer real
data questions more correctly and more efficiently than the same agent
connected to bare data tools — and does the platform's memory layer let a fact
taught in one session survive into later ones? The benchmark ablates the
platform, not the model: every run holds the model, prompt scaffold, seed
data, and task set constant and varies only the platform configuration.
The two benchmarks measure one knowledge system from two angles, not two independent features. The system has a capture/curation half and a surfacing half, and they are coupled — the first populates what the second delivers:
- Capture / curation — an agent records a fact (
memory), it becomes an insight, andapply_knowledgepromotes it into a sink: a DataHub entity or column description, or a knowledge page. - Surfacing — curated knowledge reaches the agent through cross-enrichment (DataHub metadata is attached to query and describe results automatically) and search (knowledge pages). Enrichment is one of the places knowledge surfaces, alongside knowledge pages — not a standalone feature.
flowchart LR
A[Agent teaches a fact] --> M[memory]
M --> I[insight]
I -->|apply_knowledge| S1[DataHub entity / column description]
I -->|apply_knowledge| S2[knowledge page]
S1 -->|cross-enrichment| AG[Agent, a later session]
S2 -->|search| AG
The two suites test the two halves. S1–S3 (the arm ablation) tests the surfacing half: with curated knowledge already present in the sinks, does the platform reliably deliver it to the agent? S5 (the lifecycle protocols) tests the capture-and-propagation half: can a brand-new fact be captured, promoted, and reach a different session or user? There is no enrichment value without knowledge in the sinks, and no cross-session value without surfacing — the benchmarks are two ends of the same pipe, and the results below read as one coherent story (surfacing proven; capture-and-propagation maturing).
Methodology, arm definitions, and reproduction commands live in
bench/README.md.
The load harness's throughput numbers are a separate concern; see
Tuning and Scaling.
Reading rule: the headline is always arm-vs-arm (or, for S5, works-vs-fails) on a pinned model — the platform's effect. Model identity is disclosed but is never the subject.
How to read this page¶
Every result compares the same model on the same tasks, changing only what the agent is connected to. The shorthand in the tables:
Arms — the platform configuration being measured. Each ablates one layer:
| Arm | What the agent is connected to |
|---|---|
| A0 — raw tools | The underlying data tools directly (trino_*, s3_*), no semantic provider, no search, all cross-enrichment off. Equivalent to wiring the standalone toolkit libraries. |
| A1 — enrichment | A0 plus semantic cross-enrichment: tool results carry DataHub context automatically, but the agent still has no search and no datahub_* tools. Isolates enrichment from discovery. |
| A2 — platform | The shipped semantic-first platform: A1 plus the search tool, the search-first gate, and curated knowledge pages. |
| A3 — lifecycle | A2 plus the memory / apply_knowledge lifecycle. On the single-session S1–S3 tasks A3 has nothing seeded to recall, so it tracks A2; the lifecycle's effect is what the S5 protocols measure. |
Suites — tasks grouped by what they test:
| Suite | Question it asks |
|---|---|
| S1 — discovery | Can the agent find the right table and columns? (straightforward lookups) |
| S2 — analytical accuracy | Can the agent compute exact numeric answers (single-table, join, temporal, cross-tab, top-N)? Four tasks emit SQL graded by execution-result comparison. |
| S3 — knowledge traps | Questions with a plausible-but-wrong answer, where the disambiguating fact lives in business knowledge, not in the raw data. |
| S5 — memory lifecycle | Does a fact taught in one session get captured, recalled, promoted, transferred to a different user, and correctly updated across later, separate sessions? |
Scores:
- Accuracy — correct graded attempts / graded attempts, averaged across the k = 3 repeats. The bracket is the 95% bootstrap confidence interval.
- pass^k / pass^3 — a stricter bar than accuracy: a task counts as passed only if it passed every one of the k = 3 attempts (tau-bench pass^k). "Accuracy" is the per-attempt average; "pass^3" is all-or-nothing per task.
- Median / p90 tool calls — how many tool calls the agent made to reach an answer, read from the platform's own audit log (p90 = 90th percentile). Fewer is better — efficiency is a first-class axis, following BIRD's Valid Efficiency Score.
3/3(trap table) — passed 3 of 3 attempts on that task;0/3— failed all three.
Methodology¶
The four arms are platform config profiles (bench/config/), not code
forks — the config surface is the ablation mechanism, which is itself a product
property worth proving. Each arm turns on exactly one more layer: A0 raw tools,
A1 adds cross-enrichment, A2 adds search and curated knowledge, A3 adds the
memory/apply_knowledge lifecycle. The arms without a discovery tool (A0, A1)
disable the search-first gate, which is not persona-aware; the DataHub arms (A1,
A2, A3) run against a seeded DataHub quickstart.
Three model adapters. The canonical anthropic adapter drives an
in-process agent loop against the Messages API with prompt caching on; it
produces all published and regression numbers. A scripted adapter replays a
deterministic script with no API key for smoke validation. A claude-cli
adapter drives a real Claude Code client (claude -p) for subscription/keyless
runs (#949). Comparability caveat: claude -p reinserts Claude Code's own
system prompt, tool policy, and retries, which shift across Claude Code
releases; within one run the arm-vs-arm delta is internally valid, but
claude-cli numbers are a labeled secondary path and are never mixed with
anthropic numbers in a published table. The manifest records the adapter and,
for claude-cli, the client version.
Grading is deterministic, scoring only the first line after a mandated
FINAL ANSWER: marker: numeric answers compare the first decimal-bearing
number; entity answers must name a correct alias and must not name any of the
task's generated trap answers. SQL-producing S2 tasks are graded BIRD-style — the
grader executes both the candidate and a reference query and compares result
sets as multisets of rows, so two different-but-equivalent queries both pass.
A pinned LLM judge scores only the one thing the deterministic graders
cannot: whether an S3 answer carried the required caveat. The judge's agreement
with human labels is measured over a committed 30-item calibration set and
published with any judged scores.
The audit log is the measurement instrument. The harness mints a session
handle via platform_info, threads it invisibly, and reads efficiency metrics
back from the admin audit API with delivery: sync; a run fails loudly when a
session's audit rows fall outside the harness's client-side accounting.
Attempts that fail at the harness level (connect, adapter, audit read-back)
never grade and are reported separately.
Token basis. Every attempt records its exact token split, including
cache_read_tokens and cache_creation_tokens, so a run's spend is reported
from committed data rather than estimated. This page publishes token counts, not
dollar figures: apply the model's current per-token pricing to the counts in the
Reproducibility section to get a cost. Cache reads bill at roughly a tenth of
fresh input, which is what keeps the runs affordable — prompt caching lets the
overwhelming majority of input arrive as cache reads (see the totals below).
Reproducibility parameters (shared by both suites): dataset seed 930; k = 3 repeats; a fixed bootstrap resampling seed, so identical inputs produce identical confidence intervals. The task set and protocol set are content-hashed and pinned in each run's manifest.
Build caveat. Both suites below were produced on the development platform build
v1.102.0-9-gadfb9d90-dirty, recorded in each manifest. This is a deliberate cost decision: the S1–S3 data is complete, reproducible, and manifest-pinned, so it is reused as-is rather than regenerated for a cleaner tag, and the S5 run was made on the same build for coherence. A future release-tagged run can refresh both suites under the identical pipeline if a tagged publication warrants it.
1. Surfacing half (semantic layer) — S1–S3 arm ablation¶
Status: full phase-2 suites, four arms, k = 3, 87 tasks × 3 = 261 graded attempts per arm, zero harness failures.
Manifest:
| Field | Value |
|---|---|
| Arms | a0 (raw tools), a1 (enrichment), a2 (platform), a3 (lifecycle) |
| Model | claude-sonnet-5 (anthropic adapter, prompt caching on) |
| Repeats | k = 3 (pass^3 reported) |
| Dataset seed | 930 |
| Task-set hash | 7e727f891ee5 |
| Platform | v1.102.0-9-gadfb9d90-dirty @ commit a373ff7f98e3 |
| Semantic layer | DataHub quickstart on OpenSearch |
| Attempts | 261 per arm (all graded; zero harness failures) |
| Efficiency source | platform audit API, delivery: sync |
| Raw results | bench/results/phase2-anthropic-k3/ |
Accuracy by suite¶
Accuracy is over graded attempts; the bracket is the 95% bootstrap CI.
| Suite | a0 (raw tools) | a1 (enrichment) | a2 (platform) | a3 (lifecycle) |
|---|---|---|---|---|
| S1 discovery | 98.0% [94–100] | 98.0% [94–100] | 100.0% [100–100] | 98.0% [94–100] |
| S2 analytical | 100.0% [100–100] | 100.0% [100–100] | 97.8% [95–100] | 97.8% [95–100] |
| S3 knowledge traps | 42.7% [32–55] | 57.3% [45–68] | 98.7% [96–100] | 98.7% [96–100] |
| Overall | 83.1% [79–88] | 87.4% [83–91] | 98.5% [97–100] | 98.1% [96–100] |
| Overall pass^3 | 80% | 86% | 98% | 97% |
S1–S3 is the test of the surfacing half of the knowledge system: the business facts are already present in the sinks (seeded here), and the question is whether the platform reliably delivers them to the agent through its two surfacing channels — cross-enrichment and search. It does. The effect concentrates in S3, exactly where BIRD showed a ~20-point external-knowledge gap with hand-curated evidence; here the platform surfaces that evidence automatically. S3 accuracy climbs 42.7% → 57.3% → 98.7% as the two channels come online, a +56.0-point gain for the full platform over bare tools (95% CI +44 to +67). Crucially, the two channels carry different facts: the cross-enrichment channel alone (a1) recovers about a quarter of the gap (+14.7, CI −1 to +31) — the facts that live in DataHub descriptions — while adding search over knowledge pages (a2) closes nearly all of the rest (the class breakdown below shows exactly which facts each channel carries). S1 and S2 are near ceiling for every arm — a capable model finds tables and computes arithmetic without help — so the platform's value is not in easy lookups but where a fact outside the raw data disambiguates a plausible-but-wrong answer. This validates delivery; it does not, by itself, test whether the platform can acquire that knowledge in the first place — that is S5.
S3 knowledge-trap accuracy by class¶
Each trap is answerable plausibly-but-wrongly without the knowledge layer. The six classes separate what enrichment can carry (a fact in column/dataset metadata) from what requires the curated knowledge pages.
| Trap class | a0 | a1 | a2 | a3 |
|---|---|---|---|---|
| units_cents | 61.9% [48–76] | 88.1% [79–98] | 97.6% [93–100] | 97.6% [93–100] |
| net_revenue | 21.2% [9–36] | 48.5% [30–67] | 100.0% [100–100] | 100.0% [100–100] |
| fiscal_calendar | 0.0% [0–0] | 0.0% [0–0] | 100.0% [100–100] | 100.0% [100–100] |
| freshness_cutoff | 91.7% [75–100] | 100.0% [100–100] | 100.0% [100–100] | 100.0% [100–100] |
| tier_boundary | 0.0% [0–0] | 0.0% [0–0] | 93.3% [80–100] | 100.0% [100–100] |
| deprecated_table | 95.2% [86–100] | 95.2% [86–100] | 100.0% [100–100] | 100.0% [100–100] |
The pattern is legible, and it is a clean read on the two surfacing channels.
units_cents and net_revenue live in DataHub column and dataset
descriptions, so the cross-enrichment channel alone (a1) lifts them
materially (62→88, 21→49). fiscal_calendar and tier_boundary live only in
knowledge pages — invisible to enrichment — so a0 and a1 score 0% and only the
search channel (a2) recovers them. Same knowledge system, two delivery
channels: each trap is defeated exactly when the channel that carries its fact is
switched on. Whichever sink a fact is promoted to, the platform surfaces it.
Efficiency (median tool calls by suite)¶
Fewer is better. Read from the audit log, not self-reported.
| Suite | a0 | a1 | a2 | a3 |
|---|---|---|---|---|
| S1 discovery | 6 | 5 | 8 | 7 |
| S2 analytical | 6 | 6 | 9 | 9 |
| S3 knowledge traps | 16 | 11 | 10 | 11 |
On the knowledge traps, the platform reaches a correct answer in fewer calls (16 → 10) than bare tools reach a mostly-wrong one: without the disambiguating fact, the model flails — issuing exploratory queries and reasoning in circles — whereas the enriched, search-first path retrieves the fact and answers. On the easy suites the platform pays a small friction cost (a few extra calls, largely SEARCH_REQUIRED refusals while the model adopts the search-first workflow), consistent with the platform's value concentrating where knowledge is load-bearing rather than on trivial lookups.
Why a2 ties a3 here (and what it does not mean)¶
On S1–S3, arm a3 (platform + memory lifecycle) tracks arm a2 (platform) because
these tasks exercise only the surfacing half of the knowledge system. Each
task is one fresh session with the knowledge already seeded in the sinks and
nothing new to capture, so the memory/apply_knowledge capture path has
nothing to add to what a2 already surfaces. a2 ties a3 because S1–S3 does not
test the capture-and-propagation half — not because that half is without
value. Its value is inherently cross-session (teach once, answer forever), which
single-session accuracy cannot capture by construction. And it is that half that
produces the sink contents S1–S3 shows are so effective once present — the two
halves are the same pipe. That capture-and-propagation capability is measured
separately, and directly, by the S5 lifecycle protocols below.
2. Capture-and-propagation half (memory / knowledge lifecycle) — S5 lifecycle protocols¶
Status: three full lifecycle runs on the canonical anthropic adapter, arm a3,
k = 3 each — one shared-store run and two isolated replicates. Unlike the
S1–S3 ablation, S5 has no meaningful "memory off" baseline — a recall task with no
memory is trivially 0% — so it does not report an accuracy delta over a baseline.
Instead it measures whether the memory subsystem works reliably: across fresh
sessions, is a taught fact captured, recalled, surfaced, promoted to shared
knowledge, transferred to a different user, and correctly updated?
Three runs are reported for a reason worth stating up front. The shared-store run is a single benchmark process: all 45 attempts run against one knowledge/memory store, so a protocol's three repeats are not independent (an earlier attempt's promotion persists as searchable knowledge). Each isolated run removes that coupling — three independent passes of every protocol at k = 1, with the platform reset to clean seeded state between passes (fresh Postgres, DataHub descriptions and knowledge pages re-seeded), then merged into one k = 3 result. Two isolated replicates were run under identical configuration. The second replicate is what makes the S5 numbers trustworthy: comparing the two isolated runs to each other separates a real effect from run-to-run noise, and it turns out that most of the metric-by-metric differences at this scale are noise. The honest scorecard below is therefore read as a range across runs, not a set of point estimates, and the interpretation is built on what stays stable across all three.
S5 is not single questions but multi-episode protocols, each a sequence of fresh sessions that exercises the teach-once-answer-forever lifecycle and verifies every state transition through the platform's own admin APIs (the insights and changesets endpoints), never inferred from a transcript. Each protocol teaches a novel definition — deliberately absent from the seeded knowledge and catalog — so the intended path to the answer is the taught memory (recall) or the promoted knowledge (transfer), not a pre-seeded fixture (see the limitations on re-derivability below). Ground truth is computed from the dataset, never hand-typed, exactly as the S1–S3 truths. Each attempt consumes two pool identities (a teacher and a learner) so the search-first gate's per-user discovery scope never leaks between attempts.
Manifest (both runs share arm a3, model claude-sonnet-5 with prompt caching,
dataset seed 930, protocol-set hash 0920c55292d1, 15 protocols — 10 promote,
5 supersede — and DataHub-quickstart enrichment with Ollama nomic-embed-text
memory embeddings; every lifecycle state transition is verified through the
admin insights + changesets API, never inferred from a transcript):
| Field | Shared-store | Isolated run 1 | Isolated run 2 |
|---|---|---|---|
| Regime | 45 attempts, one shared store (k-repeats coupled) | 3 independent k = 1 passes merged to k = 3 | 3 independent k = 1 passes merged to k = 3 |
| Platform build | v1.102.0-9-gadfb9d90-dirty | v1.102.0-10-g32d61254-dirty | v1.102.0-10-g32d61254-dirty |
| Transcripts | complete | passes 2–3 lost to a harness bug (metrics complete; see Limitations) | complete (all 3 passes) |
| Raw results | bench/results/s5-anthropic-k3/ |
bench/results/s5-anthropic-k3-isolated/ |
bench/results/s5-anthropic-k3-isolated-v2/ |
The isolated runs used a later development-build string than the shared-store run
because they rebuilt the platform from the current tree; the platform code
(cmd/ pkg/ internal/ go.mod go.sum) is byte-identical between the two build
commits (verified by diff), so the difference is benchmark-harness commits only
and the runs are directly comparable.
Lifecycle scorecard¶
Each metric is a numerator/denominator over the applicable, non-harness-failed runs. Duplicate rate is lower-is-better; every other metric is higher-is-better.
| Metric | Shared-store | Isolated run 1 | Isolated run 2 | What it measures |
|---|---|---|---|---|
| Capture rate | 80.0% (36/45) | 84.4% (38/45) | 82.2% (37/45) | the agent recorded the taught fact and entity-linked it (verified via the insights API) |
| Personal recall | 84.4% (38/45) | 88.9% (40/45) | 84.4% (38/45) | a fresh same-identity session answered the fact-dependent question correctly — graded on answer-correctness like an S1–S3 question, not on proven retrieval (see limitations) |
| Unprompted surface | 100.0% (36/36) | 100.0% (38/38) | 100.0% (37/37) | among captured runs, search surfaced the saved memory itself, unprompted — the cleaner "memory was actually used" signal |
| Transfer rate | 42.3% (11/26) | 43.3% (13/30) | 46.7% (14/30) | a different identity answered correctly after the reviewer promoted the insight to shared knowledge |
| Update correctness | 100.0% (10/10) | 100.0% (8/8) | 100.0% (7/7) | a correction flipped a later recall to the new value |
| Duplicate rate | 10.0% (1/10) | 0.0% (0/8) | 42.9% (3/7) | supersede left more than one live insight (lower is better) |
| Abstention | 100.0% (45/45) | 100.0% (45/45) | 95.6% (43/45) | the agent refused to fabricate a fact it was never taught |
| Full-lifecycle pass^3 | 20.0% (3/15) | 26.7% (4/15) | 20.0% (3/15) | every applicable stage passed all 3 attempts of the protocol |
What is stable across all three runs — read these as the firm results. The
platform surfaced every captured memory unprompted (unprompted surface 100% in
all three) and flipped every detected correction to the new value (update
correctness 100% in all three). Abstention held at 96–100% — the platform did not
fabricate facts it was never taught, the failure mode LongMemEval warns about.
And transfer sits at 42–47% across all three runs: a fact promoted to shared
knowledge is reused by a different identity under half the time. That is the
clearest, most reproducible finding — a genuine reliability ceiling on
cross-identity transfer, and the sharpest lever for improvement. Where that
ceiling comes from matters, and the two-halves framing localizes it: S1–S3 has
already validated the surfacing half — enrichment and search reliably deliver
knowledge that is present in the sinks — so a transfer failure is very unlikely to
be a surfacing failure. It points upstream, at the capture-and-propagation
half: either the apply_knowledge promotion did not land in the aspect
enrichment reads (see the note in Limitations on entity-type coverage), or the
second identity surfaced the fact but did not use it. Transfer is a capture/
propagation problem, not a delivery problem — which is exactly where to look.
What is not stable — most of the rest, and this is itself the finding. The two isolated replicates ran under identical configuration, differing only in the model's run-to-run stochasticity, yet they disagree materially: duplicate rate is 0% in one and 42.9% in the other, pass^3 is 26.7% then 20.0%, and capture and personal recall each move a few points. The apparent "isolation lift" over the shared-store run that isolated run 1 alone suggested (capture 80→84%, recall 84→89%, pass^3 20→27%) is not reproduced by isolated run 2 (82%, 84%, 20%) — so that lift was largely run-to-run variance, not the shared-store confound. At this scale (15 protocols, k = 3, with applicable denominators as small as 7–10 on the supersede metrics) the sampling noise is wide, and no single run's capture/recall/duplicate/pass^3 value should be read as a point estimate. The second replicate earned its place by exposing exactly this: the confound's effect on the scorecard is within the noise between two otherwise-identical runs. The robust reading is the range across runs, anchored on the cross-run-stable metrics above.
Robustness to transient API errors. All three runs recorded zero harness failures and zero episode-level errors (45/45 attempts graded each), and the run logs show no overload/rate-limit/5xx/retry markers. The harness excludes any adapter- or transport-level failure from the metrics and counts it separately, so a Claude API outage would appear as an excluded harness failure, never as a silently mis-graded answer; the agent's budget is counted in tool calls, not wall-clock, so outage latency does not change outcomes. The two isolated replicates ran about an hour apart, so a sustained outage window would likely have struck one and not the other — both were clean.
Limitations¶
Run-to-run variance dominates several metrics. The two isolated replicates are identical in configuration, so their disagreement is a direct read on sampling noise: duplicate rate spans 0–43%, pass^3 spans 20–27%, and capture and recall each move a few points. With 15 protocols and applicable denominators as small as 7–10 on the supersede metrics, this is expected — these are small samples. Treat capture, recall, duplicate rate, and pass^3 as ranges, not point estimates, and weight the cross-run-stable results (unprompted surface, update correctness, and the ~45% transfer ceiling) most. A larger protocol set and more replicates would tighten the noisy metrics; that is the natural next investment.
The shared-store confound is real in mechanism but small in effect. In the shared-store run, promotions from earlier attempts persist as globally searchable knowledge pages, and two consequences are visible in its transcripts:
- Capture failures from discovery-budget exhaustion. In some later attempts
the teacher spent its entire tool-call budget searching for and trying to read
a page a prior attempt had promoted, and hit the 30-call cap before invoking
the capture tool — recorded correctly as a capture miss (the insights API
confirms zero insights for that identity). In
lc-house-tierattempt 2, for example, the teacher's transcript ends "I ran out of tool-call budget before I could actually write the memory record." - Recall can be aided by shared knowledge. A fresh session can reach the right answer via a prior attempt's promoted page rather than its own memory.
The isolated runs remove this coupling by resetting the platform to clean seeded state between passes (fresh Postgres via a volume wipe, DataHub descriptions and knowledge pages re-seeded, verified clean before each pass). But the effect that removal produces is within the variance between the two isolated replicates — so while the mechanism is documented and real, the scorecard cannot claim a confound cost larger than the run-to-run noise. Reporting all three runs, rather than a single shared-store-vs-isolated pair, is what makes that honest.
Personal recall is graded on answer-correctness, exactly like an S1–S3 question, so some taught definitions being partly re-derivable from the data can inflate it; read it alongside unprompted surface, which is the cleaner evidence that the saved memory itself was used.
Transfer is a capture/propagation limit, not a surfacing one. Because S1–S3
validates that enrichment and search deliver knowledge already in the sinks, the
~45% transfer ceiling is best diagnosed upstream: the apply_knowledge promotion
landing in an aspect enrichment reads (mcp-datahub GetEntity fully populates
dataset and dashboard entities but reads other entity types more sparsely, so a
promotion to a different entity type may surface incompletely), and whether the
second identity that did see the fact actually used it. This run does not yet
instrument that split; doing so is the natural next step for closing the gap.
Raw-data note. Isolated run 1's per-attempt records (final answers, tool-call counts, tokens, audit) are complete, but a harness bug pointed all three of its passes at one transcript directory, so only the last pass's turn-by-turn transcripts survived on disk. No metric or cost derives from the lost transcripts. The bug is fixed (each pass now writes its own directory) and isolated run 2 preserves all three passes' transcripts; it is reported alongside run 1 rather than replacing it.
Reading S5 correctly¶
S5 answers a different question than S1–S3. S1–S3 asks "is the platform's answer more correct?" and reports an accuracy delta between arms. S5 asks "does the memory subsystem reliably carry a fact across sessions and users?" and reports whether each stage of that lifecycle works — because the alternative (an agent with no memory) does not fail less accurately, it simply cannot do the task at all. A recall or transfer task run without the memory layer scores 0% by construction, so a baseline comparison would be vacuous; the honest measurement is reliability of the mechanism itself, which is what this scorecard reports.
3. Reproducibility and token spend¶
Both suites are reproducible from committed data and a recorded platform build. Spend is reported as token counts (input / output / cache-read / cache-write); apply the model's current per-token pricing to get a dollar figure.
Semantic layer (S1–S3), reused as-is. From a booted arm:
make bench-up BENCH_ARM=a0 # then a1, a2, a3 (a1/a2/a3 need a seeded DataHub quickstart)
make bench-run BENCH_ARM=a0 LLM=anthropic MODEL=claude-sonnet-5 K=3
make bench-compare # cross-arm tables + bootstrap CIs
Token spend across all four arms (1,044 graded attempts), from the committed per-attempt records: 17,728 fresh input, 1,725,111 output, 80,695,457 cache-read, and 4,555,362 cache-write tokens (≈ 87.0M total). Prompt caching is what keeps this affordable: the arms read ~80.7M cached input tokens against only ~4.6M freshly written, so cache reads dominate the input. The lifecycle arm (a3) is the heaviest single arm because its larger memory/search context is re-sent each turn.
Memory layer (S5). Boot the a3 arm with the metrics port overridden to
avoid the DataHub Kafka :9092 clash, and with ollama serve +
nomic-embed-text available so supersede detection has an embedding provider:
make bench-up BENCH_ARM=a3 BENCH_METRICS_ADDR=:9095
make bench-lifecycle LLM=anthropic MODEL=claude-sonnet-5 K=3
make bench-lifecycle-report RESULTS=build/bench-results/lifecycle-a3-<stamp>/lifecycle-a3.json
Every bench-lifecycle (and bench-supersede) invocation writes into its own
timestamped run directory under build/bench-results/ and refuses to overwrite
an existing results file, so no run's data is ever lost; the report targets
therefore take the run to summarize via RESULTS= (they list the available run
directories when it is unset).
Token spend for the shared-store S5 run, from the committed records: 2,578 fresh input, 356,015 output, 23,913,219 cache-read, and 1,502,311 cache-write tokens (≈ 25.8M total) across the 45 attempts, bounded during the run by a spend watchdog. The local Ollama embedding adds no API tokens.
Each isolated run is three independent single-pass runs with a clean reset between them, merged into one k = 3 result. Every pass writes to its own directory so all raw transcripts are preserved:
D=bench/results/s5-anthropic-k3-isolated-v2
for pass in 1 2 3; do
make bench-down && make e2e-down # wipe Postgres volume
make bench-seed-datahub # reset DataHub descriptions to seed
make bench-up BENCH_ARM=a3 BENCH_METRICS_ADDR=:9095
build/benchrun -lifecycle -arm a3 -url http://localhost:8098 -credential "$KEY" \
-protocols bench/protocols -k 1 -llm anthropic -model claude-sonnet-5 \
-out "$D/pass$pass/lifecycle-a3.json"
done
build/benchrun -lifecycle -merge "$D/pass1/lifecycle-a3.json,$D/pass2/lifecycle-a3.json,$D/pass3/lifecycle-a3.json" \
-out "$D/lifecycle-a3.json"
The two isolated replicates spent, respectively: 2,438 / 296,917 / 21,260,051 / 1,328,127 and 2,438 / 286,587 / 21,315,216 / 1,291,098 tokens (input / output / cache-read / cache-write; ≈ 22.9M total each) across their 45 attempts, also watchdog-bounded.
Regression gate. Any committed run can serve as a baseline for future runs:
make bench-run BENCH_ARM=a2 LLM=anthropic K=3 BASELINE=bench/results/phase2-anthropic-k3/full-a2/results.json
benchrun -baseline <results.json> compares the fresh run to the baseline
suite-by-suite and exits nonzero if any suite regresses beyond the default
thresholds (accuracy or pass^k drop > 5 points, or median tool calls > 1.25×),
so a broken enrichment path or a persona that stops granting search fails CI
loudly. The gate's failure behavior is covered deterministically by unit tests,
and a scripted, no-API-key smoke of the full pipeline (plus a live self-check of
the gate) runs on demand via the workflow_dispatch path of the Bench Harness
CI workflow.
Total published token spend: ≈ 158.6M tokens — 25,182 fresh input, 2,664,630
output, 147,183,943 cache-read, and 8,676,898 cache-write across all runs
(S1–S3 reused as-is, plus the shared-store and two isolated S5 replicates). Cache
reads are ~93% of the total, which is why prompt caching is the load-bearing cost
control; apply current claude-sonnet-5 pricing to these counts for a dollar
figure.
4. Honest caveats¶
- Results are model-dependent; the headline is arm-vs-arm (or, for S5, works-vs-fails) on a single pinned model, never model-vs-model.
- The seed dataset is small by design (fixed seed, airgapped); absolute accuracies are not real-world estimates. What the ablation isolates — the platform's effect, holding everything else constant — is the point.
- All runs used development builds, not a release tag: S1–S3 and the
shared-store S5 run on
v1.102.0-9-gadfb9d90-dirty, the isolated S5 run onv1.102.0-10-g32d61254-dirty(adjacent builds differing only by benchmark-harness commits). The manifests pin each; a tagged run can refresh all under the same pipeline. - CIs are percentile bootstrap over graded attempts with a fixed resampling seed — reproducible, but they do not model task-selection variance.
- S1–S3 tests the surfacing half, not the capture-and-propagation half, so a2 tying a3 there reflects the test scope, not the value of capture; the two are one knowledge system (capture populates the sinks that surfacing delivers), and capture-and-propagation is measured by S5.
5. Result history¶
| Date | Suite | Arms | Model | Attempts | Raw results |
|---|---|---|---|---|---|
| 2026-07-13 | 1 (pilot) | a0, a2 | claude-sonnet-5 | 60 | bench/results/v1.102.0-pilot/ |
| 2026-07-13 | 3 (S5 pilot) | a3 | claude-sonnet-5 | 13 protocols | bench/results/v1.102.0-s5-partial/ |
| 2026-07-14 | Semantic (S1–S3) | a0, a1, a2, a3 | claude-sonnet-5 | 261/arm | bench/results/phase2-anthropic-k3/ |
| 2026-07-14 | Memory (S5, shared-store) | a3 | claude-sonnet-5 | 15 protocols × 3 | bench/results/s5-anthropic-k3/ |
| 2026-07-14 | Memory (S5, isolated run 1) | a3 | claude-sonnet-5 | 15 protocols × 3 passes | bench/results/s5-anthropic-k3-isolated/ |
| 2026-07-14 | Memory (S5, isolated run 2) | a3 | claude-sonnet-5 | 15 protocols × 3 passes | bench/results/s5-anthropic-k3-isolated-v2/ |
Cold-start knowledge growth (upcoming)¶
A cold-start knowledge-growth suite (make bench-cold-start, issue #963) is
implemented and runnable but not yet published here: it starts the a3 arm from
an empty enrichment layer (undocumented DataHub, no knowledge pages) and teaches
a six-lesson curriculum — one fact per S3 trap class, promoted to the same
DataHub descriptions and knowledge pages the A2 seed pre-loads — re-running the
fixed S3 trap suite with a fresh, never-taught evaluator identity after each
promotion. The result is a learning curve of accuracy, per-trap-class trap
resistance, and enrichment coverage as promoted knowledge accumulates toward the
A2 ceiling, directly exercising the coupling between the two halves above (the
lifecycle populates the sinks the surfacing layer delivers from). Its learning
curve will be added to this page once a real model run completes; the loop
itself is validated with no API key by make bench-cold-start-smoke.