$ capsul benchmark
Measured, including what the saving cost.
Every number here comes from a run we can reproduce. Intervals sit next to medians, and the rows where an answer check was lost say so on the row rather than in a footnote.
/benchmark · September 2026
Measured with Claude and Codex.
What actually goes over the wire, model by model.
The same five short questions, to the same model, with and without capsul. Bars show the reduction in cache-weighted input. These are separate campaigns; savings vary with the task and cache.
share of the input capsul does not send
saved by capsulstill sent= claude -p
Haiku 5.5
−60 %
- claude -p
- 270 145
- capsul
- 107 326
Haiku 4.5
−69 %
- claude -p
- 298 826
- capsul
- 93 693
Sonnet 5.5
−70 %
- claude -p
- 240 175
- capsul
- 71 726
Sonnet 5
−63 %
- claude -p
- 295 652
- capsul
- 110 166
Sonnet 4.6
−50 %
- claude -p
- 169 531
- capsul
- 85 171
Opus 4.6
−64 %
- claude -p
- 230 791
- capsul
- 82 199
Opus 4.7
−62 %
- claude -p
- 257 855
- capsul
- 96 722
Opus 4.8
−80 %
- claude -p
- 347 589
- capsul
- 71 236
Opus 5
−69 %
- claude -p
- 303 384
- capsul
- 95 151
Opus 5.5
−69 %
- claude -p
- 266 953
- capsul
- 82 584
Fable 5
−66 %
- claude -p
- 264 411
- capsul
- 89 855
Fable 5.1
−69 %
- claude -p
- 277 520
- capsul
- 86 622
Method
Two arms, with the same model imposed on both. The bare arm runs the agent CLI the way anyone runs it; the capsul arm sends the same question through capsul on the same model. Comparing two different models would compare price lists, not tools.
The unit is cache-weighted input: fresh tokens at full weight, cache reads at a tenth. On a subscription the raw token is not what gets metered, and a raw count flatters whichever arm happens to cache better.
Every cell is kept. Nothing is dropped for being inconvenient, and each campaign states how many cells were run and how many were discarded. In the latest Claude campaign that second number is zero: all 360 cells were measured.
Each question is asked three times per arm and the three runs are averaged. The first run writes the prompt cache and the next two read it, so taking the median would keep the warm run two times out of three and flatter every gain. The average is the more conservative figure, and it is the one published.
Intervals are bootstrapped at 90 percent and drawn on the same axis as the gain, with the line at one visible underneath. An interval that crosses that line means the measurement establishes nothing, and it is meant to be seen.
What these numbers do not say
- A median is not a promise. The twelve per-model gains run from x1.99 to x4.88, with x3.06 in the middle, and within one model the gain moves with the question: on Opus 4.8 it runs from x2.44 on one task to x8.64 on another.
- None of the twelve Claude models lose an answer check with capsul, and six gain at least one: Haiku 4.5 gains two; Haiku 5.5, Opus 4.6, Opus 4.8, Opus 5.5 and Fable 5.1 gain one each. A saving bought with a wrong answer is not a saving, so the count is stated here rather than left out.
- The protocol is short, one-line questions, which is where the gain is widest. An earlier run on ten-line prompts, on 2026-09-04 with 126 cells, gave x1.09 to x3.34 with two rows establishing nothing. If you write long prompts, that run describes you and this table does not. On one published task the gain is close to nil: x1.05 on Sonnet 4.6.
- Campaigns are separate. Figures from different dates were measured on different versions of the code, so a row is comparable with its own campaign and not necessarily with another.
- These figures measure input sent to a model. They do not measure how much of a subscription quota that turns into, which depends on weighting we do not control.
The raw data
The most recent campaign is published as JSON, with its protocol and its per-task figures. Anyone who does not take our word for the table can read the file.