Does /prompt actually save tokens?
5 already-merged bugs, rolled back to the commit right before each fix. Same model, same empty-of-docs repo, twice: once with a plain bug report, once with the prompt /prompt generated. The only variable was the briefing.
Promise-based HTTP client for the browser and Node.js — one of the most widely used packages in the JS ecosystem.
Tokens to fix the bug
-63%Turns to fix the bug
-52%Cost to fix the bug
-53%Summed across all 5 tasks. Execution cost only — generating the prompts is separate and one-time per task.
Setup.
Exactly what ran, in order.
5 already-merged pull requests from axios/axios were picked. For each one, the repo was checked out at the commit right before the fix — the bug is genuinely still there.
docs/, examples/, README.md, AGENTS.md and CLAUDE.md were removed from the working copy for both runs — /prompt exists for projects that don't already have great docs, so that's the scenario we tested.
Both runs used Claude Sonnet 5 through the real Claude Code CLI, with full file-edit permissions. The only thing that changed between runs was the briefing it received.
The "raw task" run got the bug described in plain language, with the whole repo to explore. The "/prompt" run got the prompt /prompt generated from a .aether/docs knowledge base — pointing at the exact files, conventions and acceptance criteria.
The files each run actually touched were checked against the files the real pull request changed — not a guess, a direct diff.
5 bugs, task by task.
Both runs edited the correct file every time. The gap is how much it cost to get there.
Tokens spent per task, raw vs. /prompt
Log scale — hover a row for exact figures
| Bug | File(s) | Tokens | Turns | Cost | Δ |
|---|---|---|---|---|---|
| AxiosError doesn't use AggregateError details #11059 · axios/axios | lib/core/AxiosError.js | 1.34M 315K | 29 7 | $0.79 $0.25 | -76% |
| NO_PROXY doesn't special-case a bare `*` entry in a list #11053 · axios/axios | lib/helpers/shouldBypassProxy.js | 413K 280K | 9 6 | $0.32 $0.25 | -32% |
| data: URL size estimate wrong in the maxContentLength guard #11061 · axios/axios | lib/helpers/estimateDataURLDecodedBytes.jslib/adapters/http.js | 4.23M 1.09M | 56 24 | $2.46 $0.85 | -74% |
| toJSONObject doesn't serialize Set values #11044 · axios/axios | lib/utils.js | 986K 955K | 23 22 | $0.56 $0.50 | -3% |
| A synchronous interceptor error doesn't reject the request #11071 · axios/axios | lib/core/Axios.js | 1.43M 486K | 26 9 | $0.93 $0.52 | -66% |
What was actually broken, per PR:
When the underlying error is an AggregateError (e.g. multiple connection attempts failing), AxiosError.from() used error.message directly — which is empty in that case — instead of synthesizing a message from error.errors[].
NO_PROXY="*" (the whole string) was already handled as "bypass everything" — but a bare * as one entry inside a multi-entry list (e.g. "localhost,*,.example.org") fell through to normal host/port matching and never bypassed the proxy.
The guard used the final decoded byte length, but it needs to reflect the size of the intermediate buffer allocation Node's base64 decoder actually makes — which can be larger than the final result, letting oversized payloads slip past the guard.
toJSONObject had no branch for Set — a Set nested in a config object fell into the generic object branch instead of being serialized as an array.
If a request interceptor threw synchronously, Axios called onRejected without checking it existed or using its result, then called dispatchRequest unconditionally — the request went out anyway instead of rejecting.
What this benchmark doesn't show
- —Numbers above are execution only — the cost of running Claude Code to actually fix each bug. Generating the /prompt itself is a small, separate one-time cost per task, not included in these charts.
- —Generating .aether/docs with /genesis is a one-time cost per project, not per task — it wasn't measured here, and it amortizes across every future /prompt you run.
- —Both runs found the correct file 5 out of 5 times. /prompt's advantage in this test is efficiency and cost, not accuracy — raw exploration got there too, it just took more turns to do it.
- —Task 4 (Set serialization) barely moved the needle — /prompt saved 3% here, not the 30–75% seen elsewhere. Not every task benefits equally; simple, single-file bugs leave less room to save.
Try it on your own codebase.
Run /genesis once, then /prompt for any task — no docs required.