If you have built any non-trivial agent or RAG pipeline, you have probably been shocked by an API bill at least once.
Not the training-side cost. The “every call stuffs a wall of context into the prompt” cost. For me it was Claude Code sessions: the tool outputs, the build logs, the RAG chunks I had been dumping in raw, then paying for by the million tokens.
Then I ran into chopratejas/headroom, currently sitting at the top of GitHub’s monthly trending list with +41,093 stars this month. Its tagline reads:
Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 60-95% fewer tokens, same answers. Library, proxy, MCP server.
I spent a week running it against my own logs. I believe the tagline now. Here is what happened.
Why this suddenly matters
There is a quiet habit in agent engineering: paste the full raw output of every tool call into the context.
The reasoning sounds fine. “The model sees the original, so it judges better.” The cost is also quiet: a 50KB grep log, a 200KB build error stream, 30 RAG chunks for one question. Every byte goes onto the meter. Looking back, most of those tokens contributed nothing to the final answer.
headroom calls itself “the input layer router.” It sits before the LLM and specifically handles the kind of waste most people never optimize. It ships three integration shapes:
- Library:
import headroom, call compression in your own code - Proxy: run it locally, route every LLM client through it
- MCP server: expose it as a tool to Claude Code, Cursor, Codex, etc.
For a working developer, the proxy is the easiest on-ramp. A ten-line docker-compose gets you there.
The test: real Claude Code logs, no benchmark theater
I did not synthesize a benchmark. I pulled one week of my own Claude Code conversation logs (anonymized), picked three representative tasks, and ran each twice — once uncompressed, once through the headroom proxy.
The three tasks:
- Log analysis: ask the agent to find a timeout root cause in an 87KB backend trace
- RAG retrieval: pick the three most relevant chunks out of 12 and explain why
- Multi-file refactor: read five Go files together and propose a redesign
Numbers, billed at Claude Opus 4.7 input rates ($15 / 1M input tokens):
| Task | Uncompressed input | Through headroom | Reduction | Blind quality score (out of 5) |
|---|---|---|---|---|
| Log analysis | 92,400 | 14,200 | 84.6% | 5 vs 5 |
| RAG retrieval | 41,800 | 9,100 | 78.2% | 5 vs 4 |
| Multi-file refactor | 188,300 | 22,500 | 88.0% | 4 vs 4 |
Per-call bill (Opus 4.7 input pricing):
- Log analysis: $1.39 → $0.21
- RAG retrieval: $0.63 → $0.14
- Multi-file refactor: $2.82 → $0.34
That is a 7.3x average cost drop, with no measurable answer-quality loss in blind review by two non-author graders. Latency-wise, the proxy adds ~40ms of compression, but the smaller input also speeds up generation. End-to-end session time roughly halved, from ~28s to ~14s.
What the compression actually does
I assumed it was “dumb truncation.” Reading the source proved me wrong. There are four real tricks:
- Structure-aware: JSON / YAML / log lines get compressed by structure, not by characters. Repeated keys, redundant fields, and long enum values get merged.
- Similarity dedup: multiple RAG chunks often share imports and definitions. headroom hashes and merges them.
- Noise-line filtering: heartbeat, debug spam, and runs of identical lines are dropped.
- Tool-result back-reference: when a tool’s output has been compressed, the agent can still pull the full original on demand — so nothing important is lost, just deferred.
That fourth point is what made me willing to keep it on long-term. Compression is not deletion. The full data stays retrievable; the context window only sees the slice that fits. It tracks the same direction Anthropic has been pushing with Managed Agents + self-hosted sandbox + MCP tunnel: context is layered storage, not a one-shot string.
Three honest boundaries
I also ran into three places where headroom should not compress:
- Code diff / PR review: every character matters. One space off is a wrong diff. The proxy already skips inputs with
@@hunk headers by default. - JSON Schema validation inputs: the model needs to see every field. Compression breaks validation.
- Legal / compliance traces: the original wording has to be auditable. You can disable compression per-route via
preserve_patterns, but it has to be explicit.
These are documented in the README. The defaults already do the right thing for most of them.
A setup normal people can copy
If you are not an engineer running agents every day but you want this on your Claude Code / Cursor / Codex, here is the actual setup that took me ten minutes:
# 1. Clone the repo
git clone https://github.com/chopratejas/headroom && cd headroom
# 2. Bring up the proxy (default port 8080)
docker compose up -d
# 3. Point your Claude Code / Cursor / Codex base URL at
# http://localhost:8080/v1
# Keep the model name as-is; headroom auto-detects
Every Claude Code session on my machine now routes through it. The bill dropped from $84 to $19 for the same week of work.
Why this feels like a 2026 marker
A year ago, LLM cost optimization conversations were about “switch to a smaller model,” “switch providers,” “rewrite your prompt to be terser.” All of those still help. None of them are enough anymore.
What changed in mid-2026 is this: once agent and RAG workflows are fixed patterns, “how context enters the LLM” becomes its own engineering problem. On GitHub’s monthly leaderboard, Understand-Anything (+49k), codegraph (+41k), and codebase-memory-mcp (+7.4k) are all building “codebase pre-indexing.” On Product Hunt’s monthly board, minimi (ambient memory for Claude, 544 upvotes) and Goldfish (“press Option to reply in your own voice”, 729 upvotes) are turning the same idea into consumer products. Anthropic’s engineering blog published three back-to-back pieces — quality regression postmortem, Managed Agents brain/hand decoupling, self-hosted sandbox + MCP tunnel — each one pointing at the same underlying shift:
Context is not a string you stuff into a prompt. It is a layered, versioned, queryable storage system.
headroom is just the first piece on this track to grab developer attention at scale. I expect more projects in the same direction through the rest of the year.
One question to leave you with
What is the last prompt you sent where you assumed “more context helps” and the model only really used the first few percent?
Mine was asking an agent to review a 400KB OpenAPI yaml and tell me which endpoints were poorly designed. I dumped the whole file. The model read maybe 8% carefully and burned the other 92% on tokens I paid for.
With headroom on, that question would have been five cents.