Why Long AI Coding Sessions Get Expensive (Quadratic Attention Explained)

Type "rename this variable" as the 8th message of a Claude Code session. Now type it again as the 80th. Same words, same answer, and with my sample numbers the second one costs about three times as much. Turn prompt caching off and it's six. Nothing's broken. A long session walks into a quadratic curve twice: once inside the model, where every token looks at every earlier token, and once on your bill, where every turn ships the whole conversation again. Caching makes the repeat cheaper. It doesn't change the shape. The model has no memory Claude Code's docs put it bluntly: "The model doesn't remember anything between requests, so Claude Code re-sends the full context." And one message from you is rarely one request. Every tool call sends its result back as a new request carrying everything before it. Everyone checks everyone, so the work is n² Inside the model, each token gets a query (what it's looking for), a key (what it offers) and a value (what it hands over). A token compares its query with every key, softmax turns the scores into weights, and the output is the weighted mix of values: Attention(Q, K, V) = softmax(Q·Kᵀ / √d_k) · V Q·Kᵀ is every query against every key: n tokens, an n × n grid. Language models only look backwards, which trims the grid to a triangle of about n²/2 cells. That's still quadratic. Double the tokens, four times the work. The full version has a live grid you can type into and double. Read more and try it here. FlashAttention fixed the memory side by never storing the whole grid. It still computes every cell, so the time stays quadratic. The KV cache helps, but not how you'd think Replies come out one token at a time. The KV cache keeps the keys and values of old tokens in GPU memory so they aren't recomputed. Plenty of explainers say this makes generation linear. It doesn't. Each new token still does one dot product per earlier token, so a whole reply is still about n²/2. What the cache saves is the rerun, which would push the total toward cubic. None of this shows up on your bill directly. Anthropic says a 900k-token request "is billed at the same per-token rate as a 9k-token request." So what makes the bill quadratic? Your bill is quadratic for a dumber reason Say a session starts with S tokens (system prompt, tools, CLAUDE.md) and each turn adds about a. Turn k sends roughly S + k·a, so K turns add up to about S·K + a·K²/2. The same triangle again: every turn is a row, and every row re-sends everything above it. Prompt caching lowers the slope, not the shape If the start of a request exactly matches something processed recently, the API skips the recompute and bills that part much cheaper. Anthropic doesn't say what it stores. Open-source servers like vLLM cache the KV blocks of earlier requests, so my guess is it's the same idea. On Claude Sonnet 5.5, per million tokens: $2 input, $2.50 to write the 5-minute cache, $0.10 to read it, $10 output (Anthropic's pricing page, October 2026). A 20× discount on reads sounds like it solves everything, but the cached part is exactly the part that grows. You still re-read the whole history every turn, just cheaper, and a fraction of a quadratic is still a quadratic. My sample session: Sonnet 5.5, a 20k-token start, 3k tokens added and 4 API requests per message. Message 8 costs about $0.05 and message 80 about $0.16. The whole 80 messages come to about $8.10, or about $112 with no caching. Cache reads are around two-thirds of the cached bill. That's the re-sending, on sale. Watch the timer, too. Claude Code's docs say the cache lasts an hour on a subscription but five minutes on an API key or once you're on usage credits. Grab a coffee on an API key and your next message rewrites the whole context at the write price. The full post has a simulator with sliders for all of this. Try it here. What actually moves the curve /clear resets the triangle. The system prompt and tools stay cached. Anthropic's docs: "/clear costs nothing." /compact is a big request itself. It reads the whole conversation to write the summary, so it saves about what /clear does but pays for one large read first. Long breaks hurt most late in a session. A 20-minute break before message 60 on the 5-minute cache takes that one message from about $0.13 to about $0.72. Switching models throws the cache away. So does connecting an MCP server whose tools load up front. Subagents get their own context. Their noisy output stays out of your main conversation, so it isn't re-sent on every later message. What this can't tell you It's a model with sample values, not your bill. Run /usage in Claude Code for your real numbers. On Pro or Max you hit usage limits rather than a dollar figure, but the same curve decides how fast. The attention maths is textbook, not Claude. Anthropic doesn't publish its architecture, so tricks like grouped KV heads change the constants, though not the n² shape of standard attention. What I'd actually do Treat a session like a branch, not a home. One task per session, /clear when the task changes. Keep CLAUDE.md short, since it rides along on every request. Pick the model at the start and leave it alone. On an API key, if you wander off mid-task, set promptCacheTtl to 1h or /compact before you go. None of that changes the shape. It just keeps you on the cheap end of it. Read more Your AI coding session gets pricier with every message, full version with live demos More posts: smiit.in/writing How long do your sessions usually run before you /clear? Tell me in the comments.