I think people underestimate the effect of context length and tool calls on token usage.
Imagine a current context length of 15,000 tokens. The model then researches your codebase by doing multiple tool calls like terminal execution or MCP. Let's say it needs 10 tool calls.
Not even considering the increase in context due to the calls themselves, this would be 150,000 input tokens (mostly cached).
Now let's say the same code research happens later in the session at a context length of maybe 150,000 tokens.
Boom, 1,500,000 input tokens (mostly cached).
Depending on how many tool calls the model needs in what stage of the session, token usage can differ greatly. And imho this has a huge impact on how fast your token quota drains in
@ChatGPT or
@claudeai or whatever other provider you're using.
Each sequential model → tool → model round trip generally causes another model call over the increased context. Multiple parallel tool calls could share a single round trip.
And the output of tool calls (depending on the implementation etc.) normally reaches the next turn's input again.
I guess there's a lot of room for optimization here. Maybe someone has a clever idea.
Cheers.