Definition
An agent reply can be expensive even when it is short because cost is accumulated across the model invocations and context used to produce it, not determined only by the length of the visible output.
The agent’s final answer is three sentences. Counting those words tells you almost nothing about what it cost to produce them.
The model may need the original task, system instructions, tool definitions, earlier messages, file contents, command output, errors, and every result that made the final answer possible. The reply is short because the work is finished—not because the work was small.
Open the bill
Drag the playhead backward to see how a $0.52 sentence was assembled.
Three sentences. Cheap, right?
The answer is short. Producing it is not.
$0.52 is an illustrative figure on synthetic rates; real provider pricing varies. The 120.4k is the input context of this reply's final model invocation. The reply runs several invocations, so the cumulative tokens read are higher.
You pay per model invocation, not per turn
A turn is a useful conversational unit. Billing happens at a lower level: model invocations. Each invocation has an input and an output. Providers may also distinguish cache writes, cache reads, reasoning tokens, or other categories.
If a model requests three tools one after another, the harness may need four invocations: one before each tool and one to produce the final reply. That pattern is common, not universal. Parallel tool calls, local planning, and harness-specific loops can change the count.
Turns organize the conversation. Invocations carry the token ledger.
Why an agent’s context window grows every turn
Models do not automatically remember a prior request. The harness constructs context for each invocation. In a straightforward, uncompacted session, that context tends to accumulate:
- system instructions and tool definitions
- the user’s task
- earlier assistant output
- tool calls and results
- file contents and command output
- corrections, failures, and verification
In the specimen’s final model invocation, the model operates over roughly 120,000 context tokens to produce 64 output tokens. That 120.4k is the input context of one invocation. The reply runs several invocations, so the cumulative tokens read across them are higher. The visible answer is less than one tenth of one percent of the text involved in that single invocation.
Context does not always grow monotonically. Harnesses can truncate old results, summarize the conversation, reset the window, retrieve only selected evidence, or move work into a child session. Each choice trades completeness, continuity, latency, and cost.
Input tokens vs. output tokens: what each one costs
| Fresh input | Context processed as new input for an invocation. |
|---|---|
| Cache write | A reusable prompt prefix stored under provider-specific rules. |
| Cache read | Previously stored prompt tokens reused at a lower price when the prefix matches. |
| Output | Tokens generated by the model, including tool requests and the visible reply. |
Rates vary by model, provider, and date. The important distinction survives those changes: a short output can sit on top of a large input.
What prompt caching changes (and what it doesn’t)
Prompt caching can make repeated prefixes much cheaper. If the beginning of one request matches a cached prefix from an earlier request, the provider can charge a lower cache-read rate for those tokens.
Caching changes price, not the model’s logical context. The model still conditions its response on the cached tokens. They have not vanished from the request’s meaning; they are accounted for differently.
Caching is also conditional. Providers differ on eligible content, minimum sizes, expiration, write cost, and whether a matching prefix survives changes later in the prompt. A cache-aware estimate should separate fresh input, cache writes, cache reads, and output instead of multiplying one token total by one rate.
Why a short agent reply can cost more than a long one
Early in a session, a model may generate a long explanation over a small context. Late in the same session, it may generate one sentence over a much larger context. Output length shrinks while input grows.
This reverses the intuition people bring from ordinary chat: the shortest visible message can be the most expensive turn.
That is not a universal law. Compaction, a context reset, a cheaper model, or a high cache-hit rate can reverse it. It is a question the invocation ledger can answer for a particular session—not a claim the transcript can settle by appearance.
How to measure agent cost: name the boundary first
“What did this reply cost?” can refer to several boundaries:
- the final invocation alone
- every invocation in the final turn
- the parent session through the reply
- the parent plus any child sessions
- the cached price paid
- the fresh-input counterfactual
Name the boundary and use the matching ledger. Cost is not hiding in the three sentences. It is distributed across the context and invocations that made them possible.
Why cost attribution needs the invocation ledger
Your bill is generated by the execution. The visible answer’s length barely figures into it. When only the output or a provider total is visible, the useful questions get hard to ask:
- cost is difficult to attribute to a turn, a session, or a tree
- repeated context and cache behavior are hard to inspect
- the most expensive invocations are hard to locate
None of this means large context is automatically waste. It means the expensive parts stay attributable and inspectable only when per-invocation records exist. See telemetry for agents.