Anatomy of an $8 feature: the itemised receipt for code no human wrote
In 10 things we learned from 100 billion tokens we mentioned that the median story CodeHerder ships costs around $7.50 and takes under an hour. That number drew more questions than anything else in the post. So here is the itemised receipt.
This one is a single week rather than the whole history: 20 to 26 July 2026, 944 shipped stories, 312,083 API calls. A week is long enough for the cohorts below to have real sample sizes and short enough that the model configuration didn’t change underneath it, which matters more than it sounds. Cost per story is sensitive to which model each stage runs on, so a receipt averaged over two months of changing configuration describes a machine that no longer exists.
The important result isn’t that a story costs $8. It’s that a staged workflow makes the cost, the delay and the failure point addressable to a specific stage.
The unit, and the machine
CodeHerder is an orchestrator for AI coding agents. You file a story. It walks that story through a fixed sequence of stages:
todo → plan → code → review → merge → verify → doneEach stage is handed to a fresh agent, Claude Code or Codex, spawned on one of your own machines in an isolated git worktree. The agent does one job and exits. Any stage can reject work back to an earlier one. Because each stage is a recorded object rather than a prompt convention, every dollar and every minute belongs to a named stage. That’s what makes a receipt possible.
Throughout, “story” means one user story: the unit of work that gets filed, planned and shipped. “Done” is the terminal state, reached only after post-merge verification passes. Dollar figures are notional list price, because we run on flat-rate subscriptions, so these are what the tokens would cost at published API rates rather than a bill we paid. Currency to two decimals, time to one, percentages to one.
Summary
| window | 20–26 July 2026 |
| stories analysed | 944 |
| shipped with no rework | 87.6% |
| median clean-run cost | $7.57 |
| median clean-run lead time | 42.5 min |
| largest cost stage | plan (38.7%) |
| post-merge rejection rate | 3.4% |
One cohort, two averages
87.6% of shipped stories, 827 of 944, clear every gate without being sent back once. That’s the run worth itemising, because it’s what normally happens.
Two figures for that same cohort turn up throughout this post, and they aren’t in conflict. The median clean run is $7.57, so half of clean runs cost less than that. The mean is $9.38, pulled upward by the expensive tail.
The gap between them is the tail, and it’s why the tables below need care. Medians don’t add. Stacking the median of each stage gives $6.87 for a story whose median total is $7.57, and the missing 70 cents is real. So instead of stacking, each table describes a cohort of real stories near the median: stories whose actual total lands close to the median total. The rows sum because they describe things that happened. Where a table reports a mean, it says so.
Where the $8 goes
Stories within $0.50 of the median total, with all five stages attributed. n=97, figures are means within that cohort.
| stage | cost | share | what happens |
|---|---|---|---|
| plan | $2.91 | 38.7% | read the codebase, write an implementation plan |
| code | $1.97 | 26.2% | implement in an isolated worktree, open the merge request |
| review | $1.11 | 14.7% | a different agent reviews the diff |
| merge | $0.50 | 6.6% | run the repo’s own host CLI to land it |
| verify | $1.04 | 13.8% | confirm the merged result actually works |
| total | $7.53 | 100% |
So in our default configuration, planning costs considerably more than implementation. Plan is 48% more expensive than the code stage that consumes its output, and it’s the largest line on the receipt by a clear margin.
Two causes, and one of them is a choice we made. Plan, review and verify run 100% of their calls on the reasoning tier this week, while code runs 93.9% and merge 99.6% on the faster one, because each stage declares its own model tier. Part of this gap is therefore a configuration decision, visible in the data rather than hidden in a prompt.
The rest is workload: reading enough of a large codebase to plan a correct change appears to cost more than emitting the resulting diff. A common assumption about LLM coding cost is that generation is the expensive part. In our data it isn’t.
If you’re budgeting agents, budget for comprehension, and check which model your planning stage is using before you conclude your bill is a code-generation problem. The caveat is that we can’t separate the tier effect from the workload effect without running the same stories on a flat model assignment, and we haven’t done that.
Where the time goes
The five working stages of a typical clean story, as medians. n=799 stories with all five stages recorded.
| stage | wall clock | median API calls |
|---|---|---|
| plan | 10.3 min | 35 |
| code | 11.8 min | 74 |
| review | 3.6 min | 24 |
| merge | 4.6 min | 24 |
| verify | 3.8 min | 23 |
| total | 34.1 min |
Plan and code together account for most of the 34.1 minutes; the three gate stages combined add barely more time than code alone.
Two definitions, because this table is uninterpretable without them. An API call is one billable request to the model, along with the tool calls it makes. A single model response often arrives as several pieces, some thinking, some text, some tool invocations, and all of those pieces belong to one call and count once. Synthetic messages the harness injects don’t count either.
So 74 calls at the code stage means the coding agent went round its read-edit-test loop 74 times. Wall clock is how long the story sat in that stage, measured between consecutive status transitions, which is what someone waiting on the story experiences.
These are medians rather than means, deliberately. A stage’s dwell includes any time the story spent queuing for machine capacity or waiting on a CI pipeline, so means run several times the medians and a stack of means would overshoot the lead time it’s meant to explain. The medians total 34.1 minutes against a 42.5-minute median lead time; the remainder is the gaps between stages, where a story sits between transitions. Read 34.1 as “the five working stages of a typical story” and 42.5 as “filed to done.”
The two columns rank the stages differently, which is the interesting part. Review and verify look nearly free at under 4 minutes, and they’re cheap in calls too, at 23–24 each. Merge reads like a 5-minute formality and takes 24 calls, about the same as review or verify: conflict resolution, CI waits and pipeline retries all land at merge, and a stage that waits on a pipeline can re-enter several times.
Review approved it. Merge accepted it. It was still broken.
Verify runs after the code is on the main branch. That ordering is deliberate, and it’s the most important thing on this receipt.
| gate | question it asks | resolutions | sent back | reject rate |
|---|---|---|---|---|
| review | is this diff correct? | 1,112 | 139 | 12.5% |
| merge | can this actually land? | 982 | 0 | 0.0% |
| verify | does it work now that it’s merged? | 959 | 33 | 3.4% |
Review catches by far the most of the three gates. Merge is the one stage that rejected nothing this week; verify, running after the code has already landed on main, still catches 3.4%.
Counts here are stage transitions across all story tasks in the window, not distinct stories. One story can be sent back twice and appear twice. The cohort tables further down count stories instead, which is why their numbers are smaller.
Merge is worth reading carefully. It sent nothing back to code this week, which is what you’d hope for from a stage whose job is mechanical. What it does instead, when it can’t proceed, is stop: a protected branch, an external approval, a permission a human has to grant. That isn’t a quality gate firing, it’s the workflow refusing to guess, and those stories ship once the human grants the thing only a human can grant.
Those 33 post-merge send-backs came from stories that had already passed review and landed cleanly. Post-merge verification catches what a diff review structurally can’t: a change that was correct against the tree it was written against and wrong against everything that merged in the meantime, or an incomplete implementation, or an environment problem.
What’s true regardless of the mix is that some failures only become observable against the merged state, which diff review alone can’t reliably reproduce. It’s the same reason human teams run CI on main rather than treating a green MR pipeline as proof. If your agents open merge requests, something has to check the merged result: pre-merge review is not a substitute, however good it is.
What a catch costs
Stories are grouped by which gate sent them back, so each story appears once. These are means, because the tail is the point.
| outcome | stories | share | mean cost | mean code stage | median lead |
|---|---|---|---|---|---|
| clean run, no gate fired | 827 | 87.6% | $9.38 | $2.88 | 42.5 min |
| review sent it back (pre-merge) | 93 | 9.9% | $28.43 | $13.89 | 194.2 min |
| verify sent it back (post-merge) | 16 | 1.7% | $26.78 | $11.39 | 321.2 min |
| both gates sent it back | 8 | 0.8% | $54.40 | $24.74 | 347.9 min |
A single send-back increases mean story cost by roughly 3×, and mean code-stage cost by roughly 4.8×, from $2.88 to $13.89. Median lead time goes from 43 minutes to 194 minutes.
Note which column moves. Planning spend stays roughly flat whether a story is rejected or not; the cost lands on the code stage, because rejection means re-entering implementation with a fresh context.
What review buys
It’s the obvious question to ask of a gate that costs money on every story and fires on one in eight. Review costs $1.11 on a median story, a fixed 15% premium. What does it return?
The clearest answer is calendar time. Stories review catches reach done in 194.2 minutes; stories that get past review and are caught after the merge take 321.2 minutes, 127 minutes longer per defect, on top of a window where a known-bad change is sitting on the main branch. Catching it before the merge is what keeps that window closed.
The second answer is coverage. Review and verify ask different questions, and each catches things the other structurally can’t. Verify checks that the merged result works; it can’t tell you the diff was badly judged, carried the wrong abstraction, or solved the wrong problem correctly. Some fraction of what review catches would never have surfaced downstream at all: it would have shipped.
This dataset can’t support a break-even ratio. The comparison you’d need is our workflow run with review switched off, and we’ve never run it that way. Review is insurance: a fixed, small, known premium against a variable cost this dataset can’t price. We keep it on, and the 127-minute figure is the part we can actually measure.
The shape of the distribution
Clean-run shipped stories: p10 $4.76, p50 $7.57, p75 $10.83, p90 $16.40.
p90 sits at 2.2× the median. We don’t have a comparable public dataset to say whether that’s tight for agentic workloads, so treat it as a description rather than a boast: under a bounded workflow, this workload was considerably more predictable than we expected, with no long tail of stories quietly spiralling into three figures.
The reason is that rework is capped. A stage that keeps getting rejected escalates to a stronger model and then hard-stops at a configured limit, so a story can’t bounce between code and review indefinitely. Read the tight distribution as a property of that cap as much as of the models.
Four things the receipt argues for
Spend expensive models on judgment, not mechanics
Plan, review and verify run on the reasoning tier; code and merge run almost entirely on the faster one. That single split is most of the shape of the cost table, and it’s the lever with the most leverage: the same story costs materially different amounts depending on which reasoning model the judgment stages use.
Isolate implementation environments
Every code stage gets its own git worktree on a real machine, so parallel stories can’t collide. Isolation is also what makes per-stage attribution meaningful in the first place.
Make rejection a measured state transition
Because a send-back is recorded rather than implied, reject loops can be counted, priced and capped. Of the stories review sent back this week, 81.3% needed exactly one round and 93.5% no more than two.
Keep the orchestrator inside existing repository permissions
The merge stage runs the repo’s own host CLI, gh or glab, as an agent stage, with the repo’s
branch protection intact. CodeHerder never becomes a git-host client. That’s why merge appears on
this receipt as a normal $0.50 stage, and why a story can correctly stop and wait for a human.
The point of the receipt
A story costing $7.57 is a fact about our codebase and our model configuration in one particular week. Yours will differ, and so will ours next month. What transfers is the structure that made the figure knowable: every retry, dollar and delay attributable to a named stage.
That’s the difference between agentic coding you supervise and agentic coding you can operate. Not that the agents work unattended, since plenty of tools will do that, but that the work they do unattended is measurable, attributable and controllable after the fact.
Method notes
The dataset is 944 shipped stories from 20 to 26 July 2026. Dollar figures are notional list price on flat-rate subscriptions: an estimated equivalent usage cost at published per-model API rates, not an invoice. Each call is priced at its own model’s rate, with uncached input, cache writes, cache reads and output priced separately, and cache writes priced according to how long the cache entry lives, which is tracked per call.
The billing unit is one API call, deduplicated. The harness writes one transcript line per content block, and every line from a single model response repeats that response’s usage figures, so counting lines rather than calls multiplies both tokens and dollars. Calls here are collapsed back to one row per model response before anything is summed.
Cost is attributed to the stage a session was activated for, not the task’s live status when the call happened. Those differ whenever a call lands after the task has moved on, and grouping by live status files planning work under coding. If you’re building analytics over agent runs, this is the trap.
96.4% of calls resolve to exactly one stage this way; the rest belong to sessions that span more than one stage, which is why the cost and time cohorts both require all five stages attributed rather than working from every story.
Wall clock is the gap between consecutive status transitions, capped at 48 hours to drop overnight queue waits, and reported as medians because means are inflated by queueing. Agent-busy time is a separate measurement, and because sessions overlap and re-enter it isn’t a simple sum, so it isn’t reported here.
The gate resolution counts are transitions out of each gate across all story tasks in the window, so they don’t equal the 827 clean shipped stories. They’re lower at later stages because stories drop out along the way, and they exceed the per-story counts wherever a story passed the same gate twice.
All of this is one herd’s production history, weighted toward the kind of software we build, not an industry benchmark.
