We filed the same vague brief 84 times
Every few days, someone on our team files the same task. The first line has not changed since July. This is the brief as we file it today:
Pick one thing about the marketing website to improve, it could be UI, UX, SEO, Chat Bot optimisation, anything, don’t restrict yourself to this code base for ideas either, feel free to research things and bring back good ideas.
Remember we are a agentic coding automation company with a focus on autonomous agents delivering software cost effectively, our audience is developers but also non technical executives or product people who might have ideas but not the technical skills to ship… codeherder unlocks them!
Make the marketing site awesome.
Don’t add chat bots or AI features to the website, that’s tacky, content is king! Use the /humanizer skill to make website copy less AI and more natural sounding.
Always use simplified technical english (ASD-STE100)
The July version stopped at “Make the marketing site awesome.” We added the last two paragraphs later. One asks for no chat bots. The other sets the house writing style.
The brief names no page, no metric and no deadline. Nobody says what “awesome” means.
We keep filing it on purpose. It is the kind of brief a product person writes when they have an idea but not a spec. As of 29 September 2026, the herd has finished it 84 times. Each run ended in a merged change. This post shows what happened, what it cost, and what it means if you cannot code.
What the herd does with a vague brief
A run passes through five stages. A different agent can work each one.
- Plan. An agent reads the site and picks one change. It writes a plan and a list of checks that must pass. A human can read this plan before any code exists, if they choose to.
- Code. An agent builds the change and opens a merge request.
- Review. A second agent reads the merge request and can send it back.
- Merge. The change lands on the main branch.
- Verify. An agent checks that the change works on the live site.
The plan stage does the most to make a vague brief safe. “Make it awesome” becomes one sentence such as “make the security page say only what is true”. You can agree or disagree with that sentence in a minute. You cannot do that with a pull request full of code.
What it shipped
Nobody chose these changes in advance. The plan stage picked each one. Some of the results:
- A page for leaders that answers the questions of a non-technical buyer.
- A security page rewritten so each claim matches what the product does.
- A page on going from prototype to production.
- A page that lists integrations.
- A self-hosting guide.
- A brief checker that tells you if your brief is too vague.
- A site search that suggests pages when a query finds nothing.
- Header menus that fit on a laptop screen.
- A plain-text summary of the site for other tools to read.
Every one of the 84 runs ended in a merge, and every outcome reads “succeeded”.
What it cost
We pulled the cost and time of every finished run. Two runs report no spend, so we left them out of the cost figures. That leaves 82 runs. Time figures cover all 84.
| Measure | Value |
|---|---|
| Finished runs | 84 |
| Runs with spend data | 82 |
| Total spend | $1,766.53 |
| Median cost per run | $17.51 |
| Cheapest run | $4.67 |
| Most expensive run | $84.82 |
| 10th percentile cost | $6.77 |
| 90th percentile cost | $36.29 |
| Median time, filed to done | 90 minutes |
| Runs done in under two hours | 47 of 84 |
| Runs with zero loopbacks | 44 of 84 |
The time includes waiting in the queue and waiting for a human. A loopback means a review or a check sent the work back to an earlier stage. The most any run had was five.
The cost is not spread evenly. We can only split cost by stage for the most recent runs, because the cost window covers 30 days. In those 13 runs, code took about 56% of spend. Plan and review took about 16% each. Merge took about 9% and verify about 4%.
Verify is the cheapest stage. It checks the change on the live site after the merge.
One run, from start to finish
Take the run that built the page for leaders. It took one hour and nine minutes and cost $17.90. It had eight stage transitions and one loopback.
| Stage | Cost |
|---|---|
| Plan | $1.12 |
| Code | $13.20 |
| Review | $1.53 |
| Merge | $1.46 |
| Verify | $0.59 |
The plan cost about a dollar. A human could read it before the $13 of coding started. If the plan had been wrong, that dollar was the price of finding out.
It got more expensive
Early runs were cheap. The first 20 runs with spend had a median cost of $7.54 and a median time under 40 minutes. The most recent 20 had a median cost of $17.90 and took about two hours.
We see two reasons. The easy wins ran out. An early plan could say “the hero text is vague” and fix it in one file. A later plan has to find something new on a site that already has about 60 pages.
The later plans also measure more before they choose. They check what the site already says, what the build gates allow, and what an earlier run changed. That work costs money. We think it is worth it, because it stops the herd from redoing old work. But we did not plan for the price to rise, and we cannot prove the extra spend always pays off.
What this means if you cannot code
A vague brief is fine when a plan sits between the brief and the code. The plan turns your idea into something you can check. Your job is to read it.
You still need to write a brief that has a clear end point. Our brief worked because the site gives the herd a lot to measure. A brief for a new product needs more detail. Read how to write a brief to see what to add, and how to ship without an engineer for the full path.
If you want to guess what your own idea would cost, use the cost calculator. Then paste your idea into the brief checker and see what it asks you to add.
