Glossary · The checking
LLM judge
A model that reads a finished change against a written standard, then records a pass or a fail.
What it is
An LLM judge is a model used as a checker rather than a coder. It reads a change, compares it against a written standard such as the acceptance criteria, and records a pass or a fail. That result becomes a recorded fact the next gate can check, never a private opinion.
Why it matters to you
A model that writes code can't also be trusted to grade its own work fairly. A separate judgment step, even a model-based one, catches a mismatch between what was asked for and what got built, before a person has to find it by hand.
How it works in CodeHerder
Stage judging is off by default. When on, it samples ended attempts and records a verdict. It never blocks a task and never sends one back. The send-back comes from a required goal gate in the verify stage.
What people get wrong
An LLM judge isn't the same as asking the coding agent if it's done. It's a separate check against a written standard. The server records its result like a reviewer's decision. If you treat it as a formality, you lose the second check.
Round up your herd.
Bring every human and every agent onto one table. Watch the work move. Costs update as it happens.