Glossary · The checking
LLM judge
A model that reads a finished change against a written standard, then records a pass or a fail.
What it is
An LLM judge is a model used as a checker rather than a coder. It reads a change, compares it against a written standard such as the acceptance criteria, and records a pass or a fail. That result becomes a recorded fact the next gate can check, never a private opinion.
Why it matters to you
A model that writes code can't also be trusted to grade its own work fairly. A separate judgment step, even a model-based one, catches a mismatch between what was asked for and what got built, before a person has to find it by hand.
How it works in CodeHerder
CodeHerder's built-in review stage requires an LLM-judge verification to pass before a task can move to merge. A fail routes the task straight back to the stage that needs fixing. A pass doesn't carry forward from an earlier attempt: a task sent back for rework needs a fresh pass on its next attempt.
What people get wrong
An LLM judge isn't the same as asking the coding agent whether it's done. It runs as its own separate check, against a written standard, and its result is recorded and gated the same way a human reviewer's decision would be. Treating it as a formality defeats the point of having a second, independent check at all.
Keep going
Related terms
Quality gate
A server-enforced checkpoint a task must clear before it can move to the next stage.
Code review
A dedicated check of a change against the task, run by an agent or a person, before it can merge.
Acceptance criteria
A written checklist that defines what a task must deliver, checked before work begins and before it can ship.
Round up your herd.
Bring every human and every agent onto one table. Watch the work move. Costs update as it happens.