Composition outcomes

Auto tasks hand the agent a judgment call: how much process does this piece of work actually need? Composition outcomes is where you find out whether that judgment call is paying off. It compares tasks whose workflow the agent chose for itself against tasks that ran a hand-pinned or default workflow of the same type, so you can see whether success rate, turnaround time, and rework actually differ — and by how much.

Where to find it

There’s no web-app page for this report yet — it’s CLI and API only.

ch quality composition

Pass a workspace name, slug, slug path, or full ID as the first argument to scope it elsewhere; without one, the command uses the workspace the CLI is pointed at. See Using the ch CLI for how ch resolves a workspace reference. Add --json to get the raw data instead of the table.

Any member of the workspace can run this report — there’s no extra role or plan requirement.

Choosing a window

The CLI takes --window today, --window yesterday, --window week, or --window month — the default is month, wider than most other quality reports default to, since composition is a comparatively rare event and a shorter window often won’t have enough tasks to say much.

For a custom range, use --since instead, with a relative duration (3d, 12h, 45m) or a full RFC 3339 timestamp — the same grammar as ch costs --since. --since and --window are mutually exclusive, and --since can’t reach back more than 90 days.

By default the report cascades: it covers the workspace you asked for and every workspace nested beneath it. Add --strict to scope it to exactly that one workspace instead.

What each number means

Every row compares tasks of one type (story, bug, and so on) that finished during the window, broken out by where their workflow came from:

Source	What it means
auto_composed	The task was filed as auto, and the agent chose its own workflow.
human_pin	Someone set a workflow override on this task by hand.
static	The task ran its type’s normal, unmodified workflow — nothing was chosen or pinned.

For each type/source pair, the table shows:

Column	What it tells you
TASKS	How many tasks of that type and source reached a terminal state during the window.
SUCCESS	The share of those tasks that finished at a success terminal — normally done, or a custom workflow’s own declared success point. A cancelled or archived task counts as not successful here.
MEDIAN TIME	The median time from when the task started to when it reached its terminal state.
LOOPBACKS	The mean number of times a task in this cell was sent back to an earlier stage for rework.
VERIF-FAIL	The mean number of verification failures per task in this cell.
MEDIAN COST	The median model spend for a task in this cell.

Note that TIME and COST are medians, while LOOPBACKS and VERIF-FAIL are means — the mix is deliberate, not a typo, so don’t read every figure on the page as “the average.”

Reading the table

A figure with no sample behind it prints —, never 0, so “nothing happened here” never gets mistaken for a measured zero. Any cell built from fewer than 5 tasks is marked (n<5) — treat those as too thin to draw a conclusion from. If the window has no terminal tasks at all, the table area prints (no data).

Sample output, with made-up numbers:

ch quality composition — window Jul 24 00:00 → Aug 23 00:00

  TYPE   SOURCE         TASKS    SUCCESS  MEDIAN TIME  LOOPBACKS  VERIF-FAIL  MEDIAN COST
  story  auto_composed  22       91%      1h48m0s      0.3        0.1         $1.85
  story  human_pin      3 (n<5)  67%      2h30m0s      0.3        0.0         $2.40
  story  static         64       78%      3h12m0s      0.6        0.3         $2.10
  bug    auto_composed  9        100%     52m0s        0.1        0.0         $0.90
  bug    static         31       87%      1h20m0s      0.4        0.1         $1.15

  seal failures: none

  Observational, not a randomised trial: auto is opt-in and chosen for particular tasks, so a gap
  between sources is confounded by that selection. Cells marked (n<5) are too small to read as a verdict.

The seal failures line

Every render prints a short line about “seal failures” — CodeHerder’s own bookkeeping catching up on a handful of recently finished tasks. none is the healthy, normal reading, and it means every task is fully accounted for. If a small number show up instead, CodeHerder retries them on its own; there’s nothing for you to do.

Read this before you act on it

This is the single most important thing to know about this page: a gap between sources is not proof that one approach is better. Auto composition is opt-in — someone chose to file that task as auto, for that particular piece of work — so the tasks in each row weren’t assigned at random. If auto_composed shows a higher success rate than static for the same type, it may be because auto tasks tend to get filed for work that was already going to go smoothly, not because letting the agent choose the workflow caused the improvement. Treat the numbers as a starting point for a question, not a verdict — especially in any row marked (n<5).

What --json adds

--json from the CLI returns the same rows as the table, plus more than it renders: mean escalations and mean advance rejections per cell, median files changed per cell, and a totals cell rolled up across everything. Reach for --json if you want to script against this data or build your own comparison outside the table.

The underlying API response carries two more rollups that the CLI doesn’t currently pass through: one grouped by source alone (across all types), and one grouped by the exact ordered list of stages a composition chose. If you need those, call the API directly rather than the CLI.

Related guides

• Auto tasks — the agent chooses the workflow — file the tasks this report measures, and set guardrails on what the agent may choose
• Review debt — the other side of the review picture: latency, rework, and cost
• Monitoring your agents — first-pass rate and the rest of fleet health
• Understanding costs — what the cost figures on this page actually mean
• Customising a task’s workflow — how a hand-pinned workflow (human_pin) gets set
• Stage signals — the sibling quality report: whether each stage’s own gate is calling it right, attempt by attempt