Trust targets and reliability

Open Dashboard → Trust. Choose the last 7, 30 or 90 days. Each row shows the current value, the previous window, the number of observations behind it, a target and a plain-language explanation of the counts. No model is called to calculate this scorecard.

What the ten measures mean

Measure	What is counted	Direction
Completion rate	Cards that reached a done outcome or a merged pull request, out of cards whose agent work started in the window. Parallel branch cards count with their parent.	Higher
First-pass QA	Done card outcomes with no station revisit.	Higher
Human rescues per pull request	Unplanned human interventions on the cards behind each merged pull request, averaged. Required approvals are separate.	Lower
Auto-merge rate	Merged pull requests recorded as merged by CodeBook, out of all merged pull requests.	Higher
Rollback rate	Merged pull requests whose change was reverted or whose deployment or verification remains failed.	Lower
Not-scored rate	Scorer results recorded as not scored, out of all scorer results in the window.	Lower
Runner failure rate	Failed, errored, timed-out or lost command/shell runs, out of command runs with a verdict. Queued and cancelled work is excluded.	Lower
Browser check failure rate	The same failure measure for browser checks.	Lower
Cost per completed card	Factory spending during the window divided by done card outcomes. Spending on unfinished work is included. Uses dollars when model prices are available, otherwise tokens.	Lower
Median completion time	Median time from kickoff to merge for pull requests merged in the window.	Lower

An auto-merge rate measures who merged; it is different from the dashboard's share of pull requests completed without a human rescue. A low failure rate also needs context: inspect the cards and whether required checks actually ran.

Set your own targets

Enter a value beside a measure and select Save. Percentages use 0–100, rescue targets use interventions per pull request, and time targets use hours. Check the displayed unit before setting a cost target: it can be dollars or tokens depending on your model prices.

Suggest fills the input with a starting value; it saves nothing. Clear removes a saved target. Targets belong to the factory and should reflect its kind of work. They do not change routing, merge rules, budgets or permissions.

The status is one of:

• Met — the value meets the saved target in the desired direction.
• Needs improvement — it does not meet the target.
• No target — there is enough data, but no target has been saved.
• Too little data — fewer than five observations, or no usable value. This is not a pass or a failure.

The summary counts targets that can actually be graded. Missing evidence does not earn a green result.

Use the evidence

Start with a measure that misses its target, then follow the relevant dashboard, scorer results, runner evidence or pull requests. High not-scored rates often mean the judge is missing a diff, not that the code is good or bad.

These are observations of recorded work. Merge, rollback and deployment measures need real pull requests and reporting connections; a new factory may have nothing to count yet. Compare a representative set of tasks before changing policy, and use reviewed changes or agent evaluations to test an improvement.