The factory dashboard
Inspect productivity, spending, scorer evidence, Trust targets and improvement results over the current and previous windows.
A factory that works by itself is measured by how rarely it needs you. The Dashboard tab on every factory shows that, and whether it is getting better.
Open a factory and choose Dashboard. Pick the window — the last 7, 30 or 90 days. Every headline number is shown beside the same number for the window before, so you can see the direction as well as the level.
The dashboard has five tabs: Productivity, Spending, Scorers, Trust and Self-improvement. The selected tab and any open scorer are kept in the URL for sharing with people who can access the factory. For setup rather than measurement, use the Getting started checklist on Overview.
Opening the dashboard does not call a model to calculate its metrics. The totals use stored work records; scorer judgments and observer proposals come from their separate metered model calls.
Productivity
| Number | What it means |
|---|---|
| Human interactions per pull request | Every time a person acted on the cards behind a merged pull request, averaged. Lower is more autonomous. |
| Merged with no person | The share of merged pull requests that nobody touched at all. |
| First pass | The share of finished cards that never came back to a station. |
| Kickoff → PR | From the first run on the card to the pull request being opened. |
| Kickoff → merge | From the first run to the merge. |
| PR → first human review | From the pull request opening to the first time a person approved, rejected, answered or commented. Pull requests nobody touched are left out. |
| First human review → merge | From that first review to the merge. |
| Merge → deployed | From the merge to your deploy reporting success, when deploys are watched. |
| Pull requests | Opened and merged, by day. |
| Total runs | Every run, by agent or by station. |
Cycle times are medians, so one slow card does not move them.
What counts as a human interaction
One interaction is one of these, made by a person:
- moving a card to another station
- starting or retrying a run
- stopping a run
- approving or rejecting a run
- answering something the factory asked a person
- deciding a review
- approving or refusing a command on your runner
- commenting on a card
Reading, opening settings, and anything an agent or CodeBook did are not counted. Neither is what the manager does when you ask it to in conversation — asking is a conversation, not an intervention on the card.
Required, or a rescue
"2.5 interactions per pull request" does not say whether that is good. It depends on which kind.
| Kind | What it is | Examples |
|---|---|---|
| Required by your rules | Your own rules asked for a person. The factory is working as designed; this number moves when you change the rules. | Approving or rejecting a run, deciding a sampled review, approving a runner command |
| Unplanned rescue | The factory could not finish by itself and a person stepped in. This is the number to drive towards zero. | Moving a card by hand, starting or stopping a run by hand, answering what the factory had to ask |
| Comments | Neither asked for nor a rescue by itself. | A comment on a card |
The dashboard shows each per pull request, the breakdown by kind, and Merged with no rescue — the share of merged pull requests that needed no unplanned help. An approval your rules asked for does not count against it.
The panel also shows which kinds of interaction make up the total and who the factory leaned on most. That is where to look for what to automate next: a factory with many approvals may be asking for approvals it no longer needs.
Interactions are counted from the day the dashboard was added. Approvals, manual starts, answers and comments from before that day are filled in from older records; card moves from before that day are not, because the older record cannot tell a person's move from some of CodeBook's own.
Spending
Tokens by day, by agent and by model, and two numbers that tell you what a piece of work costs: tokens per merged pull request and tokens per finished card.
Dollars
CodeBook counts tokens. What a token costs is set per model in Bookbag: Admin → AI, the Price ($ / 1M tokens) field beside each model. Type what a million tokens through that model costs you — one figure for input and output together.
CodeBook also records reported input, output and cached-input token counts for routed model attempts and scorer judgments. If SSO supplies both separate prices and usage by kind, those are priced separately; otherwise the blended price is used. The current SSO price editor supplies the blended figure. Missing token splits are not guessed.
Once a model has a price, Spending shows:
- AI spend for the window, by day, by model and by agent.
- Cost per merged pull request — every run on the cards behind it.
- Cost per autonomous pull request — the same, for pull requests that needed no unplanned rescue.
On a scorer's page each judged run shows what the run cost, and — separately — what judging it cost, because measuring costs money too.
A model with no price is never counted as free. Its tokens are left out of the dollar figure, and the page says so and names the model. Leave a price empty when you do not want money shown for a model; type 0 only for a model that really costs nothing.
These figures report what the work cost. They bill nobody: what your organization is charged is its plan.
Scorers
Sample completed station work with scorers. Read pass rate together with coverage, not-scored reasons and human agreement, then inspect the actual evidence. Scorers measure quality without blocking work; station evaluators remain the reviews that can stop a card.
Trust
The Trust scorecard compares ten measures with targets you save: completion, first-pass QA, human rescues, auto-merges, rollbacks, unscored work, command/browser failures, cost per completed card and time to merge. Each includes its sample size. Suggested targets do nothing until saved; too little data never counts as meeting a target.
Self-improvement
Inspect recurring failures and the observer's proposals. Accepting instructions creates an inactive version. Test it before release, compare its results and follow the improvement report to see which proposals were evaluated, improved, regressed or released. The observer does not apply its own changes.