Review and schedule factory changes
Draft named edits, inspect their impact, request an independent review and publish now or later.
Use Factory settings → Changes to propose changes to the factory definition. Each change has a draft, a review state and a history. Creating a draft does not alter the live factory.
Draft the change
Enter a title and select Create draft. Use the edit builder to choose an operation, workflow and station. You can change workers, instructions, routing and other supported settings, or add and rearrange workflow structure.
The panel lists the edits and their diff. Remove an edit to take it out of the draft. Edit title and description lets you explain the purpose of the change.
Inspect and request review
Read the before-and-after Test factory result and new findings. If the live factory has changed since the draft began, inspect the drift notice. Rebasing can drop edits that no longer apply and requires a fresh approval.
Select Request review. A change that worsens the simulation may require a reason. Another person with write access reviews the draft; its author cannot approve their own change.
The reviewer can approve, reject, or request changes. Rejection and a request for changes require a note so the author knows what to fix.
Publish now or later
An approved change can be published immediately or scheduled with Publish later. The date/time input uses your browser’s local time.
Scheduled publication is checked periodically by the backend workforce scheduler. The service rechecks the approval and current definition when the time arrives. If it cannot publish, the change returns to approved with a history entry and an alert explaining why. A stopped backend cannot publish scheduled work.
Cancel schedule removes the scheduled time. Withdraw closes a change without publishing it. Published changes become factory versions.
Model policy changes
In a factory that requires another person's approval, saving or removing a model policy in Factory settings → Models (or a station's Worker tab) does not change the live setting. The editor asks for a reason and offers Request model policy change, which opens an ordinary change request with one model-policy edit. It does nothing until a second writer approves and publishes it; the requester cannot approve their own request.
The request's preview shows Models resolved for this review: the exact provider and model each changed factory or station resolves to in your organization right now. Approval binds those models. If, before publication, a pinned model is switched off or removed, a model name now resolves to a different provider, the organization default changes under an inherited pin, or the automatic pool changes, publication is refused and nothing changes. Withdraw the request and ask again for a new approval. The check runs again inside publication, so a change to the catalogue while publishing rolls back the whole publication. A request naming a model the organization does not have is refused.
Model policy edits are structural: they cannot be tried as a canary.
Try a canary
For an approved change that affects per-run settings, choose a Canary share from 5% to 50% and select Start canary. Eligible settings include the worker, instructions, routing, evaluators and retry limit. Structural, contract and policy changes must be published or scheduled instead. Human evaluators need an assigned reviewer before the canary starts.
The queue records the settings, routing candidates and evaluators for each selected run. Stop canary returns new work to the live settings; existing runs retain their recorded configuration. Current permissions and station revision checks still apply. Promote to all runs publishes the approved change.
Use Refresh results to compare completed runs, pass rates, first-attempt pass rates and median tokens with other runs at the same stations. The report requires at least ten finished runs on each side before showing a comparison verdict. Stopping, promoting or withdrawing the canary fixes the comparison period at that recorded event. Later edits do not add new control runs or erase the original controls. If its ending history is unavailable, CodeBook says the comparison is unavailable. Restarting a canary starts a new comparison period and excludes fallbacks from an earlier period.
These controls have automated coverage. A local authenticated walkthrough also exercised two-person approval, candidate allocation, comparison data and stopping a canary with a scripted model. The browser walkthrough also confirmed that stopping preserves both groups and keeps the minimum-sample warning. It did not establish real-model quality, test every external runtime or prove a production promotion. See implementation status for the remaining verification limits.
History and limitations
Expand Change history to read edits, review decisions, scheduling and publication events. Permissions and revision checks are enforced on the server, including when multiple people have the panel open.
For a change to the factory graph, use reviewed publication or environment promotion.