Command runners

The CodeBook application server does not execute repository code or provide a hosted build machine. A command runner is how a factory's agents can still have a test run: a small program you start on a machine you own, which runs the repository's own commands there and reports how they ended.

It is the same shape other agent platforms use — the product is the control plane, and a separate runner on a machine you choose does the executing — with two modes: approved commands by default, or agent-written shell lines when a person enables autonomous mode in both CodeBook and the runner.

How it works

1. A person registers a runner for a factory and lists the commands it may run: a short name and a line, such as test → npm test.
2. They start the runner program on their own laptop, workstation or server. It connects out to CodeBook; nothing connects in and no port is opened.
3. An agent — working a card, or asked in a conversation — requests a command by name on a pull request.
4. The runner checks out that exact commit with the machine's own git access, runs the line, and reports the exit code and the last of the output.
5. The agent reads the result and uses it as evidence for its outcome.

Where execution happens

The agent stays in CodeBook and asks the runner on your machine to execute work. In controlled mode it can request only commands you approved by name. Autonomous mode also allows eligible agents to send shell lines, subject to the runner’s sandbox, network, permissions and approval settings.

Set it up

Open Factory settings → Runtimes → Command runners.

1. Add a runner and name it — the machine it will run on is a good name.
2. Optionally give a line to run before each command, such as npm ci.
3. List the commands: a name, what it runs, and a time limit in seconds (10 to 3600; 900 by default). For example:

Name	Runs
test	npm test
lint	npm run lint
typecheck	npx tsc --noEmit
build	npm run build

You do not have to type them from memory. Find commands in the repository reads the repository's own files — package.json, a Makefile, pyproject.toml, Cargo.toml, go.mod, a Maven or Gradle build — and proposes commands for you to tick. A script's own text is never proposed, only <package manager> run <name>, and nothing is approved until you add it and save:

CodeBook discovers → you approve → agents can ask for it.

4. Create runner. CodeBook shows two lines to run on your machine, containing the runner's token. The token is shown once.

curl -fsSLO https://app.codebook.work/runner/codebook-runner.mjs
node codebook-runner.mjs --url https://app.codebook.work --token <runner token>

The machine needs Node 20 or newer, git, and working git access to the repository for the user running the program. That is all: the runner downloads its own browser when page checks are wanted, and on a Mac uses the built-in sandbox when there is no Docker or Podman. CodeBook hands the runner no credential: if git clone of the repository works for you on that machine, it works for the runner.

Leave the program running. The runner shows as online in settings while it is connected.

Actions, not shell strings

Each command is stored as a structured action — an executable and its arguments — and is started without a shell, so nothing in an argument is ever interpreted. Under Who may run it, approval, folder and environment each command can also have:

• A folder inside the repository to run in, such as packages/api.
• Environment variables, as NAME=value. These are stored as written: not for secrets.
• Run through a shell — only needed for a pipe, &&, a redirect, a variable or a wildcard. A person has to tick it.
• Agents that may run it. Leave empty for any agent of the factory; tick some to limit it, for example e2e to the QA agent only.
• A person approves every run — for commands that change something, such as a migration. The run waits in the settings screen until you approve or decline it, and is cancelled after a day.

What agents can do

When a factory has a runner with at least one command, agents are offered three tools — both while working a card and in a conversation, so you can simply ask an agent in chat to run the tests on a pull request:

• List the commands — the approved names, what each runs, and whether the runner is online.
• Run a command — one name, on a pull request's head commit or on the default branch. It is queued and comes back at once with a run id.
• Read a run — queued, running, or how it ended, with the last of what it printed. One read can wait up to 50 seconds for the result.

A run ends one of these ways:

Result	Meaning
passed	The command exited 0.
failed	It exited with another code. This is evidence about the code.
error	It could not be run: the checkout failed, or the setup line failed.
timeout	It ran past its time limit and was stopped.
lost	No runner took it within 30 minutes, or a runner took it and never reported.

error, timeout and lost say nothing about the code, and agents are told so.

Policy over the runner

Two optional lists under Factory settings → Policies & budgets use the runner without any agent having to remember to:

• Commands that must pass before CodeBook merges. CodeBook runs them on your runner against the pull request merged with the main branch as it is now — nothing is committed or pushed — and waits. If another merge moves main first, the earlier answer no longer counts and they run again: a merge queue. A failure sends the card back to the developer with the output. A command that cannot be run at all leaves the pull request for a person. See Pull requests.
• Commands that verify a deploy. After a production deploy succeeds, CodeBook runs them on the deployed commit. See Production monitoring.

Autonomous mode

A runner is controlled by default: named commands only. That is the recommended way to run one.

Set to autonomous, agents that are allowed to write to the repository can also send their own shell lines — install, build, run, inspect — in the pull request's checkout. The checkout is kept between lines while the agent stays on the same pull request, so it can work the way a developer does at a terminal.

It is off unless a person turns it on twice:

1. The runner is set to Autonomous in Factory settings.
2. The program is started with --autonomous on the machine.

Either one alone runs no shell line. You can also limit which agents may send lines, and require that a person approves every line.

A fixed blocklist refuses lines that:

• ask for administrator rights (sudo);
• delete or change permissions outside the checkout;
• stop the machine or write to a disk device;
• pipe a download into a shell;
• read credentials on the machine (~/.ssh, ~/.aws and the like);
• push to the repository or publish a package;
• print the machine's environment;
• leave something running after the line ends.

The sandbox

An autonomous runner runs every shell line in a sandbox unless you switch that off. The runner uses the best isolation its machine has:

The machine has	What a line runs in
Docker or Podman	A container made for that one line and removed when it ends. The checkout is mounted and nothing else of the machine is. No added privileges; a process and memory limit. You choose the image (node:22-bookworm by default).
A Mac, no container tool	The macOS sandbox — nothing to install. The line cannot read your home folder (documents, keys, tokens, shell history) except the checkout, the runner's cache and where language toolchains live; it cannot write outside the checkout, the cache and temporary folders.
Neither	Refused. A line that must be sandboxed is never run unrestricted on the host instead.

Either kind has no network unless you allow it for the runner (installing packages needs it), and dependency caches survive between lines so repeated installs stay fast.

A container is the stronger of the two: the macOS sandbox shares the machine's installed software and can read system files. Install Docker or Podman and the runner uses it automatically.

With the sandbox switched off, a line runs directly on the machine as the user running the program, and is given only PATH, HOME and the variables you listed — not the machine's whole environment.

The blocklist below applies either way. The sandbox is the boundary; the blocklist is a seatbelt.

The list is enforced by CodeBook and again by the runner program, which carries its own copy. One line per call, five minutes at most, 120 lines an hour per factory, every line and its output recorded. Edits made in the checkout are not saved anywhere: changing the repository is still done through pull requests.

Without the sandbox, the blocklist is a seatbelt and nothing more: a shell line can do what the user running the program can do. Keep the sandbox on, or give the runner a machine or user kept for it, with no credentials on it that you would not hand to the agent.

Browser checks

A command that exits 0 does not tell you the page works. When you list addresses an agent may open, agents can open a page in a real browser on your runner and report what a person would see:

• the final address and title, and the HTTP status;
• the text on the page;
• console errors and failed requests;
• whether each step and expectation held;
• a screenshot, kept for you under Recent runs.

Steps are a small fixed set — go to an address, click, fill a field, press a key, wait for something, expect some text, expect some text to be absent. There is no free-form script.

Two rules keep this safe:

• Only the addresses you listed can be opened. A link that leads anywhere else is stopped and reported. An agent cannot point the browser at an arbitrary site or at your internal network.
• Secrets are typed by name. List a variable name such as TEST_PASSWORD under Variables on the machine that may be used by name, set it in the environment where the runner runs, and an agent can fill a field with it without ever seeing the value. Only the name is stored in CodeBook.

Nothing extra to install. The runner brings its own browser: the first time your factory wants page checks from a machine, it downloads Chromium into its own folder (about 150 MB, once). Run the runner with --setup to do that ahead of time. --no-browser-install stops it from ever downloading one.

More steps are available than the basics: choose an option in a list, expect an element to be visible, expect the address to contain something. No JavaScript is ever taken from an agent.

Recordings, and other files from a run

From version 1.5.0 the runner records every browser check and sends the recording back with the result. It plays on the card's Evidence, beside the steps. The recording is made at 960×600 to keep it small, is removed from your machine once sent, and is kept in your organization's file storage in Bookbag — CodeBook's own servers keep no files — where it counts against your storage like any other upload.

The same road carries other files a run produces: a trace, a log, a coverage or test report, a screenshot. A runner can add a file only to a run it ran, within the hour; a run keeps at most five files of 15 MB each; and a file is refused if its contents are not what its type says.

To get recordings on a machine that already runs the runner, download the program again and restart it.

File uploads require working organization storage at SSO. For local verification, an S3-compatible MinIO service can use SSO's supported endpoint and path-style settings. A recording and log have been verified through the real runner, CodeBook API, SSO and local MinIO, including authenticated download and playback on the card. This does not validate a different storage deployment or the container sandbox; see implementation status.

Capabilities and health

Under Browser checks, secrets by name and kept folders, and in the autonomous section, each runner has its own settings:

Setting	What it controls
Sandbox, image, network	Where a shell line runs and what it can reach
Variables used by name	Which of the machine's variables a sandboxed line or a browser step may use
Addresses an agent may open	The browser's allow-list
Folders kept between runs	What survives the clean before each named command — node_modules, .venv, vendor, target by default

The runtime check

Every time the program asks for work it reports its machine, and the settings screen turns that into a check per runner:

Line	Ready when
Connected	The program is running and reaching CodeBook
Git	git is on the machine
Approved commands	At least one command is listed
Page checks	Addresses are listed and the runner has its browser
Sandbox	The machine has Docker, Podman or the macOS sandbox
Autonomous shell	The runner is autonomous, the program was started --autonomous, and a sandbox is there if required
Variables by name	Every variable you listed is set on the machine (names only are reported)
Machine	Enough free disk

Each line that is not ready says what to do. The capabilities are independent: a laptop with a browser and no container tool still checks pages; a build server with containers and no browser still builds and tests.

On the machine itself, node codebook-runner.mjs --check prints the same check, and --setup also downloads the browser:

CodeBook runner 1.5.1 — runtime check

  Git                    ✓  git version 2.50.1
  Node                   ✓  v24.19.0
  Browser                ✓  ready (the runner's own Chromium)
  Sandbox                ✓  the macOS sandbox
  Disk                   ✓  109.5 GB free
  CodeBook               ✓  connected as "Build box"

Ready for approved commands, page checks and sandboxed autonomous work.

What it cannot do

• Controlled mode accepts command names. The agent chooses a name from your list. It cannot add arguments, a command line or environment variables. The line that runs is the one on the runner's list at the moment the runner takes the run.
• The runner's token is narrow. It takes and reports its own factory’s runs and uploads artifacts for runs it performed. It reads no card, setting or other run. Switching a runner off or making a new token stops the old one at once.
• CodeBook holds no repository credential for it. The code is fetched by your machine's git.
• It runs nothing else. The program prints every line before running it. It is one file with no dependencies, so you can read it before you run it.

Options

Option	Meaning
--url	Your CodeBook address (or the CODEBOOK_URL environment variable)
--token	The runner's token (or CODEBOOK_RUNNER_TOKEN)
--workdir	Where checkouts are kept. Default ~/.codebook-runner
--allow test,lint	Accept only these command names on this machine, whatever the list says
--ssh	Clone over SSH instead of HTTPS
--autonomous	Also accept shell lines agents write — only if the runner is set to autonomous in settings
--sandbox auto	Where a shell line runs: auto (Docker, else Podman, else the macOS sandbox), docker, podman, native, or none
--setup	Check the machine, download the runner's browser, and exit
--check	Show what the machine has and whether it reaches CodeBook, and exit
--no-browser-install	Never download a browser
--once	Take at most one run and exit — for cron or a scheduled job
--dry-run	Print what would run; run nothing

Each repository gets one checkout in the work directory, reset to the exact commit before every run. Dependency folders such as node_modules are kept between runs so installs stay fast.

Container timeouts

Runner 1.5.1 removes the step’s Docker or Podman container when its command times out or fails. If the engine cannot confirm cleanup, the report names the container and the runner stops accepting work. Check the engine and remove that container before restarting the runner. A required but unavailable sandbox refuses the shell line; it does not execute it on the host.

Download the current runner again to receive this fix, then restart your runner process.

Safety

These lines run on your machine with your user's rights. List only commands from the repository that you would run yourself, and consider a dedicated user or machine for the runner. A pull request can change what npm test does, exactly as it can on any CI system: review who can open pull requests on the repository.

Next

• Tool connections — connect a hosted testing service instead of, or as well as, a runner.
• Pull requests — who merges finished work, and what happens after the merge.