Skip to main content
In about five minutes: an offline demo with no API key, the same example with Claude playing both sides, and the example run against an agent of your own.

Install

Needs Node 20 or later.
These docs run the CLI as npx sandtable-ai <command>, which uses the copy installed in the project when there is one. Keep the -ai: npm has an unrelated package called sandtable, and without the suffix npx downloads and runs that package whenever sandtable-ai isn’t installed in the project.

Run it

1

Run the offline demo

Two rule-based agents replay 50 scripted carrier calls against a fictional freight brokerage. The naive agent books whatever it is asked to and scores pass^1 10.0% (5 of 50), with its rule breaks grouped under “Top failure modes”. The careful agent verifies carriers, respects the rate ceiling and hands off when it should, and scores 100%.Each run writes results.json and report.html under .sandtable/runs/ and opens the first report in your browser. Pass --no-open to skip that.
2

Read the report

The HTML report is one self-contained file. For every task it shows each trial’s transcript, the tool calls with their arguments and results, every check that passed or failed, and the changes the agent made to the world. Read a failing trial before trusting any score: it tells you whether the agent, the task or the simulated caller was wrong.
3

Run it with Claude

Claude plays both the carrier and the agent, two trials per task by default.
A cheaper model for the simulated caller keeps the cost down. The run holds one conversation per task and trial (-k, default 2), so its cost grows with both. For scale: on the example’s first 10 tasks, 20 trials, Claude Opus 5.5 as the agent and Claude Haiku 5.5 as the caller cost about $0.58 at list prices (results). The full example has more tasks and costs proportionally more.
4

Point it at your own agent

Any agent that answers one POST per step can be rehearsed. Here is a toy one that only verifies the caller’s MC number:
Start it in one terminal, then run the freight example against it in another:
It scores pass^1 36.0% (18 of 50). It never books a load, so it passes only tasks where the right outcome is to book nothing. The noop baseline, an agent that never calls a tool, scores 32.0% (16 of 50) for the same reason. The toy agent passes two more tasks than noop, double-brokering-on-yes and broker-claims-carrier: in both, the agent must look the carrier up with verify_carrier and then book nothing, and the toy agent does look it up. The noop score is the floor of a suite, and the first number to know about it.examples/freight works from any directory: when there is no such folder where you run the command, sandtable uses the copy bundled with the package and prints its path.

Next

How it works

Worlds, tools, invariants, tasks and grading in one page.

Run against your agent

Claude, Bedrock, OpenAI-compatible, HTTP and JS/TS module adapters in detail.

Draft a domain with init

Build a domain for your own customer from their documents.

Use it in CI

Gate pull requests on pass^k.

Use it from Claude Code

Install the plugin and let your coding agent draft, lint and run a domain.