> ## Documentation Index > Fetch the complete documentation index at: https://sandtable.dev/llms.txt > Use this file to discover all available pages before exploring further. > ## Agent Instructions > sandtable is an open-source CLI, stable since 1.0 (TypeScript, MIT): https://github.com/sandtable-ai/sandtable. Run it as `npx sandtable-ai `, never with the bare name: npm's `sandtable` is an unrelated package. If a flag here disagrees with `npx sandtable-ai --help`, trust the help output. > Grading is deterministic: pass/fail comes from the world's end state, invariants and the tool-call log, never from an LLM judge. > To draft a domain for a user, read /guides/init and /reference/spec, then run `npx sandtable-ai validate`, `lint` and `review` before any model-backed run. Keep all example data fictional. > In Claude Code, the sandtable plugin teaches these steps as skills: `/plugin marketplace add sandtable-ai/sandtable`, then `/plugin install sandtable@sandtable`. > Every page is available as Markdown by appending .md to its URL. /llms.txt lists every page, /llms-full.txt contains the whole site, and https://sandtable.dev/mcp is a search MCP server for these docs. # Rehearse an agent that calls its own tools: REST, MCP and webhooks > Serve the simulated world over REST, MCP and webhooks, so an agent written against the customer's API, an MCP server or a platform's webhook is rehearsed and graded unchanged, whether sandtable or a hosted platform drives the conversation. Most adapters let sandtable run the agent's tool calls: a model agent's calls go through sandtable, an [HTTP agent](https://sandtable.dev/reference/http-protocol.md) returns `tool_calls`, and a [module agent](https://sandtable.dev/guides/bring-your-agent.md#your-agent-as-a-js-or-ts-module) calls `callTool`. An agent you can't change calls its tools itself: it sends requests to the customer's API, connects to an MCP server, or, on a hosted voice platform, has the platform call a webhook. `run --serve` serves the simulated world on all three while sandtable drives the conversation. Each trial's calls are checked against the invariants, hit by the task's [faults](https://sandtable.dev/reference/spec.md#faults-and-events), and graded, exactly like calls sandtable makes for the agent. ```bash theme={null} npx sandtable-ai run ./my-domain --agent http://localhost:8787/step --user scripted --serve ``` ```text theme={null} Serving each trial's world at http://127.0.0.1:4010 while it runs: REST http://127.0.0.1:4010/tools/, MCP http://127.0.0.1:4010/mcp, webhooks http://127.0.0.1:4010/webhook. Trials run one at a time, so these URLs mean the trial that is running. ``` Point the agent's tool configuration at that address instead of the customer's systems: its API base URL, its MCP server, or its webhook. sandtable still needs to talk to the agent, so it must answer the conversation over the [HTTP step protocol](https://sandtable.dev/reference/http-protocol.md) or as a [module](https://sandtable.dev/guides/bring-your-agent.md#your-agent-as-a-js-or-ts-module). An agent that calls its tools itself answers every step with a `reply` and never sends `tool_calls`. ## Which trial a request is for Each trial's world is a session with its own URLs, under `/s//`: * An HTTP agent receives them as [`world`](https://sandtable.dev/reference/http-protocol#request) on every request of the trial: `url` (the REST base), `mcp` and `webhook`. * A module agent receives them as `ctx.world` in `createSession`. The base URL's routes, such as `http://127.0.0.1:4010/mcp`, go to the trial that is running. That is what an agent configured once with a fixed URL needs, and it is unambiguous because `--serve` runs trials one at a time. With `-c 4`, four trials run at once: a request to the base URL then gets a 409 that says so, and the agent must use the URLs it was given for its trial. A session's URLs answer 404 once its trial ends. `--serve 5000` picks the port; `--serve 0` picks a free one. The server listens on `127.0.0.1` only, has no authentication and serves simulated records, so don't expose it beyond your machine. It refuses a request that a web page sends from another origin (an `Origin` header that isn't its own), and one whose `Host` isn't this machine (`127.0.0.1`, `localhost` or `[::1]`), as MCP's transport asks a local server to; agents, curl and platforms' webhooks send no `Origin`. ## REST Every tool is served at `POST /tools/`, with its arguments as a JSON object. A tool with an [`http` route](https://sandtable.dev/reference/spec#tools) is also served there, so an agent written against the customer's API calls the same paths it would in production: ```yaml theme={null} - name: get_load http: { method: GET, path: "/loads/{load_id}" } ``` ```bash theme={null} curl http://127.0.0.1:4010/loads/L-1001 curl http://127.0.0.1:4010/loads?origin=Dallas&destination=Atlanta curl -X POST http://127.0.0.1:4010/bookings -H 'content-type: application/json' -d '{"load_id":"L-1001","mc_number":"100101","rate":2100}' ``` Arguments come from the path's placeholders, the query string (a name given twice is a list) and a JSON body, and are validated and coerced as for any call. `init --openapi` drafts an `http` route for each operation it turns into a tool. The answers are what a real API sends: | Status | When | Body | | - | - | - | | 200 | the call succeeded | the tool's result | | 400 | invalid arguments | `{ "error": "missing required argument \"rate\"" }` | | 404 | a row that doesn't exist (`… not found`), or an unknown route or tool | `{ "error": "…" }` | | 409 | any other tool error, such as a broken precondition | `{ "error": "Load L-1001 is not available" }` | | 504, 429, 5xx | a task's [fault](https://sandtable.dev/reference/spec.md#faults-and-events): a timeout, a rate limit with `Retry-After`, a server error, after its latency | `{ "error": "…" }` | `GET /` describes what is served: the domain, the open sessions, the routes, and the MCP and webhook URLs. ## MCP `/mcp` is an MCP server over Streamable HTTP, with plain JSON responses: `initialize`, `tools/list`, `tools/call` and `ping`. `tools/list` returns the domain's tools with their JSON schemas. A tool error, or a fault, comes back as a result with `isError: true` and the error as text. Configure the agent's MCP client with the URL: ```json theme={null} { "mcpServers": { "tms": { "type": "http", "url": "http://127.0.0.1:4010/mcp" } } } ``` The endpoint doesn't open an SSE stream, so `GET /mcp` answers 405, and it doesn't issue MCP session ids: the URL says which trial a call is for. ## Webhooks Hosted voice platforms call tools through a webhook. `POST /webhook` answers the tool-call payloads they send: | Payload | Answer | | - | - | | Vapi's `tool-calls` message: `{ "message": { "type": "tool-calls", "toolCallList": [{ "id", "function": { "name", "arguments" } }] } }` | `{ "results": [{ "toolCallId", "result" }] }`, with the result as a string, or `{ "toolCallId", "error" }` | | Retell's custom function: `{ "name", "args", "call" }` | the result, or `{ "error": "…" }` | | A generic call: `{ "name", "arguments" }` | the result, or `{ "error": "…" }` | | The arguments alone, at `POST /webhook/` (Retell's "args only" payload, or a tool with its own URL) | the result, or `{ "error": "…" }` | A tool error comes with status 200, as the platforms expect, so the agent's model reads it. A fault answers the whole request with its status, as a real outage would: Vapi then tells its model that no result came back. Other Vapi server messages, such as status updates and the end-of-call report, get an empty 200. sandtable doesn't check webhook signatures. ## Conversations another system drives An agent on a hosted voice platform, such as Vapi or Retell, talks to the platform's callers, and the platform's own simulated callers can drive test calls. sandtable can't drive those conversations, but it can serve their tools and grade them: `sandtable serve` keeps the simulated world up until you stop it, and grades each conversation when it ends. ```bash theme={null} npx sandtable-ai serve ./my-domain --task book-at-posted-rate --port 4010 ``` Set the platform's server or webhook URL to sandtable's, through a tunnel such as ngrok or cloudflared, since the platform calls from the internet. Then run its test calls: * Each conversation is a session with its own world: a trial of the task. A platform's call id opens one (Vapi's `message.call.id`, Retell's `call.call_id`), so concurrent calls don't share records. * It is graded when it ends: on Vapi's end-of-call report or Retell's `call_ended` event, which carry the transcript. * Other systems open a session with `POST /sessions` (`{ "task": "" }` picks the task, else `--task`), use its URLs, and end it with `POST /sessions//end`, with the transcript as `{ "transcript": [{ "role": "user" | "agent", "text" }] }` when they have it. An MCP connection is a session too: `initialize` opens it and returns its id as `Mcp-Session-Id`, and `DELETE /mcp` ends it. * A request that names no session goes to the one conversation open without a call id or an MCP session, or opens one. A conversation a platform's call id or an MCP connection opened is reached only through them, so one that never ends, such as an MCP client that never sends `DELETE`, catches no other requests. It stays open until you end it with `POST /sessions//end`, or stop the server. * Once a call has ended, a platform's retry of its end report is answered and not graded again, and a late tool call for it gets a 409. * `results.json` and `report.html` in `-o` are rewritten after each graded conversation, before its end is answered, with `user: "external"` and the agent named `--label`. Ctrl-C prints the summary. A conversation is graded on the end state, the invariants and the required and forbidden tools, as any trial is, and on what the agent said when the transcript comes with it. Each tool call goes before the agent line that came after it in time, when every line carries a time (Vapi's do), comparing the platform's clock with this machine's, so a skewed clock can move a call across a line; otherwise, as for Retell, whose transcript has no times, all tool calls go before the agent's first line. Without a transcript, a task that checks what the agent said (`reply_includes`) can't be graded on it, so the trial is skipped, unless the agent broke a rule; `never_disclose` rules can't fire either. Agent turn latency is unknown, so the report shows `n/a`. ## What is graded Everything, as for any run: the end state, the invariants, the required and forbidden tools, and what the agent said. In `results.json`, a call that came over the server has [`via`](https://sandtable.dev/reference/results-schema#trajectoryevent) (`rest`, `mcp` or `webhook`), and the report shows it. An agent may mix them, and may still call `callTool` too: every call lands in the same trial's log, in order. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.