Rehearse an agent that calls its own tools: REST, MCP and webhooks

Most adapters let sandtable run the agent’s tool calls: a model agent’s calls go through sandtable, an HTTP agent returns tool_calls, and a module agent calls callTool. An agent you can’t change calls its tools itself: it sends requests to the customer’s API, connects to an MCP server, or, on a hosted voice platform, has the platform call a webhook. run --serve serves the simulated world on all three while sandtable drives the conversation. Each trial’s calls are checked against the invariants, hit by the task’s faults, and graded, exactly like calls sandtable makes for the agent.

npx sandtable-ai run ./my-domain --agent http://localhost:8787/step --user scripted --serve

Serving each trial's world at http://127.0.0.1:4010 while it runs: REST http://127.0.0.1:4010/tools/<name>, MCP http://127.0.0.1:4010/mcp, webhooks http://127.0.0.1:4010/webhook.
  Trials run one at a time, so these URLs mean the trial that is running.

Point the agent’s tool configuration at that address instead of the customer’s systems: its API base URL, its MCP server, or its webhook. sandtable still needs to talk to the agent, so it must answer the conversation over the HTTP step protocol or as a module. An agent that calls its tools itself answers every step with a reply and never sends tool_calls.

Which trial a request is for

Each trial’s world is a session with its own URLs, under /s/<session>/:

An HTTP agent receives them as world on every request of the trial: url (the REST base), mcp and webhook.
A module agent receives them as ctx.world in createSession.

The base URL’s routes, such as http://127.0.0.1:4010/mcp, go to the trial that is running. That is what an agent configured once with a fixed URL needs, and it is unambiguous because --serve runs trials one at a time. With -c 4, four trials run at once: a request to the base URL then gets a 409 that says so, and the agent must use the URLs it was given for its trial. A session’s URLs answer 404 once its trial ends. --serve 5000 picks the port; --serve 0 picks a free one. The server listens on 127.0.0.1 only, has no authentication and serves simulated records, so don’t expose it beyond your machine. It refuses a request that a web page sends from another origin (an Origin header that isn’t its own), and one whose Host isn’t this machine (127.0.0.1, localhost or [::1]), as MCP’s transport asks a local server to; agents, curl and platforms’ webhooks send no Origin.

REST

Every tool is served at POST /tools/<name>, with its arguments as a JSON object. A tool with an http route is also served there, so an agent written against the customer’s API calls the same paths it would in production:

- name: get_load
  http: { method: GET, path: "/loads/{load_id}" }

curl http://127.0.0.1:4010/loads/L-1001
curl http://127.0.0.1:4010/loads?origin=Dallas&destination=Atlanta
curl -X POST http://127.0.0.1:4010/bookings -H 'content-type: application/json' -d '{"load_id":"L-1001","mc_number":"100101","rate":2100}'

Arguments come from the path’s placeholders, the query string (a name given twice is a list) and a JSON body, and are validated and coerced as for any call. init --openapi drafts an http route for each operation it turns into a tool. The answers are what a real API sends:

Status	When	Body
200	the call succeeded	the tool’s result
400	invalid arguments	`{ "error": "missing required argument \"rate\"" }`
404	a row that doesn’t exist (`… not found`), or an unknown route or tool	`{ "error": "…" }`
409	any other tool error, such as a broken precondition	`{ "error": "Load L-1001 is not available" }`
504, 429, 5xx	a task’s fault: a timeout, a rate limit with `Retry-After`, a server error, after its latency	`{ "error": "…" }`

GET / describes what is served: the domain, the open sessions, the routes, and the MCP and webhook URLs.

MCP

/mcp is an MCP server over Streamable HTTP, with plain JSON responses: initialize, tools/list, tools/call and ping. tools/list returns the domain’s tools with their JSON schemas. A tool error, or a fault, comes back as a result with isError: true and the error as text. Configure the agent’s MCP client with the URL:

{ "mcpServers": { "tms": { "type": "http", "url": "http://127.0.0.1:4010/mcp" } } }

The endpoint doesn’t open an SSE stream, so GET /mcp answers 405, and it doesn’t issue MCP session ids: the URL says which trial a call is for.

Webhooks

Hosted voice platforms call tools through a webhook. POST /webhook answers the tool-call payloads they send:

Payload	Answer
Vapi’s `tool-calls` message: `{ "message": { "type": "tool-calls", "toolCallList": [{ "id", "function": { "name", "arguments" } }] } }`	`{ "results": [{ "toolCallId", "result" }] }`, with the result as a string, or `{ "toolCallId", "error" }`
Retell’s custom function: `{ "name", "args", "call" }`	the result, or `{ "error": "…" }`
A generic call: `{ "name", "arguments" }`	the result, or `{ "error": "…" }`
The arguments alone, at `POST /webhook/<tool>` (Retell’s “args only” payload, or a tool with its own URL)	the result, or `{ "error": "…" }`

A tool error comes with status 200, as the platforms expect, so the agent’s model reads it. A fault answers the whole request with its status, as a real outage would: Vapi then tells its model that no result came back. Other Vapi server messages, such as status updates and the end-of-call report, get an empty 200. sandtable doesn’t check webhook signatures.

Conversations another system drives

An agent on a hosted voice platform, such as Vapi or Retell, talks to the platform’s callers, and the platform’s own simulated callers can drive test calls. sandtable can’t drive those conversations, but it can serve their tools and grade them: sandtable serve keeps the simulated world up until you stop it, and grades each conversation when it ends.

npx sandtable-ai serve ./my-domain --task book-at-posted-rate --port 4010

Set the platform’s server or webhook URL to sandtable’s, through a tunnel such as ngrok or cloudflared, since the platform calls from the internet. Then run its test calls:

Each conversation is a session with its own world: a trial of the task. A platform’s call id opens one (Vapi’s message.call.id, Retell’s call.call_id), so concurrent calls don’t share records.
It is graded when it ends: on Vapi’s end-of-call report or Retell’s call_ended event, which carry the transcript.
Other systems open a session with POST /sessions ({ "task": "<id>" } picks the task, else --task), use its URLs, and end it with POST /sessions/<id>/end, with the transcript as { "transcript": [{ "role": "user" | "agent", "text" }] } when they have it. An MCP connection is a session too: initialize opens it and returns its id as Mcp-Session-Id, and DELETE /mcp ends it.
A request that names no session goes to the one conversation open without a call id or an MCP session, or opens one. A conversation a platform’s call id or an MCP connection opened is reached only through them, so one that never ends, such as an MCP client that never sends DELETE, catches no other requests. It stays open until you end it with POST /sessions/<id>/end, or stop the server.
Once a call has ended, a platform’s retry of its end report is answered and not graded again, and a late tool call for it gets a 409.
results.json and report.html in -o are rewritten after each graded conversation, before its end is answered, with user: "external" and the agent named --label. Ctrl-C prints the summary.

A conversation is graded on the end state, the invariants and the required and forbidden tools, as any trial is, and on what the agent said when the transcript comes with it. Each tool call goes before the agent line that came after it in time, when every line carries a time (Vapi’s do), comparing the platform’s clock with this machine’s, so a skewed clock can move a call across a line; otherwise, as for Retell, whose transcript has no times, all tool calls go before the agent’s first line. Without a transcript, a task that checks what the agent said (reply_includes) can’t be graded on it, so the trial is skipped, unless the agent broke a rule; never_disclose rules can’t fire either. Agent turn latency is unknown, so the report shows n/a.

What is graded

Everything, as for any run: the end state, the invariants, the required and forbidden tools, and what the agent said. In results.json, a call that came over the server has via (rest, mcp or webhook), and the report shows it. An agent may mix them, and may still call callTool too: every call lands in the same trial’s log, in order.

Get started

Concepts

Guides

Reference

Resources

Rehearse an agent that calls its own tools: REST, MCP and webhooks

Which trial a request is for

REST

MCP

Webhooks

Conversations another system drives

What is graded

​Which trial a request is for

​REST

​MCP

​Webhooks

​Conversations another system drives

​What is graded

Which trial a request is for

REST

MCP

Webhooks

Conversations another system drives

What is graded