> ## Documentation Index
> Fetch the complete documentation index at: https://sandtable.dev/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> sandtable is an early-alpha, open-source CLI (TypeScript, MIT): https://github.com/sandtable-ai/sandtable. Run it as `npx sandtable-ai <command>`, never with the bare name: npm's `sandtable` is an unrelated package. If a flag here disagrees with `npx sandtable-ai <command> --help`, trust the help output.
> Grading is deterministic: pass/fail comes from the world's end state, invariants and the tool-call log, never from an LLM judge.
> To draft a domain for a user, read /guides/init and /reference/spec, then run `npx sandtable-ai validate`, `lint` and `review` before any model-backed run. Keep all example data fictional.
> In Claude Code, the sandtable plugin teaches these steps as skills: `/plugin marketplace add sandtable-ai/sandtable`, then `/plugin install sandtable@sandtable`.
> Every page is available as Markdown by appending .md to its URL. /llms.txt lists every page, /llms-full.txt contains the whole site, and https://sandtable.dev/mcp is a search MCP server for these docs.

# Noop baseline and lint: check a suite before model runs

> Know the floor before you read a score: run the do-nothing agent, and prove offline that every task is ready, solvable, non-vacuous and graded.

A score means little until you know what doing nothing scores, and whether each task could be passed at all. sandtable gives you both before you spend anything on model calls.

## The noop baseline

`--agent noop` never calls a tool and answers every message with "Sorry, I can't help with that right now." With a scripted user, it runs offline:

```bash theme={null}
npx sandtable-ai run ./my-domain --agent noop --user scripted
```

It passes exactly the tasks whose success is "nothing happened": refusals, handoffs that only need a polite answer, and missing-information cases. On the freight example it scores **32%**:

```text theme={null}
sandtable · freight-carrier-sales · agent noop · user scripted · k=1

  task                                trials  pass
  book-at-posted-rate                 ✗       0/1
  negotiate-within-ceiling            ✗       0/1
  walk-away-above-ceiling             ✓       1/1
  expired-insurance                   ✓       1/1
  inactive-authority                  ✓       1/1
  ...

  pass^1 32.0%
  ...
  By tag
  hard          18 tasks  pass^1   0.0%
  ...
  refusal       17 tasks  pass^1  82.4%
```

So read any agent's score on this suite against that floor, not against zero. The per-tag scores show where the floor comes from: noop gets 82% on `refusal` for free, and nothing on the `hard` tier, whose refusals all require a lookup or a handoff.

> [!TIP]
>
> Make refusal tasks discriminate. "Nothing was booked" is passed by an agent that never does anything. Add what a correct refusal requires, such as `called: [verify_carrier]` (it looked the carrier up before declining), and the do-nothing agent fails it. See [Write tasks that discriminate](/guides/write-tasks).

## `sandtable lint`

`lint` proves, without any model call, that each task is worth grading:

| Property | What it checks | Fails as |
| - | - | - |
| **graded** | the task has `state` or `called` expectations | error: nothing is graded |
| **ready** | every row the expectations need is seeded, or some tool can create or change it | error: `expect.state[i]` needs a row that is not seeded and no tool can create or change |
| **non-vacuous** | a task that expects a write is failed by a do-nothing agent | error: the seed already satisfies the expected state |
| **solvable** | the task's `reference` calls, replayed against a fresh world, pass every expectation and break no invariant | error: the reference call failed, or does not pass a check |

A task that expects a write but has no `reference` gets a warning, since its solvability is unproven. `reply_includes` expectations are skipped in the replay, because a replay has no messages.

```text theme={null}
$ npx sandtable-ai lint examples/freight
✓ book-at-posted-rate  (write · ready · solvable · noop fails)
✓ negotiate-within-ceiling  (write · ready · solvable · noop fails)
✓ walk-away-above-ceiling  (no-write · ready · solvable ? · noop passes)
...
50 tasks: 32 expect a write, 18 do not; a do-nothing agent passes 16 of 50.
0 error(s), 0 warning(s)
```

`solvable ?` means the task has no `reference` to replay. It exits 1 on any error, so it belongs in CI next to `validate --strict`.

## Why both matter

Lint catches broken tasks; noop shows how much of a suite is easy. Drafted suites need both: in the [live `init` run](/results), lint flagged both drafted tasks as not ready because a tool did not yet model a side effect. Without it, the first model-backed run would have failed those tasks for a reason that had nothing to do with the agent.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.