setup_sql: let a caller adjust a new instance's rows with SQL at reset

specs/projects/startup_sql/functional_spec.md

@@ -0,0 +1,159 @@
+---
+status: complete
+---
+
+# Functional Spec: Startup SQL
+
+## Summary
+
+`setup_sql` is a new parameter of `world.instance(...)`, and therefore of OpenEnv `reset`, of the
+HTTP API's instance creation and of `seahaven mcp --reset-options`. It holds SQL that the framework
+runs against the new instance after the fixture is copied (or the blank schema is built) and before
+any startup hook. An eval uses it to adjust the starting state of a served world without writing
+code.
+
+It reuses the startup step of instance creation. It is not a startup keyword and not an
+`instance_startup` hook: `startup` stays the world's own keyword namespace.
+
+## The interface
+
+The single string below is the preferred design. It depends on a new writable connection that
+attaches every node. If the architecture step finds that connection adds disproportionate
+complexity or risk (locking, triggers, seeding, attach limits), it may push back to the fallback:
+`setup_sql: str | Mapping[str, str] | None`, where a string is the root's SQL and an object maps a
+canonical node path to that node's SQL, run on each node's own connection. The fallback needs a
+per-world reset schema and a console change to render well; the architecture must cover both if
+it is chosen.
+
+```py
+world.instance(
+    fixture=None, *, seed=None, now=None, clock_mode=None, state_format=None,
+    control_tools=False, startup=None,
+    setup_sql: str | None = None,
+)
+```
+
+Over OpenEnv:
+
+```jsonc
+{"type": "reset", "data": {
+  "fixture": "agency",
+  "seed": 7,
+  "setup_sql": "UPDATE issues SET due_at = '2026-10-01T00:00:00.000Z' WHERE key = 'ENG-12'; DELETE FROM comments WHERE issue_id = (SELECT id FROM issues WHERE key = 'ENG-12');",
+  "startup": {"user_id": "a0b45f75-0917-49b9-9efd-d0279f2dd73d"}
+}}
+```
+
+`setup_sql` is one string. It may hold several statements separated by `;`, and they run in the
+order written. `null` or omitted means no setup SQL. An empty or whitespace-only string is accepted
+and runs nothing.
+
+### Table names in a composed world
+
+The SQL runs on one connection that sees every node, named as `inst.inspect()` names them:
+
+- the root's tables unqualified (`issues`), or as `main.issues`;
+- an added node's tables under a schema named by its canonical path, with `/` replaced by `__`:
+  `payments.charges`, `payments__tax.rates`.
+
+A statement may read and write across nodes, for example
+`INSERT INTO payments.charges (...) SELECT ... FROM orders`. A world that adds no other world only
+ever uses unqualified names.
+
+## Behaviour
+
+### When it runs
+
+Instance creation, in order:
+
+1. Copy the fixture, or build the blank schema.
+2. **Run `setup_sql`** in its own transaction, on a writable connection to the root's file with
+   every other node attached writable under its schema name. Commit, then close the connection.
+3. Open every node's transaction and run the startup hooks, as today. A hook sees the rows
+   `setup_sql` wrote: for example, SQL that inserts a user and `startup={"user_id": <that id>}`
+   together work.
+4. Commit, and the instance is ready for its first call.
+
+Step 2 commits before step 3 so that its write locks on the attached files cannot block a hook.
+Creation is still all or nothing: a failure at any step removes the whole instance directory, as
+a failing startup hook does today.
+
+### The change log and the state document
+
+- Rows `setup_sql` writes are **not** in the change log. Change-log sessions are opened only per
+  tool call; creation runs before any exists. They are part of the starting state, as rows a startup
+  hook writes are today. A grader folding the log sees only what the agent's calls changed.
+  `call_count` is unchanged.
+- The state document's envelope gains a field, `setup_sql`, beside `startup`: the string exactly as
+  given, or `null` when there was none (and before the first `reset` over a server).
+- `fixture.nodes[path].file_sha256` keeps naming the fixture file each node was copied from. A
+  reader reconstructs the starting state from `fixture` plus `setup_sql` plus `startup`.
+
+### What SQL is allowed
+
+The docs tell an author to use `INSERT`, `UPDATE` and `DELETE`, and to change the schema with a new
+world version, not with setup SQL.
+
+Runtime enforcement is part of v1, through `seahaven.sandbox.Authorizer`, which already
+default-denies everything but row reads, row writes and allowlisted functions. It refuses schema
+changes (`CREATE`, `DROP`, `ALTER`), `ATTACH`/`DETACH`, `PRAGMA` writes, transaction control
+(`BEGIN`, `COMMIT`, `SAVEPOINT`) and functions outside the `run_sql` allowlist. Read and write
+access covers every table of every node.
+
+A refusal fails loudly: instance creation aborts with a `WorldBug` whose message names the
+statement's position, what was refused, and what to do instead. For example: "setup_sql statement
+2 was refused: CREATE TABLE is a schema change. setup_sql may only read and write rows (INSERT,
+UPDATE, DELETE); change the schema in the world and bump its version."
+
+Transaction control would end the setup transaction early, and `ATTACH` and `DETACH` would change
+the setup connection's attachments, which are the framework's.
+
+`random()` and `randomblob()` draw from the root node's seeded stream, whichever node the row lands
+in, so the same `seed` and the same `setup_sql` give the same starting state.
+
+### Errors
+
+A failure aborts instance creation with `seahaven.WorldBug` and nothing is left behind, exactly as
+when a startup hook raises. Over OpenEnv the session is
+left fresh (no instance, no episode) and open for another `reset`; the frame is `EXECUTION_ERROR`.
+
+Refused before anything is copied (the checks `world.instance` already runs first):
+
+- `setup_sql` is not a string or `null`.
+
+Refused while running:
+
+- A statement SQLite rejects (syntax, constraint, missing table). The message names the
+  statement's position in the string (1-based) and SQLite's own text.
+- A statement the enforcement refuses, named the same way.
+
+## Every entry point
+
+`setup_sql` is a `world.instance` parameter, so each place that forwards instance options takes it:
+
+| Entry point | How it is passed |
+|---|---|
+| `world.instance(...)` | `setup_sql=` |
+| OpenEnv `reset` (WebSocket and `SeahavenEnv.reset`) | `"setup_sql"` key; published in the reset schema from `GET /schema` |
+| HTTP API instance creation (`PUT` body) and `seahaven serve` reset options | `"setup_sql"` key |
+| `seahaven mcp --reset-options` | `"setup_sql"` key |
+| `/ui` reset form | rendered from the reset schema as a multi-line text box; no console change |
+
+It is always available. There is no operator flag to turn it off.
+
+## Out of scope
+
+- Bound parameters (`:now`, `:seed`). Statements are plain literal SQL.
+- Setup SQL on `add_world(...)` (a host fixing a child's SQL at composition time).
+- A per-world opt-out.
+- Turning setup SQL into a fixture (freezing). Authors who reuse the same setup across many evals
+  should make a fixture instead; the docs say so.
+
+## Documentation
+
+- `serving_and_openenv.md`: a row in the reset argument table and one example.
+- `db_schema_and_fixtures.md` or `authoring.md`: when to use `setup_sql` versus a fixture or a
+  startup hook, and the advice to write rows only.
+- `state.md`: the new envelope field.
+- `reference/api.md`: the `world.instance` signature.
+- `composition.md`: the schema names an added node's tables have in `setup_sql`.

specs/projects/startup_sql/implementation_plan.md

@@ -0,0 +1,23 @@
+---
+status: complete
+---
+
+# Implementation Plan: Startup SQL
+
+## Phases
+
+- [x] Phase 1: Core. `seahaven/setup_sql.py` (`run_setup_sql`, `split_statements`,
+  `SetupAuthorizer`, `_explain`), `SETUP_STREAM`, the `setup_sql` parameter on `World.instance`,
+  `InstanceManager.create` and `Instance` (stored as `inst.setup_sql`), the pre-copy type check, and
+  the pytest marker signature. Tests in `tests/test_setup_sql.py` through `world.instance(...)`:
+  behaviour, composed worlds, FTS5, startup hooks seeing setup rows, seeding, change log, every
+  refusal and error message, rollback leaving no directory, and the unit tests.
+- [x] Phase 2: Entry points and state document. `setup_sql` in the state document envelope and
+  `SeahavenState`; OpenEnv `reset`, `SeahavenResetRequest` and `RESET_ORDER`; the stale
+  `cli/mcp.py` comment; console mock schema and smoke order; every pinned test and
+  `tests/state_v1.schema.json`. Tests for OpenEnv (in process and over the WebSocket), HTTP `PUT`,
+  `seahaven mcp --reset-options`, and the projecttracker test (SQL inserts a user, `startup` names
+  them, `create_issue` is attributed to them).
+- [x] Phase 3: Docs. Updates to six existing pages, no new page: `serving_and_openenv.md`, `db_schema_and_fixtures.md` (new short section with
+  a runnable example), `state.md`, `composition.md`, `reference/api.md`, `reference/cli.md`, per
+  the AGENTS.md docs style.

specs/projects/startup_sql/phase_plans/phase_1.md

@@ -0,0 +1,127 @@
+---
+status: complete
+---
+
+# Phase 1: Core
+
+## Overview
+
+`setup_sql: str | None` becomes a parameter of `world.instance(...)`. Instance creation runs it on
+one writable connection to the root's file with every added node attached, after the files are in
+place and before any node connection is opened, in one transaction, under an authorizer that allows
+only row reads and row writes. Startup hooks then see the committed rows. Any failure is a
+`WorldBug` naming the statement, and creation leaves no directory behind.
+
+The entry points (OpenEnv, the state document) and the docs are phases 2 and 3. HTTP and MCP derive
+their keys from the `World.instance` signature, so they take the key from this phase on; the one
+test that pins those keys is updated here so the suite stays green.
+
+## Steps
+
+1. `src/seahaven/ids.py`: add `SETUP_STREAM = b"setup"` beside the other streams, to `__all__`, and
+   one sentence in the stream comment saying it is the setup connection's door.
+2. New `src/seahaven/setup_sql.py`:
+
+   ```py
+   __all__ = ["SetupAuthorizer", "run_setup_sql", "split_statements"]
+
+   def run_setup_sql(
+       sql: str, *, root: Path, attachments: Sequence[tuple[str, Path]], clock: Clock, seed: bytes
+   ) -> None: ...
+
+   def split_statements(sql: str) -> list[str]: ...
+
+   @dataclass(frozen=True)
+   class Refused:
+       action: int
+       third: str | None
+       fourth: str | None
+       description: str      # the base class's refusal text
+
+   class SetupAuthorizer(Authorizer):
+       refused: Refused | None
+       def reset(self) -> None: ...
+       def __call__(self, action, third, fourth, database, trigger, /) -> int: ...
+
+   def _explain(refused: Refused | None, error: apsw.Error, schemas: Sequence[str]) -> tuple[str, str]
+   def _every_table(conn: apsw.Connection) -> list[str]
+   def _failure(index, count, statement, refused, error, schemas) -> WorldBug
+   ```
+
+   - `split_statements`: split on `;`, join pieces until `apsw.complete` is true, drop
+     whitespace-only pieces, keep a trailing fragment as the last statement.
+   - `run_setup_sql`: per architecture — open, `_harden`, `synchronous OFF`, busy timeout 0, random
+     functions on `SETUP_STREAM`, clock functions, `ATTACH ... AS ?` with bound parameters, list every
+     table (including FTS5 shadow tables), `BEGIN`, install the authorizer, run each statement on
+     one cursor and drain it, remove the authorizer, `COMMIT`; on an `apsw.Error` roll back and raise
+     the `WorldBug` from it; always close the connection and the clock helper.
+   - `SetupAuthorizer`: `read_only=False`, `ALLOWED_FUNCTIONS`; additionally allows the read-only
+     pragmas `db._READ_ONLY_PRAGMAS` (the list the inspection connection uses), so the error
+     message "Only read-only pragmas such as table_info are allowed" is true. Records the first
+     refusal of a statement; `reset()` clears it.
+   - `_explain`: the reason and fix per the architecture's error table; message shape
+     `setup_sql statement {i} of {n} {was refused|failed}: {reason} {fix}\n  {statement}` with the
+     statement whitespace-collapsed and cut at 200 characters with `...`.
+3. `src/seahaven/instances.py`:
+   - `_attachment_pairs(nodes: Iterable[Node], directory: Path) -> list[tuple[str, Path]]`, used by
+     `Instance._attachments()` and by `create`.
+   - `_check_setup_sql(setup_sql: object) -> None`: `WorldBug("setup_sql takes a string of SQL
+     statements separated by ';', not {type}")` unless `str` or `None`; called beside
+     `_check_startup_keywords`, before any directory exists.
+   - `create(..., setup_sql: str | None = None)`: after the copy or blank build and before the
+     `_open_node` loop, `run_setup_sql(...)` when `setup_sql is not None`, seeded with
+     `node_seed(base, ROOT_PATH)`. Docstring line.
+   - `Instance.__init__(..., setup_sql: str | None)`: stored as `self.setup_sql`.
+4. `src/seahaven/world.py` `World.instance`: `setup_sql: str | None = None` after `startup`,
+   docstring sentence, forwarded.
+5. `src/seahaven/pytest_plugin.py` `_MARKER_SIGNATURE`: add `setup_sql=None` after `startup=None`.
+6. Pins the new parameter breaks: `tests/test_cli_mcp.py` key list; `tests/test_world.py`
+   framework-parameter list; `SETUP_STREAM` added to the door lists in `tests/test_ids.py` and
+   `tests/test_composite_instance.py`.
+
+## Tests
+
+New `tests/test_setup_sql.py`, every behaviour test through `world.instance(...)`, on small inline
+worlds (a leaf `notes`-style world, a host adding `payments` which adds `tax`, and an FTS5 world).
+
+Behaviour:
+- `test_rows_written_by_setup_sql_are_there_for_the_first_call`: inserted row read by the first
+  tool call and by `inst.inspect()`.
+- `test_statements_run_in_the_order_written`: a later `UPDATE` sees an earlier `INSERT`'s row.
+- `test_a_semicolon_inside_a_literal_does_not_split`.
+- `test_unqualified_names_write_the_root_and_qualified_names_write_a_node`: `payments.charges` and
+  `payments__tax.rates`.
+- `test_one_statement_reads_one_node_and_writes_another`: `INSERT INTO payments.charges ...
+  SELECT ... FROM <root table>`.
+- `test_setup_sql_runs_on_a_copied_fixture`: a frozen fixture plus setup SQL.
+- `test_an_update_through_setup_sql_is_found_by_fts5`: `UPDATE` then a `MATCH` in the first call;
+  the old term no longer matches.
+- `test_a_startup_hook_sees_the_rows_setup_sql_wrote`: SQL inserts a user, `startup={"user_id":
+  ...}` names them, the hook reads the row.
+- `test_the_same_seed_and_setup_sql_give_the_same_random_rows`, and
+  `test_setup_sql_draws_from_a_stream_of_its_own`: an instance's own `random()` in a tool call is
+  the same with and without setup SQL that drew from `random()`; and different seeds differ.
+- `test_the_clock_functions_read_the_instances_clock`: `datetime('now')` is the instance's `now`.
+- `test_setup_sql_rows_are_not_in_the_change_log`: `change_log()` empty and `call_count == 0`.
+- `test_setup_sql_is_kept_on_the_instance`: `inst.setup_sql` is the string as given, `None` when
+  omitted.
+- `test_an_empty_string_whitespace_or_a_comment_changes_nothing` (parametrized).
+- `test_a_select_runs_and_changes_nothing`, and `test_read_only_pragmas_and_fts5_commands_are_allowed`.
+
+Refusals and errors (each asserts the whole `WorldBug` message shape: statement number of count,
+"was refused"/"failed", reason, fix):
+- `test_a_refused_statement_names_what_was_refused` parametrized over every row of the
+  architecture's Refusals table.
+- `test_a_refusal_rolls_back_earlier_statements_and_leaves_no_directory`.
+- `test_a_syntax_error_names_the_statement_and_sqlites_message`,
+  `test_a_constraint_failure_fails_the_instance`,
+  `test_a_missing_table_in_a_composed_world_names_the_schemas`, and the same in a leaf world with no
+  hint.
+- `test_a_non_string_setup_sql_is_refused_before_anything_is_copied` (no directory created).
+- `test_a_long_statement_is_cut_in_the_message`.
+
+Unit tests:
+- `split_statements` cases: literal `;`, quoted identifier, comment, trigger body, trailing fragment,
+  empty and whitespace, comment-only piece.
+- `_explain` per row of the error table, including the ATTACH fix with and without added nodes.
+- `SetupAuthorizer.reset` clears the recorded refusal and records only the first.

specs/projects/startup_sql/phase_plans/phase_2.md

@@ -0,0 +1,61 @@
+---
+status: complete
+---
+
+# Phase 2: Entry points and state document
+
+## Overview
+
+Phase 1 made `setup_sql` a `world.instance(...)` parameter. HTTP `PUT` and
+`seahaven mcp --reset-options` already take it, because both read their keys off that signature.
+This phase puts it on the OpenEnv wire (the `reset` signature and the published reset schema),
+reports it in the state document's envelope and in `SeahavenState`, updates the console mock and
+smoke test, and updates every test that pins the envelope or the reset keys. Each entry point gets
+a test that drives `setup_sql` through it.
+
+## Steps
+
+1. `src/seahaven/state.py` `envelope()`: add
+   `"setup_sql": instance.setup_sql if instance is not None else None` directly after `startup`.
+2. `src/seahaven/openenv/env.py`:
+   - `SeahavenState`: `setup_sql: str | None = Field(default=None, description=...)` after
+     `startup`; the field comment's count of null-defaulted fields becomes seven.
+   - `RESET_ORDER = ("fixture", "startup", "setup_sql", "now", "clock_mode", "state_format")`.
+   - `SeahavenResetRequest`: `setup_sql: str | None = Field(default=None, description=...)`.
+     `for_world` is unchanged.
+   - `SeahavenEnv.reset`: `setup_sql: str | None = None` after `startup`, before `**unknown`;
+     forwarded to `InstanceManager.create`; one docstring clause.
+3. `src/seahaven/cli/mcp.py`: the comment over `RESET_OPTION_KEYS` no longer says "five names".
+4. Console: `tools/console/mock/server.mjs` reset schema gains `setup_sql` (titled "Setup Sql")
+   after `startup`; `tools/console/e2e/smoke.mjs` form order includes "Setup Sql" between the
+   startup keywords and "Now", and checks it renders as a `textarea`. No console source change.
+5. Pins: `tests/test_env.py` `RESET_PARAMETERS` and `DECLARED_DESCRIPTIONS`;
+   `tests/test_reset_schema.py` property order; `tests/state_v1.schema.json` (`required`,
+   `properties` as `["string", "null"]`); `tests/test_state.py` key order and no-instance nulls;
+   `tests/test_server.py` `EPISODE` and `expected_document`; `tests/test_client.py`
+   `DOCUMENT_FRAME` and the pre-reset nulls.
+
+## Tests
+
+- `tests/test_state.py::test_the_setup_sql_is_reported_exactly_as_given`: the string as given,
+  `""` stays `""`, `None` when omitted, and the setup row is not in the log.
+- `tests/test_state.py::test_a_built_in_document_validates_against_the_published_schema`: now with
+  `setup_sql`, so the string branch of the schema is exercised.
+- `tests/test_env.py::test_setup_sql_reaches_the_instance_through_reset`: in process, on a fixture;
+  rows visible to the first call, `state.setup_sql`, empty log.
+- `tests/test_env.py::test_a_failing_setup_sql_leaves_the_session_as_a_fresh_one`: `WorldBug`, no
+  instance, old directory gone, `setup_sql` null, another `reset` works.
+- `tests/test_server.py::test_setup_sql_reaches_the_instance_over_the_wire`: a refused statement is
+  an `EXECUTION_ERROR` frame; the next reset with good SQL has the row and the state field.
+- `tests/test_server.py` whole-document tests: `setup_sql` is in `EPISODE`, so the document over the
+  WebSocket (typed and stock client) carries it, and its row is not in the log.
+- `tests/test_reset_schema.py`: `setup_sql` in the published order, as a nullable string titled
+  "Setup Sql".
+- `tests/test_http_server.py::test_a_put_body_with_setup_sql_starts_the_instance_with_its_rows`, and
+  two new rows in the refused-`PUT` table (non-string, refused statement) answering 400.
+- `tests/test_cli_mcp.py::test_setup_sql_is_a_reset_option` and
+  `tests/test_mcp_process.py::test_reset_options_carry_setup_sql_to_the_instance` (a real MCP
+  process; the first call reads the row).
+- `worlds/projecttracker/tests/test_issues.py::test_a_user_setup_sql_inserts_can_be_the_viewer_who_files_an_issue`:
+  `agency` plus SQL inserting a user plus `startup={"user_id": ...}`; `create_issue` is attributed
+  to that user, and `users` is not in the change log.