Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,11 +100,44 @@ That guard exists because the opposite behaviour hid a bug for five months (fixe

Every failure is logged as `SQL statement failed` with `reasons` and `sql_shape` and counted in `mcp_sql_failures_total`. `sqlShape()` removes all customer content — literals become `?`/`0`, table and column names become `x` — so the log shows which constructs models get wrong without storing data. Never log the raw statement or its parameters. How to read and act on the log: README, "SQL failure log".

`runSql()` also adds hints to SeaTable's errors, in `ERROR_HINTS`: `no such column/table` points to `get_schema` (reason `unknown_name`), `failed to convert value` and `type mismatch` explain select options and the date format (reason `invalid_value`). In production the failures were guessed names, not dialect problems, and models repeated them.

`truncationNotice()` adds a `notice` field to `query_sql` results that may be cut, and `list_rows` does the same when `has_more` is set. A `LIMIT` below 100 is treated as a deliberate top-N and stays quiet.

SeaTable SQL behaviour verified against 7.0, which the tool texts rely on:

- A `SELECT` without `LIMIT` returns 100 rows, also with `GROUP BY`, and does not say it cut. `LIMIT` goes up to 10,000 and `OFFSET` works (`LIMIT n OFFSET m` and `LIMIT m, n`).
- A link column can't be a join condition: `= other.col` and `= other._id` return nothing, `HAS ANY OF (other.col)` is an error. `GROUP BY` on it works, `` `Link` = 'display value' `` filters, and the link column on the other table supports `IS NULL`.
- `ORDER BY` accepts columns outside the `SELECT` list.
- A deleted API token answers 403 on a self-hosted 7.0 and 404 on SeaTable Cloud. `TokenValidator.check()` treats both, and 401, as revoked.

To see how real models use the tools, run the series in `docs/testing-with-models.md`.

### Server instructions and tool annotations

The `instructions` in `src/mcp/server.ts` reach every client, and the SeaTable AI chat plugin (`seatable-plugin-ai-chat`) embeds them in its system prompt instead of keeping its own SQL rules. A change there takes effect for all clients on the next deployment, without a plugin release. If a rule moves here from a client, deploy the server first.

`tests/server.spec.ts` pins every tool as read-only, additive or destructive. A tool that can overwrite or remove existing data must be destructive; directory reviewers call the tools and compare against the hints.

### Container

The image's `CMD` runs `node --permission --allow-fs-read=/app`. Writing files, reading outside `/app`, child processes, worker threads and native addons fail with `ERR_ACCESS_DENIED`, so a new feature that needs one of these has to change the flags deliberately. The image has no `wget`, `nc`, npm or apk; health checks use `node`. See README, "Container hardening".

### Schema Utilities

- `src/schema/map.ts` — converts SeaTable metadata to `GenericSchema` format
- `src/schema/validate.ts` — validates row data against schema before writes

## Releases

The workflow runs on tags. `testing-vX.Y.Z` only builds the image `testing-X.Y.Z`. `release-vX.Y.Z` also publishes `:latest`, npm (irreversible) and the MCP Registry.

1. Bump the version in the PR (`npm version X.Y.Z --no-git-tag-version`). The npm job fails if the tag and `package.json` differ.
2. After the merge, build the image and scan it locally with the workflow's Trivy parameters. The workflow pushes the image **before** it scans, so a red scan has already published.
3. Set an annotated tag `release-vX.Y.Z` on `main`. Its message is the changelog: name breaking changes and what deployments with their own compose file have to do.

The commit messages of this public repo should not contain table or column names from customer logs.

## Known Limitations

- HTTP sessions are in-memory and do not survive restarts.
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -486,6 +486,8 @@ node scripts/mcp-call.cjs list_tables '{}'
node scripts/mcp-call.cjs list_rows '{"table": "Tasks", "page_size": 5}'
```

To check how real models use the SQL tools, see [Testing the SQL tools with a model](docs/testing-with-models.md).

## Troubleshooting

| Issue | Solution |
Expand Down
68 changes: 68 additions & 0 deletions docs/testing-with-models.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# Testing the SQL tools with a model

Unit tests can't show whether a model actually uses the tools well. Run this series after changing tool descriptions, the server instructions or the SQL checks, and ideally with a small model too: a strong model hides gaps in the instructions that a weaker one runs into.

Check two things for each prompt: whether the answer is right, and which calls it took. A wrong answer can come without any error, for example from a result cut at 100 rows.

## Test data

Create a base with two tables. Import the CSV files, then set the column types.

**Kunden:** `Umsatz` number, `Status` single select (`aktiv`, `inaktiv`), `Kunde seit` date.

```csv
Name,Ort,Umsatz,Status,Kunde seit
Müller GmbH,Mainz,12000,aktiv,2021-03-01
Schmidt AG,Wiesbaden,4500,inaktiv,2019-07-15
Weber KG,Mainz,8800,aktiv,2023-11-20
Fischer OHG,Frankfurt,300,aktiv,2024-01-05
Becker & Söhne,Frankfurt,15200,aktiv,2020-05-12
Hoffmann IT,Darmstadt,2100,aktiv,2022-09-30
Klein Logistik,Mainz,6700,inaktiv,2018-02-14
Wagner Bau,Wiesbaden,950,aktiv,2025-04-01
```

**Bestellungen:** `Kunde` link to `Kunden`, `Betrag` number, `Datum` date. Use a real link column: it is what makes prompts 1 and 2 hard. SeaTable then also creates a link column `Bestellungen` in `Kunden`.

```csv
Nr,Kunde,Betrag,Datum
B-1001,Müller GmbH,500,2026-07-03
B-1002,Müller GmbH,700,2026-07-18
B-1003,Weber KG,250,2026-07-22
B-1004,Becker & Söhne,3200,2026-08-05
B-1005,Becker & Söhne,1800,2026-08-19
B-1006,Hoffmann IT,420,2026-08-27
B-1007,Müller GmbH,950,2026-09-02
B-1008,Weber KG,1300,2026-09-15
B-1009,Becker & Söhne,640,2026-10-08
B-1010,Hoffmann IT,2100,2026-10-21
```

## Prompts

The prompts are in German on purpose, to match the data. Run prompt 7 last, since it changes data, and set both customers back to `aktiv` afterwards.

| # | Prompt | Tempts the model to | Good path | Right answer |
| --- | ---------------------------------------------------------------------------------- | --------------------------------------------------- | ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ |
| 1 | Welche Kunden haben noch keine Bestellung? | subquery, `LEFT JOIN` | one `query_sql`: ``WHERE `Bestellungen` IS NULL`` | Schmidt AG, Fischer OHG, Klein Logistik, Wagner Bau |
| 2 | Zeig mir pro Kunde die Summe aller Bestellungen und den Ort. | `JOIN`, a join on the link column (returns nothing) | `` GROUP BY `Kunde` ``, then a second query for the cities | Becker & Söhne 5.640 (Frankfurt), Hoffmann IT 2.520 (Darmstadt), Müller GmbH 2.150 (Mainz), Weber KG 1.550 (Mainz) |
| 3 | Welcher Kunde hat die größte Einzelbestellung? | `WHERE x = (SELECT MAX…)` | `ORDER BY … DESC LIMIT 1` | Becker & Söhne, B-1004, 3.200 |
| 4 | Teile die Kunden nach Umsatz ein: ab 10.000 groß, ab 2.000 mittel, darunter klein. | `CASE WHEN` | `ifs()`, or read the rows and classify them | groß: Müller, Becker · mittel: Weber, Klein, Schmidt, Hoffmann · klein: Wagner, Fischer |
| 5 | Zeig die ersten drei Buchstaben jedes Namens zusammen mit dem Ort in einer Spalte. | `SUBSTRING`, `CONCAT` | `concatenate(mid(…), …)` | e.g. "Mül Mainz", "Bec Frankfurt" |
| 6 | Wie viele Bestellungen gab es pro Monat? | `DATE_FORMAT`, `EXTRACT` | `month()`/`year()` with `GROUP BY` | July 3, August 3, September 2, October 2 |
| 7 | Setze alle Kunden aus Frankfurt auf inaktiv. | sending the `UPDATE` to `query_sql` | `execute_sql` with `UPDATE … WHERE` | 2 rows: Fischer OHG, Becker & Söhne |

## What to watch

Follow the server log while testing:

```bash
docker logs -f --since 1m seatable-mcp 2>&1 | grep --line-buffered -E '"tool":|SQL statement failed' \
| jq -rc '[.time[11:19], .tool, .msg, (.reasons // empty | join(",")), (.sql_shape // empty)] | join(" ")'
```

- **Ideally no rejection at all**: the model follows the description and asks simple queries.
- **After a failed call**, the next one should restructure the statement, not repeat it with small changes.
- **A row count of exactly 100** without a `notice` in the result would be a bug in the truncation notice.

For the cause of a failure, see README, "SQL failure log".
Loading