feat(traverse): one walk() for any AST; walkSql(text) in plpgsql-parser

.agents/skills/ast-traversal/SKILL.md

@@ -0,0 +1,196 @@
+---
+name: ast-traversal
+description: How to walk PostgreSQL SQL and PL/pgSQL ASTs in this monorepo — choosing between walk, walkSql, walkSqlAst, walkPlpgsqlAst, and traverse; statement context, visitor composition, abort, and mutation. Use when reading, validating, or rewriting SQL/PL/pgSQL ASTs.
+---
+
+# AST Traversal
+
+Everything in this repo that inspects or rewrites SQL goes through one of five
+functions. Pick the right one first; the rest of the work follows from it.
+
+## Choosing a function
+
+| Function | Package | Walks | Mutates |
+|---|---|---|---|
+| `walk(ast, visitors, opts?)` | `@pgsql/traverse` | any AST — parsed script, `ParseResult`, SQL node, PL/pgSQL node | no |
+| `walkSql(text, visitors, opts?)` | `plpgsql-parser` | SQL **text**: parses, hydrates PL/pgSQL bodies, then `walk`s | no |
+| `walkSqlAst(ast, visitor)` | `@pgsql/traverse` | SQL AST only | no |
+| `walkPlpgsqlAst(ast, visitor, opts?)` | `@pgsql/traverse` | PL/pgSQL AST only | no |
+| `traverse(ast, mutableVisitor)` | `@pgsql/traverse` | SQL AST, with insert/remove/replace | yes |
+
+Decision rules:
+
+- **Have a string?** `walkSql`. It is the only entry point that parses.
+- **Have an AST and don't care which universe it is from?** `walk`.
+- **Want exactly one node universe and no statement context?** `walkSqlAst` or
+  `walkPlpgsqlAst`. These are the primitives `walk` is built on; reach for them
+  when writing a reusable visitor that some other walker will drive (this is why
+  `@pgsql/transform` exports visitor *factories* rather than walkers).
+- **Need to change the tree structurally?** `traverse`. Note that read-only
+  walkers hand you the real node objects, so field-level edits (renaming a
+  schema, rewriting a name list) work under `walk` too — `traverse` is for
+  inserting, removing, and replacing nodes.
+
+Never hand-roll `transformSync(sql, ..., { hydrate: true })` plus a
+per-statement loop plus a PL/pgSQL walk. That harness *is* `walk`.
+
+## Visitors
+
+A visitor is either a callback (fires on every node) or an object keyed by node
+tag. SQL tags and `PLpgSQL_*` tags may be mixed in one object — `walk` routes
+each node to the right walker.
+
+```ts
+import { walk } from '@pgsql/traverse';
+
+walk(ast, {
+  RangeVar: (path) => console.log(path.node.relname),
+  PLpgSQL_stmt_dynexecute: (path) => console.log('dynamic EXECUTE', path.node)
+});
+```
+
+Pass an **array** of visitors to run independent concerns in a single parse:
+
+```ts
+walkSql(sql, [blockedSchemas, readOnlySchemas, blockedFunctions]);
+```
+
+Each callback receives `(path, ctx)`:
+
+- `path` — a `NodePath` (SQL) or `PlpgsqlNodePath` (PL/pgSQL): `tag`, `node`,
+  `parent`, `keyPath`. This is *structure*: where the node sits.
+- `ctx` — a `WalkContext`: `stmtTag`, `stmtIndex`, `isWrite`, `isRead`,
+  `insideFunction`, `functionName`, `abort()`. This is *situation*: what the
+  node is part of.
+
+The reserved `statement` key fires once per top-level statement, before its
+children — the hook for per-statement setup or classification.
+
+## Control flow
+
+Two distinct mechanisms, do not confuse them:
+
+- `return false` — skip this node's children, keep walking siblings.
+- `ctx.abort(reason?)` — end the whole walk. `walk`/`walkSql` return
+  `{ aborted, reason, reasons }`. This is what a validator wants: the first
+  rejection ends the work.
+
+```ts
+const result = walkSql(sql, {
+  RangeVar: (path, ctx) => {
+    if (ctx.isWrite && path.node.schemaname === 'audit') {
+      ctx.abort(`cannot write to ${path.node.schemaname}`);
+    }
+  }
+});
+if (result.aborted) reject(result.reason);
+```
+
+Unparseable input from `walkSql` comes back as `{ aborted: true, reason }`
+rather than a thrown error, so "rejected" and "not understood" are one code path.
+
+## Worked examples
+
+### Collect every table a script touches, including inside function bodies
+
+```ts
+import { loadModule, walkSql } from 'plpgsql-parser';
+
+await loadModule(); // once per process: libpg-query is WASM
+
+const tables = new Set<string>();
+walkSql(sql, {
+  RangeVar: (path, ctx) => {
+    const name = path.node.schemaname
+      ? `${path.node.schemaname}.${path.node.relname}`
+      : path.node.relname;
+    tables.add(ctx.insideFunction ? `${name} (via ${ctx.functionName})` : name);
+  }
+});
+```
+
+### Classify statements without a second parse
+
+```ts
+walkSql(sql, {
+  statement: (path, ctx) => {
+    console.log(ctx.stmtIndex, path.tag, ctx.isWrite ? 'write' : 'read');
+  }
+});
+```
+
+### Rewrite schema names on a parsed script
+
+Field-level rewrites work through the read-only walker because `path.node` is the
+live node. This is exactly how `@pgsql/transform` renames schemas:
+
+```ts
+import { transformSync } from 'plpgsql-parser';
+import { walk } from '@pgsql/traverse';
+
+const out = transformSync(sql, (ctx) => {
+  walk(ctx, {
+    RangeVar: (path) => {
+      const to = mapping.get(path.node.schemaname);
+      if (to) path.node.schemaname = to;
+    }
+  });
+}, { hydrate: true, pretty: true });
+```
+
+`transformSync` gives the callback a parsed script (`{ sql, functions }`), which
+`walk` dispatches over directly — statements first, then every hydrated body.
+
+### Restructure the tree
+
+```ts
+import { traverse } from '@pgsql/traverse';
+
+traverse(ast, {
+  RawStmt: {
+    enter: (path) => {
+      if (isRedundant(path.node)) path.remove();
+    }
+  }
+});
+```
+
+### Drive a reusable visitor from a primitive
+
+When a caller already owns the traversal loop (per-statement state, custom
+ordering), build the visitor separately and hand it to the primitive:
+
+```ts
+import { walkSqlAst } from '@pgsql/traverse';
+
+for (const stmt of parseResult.stmts) {
+  walkSqlAst(stmt.stmt, createFactsVisitor(factsFor(stmt)));
+}
+```
+
+## Options
+
+`walk` and `walkSql` share:
+
+- `walkFunctionBodies` (default `true`) — walk hydrated PL/pgSQL bodies. Under
+  `walkSql`, `false` also skips the PL/pgSQL parse, so turn it off when the
+  visitors only care about top-level SQL.
+- `walkSqlExpressions` (default `true`) — recurse into the SQL expressions inside
+  those bodies.
+- `sqlVisitor` — override the visitor used for those SQL expressions.
+
+## Gotchas
+
+- **Call `loadModule()` before any parse.** `libpg-query` is WASM; `parseSync` /
+  `walkSql` throw `WASM module not initialized` otherwise.
+- **PL/pgSQL bodies are opaque until hydrated.** Without `{ hydrate: true }` a
+  function body is a query string, and no SQL visitor will ever fire inside it.
+- **Untagged typed fields exist.** `CreatePolicyStmt.table` is a bare `RangeVar`
+  with no `{ RangeVar: ... }` wrapper. `walkSqlAst` handles this via the runtime
+  schema, which is why hand-written recursion over `Object.keys` misses nodes.
+- **`@pgsql/traverse` must stay parser-free.** It depends on types only; adding a
+  parser dependency would cycle and pull WASM into the leaf package. Anything
+  that needs to parse belongs in `plpgsql-parser` or above.
+- **Fixtures are the regression gate for transform work.** After changing any
+  traversal in `@pgsql/transform`, `pnpm --filter @pgsql/transform test` output
+  must be byte-identical; see the `testing-fixtures` skill.

.agents/skills/code-generation/SKILL.md

@@ -13,7 +13,7 @@ Three packages generate TypeScript from the PostgreSQL protobuf definition at `_
 | Package | Script | What it generates |
 |---------|--------|-------------------|
 | `@pgsql/utils` | `npm run build:proto` | AST helper functions (`src/`), wrapped helpers (`wrapped.ts`), runtime schema (`runtime-schema.ts`) |
-| `@pgsql/traverse` | `npm run build:proto` | Visitor-pattern traversal utilities |
+| `@pgsql/traverse` | `npm run build:proto` | Runtime schema driving the SQL walker (`walk` / `walkSqlAst`); see the `ast-traversal` skill |
 | `@pgsql/transform-ast` | `npm run build:proto` | Multi-version AST transformer utilities |
 
 Each package has a `scripts/pg-proto-parser.ts` that configures `PgProtoParser` with package-specific options (which features to enable, output paths, type sources).

.agents/skills/testing-fixtures/SKILL.md

@@ -168,6 +168,30 @@ npm run kitchen-sink   # generate transform-specific kitchen-sink tests
 npm run test:ast       # run AST round-trip validation
 ```
 
+## Changing a Walker
+
+`@pgsql/transform` drives its whole pipeline (schema mapping, routing, qualify,
+role and extension transforms, round-trip validation) through the walkers in
+`@pgsql/traverse` — see the `ast-traversal` skill. Its fixtures are therefore the
+regression gate for any traversal change, including ones made in another package:
+
+```bash
+pnpm --filter @pgsql/traverse test
+pnpm --filter plpgsql-parser test
+pnpm --filter @pgsql/transform test   # fixtures + snapshots must be unchanged
+```
+
+A traversal change that alters fixture output is a behavior change, not a
+refactor. Two failure modes to look for specifically:
+
+- **Nodes no longer reached** — untagged typed fields (e.g. `CreatePolicyStmt.table`)
+  are only found via the runtime schema, so a hand-rolled recursion silently drops
+  them and a fixture loses a rename.
+- **Nodes reached twice** — a visitor fired both by an outer walk and by a nested
+  one double-applies edits.
+
+Never edit a fixture or snapshot to make a walker change pass.
+
 ## Package Scripts Reference
 
 ### `packages/deparser` (primary fixture pipeline)

.github/workflows/run-tests.yaml

@@ -23,6 +23,7 @@ jobs:
           - pg-proto-parser
           - '@pgsql/quotes'
           - '@pgsql/transform-ast'
+          - '@pgsql/traverse'
           - '@pgsql/transform'
           - '@pgsql/scripts'
     steps:

AGENTS.md

@@ -10,12 +10,12 @@ A pnpm monorepo for PostgreSQL AST parsing, deparsing, and code generation. All
 |---------|-----------|---------|
 | `pgsql-parser` | `packages/parser` | Parse SQL to AST (wraps `libpg-query` WASM) |
 | `pgsql-deparser` | `packages/deparser` | Convert AST back to SQL (pure TypeScript) |
-| `plpgsql-parser` | `packages/plpgsql-parser` | Parse PL/pgSQL to AST |
+| `plpgsql-parser` | `packages/plpgsql-parser` | Parse PL/pgSQL to AST; `walkSql(text, ...)` for text-in traversal |
 | `plpgsql-deparser` | `packages/plpgsql-deparser` | Convert PL/pgSQL AST back to SQL |
 | `pgsql-types` | `packages/pgsql-types` | Narrowed TypeScript types inferred from SQL fixtures |
 | `@pgsql/types` | (published from proto-parser codegen) | Core TypeScript type definitions for PostgreSQL AST nodes |
 | `@pgsql/utils` | `packages/utils` | Type-safe AST node creation utilities |
-| `@pgsql/traverse` | `packages/traverse` | Visitor-pattern AST traversal |
+| `@pgsql/traverse` | `packages/traverse` | Visitor-pattern traversal of SQL and PL/pgSQL ASTs: `walk`, `walkSqlAst`, `walkPlpgsqlAst`, `traverse` |
 | `@pgsql/transform-ast` | `packages/transform-ast` | Multi-version AST transformer (PG 13-17) |
 | `@pgsql/transform` | `packages/transform` | SQL schema transformation, statement classification (AST facts), qualification, round-trip validation |
 | `@pgsql/quotes` | `packages/quotes` | SQL identifier/string quoting and keyword classification |
@@ -37,6 +37,7 @@ Detailed workflow documentation lives in `.agents/skills/`:
 
 | Skill | Path | Covers |
 |-------|------|--------|
+| **AST Traversal** | `.agents/skills/ast-traversal/SKILL.md` | Walking SQL and PL/pgSQL ASTs: choosing `walk` / `walkSql` / `walkSqlAst` / `walkPlpgsqlAst` / `traverse`, statement context, visitor composition, abort, mutation |
 | **Testing & Fixtures** | `.agents/skills/testing-fixtures/SKILL.md` | Fixture-based testing pipeline, adding new test fixtures, kitchen-sink workflow, PL/pgSQL fixtures, transform tests |
 | **Code Generation** | `.agents/skills/code-generation/SKILL.md` | Protobuf codegen (`build:proto`), type inference/generation (`pgsql-types`), keyword generation (`@pgsql/quotes`), version-specific deparsers |
 
@@ -115,6 +116,12 @@ Version configuration lives in `config/versions.json` — maps PG versions (13-1
 
 - TypeScript throughout, compiled to both CJS and ESM
 - `@pgsql/types` provides all AST node types — use them for type safety
+- Traversal: reach for `walk` from `@pgsql/traverse` (any AST: SQL, PL/pgSQL, or a
+  parsed script) or `walkSql` from `plpgsql-parser` (SQL text). Use the
+  `walkSqlAst` / `walkPlpgsqlAst` primitives only when you deliberately want a
+  single node universe with no statement context, and `traverse` when you need to
+  mutate. Never hand-roll a `transformSync(..., { hydrate: true })` +
+  per-statement loop harness — that is what `walk` is for
 - `@pgsql/quotes` handles SQL identifier quoting — use `QuoteUtils` methods
 - Test files go in `__tests__/` within each package
 - Fixture SQL files go in `__fixtures__/kitchen-sink/` (see testing-fixtures skill)

README.md

@@ -149,19 +149,30 @@ const walker: Walker = (path: NodePath) => {
 
 walk(ast, walker);
 
-// Using a visitor object (recommended for multiple node types)
-const visitor: Visitor = {
+// Using a visitor object (recommended for multiple node types).
+// SQL and PL/pgSQL node tags may be mixed; the second argument to each
+// callback describes the statement the node belongs to.
+walk(ast, {
   SelectStmt: (path) => {
     console.log('SELECT statement:', path.node);
   },
-  RangeVar: (path) => {
+  RangeVar: (path, ctx) => {
     console.log('Table:', path.node.relname);
-    console.log('Path to table:', path.path);
-    console.log('Parent node:', path.parent?.tag);
-  }
-};
+    console.log('Is a write target:', ctx.isWrite);
+    console.log('Inside function:', ctx.functionName);
+  },
+  PLpgSQL_stmt_dynexecute: (_path, ctx) => ctx.abort('dynamic EXECUTE')
+});
+```
+
+Starting from SQL **text** instead of an AST? Use `walkSql` from
+`plpgsql-parser`, which parses, hydrates PL/pgSQL bodies, and then walks:
+
+```typescript
+import { loadModule, walkSql } from 'plpgsql-parser';
 
-walk(ast, visitor);
+await loadModule();
+walkSql('SELECT * FROM users', { RangeVar: (path) => console.log(path.node.relname) });
 ```
 
 ## 📦 Packages 
@@ -175,7 +186,7 @@ walk(ast, visitor);
 | [**pg-proto-parser**](./packages/proto-parser) | PostgreSQL protobuf parser and code generator | • Generate TypeScript interfaces from protobuf<br>• Create enum mappings and utilities<br>• AST helper generation |
 | [**@pgsql/transform-ast**](./packages/transform-ast) | Multi-version PostgreSQL AST transformer | • Transform ASTs between PostgreSQL versions (13→17)<br>• Single source of truth deparser pipeline<br>• Backward compatibility for legacy SQL |
 | [**@pgsql/transform**](./packages/transform) | SQL transformation & classification | • Schema-name rewriting (incl. PL/pgSQL bodies)<br>• Per-statement AST facts (`classifyStatements`)<br>• Qualification & round-trip validation |
-| [**@pgsql/traverse**](./packages/traverse) | PostgreSQL AST traversal utilities | • Visitor pattern for traversing PostgreSQL AST nodes<br>• NodePath context with parent/path information<br>• Runtime schema-based precise traversal |
+| [**@pgsql/traverse**](./packages/traverse) | SQL + PL/pgSQL AST traversal | • One `walk()` for SQL ASTs, PL/pgSQL ASTs, and parsed scripts<br>• `NodePath` structure plus statement context (`isWrite`, `insideFunction`, ...)<br>• Visitor composition, skip-children, and whole-walk abort |
 
 ## 🛠️ Development