feat(traverse): one walk() for any AST; walkSql(text) in plpgsql-parser - #329
Merged
Conversation
- move the PL/pgSQL walker down into @pgsql/traverse as walkPlpgsqlAst (types-only dep on plpgsql-deparser, no parser, no cycle) - rename the SQL primitive to walkSqlAst; add walk(), which dispatches over parsed scripts, ParseResults, SQL nodes and PL/pgSQL nodes, threads a WalkContext (stmtTag/isWrite/insideFunction/functionName), composes visitor arrays and supports ctx.abort() - add walkSql(sqlText, ...) to plpgsql-parser: parse + hydrate + walk - remove visit() and walkParsedScript() - collapse the hand-rolled hydrate harnesses in @pgsql/transform onto walk() - docs: traverse/plpgsql-parser/transform READMEs, root README, AGENTS, new ast-traversal skill
Contributor
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
There were two functions named
walk(SQL AST in@pgsql/traverse, PL/pgSQL AST inplpgsql-parser) and no way to walk a SQL string. So the parse+hydrate+iterate harness got hand-rolled 7 times inside@pgsql/transformalone:Now there is one walker per axis:
walk(ast, visitors, opts?)@pgsql/traverseParseResult, SQL node, PL/pgSQL node, arraywalkSqlAst(ast, visitor)@pgsql/traversewalkPlpgsqlAst(ast, visitor, opts?)@pgsql/traversetraverse(ast, mutableVisitor)@pgsql/traversewalkSql(text, visitors, opts?)plpgsql-parserwalkThe PL/pgSQL walker moved down into
@pgsql/traverse(src/plpgsql.ts). That's the enabling move: it only ever needed PL/pgSQL types, which live inplpgsql-deparser(no dep on@pgsql/traverse), not the parser — so no cycle and no WASM in the leaf package. Only text-in traversal stays up inplpgsql-parser, since only it can parse.walkadds the three things every caller was reimplementing:return false(skip children) andctx.abort()(end the walk) are deliberately different mechanisms; visitor arrays let N independent policies share one parse.One implementation note worth flagging, because the obvious version is wrong:
walkdoes not iterateparseResult.stmtsand callwalkSqlAstper statement. Doing that skips bareRawStmtentries and every untagged typed field (CreatePolicyStmt.tableis a rawRangeVar) that only the runtime schema knows how to reach. Instead it makes a singlewalkSqlAstpass over the whole parse result and derives statement context on the fly — traversal is depth-first and statements are sequential, so every node after aRawStmtbelongs to it.Removed, no shims:
visit()(superseded bywalk— noNodePath, no skip-children, and itsObject.keysrecursion misses untagged typed fields; its tests were converted towalk) andwalkParsedScript()(absorbed bywalk).@pgsql/transformis unchanged in behavior: thewalk as walkSqlaliases becamewalkSqlAstsowalkSqlhas exactly one meaning ecosystem-wide, and the uniform-visitor harnesses collapsed ontowalk. Passes that keep genuine per-statement state (CTE names inqualifyUnqualified, per-statement facts inclassifyStatements) still drive the primitives directly. All 252 transform tests and 4 snapshots pass untouched — that is the regression gate.Docs:
@pgsql/traverse,plpgsql-parserand@pgsql/transformREADMEs, root README,AGENTS.md, and a new.agents/skills/ast-traversal/SKILL.mdcovering which function to pick, visitor composition, skip-vs-abort, and the gotchas (hydration, untagged typed fields,loadModule()). The transform README now states the boundary explicitly: transform owns routing/classification/qualification/round-trip policy,@pgsql/traverseowns reaching the nodes.Also adds
@pgsql/traverseto the CI test matrix — its 36 tests weren't running there.Versions are minor:
@pgsql/traverse18.5.0,plpgsql-parser18.3.0,@pgsql/transform18.12.0 (major stays reserved for PostgreSQL versions).Plan: constructive-io/constructive-planning#1336. After publish,
constructive-dbdeletesparsing/sql-ast(a fork of this that also under-translates schema names in the wire proxy) and pointsproxy/sql-filteratwalkSql.Testing
pnpm buildclean; every package's suite passes (@pgsql/traverse36,plpgsql-parser322 incl. 19 newwalkSqltests ported from the downstream fork's coverage,@pgsql/transform252 + 4 snapshots,pgsql-deparser715,@pgsql/transform-ast1437, …).pnpm lintis broken repo-wide onmain— ESLint 9 is installed but the repo only has.eslintrc.jsonand no flat config; CI does not lint. Left alone here.Link to Devin session: https://app.devin.ai/sessions/2aa454e560904773a0a6955712e0aeda
Requested by: @pyramation