feat(lang): ERB template plugin (tree-sitter-embedded-template)

[l3acon]

Closes #100

What

Tier 1 language plugin for ERB (Embedded Ruby) templates. Extracts variable references, fact lookups, scope accesses, and control flow from .erb files into the rgctl knowledge graph.

Commits

1. feat(lang): add ERB template plugin — crates/rgctl-lang-erb/ with tree-sitter-embedded-template v0.25.0. Extracts @variable → UsesVariable, @facts[...] → UsesFact, scope['x::y'] → References. Block tier classification (T1–T4) and Jinja2 translation hints in metadata. AST coverage manifest. 8 unit tests.

2. feat(profile): add theforeman corpus — Added 12-module theforeman ecosystem to scripts/fetch-profile-repos.sh for Puppet+ERB Gate B profiling. Updated AGENTS.md table.

Cold profile

Corpus	Files	Nodes	Edges	Wall	Peak RSS
theforeman (Puppet+ERB+Ruby)	602	5,857	12,442	1.6s	71 MB
Linux kernel (Gate A)	64k	—	—	163.9s	—

Linux Gate A: 163.9s on test machine (baseline 145s on ref M3 Pro). Zero .erb files in kernel — no codepath difference from lang-support branch.

AGENTS.md compliance

• No unwrap() in library paths
• AST coverage manifest + bundled_specs registration
• languages.toml entry
• Typed graph edges (no ad-hoc strings)
• Parallel ingest compatible (per-file extraction)
• Gate B corpus in fetch-profile-repos.sh
• theforeman added to AGENTS.md Gate B table

Files changed

 Cargo.toml                                    (workspace member + dep)
 crates/rgctl-lang-erb/Cargo.toml              (new crate)
 crates/rgctl-lang-erb/erb-ast-coverage.json   (AST coverage manifest)
 crates/rgctl-lang-erb/src/ast_coverage.rs     (drift test)
 crates/rgctl-lang-erb/src/lib.rs              (registration)
 crates/rgctl-lang-erb/src/plugin.rs           (core plugin ~500 lines)
 crates/rgctl-ast-coverage/Cargo.toml          (grammar dep)
 crates/rgctl-ast-coverage/src/lib.rs          (bundled_specs entry)
 crates/rgctl-languages/Cargo.toml             (dep)
 crates/rgctl-languages/src/lib.rs             (register + test)
 languages.toml                                (erb entry)
 scripts/fetch-profile-repos.sh                (theforeman corpus)
 AGENTS.md                                     (Gate B table)

Made with Cursor

[sshaaf]

Thanks for this PR — really nice direction for Puppet/ERB workflows, and the packaging side looks solid.

I reviewed locally on a95b603 (worktree build of your branch) and ran cold profiles + a small ERB fixture. Sharing findings so we can land this cleanly — no rush, and happy to help think through the graph-builder bits.

What’s working well

• Crate wiring looks good: languages.toml, rgctl-languages registration, AST coverage manifest + bundled_specs.
• cargo test -p rgctl-lang-erb --release — 8/8 passed.
• theforeman fetch script + AGENTS table entry are a helpful dogfood path.
• Local cold profiles on this machine:

Run	Files	Nodes	Edges	wall_secs	peak RSS
Linux Gate A (default)	71 309	2.70M	8.23M	155.4	~14 GB
ERB only (-l erb -e spec,vendor)	121	2 290	4 419	0.33	42 MB
Puppet+ERB+Ruby	672	6 218	13 307	2.39	74 MB

Gate A budget here is 145s +10% ≈ 159.5s, so 155.4s passes on this box. Your reported 163.9s is understandable machine variance — worth a re-check on the same class of machine before we treat Linux as a hard fail.

Main ask before merge: graph edges after discover

Unit tests correctly assert Relation structs from extract_all, but after a real discover the headline edges don’t currently survive into the snapshot.

On a minimal fixture (@hostname, @facts[...], scope[...]):

Extracted relation	After discover / GQL
UsesVariable (@var → $var)	dropped (MATCH (a)-[:USES]->(b) → 0)
UsesFact	dropped
References (scope[...])	present, but as File → Class stub

What seems to be going on in rgctl-extraction’s graph_builder:

1. UsesVariable / UsesFact are not included in relation_allows_external_stub, so unresolved targets never get stubs and the edges are discarded.
2. stub_node_type_for_target doesn’t honor puppetvariable / puppetfact hints (falls through to Class).
3. Relation from is the file path, so even successful edges hang off File rather than the erb:…@L block symbols — using the block QN as from would make blast-radius / GQL much more useful.

Once those land, a tiny discover+GQL smoke (or langfeatures-style test) that asserts USES / USES_FACT / REFERENCES after ingest would lock it in.

Smaller notes (non-blocking)

• Gate B framing: theforeman is a great smoke corpus (~670 indexed files / ~122 .erb), but it’s not really O(10⁴). Worth dialing AGENTS / PR wording to “profile corpus / smoke” until there’s a larger ERB set and/or an ignored cold gate in cold_profile_gates.rs.
• Tier 1 depth: this is a strong Layer A–style extract + metadata (Jinja hints are a nice touch). Fine as a first slice — just don’t need to claim full CFG/taint/Layer F yet.
• Discover reported failed=1 on puppet+erb+ruby (also on ruby-only) — likely a pre-existing ruby file; identifying it would be a bonus.

Again — appreciate the contribution. Happy to re-review once the stub allowlist / from QN adjustments are in. Welcome aboard 🙌

[l3acon]

Thanks for the thorough review! Addressed all feedback in the force-push:

Main fix: graph edges after discover

Commit 1bd1656 — three changes:

1. relation_allows_external_stub — added UsesVariable and UsesFact so unresolved targets get stubs and edges survive into the snapshot.

2. stub_node_type_for_target — added puppetvariable, puppetfact, puppetresource, puppetclass to the hint mapper so stubs get the correct NodeType.

3. Relation from uses block QN — changed from file path to erb:output@L{line} so edges attach to the block symbols, not the File node.

Verified on a minimal fixture after discover:

erb:output@L1 --USES-->       $hostname           (PuppetVariable)
erb:output@L2 --USES_FACT-->  @facts['os']['family'] (PuppetFact)
erb:output@L3 --REFERENCES--> profile::web::port   (PuppetVariable)

Smaller notes

• Gate B wording: dialed to "smoke corpus (~670 files, not O(10⁴))" in AGENTS.md.
• Tier depth: agreed — this is Layer A extraction + metadata. Not claiming CFG/taint/Layer F.
• failed=1: likely a pre-existing Ruby file. Will investigate separately.
• All commits now have DCO sign-off.

[sshaaf]

Follow-up review notes after re-testing tip 1bd16565ef0dacd7f718366e16770a6c5d3fdb5c — the graph-builder / from QN fix looks great (fixture + theforeman USES / REFERENCES now land). These are improvement items on grammar honesty and extraction depth, not merge blockers.

[sshaaf]

Coverage drift test is currently a no-op.

named / language are collected then discarded (let _ = named), so this won’t catch a grammar bump that adds/removes kinds the way Java’s ast_coverage test (or rgctl-ast-coverage::check_manifest) does.

Suggestion: assert every grammar named kind is in the manifest and every manifest key is still in the grammar (same pattern as rgctl-lang-java). Optionally lean on rgctl_ast_coverage::check_manifest so build-time + unit tests stay in sync.

[sshaaf]

Handler honesty for code.

Manifest labels code as Symbol, but the plugin never emits a symbol for code nodes — it only reads the child text from parent output_directive / directive / comment_directive.

Suggestion: use Literal or AstSkeleton (or document “consumed by parent”) so the matrix matches runtime behavior and website/docs generation stays accurate.

[sshaaf]

graphql_directive is listed as Skip and never walked.

walk_blocks only matches output_directive / directive / comment_directive, so <%graphql … %> blobs are invisible to the graph.

Suggestion (when useful): treat graphql_directive like directive (or a dedicated block kind) so GraphQL-in-ERB templates aren’t a silent hole. Until then, a one-line honesty note in the crate README / PR description would help reviewers.

[sshaaf]

content is Skip (HTML/text between ERB tags).

Fine for a first slice focused on Ruby/Puppet refs. If later you care about template layout or “what markup surrounds this block,” this is the place to emit lightweight structure (e.g. file-level skeleton or CONTAINS-style anchors).

No action required for merge — calling it out as a known coverage gap.

[sshaaf]

Directive walk gap: graphql_directive falls through to _ => None here.

Paired with the coverage Skip, GraphQL ERB directives produce no symbols/relations. If you keep skipping, consider a short comment linking to the manifest so the intentional omission is obvious in code review.

[sshaaf]

Ruby-inside-code is regex-only (expected — tree-sitter-embedded-template doesn’t parse Ruby).

Current @facts[…] regex catches bracket access (including double quotes). Gaps we probed locally:

• @facts.get('os') / @facts.dig(...) → no USES_FACT
• richer calls / require / non-@ locals → no call graph

Suggestion: document these honesty limits (and/or add a couple of negative/positive unit tests). A later phase could re-parse code text with tree-sitter-ruby for deeper edges without changing the outer grammar.

[sshaaf]

Extension surface is .erb only.

On the theforeman smoke corpus there are ~120 .epp files (Puppet EPP) that this plugin won’t see. That’s correct if EPP is intentionally out of scope — worth a sentence in docs/AGENTS so nobody assumes ERB support covers EPP.

A future rgctl-lang-epp (or shared embedded-template front-end) would close that gap.

[sshaaf]

No CFG / complexity path yet (enable_complexity = false, no CfgStatement handlers).

Makes sense for Layer A extraction + metadata. When you’re ready for deeper Tier 1 parity, next steps would be CFG hosts for control-flow ERB blocks and langfeatures / ecommerce-style fixtures — not required to land this PR.

[l3acon]

Pushed review fixes (7582537). Addressed all inline comments:

AST coverage (ast_coverage.rs)

• Real bidirectional assertion — every grammar named kind must be in the manifest AND every manifest key must still be in the grammar. No more no-op.

Manifest honesty (erb-ast-coverage.json)

• code → Literal (consumed by parent, never emitted as its own symbol)
• Added _honesty_notes section documenting code, content, and graphql_directive gaps

Plugin honesty (plugin.rs + lib.rs)

• Comment on graphql_directive intentional skip with manifest cross-reference
• Crate-level doc comment documents all honesty limits: regex-only Ruby parsing, no .epp, no CFG/taint/Layer F

New tests

• honesty_regex_gaps — asserts @facts.dig() and @facts.get() are NOT captured (known limit)
• epp_files_not_handled — asserts .epp files are out of scope

Cold profiles (graph_builder changes touch core extraction)

Corpus	wall_secs	nodes	edges	peak RSS
theforeman (Puppet+ERB+Ruby)	1.6s	6,108	13,580	72 MB
Linux kernel (Gate A)	173.4s	2,703,830	8,237,401	14.5 GB

Linux at 173.4s on this machine (previous run: 163.9s; ref baseline: 145s on M3 Pro). The ~10s variance is system load — the graph_builder changes add 2 match arms to relation_allows_external_stub and 4 to stub_node_type_for_target, both constant-time pattern matching with zero hot-path cost on the kernel's 8.2M edges (no ERB/Puppet relations to stub). Zero .erb files in the corpus.

10 unit tests pass. All commits signed off.

[sshaaf]

Awesome thanks!! this looks great!