Expand data-oriented design doc to cover the full compilation pipeline - #1
Open
devin-ai-integration[bot] wants to merge 5 commits into
Open
devin-ai-integration[bot] wants to merge 5 commits into
devin-ai-integration[bot] wants to merge 5 commits into
Conversation
Rename docs/compiler/HLIR_DataOriented.md to DataOrientedPipeline.md and broaden its scope from HLIR-only to the entire HLIR -> MLIR -> LLIR -> codegen pipeline, excluding ANTLR-generated lexer/parser/visitor code. Co-Authored-By: Kenneth Cochran <kenneth.cochran@gmail.com>
Author
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
Reframe the migration as a parallel DOD pipeline built behind the existing stage interfaces (new sibling ANTLR visitor -> DOD AST, plus a new DOD implementation of each pipeline stage), kept side by side with the legacy pipeline and gated on differential parity tests before the legacy path is retired. Co-Authored-By: Kenneth Cochran <kenneth.cochran@gmail.com>
Reflect that the DOD pipeline can diverge structurally from the legacy one: introduce DOD-specific abstractions (IDodFrontend, IDodIRTransformer, IDodOptimizer, IDodCodeGenerator) where shapes don't fit the existing interfaces, with the branch-by-abstraction seam at the pipeline/ orchestration level as well as per stage. Co-Authored-By: Kenneth Cochran <kenneth.cochran@gmail.com>
Phased, branch-by-abstraction plan (P0 build unblock through P16 legacy removal): shared Blob infra, blob models, DOD interfaces, differential test harness, per-stage DOD implementations, serialization, pipeline composition/selection, end-to-end parity + benchmarks, flip default, remove legacy. Includes per-phase deliverables, tests, acceptance criteria, dependencies, and risks. Co-Authored-By: Kenneth Cochran <kenneth.cochran@gmail.com>
Reframe phases as logical increments that can be grouped into a few PRs (foundation / DOD pipeline / cutover), since existing stages are simple and parity is quick; keep branch-by-abstraction, green-build, and parity invariants. Co-Authored-By: Kenneth Cochran <kenneth.cochran@gmail.com>
kennethcochran
added a commit
that referenced
this pull request
Aug 7, 2026
…ommit gate - Add CONTEXT.md as the single source of truth (Pocket #1) - Add ADRs for SoA IR and Shared StringPool decisions (Pocket #3) - Update .claude/skills/doc-sync/SKILL.md to use gstack document-release - Update .husky/task-runner.json to run gstack document-release pre-commit - Update docs/AGENTS.md with Three-Pocket Strategy - Update root AGENTS.md with mandatory pre-commit hook enforcement - Remove obsolete doc-sync.py and doc-mapping.yaml Note: --no-verify used because 3 pre-existing Pascal frontend test failures (EdgeCaseTests) block the hook independently of this docs-only commit. These failures pre-date this change and will be fixed in the next commit to restore hook health.
kennethcochran
added a commit
that referenced
this pull request
Sep 14, 2026
… frontends ADR-004 ticket #02: AST container types move from Core.IR.Ast to each frontend assembly. Pascal uses GameVM.Compiler.Pascal.Ast, CSharp uses GameVM.Compiler.CSharp.Ast. Core's public API has zero AST type references. Changes: - Moved AstNode.cs + AstBuilder.cs to Pascal and CSharp assemblies - Deleted originals from Core (IR/Ast/ and IR/Soa/AstBuilder.cs) - Updated all using directives in frontend source and test files - Removed GameVM.Compiler.Core.IR.Ast from .editorconfig GVM001 scope - Moved AstBuilderTests from Core.Tests to Pascal.Tests - All 496 tests pass override-no-doc: Structural file relocation only; CONTEXT.md already describes AST as frontend-internal per ticket #1.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Broadens the data-oriented design (DOD) proposal from HLIR-only to the entire compilation pipeline, and renames the doc to match:
docs/compiler/HLIR_DataOriented.md→docs/compiler/DataOrientedPipeline.mdThe original doc only covered making HLIR cache-friendly. This rewrite applies the same DOD approach (POD structs, SoA hot fields, 32-bit id references instead of object pointers, interned
StringPool, arenas, rebuild-free passes) to every stage orchestrated byCompileUseCase.CompileInternal:frontend AST→HLIR →
HLIRBlob→HlirToMlirTransformer→MLIRBlob→ mid optimizer →MidToLowLevelTransformer→LLIRBlob→ low optimizer →Atari2600CodeGenerator/M6502Emitter, plus capability validation.Explicitly out of scope: ANTLR-generated code (the
PascalLexer/PascalParser/PascalBaseVisitorgenerated fromPascal.g4). The DOD boundary at the frontend starts after parsing — the hand-written AST +PascalAstToHlirTransformerare in scope; the generated parse tree is only read at the boundary.Content added beyond the original:
string-typed operands and full-graph rebuilds per pass).Blobinfrastructure (Ids,Range32,StringPool,TypeTable, SoA instruction store,Arena,VersionInfo) reused across HLIR/MLIR/LLIR.BlobBasedHlirView-style view per level, gated byCompilationOptions.DataOriented), serialization/versioning, testing/benchmark plan, roadmap checklist, file layout, and open questions.Docs-only change; no code paths touched.
Note: base branch is
feature/dod-pipeline(where the original doc lives), notmain.Link to Devin session: https://app.devin.ai/sessions/c699e030f8e4442faff64b1d5bf94f24
Requested by: @kennethcochran