[GPT-5.6 Luna] Plan: minimal instructional Rust port

[AuthorOfTheSurf]

Goal

Port KobraScript correctly and minimally from JavaScript to Rust while keeping the original JavaScript implementation intact as a side-by-side learning reference. The Rust version should preserve the current language and compiler behavior, not introduce a new language revision.

The implementation should be instructional: a reader should be able to compare the existing JavaScript modules with the Rust modules and see how enums, ownership, borrowing, Result, pattern matching, and explicit state replace prototype classes and process-global mutable state.

Current implementation to mirror

The existing compiler pipeline is:

.ks source -> scanner.js -> tokens -> parser.js -> entity AST -> analyzer.js -> JavaScript generation

• scanner.js is a line-oriented scanner that emits { kind, lexeme, line, col } tokens, including comments, strings and escapes, numbers, reserved/banned words, identifiers, and operators.
• parser.js is recursive descent with precedence levels for assignments, logical/comparison/arithmetic operators, unary/postfix expressions, calls, indexing, properties, arrays, objects, function literals, and closures.
• entities/ contains 33 AST/semantic/code-generation classes covering declarations, functions, closures, conditionals, only, loops, control flow, literals, calls, indexing, dotted access, arrays, and objects.
• analyzer.js provides parent-linked contexts and checks loop-only (break/continue) and subroutine-only (return/leave) control flow.
• compiler.js orchestrates scanning, parsing, optional optimization, analysis, and JavaScript output. There is no Kobra runtime; generated JavaScript is executed by Node.
• kobra-cli.js exposes -t, -a, -o, and -i; kobra compiles and pipes the result to Node.
• README.md macrosyntax/microsyntax and the existing .ks fixtures define the compatibility surface.

The current fixture inventory is 82 good programs, 20 syntax-error programs, and 4 semantic-error programs. A baseline npm test run in this checkout reported numerous existing failures and timed out, so the Rust port should use deterministic, phase-specific tests and record any intentional parity decisions rather than assuming the JavaScript suite is currently green.

Proposed Rust layout

Keep the Rust implementation in rust/ so the JavaScript implementation and its npm workflow remain unchanged:

rust/
  Cargo.toml
  README.md
  src/
    lib.rs
    error.rs       # Diagnostic and Result types
    token.rs       # TokenKind, Span, Token
    scanner.rs     # corresponds to scanner.js
    ast.rs         # replaces entities/* with typed enums/structs
    parser.rs      # corresponds to parser.js
    analyzer.rs    # corresponds to analyzer.js and entity analysis
    codegen.rs     # JavaScript emitter
    compiler.rs    # pipeline orchestration
    bin/
      kobrac.rs    # compiler CLI
      kobra.rs     # compile and run through node
  tests/
    scanner.rs
    parser.rs
    syntax_errors.rs
    semantic_errors.rs
    good_programs.rs
    codegen.rs

Use a library for compiler logic and thin binaries for I/O, diagnostics, flags, and process exit codes. The JavaScript files should not be removed or rewritten as part of this port.

Rust design principles

1. Typed tokens: Replace stringly-typed token kinds with a TokenKind enum and retain a Span { line, column } for diagnostics. Keep lexemes where needed for identifiers and literals.
2. Borrowed scanner input: Implement scan(source: &str) -> Result<Vec<Token>, Diagnostics> without filesystem I/O or callbacks. compile_file can own the file-reading boundary.
3. Explicit parser state: Implement a Parser<'a> over &'a [Token] with a cursor. Do not reproduce the JavaScript module-global token array or shift() behavior.
4. Instructional AST: Collapse the entity class hierarchy into Program, Stmt, and Expr enums/structs. Derive useful traits such as Debug, PartialEq, and Clone only where they serve tests or ownership needs.
5. Structured diagnostics: Replace global error.count and console logging with Result and structured diagnostics. Fail fast initially unless fixture parity requires a small diagnostic accumulator. Library code must not print or panic for user-facing source errors.
6. Scoped semantic analysis: Model lexical scopes with a stack or indexed parent relationships and track in_loop / in_subroutine explicitly. Resolve names deliberately; do not recreate the JavaScript object graph.
7. Separate code generation: Emit JavaScript from the AST with match and fmt::Write/String. Preserve the current translations: == → ===, != → !==, ~= → ==, # → ||, ** and -** via Math.pow, is via typeof, :=: swap lowering, say/loge via console.log, and closure IIFEs.
8. Standard library first: Prefer std for scanning, collections, formatting, filesystem access, and basic argument parsing. Keep dependencies minimal and justify any added crate.
9. No speculative language changes: Preserve the README syntax and actual supported behavior, including $/,/.. declarations, fn, close, block forms, only, loops, control-flow statements, literals, object keys, calls, indexing, and the documented operator precedence.

Implementation phases

Phase 0: crate skeleton

• Add rust/Cargo.toml, the library, module declarations, and minimal binaries.
• Make cargo build and cargo test work before implementing compiler behavior.
• Add short rust/README.md guidance explaining the JavaScript-to-Rust module mapping.

Phase 1: tokens and scanner

• Implement TokenKind, Span, Token, diagnostics, and a hand-written scanner.
• Port scanner behavior for comments, whitespace, line/column tracking, strings and escapes, numeric literals, reserved/banned words, identifiers, and all operators.
• Port representative cases from test/test-scanner.js as table-driven Rust tests, including hello world, exchange, increments, objects, and exact positions.
• Make scanner error cases return structured diagnostics.

Phase 2: AST and parser

• Implement the typed AST and Display/debug output useful for comparing with kobrac -a.
• Port the recursive-descent parser one precedence level at a time, preserving declaration continuation, block termination (end/..), single-statement blocks (->), function/closure forms, arrays, objects, calls, dotted access, and indexed access.
• Add parser tests for representative valid syntax and all 20 syntax-error fixtures. These must fail in parsing with useful source locations before semantic analysis is attempted.

Phase 3: semantic analysis

• Implement scope creation, declaration/reference handling, and control-flow context checks.
• Port all 4 semantic-error fixtures and assert they parse successfully but fail analysis.
• Run all 82 good-program fixtures through scan, parse, and analysis and ensure no diagnostics are produced.

Phase 4: JavaScript code generation and CLI

• Implement JavaScript emission with explicit state for declaration continuation and formatting.
• Add snapshot or normalized-output tests for hello world, declarations, precedence, functions, closures, objects, loops, only, swaps, and say/loge.
• Implement kobrac equivalents for -t, -a, -o, and -i, with clear stderr diagnostics and non-zero exit status on failure.
• Implement kobra as a thin compile-and-run wrapper around Node, matching the current scope without adding a Rust interpreter.

Phase 5: compatibility sweep and documentation

• Run the complete fixture matrix in cargo test.
• Compare representative Rust-generated JavaScript with the current JavaScript compiler, normalizing formatting where appropriate.
• Document commands for running both implementations, the verified parity surface, and any behavior inherited from the existing compiler.
• Keep the Rust code readable in side-by-side comparison with scanner.js, parser.js, analyzer.js, and the entity files.

Acceptance criteria

• cargo test passes the Rust scanner, parser, semantic, fixture, and code-generation tests.
• All 82 good fixtures scan, parse, and analyze successfully.
• All 20 syntax-error fixtures fail in the parser phase.
• All 4 semantic-error fixtures fail in analysis rather than parsing.
• Representative generated JavaScript is behaviorally equivalent to the existing compiler and kobrac flags work.
• cargo run --bin kobrac -- test/kobra-code/good-programs/hello-world.ks emits JavaScript equivalent to the current output.
• Original JavaScript sources, fixtures, and root npm commands remain available and unmodified.
• User-facing source errors are structured, located, and recoverable through Result; library code does not rely on global mutable counters or hidden console output.

Suggested review slices

1. Crate skeleton plus scanner and scanner tests.
2. Typed AST plus parser and syntax-error tests.
3. Semantic analysis plus good/semantic fixture sweep.
4. Code generation, CLI binaries, and representative snapshots.
5. Full compatibility sweep and instructional documentation.

This issue is a plan only. Implementation should stay minimal, preserve the JavaScript behavior as the reference, and favor clear Rust fundamentals over abstraction for its own sake.