Fix native parser portability on Windows

.github/workflows/check.yml

@@ -10,10 +10,14 @@ permissions:
 
 jobs:
   test:
-    runs-on: ubuntu-latest
+    runs-on: ${{ matrix.os }}
     strategy:
       matrix:
+        os: [ubuntu-latest, windows-latest]
         python-version: ['3.10', '3.12']
+    env:
+      PYTHONUTF8: "1"
+      PYTHONIOENCODING: utf-8
     steps:
       - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
         with:

README.md

@@ -57,7 +57,7 @@ retain their current checks. Try the neutral fixture with
 
 The repository registers its three skills through symlinks under `.agents/skills/`, which Codex discovers when you open this project. They remain independently invocable and are scoped to this repository. Start a new project conversation if the skill selector has not refreshed. [Codex skill discovery](https://learn.chatgpt.com/docs/build-skills) documents repository scope and symlink support.
 
-Each skill's `scripts/cse.py` launcher finds the shared parser/engine from its real location, so checks work from subdirectories as well as the repository root. The repository also includes a plugin manifest at `.codex-plugin/plugin.json`; installation through a public marketplace remains untested. Copying a skill folder alone preserves its version-1 helper, but does not install the shared SysML engine. Avoid duplicate installations.
+Each skill's `scripts/cse.py` launcher finds the shared parser/engine from its real location, so checks work from subdirectories as well as the repository root. The repository also includes a plugin manifest at `.codex-plugin/plugin.json`; installation through a public marketplace remains untested. Copying a skill folder alone preserves its version-1 helper, but does not install the shared SysML engine. Avoid duplicate installations. [Windows runtime and installation boundaries](docs/windows-runtime.md) describe platform locking, UTF-8, checkout links, and the separate installed-plugin qualification.
 
 Example requests:
 

docs/windows-runtime.md

@@ -0,0 +1,54 @@
+# Native runtime portability and installation boundary
+
+The shared native parser needs Python 3.10+, Java and javac21+, and the same pinned
+pilot archive on both Linux and Windows. This change preserves parser serialization:
+POSIX uses nonblocking `flock`; Windows uses a nonblocking one-byte `msvcrt` lock
+at byte zero of the same persistent `parser.lock`. A contender must wait or time
+out. Lock errors never fall back to running Java without a lock. The lock file
+is opened without truncation and is not deleted or replaced while contenders
+may be using it. File closure and explicit release apply on parser failure too.
+
+The existing timeout bounds lock acquisition and the Java invocation together.
+Java's two-entry classpath uses the platform path separator, and source/result
+pipes carry explicit UTF-8 bytes. Java launches with `-Dstdout.encoding=UTF-8`
+and `-Dstderr.encoding=UTF-8`; Python strictly decodes both captured byte streams
+on the calling thread. Invalid or missing streams produce a structured parser
+error, including on nonzero exits. Paths with spaces remain individual subprocess
+arguments; no shell command is constructed.
+
+Run the normal required suite after installing the documented pinned parser:
+
+```sh
+python -X utf8 scripts/install_sysml_parser.py
+python -X utf8 -B scripts/check.py --require-sysml
+```
+
+The new portability suite exercises real cross-process exclusion, timeout without
+Java execution, release after exceptions/timeouts, preserved lock contents,
+nonblocking Windows byte-zero adapter behavior, import without POSIX-only modules,
+actual JVM stdout/stderr UTF-8 bytes, malformed transport errors,
+and actual native acquisition of UTF-8 text in a path with spaces. The workflow
+adds Windows alongside Ubuntu for both supported Python test versions. Hosted
+workflow configuration alone is not a completed Windows test result; record the
+actual run and revision before claiming qualification.
+
+On Windows, Git may retain tracked `.agents/skills` symlinks as small files naming
+their intended targets. The registration test checks those records, but does not
+claim such a file supplies project skill discovery. Runtime tests use the real
+`skills/<name>/scripts/cse.py` directory when a directory registration is absent.
+Installed plugins likewise require the complete real skill directories plus the
+shared scripts, verifier, reference files and qualified presentation assets.
+Do not overwrite existing legacy standalone skill files or customizations to
+repair a checkout. Project discovery through actual links and installed-plugin
+discovery remain distinct checks.
+
+Global installation is a separate qualification: use a clean accepted revision,
+verify manifest/marketplace loading through supported clients, compare installed
+payload hashes, run the installed launchers/native check, and confirm skill
+discovery in a clean session. This document does not claim a marketplace install,
+Windows availability, editor roundtrip or protected execution merely from local
+source tests. Do not restart active sessions or replace unrelated registrations.
+
+References: Python [msvcrt locking](https://docs.python.org/3/library/msvcrt.html),
+[flock](https://docs.python.org/3/library/fcntl.html), and
+[platform path separators](https://docs.python.org/3/library/os.html#os.pathsep).

scripts/check.py

@@ -42,7 +42,7 @@ def main():
             return 1
     suites = ("test_develop_conops.py", "test_scenario_split.py", "test_concept.py", "test_verifier.py",
               "test_language.py", "test_language_oracle.py", "test_influence.py", "test_events.py",
-              "test_public_language_cases.py", "test_sysml.py", "test_sysml_views.py",
+              "test_public_language_cases.py", "test_sysml_portability.py", "test_sysml.py", "test_sysml_views.py",
               "test_sysml_public.py", "test_skill_runtime.py", "test_engineering_views.py", "test_visual_style.py", "test_visual_projection.py", "test_visual_labels.py")
     for suite in suites:
         result = subprocess.run([sys.executable, "-B", str(ROOT / "tests" / suite)],

tests/test_skill_runtime.py

@@ -1,12 +1,16 @@
 """Real repo skill discovery paths and CLI use from another working directory."""
 
+import io
+import importlib.util
 import json
 import os
 from pathlib import Path
 import subprocess
 import sys
 import tempfile
 import unittest
+from contextlib import redirect_stderr
+from unittest.mock import patch
 
 ROOT = Path(__file__).resolve().parents[1]
 sys.path.insert(0, str(ROOT))
@@ -16,31 +20,47 @@
 PARSER_AVAILABLE = JAR.is_file() and (CLASSES / "SysmlBridge.class").is_file()
 
 
+def launcher(name):
+    registered = ROOT / ".agents/skills" / name
+    # Installed plugins carry real skill directories. Preserve a Windows Git
+    # symlink-record checkout rather than rewriting its registration files.
+    base = registered if registered.is_dir() else ROOT / "skills" / name
+    return base / "scripts/cse.py"
+
+
 class RuntimeTests(unittest.TestCase):
     def test_installer_explains_unsupported_java_before_download_or_compile(self):
-        (ROOT / ".tmp").mkdir(exist_ok=True)
-        with tempfile.TemporaryDirectory(dir=ROOT / ".tmp") as directory:
-            java = Path(directory) / "java"
-            java.write_text("#!/bin/sh\nprintf 'openjdk 17.0.9\\n'\n")
-            java.chmod(0o755)
-            result = subprocess.run([sys.executable, "-B", str(ROOT / "scripts/install_sysml_parser.py")],
-                                    env=dict(os.environ, PATH=directory), text=True, capture_output=True)
-            self.assertEqual(result.returncode, 1)
-            self.assertIn("requires Java 21+ and javac 21+", result.stderr)
-            self.assertNotIn("bad class file", result.stderr)
+        spec = importlib.util.spec_from_file_location("cse_test_installer", ROOT / "scripts/install_sysml_parser.py")
+        installer = importlib.util.module_from_spec(spec); spec.loader.exec_module(installer)
+        old_java = subprocess.CompletedProcess(["java", "--version"], 0, "openjdk 17.0.9", "")
+        errors = io.StringIO()
+        with patch.object(installer.subprocess, "run", return_value=old_java) as run, \
+                patch.object(installer, "urlopen") as download, redirect_stderr(errors):
+            self.assertEqual(installer.main(), 1)
+        self.assertIn("requires Java 21+ and javac 21+", errors.getvalue())
+        self.assertNotIn("bad class file", errors.getvalue())
+        self.assertEqual(run.call_args.args[0], ["java", "--version"])
+        self.assertEqual(run.call_count, 1)
+        download.assert_not_called()
 
-    def test_repo_skill_links_resolve_to_complete_skills(self):
+    def test_repo_skill_registrations_refer_to_complete_skills(self):
         for name in SKILLS:
             skill = ROOT / ".agents/skills" / name
-            self.assertTrue(skill.is_symlink())
-            self.assertEqual(skill.resolve(), ROOT / "skills" / name)
-            self.assertTrue((skill / "SKILL.md").is_file())
+            if skill.is_symlink():
+                self.assertEqual(skill.resolve(), ROOT / "skills" / name)
+                self.assertTrue((skill / "SKILL.md").is_file())
+            else:
+                # Windows Git may preserve a tracked link as its target text.
+                # This checks metadata, not project discovery through that file.
+                self.assertEqual(os.name, "nt")
+                self.assertEqual(skill.read_text().strip(), "../../skills/" + name)
+                self.assertTrue((ROOT / "skills" / name / "SKILL.md").is_file())
 
     def test_each_independent_launcher_finds_the_same_engine_from_another_directory(self):
         (ROOT / ".tmp").mkdir(exist_ok=True)
         with tempfile.TemporaryDirectory(dir=ROOT / ".tmp") as directory:
             for name in SKILLS:
-                result = subprocess.run([sys.executable, "-B", str(ROOT / ".agents/skills" / name / "scripts/cse.py"),
+                result = subprocess.run([sys.executable, "-B", str(launcher(name)),
                                          "runtime"], cwd=directory, text=True, capture_output=True)
                 self.assertEqual(result.returncode, 0, result.stderr)
                 self.assertEqual(json.loads(result.stdout)["runtime_root"], str(ROOT))
@@ -55,8 +75,8 @@ def test_joint_concept_scenario_uses_one_model_and_exports_supported_tests(self)
         with tempfile.TemporaryDirectory(dir=ROOT / ".tmp") as directory:
             local_model = Path(directory) / "user-model.sysml"
             local_model.write_bytes(model.read_bytes())
-            launcher = ROOT / ".agents/skills/develop-operational-scenarios/scripts/cse.py"
-            result = subprocess.run([sys.executable, "-B", str(launcher), "testgen", "user-model.sysml",
+            entrypoint = launcher("develop-operational-scenarios")
+            result = subprocess.run([sys.executable, "-B", str(entrypoint), "testgen", "user-model.sysml",
                                      "--output", "synthetic-tests.json"], cwd=directory,
                                     text=True, capture_output=True)
             self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
@@ -70,16 +90,16 @@ def test_joint_concept_scenario_uses_one_model_and_exports_supported_tests(self)
 
     @unittest.skipUnless(PARSER_AVAILABLE, "Install the pinned SysML parser")
     def test_plain_output_distinguishes_compilation_and_behavior_and_reports_parser_errors(self):
-        launcher = ROOT / ".agents/skills/develop-system-concept/scripts/cse.py"
-        result = subprocess.run([sys.executable, "-B", str(launcher), "check",
+        entrypoint = launcher("develop-system-concept")
+        result = subprocess.run([sys.executable, "-B", str(entrypoint), "check",
                                  str(ROOT / "examples/sysml/ack.sysml")], text=True, capture_output=True)
         self.assertEqual(result.returncode, 0, result.stderr)
         self.assertIn("no behavior evaluated", result.stdout)
         (ROOT / ".tmp").mkdir(exist_ok=True)
         with tempfile.TemporaryDirectory(dir=ROOT / ".tmp") as directory:
             model = Path(directory) / "invalid.sysml"
             model.write_text("package Broken { part def ???; }")
-            invalid = subprocess.run([sys.executable, "-B", str(launcher), "check", str(model),
+            invalid = subprocess.run([sys.executable, "-B", str(entrypoint), "check", str(model),
                                       "--syntax-only"], text=True, capture_output=True)
             self.assertNotEqual(invalid.returncode, 0)
             self.assertIn("ERROR", invalid.stdout)