Skip to content

feat(judge-node): persist a real microvm evaluation - #11

Open
CN059 wants to merge 2 commits into
devfrom
agent/real-microvm-judge-e2e
Open

CN059 wants to merge 2 commits into
devfrom
agent/real-microvm-judge-e2e

Conversation

@CN059

@CN059 CN059 commented Aug 24, 2026

Copy link
Copy Markdown
Member

Stack

This PR is stacked on #10. Its base is codex/algorithm-c-runtime-smoke so the runtime-image work remains isolated. After #10 merges, retarget this PR to dev.

Summary

  • connect the existing CLI/PostgreSQL/control-plane UDS lease path to the development Firecracker executor
  • load one bounded C source artifact from trusted deployment configuration, never from a request-selected host path
  • match source digest, exact size, text/x-csrc, and runtime manifest digest before boot
  • upload main.c through the existing bounded guest v0alpha1 message and compile/run fixed argv inside the guest
  • require the host-side expected output digest in addition to exit 0 before producing Accepted
  • align judge-node boot args and 256 MiB machine shape with the runtime contract already validated by build(runtime): validate the algorithm-c guest smoke path #10
  • persist and verify Firecracker provenance, node identity, stage evidence and teardown through a real-process KVM test

Scope and alternatives

This is a development-only bridge for one fixed Artifact. The preferred production design remains content-addressed object storage with scoped delivery. Implementing that backend in this PR would mix storage authorization and lifecycle work into the execution-chain proof. Embedding source bytes or host paths in the Judge Control request was rejected because it would weaken bounded-message and trust-boundary constraints.

No ADR is required: ADR-0005 already accepts this minimal vertical slice and the change does not alter its component, protocol or trust decisions.

Security and compatibility

  • the Evaluation request cannot select a host path, executable or argv
  • source is capped at the existing 256 KiB inline bound and checked before VMM boot
  • guest network and credentials remain absent; root block remains read-only
  • guest exit 0 cannot self-declare acceptance
  • OPENOJ_FC_PRODUCTION=1 remains fail closed
  • no canonical schema, Judge Control protobuf, guest message, database migration or public API change

Rollback is to disable the development firecracker executor and use explicit development_mock or the standalone KVM smoke; no data migration is involved.

Validation

Target environment: Linux 7.1.8 x86_64, AMD-V exposed by VMware, KVM API 12, Firecracker 1.16.1, PostgreSQL 18.6 on disposable tmpfs.

Passed:

  • strict focused real-process E2E: CLI -> PostgreSQL -> control-plane -> UDS lease -> judge-node -> Firecracker guest C compile/run -> host digest check -> persisted result, 8.21 s
  • OPENOJ_REQUIRE_KVM=1 cargo test --workspace --all-targets --locked with disposable PostgreSQL: 149 passed, 0 failed
  • cargo fmt, workspace check, workspace Clippy with warnings denied
  • cargo-deny advisories/bans/sources; existing duplicate-version warnings only
  • workspace dependency, documentation, runtime structure and four applicable Skill validators

Exact input/runtime/output digests and raw summary are in docs/validation/2026-08-25-algorithm-c-process-e2e.md.

Unverified

  • production jailer/uid/gid/cgroup/namespace/seccomp/watchdog
  • formal object-storage Artifact delivery and arbitrary user artifacts
  • HTTP/API ingress, identity and full audit correlation
  • adversarial resource, concurrent guest and real-stage cancellation matrix
  • reproducible rootfs, licensing review and release performance

Refs: FR-ARTIFACT-001, FR-EVAL-001, FR-JUDGE-001, FR-JUDGE-002, FR-RUNTIME-001, FR-RESULT-001, ACC-P0-001, ACC-P0-005, ACC-P0-008, ACC-P0-009, ACC-P0-010, ACC-P0-016, ADR-0005, RISK-EXEC-001

Base automatically changed from codex/algorithm-c-runtime-smoke to dev August 24, 2026 17:17
CN059 added 2 commits August 31, 2026 08:50
Connect the existing PostgreSQL/UDS worker path to the development Firecracker executor using one bounded deployment-configured C artifact. Match source digest, size, media type, and runtime digest before boot; compile and run fixed argv inside the guest; and let the host accept only the configured output digest.

Reuse the validated algorithm-c kernel command line and 256 MiB machine shape, retain idempotent teardown, and add a real-process KVM test that verifies the persisted Firecracker provenance and run evidence.

Refs: FR-ARTIFACT-001, FR-EVAL-001, FR-JUDGE-001, FR-JUDGE-002, FR-RUNTIME-001, FR-RESULT-001, ACC-P0-001, ACC-P0-005, ACC-P0-008, ACC-P0-009, ACC-P0-010, ACC-P0-016, ADR-0005, RISK-EXEC-001

Tests: PostgreSQL 18.6-backed OPENOJ_REQUIRE_KVM=1 cargo test --workspace --all-targets --locked (149 passed); cargo fmt --all -- --check; cargo check --workspace --all-targets --locked; cargo clippy --workspace --all-targets --locked -- -D warnings; cargo deny --locked check advisories bans sources; python3 scripts/check-workspace.py; bash scripts/check-docs.sh; runtime and four applicable Skill validators

Unverified: production isolation, object-storage Artifact delivery, HTTP/API ingress, arbitrary user artifacts, adversarial resource matrix, cancellation during real guest stages, and release performance

Security: The Evaluation request cannot select a host path or command; guest networking and credentials remain absent; source and runtime mismatches fail before VMM boot; production remains fail closed.

Compatibility: Reuses the existing canonical and guest v0alpha1 messages without schema, persistence, migration, or public API changes. The local Artifact bridge is development-only and removable when the formal Artifact backend lands.
Record the target KVM environment, immutable source/runtime/output digests, RED-to-GREEN boot assembly diagnosis, PostgreSQL-backed real-process result assertions, complete workspace gates, and remaining production and Artifact-backend gaps for commit c492288.

Refs: FR-ARTIFACT-001, FR-EVAL-001, FR-JUDGE-001, FR-JUDGE-002, FR-RUNTIME-001, FR-RESULT-001, ACC-P0-001, ACC-P0-005, ACC-P0-008, ACC-P0-009, ACC-P0-010, ACC-P0-016, ADR-0005, RISK-EXEC-001

Tests: bash scripts/check-docs.sh; evidence SHA-256 check; implementation validation recorded as PostgreSQL 18.6-backed OPENOJ_REQUIRE_KVM=1 cargo test --workspace --all-targets --locked (149 passed) plus fmt/check/Clippy/cargo-deny/workspace/runtime/Skill gates

Unverified: production isolation, formal object-storage Artifact delivery, HTTP/API ingress, arbitrary user artifacts, adversarial resource and cancellation matrix, reproducible rootfs, licensing review, and release performance

Security: Evidence is limited to the exact development host and digests; production remains fail closed and no network, credential, request-selected host path, or request-selected command was introduced.

Compatibility: Evidence-only follow-up; no protocol, schema, migration, public API, or production compatibility change.
@CN059
CN059 force-pushed the agent/real-microvm-judge-e2e branch from d6445af to adb5943 Compare August 31, 2026 00:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant