Skip to content

Phase 623: genuine signed end-to-end IRC adversarial trials and verified CI evidence - #623

Merged
EmergentMonk merged 7 commits into
mainfrom
test/phase623-end-to-end-adversarial-trial
Nov 1, 2026
Merged

EmergentMonk merged 7 commits into
mainfrom
test/phase623-end-to-end-adversarial-trial

Conversation

@EmergentMonk

@EmergentMonk EmergentMonk commented Nov 1, 2026 •

Copy link
Copy Markdown
Member

Goal

Test the actual integrated network execution path from merged QEC #620 (composed experiment/capability/firewall verification), #621 (real protected IRC command), and #622 (one-use revocation ledger). No producer-controlled ALLOW flags or mocks standing in for real socket effects.

Changes

  • tests/test_phase623_end_to_end.py: two real loopback IRC clients, real ephemeral Ed25519 signed UFF experiments and issued grants, real verifier-pinned contexts. Wire-observed valid one-shot NOTICE versus replay, independently valid cross-bundle transplant, all four revocation mechanisms, forged signed payload, policy change, wrong-session and channel fanout, saturation, and hash-chain tampering.
  • scripts/maintenance/verify_phase623_trial.py: separately validate JUnit inventory for 11 required executed test cases across 8 scenario families. Reject any skipped, failed, missing or unexpected case and emit per-run content hash/source-SHA receipt. No false stable-digest claim about JUnit timing.
  • tests/test_phase623_receipt_verifier.py: fail-closed test coverage with synthetic XML clearly not represented as executed adversarial evidence.
  • .github/workflows/phase623-adversarial.yml: real cryptography backend, actual Ed25519 sign/verify, executed live-socket trials, independent inventory verifier, retained per-run JUnit and success-only receipt artifacts.
  • docs/security/PHASE623_END_TO_END_ADVERSARIAL_TRIAL.md: test matrix and limitations.

Protected boundary remains only the non-sensitive !trusted-observe NOTICE. No production executor or decoder changes, durable authentication, independent custody, quantum hardware, global distributed replay or physical-world proof claimed.

Required gates

Draft, do not merge. Need exact-head dedicated CI; full existing Python and container suite; exact-head Codex code and security reviews with no actionable findings. Await actual CI execution before making any passing-claim. Preserve historical artifact bytes and original source/receipt schema.

Summary by Sourcery

Validate the integrated signed IRC observation path through real loopback clients and produce fail-closed, per-run CI evidence for the adversarial trial.

New Features:

  • Add real loopback-socket adversarial trials covering signed IRC observations, replay protection, composition boundaries, revocation, authorization, saturation, and receipt-chain tampering.

Enhancements:

  • Add fail-closed validation for the exact executed adversarial-test inventory and per-run evidence receipts.
  • Document the Phase 623 transport trial scope, test matrix, evidence handling, and limitations.

CI:

  • Add dedicated CI coverage using real Ed25519 cryptography, live socket execution, independent JUnit inventory verification, and retained per-run artifacts.

Documentation:

  • Document the end-to-end adversarial trial and explicitly state its evidence and security limitations.

Tests:

  • Add contract tests ensuring the inventory verifier rejects incomplete, substituted, skipped, failed, or unrelated test results.

@sourcery-ai

sourcery-ai Bot commented Nov 1, 2026

Copy link
Copy Markdown

Reviewer's Guide

This PR validates the previously integrated signed evidence, capability, firewall, IRC, and one-use replay protections through genuine Ed25519 signing and two-client loopback socket execution, then applies an independent exact-head JUnit inventory check and per-run receipt workflow. It deliberately limits claims to the non-sensitive NOTICE boundary and documents remaining limitations and required CI/review gates.

Sequence diagram for the Phase 623 signed IRC adversarial trial

sequenceDiagram
    participant C1 as IRCClient1
    participant S as IRCServer
    participant V as VerifierPinnedContext
    participant L as ObservationReplayLedger
    participant C2 as IRCClient2

    C1->>S: IRC command with signed UFF evidence and grant
    S->>V: Verify signed experiment and capability
    V-->>S: Valid pinned evidence
    S->>L: Admit operation identity
    L-->>S: ALLOW
    S-->>C1: NOTICE protected observation
    S-->>C2: No protected output

    C1->>S: Replay same operation
    S->>L: Admit operation identity
    L-->>S: REPLAYED_OPERATION
    S-->>C1: No NOTICE

    C1->>S: Forged, transplanted, revoked, or policy-mismatched evidence
    S->>V: Verify evidence and composition
    V-->>S: Denied
    S->>L: Record denial
    S-->>C1: No NOTICE
Loading

Sequence diagram for Phase 623 CI evidence verification

sequenceDiagram
    participant CI as GitHubActions
    participant T as EndToEndTests
    participant J as JUnitXML
    participant V as TrialInventoryVerifier
    participant R as PerRunReceipt
    participant A as CIArtifacts

    CI->>T: Execute real Ed25519 and loopback socket trials
    T->>J: Write 11 executed test cases
    CI->>V: verify_phase623_trial.py(JUnitXML, source SHA)
    V->>J: Check exact cases and scenario families
    J-->>V: No missing, skipped, failed, or unexpected cases
    V->>R: Emit source SHA and JUnit SHA-256
    CI->>A: Upload JUnit evidence
    CI->>A: Upload receipt only on success
Loading

File-Level Changes

Change Details Files
Adds real end-to-end adversarial coverage through the IRC transport using ephemeral signed evidence and live loopback clients.
  • Starts the actual IRC server on an ephemeral loopback port and drives two TCP clients.
  • Exercises one-shot acceptance, replay, bundle transplant, all revocation dimensions, client/channel isolation, malformed input, forged payloads, policy changes, ledger saturation, and receipt-chain tampering.
  • Asserts exact protected wire output, silence for denied paths, and ledger admission/denial reasons.
tests/test_phase623_end_to_end.py
docs/security/PHASE623_END_TO_END_ADVERSARIAL_TRIAL.md
Introduces an independent fail-closed verifier for the executed adversarial-test inventory and per-run evidence receipt.
  • Requires exactly 11 passing, non-skipped cases across eight expected scenario families.
  • Rejects missing, duplicate, unrelated, failed, errored, or skipped JUnit cases.
  • Records the source commit, JUnit content hash, sorted test identifiers, and explicit evidence limitations without claiming stable timing digests.
  • Adds synthetic-XML tests for verifier rejection and exact inventory enforcement.
scripts/maintenance/verify_phase623_trial.py
tests/test_phase623_receipt_verifier.py
Adds dedicated CI execution and artifact handling for real cryptographic and socket trials.
  • Installs and explicitly exercises the real Ed25519 backend before running the live trials.
  • Runs verifier tests, generates JUnit from the end-to-end suite, and independently validates the inventory against the checked-out commit.
  • Retains JUnit artifacts on failure and verified inventory receipts only on success.
.github/workflows/phase623-adversarial.yml

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link
Copy Markdown
Member Author

@codex review
@codex security review

Please review exact Phase 623 QEC PR #623 head 5c1b1ee5c9b6b1a036313670008e642699f51f32. Draft, no merge/release.

This PR tests the actual merged #620–#622 security composition through two real loopback IRC socket connections using ephemeral signed UFF experiment evidence (Ed25519) and independently issued test capability grants. Test-only change; no production runtime or protected decoder altered.

Executed at this SHA: Phase 623 workflow run 41355448547 SUCCESS. Real Ed25519 sign/verify executed; 11/11 adversarial socket cases passed, 6/6 verifier contract tests passed. Independent JUnit inventory guard confirmed all 8 scenario families, no skips/errors/failures, then emitted a generated SHA-256-bound per-run inventory. Both original JUnit and success-only inventory were uploaded (not committed). Full repository/container workflow 41355448553 still running as of this comment; do not infer success.

Focus: adversarial coverage gaps that could produce a protected NOTICE under replay, policy/signer/grant revocation, crossed signed proofs, alternate valid operation identity, compromised trust context, malformed IRC routing, second-client/channel leakage; async socket test false positives; JUnit verifier's completeness checks versus artifact spoofing; accurate receipt/provenance wording. Distinguish real executed observations from inferred claims, and provide minimal reproducible actionable defects only. No claims of durable replay, authenticated human identity, external custody, privileged QEC actions or physical science.

Keep exact-head review, full suite, and container CI as separate gates. Do not merge automatically.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-01T22:43:04.849802Z 7774309 Draft marked ready
🔒 Security Review ✅ Completed 2026-11-01T22:44:12.751078Z 7774309 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5c1b1ee5c9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Copy Markdown
Member Author

@codex review
@codex security review

Please review exact Phase 623 QEC PR #623 head 7774309df908f3568cf7d4055f3c2ff1c4a3f4ae. Earlier review at 5c1b1ee5 found the JUnit parameter-identity completeness defect. It is now corrected: inspect() enforces the exact 11 expected pytest node names, including each distinct revocation dimension/reason, rather than accepting four different labels under a single family. The original CI JUnit artifact confirmed the exact names. New synthetic XML regression deliberately substitutes four operation-only cases with unique labels and requires refusal, plus wrong dimension/reason and missing-parameter cases.

Executed current-head evidence: dedicated Phase 623 run 41356155761: SUCCESS, real Ed25519 sign/verify, 15 verifier contract tests passed, 11 real two-client IRC socket trials passed, JUnit inventory check passed with both per-run artifacts retained. Full Python/container run 41356155675 is a separate check; do not assume it passed while still running.

Review only actionable defects, minimal reproduction, expected/actual and executed/inferred. No changes to runtime, historical data or QEC decoder. PR stays unmerged until review and full CI clear; no automatic merge.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. More of your lovely PRs please.

Reviewed commit: 7774309df9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@EmergentMonk
EmergentMonk marked this pull request as ready for review November 1, 2026 22:42
@sourcery-ai

sourcery-ai Bot commented Nov 1, 2026

Copy link
Copy Markdown

Sorry @EmergentMonk, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 8 hours and 33 minutes by commenting @sourcery-ai review. Upgrade to get a review now.

@EmergentMonk
EmergentMonk merged commit 27bbc79 into main Nov 1, 2026
6 checks passed
@EmergentMonk
EmergentMonk deleted the test/phase623-end-to-end-adversarial-trial branch November 1, 2026 23:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant