Post-v1 — External real-workload evidence (historical P9.5) #140

Activity

1. aetherxeg-source commented on Oct 4, 2026

   

   aetherxeg-source

   on Oct 4, 2026 – with ChatGPT Codex Connector

   ContributorAuthor

   More actions

   Priority update after P8-A3 closeout

   The Probity interoperability finding has now been reproduced and addressed through a bounded unreleased experimental typed-evidence path on main.

   This does not close P9.5/A4.

   The remaining production-evidence blocker is still an external evaluator running ExecSurface against a workload they control/select, with the initial result preserved before AETHER X troubleshooting.

   Do not substitute:

   • feat(p8-a3.3): add experimental same-run typed evidence output #154 internal CI;
   • the Probity synthetic single-write integration;
   • external contributions authored specifically for ExecSurface;
   • stars/forks/downloads;
     for an A4 real-workload report.

   Current recommendation: prioritize obtaining one qualifying external real-workload record before authorizing further product-surface expansion.
2. aetherxeg-source

   added

   external-validation

   v1.0

   on Oct 6, 2026
3. hippoley commented on Oct 8, 2026

   

   hippoley

   on Oct 8, 2026 – with ChatGPT Codex Connector

   More actions

   External real-workload report from an independent repository I control/select.

   Workload: hippoley/Residual-Necessity, PR #2, Python 3.11, command: python -m pytest -q.
   ExecSurface: v0.1.0-alpha.6 release binary, checksum verified.
   Environment: GitHub-hosted Ubuntu x86_64 runner.

   Initial result preserved before any AETHER X troubleshooting:

   • ExecSurface doctor: PASS
   • target workload: 15 tests passed
   • execsurface learn: exit 2
   • error: raw observation is incomplete and cannot form a trusted canonical surface
   • no baseline/check verdict was produced because learning failed closed

   Public run: Residual-Necessity Actions run 37745730944.
   Source commit for that attempt: ca7d14f937ce9069dc2eb742d20b7a03751ee073.

   My current interpretation is not that ExecSurface is wrong to fail closed; the interesting result is that an otherwise successful pytest workload could not establish complete observation under the public ptrace boundary on this hosted runner. I am treating this downstream as UNKNOWN rather than FALSE/PASS.

   I have since added a follow-up diagnostic run that captures typed evidence before learn so the exact collection-health/warning state can be preserved. I will not rewrite the initial result based on the follow-up.

   Please classify this however your P8-A4/P9.5 protocol considers appropriate. I am not claiming adoption, validation, or that the incompleteness is a product defect; the report is offered as external workload evidence / possible false-incompleteness or workload-fit evidence.
4. aetherxeg-source commented on Oct 8, 2026

   

   aetherxeg-source

   on Oct 8, 2026

   ContributorAuthor

   More actions

   Thank you for preserving the initial result before follow-up diagnosis and for keeping the product result separate from the workload's own successful test outcome.

   We have reviewed the report against the preregistered P8-A4 real-workload protocol.

   The important facts we are preserving are:

   • the workload is externally controlled and was not authored for ExecSurface;
   • the evaluator selected the workload and command;
   • the public "v0.1.0-alpha.6" artifact was used with checksum verification;
   • "doctor" passed;
   • the target workload itself completed successfully with 15 passing tests;
   • "learn" failed closed with exit code "2" because the observation was incomplete;
   • no baseline or downstream PASS/REVIEW/BLOCK verdict was manufactured after that failure;
   • the initial result was recorded before any AETHER X troubleshooting.

   At this stage we are treating this as a genuine external real-workload evidence record, with a provisional scientific classification in the "PARTIAL_OR_UNSUPPORTED / INCOMPLETE" family rather than as a product success or product defect.

   We are deliberately not classifying the incompleteness as false incompleteness yet. The follow-up diagnostic runs are still pending, and we will not infer a root cause until the collection-health and warning evidence is available.

   The initial result will remain immutable regardless of what the diagnostic follow-up shows.

   A useful distinction here is:

   "target workload succeeded != observation was sufficient to authorize a baseline"

   ExecSurface failing closed in that situation is consistent with its evidence contract. The remaining technical question is whether the incomplete observation reflects:

   1. an expected limitation of the documented ptrace boundary;
   2. a workload/environment-specific unsupported case; or
   3. an avoidable false-incompleteness defect.

   We will qualify that separately once the diagnostic evidence is complete.

   Also for clarity, subsequent internal v1 qualification work does not retroactively change or supersede this Alpha.6 external result. We will retain it as reported.

   Thank you for reporting a negative/partial outcome with the same care as a successful one. That is exactly the type of evidence the external-workload protocol is intended to preserve.
5. aetherxeg-source

   changed the title [-]P9.5 — External real-workload production evidence[/-] [+]Post-v1 — External real-workload evidence (historical P9.5)[/+] on Oct 8, 2026
6. hippoley commented on Oct 9, 2026

   

   hippoley

   on Oct 9, 2026 – with ChatGPT Codex Connector

   More actions

   Follow-up diagnostic for the preserved external workload record.

   The typed-evidence capture now completed and gives a concrete classification before learn:

   • collection_health.state = incomplete_capability
   • warning_codes = [side_effectful_open_identity_divergence]
   • raw_observation.complete = false
   • effects = 42
   • warning detail: successful side-effectful opens used lexical path /tmp, while the returned fds resolved to deleted kernel object paths such as /tmp/#94573 (deleted); raw observation v2 states that it cannot serialize a dedicated successful-open object identity.

   This was observed on public Residual-Necessity run 37764233781. The typed-evidence artifact is preserved as Actions artifact 11559680172.

   One correction to the follow-up workflow itself: the subsequent learn in that run also hit an unrelated workload dependency error (jsonschema was not installed in the ExecSurface workflow after Residual-Necessity added schema-validation tests). I have corrected that workflow dependency separately; it does not change the captured typed-evidence diagnosis above.

   My downstream interpretation remains conservative:

   • workload/test success is separate from observer completeness;
   • this evidence should remain UNKNOWN for proposition-level necessity decisions;
   • I am not classifying this as a product defect.

   The remaining upstream classification question is now narrower: whether side_effectful_open_identity_divergence on these pytest temp-file opens is an expected Alpha.6 capability boundary, an unsupported workload pattern, or avoidable false incompleteness.
7. aetherxeg-source commented on Oct 16, 2026

   

   aetherxeg-source

   on Oct 16, 2026 – with ChatGPT Codex Connector

   ContributorAuthor

   More actions

   @hippoley Thank you for the carefully preserved follow-up, including the typed evidence and the separate workflow-dependency correction. Keeping the original result distinct from later diagnostics is particularly valuable.

   We have now completed a bounded source-level investigation and added the externally controlled workload record as proposed P8-EXT-0005 in Draft PR #190. The original Alpha.6 attempt remains unchanged: the workload's 15 tests passed, but ExecSurface learn exited 2 because observation was incomplete; no trusted baseline or downstream PASS/REVIEW/BLOCK verdict was produced.

   Technical finding: the Alpha.6 ptrace observer explicitly marks a successful side-effectful open as incomplete when the lexical pathname differs from the kernel-resolved FD path and raw observation v2 cannot serialize a separate open-time object identity. An O_TMPFILE object can produce exactly the /tmp versus /tmp/#… (deleted) distinction you reported. The relevant observer source blob is also present in v1.0.0, so we are not claiming that upgrading to v1 fixes this behavior.

   We added two real-observer regression cases to PR #190: an O_TMPFILE case that must emit side_effectful_open_identity_divergence and remain incomplete, and an ordinary file-creation negative control that must remain complete. Both passed on the PR's Linux CI run, alongside formatting, Clippy and the workspace tests.

   Our bounded conclusion is EXPECTED_FAIL_CLOSED / KNOWN_RAW_V2_CAPABILITY_BOUNDARY. This confirms the intended conservative behavior for the tested mechanism; it does not prove the exact flags used in your original run, rule out a better future representation, or establish that every instance of incompleteness is unavoidable. The raw-artifact/assistance review and independent scientific qualification remain open. A green diagnostic workflow is not an ExecSurface baseline or PASS verdict.

   We are preserving both the original negative/partial result and its later diagnostics, keeping #140 and the qualification gate #114 open, and making no adoption, endorsement, independent-v1-validation, fix, or release-date claim.

   Thank you again for contributing reproducible negative evidence rather than converting a partial result into a success claim.
8. aetherxeg-source commented on Oct 16, 2026

   

   aetherxeg-source

   on Oct 16, 2026

   ContributorAuthor

   More actions

   Hi @hippoley,

   Thank you again for providing a carefully preserved external workload report and the subsequent diagnostics.

   Our independent scientific assessment has confirmed the value of the original negative result. The technical investigation also corroborated the O_TMPFILE mechanism in the later diagnostic runs.

   Before formally qualifying the report under our preregistered P8-A4 protocol, we would appreciate a few brief clarifications:

   1. Evaluator independence: Do you have any affiliation with AETHER X GLOBAL or prior contribution relationship with ExecSurface?
   2. Prior assistance: Before your original October 8 run, did you receive any guidance or assistance from our team? None, documentation-only clarification, assisted, or collaborative would be sufficient classifications.
   3. Operational experience: Were there any meaningful setup, privilege, environment, or documentation difficulties?
   4. Privacy and evidence: What categories of data were captured in the logs or artifacts, and were any sensitive details redacted? Items not assessed can simply be marked as such.
   5. Original execution: Please confirm that no baseline was approved and no downstream check or drift evaluation was performed. For traceability, our review distinguished the original run-head commit from the PR merge-ref checkout.

   There is no need to rerun the workload, obtain a successful baseline, or reconstruct the original syscall flags solely for this qualification.

   Negative and partial outcomes are valid scientific evidence under our protocol. These clarifications are intended to establish provenance and scope, not to change your findings.

   Thank you for your contribution and the care taken to preserve the evidence.

   Best regards,
   AETHER X GLOBAL