Skip to content

Publish native HTTP and SSH pushes through resident custody - #36

Open
forhappy wants to merge 36 commits into
mainfrom
codex/native-metadata-replay
Open

forhappy wants to merge 36 commits into
mainfrom
codex/native-metadata-replay

Conversation

@forhappy

@forhappy forhappy commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

After the packed schema cutover, stock receive-pack still called removed SQL object ingestion and returned HTTP 500. This PR routes authenticated HTTP/SSH pushes through resident staging, native verification, certified joint catalog/ref publication and registered exact recovery. The resident retains ownership after request loss; authorized responses follow the durable completed root. Metadata preparation streams bounded shards and reuses existing source descriptors.

The serving pool reuses idle generations and retains their physical ownership until actual release. Push publication distinguishes mutex contention from real capacity refusal, waits under the existing custody ceiling, rechecks custody and captures its exact held command before yielding. Controller failures retain bounded diagnostics without retaining worker-owning error objects. Shutdown preserves Cell, workspace and exact recovery through physical drain.

Commit-check reads and starts also queried the removed objects table and returned HTTP 503. They now read authenticated native headers under the resident serving snapshot and pass a private, bounded commit-membership certificate to typed receivers. The final transaction verifies actual Cell/owner, current access, the exact live pin and retained generation, then rechecks reporter/context/version and writes atomically. Existing check metadata, UUID binding and creation order are reused. Durable policy rejections retain their receipts and map to the correct HTTP status. Inputs are bounded at 4 KiB and pages at 32 contexts / 256 KiB. Membership lookup reads metadata without hydrating pack bodies.

Validation: both new actual-resident receiver families pass for SHA-1/SHA-256, covering MAC tampering, actor/repository/OID substitution, another Cell, noncommit targets, retained generations, policy changes, revocation, visibility and actual producer drain. Stock-Git HTTP checks and branch-protection integrations pass, including concurrent retries, pagination, delayed policy changes and independent recovery. Full frozen-source workspace still fails: server library 715 passed / 2 failed; multi-server 77 passed / 29 failed / 9 existing ignores; three standalone aggregates fail. The two library cases (serving rollover capacity and bound expiry terminal state) pass isolated from the same binary; their workload-dependent failures remain unresolved. Unique workspace totals are 830 passed / 34 failed / 9 unexecuted ignores, excluding nested summaries and focused reruns. All-target Clippy with warnings denied, server build, formatting, diff checks and all 96 Python harness cases pass.

This PR is not release qualified. Exact parent 3e40f19 Linux push/PR Verify runs fail multi-server at 75/35/9 and 76/34/9, with all 715 server library cases passing (Rust/Cargo 1.98.1, Git 2.55.0). New-head Linux qualification is required. Highest priorities: workload-dependent library failures, Linux bulk/rejected-push custody and pre-Bind terminal refusals, native pull/ref and default-branch metadata, generated merge/rebase writers, peer residency, source-independent backup, reachable-only filtered fetch, standalone resident fixtures, complete physical custody/recovery, final DDL and large-history/team capacity gates. No tests are disabled, quotas raised, or retired SQL object/ref authority restored.

Frozen source fingerprints, failed reproductions, intermediate adapter failures, full-workload failures and isolated probes are in docs/evidence/native-checks-ci-20261005.json. Earlier validation remains attributed to its source in docs/evidence/push-admission-ci-20261005.json, serving-rollover-ci-20261005.json, native-writer-ci-20261005.json and push-workflow-ci-20261004.json. Contracts and implementation status describe the remaining gates.

The SSH publication fixture selected a port by binding and immediately
closing it, then asynchronously initialized a server before binding again.
A competing listener at that real call site reproduces AddrInUse. Retain
the original bound listener and hand it to the existing supervised startup
API for initial startup and every restoration in this fixture family.

Assert that another binder cannot claim the fixture port before handoff.
All original refusal, disconnect and cold-preparation scenarios pass in
the 105-case multi-server rerun. No bind retries or deadline changes.
Reuse the existing root purpose and ETag reservation in a bounded
canopy-pack-v1 envelope. Reject unversioned or unknown remote formats
before workspace reclamation, probes, identity writes or Cell activation.
Reuse the owner/worker fences with the new managed runtime directory;
reject and retain legacy local state. Include the boundary in release
source hashing and update affected fixtures and harness paths.

Local hard-cutover work only: the old production Git schema and command
registry still require conversion with every producer and reader before
this branch can be released. No legacy decoder or migration is added.

Validation: 661 unique workspace Rust cases, 8 isolated RustFS cases,
96 Python harness cases, warnings-denied all-target Clippy, formatting,
doctest completion, and server build. Original red format regressions
and the earlier SSH suite failure are retained separately.
Reuse the bounded typed publication contract for actual Repository Cell migrations, activation and maintenance. Initialize private empty catalog/ref roots before Ready and authenticate retained initialization on cold load without new allocation.

Production Git producer/reader conversion remains an unreleasable local cutover. Qualification reads published Cell roots through the sparse VFS, uses scoped provider keys for corruption injection and releases connection guards before awaits.
Retain the actual accepted Begin receipt in the existing logical request row,
sharing the bounded authenticated record and lookup with staging. Startup
recovers it before another Begin and checks current fence/custody separately.
Explicit Claim can recover the authenticated original after operation reaping
without restoring old custody or displacing a successor.

Update initialization/compaction admission checks, command codecs and source
hashing. Qualify both object formats, SDK expiry, fresh-owner restore, rollback,
immutability and purpose separation. Preserve initial receipts in fixture
state hashes and target fault injection at the actual publication update.

Validation: 271 publication + 3 actual startup tests; all-target workspace
Clippy with warnings denied; canopy binary build; fmt/diff and frozen sources.
Local checkpoint only: remaining producer/reader cutover and full durable
custody-command recovery are required before release.
Persist original SDK snapshots before Begin and retain positive and denied
custody phases atomically with domain execution. Reuse the typed domain
receivers, recorded receipts, namespace allocator and independent pins.
Discover pending initialization and successor grants after cold restore.

Unbind raw custody commands in production; explicit qualification fixtures
retain domain receivers. Keep this partial cutover local pending service
conversion, compact history archival and full capacity qualification.
Retain original custody and registrar commands in the staging service and
fair preparation dispatcher. Preserve both identities through cancellation,
registrar uncertainty and closed recovery; gate absent execution on the local
fence and deadlines while returning recorded knowledge first.

Remove caller-owned raw renewal APIs. Restore renewals with the existing
shared session fence so a failed fresh observation also fences old resolvers.
Charge bounded intent/body copies without raising admission or byte caps.

Qualify 286 publication and nine startup/workspace tests, warnings-denied
all-target Clippy, server build, frozen sources, dependency pins and protected
checkout files. Document asynchronous file attribution and its native studies.

This is an unpublished, unreleasable cutover checkpoint. Current-owner fencing
for cold sessions, orphan lifecycle/history archival, complete production
producer/reader/schema conversion and the remaining runtime/capacity gates
are still required.
A fresh CheckPreparation can still return the previous owner's historical lease after actual owner restoration. Reopening that result incorrectly made the old token usable. Require a target-bound server-owned authority source and compare the admitted incarnation/epoch before and after fresh lease probes. Production reuses validated Cell Control and live node advertisements.

Carry the source through preparation and staging handoff, base/frontier loading and standalone positive recovery. Failed observations permanently fence shared sessions; original known outcomes remain recoverable. Explicit registered Claim under the current owner can still restore a usable session.

Validation: 289 publication and nine real startup/workspace tests on frozen source, workspace/all-target Clippy with warnings denied, canopy binary build, formatting, dependency/protection and documentation-link checks. New SHA-1/SHA-256 regressions cover cold old-owner renewal replay, current-owner takeover and missing/corrupt authority repair without un-fencing old sessions.

This is a local unreleasable cutover checkpoint. Cold staging reconstruction, orphan intent lifecycle/history archival, production ingress/read conversion, retention and full-history/team capacity remain open.
Restore all seven original custody actions without changing identities or
receipts. Retain positive and negative history before fresh owner/lease
checks, including after closed-service recovery and SDK expiry.

Wake bound callback supervisors from the shared permanent session fence;
join cancellation and drop owned resources before returning worker credit.

Validate 296 publication and nine real startup/workspace lifecycle tests,
all-target workspace Clippy with warnings denied, server build and format.
The production cutover and full-history/team capacity gates remain open.
Preserve exact original execution uncertainty in a separate authenticated stop fact. Reuse dispatcher admission for bounded keyset discovery and exact retirement recovery, and allow a stop beside its own pending preparation so its shared session can fence and drain.

All 307 publication and nine real lifecycle tests pass, with workspace all-target Clippy, build, format and frozen-source/static checks. Full library qualification passes 585 of 590 cases; five legacy consumers still query the already-removed objects table. Production lifecycle wiring, reader/producer cutover and capacity qualification remain open. This is an unpublished implementation checkpoint, not a release.
Retire only the tracked transition's expired unresolved initialization head using the existing authenticated stop factory and exact completion. Preserve original outcomes and SDK identities; choose an explicit successor from a receipt-watermarked indexed observation and reject inconsistent state instead of treating it as absence.

Seven new production-registry/schema regression families cover both object formats, real owner restoration after SQLite deletion, original history, automatic retirement, and authority/purpose/context refusal. Full library:592pass/5fail; the same five legacy readers still query the removed objects table. Nine real lifecycle cases, Clippy, build, fmt and frozen/static checks pass. The cutover remains unpublished and incomplete.
@forhappy forhappy changed the title Bound native metadata preparation and retain artifact hash ownership Own bounded native preparation and resident staging Oct 5, 2026
@forhappy forhappy changed the title Own bounded native preparation and resident staging Bound native preparation and add resident workflow ownership Oct 5, 2026
@forhappy
forhappy marked this pull request as ready for review October 5, 2026 04:05
@forhappy forhappy changed the title Bound native preparation and add resident workflow ownership Publish native HTTP and SSH pushes through resident custody Oct 5, 2026
A busy publication mutex was reported as exhausted capacity and aborted
resident receive-pack workflows. Wait under the original custody ceiling,
then capture the held command synchronously with fresh custody checks.
Keep genuine quota refusals immediate and retain the original prepared
command, registered recovery and completion receipt.

Surface bounded controller diagnostics before and after Bind without
retaining errors that own physical worker pins. Record stop before those
payloads are dropped so credits cannot be reused before cleanup.

Add real bound-handoff contention, quota and expiry regressions plus
resident error-ownership coverage for both Git object formats. Report CI
tool versions and preserve failed reproductions and qualification limits.
The full workspace still has 34 integration failures; no tests are hidden.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant