Repository navigation
Conversation
Review the current implementation, area by area, for guards against situations that cannot occur in C-Two's real use: 16 real defects (the per-request chunk limit reproduced), about 8k production and 10k test lines of over-defense and dead code by static estimate, and the ledger's retention decisions that no longer hold. Sync the slimming ledger to the 0.10.1 release and close R20, whose guarded example code was removed in eee0fe5.
…variants Keep runtime, hosting, connection and error types at the crate root and move endpoint administration, owner control, diagnostics and contract tooling into the endpoint, owner, observe and contract modules. direct_ipc_endpoint, ping_direct_ipc and shutdown_direct_ipc take an optional endpoint context instead of separate *_with_context functions; None keeps the process configuration resolution. Generated Rust code follows the new paths.
… the resource bridge Keep the everyday API at the c_two root and move owner control, endpoint maintenance, diagnostics and contract tooling into c_two.owner, c_two.endpoint, c_two.observe and c_two.contract. Remove the Held alias, hold_stats, adopt_owner_stdin at module level, the public ConcurrencyMode and InputLifetime enums (values are string literals), the crm() identity defaults, infer_crm_from_resource with c3 contract infer, and the resource bridge with register(bridge=...). Snapshot and endpoint results are typed, c_two.error has __all__, and the native classes carry their public names.
…ats note at its source
… frames, drop dead allocator surface B8: a request is limited only by max_payload_size and max_reassembly_bytes; the global in-flight chunk count is no longer a per-request cap, and an oversized request reports a size error. B10: malformed call-control frames end the connection with a protocol error instead of being dropped. A8: invalid buddy geometry is an error instead of silently disabling buddy tiers, and MemPool constructors return Result. A2, A3, A12 and the dead server surface are removed. Callers outside these crates are migrated separately.
…lly, retire contended drops C8: a malformed shutdown ACK is a protocol error instead of an unacknowledged result. D4: first initialization scans the whole namespace directory instead of failing after 64 unrelated entries. D-drop: a listener Drop that meets gate contention hands its lease and gate pin to a short-lived thread that uses the close path's bounded wait, so contended drops no longer leave sockets and leases behind. D9: the arbitrary 4096-address sweep cap is removed.
…ed registry output F5c: mesh peers must speak the current protocol version. F6: the HTTP register path and c3 relay --upstream share one registration that retries only transport errors, never a version mismatch. F9: the arbitrary 1024-owner cap is removed. H10: c3 registry deserializes responses by type and reports malformed ones instead of printing no routes. H11: the TypeScript transport checks only the NACK envelope shape. F1, the unused RelayAwareHttpClient entry points and the F10 duplicates are removed.
…ove dead configuration surface H3: C2_RELAY_ROUTE_MAX_ATTEMPTS is removed; Core does one observation plus at most one refresh. H12: explicit max_execution_workers values are not capped. A8: c2-config rejects pool segment sizes that overflow u32. G4: relay connect errors keep their own type. E13: malformed control addresses are errors, and the swallowed duplicate Host shutdown is gone. cc.close() of a non-proxy raises TypeError, closed proxies keep hasattr working, and signature errors are no longer hidden. E3, E6, E7, H2, H5, the duplicate config validators and keys, G1, G6 and G-slot are removed.
…nd route-watch recovery code Callers of the changed allocator, chunk, server-address and lease APIs are migrated across c2-ipc, c2-core, c2-http, the Python native module, the Rust SDK and codegen tests. C1 removes the SyncClient preallocation API, injected pools and the pool-less IpcClient constructors; C3 removes the IPC route-watch recovery path with its Core and relay mappings; the remaining C8 dead branches are gone. H13: c2-contract exports canonical_json and lower_hex for codegen, and the relay-id suffix is no longer re-sanitized.
…ET_DIR in test fixtures The Core geometry test no longer treats 513 chunks as invalid: B8 removed the per-request chunk-count cap, so valid geometry over the reassembly budget is a capacity error, which a new regression case asserts. The c3 locators and the portable matrix and cross-language harnesses look for cargo outputs under CARGO_TARGET_DIR when it is set.
…cript IPC test After B8 the server rejects an oversized chunked upload either because the reassembly size exceeds its limit or because global chunk-processing permits are exhausted; both are ResourceUnavailable. The removed per-request chunk-count protocol error is no longer an accepted outcome.
… limit The relay client lifecycle no longer has a fixed owner limit, so the watch test that filled 1024 slots to force a watch failure tested a path that no longer exists. Three tests now drive the same behaviors through a real transport failure in a test-owned endpoint root: a closed lifecycle stops the retry, a live acquire succeeds while the watch is unavailable, and a recovered watch observes a real upstream withdrawal. Two comments that still described the owner limit are corrected.
The Drop tests compared a fixed gate hold with the background worker's bounded retry, so a slow scheduler could let the worker retire the slot before the test expected a reapable record. Each test now holds the gate until an independently opened lease description proves the worker has released ownership, then checks the exact outcome. The contended-Drop test confirms through a channel that Drop returns while the gate is held and accepts either settled state. An assertion that read a raw gate descriptor after its ownership moved to the worker is removed.
- c2-mem adds PeerMappingCache: each admitted remote read is a move-only PeerRead; the last read closes a dedicated view at once and starts a buddy view's idle window, retired by gc_idle. c2-ipc and c2-server adopt it in F2/F3; Node uses it now. - Python MemPool.free accepts each PoolAlloc once and only from its own pool, so a double or foreign free no longer corrupts the allocator. - Node reads the same C2_IPC_* client settings as Python and Rust. - TS and Node fail only the affected call for a well-framed bad reply or chunk and for local failures; the connection stays usable. - Node settles buffered data in order before recording the peer end, so a release frame read just before EOF no longer fails a completed call. c2-mem-ffi's connect entries adopt F4's caller-deadline API in F2.
…ting sent calls - c2-ipc: a well-framed bad reply or chunk, and a local failure such as opening a peer mapping, fail only their own call. Release-channel violations, including a zero loan id, and bookkeeping exhaustion deactivate the connection while calls already sent finish; only untrusted framing fails the other calls in flight. - Client peer mappings use PeerMappingCache: a dedicated view closes at its last read and a buddy view retires after the idle decay. The GC period keeps its 5 ms floor. - Connect and handshake follow the caller's ConnectAttempt deadline, and writes use the configured write_timeout. - ClientPool replacement, discard and sweep, and relay client retirement no longer abort sent calls: the old connection closes after their replies. Explicit close_all and Runtime shutdown still fail calls in flight. - An early reply stops a chunked upload at the frame boundary and keeps the connection, in Rust and in the generated TypeScript transport. - The C ABI 7 connect entries take a nullable timeout_millis, and the TypeScript IPC transport adds connectTimeoutMs. - ResponseLease and SyncClient::lease_response are removed. - Restored tests for invalid reply geometry and malformed, duplicate or oversized chunks with another call in flight, and the relay partial write after a real eight-byte IPC prefix.
…ose, serve budget - c2-server: replies wait for writer queue space instead of ending the connection. Receive-loop replies reserve the server's pending budget, and a call keeps its pending permit until its reply is written. - The listener closes after accepted calls and their queued writes drain. Deactivation keeps write access for accepted calls, and an unpublished SHM reply falls back to inline or chunked bytes. - Server peer mappings use PeerMappingCache and retire in the maintenance task, which keeps running for connections retained after shutdown. - Host startup binds the listener under the startup deadline and reports StartupTimeout. - Concurrent unregister of one route has a single winner. - Python ipc_overrides accept write_timeout, max_dedicated_segments and spill_threshold. - cc.serve(shutdown_timeout=30.0) validates the budget on entry and returns the shutdown outcome. - The 60 s retained-hold warning is removed with LocalCopyTracker.sweep_retained and RuntimeSession.sweep_local_copies. - Exact-byte reassembly tests count the chunk-arrival marks.
gRPC (core, Python sync and aio, Go, Java), Ray actors, Go net/http, Java ExecutorService, Python concurrent.futures, ZeroMQ linger, Dask, tokio, Erlang supervisors and Cap'n Proto, against pinned sources, with CPython's exit order for threads blocked in calls.
- AGENTS.md: deactivation, eviction and Runtime shutdown let calls already sent finish; a caller stops waiting only through its own call deadline or a genuine failure. - CONTEXT.md: deactivation also lets calls already sent settle. - Release notes: failure scope, cached-client deactivation, write backpressure, drain before listener close, connect deadlines, the serve shutdown budget, the three IPC keys, the removed hold warning. - Python usage, C ABI, TypeScript binding README and runtime lifecycle follow the same changes; the design notes the removed sweep. - Backlog: fix-block status, review findings, the next round, and post-dispatch outcome query and deduplication as a later item.
- c2-ipc has one closing path, deactivation: explicit close, close_all, pool replacement, discard and sweep, and unpublished candidates stop new calls and loans, fail only calls not yet written as pre-dispatch, and close the connection after calls already sent and their loans settle. Only untrusted framing and a stopped reader fail sent calls. - Continuation chunks of a started call are written after deactivation, and DISCONNECT waits until every dispatched call has replied. - A submission waiting for queue capacity fails pre-dispatch as soon as the connection deactivates instead of waiting behind a stuck write. - The peer mapping cache is released when the physical connection retires, so an evicted client no longer keeps the server's segments. - Relay shutdown and eviction deactivate upstream clients; forwarded calls in flight complete. - Node uses write_timeout, a zero loan id deactivates the connection as in Rust, peer dedicated indexes follow the wire range, and a local allocation failure fails only its call. - Generated TypeScript close() lets calls already sent, including chunked uploads, finish. - Tests cover Runtime shutdown with a call in flight in Core and Python. - c2-mem-ffi tests serialize environment changes, socket pair creation and compiler launches; the suite had failed intermittently because a child process inherited a test socket and endpoint tests changed the process environment.
- A well-framed request with a malformed descriptor or call control gets an error reply and the server keeps parsing; an undecodable descriptor or zero loan id only deactivates. A local mapping failure fails only its call. - Deactivation no longer cancels accepted work: admitted calls wait for their permits and reply, and chunked requests admitted before it keep their continuation chunks. A later call receives ResourceClosed saying it was not executed. Peer end still drops accepted calls that have not started, as in 0.10.1. - Shutdown moves admission to draining: new calls are refused, accepted work of the current generation finishes, and routes close after it. - A reply that cannot be allocated after the method ran reports ResourceOutputSerializing instead of ResourceUnavailable. - Core starts outgoing client deactivation at the beginning of Runtime shutdown, finishes the Host drain, and keeps observing the clients in the background; the caller's budget bounds only its own wait.
- Release notes: shutdown, eviction and relay shutdown let calls already sent finish; malformed requests fail only their call; a call refused after deactivation reports it was not executed; reply allocation failure after execution reports ResourceOutputSerializing; Node write timeout, zero loan ids and dedicated indexes match Rust. - Python usage (English and Chinese), runtime lifecycle and the TypeScript binding README state the semantics and the cost: a thread waiting without a call deadline on a stopped server keeps waiting. - Backlog: the G round is accepted; MemPool cleanup and the full CI remain.
- MemPool no longer opens or reads peer backings: ensure_peer_segment, ensure_peer_dedicated, open_dedicated_at, validate_data_at, copy_data_at and data_ptr_at are removed with the peer branches of stats, GC and internal creation. open_peer only validates a peer pool identity for PeerMappingCache, and peer pools still refuse allocation, release and owner access. - Tests that used the removed calls now go through PeerMappingCache or are covered by its tests. - The ownership gate proves the same properties on PeerRead: no raw mapping, no write and no unsafe-free copy during a borrow. The probe for adopting a dedicated mapping is removed with that entry point.
- Release notes list the removed MemPool peer access and state that receivers read peer blocks through PeerMappingCache. - The c2-config comment on MAX_IPC_POOL_SEGMENTS names the check that PeerMappingCache performs. - Backlog: the batch is code-complete; the full CI remains.
- Node loader: read libuv's uv_hrtime on every platform; strict C11 on glibc hides CLOCK_MONOTONIC. - c2-server: tempfile is a dev-dependency on every target, since a cross-platform test uses it. - Source validation and the local runner fetch core/Cargo.lock before the Node protocol fixtures build offline. - The ipc_memory matrix test injects the Windows Job seam instead of letting the harness read Popen._handle from a fake.
…l state - A deactivated client detaches its cached request and reassembly pools; the read, write and GC tasks drop their storage before the physical retirement fence, so a completed close no longer leaves idle buddy backing charged. - A server stops retaining idle backing before it publishes Stopped or Failed. Live allocations keep their backing until their last release; a restart re-enables retention and prewarms again. - A client that rejects a malformed handshake reply may leave bytes unread, which Linux reports to the peer as a reset; the lifecycle test accepts that peer-end proof.
- Build both environments in CI's source mode. Windows Python does not search PATH for an extension's DLL dependencies, so the system-linked native module could not be imported. - Every failure names its side and category: tool failure, scenario not established, or correctness failure. Candidate failures and tool failures on either side fail the run; baseline failures are recorded with their samples. - Linux mapping convergence counts MAP_SHARED mappings, keeps the total VMA count as data, and no longer truncates its wait at 15 seconds.
- c2-config owns time-budget validation: finite, non-negative and within the range every supported monotonic clock can represent (signed 64-bit seconds). Call, connect, shutdown, serve, startup and direct control budgets share it, and deadlines still check the actual clock addition. Windows now rejects the same values as Unix, and an oversized shutdown budget returns a configuration error instead of panicking. CallTimeoutError is replaced by TimeoutError. - Windows test fixtures confirm the server's handshake with a ping, wait for the writer to drain before using the book directly, write closing frames without a native flush, and match parallel replies by request id. A busy pipe within a caller's connect deadline now reports that deadline, as the connect-deadline rule requires.
- Linux mapping counts are three fields of the common {value, reason}
shape: total, shared and the shared mapping detail.
- Remove the FastDB CoreSDK preparation the source-mode build no longer
needs, and the job-level link mode it contradicted.
- Measure with perf_counter: time.monotonic on Windows ticks at 15.6 ms.
- Workers report their own pid in the ready marker, because a venv's
python.exe on Windows is a launcher whose pid is not the worker's.
- Worker output goes to files, not pipes that can fill and deadlock.
- Coordination markers wait out Windows file-sharing conflicts within the
existing deadlines; other platforms raise at once.
On macOS CI the standard HTTPServer bind hung in getfqdn('127.0.0.1'). The test now uses the existing loopback fixture, which binds without a reverse lookup, and fails if a reverse lookup is attempted.
…and the fast-path experiment
The protocol batch and the defect batch closed C02, R07's re-evaluation condition, R13's cancellation path, R19, R34, R35 and the two owner control-session refactors; remove or rewrite their ledger rows, drop the superseded R35 heartbeat analysis, and record that the fork's socu/protocol-batch is now the only remote copy of the stack.
…failures under repair
…dows closes A pre-handshake shutdown control stream now stays open until the client closes or the configured write timeout expires, so a Named Pipe peer reads the one-byte ACK before the server drops the stream; the hard-coded 50 ms post-shutdown wait uses the same budget. DISCONNECT records PeerCloseMessage as the first end cause before its ACK is queued, so a failed ACK write cannot record IoError first. The Node endpoint fixture accepts a read error after explicit close when c2-local reports peer end.
…ining chunks Each continuation of a refused chunked request used to queue its own capacity error, which exhausted the reply budget and deactivated the connection, so the next call on it failed. A chunked request now becomes settled at its first error reply: one reply per connection and request id, including when the receive loop and a chunk task race to refuse it. Its remaining chunks are discarded before they take a chunk-processing permit, returning any SHM loan they carry. A late chunk of an assembly that GC removed still gets one error. The settled marker lives in the chunk registry and goes at the request's last chunk, at connection end, or when the existing GC finds it older than chunk_assembler_timeout. Identity and generation checks that only request-id reuse could trigger are removed; clients never reuse a request id on a connection.
…limit and the PR CI repair
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Four accepted batches, stacked on 0.10.1 (
606b8f13):docs/split into reference, decisions, design, acceptance and research; aCONTEXT.mdglossary; AGENTS.md kept under 32 KiB.RouteAckremoved; deactivation, eviction and Runtime shutdown no longer abandon dispatched calls.Relative to 0.10.1,
core/,sdk/andcli/change by +38,599 / −69,293 lines.Breaking changes
IPC handshake 15, SHM 4, C/FFI ABI 7 and relay mesh 7, with no compatibility path to 0.10.x peers. Removed and renamed public names are listed in
docs/releases/unreleased.md.Performance
Paired benchmark against the batch baseline
b861f994, five pairs per platform (run 41444159385): 1-byte calls are 1.13–1.21× slower at P50 on Linux and Windows and 1.25–1.28× on macOS (about 40–60 µs more per call, throughput 10–17% lower); 64 KiB calls through the buddy pool stay within 6% on Linux and Windows and are 5–20% slower with the pool disabled; 4 MiB calls are 6–14% faster on Linux and Windows and 0.99–1.07× on macOS. The small-call regression misses the design threshold and was accepted; a write fast-path experiment recovered about 9% and was not adopted (record).Validation
d646fc12: full fork CI run 41444148760 passed on Linux (CPython 3.12 and free-threaded 3.14t), macOS, Windows Server 2025 x64 and Windows 11 ARM64; the paired benchmark above passed on all four.2ec4ddb2,55aca65aand63979601change documentation only.f0f9efaa), 41192397960 (e1630cf1), 41218012397 (08b8c65c).