Repository navigation
Fix escaped origin identities, endpoint probes and relay error propagation - #80
Conversation
Keep encoded path boundaries and separators distinct so different immutable origin objects cannot reuse write-once blocks. Split digest segments before unescaping and document coordinated upgrades for affected URL identities. Add literal identity vectors and real cache-isolation regressions without changing ChunkKey or HRW.
Keep endpoint size and Range support separate from digest-addressed content. Add a dedicated coalescing probe group so concurrent origins cannot share metadata flights, while refreshed signatures still coalesce and cached blocks remain shared across registries. Cover both probe orders, unknown totals, independent stalled probes and cross-origin cache reuse.
Prevent net/http from ending a failed chunked stream as a successful response. Abort after committed source errors, propagate known relay geometry mismatches, and retain clean early failures. Cover both response framing modes and interruption across a real intermediate engine; update protocol behavior and upgrade notes.
Charge definitive block protocol failures to the parent circuit rather than leaving it healthy after a clean HTTP EOF. A soft failure would be reset by every later transport success, so quarantine the invalid parent immediately and retain normal cooldown recovery. Add a real repeated-request regression proving origin fallback and cache reuse. Address Copilot feedback on PR #1; refs #4.
Prevent pre-quarantine transport completions from clearing newer circuits or consuming half-open probe slots. Capture entry identity and generation for block and roster requests, then check and record their outcome under one lock. Validate relay geometry before recording success through StreamValidated, using the bounded request context. Add deterministic concurrent regressions for delayed Get/Stream/roster success, stale failure after recovery and validator context bounds. Address the second Copilot review on PR #1; refs #4.
Derive the expected chunk key from the supplied Origin URL, namespace and block index before cache lookup or upstream I/O. A malformed requester must not blame an honest parent for an unrelated object geometry. Preserve cache-only reads without Origin and decline caller addressing errors cleanly. Cover buffered/streaming and warm/cold relays, wrong URL/chunk/namespace pairs, healthy circuits and valid control reads. Address the third Copilot review on PR #1; refs #4.
…fix/origin-identity-and-relay-errors # Conflicts: # docs/fetch.md # docs/peer.md
Record upstream issue references and Go 1.27 coverage. Clarify that malformed URLs are rejected before metadata caching and roster success cannot consume outstanding data probes.
Coldwings
left a comment
There was a problem hiding this comment.
Self-review: no remaining blocking findings in the current PR diff.
Reviewed escaped path/digest segmentation, field-separator exclusion, endpoint metadata and probe/content singleflight separation, requester key/Origin consistency before cache access, response commitment/error handling, and breaker entry/generation lifetime across Get, Stream and roster requests. Half-open recovery keeps admitted siblings live until all finish, with failure veto and separate roster slot ownership. Cut-through forwarding and digest block reuse remain intact; trusted immutable origins and uniform cluster configuration remain explicit preconditions.
Real HTTP regressions reproduce all three defects on current upstream main and pass on this branch. Reviewed the four historical Copilot concerns against their fixes and deterministic regressions; older fork reviews do not substitute for review of this head. Original hash constructions and independent golden vectors are unchanged, and the Python reference checks pass.
Main-module vet, race tests, coverage tests and build pass with Go 1.27.0 (92.5% total coverage). Separate K8s-module vet, race tests, coverage tests and build pass. Formatting, diff checks and all eight documentation checks pass with no advisory notes. Package contracts, CHANGELOG and proposed ADR-0004 describe the coordinated upgrade and cold-cache requirement for affected non-digest URLs.
Residual validation scope: local real-HTTP tests establish transport/caching behavior, not Kubernetes end-to-end throughput at 100 Gbps. Remote CI and Copilot review are being checked separately.
|
Independent review of the current PR revision found one blocking API composition defect and one documentation correction.
I will fix the composition defect and documentation, rerun the required checks, and obtain both independent reviews on the updated head before considering merge. |
Keep underlying Prober semantics for shared probes and renewed-credential retries so nested content-keyed wrappers cannot merge endpoint capabilities. Add regressions and clarify escaped separator and HTTP stream framing contracts.
|
Addressed both findings from the first independent review in The shared request and credential-refusal retry now use the same operation callback. For probes, that callback delegates to an inner Corrected the obsolete separator documentation and clarified the completed Content-Length boundary: a later source error cannot be expressed by truncation once all advertised bytes are already sent, so fallible streaming sources must omit the sizer. Main-module and separate K8s-module vet/race/coverage/build checks pass, along with the independent Python goldens, formatting/diff checks and all eight documentation checks. Both independent reviewers are rechecking the updated head; remote CI is rerunning. |
Coldwings
left a comment
There was a problem hiding this comment.
Self-review of the updated revision: no remaining blocking findings in the implementation review.
The nested-wrapper defect identified by independent review is fixed in both the shared probe and the renewed-credential retry; the same operation callback preserves inner Prober semantics. Both new regressions fail before the fix, and the probe suite passes ten race runs after it. Reviewed cancellation, bounded flight lifetime, refusal retry, and unchanged content-fetch semantics. Corrected stale separator documentation and narrowed the HTTP error-signal contract to incomplete Content-Length or unfinished chunked framing.
Full main-module vet/race/coverage/build checks pass (Go 1.27.0, 92.4% combined coverage); separate K8s-module vet/race/coverage/build, independent Python hash/golden checks, formatting/diff and all eight docs checks pass. Independent reviews and CI are being checked against this updated head before merge.
|
Independent re-review of the updated PR revision: no blocking findings from either reviewer.
Residual limits: custom fetchers without Prober must preserve endpoint isolation themselves; unresolved geometry deliberately prevents caching rather than promising validation. Failure-first cohort ordering, permit eviction/reuse and cancellation during cohort recovery have less direct test coverage, with no defect found by static review. Local tests do not establish Kubernetes end-to-end throughput at 100 Gbps. |
Mark ADR-0004 accepted in the merge-bound revision so the decision and its implementation enter main together after review.
|
Final independent review: both reviewers report no blocking objections for the merge-bound revision. The final increment only changes ADR-0004 and its index from proposed to accepted, so the decision takes effect together with its implementation when this PR merges. Source and tests are identical to the previously independently reviewed revision; both reviewers verified the exact final diff and retained their no-blocker verdicts. All earlier findings are addressed. The documented validation limits remain unchanged. Final self-review also confirms that the ADR status/index agree and all eight local documentation checks pass. The final revision's CI must finish successfully before merge. |
Coldwings
left a comment
There was a problem hiding this comment.
Final self-review: no blocking findings. Verified that the merge-bound ADR status and index both say accepted, and all eight local documentation checks pass. Source/tests/modules are unchanged from the fully validated preceding revision. Both independent reviewers separately verified this exact final head and retain their no-blocker verdicts. Remote CI is checked separately before merge.
Fixes #77
Fixes #78
Fixes #79
Distinct escaped origin paths could return another object's cached bytes, endpoint-specific Range metadata could leak across origins serving the same digest, and a failed chunked peer relay could complete as HTTP 200 without a read error.
Changes
fetch.Prober; nested wrappers preserve that interface on shared probes and credential-refusal retries.http.ErrAbortHandlerwhen sources fail; propagate known relay-length mismatches and never cache partial or invalid blocks. Quarantine the actual faulty parent so later requests can fall back to origin.Validation
With Go 1.27.0, the main module passes:
Combined statement coverage is 92.4%; chunk 99.1%, fetch 92.4%, engine 88.4%, peer 90.5%, registry 90.5%. Formatting and diff checks pass. All eight documentation checks pass, with no advisory notes.
The separate
providers/k8smodule passes vet, race tests, coverage tests and build (provider package 81.8% coverage). The main module remains dependency-free.Regression tests copied to current upstream main reproduce all three defects: distinct equal-length escaped objects collide, both probe orders mix capabilities/unknown totals, a stalled endpoint blocks an independent probe, and a chunked source failure returns no read error. They pass on this branch. Independent review also reproduced a nested-wrapper probe composition defect; regressions fail before delegation is fixed and pass after it, including ten consecutive race runs. Real HTTP tests cover incomplete Content-Length failure framing, multi-hop truncation, quarantine/fallback, and requester identity consistency; deterministic concurrent tests cover stale completions and half-open sibling outcomes. A source error after the entire advertised Content-Length has already been flushed cannot be signaled by truncation; fallible streaming sources must omit the sizer and use chunked framing.
Compatibility and rollout
Digest identities, ChunkKey construction, HRW and independent golden vectors are unchanged. Affected non-digest URL identities now retain escaped paths; upgrade all nodes together when serving encoded separators, reserved characters or non-ASCII paths. Cache stores restart cold; do not carry old URL-keyed entries through an external warm-cache layer. Each new endpoint performs its own one-byte capability probe, even when its digest blocks are cached elsewhere.
Review history
The earlier PR was incorrectly opened against the fork and is closed: Coldwings/dart#1. Its four Copilot findings were fixed with regression tests and resolved; those older reviews do not establish approval of this PR's current head. This PR targets
data-accelerator/dart:mainfromColdwings:fix/origin-identity-and-relay-errorsand uses the upstream issues above.Checklist
go vet ./.../go test ./... -race -count=1/go test ./... -cover -count=1all pass (AGENTS.md)docs/<pkg>.mdupdated for behavior/contract changes; no new packagebash scripts/check-docs.shpasses