Repository navigation
fix(projection): settle Codex passes after session project_path collisions - #4326
ScriptedAlchemy wants to merge 6 commits into
Conversation
…ect-path-collision
…ect-path-collision
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
There was a problem hiding this comment.
Devin Review found 1 potential issue.
1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
| // Two concrete working directories name one session: it was read | ||
| // through another rollout, or a resumed rollout moved its cwd. The | ||
| // stored cwd stays, like the session's title and annotations, and each | ||
| // message row keeps its own source. Replaying the same records in the | ||
| // same order therefore reaches the same row, instead of dropping every | ||
| // later message of the session as a collision. | ||
| actual.project_path.clone() |
There was a problem hiding this comment.
🔴 Later working directories lose scoped messages
When a profile session moves directories, reconcile_session_rows_detailed keeps the first path for every message. Scoped searches filter by that session path, hiding later messages from their directory.
Learn more
A session row contains one project path, but the same profile session can now contribute messages from several working directories. The project-scoped message count and search join every message to that single row and select by its path push_project_identity_predicate. Keeping the first directory lets the projection finish but does not preserve the directory of later messages for these reads. Git history attribution also falls back to the session path session_activity_rows.
Example: A profile Codex session emits message A under /work/one and message B under /work/two. The stored session path remains /work/one. A search scoped to /work/two returns neither message, while /work/one includes both.
Recommended fix: Preserve each message's working-directory identity in a queryable projection and use that identity for directory-scoped message reads. Keep the session-level fallback only for messages without their own directory evidence; cover a session spanning two directories with scoped search and count tests.
Was this helpful? React with 👍 or 👎 to provide feedback.
This branch has not been deployed
1 participant
Fixes #4279
What was broken
Two defects in the Codex session projection made a collision turn into a pass that never settles.
reconcile_session_rows_detailedtreated two different concreteproject_pathvalues for one session as a conflict. A session row written earlier under another working directory (hook payload, LCM, older build) therefore refused every later message of that session asprojection session output collided ... project_path. Each refused message is parked as a terminal collision and its text is never projected.projection_queue_has_itemscounted terminal rows. They are never drained again, but the host drain reporteddeferredfor them forever, so every user-scope pass endedBackpressured/ingest_pass_backpressured. That is thehistorical_retry:ingest_pass_backpressuredin the report. It also means rows already parked on a beta.88 profile keep every pass backpressured until this lands.Fix
project_pathstays when a session names another cwd, like the title and annotations. Each message row keeps its own source.transcript_path, lineage andproject_keyconflicts still report a collision.projection_queue_has_itemsignores rows parked on the terminal deadline.Evidence
User-scope Codex pass over a synthetic rollout whose session row already exists under another cwd (
transcript_ingest_suite):Backpressured { admitted_units: 1, rejected_units: 1 })Complete)Mutation checks:
project_pathchange:user_scope_projects_a_codex_rollout_whose_cwd_differs_from_the_stored_sessionfails withstill retrying: ... ingest_pass_backpressured.projection_queue_has_itemsfilter:user_scope_settles_after_a_genuine_codex_session_collisionfails with the same message, andgenuine_session_collision_is_recorded_and_later_sessions_projectfails with "a parked terminal collision is not pending work".Verified after merging origin/master:
tracedecay-global-dbandtracedecay-sessionsunit_test,//crates/tracedecay:transcript_ingest_suite,:session_suite, clippy (--config=clippy) andrustfmt_check_2024for the touched crates.About the CPU in the report
The reporter's beta.88 (950bf21) does not contain #4153, #4154 or #4173, all merged 2026-10-16/17. They address the bulk of the sustained load: #4154 removed a per-replay-sweep
GROUP BYover all observations that wrote about 148 GB of TEMP in 4 hours, and #4153 removed the whole-store projection output-state rebuild that stalled large stores. #4173 fixes the mimalloc owner-heap crash and does not affect CPU. This PR removes the unending backpressure retry and the dropped messages. It does not claim to account for the full 1.4 cores.Not in this PR