Skip to content

ccp: read the resend markers and book replays on cumulative quantity (ibx#320, ibx#260) - #361

Closed
userFRM wants to merge 1 commit into
deepentropy:mainfrom
userFRM:fix/replay-restates-history
Closed

ccp: read the resend markers and book replays on cumulative quantity (ibx#320, ibx#260)#361
userFRM wants to merge 1 commit into
deepentropy:mainfrom
userFRM:fix/replay-restates-history

Conversation

@userFRM

@userFRM userFRM commented Jul 30, 2026

Copy link
Copy Markdown

Problem

At session start the gateway replays recent activity. A New/New report inserts the order through the cross-session recovery path, then its executions arrive behind it carrying their original ExecIDs and a resend marker. By then the order is tracked, so each execution books through the normal path — a fill event, a position move — for something that happened before the process started. The ExecID window cannot stop it: a fresh process has never seen those IDs. Neither 97 (PossResend) nor 43 (PossDupFlag) was read anywhere in the handler.

What this changes

The recovery insert seeds filled from the record's own CumQty instead of assuming zero, so a recovered order arrives with the quantity it has actually filled rather than looking untouched.

A marked report books on what it adds to that baseline, not on the increment it reports. Its CumQty is the order's cumulative filled quantity as of that execution, so what is news to this session is CumQty minus what the order already holds — nothing when it restates. Reading the cumulative figure is what makes replay safe in general:

  • the same report twice adds nothing the second time;
  • copies arriving out of order settle on the true total instead of losing whichever lands behind a later one;
  • a session long enough to have rolled those executions out of the ExecID window is not overcounted.

Live reports are unchanged: an increment is new quantity by definition, and the window is what stops a repeat.

A marked report is not vetoed outright. A CCP reconnect keeps this state and the gateway replays recent executions on the new session, so a fill that executed during the outage arrives marked with an ID this session has never seen and is the first news of it. It carries cumulative quantity the order lacks, so it books.

An execution with no ExecID is deduplicated on its content. It skipped the window entirely, so a replayed copy booked twice — the shape a replay takes, and precisely when the window matters. The fallback key includes CumQty, the one field that separates two slices of the same size at the same price inside one timestamp tick, including across a replacement that raised the total and left the same quantity behind twice. The key is spent only where the fill can actually book, so an execution arriving ahead of the recovery record for its order is not refused when that record finally lands.

update_order_filled saturates rather than wrapping, so a quantity large enough to overflow the counter cannot take an order's filled quantity round to nothing.

Tests

  • a_resent_execution_does_not_book_a_fill — both markers, with a live positive control.
  • a_fresh_process_does_not_book_the_history_it_is_replayed — end to end: recovery record then its replayed execution.
  • a_resent_execution_carrying_new_quantity_is_still_booked — the fill that ran during an outage.
  • a_replay_of_booked_history_adds_nothing_to_the_order — unordered, ExecID-less replay of already-booked executions, and a genuinely new one in the same batch.
  • a_marked_execution_delivered_twice_books_once.
  • a_raised_total_does_not_collapse_two_slices_into_one.
  • a_key_is_not_spent_before_the_order_exists.
  • two_same_priced_slices_in_one_tick_are_not_one_execution, an_execution_without_an_exec_id_is_still_deduplicated.
  • update_order_filled_saturates.

Each fails by name against a compiling reversion of the production change it covers.

No new wire traffic: this reads tags the gateway already sends.

The dedup window stops being the guard for a replayed execution, which is what made its depth a cliff (#344): a batch deeper than the window no longer held its own head, and for an order this session tracks the window was the only thing between the second copy and a second booking.

Closes #320. Closes #260. Closes #344.

Test plan

  • Mutation: reverting the resend path to the live one fails a_resent_execution_does_not_book_a_fill, a_fresh_process_does_not_book_the_history_it_is_replayed, a_marked_execution_delivered_twice_books_once and a_replay_of_booked_history_adds_nothing_to_the_order by name.
  • Mutation: keying the fallback on LeavesQty instead of CumQty fails a_raised_total_does_not_collapse_two_slices_into_one.
  • Mutation: spending the key before the order lookup fails a_key_is_not_spent_before_the_order_exists.
  • Mutation: seeding the recovery insert with zero fails a_fresh_process_does_not_book_the_history_it_is_replayed.
  • Mutation: not recording a marked ExecID fails a_marked_execution_is_remembered_for_its_unmarked_twin.
  • a_replay_deeper_than_the_exec_id_window_does_not_double_count covers ccp: the ExecID window is 1024 entries, so a replay batch deeper than that double-counts its own head #344 — the window is no longer the guard for a replayed execution.
  • cargo check --offline clean on --lib, --lib --features python, --bins, --examples, and each integration target individually.
  • tests/ib_paper_compat compared against a clean checkout of the base commit — identical sorted diagnostic sets.
  • cargo test --offline --lib — only the two known config::expiry_tests failures, which fail on the base commit for missing legacy tzdata (fixed separately in config: resolve the legacy timezone names IB states its times in (ibx#335) #336).

@userFRM
userFRM force-pushed the fix/replay-restates-history branch from 307fdf9 to aaa0c51 Compare July 30, 2026 14:56
@userFRM userFRM changed the title ccp: adjudicate a resent execution instead of trusting or ignoring it (ibx#320, ibx#260) ccp: read the resend markers and book replays on cumulative quantity (ibx#320, ibx#260) Jul 30, 2026
@userFRM
userFRM force-pushed the fix/replay-restates-history branch from aaa0c51 to faaf8bf Compare July 30, 2026 18:43
At session start the gateway replays recent activity: a New/New report inserts the order through the cross-session recovery path, then the order's executions arrive behind it carrying their original ExecIDs and a resend marker. By then the order is tracked, so each execution books through the normal path — a fill event, a position move — for something that happened before the process started. The ExecID window cannot stop it, because a fresh process has never seen those IDs. Neither `97` (PossResend) nor `43` (PossDupFlag) was read anywhere in the handler.

The recovery insert now seeds `filled` from the record's own CumQty instead of assuming zero, so an order arrives with the quantity it has actually filled rather than looking untouched.

A marked report is then booked on what it adds to that baseline rather than on the increment it reports. Its CumQty is the order's cumulative filled quantity as of that execution, so the quantity that is news to this session is CumQty minus what the order already holds, and nothing when it restates. Reading the cumulative figure is what makes replay safe in general: the same report twice adds nothing the second time, copies arriving out of order settle on the true total instead of losing whichever lands behind a later one, and a long session that has rolled those executions out of the ExecID window is not overcounted. Live reports are unchanged — an increment is new quantity by definition, and the window is what stops a repeat.

A marked report is not vetoed outright. A CCP reconnect keeps this state and the gateway replays recent executions on the new session, so a fill that executed during the outage arrives marked with an ID this session has never seen and is the first news of it. It carries cumulative quantity the order lacks, so it books.

An execution with no ExecID skipped the window entirely and a replayed copy booked twice, which is the shape a replay takes and precisely when the window matters. It now falls back to a key built from the fields that identify an execution, including CumQty — the one field that separates two slices of the same size at the same price inside one timestamp tick, including across a replacement that raised the total and left the same quantity behind twice. The key is spent only where the fill can actually book, so an execution that arrives ahead of the recovery record for its order is not refused when the record finally lands.

`update_order_filled` saturates rather than wrapping, so a quantity large enough to overflow the counter cannot take an order's filled quantity round to nothing.

The dedup window stops being the guard for a replayed execution, which is what made its depth a cliff: a batch deeper than the window no longer held its own head, and for an order this session tracks the window was the only thing standing between the second copy and a second booking. A marked report now adds what it carries above the order's own quantity, whether or not its ExecID is still remembered.

A marked report is recorded in the ExecID window even though the cumulative figure is what judged it: the same execution can arrive again without its marker, and the window is the only thing that can catch that copy.

Where the reconciled quantity is not the increment the report states — a report arriving after one this client never saw — the fill carries the reconciled quantity at that report's price rather than one execution's own terms. The order total and the position are right; the execution detail is approximate, and the handler logs it as such. Making it exact needs a capture establishing whether replayed executions can arrive out of order at all.

No new wire traffic: this reads tags the gateway already sends.

Closes deepentropy#320.
Closes deepentropy#260.
Closes deepentropy#344.
@userFRM
userFRM force-pushed the fix/replay-restates-history branch from faaf8bf to 93bdc87 Compare July 30, 2026 19:53
@userFRM

userFRM commented Aug 26, 2026

Copy link
Copy Markdown
Author

Closing this. It's in #409 along with the rest of the fork, which is easier to take in one piece than sixty separate branches.

@userFRM userFRM closed this Aug 26, 2026
userFRM added a commit to userFRM/ibx that referenced this pull request Aug 30, 2026
@userFRM
userFRM deleted the fix/replay-restates-history branch August 30, 2026 12:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment