Repository navigation
Storage TTL policy, outstanding-share finalization, TWAP accumulator, and batched redemption (#16, #14, #11, #15) - #156
Conversation
…cleDesk#16) Soroban archives entries when their TTL lapses, and an archived entry is unreadable until restored; an archived instance entry makes a contract uncallable. Four defects shared that root cause: - market-core bumped Market/Position to a fixed 60 days regardless of close_time, so a market closing further out could archive before it could be resolved and the resolver's get_market cross-call would fail. - resolver/treasury/reasoning-registry never extended instance storage, so Admin/MarketCore/Config/Daily/NextId lived only as long as the deploy-time default TTL. - the resolver extended only Price/Signers; Attestation/Count/Proposal/State were never extended, and a missing Count reads as 0 via unwrap_or(0) — silently resetting an in-progress signer tally. - the registry extended Agent only at register_agent, so an agent publishing for 60+ days was archived and publish_trace failed with AgentNotRegistered. Each contract now carries an identical src/ttl.rs (duplicated because contract crates cannot path-depend on each other — ADR 0002) with the shared policy: per-market entries target close_time + 90-day settlement horizon, clamped to max_ttl(), threshold = target/2 so hot paths rarely rewrite; instance target 30 days, threshold 7 days. - market-core: Market/Position bumps use ttl::market_target(close_time). - resolver: instance bumped via instance_get; all six enumerable per-market keys extended has()-guarded in bump_market_entries after every mutation; Attestation extended where written and, for configured signers, in finalize_signers; state() stays a pure view. - treasury: instance bumped wherever instance storage is read; exposure entries keyed to close_time (create-market arg, or a get_market read). - reasoning-registry: instance bumped on NextId access; Agent uses the long-lived target and is refreshed by every successful publish_trace. - docs/architecture.md documents the policy and links the keeper doc. Regression coverage: signer_vote_entries_survive_dispute_period (tally survives 61 days of silence), market_entry_survives_until_close_plus_horizon, position_ttl_tracks_close_time, instance_survives_after_long_idle, bump_market_entries_skips_missing_keys, instance_ttl_extended_on_agent_buy, agent_entry_extended_on_publish — all advancing sequence and timestamp together and asserting TTLs through testutils. Vendored wasm refresh and bindings regeneration land in the final commit of this branch (one refresh for all four issues).
…ims residual collateral (OracleDesk#14) A final market's collateral could sit in market-core forever: nobody could tell 'fully settled' from 'waiting on holders' because there was no running total of user-held shares, and void per-holder floors left rounding dust unclaimable by design. - SettleState side entry (DataKey::Settle(u64)): outstanding_yes/no, finalized_at, horizon_secs, paid_out, closed. Maintained by create_market, buy, sell (now checked_sub on sets_minted), redeem (both sides decrement, even at payout 0) and claim_pool_remainder; resolve/void_market stamp finalized_at. - Per-market redemption horizon snapshotted at creation from an admin default (365 days, bounded to 180..=1825 days via set_default_redeem_horizon). - finalize_market: permissionless after finalized_at + horizon; reserves settle_upper (aggregate floor >= sum of per-holder void floors, proven in math.rs and the Python model) plus any unclaimed LP remainder, then sweeps the market's entire remaining balance - expired claims, dust, remainder, fees - to the treasury and closes the market. Redeem/claim after close return RedemptionClosed; legacy markets without a Settle entry return NotSupported and stay redeemable. - Views settle_state / redeem_deadline; MarketFinalized event; debug_assert of reserve+outstanding==sets on pre-payout transitions. - Docs: architecture.md lifecycle + TTL interaction, STATUS.md limitation. Treasury redeem() doc notes the agent must redeem before the deadline.
…eads (OracleDesk#11) Spot price alone is a single-trade-grabbable reference; anything automated (agents, frontends) needs a price that requires holding a distortion. - Per-market accumulator (Uniswap-v2 semantics): every buy/sell folds the pre-trade price into price_cumulative before reserves move. Same-timestamp trades only refresh the price - nothing accumulates within one ledger, so moving the TWAP requires holding the price across a ledger boundary. - State lives in Oracle(id) and a 16-slot ring of Obs(id, k) entries, so the Market entry's XDR is unchanged and existing readers/decoders keep working. - Views: observe(id, seconds_ago), twap(id, window_secs) (floor division) and price_cumulative(id) (= observe at now). Between bracketing observations the cumulative is chord-interpolated (exact - piecewise-linear between trades); past the newest observation the last price is extended counterfactually; earlier than the surviving ring returns ObservationTooOld; legacy markets return NotSupported. - Overflow: checked i128 arithmetic throughout; the 10-year alternating-trade test bounds the accumulator at elapsed * MAX_PROB_BPS, ~24 orders of magnitude below i128::MAX. - Oracle/Obs entries share the market's close-time-aware TTL target (OracleDesk#16); unwritten ring slots are never written and missing slots are skipped. - agents: MarketState.twapBps and decideTrade({priceSource: "spot"|"twap"}), readMarket() in the stellar adapter, trader PRICE_SOURCE/TWAP_WINDOW_SECS env knobs, twapFromObservations() BigInt mirror + tests. - Bindings regenerated; architecture.md, STATUS.md and frontend-integration.md document the semantics and caveats.
…n cap (OracleDesk#15) redeem_many / claim_pool_remainders settle up to MAX_BATCH (46) markets in one call, paying the total in a single token transfer. Validation rejects empty, over-cap and non-ascending id lists; markets that cannot pay out right now come back as per-item skips instead of aborting the batch, and the per-market Redeemed / PoolRemainderClaimed events let the frontend reconcile receipts without an indexer. The cap is backed by a committed measurement table (see the MAX_BATCH docstring; regenerate with `cargo test -p market-core measure_redeem_many_resources -- --ignored --nocapture`): the direct call measures 145 reads / 95 writes / 240 footprint at N=46, and the treasury wrappers add exactly two read entries (147 / 95 / 242). 46 keeps the worst-case disk-read interpretation at 147/200 = 73.5% of the mainnet limit, under the 75% headroom line; N=47 would sit at exactly 75%. Deviation from the issue sketch: `BatchItem.skipped: Option<SkipReason>` does not compile as a `#[contracttype]` field — under the `testutils` feature the derive needs `From<SkipReason> for ScVal`, which is unwritable from this crate (E0119 against the derive's own TryFrom via core's blanket impl; the orphan rule blocks impls on Option, E0117). The equivalent `BatchStatus { Settled, Skipped(SkipReason) }` union is used instead and documented on the type. Error codes continue at 23-27, reserving 19-22 for the parallel fees/breakers PR OracleDesk#153. Treasury: agent-gated redeem_many / claim_pool_remainders forwarders that act for the treasury's own positions, with cap-sized and wrong-caller tests. Bindings regenerated; market-core wasm re-vendored into resolver/treasury; frontend docs gain the chunking, simulate-before-sign and archived-entry-restore rules.
Regenerating the bindings npm-installs the package, which churned the lockfile's root name (@oracledesk/bindings-market-core vs the stale market-core) and bumped three transitive versions off registry drift. None of that is required by the interface change, so the lockfile goes back to develop's exact bytes; only src/index.ts carries the regen.
…k#16) OracleDesk#16's TTL work touched doc comments on contractimpl methods, and those comments are part of the contract spec — so freshly generated bindings for these two packages carry new doc text (and the spec-entry order the pinned stellar CLI 27.1.0 emits). market-core and treasury were already regenerated with the same CLI earlier in this branch. Found by running scripts/gen-bindings.sh (the command the nested CI job runs) against the branch's built wasm.
|
@Teescom Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
# Conflicts: # oracledesk-stellar/agents/bin/trader.ts # oracledesk-stellar/agents/core/strategy.test.ts # oracledesk-stellar/agents/core/strategy.ts # oracledesk-stellar/contracts/market-core/src/lib.rs # oracledesk-stellar/contracts/market-core/src/math.rs # oracledesk-stellar/contracts/resolver/market_core.wasm # oracledesk-stellar/contracts/resolver/src/lib.rs # oracledesk-stellar/contracts/treasury/market_core.wasm # oracledesk-stellar/contracts/treasury/src/lib.rs # oracledesk-stellar/docs/STATUS.md # oracledesk-stellar/docs/architecture.md # oracledesk-stellar/docs/frontend-integration.md # oracledesk-stellar/model/fpmm_model.py # oracledesk-stellar/packages/bindings/market-core/src/index.ts # oracledesk-stellar/packages/bindings/resolver/src/index.ts # oracledesk-stellar/packages/bindings/treasury/src/index.ts
Reconciles the four-issue batch (OracleDesk#16 close-time-aware TTL, OracleDesk#14 outstanding shares + finalize_market, OracleDesk#11 cumulative-price TWAP, OracleDesk#15 batch redemption) with develop's resolver AttestedOutcome migration and treasury rewrite. Resolution notes: - resolver: keep develop's AttestedOutcome/V2 data model and all of its tests; re-apply the OracleDesk#16 close-time-aware TTL bumps, and extend bump_market_entries to sweep the CountV2/ProposalV2/Snapshot keys that replaced the legacy Count/Proposal in the migration. - treasury: adopt develop's bucketed-usage / agent-scoped-auth model and re-graft OracleDesk#15's redeem_many + claim_pool_remainders as thin delegates so the batch surface stays on the treasury contract. - market-core / reasoning-registry: take the union of both sides.
Six of the twenty-one rustfmt hunks are this branch's own code (two in market-core, four in resolver); the remaining fifteen are drift already on develop in resolver/treasury, which rustfmt normalizes because the gate is per-file. reasoning-registry is excluded: it does not parse on develop.
Merge conflicts resolved — remaining CI red is pre-existing on
|
What this does
Implements four scheduled issues, one commit per issue, based on
develop@ 67faa89:79efc79f016b98finalize_marketresidual sweep0aec7d3PRICE_SOURCE=twap99da493,460c22aMAX_BATCH56bbf26,ff7724bCloses #16
Closes #14
Closes #11
Closes #15
Checklist
cargo fmt --checkandcargo clippy --workspace --all-targets -- -D warningspasscargo test --workspacepassesmake bindings) and included in this PRmarket-core's interface changed:resolver/treasury's vendoredmarket_core.wasmrefresheddocs/STATUS.mdupdated if this changes what's verified/unverifiedGates (run locally on the final tree, unpiped)
CI-equivalent drift checks, run exactly as the
workflowssteps do:Note: the nested
oracledesk-stellar/.github/workflows/ci.ymldoes not run on GitHub (only repo-root workflows do), so no check runs are expected on this PR — the block above is that workflow's full job set, replicated locally with stellar CLI 27.1.0 (the CI pin).#16 — close-time-aware storage TTL policy
Four archival defects shared one root cause (entries living on a fixed 60-day bump or a deploy-time default instead of per-market needs):
Market/Positionto a flat 60 days regardless ofclose_time— a market closing further out archived before it could be resolved, and the resolver'sget_marketcross-call then failed until someone paid for a restore.Admin/MarketCore/Config/Daily/NextIdlived only as long as the deploy default; an archived instance makes the whole contract uncallable.Price/Signers;Attestation/Count/Proposal/Statewere never extended — and a missingCountreads as 0 viaunwrap_or(0), silently resetting an in-progress signer tally.Agentonly atregister_agent, so an agent publishing for 60+ days was archived andpublish_tracefailed withAgentNotRegistered.Each contract now carries an identical
src/ttl.rs(duplicated because contract crates cannot path-depend on each other — ADR 0002): per-market entries targetclose_time + 90-day settlement horizon, clamped tomax_ttl(); threshold = target/2 so a frequently touched entry is rarely rewritten; instance target 30 days / threshold 7. All six enumerable resolver keys are extendedhas()-guarded inbump_market_entries; treasury exposure entries are keyed toclose_time; the registry refreshesAgenton every successfulpublish_trace;state()stays a pure view.Rent cost comparison (old vs new policy) — measured
Same harness, same call, three builds (one 1-USDC
buyat ts=1000 on a market created identically, closing 10,000,000 s ≈ 115.7 days out; the committedmeasure_buy_resourcesharness, run with--ignored --nocapture):persistent_rent_ledger_bytesThe bumped entry is 212 bytes: 212 × 1,036,800 = 219,801,600 vs 212 × 3,555,000 = 753,660,000 — ×3.43. The test environment's
max_entry_ttl(6,312,000 ledgers) does not bind here; on mainnet the same market clamps tomax_ttl()= 3,110,400 ledgers (180 days), i.e. ×3.0.That multiplier is the price of actually coming due: the old 60-day cover cannot reach a market that closes beyond 60 days — archival then blocks resolution until a restore operation (which itself rents the restored TTL). Steady state is unchanged where the old policy did work:
extend_ttlis a no-op while remaining TTL ≥ target/2 (both rows above are exactly 1 bump — a freshly touched entry), so a market touched at least once per half-target pays only on real re-extensions. Instance storage: old extended to 60 days on every instance read with a 30-day threshold; the new 30/7 pair rents half the ledgers per extension and bounds a quiet contract's pre-paid cover at 30 days.Full constants and the CAP-0066 rent formula are in the
ttl.rsmodule header (identical in all four crates); the policy is documented indocs/architecture.md§ Storage TTL policy, linkingdocs/wave-issues/14-storage-ttl-keeper.md.Regression tests advance ledger sequence and timestamp together and assert TTLs through testutils, including
signer_vote_entries_survive_dispute_period(a signer tally survives 61 days of silence — the silent-reset bug),market_entry_survives_until_close_plus_horizon,position_ttl_tracks_close_time,instance_survives_after_long_idle,bump_market_entries_skips_missing_keys,instance_ttl_extended_on_agent_buy,agent_entry_extended_on_publish.#14 — outstanding shares +
finalize_marketSettleStateside entry per market tracksoutstanding_yes/no,finalized_at,horizon_secs,paid_out,closed; theMarketXDR is untouched (side-entry design, same as Add a cumulative-price TWAP accumulator to market-core for manipulation-resistant reads #11 below).reserve_<side> + outstanding_<side> == sets_mintedinvariant is asserted in tests after each operation type.finalize_marketis permissionless and deadline-gated: before the deadline it reverts (RedemptionWindowOpen); after it, it sweeps the market's entire remaining balance — expired claims, void rounding dust, unclaimed LP residue — in one transfer, proven by a balance-delta test (finalize_after_deadline_sweeps_exact_residual).redeem_before_deadline_never_underpaid).redeem/claim_pool_remainderrevert withRedemptionClosed; a secondfinalize_marketalso reverts.SettleStatebookkeeping) are explicitly unsupported for finalization (NotSupported, tested vialegacy_market_cannot_finalize) and stay redeemable through the old paths (legacy_market_still_redeems_without_deadline).wrong_caller_cannot_set_default_redeem_horizontest;sellnow useschecked_subonsets_minted.docs/architecture.mdanddocs/STATUS.mddescribe the deadline and the dust sweep.#11 — TWAP accumulator, and the
buydeltasDesign: a Uniswap-v2-style cumulative-price accumulator updated in
buy/sellbefore reserves change, with same-timestamp semantics handled exactly as the v2 oracle does; a bounded 16-slot observation ring stored as side entries (MarketXDR unchanged);observe,twapandprice_cumulativeviews with interpolation andObservationTooOld; overflow analysis documented in code and exercised by a simulated 10-year-lifetime test; a single-ledger manipulation test provingtwapis unaffected by one trade.docs/frontend-integration.mddocuments the reads and their caveats (observation coverage, counterfactual extension after the last trade,NotSupportedfor pre-accumulator markets); agents readPRICE_SOURCE=twapthroughreadMarketwith unit tests (.env.exampledocuments the knob).CPU and write-byte deltas for
buy(required by the issue)Same test body (
measure_buy_resources, committed) on the parent buildf016b98and on this branch; the call is a 1-USDCbuyat a new timestamp — the full accumulator write path (worst case; same-timestamp trades only update in place):The rent delta is the new observation ring entry being bumped alongside the market (bumps 1 → 2) at the shared close-time-aware target; the accumulator adds no event payload. In absolute terms the whole
buycosts 569,672 instructions — 0.14% of the 400M mainnet budget — and 2,156 write bytes (1.6% of the budget) per new-timestamp trade.#15 — batched redemption,
MAX_BATCHmeasuredredeem_many(market_ids)andclaim_pool_remainders(market_ids)take ascending, deduplicated ids (elseInvalidAmount), reject empty/over-cap batches (BatchTooLarge), skip per-market problems with explicit reasons (SkipReason::{NotFound, NotFinal, NoPosition, NotCreator, AlreadyClaimed}) instead of aborting the call, and pay via one aggregated transfer per call — balance-delta test provesafter − before == result.totalandresult.total == Σ item payouts.Redeemed/PoolRemainderClaimedevents are emitted identically from the single-market entrypoints; mixed settlement kinds (Yes/No/void) are tested in one batch; open markets are skipped;Settle/Positionbookkeeping is the same code path as single redeem.redeem_many/claim_pool_remaindersexist and are agent-gated (wrong_callertests for both, plus a cap-sized success test through the treasury atmax_batch()).docs/frontend-integration.mddocuments chunking via the newmax_batch()view, simulating before signing, and restoring archived entries in a separate transaction.MAX_BATCH= 46, backed by the committed measurement tableMeasured with the committed
measure_redeem_many_resourcesharness; mainnet limits are the SDK's 2026-07-10 snapshot (InvocationResourceLimits::mainnet(), enforced by the test env). Direct market-core path, N redeemed markets:Treasury wrapper path at the same N=46: 147 reads, 95 writes, 242-footprint, 33.1M insns, 15,240 write B, 7,964 event B (the wrappers add a constant two read entries and no extra writes).
At 46 the binding axis (footprint, 5N+10) is 240/400 = 60% of the ledger-entry limit; write entries 95/200 = 47.5%; events 7,964/16,384 = 49%; instructions 5–8%, memory 9–13%. Worst case, if every touched entry had to be restored from archived state, the 3N+7 reads would be 147/200 = 73.5% on the treasury path (145/200 = 72.5% direct) — under the 75% headroom line with room for tx overhead the harness cannot see (auth entries, source account). N=47 measures exactly 150/200 = 75.0% with zero margin, so 46 is the largest defensible cap (the issue's "50" was an illustration; this is the measured answer). The table and this rationale live in the
MAX_BATCHdoc comment;max_batch()exposes the cap so frontends chunk trustlessly instead of hardcoding it.Deviations from the issue text (deliberate, documented in code)
BatchResult.itemsusesBatchStatus::Settled | Skipped(SkipReason)instead of a struct withskip_reason: Option<SkipReason>—Option<T>inside a#[contracttype]struct does not compile in this workspace's test setup (the SDK 27 derive hits E0117/E0119 via testutils). The union carries identical information.ObservationTooOld23,RedemptionWindowOpen24,RedemptionClosed25,NotSupported26,BatchTooLarge27): the open PR feat: fees, breakers, trade bounds, transfers + migration (#8 #10 #12 #13) #153's branch introduces its own codes 19–22. The sets are disjoint — whichever lands second keeps its own numbering, and only the generated bindings embed the values.MAX_BATCHis 46, not the issue's illustrative 50 — reason above.Notes for the reviewer
market_core.wasm(26,137 → 47,533 bytes incontracts/resolver/andcontracts/treasury/) was rebuilt with stellar CLI 27.1.0 (the CI pin; homebrew's 28.x reorders spec entries) and the drift check above exits 0.packages/bindings/*/package-lock.jsonfiles are byte-identical todevelop's (see56bbf26): regenerating bindings npm-installs the package, which churns them with registry drift that has nothing to do with the interface change.ff7724bregeneratesresolver/reasoning-registrybindings — Make storage TTL extension close-time-aware and add missing instance TTL bumps #16 touched doc comments oncontractimplmethods, and those comments are part of the contract spec, so the committed bindings for those two packages were stale without it (found by running the CI job's owngen-bindings.shstep against the branch build).max_batch,observe,twap,price_cumulative,redeem_deadline; typesSettleState,BatchResult/BatchItem/BatchStatus/SkipReason; entrypointsfinalize_market,redeem_many,claim_pool_remainders(also mirrored on treasury),set_default_redeem_horizon.docs/architecture.md(TTL policy + settlement design),docs/STATUS.md(deadline/dust sweep, TWAP, batch) anddocs/frontend-integration.md(batch chunking + TWAP caveats) are updated in the same commits.