Skip to content

feat(market-core): counter-indexed open/category/holder market directories (#25) - #240

Open
achiever2110 wants to merge 3 commits into
OracleDesk:developfrom
achiever2110:feat/market-directories-25
Open

achiever2110 wants to merge 3 commits into
OracleDesk:developfrom
achiever2110:feat/market-directories-25

Conversation

@achiever2110

Copy link
Copy Markdown

This PR resolves the following assigned issues:

What

Counter-indexed on-chain market directories in market-core, following the
reasoning-registry MarketTraceCount/MarketTrace pattern: no directory is
ever an unbounded Vec in a single entry, every entry is its own persistent
ledger key, and removals are O(1) swap-removes backed by reverse-index keys.

Changes made (all in oracledesk-stellar/contracts/market-core/src/lib.rs + two docs)

Keys (appended after Band(u64) at the DataKey tail, never reordered):
OpenCount, OpenAt(u32), OpenIndex(u64), CategoryCount(Category),
CategoryAt(Category, u32), HolderCount(Address), HolderAt(Address, u32),
HolderIndex(Address, u64). OpenCount is persistent, not instance, so the
hot lifecycle counter does not fatten every call's instance write.

Error: DirectoryEntryArchived = 27 (appended at the current develop
tail, after PendingAdminExpired = 26; uniqueness of 1..=27 verified).

Helpers (private): generic dir_remove<F, G> (swap-remove; no-op when the
reverse key is absent — a never-indexed legacy market — and traps with
DirectoryEntryArchived if the last *At slot was archived while its reverse
key survives), dir_write / dir_write_u32 (set + extend_ttl with the house
BUMP_THRESHOLD/BUMP_TO), open_add / open_remove, category_add,
holder_add / holder_remove, and the shared paginated reader dir_page.

Wiring: create_market appends to the open + category directories and
passes was_empty = true for the creator's seed position (so treasury/creator
seeded positions list under the creator — the issue's explicit edge case).
resolve / void_market call open_remove. save_position gained a private
was_empty: bool parameter (it is a private fn, not an entrypoint, so the
frozen public shape is untouched) and updates the holder directory only on
empty/non-empty transitions
; all six call sites are threaded — buy / sell
compute was_empty from the position they already loaded (zero extra reads),
redeem passes false (it always empties → always removes), and the position
transfer derives to_was_empty from the pre-mutation position with from
guaranteed non-empty by the balance checks. Directory writes live in the
effects sections, after require_auth and all validation.

Views (new #[contractimpl] entrypoints): open_count(),
open_markets(start, limit), category_count(category),
markets_by_category(category, start, limit), holder_count(holder),
markets_of(holder, start, limit) — each capped at MAX_PAGE = 50, returning
a short vector at the end and an empty vector past it, TTL-extending only the
slots they touch. Pagination is snapshot-free: swap-removes reorder
entries, so pages across concurrent transitions can skip or repeat ids;
clients must de-duplicate (documented in both docs files).

Backfill: admin-only rebuild_open_index(start_id, end_id), bounded to
MAX_BACKFILL_BATCH = 200 ids per call, idempotent (skips ids that already
have an OpenIndex), appends only still-Open markets. Holder directories
cannot be backfilled on-chain — documented in docs/architecture.md and
docs/frontend-integration.md as required.

Docs: docs/architecture.md gains an "On-chain market directories"
section (key table, swap-remove semantics, rent-restore requirement, cost
table, backfill limits); docs/frontend-integration.md gains a paging example
and the de-duplication caveat.

Tests (authored, NOT run on this host — see verification)

12 new tests inside the existing mod test: create/resolve/void tracking;
swap-remove at first/middle/last with exact slot-shape assertions; append-only
category behaviour; holder add on first buy / remove on full sell; zero
directory writes on an additional buy (write_entries delta pinned to 3 vs a
first buy); redeem removal (winning and losing holders); creator-seed listing;
pagination caps + out-of-range pages; a deterministic-LCG randomized
consistency test over 90 mixed lifecycle steps cross-checking all three
directories against raw Market/Position scans (dense slots, exact counts,
matching back-pointers); rebuild_open_index_admin_only (targeted mock_auths,
idempotency, batch bounds); directory_remove_traps_on_archived_slot
(#[should_panic] on Error(Contract, #27)); and directory_write_costs,
which records the per-path env.cost_estimate() table.

Acceptance criteria

  • Open/category/holder directories with counter-indexed persistent keys,
    no unbounded Vec per entry.
  • O(1) swap-remove, tested at first/middle/last positions.
  • save_position touches the holder directory only on empty/non-empty
    transitions, verified by holder_directory_unchanged_on_additional_buy.
  • open_markets / markets_by_category / markets_of capped at
    MAX_PAGE.
  • Randomized consistency test over mixed lifecycle sequences
    (directory_consistency_random_sequences).
  • [~] Measured extra write cost documented in docs/architecture.md — the
    entry-count table is documented and derivable from the helpers; the
    byte-exact env.cost_estimate() numbers are authored-test derivations
    only, because no cargo test could run on this machine (below).
  • Admin open-market backfill exists; holder-backfill limitation documented.
  • Refused: vendored wasm regeneration and make bindings. This host
    cannot compile the workspace, so the wasm blobs cannot be rebuilt or
    hashed, and bindings codegen requires the built wasm. No *.wasm,
    *.wasm.sha256 or packages/bindings/** file is touched by this PR
    (git diff --name-only .wasm count = 0). A maintainer with a working
    toolchain needs to run make build && make bindings before or after
    merge.

Verification actually performed (and its honest limits)

This machine cannot compile the workspace at the default branch: the pinned
rust-toolchain.toml (1.90.0, required for bit-identical wasm per #58) is
rejected by soroban-sdk@27.0.6 (MSRV 1.91), so cargo check/test fails at
dependency resolution, and cargo +stable dies on pre-existing
oracledesk-types errors. I did not attempt to repair either, per scope.
No test was executed and I claim none was.

What did run:

  • cargo +stable fmt -p market-core -- --check → exit 0 at the base commit
    and exit 0 after this change
    (rustfmt fully parses the file, which
    validates syntax and brace balance of the diff; it does not type-check).
  • Discriminant audit: Error is unique and dense 1..=27; the DataKey
    additions are append-only after Band(u64).
  • Every new DataKey variant is referenced by the helpers/views/tests; no
    todo!() / unimplemented!(); all six save_position call sites updated.

Collision disclosure (independent-PR policy)

Error::DirectoryEntryArchived = 27 and the eight DataKey variants append at
the current develop tail (5ede00a8). They will need re-appending (and the
#27 should_panic expectation bumping) if #237 or #239 — which also append to
these enums — merge first. This branch deliberately does not rebase onto our
other open branches; last-merged rebases.


Generated by an AI coding agent on the achiever2110 account; human-reviewed
to the extent described above (fmt + audit only — no compile, no test run).

…ories (OracleDesk#25)

Adds on-chain market directories following the reasoning-registry
MarketTraceCount/MarketTrace pattern: an open-markets set with O(1)
swap-remove, an append-only per-category list, and a per-holder directory
maintained inside save_position on empty/non-empty transitions only.
Paginated views (open_markets, markets_by_category, markets_of) cap pages
at MAX_PAGE=50; admin rebuild_open_index backfills legacy open markets in
bounded batches. Error::DirectoryEntryArchived = 27 appends at the current
develop tail.
…g example (OracleDesk#25)

Documents the counter-indexed open/category/holder directories (keys,
O(1) swap-remove, rent-restore requirement, snapshot-free pagination),
the per-transition write-count table as authored-test derivations, the
admin open-market backfill with its holder-directory limitation, and a
frontend paging example for open_markets / markets_by_category /
markets_of.
@achiever2110

Copy link
Copy Markdown
Author

Follow-up to the collision disclosure: our own PR #184 (abercrombie04, hot/cold storage split) also appends at the market-core Error tail with MetaUriTooLong = 27 and adds DataKey variants alongside. So discriminant 27 and the DataKey tail collide with #184 as well as #237/#239, per the independent-PR policy (last PR merged rebases). Whichever of #184 / #240 merges second needs a one-hunk re-append: bump DirectoryEntryArchived to the next free discriminant and update the should_panic expectation Error(Contract, #27) in the directory_remove_traps_on_archived_slot test. No rebase done now; both PRs stand against develop@5ede00a8 as authored.

anitahhhh added a commit to anitahhhh/OracleDesk-SmartContract that referenced this pull request Oct 22, 2026
…ils precisely (OracleDesk#24)

ADR 0010 listed the Python mirror as follow-up work it could not do. That was
wrong about the constraint: model/fpmm_model.py runs here, so its checks are
executed rather than merely authored. Only the TypeScript mirror stays out,
because Node on this host cannot strip types.

Also drops the claim that resolver's Error 44 collides with OracleDesk#239/OracleDesk#240/OracleDesk#241: it
is OracleDesk#241 (QuorumNotReached) and OracleDesk#242 (EventTimeMismatch) that reach 44, verified
from each PR's own patch.

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add on-chain market directories indexed by status, category and holder

1 participant