# ISP-087: Contract WASM-hash upgrade migration vs bindings regen vs indexer parser version gate — clients, indexers and contracts on three versions at once #757

• Type: Integration + Spike
• Severity: P0/Critical
• Difficulty: Extreme
• Estimate: 24h
• Touch surface: 15 files

1. Background & Context

HelPhone upgrades Soroban contracts by deploying new WASM and updating the hash each proxy points to. The pipeline spans three repos-in-one: Rust in contracts/aegis_vault, contracts/emergency_vault, contracts/helphone_dao, contracts/maintainer_vault (each with src/lib.rs, Cargo.toml, Cargo.lock); TypeScript bindings in src/lib/contract.ts, src/lib/treasury.ts, src/lib/footprint.ts, src/lib/txBatcher.ts regenerated by scripts/generate-soroban-bindings.js (stellar-cli bindgen + manual patches); and the off-chain indexer in server/indexer/daemon.ts, parser.ts, exporter.ts, rentRenewer.ts that decodes events and ledger entries by schema version. soroban-contract.md documents the intended atomic upgrade: deploy → verify hash → regen bindings → bump parser gate → deploy frontend + indexer together.

In practice the three move independently. Contract engineers deploy a new emergency_vault WASM to fix a rent bug (ISP-085) and update the on-chain hash; frontend deploys daily via render.yaml static hosting and picks up regenerated bindings immediately; the indexer deploys weekly via Dockerfile.hermetic and still runs the old parser.ts that expects the old event schema (e.g., DispatchCreated { id, geohash } vs new { id, geohash, seq, ttlClass }). src/lib/rpcHealth.ts routes different clients to different RPC hosts with different ledger lags, so some users read new state through old bindings (decode failure) while the indexer writes old projections from new events (silent field drops). scripts/export-contract-state.sh snapshots mix versions without recording which WASM hash produced which row.

The version gate that should prevent this — parser.ts checking contractVersion / WASM hash before decoding — is either absent or advisory (console.warn instead of halt). server/db/migrator.ts + partitioning.js have no column for wasmHash or schemaVersion, so exporter.ts overwrites new-schema rows with old-schema defaults. swagger.json + server/routes/docs.ts document REST shapes derived from indexer projections, so API consumers see a fourth version (stale projection served as current). scripts/detect-api-drift.js pins npm surfaces but knows nothing about WASM hashes, so CI stays green while production serves three truths.

This spike must design a hard version discipline (hash-pinned bindings, parser gates that halt, migration windows with dual-read) and prove it on testnet with a live upgrade that keeps all three in sync or fails loudly.

2. Current Broken Behaviour

After a testnet emergency_vault upgrade adding seq: u64 to DispatchCreated, new dispatches created via fresh contract.ts bindings succeed on-chain but the indexer logs unknown field seq, dropping and serves projections without seq, breaking ISP-084 ordering; old frontend clients with stale bindings fail with failed to parse event: missing field seq or submit create_dispatch with wrong arity and get InvalidArgs. scripts/export-contract-state.sh snapshot contains rows with and without seq and no version column. parser.ts emits no error (warn-only gate), render.yaml shows frontend and indexer on different deploy SHAs, and node scripts/generate-soroban-bindings.js --check reports drift that nobody blocks on.

3. Desired End State

Upgrades are atomic-logical: every WASM hash maps to exactly one bindings version and one parser version, recorded in a registry (contracts/versions.json or equivalent); parser.ts hard-gates (refuses to index unknown hashes, alerts, backfills after upgrade); contract.ts bindings embed their expected hash and refuse to submit to mismatched contracts with an actionable error; exporter.ts + DB store wasmHash + schemaVersion per row; frontend + indexer deploy in lockstep (or indexer leads with dual-read during window); and export-contract-state.sh snapshots are version-stamped. A testnet upgrade drill demonstrates zero silent drops and clean rollback.

4. Files / Systems Involved

• contracts/emergency_vault/src/lib.rs
• contracts/aegis_vault/src/lib.rs
• contracts/helphone_dao/src/lib.rs
• contracts/maintainer_vault/src/lib.rs
• scripts/generate-soroban-bindings.js
• src/lib/contract.ts
• src/lib/treasury.ts
• src/lib/footprint.ts
• src/lib/txBatcher.ts
• server/indexer/daemon.ts
• server/indexer/parser.ts
• server/indexer/exporter.ts
• scripts/export-contract-state.sh
• soroban-contract.md
• docs/soroban-footprints.md

5. Reproduction Steps

1. Run cat soroban-contract.md; ls contracts/*/src/lib.rs; cat scripts/generate-soroban-bindings.js | head -100 to map upgrade pipeline.
2. Run node scripts/generate-soroban-bindings.js --check to capture current drift status.
3. Record current hashes: stellar contract fetch --id <emergency_vault> --network testnet | sha256sum for each of the four contracts.
4. Make a schema-breaking change in a branch (add seq: u64 to DispatchCreated in contracts/emergency_vault/src/lib.rs), build: cargo build --target wasm32-unknown-unknown --release -p emergency_vault.
5. Deploy to testnet: stellar contract deploy --wasm target/wasm32-unknown-unknown/release/emergency_vault.wasm --source deployer --network testnet and update hash pointer.
6. Regen bindings only for frontend: node scripts/generate-soroban-bindings.js --contract emergency_vault and deploy frontend preview (npm run build && npm run preview).
7. Create dispatch via new frontend, observe indexer (npx tsx server/indexer/daemon.ts --once) warn-and-drop seq.
8. Create dispatch via old frontend (stashed bindings), observe InvalidArgs on-chain error.
9. Run bash scripts/export-contract-state.sh --network testnet --out /tmp/mixed.json && jq '.[].schemaVersion' /tmp/mixed.json | sort | uniq -c to prove mixed-version snapshot.
10. Confirm render.yaml frontend SHA ≠ indexer SHA and no gate blocked either deploy.

6. Spike Investigation Plan

Experiment A (5h): Schema-diff census. Diff old vs new events/storage for all four contracts (cargo soroban inspect, stellar contract bindings json). Enumerate breaking (field add/remove/rename, arity change) vs additive (new optional event) changes. Build version-compatibility matrix (which old readers tolerate which new writes). Decide SemVer-for-contracts policy (major = breaking decode, minor = additive).

Experiment B (5h): Hard-gate prototype. Add EXPECTED_WASM_HASH + SCHEMA_VERSION constants to parser.ts (fetched on-chain at startup via rpcHealth.ts host), halt-with-alert on mismatch (not warn), and contract.ts runtime hash check before invoke (fetch getLedgerEntries instance + compare). Measure false-halt rate under RPC lag (different hosts report different ledgers during upgrade window). Design dual-read window: indexer runs old+new parsers for N ledgers, exporter.ts tags rows with version.

Experiment C (4h): Registry + bindings pinning. Prototype contracts/versions.json mapping {contract, wasmHash, bindingsVersion, parserVersion, deployedAtLedger}. Teach scripts/generate-soroban-bindings.js to stamp bindings with hash + fail --check in CI on unregistered combos. Teach scripts/export-contract-state.sh to include hash/version per row and refuse mixed snapshots without --allow-mixed flag.

Experiment D (4h): Deploy choreography. Coordinate frontend (render.yaml static) + indexer (Dockerfile.hermetic) upgrade order: indexer-first with dual-read, then frontend, then deprecate old parser. Test rollback: revert hash pointer, confirm old bindings + old parser resume cleanly from version-stamped checkpoint (link ISP-088 checkpoint interplay). Time each phase.

Experiment E (6h): Soak + drift integration. Extend scripts/detect-api-drift.js (or new detect-wasm-drift.js) to compare on-chain hashes vs registry vs bindings vs parser constants in CI. Run upgrade drill end-to-end on testnet with 500 dispatches across the window; assert zero drops, zero InvalidArgs, version-tagged snapshots. ADR with upgrade runbook.

7. Acceptance Criteria

• Compatibility matrix committed distinguishing breaking vs additive contract changes per contract
• contracts/versions.json (or equivalent registry) exists with hash→bindings→parser mapping; CI fails on unregistered combo
• parser.ts hard-gates on unknown WASM hash (halts + pages, does not warn-and-drop); test proves halt on staged mismatch
• contract.ts bindings embed expected hash and refuse mismatched submit with actionable error (not opaque InvalidArgs)
• Dual-read window demonstrated: indexer handles old+new events for ≥500 ledgers with version-tagged rows, zero silent drops
• exporter.ts + DB rows carry wasmHash + schemaVersion; export-contract-state.sh snapshots version-stamped, mixed snapshots explicitly flagged
• Upgrade drill on testnet with 500 cross-window dispatches: 100% indexed with correct version tags, old clients get clean error (not corrupt submit)
• Rollback drill succeeds: revert hash, old stack resumes from checkpoint without reindex-from-genesis
• soroban-contract.md + docs/soroban-footprints.md updated with versioning policy + runbook (who deploys what in which order)
• scripts/generate-soroban-bindings.js --check green in CI and wired as required check (not advisory)
• render.yaml + Dockerfile.hermetic deploy order documented and enforced (indexer-first) with SHA linkage
• No triple-version serving in production: probe endpoint reports {contract, hash, bindings, parser}一致性

8. Why This Takes 24h / Why It Is Extremely Hard

• Three release cadences (contracts: slow + audited, frontend: daily, indexer: weekly) with two deploy targets (render.yaml vs Dockerfile.hermetic) guarantee skew windows; designing a discipline that survives independent deploys requires distributed-version thinking, not just a checklist.
• Soroban upgrade semantics (WASM hash pointers, instance vs code, update auth) are subtle and testnet-only rehearsal takes hours (builds, deploys, ledger waits, faucet limits).
• Backward-compat analysis needs Rust + XDR + TypeScript literacy simultaneously: a field-add that looks additive in Rust can be breaking in bindgen output or parser.ts manual decoding.
• Dual-read doubles indexer complexity (two schemas, version-tagged storage, backfill) and interacts with rent/archival (ISP-085) and ordering seq (ISP-084) — the spike must avoid solving versioning by breaking economics or ordering.
• Proving zero silent drops requires cross-window load (500+ dispatches) with version-aware assertions across chain + indexer + API + snapshot — a harness that doesn't exist and must be built during the spike.

9. Risks & Blast Radius

Hard-gating the parser means a missed registry update halts indexing entirely (availability loss) instead of degrading — a wrong hash string bricks the indexer until redeploy. Hash-pinned bindings mean a legitimate emergency contract fix (ISP-085/086) can't be used by the frontend until bindings regen + redeploy completes, delaying response. Dual-read doubles RPC + DB load and can exhaust poolManager.ts connections or RPC quotas via rpcHealth.ts. Version-tagged DB columns require migrator.ts migration under load (ISP-088 lock interplay). A bad rollback (revert hash but not parser) re-creates the triple-version split the spike was meant to kill, now with version tags proving the failure permanently.