feat: core engine, venue adapters, reconciliation, and partitioning sweep

[sparkysolutions10-pixel]

Summary of Changes

This PR completes a comprehensive engine hardening, multi-venue adapter expansion, data integrity, and aggregation sweep resolving 22 tracked issues:

1. Pricing, Routing & Aggregator

• Dynamic Fee Reading (Fees are assumed, not read #298): Reads dynamic ps.fee_bp with fallback to pool fee tier / default in src/pricing/depth.ts.
• Multi-Venue Route Splitting (Route recommendations are not split across venues #296): Implements RouteSplit[] support with proportional amounts for cross-venue routing execution in src/aggregator/bestRoute.ts.
• Price Provenance & Input Attribution (Nothing records which inputs produced a published price #245): Captures complete attribution metadata (spotSources, medianPrice, selectedSources) across pricing aggregation in src/pricing/aggregate.ts.
• Cross-Venue Reconciliation (Nothing reconciles two venues quoting the same pair #239): Adds cross-venue price reconciliation and divergence monitoring at GET /price/:assetA/:assetB/reconciliation and src/aggregator/reconciliation.ts.
• VWAP Unit Tests (Unit-test src/aggregator/vwap.ts, the module every /price answer comes from #173): Adds 13 comprehensive unit tests validating time windows, volume weighting, and outlier filtering in src/__tests__/vwap.test.ts.

2. Database & Data Integrity

• Strict Network Predicate (Nothing fails when a query forgets the network predicate #247): Enforces network scoping across all query paths for price_points, pool_snapshots, and price_aggregates.
• Batch Deduplication & Idempotency (Nothing stops the same trade being written twice #237): Guarantees unique ingestion on (network, source, eventId) in batch upserts.
• Timestamp Disambiguation (Timestamps mix chain time and wall clock #240): Explicitly separates on-chain ledger timestamp (chainTimestamp) from Lens observation time (observedAt).
• Table Range Partitioning (price_points has no partitioning, though its key is built for it #242): Implements monthly range partitioning and safe partition pruning in src/db/partitioning.ts.
• Historical Gap Detection & Backfill (There is no way to backfill #241): Implements gap detection across ledger sequences and historical backfilling in src/ingest/backfill.ts.
• High-Precision Calculations (Prices are stored to 18 decimal places and computed as floats #230): Replaces float casting with Postgres ::numeric precision preserving 18 decimals in src/routes/candles.ts.

3. Venue Adapters & Ingestion Architecture

• Consolidated Ingesters (Classic AMM and Soroswap ingesters live outside src/ingest/ #225): Consolidates ingester logic into src/ingest/ with backwards-compatible proxies in src/ingesters/.
• Venue Heartbeat & Offline Monitoring (A venue that goes quiet is indistinguishable from a venue with no trades #219): Adds venue liveness tracking and quiet detection exposed at GET /venues/status.
• Phoenix DEX AMM Adapter (No adapter for Phoenix, one of Soroban's largest DEXs #210): Implements Phoenix DEX adapter registered in venueAdapters.
• Blend Lending Market Adapter (Blend lending markets have no price or rate ingestion #212): Implements Blend market rates adapter and REST endpoint at GET /blend/rates/:asset.
• Dynamic Decimal Scaling (A venue's decimals are assumed, never read #227): Scales token decimals dynamically in Soroswap calcSpotPrice.
• Deduplicated Aggregator Volume (Volume is summed across venues that may be quoting the same trade #244): Filters duplicate volume across multi-venue routing events in src/routes/volumes.ts.

4. API, Security & Bazaar Discovery

• Daily Request Quota (ratePerDay is stored, returned, advertised — and never enforced #175): Enforces ratePerDay limits per API key with HTTP 429 quota exhaustion in src/api/auth.ts.
• Normalized Redis Keys (The aggregate-refresh worker writes a cache key nobody reads #166): Unifies Redis cache key format across ingester workers and REST endpoints.
• Facilitator Spend Ceiling & Allowlist (Harden the x402 facilitator: spend ceiling, caller allow-list, settle idempotency #147): Adds ceiling checks, caller allowlist, and auto-cataloging hook in src/routes/facilitator.ts.
• Bazaar Auto-Cataloging & Natural Language Search (Bazaar: automatic cataloging from the discovery extension #130, Bazaar: GET /discovery/search with real ranking and a quality evaluation #129): Implements Bazaar auto-cataloging and ranked discovery search with relevance and metadata quality scoring at GET /discovery/search.

---

Closes #298
Closes #296
Closes #247
Closes #245
Closes #244
Closes #242
Closes #241
Closes #240
Closes #239
Closes #237
Closes #230
Closes #227
Closes #225
Closes #219
Closes #212
Closes #210
Closes #175
Closes #173
Closes #166
Closes #147
Closes #130
Closes #129

[Miracle656]

Changes requested: one issue per PR, and only issues assigned to you

This is a process requirement, applied to all 23 of your open PRs in this repo. It is not a judgement about the code, which I have not been able to review — and that is the first half of the problem.

The bundling

Across your open PRs, 150 distinct issues are claimed. This one claims several. #483 claims 22, #481 claims 20, #477 claims 20, #474 claims 20.

A PR that closes twenty issues cannot be reviewed. Each of those issues has its own acceptance criteria — typically five — so twenty issues is a hundred separate things to verify in one diff, and if any one of them is wrong the whole PR is blocked while it is fixed. Every batch I published says the issues are deliberately independent so they can be worked in parallel, by different people, without colliding. Bundling them removes exactly that property.

The assignments

135 of the 150 issues claimed are assigned to other contributors — 67 different people.

This PR is an example: it names issues currently assigned to other contributors who are working on them now. Several have open PRs of their own. If this merged as it stands, those people would have done the work and received nothing for it, and I would have no way to tell the two efforts apart.

I am not assuming anything about why. Assignments change, Drips and GitHub can disagree, and a wide-ranging branch picks up issue numbers easily. But the effect on the people holding those assignments is the same regardless of intent, which is why I have to act on it.

What I need

• One issue per PR. The issue number in the title, its acceptance criteria met, and nothing else in the diff.
• Only issues assigned to you, or unassigned ones you have claimed through Drips. If an issue is assigned to someone else, it is theirs.
• Re-open the work that meets both conditions as individual PRs. I will review them in the order they arrive and at the same pace as everyone else's.

Nothing is being closed. These stay open and are yours to re-cut, withdraw or split as you choose.

Two things worth knowing before you re-cut

Some of this has already landed. #488 (SSRF guard for bazaar_search) and #487 (the paid_call host allow-list) cover the same ground as your #449; #486 covers the MCP README. Check current main before rebuilding something that is already there.

main has moved a long way today — #306, #309, #252, #256, #355, #381, #390, #402, #476, #488 and more. Cut from current main, not from where these branches started.

If you believe an issue named here is genuinely yours, say which and I will check the assignment myself — I would rather be corrected than have someone lose work.

[Miracle656]

Following up, because I want to be straight with you rather than leave 21 PRs sitting in silence.

Between 22:26 and 00:42 on 18–19 October, all 21 of these received exactly one new commit each: Merge branch 'upstream/main'. I checked them individually rather than assuming. Nothing else changed.

That's worth doing — several were conflicting and some still are — but it doesn't touch either thing I asked for. The review wasn't about staleness. It was that 135 of the 150 issues these PRs claim are assigned to 67 other contributors, and that a PR closing twenty issues can't be reviewed at all. A merge commit changes neither.

So the changes-requested stands on all 21, and I'm not going to re-review them in this state — not as a penalty, but because the question in front of me isn't about your code. I genuinely don't know how to merge a PR that would close twenty issues when eighteen of them belong to people who are working on them right now, several with their own open PRs. If I merge it, they did the work and get nothing, and I can't tell the two efforts apart afterwards. That's the part I can't get past.

The offer from the review is unchanged and still open:

• One issue per PR, that issue's number in the title, its acceptance criteria met, nothing else in the diff.
• Only issues assigned to you, or unassigned ones you've claimed through Drips.

Re-cut on that basis and I'll review them at the same pace as everyone else's — I merged four PRs today and I'm working through the queue continuously.

And the correction offer is also still open, and still sincere: if any issue named in these is genuinely yours, tell me which and I'll check the assignment myself. Assignments drift, Drips and GitHub disagree sometimes, and a wide branch picks up issue numbers easily. I'd much rather be corrected than have anyone lose work — you included. In eleven days nobody has named one, so I've had nothing to check.

Nothing here is being closed. These stay open and yours to split, re-cut or withdraw.

One practical note if you do re-cut: main has moved a long way again since the review, and some of this has landed in the meantime. Check current main before rebuilding anything.

[Miracle656]

Correction to what I just wrote, and an apology for it.

I checked the 21 bundled PRs and not the rest of the repo, so I missed that you did re-cut: #526 through #540, opened this morning, each naming one issue, each between 70 and 400 lines. That is exactly what the review asked for and it deserved to be acknowledged rather than met with a complaint about merge commits. My comment above was written without that in front of me. Sorry.

So let me be useful instead. I went through all thirteen and checked the assignment on every issue they name. Five are clear and I'll review them; eight collide, four of those with a PR that already exists.

Clear — unassigned issues, yours to earn. These go in my review queue now:

PR	Issue
#526	#175	ratePerDay never enforced
#527	#129	/discovery/search with ranking
#528	#130	automatic cataloging on settlement
#531	#84	liquidity-depth & slippage
#532	#147	harden the x402 facilitator

Already assigned to someone else:

PR	Issue	Assigned to	Their PR
#529	#290	dmoonventure070-art	—
#530	#272	mansur-codes	—
#533	#273	Azintar	—
#534	#334	CollinsKRO	#544
#535	#335	CollinsKRO	#543
#536	#347	Oluoma123	—
#537	#352	titankrypt-gif	#553
#538	#411	esanbasirat	#555

On those eight the assignee keeps the claim, and I'm going to review their PRs first — #543, #544, #553 and #555 before #534, #535, #537 and #538. Not because yours are worse; I haven't read either yet. Because they hold the assignment, and if I review in any other order I'm deciding by who got there first, which is not how the assignments are supposed to work.

If one of those four lands, the matching PR of yours closes as superseded. If an assignee's PR needs changes and goes quiet, the issue frees up and yours is the obvious thing to look at — tell me and I'll pick it up.

#540 (read the version from package.json, rename to lens-mcp-server) names no issue at all, which makes it a plain chore. Thirty lines. I'll just review it.

And the correction offer stands with a shorter path now: for #529, #530, #533, #536, if you believe the assignment is stale — the assignee has gone quiet, or Drips shows it differently — say which and I'll check it. That's four to look at instead of a hundred and thirty-five, which is a conversation I can actually have.