Repository navigation
fix(backend): refactor GET /admin/users correlated subqueries to LEFT… - #441
Open
Onesimus-dev wants to merge 1 commit into
Open
Onesimus-dev wants to merge 1 commit into
Onesimus-dev wants to merge 1 commit into
Conversation
… JOIN aggregates (issue arflexx#256)
|
@Onesimus-dev Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fix(backend): refactor GET /admin/users correlated subqueries to LEFT JOIN aggregates (issue #256)
Problem
GET /api/v1/admin/users executed two correlated subqueries for every
user row returned:
transactionsto compute each user's net balance.trade_offersto compute each user's trade count.With a page of 50 users, this fired 100 sub-selects. With 1 000 users
loaded (e.g. a large page or a downstream report), the planner issued
2 000 correlated lookups — one per user per metric — causing query
timeouts at scale. The complexity was O(n × m) where n is the number
of users in the result and m is the average number of rows per user in
transactions/trade_offers.Root Cause
Correlated subqueries in the SELECT list are evaluated row-by-row by
the PostgreSQL executor. They cannot be hoisted into a single scan or
a hash join, so the cost grows linearly with the result set size
regardless of indexes.
Fix
server/src/routes/admin.ts
Replaced both correlated subqueries with pre-aggregated grouped
subqueries joined via LEFT JOIN:
tx_agg: a single pass overtransactions— GROUP BY user_id,SUM(CASE WHEN direction = 'credit' THEN amount ELSE -amount END).
Joined to
userson user_id.tr_agg: a single pass overtrade_offersvia a UNION ALL ofseller_id and buyer_id columns — GROUP BY participant_id, COUNT(*).
Joined to
userson participant_id.The planner now executes two O(1) hash-aggregate scans (one per joined
subquery) rather than O(n) correlated lookups. Total query cost is
O(T + O) where T = rows in transactions and O = rows in trade_offers,
independent of the page size or the number of users returned.
server/src/db/migrations/010_admin_users_query_index.sql
Added three idempotent indexes (IF NOT EXISTS) to support the new
aggregation scans:
idx_transactions_user_id ON transactions(user_id)
Allows the tx_agg GROUP BY user_id to use an index scan instead of a
sequential scan.
idx_trade_offers_seller_id ON trade_offers(seller_id) WHERE seller_id IS NOT NULL
idx_trade_offers_buyer_id ON trade_offers(buyer_id) WHERE buyer_id IS NOT NULL
Partial indexes on the UNION ALL inputs reduce index size to only the
rows that contribute to the participation count.
Migration 009 already added idx_transactions_user (user_id, created_at)
for per-user history queries, but the planner may choose a seq scan for
a bare GROUP BY user_id aggregate. The new single-column index gives the
planner a lighter, more targeted option for the aggregation path.
Acceptance Criteria Met
aggregation joins — expected to complete well under 500 ms at
10 000 users / 100 000 transactions.
Closes #256
closes #244
closes #239
closes #242