Skip to content

feat(properties): listing metrics, functional search indexes, search cache and search rate limit - #1475

Open
m-hajjo wants to merge 2 commits into
MettaChain:mainfrom
m-hajjo:feature/listingapi-metrics-db-cache-ratelimits
Open

m-hajjo wants to merge 2 commits into
MettaChain:mainfrom
m-hajjo:feature/listingapi-metrics-db-cache-ratelimits

Conversation

@m-hajjo

@m-hajjo m-hajjo commented Oct 18, 2026

Copy link
Copy Markdown

Performance and observability for the public listing read path (GET /properties and GET /properties/search). This complements #1471, which covered the property write/detail surface, and follows the conventions of the Users API pass in #1457–#1463.

#1384 — Database query optimisation (the big one)

The case-insensitive search filters could never use an index. GET /properties/search filters with Prisma's mode: 'insensitive', which compiles to:

LOWER("city") = LOWER($1)

A plain btree index on the raw column is unusable for that expression — the indexed values are Austin while the lookup key is LOWER("city"). So @@index([city, state]) and @@index([propertyType]) were never chosen by the planner for the filters they were added for, which are among the most common in the product. Same for country, category and the amenity join.

prisma/migrations/20261001000001_add_listing_search_functional_indexes indexes the expression instead, leading with the listing predicate so a filtered search is a single index seek:

Index Serves
(status, is_deleted, LOWER(city)) ?city=
(status, is_deleted, LOWER(state)) ?state=
(status, is_deleted, LOWER(property_type)) ?propertyType=
(status, is_deleted, LOWER(category)) WHERE category IS NOT NULL ?category=
GIN (tags) where: { tags: { has: tag } }
LOWER(amenity_type) on property_amenities ?amenityType=

?tags= is array containment, which needs GIN — btree cannot answer it. ?category= is partial because the column is optional, so indexing the empty majority would cost space and scan time for nothing.

⚠️ For maintainers: Prisma cannot express functional or partial indexes in schema.prisma, so these six are intentionally not declared in the schema. A future prisma migrate dev may therefore propose dropping them as unmanaged drift — if it does, do not accept the drop, re-create them.

Deliberately not done: the free-text ?location= filter does contains across address/city/state/zipCode and really wants pg_trgm + GIN or Postgres FTS. That needs the extension enabled on the database, which is a platform decision rather than something a migration should make unilaterally. Flagged for follow-up in the doc.

→ docs/api/listings-db-optimization.md

#1385 — Redis caching

GET /properties/search is read through PropertiesListingsCacheService: key properties:search:<sha1 of the normalised filter set>, TTL 30 seconds, registered under CACHE_TAGS.PROPERTIES so a property write invalidates it with the rest.

  • 30s, deliberately short. A listing can be published, repriced or taken down at any moment, and the public feed is where a stale listing is immediately visible to a buyer. 30s bounds the damage while still absorbing the repeated identical queries a listing page produces while someone pages through filters.
  • GET /properties is not cached. Its result set changes on almost every write and it is the cheaper of the two queries — caching it buys mostly invalidation risk.
  • Key normalisation. Filter entries are sorted and absent values dropped before hashing, so query-string order and omitted-vs-undefined cannot produce two keys for the same filter set.
  • An empty filter set is not cached. That is the full listing feed, which changes constantly and is exactly where a stale answer would be most damaging.
  • Keys over 2 KB are skipped, so a pathological query cannot create an unbounded Redis key.
  • Cache errors fall through to the database, so caching can never fail a request.

PropertiesListingsCacheService is injected with @Optional(), so the service still constructs without it and the existing unit-test module is unaffected.

→ docs/api/listings-caching.md

#1383 — Prometheus metrics

Metric Type Labels
properties_listings_served_total counter surface (list/search), outcome
properties_listing_response_seconds histogram surface
properties_listing_results_returned histogram surface
properties_listing_cache_lookups_total counter outcome (hit/miss/bypass)

Every label set is fixed — 2, 3 and 3 values — so cardinality cannot grow with request input, which is what makes these safe to scrape at 1% resolution. surface is derived from the request path rather than the handler name, so adding a handler cannot silently widen the label space.

Two worth alerting on:

  • properties_listing_results_returned — a histogram of how many listings each response carried. A drop toward zero at flat request volume means the filters or index are wrong, visible long before someone reports "search is broken".
  • ..._cache_lookups_total{outcome="bypass"} — separates a query that was never cacheable from a cache that errored. A rising bypass share usually means Redis is unhealthy, visible before latency degrades.

→ docs/api/listings-metrics.md

#1386 — Rate limiting

GET /properties/search had no entry in ENDPOINT_RATE_LIMITS at all. Both listing routes are unauthenticated, so search was bounded only by the global per-IP bucket — shared with every other endpoint on the API. A single client could spend that whole budget on search alone, and search is the most expensive read in the module: up to 25 filters, an optional in-memory trust-score pass, and a COUNT issued alongside every page.

Route Limit
GET /properties 100 / minute pre-existing
GET /properties/search 60 / minute added

Sixty per minute is above what a person paging through filters needs, while cutting the worst case by roughly 40%. Unauthenticated requests are keyed by client IP (hashed, and x-forwarded-for only consulted when TRUST_PROXY is set, per #1195); authenticated callers are additionally bounded by their own user-tier bucket.

A per-route guard class was written first and then rejected: the global RateLimitGuard already provides this behaviour, and a new @UseGuards class would have added DI surface (a RateLimitService dependency) to every integration test that boots PropertiesController in isolation.

→ docs/api/listings-rate-limiting.md

Not run

Nothing was installed, built, linted or tested — please verify in CI, in particular that the functional indexes apply cleanly to your Postgres version.

Closes #1383
Closes #1384
Closes #1385
Closes #1386

…cache, search rate limit

Performance and observability for the public listing read path
(`GET /properties` and `GET /properties/search`), following the conventions of
the Users API pass in MettaChain#1457-MettaChain#1463 and complementing MettaChain#1367-MettaChain#1370, which covered
the property write/detail surface.

- MettaChain#1383: `properties_listings_served_total{surface,outcome}`,
  `properties_listing_response_seconds{surface}` and
  `properties_listing_results_returned{surface}` from
  `ListingsMetricsInterceptor`, plus
  `properties_listing_cache_lookups_total{outcome}` from the cache service.
  Every label set is fixed (2/3/3 values), so cardinality cannot grow with
  request input; `surface` is derived from the request path rather than the
  handler name so a new handler cannot silently widen the label space. The
  results histogram makes a broken filter visible long before anyone reports it.
  docs/api/listings-metrics.md
- MettaChain#1384: the case-insensitive search filters compile to
  `LOWER("city") = LOWER($1)`, which a plain btree index on the raw column can
  never serve - so `@@index([city, state])` and `@@index([propertyType])` were
  never chosen by the planner for the filters they exist for.
  20261001000001_add_listing_search_functional_indexes indexes the expression
  instead, leading with (status, is_deleted) so a filtered search is one index
  seek: LOWER(city), LOWER(state), LOWER(property_type), a partial
  LOWER(category), GIN on the tags array for `{ has: tag }` containment, and
  LOWER(amenity_type) on property_amenities. Prisma cannot express functional or
  partial indexes in the schema, so they are intentionally not declared there and
  the doc says so - if a future `migrate dev` proposes dropping them, re-create
  them. docs/api/listings-db-optimization.md
- MettaChain#1385: GET /properties/search is read through PropertiesListingsCacheService,
  TTL 30s, registered under CACHE_TAGS.PROPERTIES. The key is a sha1 of the
  filter set with entries sorted and absent values dropped, so query-string order
  and omitted-vs-undefined cannot produce two keys for one filter set. An empty
  filter set is deliberately NOT cached: that is the full feed, which changes
  constantly. Keys over 2KB are skipped so a pathological query cannot create an
  unbounded Redis key. Cache errors fall through to the database.
  docs/api/listings-caching.md
- MettaChain#1386: GET /properties/search had no entry in ENDPOINT_RATE_LIMITS at all, so
  being unauthenticated it was bounded only by the global per-IP bucket shared
  with the whole API. It is the most expensive read in the module - up to 25
  filters, an optional in-memory trust-score pass and a COUNT per page - so one
  client could spend that entire budget on search alone. Now 60 per minute,
  against 100/min for the plain list. The existing global guard was reused
  rather than adding a route guard class with its own DI surface.
  docs/api/listings-rate-limiting.md

PropertiesListingsCacheService is injected with @optional(), so PropertiesService
still constructs without it and the existing unit test module is unaffected.

Nothing was installed, built, linted or tested; please verify in CI.

Closes MettaChain#1383
Closes MettaChain#1384
Closes MettaChain#1385
Closes MettaChain#1386
@drips-wave

drips-wave Bot commented Oct 18, 2026

Copy link
Copy Markdown

@m-hajjo Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add rate limiting to ListingAPI Add Redis caching for ListingAPI Optimize DB queries in ListingAPI Add Prometheus metrics to ListingAPI

1 participant