Skip to content

fix(benchmarking): return 404 on missing benchmarks, validate metrics/thresholds, and guard definition auth (closes #911) - #914

Closed
mopxam-art wants to merge 1 commit into
Heliobond:mainfrom
mopxam-art:fix/benchmarking-validation-and-auth
Closed

mopxam-art wants to merge 1 commit into
Heliobond:mainfrom
mopxam-art:fix/benchmarking-validation-and-auth

Conversation

@mopxam-art

Copy link
Copy Markdown

Overview

Closes #911 by standardizing HTTP 404 responses for missing benchmark resources, validating metrics against the supported set, enforcing non-negative ordering on thresholds, guarding custom definition collisions with built-in benchmarks, and protecting benchmark mutations with admin bearer authentication.

Changes

  • src/routes/benchmarking.ts:
    • Updated GET /benchmarks/:id and GET /:id/trend to return HTTP 404 (notFound) when a requested benchmark id does not exist.
    • Added requireAdminBearer middleware to POST /benchmarks to ensure consumer API keys cannot mutate global benchmark definitions.
    • Validated metric against KNOWN_METRICS with strict 400 rejection on unknown metrics.
    • Validated thresholds non-negativity and monotonic order (poor <= fair <= good <= excellent).
    • Rejected direct and slug-derived id collisions against built-in benchmarks.
  • src/lib/benchmarking.ts:
    • Exported KNOWN_METRICS, isKnownMetric, isBuiltInBenchmark, getBuiltInBenchmarkIds, and clearCustomBenchmarks.
    • Hardened defineCustomBenchmark to reject built-in ID overwrites.
  • src/middleware/errors.ts: Exported standard notFound(message) helper returning ApiError(404, 'not_found', message).
  • src/__tests__/benchmarking.test.ts: Fixed mislabeled test asserting 400 to assert 404.
  • src/__tests__/benchmarking-5d-matrix.test.ts: Added 111 comprehensive tests across boundaries, combinatorics, parameter variations, negative controls, and lifecycle.

Verification

  • All 121 benchmarking tests pass green (npx jest src/__tests__/benchmarking*).
  • TypeScript typecheck passes with 0 errors (npx tsc --noEmit).
  • ESLint passes with 0 warnings/errors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Benchmarking route: not-found returns 400, POST accepts unvalidated metric/thresholds/id, and any consumer key mutates global definitions

2 participants