Skip to content

feat(audit): make audit records tamper-evident and queryable - #142

Open
woahwhattheheck wants to merge 11 commits into
RemitFlow:mainfrom
woahwhattheheck:latch/remitflow-127-audit-integrity
Open

feat(audit): make audit records tamper-evident and queryable#142
woahwhattheheck wants to merge 11 commits into
RemitFlow:mainfrom
woahwhattheheck:latch/remitflow-127-audit-integrity

Conversation

@woahwhattheheck

@woahwhattheheck woahwhattheheck commented Sep 24, 2026

Copy link
Copy Markdown

Closes #127

Problem

Sensitive transfer investigations are unreliable if audit records can be changed silently or cannot be correlated to a request.

What this PR does

Acceptance criterion How it is met
Tampering is detectable Each entry carries chainSeq, prevHash, and entryHash (SHA-256 over a canonical payload). verifyIntegrity() / GET /api/audit/integrity recomputes the chain.
Every privileged mutation has one outcome event Create / claim / cancel / archive / unarchive / user-create emit a single outcome: success event. Retries with the same (action, target, correlationId, outcome) return the existing entry instead of appending.
Authorized operators can filter without exposing secrets GET /api/audit remains behind audit:read. New filters: action, scope, outcome, correlationId, actor. Changes are redacted at write; actors are HMAC fingerprints, never raw tokens.

Design tradeoffs

  • Hash chain over Merkle trees / external anchoring. Enough to detect silent in-process edits for the in-memory store without inventing a persistence layer the rest of the app does not have.
  • Dedup keyed by correlation id. Without a correlation id, appends remain unrestricted so internal/test callers are unchanged. With one, a retried mutation cannot inflate the trail.
  • Actor fingerprints, not raw tokens. Operators can still filter by who acted; captured audit responses cannot be used to recover API tokens.
  • Compat aliases kept. resourceId / requestId / payload continue to work alongside target / correlationId / changes.

Compatibility impact

  • Additive fields on audit entries and additive query params on GET /api/audit.
  • New route: GET /api/audit/integrity (audit:read).
  • Archive / unarchive now write audit events (previously silent).
  • Optional env: AUDIT_ACTOR_SECRET (falls back to PAGINATION_CURSOR_SECRET).

Tests

test/auditIntegrity.test.js covers:

  • Integrity-chain linking and clean verification
  • REGRESSION: silent field edit breaks the chain
  • REGRESSION: broken prevHash is detected
  • Redaction of nested secrets at rest and over HTTP
  • Duplicate-event suppression (including transfer-create retry)
  • Query authorization (401 / 403 / integrity scope)
  • Correlation / scope / actor filters without leaking tokens
  • Archive / unarchive each emit one outcome event

Verification

npm test
# tests 270
# pass  270
# fail  0

Baseline on upstream/main was 256. No unrelated tests were skipped, deleted, or weakened.

Out of scope

  • External blockchain anchoring or durable disk WAL
  • Broad rewrites of unrelated services or user flows

woahwhattheheck and others added 11 commits September 24, 2026 18:19
@woahwhattheheck
feat(audit): make audit records tamper-evident and queryable
557a2ba 
Add hash-chained integrity metadata, redacted attribution fields, duplicate
outcome suppression, and authorized filters so transfer investigations can
detect silent tampering and correlate privileged mutations without exposing
secrets.
@woahwhattheheck
fix(audit): retain distinct archive outcomes within one correlation
460f829 
An actual HTTP archive/unarchive/archive/unarchive sequence returned four successful state changes but only two audit outcomes when the caller reused a correlation ID. The later mutations were suppressed as retries while integrity still reported a valid, incomplete trail.

Add optional mutationId to audit records and derive it from the existing monotonic updatedAt for archive/unarchive. Include normalized actor, scope and state-change identity in the structured replay key, preserving caller correlation fields and legacy first-write-wins behavior. Bind the optional ID in the integrity hash; entries without it retain their prior field shape and hash calculation. Archive no-ops still add no event.

Three new regressions fail on the previous source. The actual app now records all six state changes across three cycles under the same correlation, including actor changes and repeated archive no-ops. Explicit mutation replay, actor/scope isolation and mutation-ID tamper detection pass. Existing creation-replay and redaction cases remain intact.

Validation: npm test passes 273/273, 0 failed, 0 skipped; audit suite 17/17; git diff --check passes. Executing the previous auditCrypto implementation against a current legacy entry confirms the original hash is preserved. Node24.19.0 with the exact existing package-lock dependency tree; no dependency changes.

Operation: REMIT142-AUDIT-CYCLE-20261003-443D
@woahwhattheheck
fix(audit): detect tampering through compatibility aliases
2a87aa7 
Verify resourceId, requestId, and payload against their hash-bound canonical
fields without changing stored hashes. Equivalent payload copies stay valid.

Three HTTP regressions fail on prior source. Full npm test: 277 passed.
@woahwhattheheck
fix(audit): fingerprint credentials that resemble actor references
0044e2e 
Privileged transfer and user mutations now pass raw credentials through the explicit actorToken input. Always fingerprint that input, including actor-prefixed, fingerprint-shaped and literal system tokens, while preserving legacy precomputed references, actor-scoped replay, archive-cycle identities and compatibility-alias integrity checks.

Actual createApp HTTP: 44 requests across four configured synthetic-token profiles. All mutation outcomes remain successful, each profile retains seven audit events, fingerprint filtering returns those events, retries reuse the original transfer and integrity remains valid. The three lookalike profiles previously stored the raw credential and returned zero results for its fingerprint.

Node 24.19.0, all 76 retained package-lock versions exact, no install. npm test -- --test-concurrency=1: 280 pass, 0 fail/skips. Focused audit suite: 24 pass; unchanged parent source with the same tests: 21 controls pass and three added regressions fail. Four changed JS files pass node --check; git diff --check passes. The HTTP exercise uses the repository's existing mock settlement adapter; no external settlement or Node 22 run is claimed.
@woahwhattheheck
docs: explain audit integrity investigation and actor secret setup
eed04bf 
Document authenticated audit filters, opaque actor references, and inspecting
valid even when the integrity endpoint returns HTTP 200. Explain the existing
import-time secret precedence and process-local chain lifetime in the README
and environment template.

Source and complete documentation readback are the verification for this
continuation. Runtime and maintained tests were not rerun during the execution
environment outage; prior source verification remains historical.
@woahwhattheheck
fix(audit): preserve own JSON keys in redacted integrity evidence
d2962a3 
Assigning a parsed __proto__ key to a plain clone invokes the inherited
setter. The key then disappears from serialized changes and the canonical
event hash, so editing that evidence does not change the hash. Construct
own data properties during recursive redaction to retain the field while
still redacting secrets nested beneath it.

Add three maintained node:test cases for own-key and prototype preservation,
nested redaction plus JSON round trips, mutation-sensitive event hashes,
and the unchanged canonical digest for ordinary JSON. On the preceding
source, two regressions fail and the compatibility control passes. All
three pass after this change on Node 24.19.0:

  node --test test/auditCrypto.test.js

Both changed files also pass node --check. The current full Express/HTTP
suite was not run; these tests use the actual module and Node built-ins.
No dependency, hash format, actor identity, replay, or storage change.
@woahwhattheheck
Include every array slot in audit integrity hashes [skip ci]
2bee10a
@woahwhattheheck
fix(audit): preserve pending claims on adapter failure [skip ci]
7192af8
@woahwhattheheck
perf(audit): reduce canonical encoding allocations
d4f7be3 
Preserve canonical bytes, existing integrity hashes and sparse-array handling while avoiding temporary item arrays. Include paired measurements and the reusable benchmark for the production encoder and hash function.
@woahwhattheheck
fix(audit): report malformed payloads as failed integrity checks
887d5b1 
Catch failures from hash recomputation and compatibility payload encoding. Return the broken index and a fixed reason without exposing the payload exception or changing stored hashes. Add two focused regressions covering cyclic changes and a throwing payload accessor.
@woahwhattheheck
fix(audit): cover prefixed credential key
93b24ba
@woahwhattheheck woahwhattheheck mentioned this pull request Oct 6, 2026
Open
@woahwhattheheck

woahwhattheheck commented Oct 6, 2026

Copy link
Copy Markdown
Author

Current-head QA at 93b24ba70075e9bd547997aa6ec027582e6c2de4: #127’s hash-chain verification, redacted actor/change storage, mutation deduplication, audit:read filtering, and transfer/user mutation coverage are present. Admin is read-only, so there is no uncovered admin write route. I did not run a local suite. GitHub shows no commit status contexts; CI run 37211230621 is action_required, not a test failure. Source unchanged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

security(backend): make audit records tamper-evident and queryable

1 participant

@woahwhattheheck