Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions docs/CACHE_INVALIDATION_POLICY.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# User cache invalidation policy (#514)

Redis caching is an optional, best-effort optimization. PostgreSQL remains authoritative.
Expand Down Expand Up @@ -56,6 +56,29 @@
all stale reads. The number of deleted keys is fixed; payload validation/serialization and
Redis memory usage scale with payload size.

## Cross-pod invalidation (#611)

In multi-pod deployments, cache invalidation events are broadcast via Redis pub/sub:

- **Channel**: `neurowealth:cache-invalidation` (configurable via `CACHE_INVALIDATION_CHANNEL`)
- **Event format**: `{ userId, correlationId, timestamp, originPodId }`
- **Behavior**: When a mutation invalidates a user's cache on pod A, the event is published to Redis; pod B receives it and invalidates the same keys locally
- **Origin filtering**: Pods ignore events they published themselves to avoid duplicate invalidation
- **Failure mode**: If Redis is unavailable, invalidation is pod-local only. Stale data on other pods expires via TTL or is refreshed on next read.

This provides bounded consistency SLA: mutations are visible on other pods within the Redis pub/sub delivery latency (typically <100ms) plus processing time. Clients reading stale data will see corrections on their next request due to cache validation or TTL expiration.

### Configuration

```env
CACHE_INVALIDATION_CHANNEL=neurowealth:cache-invalidation
REDIS_URL=redis://...
```

### Startup and shutdown

The cache invalidation subscriber is initialized during application startup (best-effort, never blocks startup) and gracefully shut down during termination. The subscriber uses a dedicated Redis connection (via `createRedisSubscriber`) to avoid interfering with the shared cache client.

## Verification

```bash
Expand Down
24 changes: 24 additions & 0 deletions docs/DATA_RETENTION_POLICY.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,9 @@ RETENTION_PROTOCOL_RATES_DAYS=180
RETENTION_YIELD_SNAPSHOTS_DAYS=90
RETENTION_ATTRIBUTION_DAYS=365

# Dry-run mode (logs counts without deleting)
DATA_RETENTION_DRY_RUN=false

# Archival Settings
ARCHIVAL_ENABLED=true
ARCHIVAL_STORAGE_PROVIDER=s3
Expand All @@ -166,6 +169,27 @@ RETENTION_INTERVAL_MS=86400000 # 24 hours
ARCHIVAL_INTERVAL_MS=604800000 # 7 days
```

## Dry-Run Mode

The data retention job supports a dry-run mode for safe testing and validation:

- **Enable**: Set `DATA_RETENTION_DRY_RUN=true`
- **Behavior**: Logs the number of rows that would be deleted per table without performing any deletions
- **Use cases**:
- Validate retention policies before deployment
- Estimate deletion impact without risk
- Debug retention job behavior in production-like environments
- Generate retention reports for compliance auditing

Dry-run mode applies to all retention tables:
- `auth_nonce` (expired nonces)
- `processed_event` (events older than retention window)
- `dead_letter_event` (RESOLVED entries older than retention window)
- `agent_log` (logs older than retention window)
- `user_event` (events older than retention window and per-user cap overages)

Logs include per-table counts (`wouldDelete`) and retention parameters (retention days, max per user for user events).

## Monitoring and Alerts

### Metrics to Monitor
Expand Down
Loading
Loading