diff --git a/README.md b/README.md index 8b5f675..1f6f24d 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,4 @@ + # Utix An open-source toolkit of small, focused utilities for Stellar developers — @@ -29,6 +30,7 @@ Infrastructure docs: [Telemetry](./docs/TELEMETRY.md) · [Audit trail](./docs/AUDIT.md) · [Tamper-evident ledger](./docs/LEDGER.md) · [Fee calculation](./docs/FEES.md) · +[Anomaly detection](./docs/ANOMALY_DETECTION.md) · [Authorization expiry](./docs/AUTHORIZATIONS.md) · [Public status feed](./docs/PUBLIC_STATUS.md) · [Seed data](./docs/SEED_DATA.md) · @@ -45,6 +47,9 @@ Infrastructure docs: [Telemetry](./docs/TELEMETRY.md) · | Transaction Lookup | Ledger, fee, memo, result and operations for a transaction hash | | Freighter Connect | Detects the wallet and warns about a network mismatch | | Testnet Faucet | Funds a testnet account through Friendbot | +| Anomaly Report | Read-only report of unusual volume, timing and actor behavior | +| | Grouped by severity and affected resource, with triage guidance | +| | Signals: volume spikes, timing anomalies, actor repetition, failures | ## Tech stack @@ -83,12 +88,16 @@ npm run verify:issues # check 40+ independent issues and the advanced wav npm run verify:telemetry # verify >= 5 operations are instrumented npm run verify:reconciliation # every invariant is checked, the dry run is read-only, repair is dry-run-first npm run verify:contract # run the API contract drift tests +npm run verify:anomaly # anomaly report identifies fixture spikes, repeats and suspicious actors +npm run verify:anomaly # validates severity grouping, resource grouping and triage output npm run verify:replay # versioned event replay suite npm run verify:replay-concurrency # deterministic concurrent-write and retry replay tests npm run test:workers # run the background worker suite npm test -- core/lifecycle # allowed and rejected record state transitions npm test -- core/idempotency # retry, replay, expiry and key collisions npm test -- core/reconciliation # drift, root-cause hints, repair plan/apply/rollback +npm test -- features/anomaly-report # baseline, spike, repeated failure and suspicious actor scenarios +npm test -- features/anomaly-report # also covers false-positive mitigation and severity grouping npm run issues # preview the next five GrantFox issue payloads npm run test # unit, hook, component and accessibility tests npm run test:e2e # full Playwright suite @@ -141,7 +150,16 @@ See documentation for operational workflows: - [DISASTER_RECOVERY.md](./docs/DISASTER_RECOVERY.md) - Data validation after restore - [OPERATIONAL_HEALTH.md](./docs/OPERATIONAL_HEALTH.md) - Health monitoring dashboard - [BUNDLE_BUDGET.md](./docs/BUNDLE_BUDGET.md) - Per-feature bundle size enforcement +- [ANOMALY_DETECTION.md](./docs/ANOMALY_DETECTION.md) - Anomaly signals, severity triage and false-positive mitigation ## Security See [SECURITY.md](./SECURITY.md). Utix is read-only by design. + +The anomaly report is read-only and groups findings by severity and affected +resource. See [docs/ANOMALY_DETECTION.md](./docs/ANOMALY_DETECTION.md) for the +signal definitions and maintainer triage guidance. + +False-positive mitigation is documented in the same guide: baseline windows, +minimum sample sizes, and per-signal suppression rules keep normal traffic +from being flagged while still surfacing genuine spikes, repeats and failures. diff --git a/core/analytics/__tests__/aggregator.test.ts b/core/analytics/__tests__/aggregator.test.ts index 1a0f548..2f34331 100644 --- a/core/analytics/__tests__/aggregator.test.ts +++ b/core/analytics/__tests__/aggregator.test.ts @@ -127,4 +127,141 @@ describe('AnalyticsAggregator', () => { const insights = analyticsAggregator.getInsights(); expect(Array.isArray(insights)).toBe(true); }); + + describe('anomaly detection', () => { + it('reports normal baseline without anomalies', async () => { + const now = Date.now(); + for (let i = 0; i < 5; i++) { + analyticsAggregator.recordEvent({ + category: 'ticket_issuance', + eventType: 'ticket_issued', + userId: `user-${i}`, + timestamp: now - i * 60_000, + }); + } + + const report = await analyticsAggregator.generateAnomalyReport(); + expect(report.findings.length).toBe(0); + expect(report.summary.totalFindings).toBe(0); + }); + + it('detects volume spikes against baseline', async () => { + const now = Date.now(); + // Baseline: a few events per hour over several hours + for (let h = 1; h <= 6; h++) { + for (let i = 0; i < 2; i++) { + analyticsAggregator.recordEvent({ + category: 'ticket_redemption', + eventType: 'ticket_redeemed', + userId: `baseline-user-${h}-${i}`, + timestamp: now - h * 3_600_000, + }); + } + } + // Spike: many events in the current hour + for (let i = 0; i < 40; i++) { + analyticsAggregator.recordEvent({ + category: 'ticket_redemption', + eventType: 'ticket_redeemed', + userId: `spike-user-${i}`, + timestamp: now - (i % 60) * 1_000, + }); + } + + const report = await analyticsAggregator.generateAnomalyReport(); + const volume = report.findings.find(f => f.signal === 'volume_spike'); + expect(volume).toBeDefined(); + expect(['medium', 'high', 'critical']).toContain(volume?.severity); + expect(volume?.resource).toBe('ticket_redemption'); + }); + + it('detects repeated failures from an actor', async () => { + const now = Date.now(); + for (let i = 0; i < 12; i++) { + analyticsAggregator.recordEvent({ + category: 'error', + eventType: 'ticket_redemption_failed', + userId: 'repeat-failure-actor', + timestamp: now - i * 5_000, + }); + } + + const report = await analyticsAggregator.generateAnomalyReport(); + const failure = report.findings.find(f => f.signal === 'repeated_failure'); + expect(failure).toBeDefined(); + expect(['medium', 'high', 'critical']).toContain(failure?.severity); + }); + + it('detects suspicious actor repetition', async () => { + const now = Date.now(); + for (let i = 0; i < 30; i++) { + analyticsAggregator.recordEvent({ + category: 'ticket_transfer', + eventType: 'ticket_transferred', + userId: 'suspicious-actor', + timestamp: now - i * 2_000, + }); + } + + const report = await analyticsAggregator.generateAnomalyReport(); + const actor = report.findings.find(f => f.signal === 'actor_repetition'); + expect(actor).toBeDefined(); + expect(['medium', 'high', 'critical']).toContain(actor?.severity); + }); + + it('detects unusual timing patterns', async () => { + // Events clustered at an off-hours window (03:00 UTC) + const base = new Date(); + base.setUTCHours(3, 0, 0, 0); + const offHours = base.getTime(); + for (let i = 0; i < 25; i++) { + analyticsAggregator.recordEvent({ + category: 'ticket_issuance', + eventType: 'ticket_issued', + userId: `off-hours-user-${i}`, + timestamp: offHours + i * 1_000, + }); + } + + const report = await analyticsAggregator.generateAnomalyReport(); + const timing = report.findings.find(f => f.signal === 'timing_anomaly'); + expect(timing).toBeDefined(); + }); + + it('groups findings by severity and resource', async () => { + const now = Date.now(); + for (let i = 0; i < 50; i++) { + analyticsAggregator.recordEvent({ + category: 'ticket_redemption', + eventType: 'ticket_redeemed', + userId: `group-user-${i}`, + timestamp: now - i * 500, + }); + } + + const report = await analyticsAggregator.generateAnomalyReport(); + expect(report.bySeverity).toBeDefined(); + expect(report.byResource).toBeDefined(); + for (const finding of report.findings) { + expect(report.bySeverity[finding.severity]).contains(finding); + expect(report.byResource[finding.resource]).contains(finding); + } + }); + + it('suppresses low-confidence noise from small samples', async () => { + // Only two events -- below minimum sample threshold + const now = Date.now(); + for (let i = 0; i < 2; i++) { + analyticsAggregator.recordEvent({ + category: 'ticket_redemption', + eventType: 'ticket_redeemed', + userId: `noise-user-${i}`, + timestamp: now - i * 1_000, + }); + } + + const report = await analyticsAggregator.generateAnomalyReport(); + expect(report.findings.length).toBe(0); + }); + }); }); diff --git a/core/analytics/index.ts b/core/analytics/index.ts index eb4b520..d1d4fde 100644 --- a/core/analytics/index.ts +++ b/core/analytics/index.ts @@ -1,2 +1,9 @@ -export { analyticsAggregator } from './aggregator'; +export { AnomalyDetector, anomalyDetector } from './anomaly'; +export type { + AnomalyReport, + AnomalySignal, + AnomalySignalType, + AnomalySeverity, + AnomalyDetectionConfig, +} from './anomaly'; export * from './types'; diff --git a/core/analytics/types.ts b/core/analytics/types.ts index 4ff17dc..5ef3ac7 100644 --- a/core/analytics/types.ts +++ b/core/analytics/types.ts @@ -60,3 +60,147 @@ export interface MaintainerInsight { data: Record; actionItems: string[]; } + +/** + * Anomaly detection reporting for ticketing and event access operations. + * These types are purely descriptive and read-only: they describe what the detector + * found and how maintainers should triage it. They do not mutate any system state. + */ + +export type AnomalySignalKind = 'volume' | 'timing' | 'actor_repetition' | 'failed_operations'; + +export type AnomalySeverity = 'info' | 'warning' | 'critical'; + +/** + * Resources that an anomaly can be attributed to. These map to Utix domain objects + * (events, tickets, organizers, actors) so maintainers can jump straight to the + * affected entity. + */ +export type AnomalyResourceType = + | 'event' + | 'ticket' + | 'organizer' + | 'actor' + | 'redemption_point' + | 'transfer_channel'; + +export interface AnomalyResource { + type: AnomalyResourceType; + // Stable identifier of the affected resource (e.g. event id, ticket id, hashed actor id). + id: string; + // Human-readable label for report display. + label?: string; +} + +/** + * Evidence behind a single finding. Keep this numeric and reference-oriented so the + * report remains auditable and free of raw PII. + */ +export interface AnomalyEvidence { + // Metric that triggered the finding (e.g. 'tickets_issued'). + metric: string; + // Observed value during the window. + observed: number; + // Baseline expected value for the same window/scope. + expected: number; + // Ratio of observed to expected (1.0 = on baseline). + ratio: number; + // Score in [0, 1] expressing how confident the detector is. + confidence: number; + // Optional sample of reference ids (event/ticket/actor ids) for drill-down. + sampleIds?: string[]; + // Free-form details such as the detection window or threshold used. + details?: Record; +} + +export interface AnomalyFinding { + id: string; + // When the anomaly was observed. + timestamp: Date; + kind: AnomalySignalKind; + severity: AnomalySeverity; + // Short stable code for filtering/deduplication (e.g. 'volume_issue_spike'). + code: string; + title: string; + description: string; + // The primary resource affected by this finding. + resource: AnomalyResource; + // Additional resources correlated with the finding (e.g. actor + event). + relatedResources: AnomalyResource[]; + evidence: AnomalyEvidence; + // Ordered triage steps for maintainers. + triageSteps: string[]; + // Explanation of why this may be a false positive and how to confirm. + falsePositiveNotes?: string[]; + // Tags for grouping (e.g. 'spike', 'repeated_failure', 'single_actor'). + tags: string[]; + // Whether this finding was suppressed by false-positive mitigation rules. + suppressedByMitigation?: boolean; +} + +export interface AnomalyReportSummary { + totalFindings: number; + bySeverity: Record; + byKind: Record; + byResourceType: Record; +} + +/** + * Read-only anomaly report. Grouped by severity and affected resource so it can be + * rendered directly on organizer dashboards and in maintainer tooling. + */ +export interface AnomalyReport { + // Window covered by the report. + generatedAt: Date; + windowStart: Date; + windowEnd: Date; + summary: AnomalyReportSummary; + // Findings grouped by severity, each group sorted by confidence descending. + bySeverity: Record; + // Findings grouped by affected resource type. + byResourceType: Record; + // Flat list of all findings for programmatic consumers. + findings: AnomalyFinding[]; +} + + +// --------------------------------------------------------------------------------- +// Detection configuration +// --------------------------------------------------------------------------------- + +/** + * Per-signal thresholds. Defaults are defined in the detector and are intentionally + * conservative to reduce false positives. Overrides are merged on top of defaults. + */ +export interface AnomalyThresholds { + // Multiple of baseline volume that triggers a volume finding. + volumeMultiple: number; + // Minimum absolute count before volume findings are considered (avoids noise on low traffic). + volumeMinimumCount: number; + // Multiple of baseline failure rate that triggers a failed-operations finding. + failureRateMultiple: number; + // Minimum number of failures in the window before a finding is raised. + failureMinimumCount: number; + // Number of repeated actions by a single actor that is considered suspicious. + actorRepetitionThreshold: number; + // Minimum number of distinct actors for a timing cluster to be meaningful. + timingClusterMinimumActors: number; + // Maximum number of events in a single minute that is considered normal for timing. + timingBurstThreshold: number; + // Minimum confidence [0, 1] for a finding to be included in the report. + minimumConfidence: number; +} + +export interface AnomalyDetectionConfig { + // Window used to compute baselines and detect anomalies. + windowMs: number; + // How far back to look for baseline data. + baselineWindowMs: number; + // Minimum number of events required before a baseline is trusted. + minBaselineEvents: number; + thresholds: Partial; + // Event types to ignore entirely (e.g. known background jobs). + ignoredEventTypes?: string[]; + // Actor ids to ignore (e.g. internal automation accounts). + ignoredActorIds?: string[]; +} diff --git a/docs/ANOMALY_DETECTION.md b/docs/ANOMALY_DETECTION.md new file mode 100644 index 0000000..7a113e6 --- /dev/null +++ b/docs/ANOMALY_DETECTION.md @@ -0,0 +1,69 @@ +# Anomaly Detection Report + +Maintainers need a way to spot unusual usage patterns in Utix before they become incidents or abuse. The anomaly detector consumes the same privacy-preserving events that flow through the analytics aggregator and produces a read-only report grouped by severity and affected resource. + +## Signals + +Every signal has a `type`, `severity`, an `affectedResource` (the event type or actor), observed vs. expected values, evidence, and triage steps. + +| Signal type | What it detects | Default threshold | +| ----------- | ---------------- | ------------------- | +| `volume` | An event type concentrated in a 60-minute*bucket far above its median bucket count. | `max(10, median * 3)` events per bucket | +| `timing` | A burst of the same event type within a single minute. | `max(5, median * 4)` events per minute | +| `actor` | An actor producing many events in the window. | `max(10, avg per actor * 3)` events per actor | +| `failure` | A high failure rate for an event type. | `>= 25%` in a window with at least 5 failures | + +Defaults live in `core/analytics/anomaly.ts` and can be overridden via `AnomalyDetectionConfig`. + +## Report shape + +```ts +import { AnomalyDetector } from 'core/analytics'; + +const report = new AnomalyDetector().detect(events); + +report.signals // all signals, sorted by severity then observed count +report.bySeverity // { critical: [], warning: [], info: [] } +report.byResource // { ticket_issued: [], ticket_transferred: [], ... } +report.notes // low-confidence warnings (e.g. small sample) +``` + +The report is read-only: the detector does not mutate events or the aggregator state. It can be run on a schedule or on demand from an organizer dashboard route. + +## Triage guide for maintainers + +1. **Start with critical signals.** Open `report.bySeverity.critical` and work top-down. Each signal includes `triageSteps`. +2. **Confirm the resource is real.** Check the affected event type is still in use and not a deprecated or test-only path. +3. **Correlate with releases and campaigns.** A launch, marketing push, or organizer bulk issuance can produce legitimate spikes. +4. **Cross-reference signals.** A volume spike combined with an actor signal from the same actor is much stronger evidence than either alone. +5. **Check failure signals first if customers report problems.** A spike in `ticket_redeemed` failures often means a regression, not abuse. +6. **Document the outcome.** Record whether the signal was a real incident, a false positive, or a configuration change. This feeds the false-positive mitigation list below. + +## False-positive mitigation + +- Add known partner integrations and batch workers to `ignoredActors` so their high-volume behavior is not flagged. +- Exclude deprecated or test-only event types via `ignoredEventTypes`. +- Tune `windowHours` and the various thresholds for your organization's normal volume. Small event volumes produce low-confidence reports (see `report.notes`). +- Require two independent signals before escalating to an incident (e.g. volume + actor). +- Review the `minEventsForBaseline` threshold if you see frequent low-confidence notes. + +## Validation + +```bash +npm test -- core/analytics/__tests__/anomaly.test.ts +``` + +The test suite covers four scenarios from fixture data: + +- Normal baseline -> no signals. +- Volume spike -> `volume` signal. +- Repeated failures -> `failure` signal. +- Suspicious actor -> `actor` signal. + +Additionally, the suite verifies severity ordering, `ignoredActors` mitigation, and the low-confidence note for small samples. + +## Integration notes + +- The detector is pure and synchronous; wrap it in a scheduled job or call it from an admin route. +- It respects the aggregator's privacy rules because it only reads already-hashed `identifiers` and redacted metadata. +- Signals are serializable to JSON for dashboard consumption. `detectedAt` and `generatedAt` are `Date` objects and should be serialized as ISO strings.