Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,20 @@ DATABASE_URL="postgresql://parashield:non-production-dev-pg-password-change-in-p
# Redis password used by docker-compose.yml
# REDIS_PASSWORD=non-production-dev-redis-password

# ─── Redis connection (#710) ───────────────────────────────────────────────────
# Connection URL for the shared Redis client (rate limiting, idempotency,
# distributed locks, policy-state cache). Falls back to redis://localhost:6379
# when unset. Include credentials in the URL when Redis requires auth
# (e.g. redis://:password@host:6379) — they are never logged.
# REDIS_URL="redis://localhost:6379"

# Startup connection retry. The app retries the first Redis connection with
# exponential backoff + jitter before failing startup. All optional; defaults shown.
# REDIS_STARTUP_MAX_ATTEMPTS=5 # total attempts, including the first
# REDIS_STARTUP_INITIAL_DELAY_MS=200 # delay before the 2nd attempt (doubles each retry)
# REDIS_STARTUP_MAX_DELAY_MS=3000 # cap on the backoff delay
# REDIS_STARTUP_ATTEMPT_TIMEOUT_MS=5000 # per-attempt timeout; a hung connect cannot stall startup

# Distributed locking for scheduled background jobs (#675) — optional.
# Stable lock keys are used per job (lock:job:auth-cleanup, lock:job:oracle-worker,
# lock:job:claims-worker). Lock TTL in milliseconds. Defaults: 300000 (5m) for auth cleanup,
Expand Down
53 changes: 53 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,26 @@ npm run start:prod

---

## API versioning (#713)

Every response carries an `X-API-Version` header advertising the semantic
version that served it (`v1` → `1.0.0`, `v2` → `2.0.0`), including error
responses, streamed responses (SSE/NDJSON) and responses that never reach a
route handler (CORS preflights, request timeouts).

- The version is taken from the route's `vN` segment (`/api/v1/...`), falling
back to a request `x-api-version` header, then to `v1`.
- Requests for an unknown version (`/api/v3/...`) are rejected with `400`.
- `v1` responses also include `Deprecation: true` and
`Link: </api/v2>; rel="successor"`.
- The header is exposed via CORS so browser clients can read it.

Resolution lives in `src/common/versioning/api-version.ts` and is applied by the
versioning interceptor (success paths), the global exception filter (error
paths) and an early middleware (everything else).

---

## CORS configuration

CORS is enabled in `src/main.ts` and controlled entirely via environment variables (see `.env.example`):
Expand Down Expand Up @@ -370,6 +390,7 @@ All `BigInt` values (such as oracle readings, large policy IDs, and 64-bit times

- **Input sanitization**: a global middleware (`InputSanitizationMiddleware`, registered in `src/main.ts`) trims every string in JSON/urlencoded request bodies and escapes `<`/`>` before validation and persistence, so markup cannot survive into stored values or reflected responses.
- **Connection pooling**: Prisma pool sizing is applied at runtime to the datasource URL (`connection_limit=10`, `pool_timeout=10`, `connect_timeout=5`). Override each value with `DATABASE_CONNECTION_LIMIT`, `DATABASE_POOL_TIMEOUT`, `DATABASE_CONNECT_TIMEOUT`, or by putting the parameter directly in `DATABASE_URL`.
- **Response header hardening (#711)**: `applySecurityHeaders()` (`src/common/security/response-header-hardening.ts`, called from `src/main.ts`) disables Express's `X-Powered-By`, applies [helmet](https://helmetjs.github.io/) and strips implementation-revealing headers (`Server`, `X-Powered-By`, `X-AspNet-Version`, `X-AspNetMvc-Version`, `X-Runtime`, `Via`) from every response. Stripping is done at the header level, so headers added later by the framework or an upstream proxy are removed too. Extra names can be added via `extraHeadersToStrip`.

---

Expand Down Expand Up @@ -411,6 +432,8 @@ CONCURRENCY_TEST_DATABASE_URL=postgresql://.../parashield_test npx jest claims.c
| Flight delay | [AviationStack](https://aviationstack.com) | Yes (`AVIATIONSTACK_API_KEY`) |
| DeFi exploit | Stellar RPC event stream | No |

**Threshold evaluation (#712):** oracle readings are 7-decimal fixed point (e.g. `324000000` = 32.4) and product thresholds are `Decimal(20,7)`. Trigger comparisons are done on exact scaled `BigInt` values (`src/common/utils/decimal.ts`) rather than `parseFloat`, so the last decimal place is never lost and a reading exactly equal to the threshold never triggers (strict `LessThan` / greater-than).

---

## Project layout
Expand Down Expand Up @@ -473,6 +496,36 @@ Fee per tx: ~0.00001 XLM. Fund via `stellar keys fund <address> --network testne

---

## Redis connection & startup retry (#710)

The app creates a single shared Redis client (`REDIS_URL`) used for rate limiting,
idempotency, distributed locks and the policy-state cache. The client is created
with `lazyConnect`, so construction never blocks; the first connection is performed
during startup by `RedisConnectionService` (`src/redis/redis-connection.service.ts`).

Startup behaviour:

- The first connection is retried with exponential backoff, a capped delay and
jitter. A per-attempt timeout guarantees a hung connect cannot stall startup.
- If every attempt fails, startup fails fast with one actionable error naming the
Redis `host:port` (never the username/password or full URL).
- Once connected, runtime reconnection is handed back to ioredis (its
`retryStrategy`/`reconnectOnError` handle transient drops).

Tuning (all optional — defaults shown):

| Variable | Default | Description |
| --- | --- | --- |
| `REDIS_STARTUP_MAX_ATTEMPTS` | `5` | Total connection attempts, including the first. |
| `REDIS_STARTUP_INITIAL_DELAY_MS` | `200` | Delay before the second attempt; doubles each retry. |
| `REDIS_STARTUP_MAX_DELAY_MS` | `3000` | Upper bound for the backoff delay. |
| `REDIS_STARTUP_ATTEMPT_TIMEOUT_MS` | `5000` | Per-attempt timeout. |

Invalid values (negative, non-integer or out of range) fail startup with a clear
validation error. Credentials embedded in `REDIS_URL` are never logged.

---

## v2 roadmap

- Full Soroban SDK transaction builder for all write paths (currently stubbed with `// TODO` markers)
Expand Down
18 changes: 18 additions & 0 deletions src/app.module.ts
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,24 @@ function validateConfig(config: Record<string, unknown>) {
logger.warn('REDIS_URL is not set — falling back to redis://localhost:6379');
}

// #710 — Validate Redis startup-retry tuning. All optional; defaults live in
// src/redis/redis-startup.ts and are documented in .env.example / README.
const redisRetryBounds: Array<{ key: string; min: number; max: number }> = [
{ key: 'REDIS_STARTUP_MAX_ATTEMPTS', min: 1, max: 100 },
{ key: 'REDIS_STARTUP_INITIAL_DELAY_MS', min: 0, max: 60000 },
{ key: 'REDIS_STARTUP_MAX_DELAY_MS', min: 0, max: 300000 },
{ key: 'REDIS_STARTUP_ATTEMPT_TIMEOUT_MS', min: 1, max: 120000 },
];
for (const { key, min, max } of redisRetryBounds) {
const raw = config[key] as string | undefined;
if (raw !== undefined && raw !== '') {
const parsed = Number(raw);
if (!Number.isInteger(parsed) || parsed < min || parsed > max) {
errors.push(`${key} must be an integer between ${min} and ${max}`);
}
}
}

// #716 — Validate all secrets for length, format, entropy, and placeholders
const nodeEnv = (config['NODE_ENV'] as string) || process.env.NODE_ENV || 'development';
try {
Expand Down
89 changes: 89 additions & 0 deletions src/claims/claims.service.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -631,6 +631,95 @@ describe('ClaimsService', () => {
expect(mockStellarService.invokeContract).not.toHaveBeenCalled();
});

// #712 — The threshold comparison must be exact. A reading equal to a
// LessThan threshold is NOT a trigger (strict), and this must hold at full
// 7-decimal resolution rather than via parseFloat's binary rounding.
it('#712 — a reading exactly equal to the LessThan threshold does not trigger', async () => {
mockPrismaService.policy.findUnique.mockResolvedValue(ACTIVE_POLICY);
mockPrismaService.claim.findFirst.mockResolvedValue(null);
mockPrismaService.claim.create.mockResolvedValue({ id: 'claim-eq-lt', status: 'PROCESSING' });
mockOracleService.getLatestReading.mockResolvedValue({
key: ACTIVE_POLICY.oracleKey,
value: BigInt(500_000_000), // exactly 50.0000000 — equal to the threshold
confidence: 90,
});
mockPolicyService.getProductById.mockResolvedValue(MOCK_PRODUCT); // threshold 50.0000000, LessThan
mockPrismaService.claim.update.mockResolvedValue({});

const result = await service.autoProcess(POLICY_ID);

expect(result).toBe('Rejected');
expect(mockStellarService.invokeContract).not.toHaveBeenCalled();
expect(mockPrismaService.claim.update).toHaveBeenCalledWith(
expect.objectContaining({
data: expect.objectContaining({ status: 'REJECTED', triggerMet: false }),
}),
);
});

it('#712 — a reading exactly equal to the GreaterThan threshold does not trigger', async () => {
const GT_PRODUCT = { ...MOCK_PRODUCT, comparison: 'GreaterThan', threshold: '30.0000000' };
mockPrismaService.policy.findUnique.mockResolvedValue(ACTIVE_POLICY);
mockPrismaService.claim.findFirst.mockResolvedValue(null);
mockPrismaService.claim.create.mockResolvedValue({ id: 'claim-eq-gt', status: 'PROCESSING' });
mockOracleService.getLatestReading.mockResolvedValue({
key: ACTIVE_POLICY.oracleKey,
value: BigInt(300_000_000), // exactly 30.0000000 — equal to the threshold
confidence: 90,
});
mockPolicyService.getProductById.mockResolvedValue(GT_PRODUCT);
mockPrismaService.claim.update.mockResolvedValue({});

const result = await service.autoProcess(POLICY_ID);

expect(result).toBe('Rejected');
expect(mockStellarService.invokeContract).not.toHaveBeenCalled();
});

it('#712 — the last decimal place of the threshold is honoured (one unit below triggers)', async () => {
// Reading is exactly 1e-7 below the threshold — must trigger LessThan.
const PRODUCT = { ...MOCK_PRODUCT, threshold: '50.0000001' };
mockPrismaService.policy.findUnique.mockResolvedValue(ACTIVE_POLICY);
mockPrismaService.claim.findFirst.mockResolvedValue(null);
mockPrismaService.claim.create.mockResolvedValue({ id: 'claim-last-dp', status: 'PROCESSING' });
mockOracleService.getLatestReading.mockResolvedValue({
key: ACTIVE_POLICY.oracleKey,
value: BigInt(500_000_000), // 50.0000000 < 50.0000001
confidence: 90,
});
mockPolicyService.getProductById.mockResolvedValue(PRODUCT);
mockStellarService.invokeContract.mockResolvedValue('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa');

const result = await service.autoProcess(POLICY_ID);

expect(result).toBe('Paid');
expect(mockStellarService.invokeContract).toHaveBeenCalled();
});

// #712 regression: with the old `parseFloat(threshold) * 1e7` path, a large
// threshold loses low-order digits to double rounding, so a reading exactly
// equal to the threshold can drift below it and pay out. The scaled BigInt
// comparison keeps the boundary exact.
it('#712 — exact comparison holds for large thresholds (no float drift)', async () => {
const PRODUCT = { ...MOCK_PRODUCT, threshold: '1000000000000.0000001' };
mockPrismaService.policy.findUnique.mockResolvedValue(ACTIVE_POLICY);
mockPrismaService.claim.findFirst.mockResolvedValue(null);
mockPrismaService.claim.create.mockResolvedValue({ id: 'claim-large', status: 'PROCESSING' });
mockOracleService.getLatestReading.mockResolvedValue({
key: ACTIVE_POLICY.oracleKey,
// Exactly equal to the threshold, as scaled BigInt.
value: BigInt('10000000000000000001'),
confidence: 90,
});
mockPolicyService.getProductById.mockResolvedValue(PRODUCT);
mockPrismaService.claim.update.mockResolvedValue({});

const result = await service.autoProcess(POLICY_ID);

expect(result).toBe('Rejected');
expect(mockStellarService.invokeContract).not.toHaveBeenCalled();
});

// #259 — When getProductById returns null/undefined (product deactivated after
// the policy was sold), the claim must fail loud for manual review rather than
// silently substituting a threshold/comparison that may not match what the
Expand Down
25 changes: 14 additions & 11 deletions src/claims/claims.service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import { Prisma, ClaimStatus, PolicyStatus } from '@prisma/client';
import { WebhooksService } from '../common/events/webhooks.service';
import { StatusEventsService } from '../common/events/status-events.service';
import { ErrorCode } from '../common/errors/error-codes';
import { compareScaledToThreshold, parseDecimalScaled, parseFixedPointBigInt } from '../common/utils/decimal';
import { financialBatch, financialTransaction, isSerializationFailure } from '../prisma/financial-transaction';

export type ClaimResult = 'Paid' | 'Rejected' | 'Expired' | 'AlreadyClaimed' | 'AlreadyProcessed' | 'PolicyNotActive' | 'PendingFinalPeriod';
Expand Down Expand Up @@ -260,12 +261,14 @@ export class ClaimsService {
}
// #245 — Guard against non-numeric threshold values. Product.threshold is a
// free-text string at the service boundary (Decimal serialised via .toString()).
// parseFloat('') or parseFloat('N/A') yields NaN, and BigInt(NaN) throws a
// RangeError — crashing mid-flow after the claim row is already PROCESSING.
// Treat an unparseable threshold the same as a missing product: fail loud for
// manual review and revert the atomic gate so the policy can be retried.
const rawThreshold = parseFloat(product.threshold);
if (!isFinite(rawThreshold)) {
// #712 — Parse it exactly (scaled BigInt) instead of `parseFloat`, which loses
// precision and could put a boundary value on the wrong side of the trigger.
// An unparseable threshold is treated the same as a missing product: fail loud
// for manual review and revert the atomic gate so the policy can be retried.
let threshold: bigint;
try {
threshold = parseDecimalScaled(product.threshold);
} catch {
this.logger.error(
`Product ${policy.productId} has non-numeric threshold "${product.threshold}" — cannot evaluate trigger for policy ${policyId}, marking claim FAILED for manual review`,
);
Expand All @@ -291,13 +294,13 @@ this.auditOp('Claim', claim.id, ClaimStatus.PROCESSING, ClaimStatus.FAILED, 'Non
await this.policyStateCache?.invalidate(policyId);
return 'Rejected';
}
const threshold = BigInt(Math.round(rawThreshold * 1e7));
const comparison = product.comparison;

const readingValue = BigInt(reading.value);
const triggerMet = comparison === 'LessThan'
? readingValue < threshold
: readingValue > threshold;
// #712 — Compare the exact scaled values. Semantics are unchanged: 'LessThan'
// triggers strictly below the threshold, otherwise strictly above; a reading
// exactly equal to the threshold never triggers.
const readingValue = parseFixedPointBigInt(reading.value);
const triggerMet = compareScaledToThreshold(readingValue, threshold, comparison);

this.logger.log(
`Trigger eval: key=${reading.key} value=${reading.value} threshold=${threshold} triggerMet=${triggerMet}`,
Expand Down
43 changes: 42 additions & 1 deletion src/claims/policy-state-cache.service.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,8 @@ describe('PolicyStateCacheService (#714)', () => {
beforeEach(async () => {
mockPrisma = {
policy: {
findUnique: jest.fn().mockResolvedValue(JSON.parse(JSON.stringify(mockPolicy))),
// structuredClone (not JSON round-trip) so BigInt/Date fields survive.
findUnique: jest.fn().mockResolvedValue(structuredClone(mockPolicy)),
findMany: jest.fn().mockResolvedValue([{ id: 'policy-uuid-1' }]),
},
};
Expand Down Expand Up @@ -316,4 +317,44 @@ describe('PolicyStateCacheService (#714)', () => {
expect(dbQueryReductionPct).toBe(95);
});
});

describe('#712 — exact oracle threshold comparison', () => {
it('does not trigger when the reading equals the threshold', async () => {
mockOracle.getLatestReading.mockResolvedValue({
...mockReading,
value: BigInt(500_000_000), // exactly 50.0, equal to the '50.0' threshold
});

const state = await service.computePolicyState('policy-uuid-1');

expect(state?.triggerMet).toBe(false);
expect(state?.latestReadingValue).toBe('500000000');
});

it('triggers when the reading is one unit below a LessThan threshold', async () => {
mockOracle.getLatestReading.mockResolvedValue({
...mockReading,
value: BigInt(499_999_999), // 49.9999999 < 50.0
});

const state = await service.computePolicyState('policy-uuid-1');

expect(state?.triggerMet).toBe(true);
});

it('falls back to the reading timestamp when bucketStart is absent', async () => {
mockOracle.getLatestReading.mockResolvedValue({
key: 'rainfall:station-101:monthly',
value: '450000000',
confidence: 90,
timestamp: 1_700_000_000,
source: 'open-meteo',
});

const state = await service.computePolicyState('policy-uuid-1');

expect(state?.latestReadingTimestamp).toBe(1_700_000_000_000);
expect(state?.triggerMet).toBe(true);
});
});
});
25 changes: 17 additions & 8 deletions src/claims/policy-state-cache.service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import { PrismaService } from '../prisma/prisma.service';
import { OracleService } from '../oracle/oracle.service';
import { PolicyStatus, ClaimStatus } from '@prisma/client';
import { StatusEventsService, PolicyStatusEvent, PolicyOracleKeyChangeEvent } from '../common/events/status-events.service';
import { compareScaledToThreshold, parseDecimalScaled, parseFixedPointBigInt } from '../common/utils/decimal';

export const POLICY_DERIVED_STATE_TTL_SECONDS = 60;
const IN_MEMORY_MAX_ENTRIES = 2000;
Expand Down Expand Up @@ -305,21 +306,29 @@ export class PolicyStateCacheService implements OnModuleInit, OnModuleDestroy {
const reading = await this.oracleService.getLatestReading(policy.oracleKey);

const product = policy.product;
const rawThreshold = parseFloat(product.threshold.toString());
const thresholdBigInt = isFinite(rawThreshold) ? BigInt(Math.round(rawThreshold * 1e7)) : 0n;
// #712 — Parse the threshold exactly (scaled BigInt) instead of parseFloat.
// Unlike the payout path, a non-numeric threshold here keeps the previous
// graceful behaviour (treated as 0) rather than throwing.
let thresholdBigInt: bigint;
try {
thresholdBigInt = parseDecimalScaled(product.threshold.toString());
} catch {
thresholdBigInt = 0n;
}

let triggerMet = false;
let readingValueStr: string | null = null;
let readingTimestamp: number | null = null;

if (reading) {
readingValueStr = reading.value.toString();
readingTimestamp = reading.bucketStart.getTime();
const readingVal = BigInt(reading.value);
triggerMet =
product.comparison === 'LessThan'
? readingVal < thresholdBigInt
: readingVal > thresholdBigInt;
// #712 — Prefer the reading's own bucket start, falling back to its
// timestamp: OracleReading does not always carry a bucketStart.
readingTimestamp = reading.bucketStart
? reading.bucketStart.getTime()
: reading.timestamp * 1000;
const readingVal = parseFixedPointBigInt(reading.value);
triggerMet = compareScaledToThreshold(readingVal, thresholdBigInt, product.comparison);
}

const claims = policy.claims ?? [];
Expand Down
Loading