feat:Prometheus metric cardinality guard for user-controlled label va…

readme.md

@@ -0,0 +1,142 @@
+# NeuroWealth Backend
+
+[![CI](https://github.com/Neurowealth/Backend/actions/workflows/node-ci.yml/badge.svg)](https://github.com/Neurowealth/Backend/actions/workflows/node-ci.yml)
+[![License: ISC](https://img.shields.io/badge/License-ISC-blue.svg)](LICENSE)
+[![Node.js](https://img.shields.io/badge/Node.js-22%2B-green.svg)](https://nodejs.org/)
+[![TypeScript](https://img.shields.io/badge/TypeScript-5.3-blue.svg)](https://www.typescriptlang.org/)
+[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](CONTRIBUTING.md)
+
+> Autonomous AI investment agent backend that manages and grows crypto assets on the **Stellar blockchain**. Users deposit once and the AI finds yield opportunities across Stellar's DeFi ecosystem.
+
+## Features
+
+- **AI-Powered Portfolio Management** - Autonomous rebalancing using Anthropic Claude
+- **Stellar Blockchain Integration** - Native Soroban smart contracts, claimable balances, and multi-sig support
+- **DeFi Yield Optimization** - Automated yield farming across Stellar protocols
+- **Real-time Analytics** - Monte Carlo simulations, correlation analysis, factor exposure, stress testing
+- **NLP Trading Interface** - Natural language commands for portfolio management (Telegram, WhatsApp)
+- **Circuit Breaker System** - Automated risk management with exposure caps and rebalance cost analysis
+- **Privacy & Compliance** - GDPR/CCPA erasure, Travel Rule support, audit trail
+- **Tax Reporting** - Multi-jurisdiction tax calculation (US, UK, DE, CA, AU)
+- **Referral Program** - Multi-tier referral system with fraud detection
+
+## Quickstart
+
+```bash
+# 1. Clone and install
+git clone https://github.com/Neurowealth/Backend.git
+cd Backend
+npm install
+
+# 2. Configure environment
+cp .env.example .env
+# Edit .env with your values
+# REDIS_URL defaults to redis://localhost:6379 (provided by docker-compose)
+
+# 3. Start database and Redis
+docker-compose up -d
+
+# 4. Run migrations
+npx prisma migrate deploy
+
+# 5. Start development server
+npm run dev
+```
+
+## Available Scripts
+
+| Script | Description |
+|--------|-------------|
+| `npm run dev` | Start development server with hot-reload |
+| `npm test` | Run all tests |
+| `npm run test:unit` | Run unit tests only |
+| `npm run test:integration` | Run integration tests |
+| `npm run test:coverage` | Run tests with coverage report |
+| `npm run lint` | Run ESLint |
+| `npm run typecheck` | Run TypeScript type check |
+| `npm run build` | Build for production |
+| `npm run format` | Format code with Prettier |
+| `npm run validate:spec` | Validate OpenAPI specification |
+
+## Tech Stack
+
+| Layer | Technology |
+|-------|-----------|
+| Runtime | Node.js 22+ / TypeScript 5.3 |
+| Framework | Express.js |
+| Database | PostgreSQL 14+ (Prisma ORM) |
+| Cache/Queue | Redis (ioredis) |
+| Blockchain | Stellar SDK, Soroban smart contracts |
+| AI | Anthropic Claude SDK |
+| Monitoring | OpenTelemetry, Prometheus, Sentry |
+| Security | Helmet, CORS, JWT, Zod validation |
+| Docs | Swagger UI, OpenAPI (Redocly-validated) |
+| Testing | Jest, Supertest |
+| CI/CD | GitHub Actions, Dependabot |
+| Deployment | Docker, Kubernetes |
+
+## Project Structure
+
+```
+Backend/
+├── src/
+│   ├── agent/          # AI agent, strategies, risk management
+│   ├── analytics/      # Monte Carlo, correlation, factor analysis
+│   ├── auth/           # JWT authentication, API keys
+│   ├── compliance/     # Travel Rule, KYC support
+│   ├── controllers/    # Request handlers
+│   ├── fiat/           # Fiat on-ramp (Transak)
+│   ├── jobs/           # Background jobs (erasure, treasury, reconciliation)
+│   ├── middleware/      # Auth, rate limiting, sub-accounts
+│   ├── nlp/            # Natural language processing
+│   ├── outbox/         # Transactional outbox pattern
+│   ├── privacy/        # Data mapping, erasure policies
+│   ├── referral/       # Multi-tier referral system
+│   ├── routes/         # API route definitions
+│   ├── stellar/        # Stellar SDK integration
+│   ├── strategy/       # Strategy simulation
+│   ├── tax/            # Multi-jurisdiction tax reporting
+│   ├── telegram/       # Telegram bot integration
+│   ├── whatsapp/       # WhatsApp bot integration
+│   └── validators/     # Zod request validation
+├── tests/
+│   ├── unit/           # Unit tests
+│   ├── integration/    # Integration tests
+│   └── load/           # Load tests
+├── prisma/             # Database schema & migrations
+├── docs/               # Documentation (38+ files)
+├── deploy/             # Kubernetes & monitoring configs
+└── scripts/            # Operational scripts
+```
+
+## Documentation
+
+| Document | Description |
+|----------|-------------|
+| [Documentation Index](docs/DOCUMENTATION_INDEX.md) | Master index with reading order |
+| [API Reference](docs/API_REFERENCE.md) | Full endpoint reference |
+| [Deployment Guide](docs/DEPLOYMENT.md) | Local, staging, production setup |
+| [Runbook](docs/RUNBOOK.md) | Operational runbook |
+| [OpenAPI Spec](docs/openapi.yaml) | Machine-readable API specification |
+| [Contributing](CONTRIBUTING.md) | How to contribute |
+| [Security Policy](SECURITY.md) | Vulnerability reporting & security measures |
+
+## Environment Variables
+
+See [`.env.example`](.env.example) for all required environment variables.
+
+Key variables:
+- `DATABASE_URL` - PostgreSQL connection string
+- `STELLAR_NETWORK` - `testnet` or `mainnet`
+- `STELLAR_AGENT_SECRET_KEY` - Stellar secret key
+- `VAULT_CONTRACT_ID` - Deployed vault contract ID
+- `JWT_SECRET` - JWT signing secret
+- `ANTHROPIC_API_KEY` - Claude AI API key
+
+## Contributing
+
+We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
+
+## License
+
+This project is licensed under the ISC License - see the [LICENSE](LICENSE) file for details.

src/whatsapp/formatters.ts

@@ -1,9 +1,29 @@
+/**
+ * Chat reply formatters.
+ *
+ * Each function is a pure formatter: it takes already-fetched data and
+ * returns a single multi-line string ready to send as a chat message. No I/O
+ * happens here.
+ *
+ * Formatting conventions used throughout:
+ *  - `*text*` renders as bold and `_text_` as italic (Markdown-style chat
+ *    syntax, as used by messengers such as Telegram/WhatsApp).
+ *  - Each reply is built as an array of lines joined with '\n'.
+ *  - Lists are capped (3-5 items) so replies stay short enough for chat.
+ *  - Money is shown to 2 decimal places; APY values arrive as fractions
+ *    (0.05) and are shown as percentages (5.00%). The exception is
+ *    formatGoalProgressReply, which prints its APY values as given.
+ */
+
+// Minimal view of a user's position in a protocol; only what the reply needs.
 type PositionSummary = {
   protocolName: string
   assetSymbol: string
   currentValue: number
 }
 
+// Minimal view of a transaction. `txHash` is null until the transaction has
+// been submitted on-chain, which the formatters display as "pending".
 type TxSummary = {
   txHash: string | null
   type: string
@@ -12,18 +32,26 @@ type TxSummary = {
   assetSymbol: string
 }
 
+// Minimal view of a protocol's current supply rate. `supplyApy` is a
+// fraction (0.05 = 5%).
 type ProtocolRateSummary = {
   protocolName: string
   assetSymbol: string
   supplyApy: number
 }
 
+/**
+ * Portfolio snapshot: totals plus up to the first 3 positions.
+ * Positions are shown in the order given, so callers should sort them
+ * (e.g. by value) beforehand if they want the largest ones first.
+ */
 export function formatPortfolioReply(input: {
   totalBalance: number
   totalEarnings: number
   activePositions: number
   positions: PositionSummary[]
 }): string {
+  // Cap at 3 positions to keep the message short.
   const lines = input.positions.slice(0, 3).map((position) => {
     return `• ${position.protocolName} ${position.assetSymbol}: $${position.currentValue.toFixed(2)}`
   })
@@ -33,15 +61,23 @@ export function formatPortfolioReply(input: {
     `Balance: *$${input.totalBalance.toFixed(2)}*`,
     `Earnings: *$${input.totalEarnings.toFixed(2)}*`,
     `Active positions: *${input.activePositions}*`,
+    // Fall back to a friendly empty-state line when there is nothing to list.
     lines.length ? lines.join('\n') : 'No active positions yet.',
   ].join('\n')
 }
 
+/**
+ * Yield history for a chosen period, showing only the first 5 data points.
+ * Points are displayed in the order given, so callers decide whether that is
+ * oldest-first or newest-first.
+ */
 export function formatPortfolioHistoryReply(input: {
   period: '7d' | '30d' | '90d'
   points: Array<{ date: string; yieldAmount: number }>
 }): string {
   const lines = input.points.slice(0, 5).map((point) => {
+    // The "+" prefix is hardcoded, so a negative yieldAmount would render
+    // as "+$-1.23".
     return `• ${point.date}: +$${point.yieldAmount.toFixed(2)}`
   })
 
@@ -51,6 +87,10 @@ export function formatPortfolioHistoryReply(input: {
   ].join('\n')
 }
 
+/**
+ * Earnings summary: lifetime total, recent-period total, and average APY.
+ * `averageApy` is a fraction and is converted to a percentage here.
+ */
 export function formatPortfolioEarningsReply(input: {
   totalEarnings: number
   averageApy: number
@@ -59,17 +99,25 @@ export function formatPortfolioEarningsReply(input: {
   return [
     '🧾 *Earnings Summary*',
     `Total earned: *$${input.totalEarnings.toFixed(2)}*`,
+    // The label is fixed at "30d", so callers must pass 30-day earnings here.
     `30d earnings: *$${input.periodEarnings.toFixed(2)}*`,
     `Average APY: *${(input.averageApy * 100).toFixed(2)}%*`,
   ].join('\n')
 }
 
+/**
+ * One page of the transaction list. The caller is expected to have already
+ * paginated the data; this function only formats what it is given and does
+ * not slice by `limit` (the limit is just echoed in the header).
+ */
 export function formatTransactionsReply(input: {
   page: number
   limit: number
   transactions: TxSummary[]
 }): string {
   const lines = input.transactions.map((tx) => {
+    // Show a shortened hash (first 8 chars) to keep lines compact, or
+    // "pending" if the transaction has no on-chain hash yet.
     const hash = tx.txHash ? `${tx.txHash.slice(0, 8)}...` : 'pending'
     return `• ${tx.type} ${tx.amount} ${tx.assetSymbol} (${tx.status}) [${hash}]`
   })
@@ -80,26 +128,42 @@ export function formatTransactionsReply(input: {
   ].join('\n')
 }
 
+/**
+ * Full detail for a single transaction. Unlike the list view, this shows the
+ * complete hash (italicised so it is easy to spot and copy).
+ */
 export function formatTransactionDetailReply(input: TxSummary): string {
   return [
     '🔎 *Transaction Detail*',
     `Type: *${input.type}*`,
     `Status: *${input.status}*`,
     `Amount: *${input.amount} ${input.assetSymbol}*`,
+    // `||` (not `??`) so an empty-string hash also falls back to "pending".
     `Hash: _${input.txHash || 'pending'}_`,
   ].join('\n')
 }
 
+/**
+ * Current supply APYs, showing the first 5 rates. Rates are shown in the
+ * order given, so callers should sort best-first if that is the intent.
+ */
 export function formatProtocolRatesReply(input: {
   rates: ProtocolRateSummary[]
 }): string {
   const lines = input.rates.slice(0, 5).map((rate) => {
     return `• ${rate.protocolName} ${rate.assetSymbol}: *${(rate.supplyApy * 100).toFixed(2)}% APY*`
   })
 
+  // Unlike the other list formatters, there is no empty-state message here:
+  // with no rates, the reply is just the header followed by a blank line.
   return ['🏦 *Protocol Rates*', lines.join('\n')].join('\n')
 }
 
+/**
+ * Latest status of the automated agent: its most recent action, its current
+ * state, and when it was last updated. `updatedAt` is displayed as given, so
+ * pass an already human-readable string.
+ */
 export function formatAgentStatusReply(input: {
   status: string
   action: string
@@ -113,6 +177,11 @@ export function formatAgentStatusReply(input: {
   ].join('\n')
 }
 
+/**
+ * Confirmation sent after a deposit has been accepted into the queue (not
+ * yet settled on-chain). If no protocol was chosen, it is shown as "Auto",
+ * meaning the system picks one.
+ */
 export function formatDepositReply(input: {
   amount: number
   assetSymbol: string
@@ -121,11 +190,23 @@ export function formatDepositReply(input: {
   return [
     '✅ *Deposit queued*',
     `Amount: *${input.amount} ${input.assetSymbol}*`,
+    // Covers both undefined and null (and empty string) -> "Auto".
     `Protocol: *${input.protocolName || 'Auto'}*`,
     '_Your transaction is being processed._',
   ].join('\n')
 }
 
+/**
+ * Savings-goal progress. The reply depends on the goal's status:
+ *  - ACHIEVED / MISSED / CANCELLED: a short terminal message.
+ *  - anything else (active): progress, target date, required vs actual APY,
+ *    an on-track / behind / unreachable line, and a projected completion date
+ *    when one is available.
+ *
+ * Dates are expected as ISO strings and are trimmed to YYYY-MM-DD. Unlike the
+ * other formatters, `requiredApy` and `actualApy` are printed as given with a
+ * "%" sign (no ×100 conversion), so pass them as percentages, not fractions.
+ */
 export function formatGoalProgressReply(input: {
   status: string
   targetAmount: number
@@ -158,10 +239,12 @@ export function formatGoalProgressReply(input: {
   const lines = [
     '🎯 *Savings Goal Progress*',
     `Progress: *$${input.currentAmount.toFixed(2)}* of *$${input.targetAmount.toFixed(2)}*`,
+    // slice(0, 10) keeps just the date part of an ISO timestamp.
     `Target date: _${input.targetDate.slice(0, 10)}_`,
     `Required APY: *${input.requiredApy.toFixed(2)}%* | Actual APY: *${input.actualApy.toFixed(2)}%*`,
   ]
 
+  // Exactly one status line: not reachable, on track, or behind but reachable.
   if (!input.reachable) {
     lines.push('⚠️ Target not reachable within your risk tolerance.')
   } else if (input.onTrack) {
@@ -179,6 +262,11 @@ export function formatGoalProgressReply(input: {
   return lines.join('\n')
 }
 
+/**
+ * Confirmation sent after a withdrawal has been accepted into the queue.
+ * Mirrors formatDepositReply; the closing line tells the user a follow-up
+ * message will arrive once it settles.
+ */
 export function formatWithdrawReply(input: {
   amount: number
   assetSymbol: string