Skip to content

feat(capacity_planning): capacity planning with historical usage trending (closes #127) - #174

Closed
ranjeet150 wants to merge 1 commit into
VeriNode-Labs:mainfrom
ranjeet150:feat/capacity-planning-usage-trending
Closed

ranjeet150 wants to merge 1 commit into
VeriNode-Labs:mainfrom
ranjeet150:feat/capacity-planning-usage-trending

Conversation

@ranjeet150

Copy link
Copy Markdown

Summary

Resolves #127: Capacity Planning with Historical Usage Trending.

This PR introduces a deterministic, pure-Rust capacity planning and historical usage trending subsystem affecting all services across the VeriNode architecture (, , , , , ).

Technical Invariants & Bounds (Derived from Issue #127)

  • Critical-Path Latency Target: CRITICAL_PATH_P99_MS = 100 ms.
  • Availability Target: AVAILABILITY_TARGET_BPS = 9_999 (99.99% uptime objective).
  • Canary Release Success Rate: CANARY_SUCCESS_TARGET_BPS = 9_999 (99.99% success rate).
  • Security Review Requirement: Mandatory security review sign-off (security_review_passed == true) enforced by CapacityCanaryAnalysis::passes_release_gate().
  • Reaction Cooldown: Enforces a minimum 30-second cooldown window (DEFAULT_REACTION_COOLDOWN_SECS = 30) between consecutive sizing actions.
  • Target Capacity Headroom: Maintains a 20% headroom objective (DEFAULT_TARGET_HEADROOM_BPS = 2_000).

Architecture & Implementation Details

1. Multi-Dimensional Telemetry & Historical Ring Buffer (src/capacity_planning/mod.rs)

  • ResourceMetrics: Tracks 5 resource dimensions in basis points (0..10_000): CPU, memory, IOPS, network, and worker thread/connection saturation. The peak resource saturation (peak_saturation_bps) dictates the bottleneck.
  • UsageSample: Captures point-in-time metrics, P99 critical path latency, allocated units, and throughput.
  • HistoricalUsageBuffer: Bounded FIFO ring buffer retaining chronological telemetry snapshots (default 120 samples).

2. Statistical Trending & Runway Forecasting Engine (CapacityForecaster)

  • Linear Regression: Derives saturation growth slope (slope_bps_per_sec), hourly/daily growth rates, and goodness of fit (^2$).
  • Holt-Winters Double Exponential Smoothing: Smooths transient telemetry noise to project next-window saturation levels.
  • Capacity Runway Estimation: Evaluates remaining time (seconds) to reach warning threshold (80%), critical threshold (90%), and exhaustion (100%). Non-increasing trends yield infinite runway (None).

3. Automated Sizing & Health Classification

  • Health States: Classifies services into Healthy, Warning (utilization >= 80% or runway < 48h), Critical (utilization >= 90% or runway < 12h), or Exhausted (utilization = 100% or P99 latency breach > 100 ms).
  • Sizing Decisions: Generates ScaleUp (maintaining 20% headroom), ScaleDown (safe deprovisioning under <30% saturation), or EmergencyThrottle (traffic shedding under SLA/capacity saturation), strictly bounded by min_units (1) and max_units (256).

4. Blue-Green Canary Deployment Validation Gate (CapacityCanaryAnalysis)

Enforces all release gate criteria prior to promoting capacity changes:

  1. Security review sign-off completed.
  2. Observed uptime meeting or exceeding 99.99%.
  3. Critical-path P99 latency strictly < 100 ms.
  4. Request success rate meeting or exceeding 99.99%.

5. Multi-Service Registry & Operational Dashboard

  • CapacityPlanningRegistry: Tracks up to 256 services with thread-safe isolation, providing evaluate_service, evaluate_all, promote_canary_deployment, and dashboard_snapshot.
  • Aggregates cluster-wide saturation, units allocated, health status counts, and identifies the service with the shortest runway.

6. Documentation & CI Verification

  • Registered pub mod capacity_planning; in src/lib.rs.
  • Added capacity_planning_usage_trending_test entry in Cargo.toml.
  • Documented exported types, operational principles, and canary verification in CORE.md.

Verification & Test Results

  • Co-located Unit Tests: 20 test cases in src/capacity_planning/mod.rs passed.
  • Integration Tests: 29 test cases in tests/capacity_planning_usage_trending_test.rs passed.
  • Total New Tests: 49 tests (100% pass rate).
  • Clippy Check: cargo clippy --lib --tests -- -D warnings passed with 0 warnings.
  • Formatting Check: cargo fmt --all -- --check passed cleanly.
  • Storage Layout Check: python3 scripts/storage-layout-check.py passed with 0 collisions.

…ding (closes VeriNode-Labs#127)

What was changed:
- Added src/capacity_planning/mod.rs: new pure-Rust module implementing capacity
  planning with multi-dimensional resource tracking and historical usage trending
  as mandated by issue VeriNode-Labs#127:
    * ResourceMetrics: captures CPU, memory, IOPS, network, and worker saturation in
      basis points (0..10_000). Peak saturation dictates bottleneck resource.
    * UsageSample: point-in-time observation of capacity, critical-path P99 latency,
      allocated units, and throughput.
    * HistoricalUsageBuffer: fixed-capacity ring buffer (default 120 samples) for
      chronological telemetry collection.
    * CapacityForecaster: statistical forecasting engine computing linear regression
      slope, R-squared goodness of fit, Holt-Winters double exponential smoothing,
      and runway estimation to warning (80%), critical (90%), and exhaustion (100%).
    * CapacityPlanningConfig & Sizing Decisions: automated ScaleUp, ScaleDown, and
      EmergencyThrottle recommendations maintaining 20% headroom and enforcing
      30-second cooldown windows and 1..256 unit bounds.
    * CapacityCanaryAnalysis: blue-green canary gate strictly enforcing 99.99%
      availability (AVAILABILITY_TARGET_BPS = 9_999), critical-path P99 < 100 ms
      (CRITICAL_PATH_P99_MS = 100), 99.99% success rate, and mandatory security review.
    * CapacityPlanningRegistry: multi-service coordinator tracking up to 256 services
      with evaluate_all, promote_canary_deployment, and system-wide dashboard snapshots.
    * Co-located unit tests (20 cases) covering all core routines and invariants.

- Added tests/capacity_planning_usage_trending_test.rs: standalone integration test
  suite (29 cases) verifying technical invariants, telemetry ingestion, linear
  trending, Holt-Winters forecasting, runway calculations, health state transitions,
  automated sizing, canary validation gates, and multi-service registry isolation.

- Modified src/lib.rs: registered pub mod capacity_planning with comprehensive doc-comment.

- Modified Cargo.toml: registered [[test]] capacity_planning_usage_trending_test.

- Modified CORE.md: updated exported symbols table in Services section, added
  Capacity Planning and Historical Usage Trending operational guide and runbook,
  and registered test in Testing and CI suite.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Capacity Planning with Historical Usage Trending

2 participants