fix(graph): eliminate Kùzu memory exhaustion and teardown race condition

[Yasou13]

Overview

Resolves production Profile B VM runtime failures where Kùzu graph retrieval causes memory exhaustion (OOM / high virtual memory) and teardown segmentation faults (use-after-free race condition) under timeout/cancellation.

Root Cause Analysis

1. Unbounded Cypher & Huge Parameter Lists (Memory Exhaustion / OOM):
   • Previous multi-hop queries (q2_undirected, q3_undirected) constructed single-pass monolithic Cypher queries passing tens of thousands of allowed_assertion_ids and allowed_entity_ids directly into Cypher IN $allowed_... predicates.
   • Kùzu's C++ query planner suffered exponential Cartesian joins and memory ballooning (~17GB mmap/virtual allocation), leading to out-of-memory crashes.
2. Asynchronous Teardown Race Condition (Segmentation Fault / SIGSEGV):
   • When asyncio.wait_for timed out, the Python coroutine was cancelled immediately while the native Kùzu C++ query was still executing in a background worker thread.
   • Subsequent close() teardown released the database and connection (_sync_close), causing the still-running native worker thread to access deallocated memory (use-after-free), triggering SIGSEGV.

Key Changes

1. Seed-Local Bounded Traversal (mesa_storage/kuzu_provider.py):
   • Replaced monolithic multi-hop Cypher queries with staged, seed-local 1-hop expansions (_step_expand).
   • Removed huge ID collections from Cypher queries; scope filtering (tenant/dataset/scope) is performed safely in Python.
   • Enforced strict frontier bounds: _MAX_FRONTIER_NODES = 64 and _MAX_FANOUT_PER_NODE = 32, preventing combinatorial explosion while preserving graph path discovery and ranking contracts.
2. Native Query Lifecycle & Teardown Barrier (mesa_storage/kuzu_provider.py):
   • Introduced _in_flight_native_query() context manager, _drain_event, and _shutting_down flag.
   • Graceful shutdown in close() rejects new queries and waits with a bounded timeout (_drain_event.wait(timeout)) for in-flight native C++ executions to complete before executing _sync_close() under _conn_lock.
3. Comprehensive Runtime & Hardening Tests (tests/test_v4_kuzu_runtime_closure.py):
   • test_real_scale_3_hop_legal_graph_retrieval: Validates 3-hop traversal on realistic legal graph density under strict memory/time bounds.
   • test_teardown_segfault_race_prevention: Recreates async timeout cancellation followed immediately by close() to guarantee no segfault.
   • test_shutdown_barrier_rejects_new_queries: Verifies query rejection during shutdown.
   • test_kuzu_resource_exhaustion_typed_and_non_fatal: Verifies non-fatal typed error handling.
   • test_server_survival_after_graph_outage: Verifies server resilience and fallback readiness.

Verification

• tests/test_v4_kuzu_runtime_closure.py: 5 passed in 15.6s
• tests/test_v4_graph_retrieval_hardening.py: 16 passed in 9.5s
• tests/test_v4_certification_contracts.py: 5 passed in 2.8s
• tests/test_v4_hardening_final_verification.py: 9 passed in 1.9s
• Python syntax (py_compile) and linting (ruff): Clean
• Note: Certification thresholds and contracts remain untouched. As requested, this PR is opened without merging.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.