From fb618991eea97fb0c5006759c1a487b1cdccbfea Mon Sep 17 00:00:00 2001 From: timoheimonen Date: Fri, 25 Sep 2026 21:06:38 +0300 Subject: [PATCH] docs: update --- documents/CORE_TO_CORE_WHITEPAPER.md | 7 ++++--- documents/MANUAL.md | 3 ++- documents/PROJECT_STRUCTURE.md | 14 +++++++------- 3 files changed, 13 insertions(+), 11 deletions(-) diff --git a/documents/CORE_TO_CORE_WHITEPAPER.md b/documents/CORE_TO_CORE_WHITEPAPER.md index 4807fdf..c441527 100644 --- a/documents/CORE_TO_CORE_WHITEPAPER.md +++ b/documents/CORE_TO_CORE_WHITEPAPER.md @@ -500,15 +500,16 @@ validation of current affinity behavior, current scheduler placement, or current Relevant deterministic coverage includes: - work-plan targets derived from an excluded pilot and exact duration-window boundary classification; -- responder-work overflow rejection and deterministic timer/thread-startup failure states; +- responder-work overflow rejection before thread startup; - scenario-order rotation; - loop-record sample ranges that count only samples appended to the pooled population; - CLI short/long options, supported sweeps, duplicate sweep-key rejection, and run-count guards; - calibrated console/audit messages; - schema-2 work plans, loop records, completion fields, nullable values, and affinity interpretability. -The runner also has Apple Silicon integration coverage for a headline plus sample window and for a zero-sample internal -execution path. Those tests exercise the real two-thread ARM64 handoff and are hardware/runtime-sensitive. +The runner integration test checks responder cleanup when initiator startup fails. Executable CLI integration tests +exercise the real two-thread ARM64 handoff and check calibrated JSON output, sweep completion metadata, and stdout +routing. These paths are hardware/runtime-sensitive. For manual validation, prefer several loops and inspect: diff --git a/documents/MANUAL.md b/documents/MANUAL.md index b0d40c1..b91d764 100644 --- a/documents/MANUAL.md +++ b/documents/MANUAL.md @@ -88,8 +88,9 @@ Test and coverage targets: ```bash make test # deterministic unit suite make test-script-examples # current script-example JSON entry paths +make test-llm-verifier # independent LLM artifact contract and mutation tests make test-integration # real Apple Silicon/CLI workflows -make test-all # all GTest cases, then the focused script-example entry test +make test-all # all GTest cases, then script-example and LLM verifier tests make coverage-unit # isolated LLVM report under /tmp make coverage-all ``` diff --git a/documents/PROJECT_STRUCTURE.md b/documents/PROJECT_STRUCTURE.md index 0988842..0ca68d5 100644 --- a/documents/PROJECT_STRUCTURE.md +++ b/documents/PROJECT_STRUCTURE.md @@ -389,11 +389,11 @@ installed. All test `.cpp` and `.mm` files are picked up automatically by the Ma | `test_llm_memory_kernels.cpp` | `LlmMemoryKernelIntegrationTest` | Real contiguous/paged decode/prefill ARM64 descriptor/kernel scenarios, paged tails/lookups, full-prompt writes, exact partial tiled scans, checksum, padding, worker-count, and AAPCS64 coverage | | `test_llm_metal_backend.cpp` | `LlmMetalBackendTest`, `LlmMetalBackendIntegrationTest`, `LlmMetalBackendFailureInjectionIntegrationTest` | Pure capability, contiguous/paged segmentation, argument-buffer, decode/prefill ABIs, source provenance and prefill loop-order audit, phase-aware checksum/reason mapping, paged-prefill `N+2*M`, cyclic no-duplication ownership and per-threadgroup accounted-byte evidence, plus real-device selected-profile compilation, Tier-2 slots, private table/K/V initialization, exact-tail decode, full-prompt/tiled-prefix prefill, final-ordinal write/padding detection, interruption/failure cleanup, and idempotent release | | `test_buffer_manager.cpp` | `BufferManagerTest` | Pattern mapping policy, atomic allocation cleanup, initialized content, validation, and peak accounting | -| `test_benchmark_executor.cpp` | `BenchmarkExecutorTest` | Injected phase/chain failures, continuous latency sampling, and hardware executor contracts | +| `test_benchmark_executor.cpp` | `BenchmarkExecutorTest` | Injected phase-preparation failures, continuous latency sampling, and hardware executor contracts | | `test_benchmark_runner.cpp` | `BenchmarkStatisticsCollectorTest`, `BenchmarkRunnerTest` | Status-bearing aggregation, schema-3 retained snapshots, checkpointing, interruption, and runner exception/failure seams | | `test_benchmark_work_plan.cpp` | `BenchmarkWorkPlanTest` | Exact payload/access planning, calibration, cyclic order, seed derivation, and duration classification | -| `test_gpu_bandwidth.cpp` | `GpuBandwidthParserTest`, `GpuMemoryBudgetTest`, `GpuRunnerTest`, `GpuJsonTest` | Strict standalone parsing and raw output spelling, memory budgets, fake-backend calibration/execution/failure/interruption and file/stdout checkpoint semantics, counters, and schema-1 serialization | -| `test_gpu_work_plan.cpp` | `GpuWorkPlanTest`, `GpuTimedAccumulatorOracleTest` | GPU constants, cyclic order, seed domains, pass/payload caps, calibration, vector/tail/grid geometry, frozen identities, and timed-accumulator oracle behavior | +| `test_gpu_bandwidth.cpp` | `GpuBandwidthParserTest`, `GpuMemoryBudgetTest`, `GpuRunnerTest`, `GpuJsonTest` | Strict standalone parsing and output-option diagnostics, memory budgets, fake-backend calibration/execution/failure/interruption and file/stdout checkpoint semantics, counters, and schema-1 serialization | +| `test_gpu_work_plan.cpp` | `GpuWorkPlanTest`, `GpuTimedAccumulatorOracleTest` | Cyclic order, seed domains, pass/payload caps, calibration, vector/tail/grid geometry, frozen identities, and timed-accumulator oracle behavior | | `test_gpu_metal_backend.cpp` | `GpuMetalBackendIntegrationTest` | Real-Metal capability/runtime compile, private/shared tracked resources, read/write/copy/tail correctness, timestamps, validation, and byte readback | | `test_mode_selector.cpp` | `ModeSelectorTest` | All six primary modes and aliases, complete pairwise conflict order, duplicate ownership, and opaque flag-shaped output targets | | `test_hash_utils.cpp` | `HashUtilsTest` | CommonCrypto SHA-256 standard vectors and source-provenance helper behavior | @@ -404,14 +404,14 @@ installed. All test `.cpp` and `.mm` files are picked up automatically by the Ma | `test_json_output_session.cpp` | `JsonOutputTargetTest`, `JsonOutputSessionTest` | Exact sentinel/path classification, lazy checkpoint dispatch, atomic-file parity, stdout routing/restoration, and stream failure containment | | `test_json_utils.cpp` | `JsonUtilsTest`, `JsonFileWriterTest` | UTC timestamp/statistics and atomic writer success/failure contracts | | `test_output_printer.cpp` | `OutputPrinterTest`, `OutputPrinterCustomCacheUnitsTest` | Status-aware partial output, mode/cache composition, and custom-cache size-unit boundaries | -| `test_sweep_runner.cpp` | `SweepRunnerTest` | Current-only nested standard schema-3 completion classification, complete/partial/interrupted/failed attempt accounting, schema-1 envelopes, and file/lazy-stdout checkpoint behavior | +| `test_sweep_runner.cpp` | `SweepRunnerTest` | Current-only nested standard schema-3 completion classification, complete/partial/interrupted/failed attempt accounting, schema-1 envelopes, and checkpoint cadence/failure handling | | `test_sweep_utils.cpp` | `SweepUtilsTest` | Shared sweep parsing, empty-dimension behavior, and overflow-safe Cartesian counts | | `test_pattern_validation.cpp` | `PatternValidationTest` | Pattern benchmark parameter validation | | `test_pattern_benchmark.cpp` | `PatternBenchmarkTest` | Pattern execution and statistics | | `test_pattern_work_plan.cpp` | `PatternWorkPlanTest` | Strided phases, worker reduction, random partitions, exact payload work, and calibration | | `test_core_to_core_messages.cpp` | `CoreToCoreMessagesTest` | Core-to-core console message strings | | `test_core_to_core_cli.cpp` | `CoreToCoreCliTest` | Core-to-core CLI argument parsing | -| `test_core_to_core_runner.cpp` | `CoreToCoreRunnerTest` | Calibration, work planning, cyclic scenario order, deterministic failure seams, and real ARM64 integration paths | +| `test_core_to_core_runner.cpp` | `CoreToCoreRunnerTest` | Calibration, work planning, cyclic scenario order, pooled sample ranges, and initiator-startup failure cleanup | | `test_executable_cli.cpp` | `ExecutableCliIntegrationTest` | Executable-level CLI routing, current standard schema-3 direct/sweep payloads, split stdout/stderr capture, bounded child execution, sentinel artifact rules, unsupported GPU evidence, bounded contiguous/paged LLM file/stdout/checkpoint contracts, and pattern orchestration smoke coverage | | `test_standard_kernels.cpp` | `StandardKernelIntegrationTest`, `PatternKernelIntegrationTest` | Real ARM64 standard/pattern kernel ABI, tails, boundaries, checksums, and multi-worker execution | | `test_statistics.cpp` | `StatisticsTest` | Standard multi-loop summary composition, mode filtering, loop/sample population separation, and rendered values | @@ -419,11 +419,11 @@ installed. All test `.cpp` and `.mm` files are picked up automatically by the Ma | `test_statistics_renderer.cpp` | `StatisticsRendererTest` | Shared console-summary ordering, precision, indentation, and diagnostics | | `test_timer.cpp` | `HighResTimerTest` | Deterministic Mach-time conversion, original boundary snapshots, creation failures, seconds/nanoseconds results, and tick wraparound | | `test_system_info.cpp` | `BenchmarkQosTest`, `SystemInfoTest`, `SystemInfoIntegrationTest` | Deterministic QoS and system-information success/failure contracts plus one real-provider integration smoke | -| `test_tlb_chain.cpp` | `TlbChainTest` | Spread/packed planning, every traversal policy, explicit corruption statuses, and one ASM smoke | +| `test_tlb_chain.cpp` | `TlbChainTest` | Spread/packed planning, every traversal policy, and explicit corruption statuses | | `test_tlb_measurement_scheduler.cpp` | `TlbMeasurementSchedulerTest` | Seeded balance, stop/error boundaries, callback contracts, convergence, and exact pass accounting | | `test_tlb_runtime_policy.cpp` | `TlbRuntimePolicyTest` | Runtime profiles, calibration, convergence, memory budgets, and work estimates | | `test_tlb_sweep_planner.cpp` | `TlbSweepPlannerTest` | Page-aligned base/refinement planning, stride bounds, deduplication, and source tracking | -| `test_utils.cpp` | `SeedUtilsTest`, `ProgressSpinnerTest`, `UtilsTest` | Seed-provider behavior, TTY-gated spinner rendering/cleanup, and worker-thread joining | +| `test_utils.cpp` | `SeedUtilsTest`, `ProgressSpinnerTest` | Zero-seed fallback and enabled/disabled spinner rendering/cleanup | ### tests/fixtures/ — Script-example and LLM verifier input fixtures