Tune non-generative CPU session pools

[Akshay Sonawane (apsonawane)]

Summary

• add bounded non-spinning CLM CPU component pools with 32 intra-op and 1 inter-op threads
• tune KEV-4B CPU component pools from 16 to 24 intra-op threads on a physical EPYC NUMA node
• document hardware-specific guidance and measured latency/accuracy results
• leave CUDA and WebGPU recipe policies unchanged

Results

AMD EPYC 7V12:

CLM FP32 CPU

• p50: 2,893 ms -> 1,145 ms (60.4% lower)
• p95: 3,772 ms -> 1,420 ms (62.4% lower)
• route accuracy: unchanged at 10/10

KEV-4B FP32 CPU

Combined with the stacked runtime CPU grouping change:

• p50: 4,409 ms -> 3,405 ms (22.8% lower)
• p95: 4,686 ms -> 3,647 ms (22.2% lower)
• policy checks: unchanged at 32/40

Validation

• recipe JSON parses successfully
• exporter script compiles
• matched CPU qualification used bounded pools with thread spinning disabled

Stacked on the existing non-generative Mobius recipes branch.

[Copilot]

Copilot review overview

🟡 Changes recommended

The CLM options are ignored by the exporter, and the KEV thread-count change breaks an existing test expectation.

Review effort: Balanced
Findings: 2

Open (2)

• Apply CLM component session options during export · New
• Update stale test expectation for 24 threads · New

What changed in this PR

Tunes CPU thread pools for CLM and KEV-4B and documents benchmark results.

Changes:

• Adds a 32-thread CLM CPU policy.
• Raises KEV-4B CPU threads from 16 to 24.
• Documents hardware guidance and latency results.

File	Description
kev-4b/​cpu/​README.md	Documents KEV tuning results.
kev-4b/​cpu/​kev-4b_cpu_fp32.json	Sets 24 intra-op threads.
Contrastive-LM-CLM-v0.1-8B/​cpu/​README.md	Documents CLM tuning results.
Contrastive-LM-CLM-v0.1-8B/​cpu/​CLM-v0.1-8B_cpu_fp32.json	Declares CLM session options.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.