nightly: install deps into a venv (fix PEP 668 install failure)#939
Conversation
`make nightly` ran `pip install` directly, which fails on PEP 668 externally-managed systems (Debian/Python 3.12 on the nightly server) with "This environment is externally managed". Install eval-live into a local venv instead, and run nightly_bench.py with that venv's python so its `import eval_live` resolves. The venv is gitignored. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## nightly #939 +/- ##
===========================================
- Coverage 86.65% 86.54% -0.12%
===========================================
Files 90 89 -1
Lines 27057 26804 -253
===========================================
- Hits 23446 23197 -249
+ Misses 3611 3607 -4 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Merging this PR will degrade performance by 7.48%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | Simulation | rust_rule_tableaction_hot_path[facts50000_funcs200] |
24.7 ms | 31.8 ms | -22.42% |
| ⚡ | Simulation | rust_rule_insert_loop[ops1000_funcs200] |
641.8 µs | 581.7 µs | +10.33% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing oflatt-claude:nightly-fix-venv (9f329d4) with nightly (f99482a)
Footnotes
-
233 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
|
Pushed a second fix. The venv change above got past the Cause: the crate is Fix ( Verified locally on a box with the same setup (system PATH lacks |
Problem
The nightly run failed to install after #935 was ported to render its report with eval-live:
The
make nightlytarget ranpip installagainst the system interpreter, which Python 3.12 on the Debian-based nightly server rejects under PEP 668.Fix
Install the dependency into a local virtualenv (
nightly/.venv) and runnightly_bench.pywith that venv'spython. The script doesimport eval_livein-process, so it must run under the interpreter the dependency was installed into.nightly: python3 -m venv nightly/.venv nightly/.venv/bin/pip install -q -r scripts/requirements.txt nightly/.venv/bin/python scripts/nightly_bench.pynightly/.venv/is gitignored.Verification
On Python 3.12 (same as the nightly server),
python3 -m venv+pip install -r scripts/requirements.txtinto the venv succeeds andimport eval_liveresolves from the venv. (The server needs thepython3-venvpackage, which providesensurepip— standard on these machines.)🤖 Generated with Claude Code