Repository navigation
Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces a disposable, non-production “A2” thin read-only turn candidate plus a focused test suite to freeze the experimental condition for a time-bounded adaptive read experiment (elapsed-time budget, tool-parallelism within a batch, and tool-free final synthesis).
Changes:
- Adds
scripts/try_thin_readonly_turn.py: a standalone A2 candidate runner using the OpenAI Responses API with a frozen read-only tool palette and elapsed-time termination (research deadline + final synthesis reserve). - Adds
tests/test_thin_readonly_turn.py: tests for protocol boundaries (read-only enforcement, provider call-id correlation), elapsed-time budgeting, parallel within-batch tool execution, and late-answer marking.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| scripts/try_thin_readonly_turn.py | Implements the time-bounded adaptive read-only A2 candidate, parallel tool execution, transcript emission, and read-only boundary enforcement. |
| tests/test_thin_readonly_turn.py | Adds behavioural tests covering the candidate’s time-budget semantics, tool execution behaviour, and protocol/guardrail handling. |
| if phase == "research": | ||
| if text: | ||
| transcript["last_partial_text"] = text | ||
| continue | ||
| transcript.update( |
| executor = ThreadPoolExecutor( | ||
| max_workers=len(prepared), | ||
| thread_name_prefix="a2-read", | ||
| ) |
| remaining_research_seconds = max(0.0, research_deadline - clock()) | ||
| completed_futures, _ = wait( | ||
| [future for _, _, _, future in pending], | ||
| timeout=remaining_research_seconds, | ||
| ) |
|
Retained-work reconciliation, 17 September 2026: closing this completed experimental freeze PR without merging or deleting its branch. Frozen commit The authoritative native task (legacy alias JVNAUTOSCI-2598) is completed. H2/JVNAUTOSCI-2599 records Michael's explicit REVISE decision in source comment 36625: A2 is complete as an experiment, keep PR #308 frozen as evidence, do not merge/cut over, and proceed to revised A3 under JVNAUTOSCI-2600. The retained original session confirms later successor experiments; A2 is not unfinished production work. This closes only the stale open PR status under Michael's current retained-work disposition instruction. The original description, candidate, six-run evidence and negative outcomes remain preserved; no production code or runtime has changed, and this does not authorise or revive any later rejected prototype. |
Why
A1 showed that its fixed two-tool-batch ceiling directly caused all three thin-candidate failures, while current Von produced one substantive answer from three cases in 52 minutes. A2 tests the more general hypothesis: ordinary read-only administrative and scientific work should leave route selection and recovery to the model, bounded by trusted identity, read-only capability and elapsed time rather than counts of model calls, tool batches, retries or routes.
A1 commit
207178f0and draft PR #307 remain immutable evidence. This branch descends from that frozen condition but defines a new candidate and a fresh experiment.What changed
scripts/try_thin_readonly_turn.pycandidate and its case-independent tests; no production runtime file or route is changed.tool_choice="none"for final synthesis.The tests cover capability/protocol boundaries and observable outcomes. They do not prescribe a tool order, route, workflow identity, intent class or answer wording.
Frozen comparison protocol
After this commit/PR freeze, Michael supplies three fresh unseen low-risk read cases. Each exact case is submitted once to this candidate and once to current Von in a fresh chat: exactly six runs. Within-turn adaptation is allowed; after-result retry, rephrasing, tuning, replacement cases and a seventh run are not. The A1 cases and entities are forbidden from A2 prompts, fixtures and acceptance logic. Every outcome is retained.
The comparison baseline is
origin/mainat95c6a4caad1f50b0845f2e654b97522cc586426b.Validation
pytest -q tests/test_thin_readonly_turn.py— 12 passedtool_choice="none"completed with textStopping point
Do not merge, route production traffic, delete production controller code, tune against evaluation cases or begin A3. A2 stops after the six frozen runs with evidence ready for Michael's H2 decision.