Skip to content

JVNAUTOSCI-2561 narrow replay Gmail blocker oracle - #291

Merged
witbrock merged 1 commit into
mainfrom
jvnautosci-2561-replay-gmail-blocker-oracle
Jul 3, 2026
Merged

witbrock merged 1 commit into
mainfrom
jvnautosci-2561-replay-gmail-blocker-oracle

Conversation

@witbrock

@witbrock witbrock commented Jul 3, 2026

Copy link
Copy Markdown
Member

Summary

  • narrows the replay harness Gmail OAuth/profile blocker oracle so it does not scan all task evidence as one text blob
  • ignores non-evidence LLM input/request surfaces such as llm_request, context_messages, and prompt_preview
  • avoids treating identifier fields such as contract_id, fact_id, and workflow_id as Gmail failure context
  • adds durable AGENTS guidance requiring path analysis for replay pass, fail, timeout, and typed-blocker outcomes

Root Cause

The 2560 validation replay completed correctly, but the harness classified the artifact as blocked because unrelated evidence leaves were combined: gmail_arxiv_* identifiers supplied gmail, while unrelated workflow/profile or LLM prompt text supplied auth/profile/token language.

Validation

  • .venv/bin/python -m pytest tests/test_run_authenticated_browser_workflow_replay.py -q -> 18 passed
  • .venv/bin/python -m py_compile scripts/run_authenticated_browser_workflow_replay.py -> passed
  • git diff --check -> passed
  • reclassified the real 2560 artifact: old blocker gmail_oauth_or_profile_blocker; new verdict pass, blocker null

Stack

This is stacked on PR #290 / jvnautosci-2560-capability-index-red-flag and should merge after 2560.

@witbrock
witbrock changed the base branch from jvnautosci-2560-capability-index-red-flag to main July 3, 2026 17:01
@witbrock
witbrock marked this pull request as ready for review July 3, 2026 17:01
@witbrock
witbrock merged commit eb6e4e2 into main Jul 3, 2026
@witbrock
witbrock deleted the jvnautosci-2561-replay-gmail-blocker-oracle branch July 28, 2026 12:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant