Skip to content

Add a reproducible Strands Evals state-review companion - #23

Merged
noteflowai merged 1 commit into
mainfrom
codex/strands-state-review-20260916
Sep 16, 2026
Merged

noteflowai merged 1 commit into
mainfrom
codex/strands-state-review-20260916

Conversation

@noteflowai

Copy link
Copy Markdown
Owner

Developers using Strands Evals need a concrete way to connect an improved aggregate to the state rules that changed. Add an optional companion recipe using the native 1.3.0 SDK and four existing saved Docker cases: its equally weighted two-rule mean improves from 0.75 to 0.875, while the notes rule regresses.

  • Preserve native reports and align verdicts by case/evaluator identity; reject missing, duplicate or unmatched inventories. Keep the different original EvalArc five-dimension score explicitly separate.
  • Add English/Chinese instructions, a tested dependency lock and a focused community discussion draft. The recipe accepts only the fixed published fixtures; it is not an arbitrary report importer.
  • Add an independent CI job using the real SDK with network attempts rejected. It checks serialization, forward/reverse regressions, report inventories, committed report equality and output preservation, and gates both deployments.

Local validation: native SDK check passed with zero network attempts; lint, formatting, local documentation links and official HF card validation passed. No model, AgentCore or new agent execution was performed; the Python release remains 0.12.1.

@noteflowai
noteflowai merged commit 7950257 into main Sep 16, 2026
13 checks passed
@noteflowai
noteflowai deleted the codex/strands-state-review-20260916 branch September 16, 2026 02:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant