Skip to content

Added a tool-neutral effort field and stopped deriving Codex reasoning effort from the tier - #46

Merged
dzykovic merged 5 commits into
mainfrom
feature/neutral-effort
Oct 7, 2026
Merged

dzykovic merged 5 commits into
mainfrom
feature/neutral-effort

Conversation

@dzykovic

@dzykovic dzykovic commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Summary

tier meant two things in one adapter. Every tool mapped it to a model, and Codex alone also derived model_reasoning_effort from it (xhigh for frontier, high for heavy or no tier, medium, low). An author could not ask for a model and a reasoning effort independently, Claude Code's own effort: on subagents and skills had no source spelling, and the same tier behaved differently per tool.

Agents and skills now accept a tool-neutral effort: — low, medium, high, xhigh, max, ultra — rendered into each tool's native setting the way tier and access are, and tier selects the model only (decision 0015):

effort: Claude Code agents and skills Codex agents Copilot, Cursor, OpenCode, Antigravity, Pi agents shared .agents/skills/
low … max effort: <level> model_reasoning_effort = "<level>" nothing kept as written
ultra effort: max (nearest lower level) model_reasoning_effort = "ultra" nothing kept as written
absent or empty nothing nothing nothing removed
off the scale (hiigh, High) nothing, plus one WARNING: line naming the source nothing nothing removed

An unknown value warns and is ignored instead of failing sync, because packages arrive from other authors and one typo must not block every team that installed it. Copilot's and Cursor's own skill copies never carry the key. Per-model level availability is left to the tools (Claude Code falls back to the highest supported level at or below the one set).

Behavior change: a Codex agent without effort: no longer gets model_reasoning_effort, so Codex's own setting applies and frontier and heavy now render identical Codex agents, as they already did in Copilot. The CHANGELOG carries a ### Breaking check for projects that relied on the derived effort. Decision 0013 is marked superseded in part.

Shape: one mapping, map_effort_var in engine/lib/common.sh, is the only source; the Claude and Codex agent loops read effort through the existing frontmatter_index pass, the skill-copy pass rewrites effort: in place from a table built from that same function (copy_skill_bundle_dirs_for <tool>), and the frontmatter lint the engine already runs once over every source emits the warning. No new process per file; Cursor's agent transform strips the key. The package's own agents are unchanged.

Type of change

  • CLI lifecycle or package behavior
  • Sync engine or adapter
  • Rule, agent, or skill content
  • Documentation
  • Refactor with no behavior change

Public CLI check

  • Uses only the established groups (no command changes; init writes effort: into its manifest comment)
  • Keeps preview/apply behavior explicit for planned writes (unchanged)
  • Covers automatic project alignment and CI refusal through the existing suites
  • Keeps fresh-clone package restoration behind intelligence sync (unchanged)

Verification

Starting state for every step: an empty directory initialized with git init, outside any checkout of this repository; the CLI is this PR's head, run as bash <head>/cli/intelligence from a tree extracted from the head commit (for example git archive <head-sha> | tar -x -C <dir>); in the empty repository run init --targets claude,codex,copilot,opencode. Agents go in intelligence/agents/<name>.md with name, description and the frontmatter the step names; skills go in intelligence/skills/<name>/SKILL.md. Run sync after each step's change.

  1. Tier selects the model only. Add agents with tier: frontier, tier: heavy and no tier, none with effort:; run sync. Expected: no .codex/agents/*.toml contains model_reasoning_effort, including the package's intelligence-architect and intelligence-operator; models are unchanged (frontier renders gpt-6-astra in Codex and fable in Claude); no .claude/agents/*.md contains effort:.
  2. An explicit effort reaches the tools that have a field, and only those. Add an agent with tier: light and effort: xhigh; run sync. Expected: its .claude/agents/<name>.md has effort: xhigh and model: sonnet; its .codex/agents/<name>.toml has model_reasoning_effort = "xhigh" and model = "gpt-6-luna"; its Copilot and OpenCode agent files contain no effort key; sync exits 0 with no WARN line.
  3. A level a tool lacks maps to its nearest lower level. Add an agent with effort: ultra; run sync. Expected: its Claude file has effort: max; its Codex file has model_reasoning_effort = "ultra".
  4. A typo warns and does not block. Add an agent with effort: hiigh; run sync. Expected: exit 0; the output contains a WARN line naming the agent's source file and hiigh; neither its Claude nor its Codex file carries an effort.
  5. Skills follow the same rules. Add a skill with effort: ultra and a skill with effort: hiigh; run sync. Expected: for ultra, .claude/skills/<s>/SKILL.md has effort: max and .agents/skills/<s>/SKILL.md has effort: ultra; for hiigh, neither tree's SKILL.md contains effort:, and the output contains a WARN line naming the skill's source file and hiigh.
  6. An empty effort is an absent effort. Add an agent and a skill with effort: (no value), and another agent and skill with effort: ""; run sync. Expected: exit 0; no WARN line for them; no generated file for them contains effort: or model_reasoning_effort.

Automated coverage

bash cli/tests/verify.sh printed verify ok on this head (run through WSL on Windows, all scope: both lint gates and all 18 suites).

  • e2e-lifecycle, "every built-in adapter renders every model tier": tier-only agents (each tier and none) carry no model_reasoning_effort in Codex and no effort: in Claude, and the frontier and heavy Codex agents are identical apart from name and description. It previously pinned the tier-derived ladder, so it fails on the old engine.
  • e2e-lifecycle, "effort is written once and rendered where a tool has a field", across all seven built-in targets: explicit, ultra→max, quoted, duplicated (first wins) and CRLF agents in Claude and Codex; effort leaves the model alone; off-scale (hiigh, High) and empty values carry no key anywhere; Copilot, OpenCode, Antigravity and Cursor agents and Copilot and Cursor skill copies never carry one; a body line starting effort: survives; skills in .claude/skills and .agents/skills; exactly one WARNING: line per off-scale source and none for empty values, and that line again under sync --compact, on a full render and on the unchanged run that replays it. Against the previous engine, 49 of the block's assertions failed when measured before the --compact checks were added; those four checks need a warning the previous engine never prints.
  • unit-engine: an independent expected matrix of map_effort_var for every tool and level, off-scale and empty inputs, the table the awk pass receives, lint_frontmatter_files (warns once, first occurrence only, never for empty, CRLF-safe, frontmatter only), and copy_skill_bundle_dirs_for per tool, including that bundled references and bodies are untouched.

Versioned artifacts

  • v0.18.1 is published and main already carries the pending 0.19.0 (Made status --check decide .gitignore presence with the writer's CR-blind test #55); a feature and a breaking change fit that minor, so this PR adds its lines to the existing ## [0.19.0] and leaves ## [0.18.1] unchanged
  • engine/VERSION, every example's schema_version and exact @ainova-systems/sync pin, their locks, and this repository's manifest and lock are already 0.19.0 on main; this PR moves none of them
  • No [Unreleased] section and no release date
  • No stable release is published by this PR

Notes for reviewers

Copilot AI balanced review requested due to automatic review settings October 7, 2026 10:58
@dzykovic dzykovic added the ai:processing An agent is working on this pull request label Oct 7, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Symlinked skill files bypass effort transformation and can emit unsupported or forbidden effort metadata.

Review effort: Balanced
Findings: 1 Medium severity · 1 Low severity

Open (2)
What changed in this PR

Introduces tool-neutral effort: metadata, separating reasoning effort from model-tier selection.

Changes:

  • Maps effort independently for Claude, Codex, and shared skills.
  • Adds validation, adapter transformations, and comprehensive tests.
  • Promotes versioned artifacts to 0.19.0 and documents the breaking change.
File Description
packages/​sync/​skills/​intelligence-update-context/​references/​skills.md Documents skill effort authoring.
packages/​sync/​skills/​intelligence-update-context/​references/​agents.md Documents agent effort selection.
packages/​sync/​references/​onboarding-migration.md Adds reverse-migration guidance.
packages/​sync/​references/​conventions.md Defines neutral effort mappings.
packages/​sync/​references/​adapters.md Documents adapter APIs and behavior.
intelligence.yaml Updates schema and field guidance.
intelligence.lock Aligns root lock to 0.19.0.
examples/​with-remote-skills/​intelligence.yaml Updates example version pins.
examples/​platform-with-submodules/​intelligence.yaml Updates example version pins.
examples/​platform-with-submodules/​intelligence.lock Updates example lock.
examples/​go-api/​intelligence.yaml Updates example version pins.
examples/​go-api/​intelligence.lock Updates example lock.
examples/​go-api-with-pi-and-codex/​intelligence.yaml Updates example version pins.
examples/​go-api-with-pi-and-codex/​intelligence.lock Updates example lock.
examples/​go-api-with-opencode/​intelligence.yaml Updates example version pins.
examples/​go-api-with-opencode/​intelligence.lock Updates example lock.
examples/​go-api-with-antigravity/​intelligence.yaml Updates example version pins.
examples/​go-api-with-antigravity/​intelligence.lock Updates example lock.
examples/​dotnet-api-with-react-frontend/​intelligence.yaml Updates example version pins.
examples/​dotnet-api-with-react-frontend/​intelligence.lock Updates example lock.
examples/​cli-project/​intelligence.yaml Updates example version pins.
engine/​VERSION Promotes the engine to 0.19.0.
engine/​lib/​common.sh Adds effort mapping, linting, and skill rewriting.
engine/​adapters/​cursor.sh Removes unsupported effort metadata.
engine/​adapters/​copilot.sh Removes effort from copied skills.
engine/​adapters/​codex.sh Emits explicit Codex reasoning effort.
engine/​adapters/​claude.sh Emits mapped Claude effort.
engine/​adapters/​_template.sh Exposes the effort helper.
decisions/​0015-separate-reasoning-effort-from-the-model-tier.md Records the new architecture.
decisions/​0013-add-a-frontier-model-tier.md Marks earlier behavior superseded.
cli/​tests/​unit-engine.sh Tests mappings, linting, and skill copies.
cli/​tests/​e2e-lifecycle.sh Tests effort across built-in adapters.
cli/​commands/​init.sh Adds effort guidance to manifests.
CHANGELOG.md Records 0.19.0 and migration requirements.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread engine/lib/common.sh
Comment thread packages/sync/references/onboarding-migration.md Outdated
@dzykovic

dzykovic commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

Verification - PASS

head: b80ab70 - steps: PR body "## Verification" steps 1-6 (no qa_checks matched; docs/qa-checks.md absent) - env: local (empty git init'd dir outside any checkout; head tree extracted via git -C <workspace> archive b80ab70ac1f6c49d393f7db38a9ba627126abcde | tar -x; CLI run as bash <head-tree>/cli/intelligence)

# Step Source Verdict Observed
1 Tier selects the model only: agents with tier: frontier, tier: heavy, no tier, none with effort:; run sync PR pass Across all 11 .codex/agents/*.toml present (including intelligence-architect, intelligence-operator), none contain model_reasoning_effort; across all 11 .claude/agents/*.md, none contain effort:. Models unchanged: agent-frontier.toml has model = "gpt-6-astra", agent-frontier.md has model: fable.
2 Explicit effort reaches only the tools with a field: tier: light, effort: xhigh; run sync PR pass .claude/agents/agent-light-xhigh.md has effort: xhigh and model: sonnet. .codex/agents/agent-light-xhigh.toml has model_reasoning_effort = "xhigh" and model = "gpt-6-luna". .github/agents/agent-light-xhigh.agent.md and .opencode/agents/agent-light-xhigh.md contain no effort: key. Sync exited 0, no WARN line.
3 A level a tool lacks maps to its nearest lower level: effort: ultra; run sync PR pass .claude/agents/agent-ultra.md has effort: max. .codex/agents/agent-ultra.toml has model_reasoning_effort = "ultra".
4 A typo warns and does not block: effort: hiigh; run sync PR pass Exit 0. Output contains WARN: .../intelligence/agents/agent-hiigh.md:4 effort "hiigh" is not one of low, medium, high, xhigh, max, ultra — ignored.... Neither .claude/agents/agent-hiigh.md nor .codex/agents/agent-hiigh.toml carries an effort key.
5 Skills follow the same rules: skill with effort: ultra, skill with effort: hiigh; run sync PR pass .claude/skills/skill-ultra/SKILL.md has effort: max; .agents/skills/skill-ultra/SKILL.md has effort: ultra. Neither .claude/skills/skill-hiigh/SKILL.md nor .agents/skills/skill-hiigh/SKILL.md contains effort:; output contains a WARN line naming .../intelligence/skills/skill-hiigh/SKILL.md:4 and hiigh.
6 An empty effort is an absent effort: agent+skill with bare effort:, agent+skill with effort: ""; run sync PR pass Exit 0, no new WARN line for any of the four. None of .claude/agents/agent-empty-bare.md, .claude/agents/agent-empty-quoted.md, .codex/agents/agent-empty-bare.toml, .codex/agents/agent-empty-quoted.toml, .claude/skills/skill-empty-bare/SKILL.md, .claude/skills/skill-empty-quoted/SKILL.md, .agents/skills/skill-empty-bare/SKILL.md, .agents/skills/skill-empty-quoted/SKILL.md contains effort: or model_reasoning_effort.

No qa_checks entries matched (docs/qa-checks.md does not exist in this repository).

Negative probes beyond the declared steps: an off-scale value differing only by case (effort: High) warned and was ignored exactly like hiigh (case-sensitive match against the documented scale, the same example the PR body's own summary table uses); Copilot's generated skill copy of skill-ultra (an explicit-effort skill) carries no effort: key, confirming the per-tool field restriction also holds for a tool that never gets the field at all.

Failures

None.

@dzykovic dzykovic added the ai:verified Declared behavior verified on the reported head label Oct 7, 2026
@dzykovic

dzykovic commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

Code review - PASS-WITH-WARNINGS

head: b80ab70 - base: main - files 34, +533/-98

Critical

(none)

Warning

  • intelligence/rules/engine.md:80 - the "helpers built for this" batched-helper enumeration (and the _var-form list at :85-88) still names copy_skill_bundle_dirs and omits the new copy_skill_bundle_dirs_for that claude.sh, copilot.sh and cursor.sh now call directly in its place, and omits map_effort_var from the _var family it lists by example. This file's own scope is engine/**, and dev-context-engineering requires a convention's rule to update in the same commit as the code that changes it; packages/sync/references/adapters.md received the equivalent update, this internal rule did not.

Suggestion

(none)

@dzykovic dzykovic added the ai:reviewed Diff reviewed on the reported head label Oct 7, 2026
… the engine rule and effort in the symlinked-skill warning
@dzykovic dzykovic removed ai:verified Declared behavior verified on the reported head ai:reviewed Diff reviewed on the reported head labels Oct 7, 2026
@dzykovic

dzykovic commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

Verification - PASS

head: 6a5028e - steps: PR body "## Verification" steps 1-6 (no qa_checks matched; docs/qa-checks.md absent) - env: local (empty git init'd dir outside any checkout; head tree extracted via git -C <workspace> archive 6a5028ef97e9233657a1277061c51b1bf1b9cc00 | tar -x; CLI run as bash <head-tree>/cli/intelligence)

# Step Source Verdict Observed
1 Tier selects the model only: agents with tier: frontier, tier: heavy, no tier, none with effort:; run sync PR pass None of .codex/agents/*.toml (11 files, including intelligence-architect, intelligence-operator, plus the 3 new agents) contains model_reasoning_effort. Models unchanged: agent-frontier.toml has model = "gpt-6-astra", agent-frontier.md has model: fable; agent-heavy/agent-notier render model = "gpt-6-astra" in Codex. No .claude/agents/*.md contains effort:. Sync exit 0, no WARN.
2 Explicit effort reaches only the tools with a field: tier: light, effort: xhigh; run sync PR pass .claude/agents/agent-light-xhigh.md has effort: xhigh and model: sonnet. .codex/agents/agent-light-xhigh.toml has model_reasoning_effort = "xhigh" and model = "gpt-6-luna". .github/agents/agent-light-xhigh.agent.md and .opencode/agents/agent-light-xhigh.md contain only description/model (no effort: key; the grep hit was the word "effort" inside the fixture's own description text). Sync exit 0, no WARN.
3 A level a tool lacks maps to its nearest lower level: effort: ultra; run sync PR pass .claude/agents/agent-ultra.md has effort: max. .codex/agents/agent-ultra.toml has model_reasoning_effort = "ultra". Sync exit 0.
4 A typo warns and does not block: effort: hiigh; run sync PR pass Exit 0. Output: WARN: .../intelligence/agents/agent-hiigh.md:4 effort "hiigh" is not one of low, medium, high, xhigh, max, ultra — ignored.... Neither .claude/agents/agent-hiigh.md nor .codex/agents/agent-hiigh.toml carries an effort key.
5 Skills follow the same rules: skill with effort: ultra, skill with effort: hiigh; run sync PR pass .claude/skills/skill-ultra/SKILL.md has effort: max; .agents/skills/skill-ultra/SKILL.md has effort: ultra. Neither .claude/skills/skill-hiigh/SKILL.md nor .agents/skills/skill-hiigh/SKILL.md contains effort:; output carries a WARN line naming .../intelligence/skills/skill-hiigh/SKILL.md:4 and hiigh.
6 An empty effort is an absent effort: agent+skill with bare effort:, agent+skill with effort: ""; run sync PR pass Exit 0, no new WARN for any of the four. None of .claude/agents/agent-empty-bare.md, .claude/agents/agent-empty-quoted.md, .codex/agents/agent-empty-bare.toml, .codex/agents/agent-empty-quoted.toml, .claude/skills/skill-empty-bare/SKILL.md, .claude/skills/skill-empty-quoted/SKILL.md, .agents/skills/skill-empty-bare/SKILL.md, .agents/skills/skill-empty-quoted/SKILL.md contains an effort: or model_reasoning_effort key.

No qa_checks entries matched (docs/qa-checks.md does not exist in this repository; profile does not override qa_checks).

Negative probes beyond the declared steps: an off-scale value differing only by case (effort: High) warned and was ignored exactly like hiigh (case-sensitive match against the documented scale, the boundary step 4's wording implies); Copilot's generated skill copy of skill-ultra (an explicit-effort skill) carries no effort: key, confirming the per-tool field restriction also holds for a tool that never gets the field at all.

The one commit since the prior verified head (b80ab70a) touches only a WARN message string (engine/lib/common.sh:531, symlinked-skill note) and two docs files (intelligence/rules/engine.md, packages/sync/references/onboarding-migration.md); re-running all 6 steps against this head confirms no regression.

Failures

None.

@dzykovic dzykovic added the ai:verified Declared behavior verified on the reported head label Oct 7, 2026
@dzykovic

dzykovic commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

Code review - CLEAN

head: 6a5028e - base: main - files 35, +546/-110

Critical

(none)

Warning

(none)

Suggestion

(none)

@dzykovic dzykovic added the ai:reviewed Diff reviewed on the reported head label Oct 7, 2026
@dzykovic

dzykovic commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

Outcome - MANUAL

head: 6a5028e

@dzykovic dzykovic added ai:manual Needs an owner decision ai:processing An agent is working on this pull request and removed ai:processing An agent is working on this pull request ai:manual Needs an owner decision labels Oct 7, 2026
@dzykovic dzykovic removed ai:verified Declared behavior verified on the reported head ai:reviewed Diff reviewed on the reported head labels Oct 7, 2026
@dzykovic

dzykovic commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

Verification - PASS

head: bd3b6cb - steps: PR body "## Verification" steps 1-6 (no qa_checks matched; docs/qa-checks.md absent) - env: local (empty git init'd dir outside any checkout; head tree extracted via git -C <workspace> archive bd3b6cb450bb0ddf0700c29a211bd0cf0ff20127 | tar -x; CLI run as bash <head-tree>/cli/intelligence)

# Step Source Verdict Observed
1 Tier selects the model only: agents with tier: frontier, tier: heavy, no tier, none with effort:; run sync PR pass No .codex/agents/*.toml contains model_reasoning_effort; no .claude/agents/*.md contains effort: (checked via grep -l across both directories, including the package's intelligence-architect and intelligence-operator). Models unchanged: agent-frontier.toml has model = "gpt-6-astra", agent-frontier.md has model: fable.
2 Explicit effort reaches only the tools with a field: tier: light, effort: xhigh; run sync PR pass .claude/agents/agent-light-xhigh.md has effort: xhigh and model: sonnet. .codex/agents/agent-light-xhigh.toml has model_reasoning_effort = "xhigh" and model = "gpt-6-luna". .github/agents/agent-light-xhigh.agent.md and .opencode/agents/agent-light-xhigh.md contain no effort: key. Sync exited 0, no WARN/WARNING line.
3 A level a tool lacks maps to its nearest lower level: effort: ultra; run sync PR pass .claude/agents/agent-ultra.md has effort: max. .codex/agents/agent-ultra.toml has model_reasoning_effort = "ultra".
4 A typo warns and does not block: effort: hiigh; run sync PR pass Exit 0. Output contains WARNING: .../intelligence/agents/agent-hiigh.md:4 effort "hiigh" is not one of low, medium, high, xhigh, max, ultra — ignored.... Neither .claude/agents/agent-hiigh.md nor .codex/agents/agent-hiigh.toml carries an effort key.
5 Skills follow the same rules: skill with effort: ultra, skill with effort: hiigh; run sync PR pass .claude/skills/skill-ultra/SKILL.md has effort: max; .agents/skills/skill-ultra/SKILL.md has effort: ultra. Neither .claude/skills/skill-hiigh/SKILL.md nor .agents/skills/skill-hiigh/SKILL.md contains effort:; output contains a WARNING line naming .../intelligence/skills/skill-hiigh/SKILL.md:4 and hiigh.
6 An empty effort is an absent effort: agent+skill with bare effort:, agent+skill with effort: ""; run sync PR pass Exit 0, no new WARN/WARNING line for any of the four. None of .claude/agents/agent-empty-bare.md, .claude/agents/agent-empty-quoted.md, .codex/agents/agent-empty-bare.toml, .codex/agents/agent-empty-quoted.toml, .claude/skills/skill-empty-bare/SKILL.md, .claude/skills/skill-empty-quoted/SKILL.md, .agents/skills/skill-empty-bare/SKILL.md, .agents/skills/skill-empty-quoted/SKILL.md contains effort: or model_reasoning_effort.

No qa_checks entries matched (docs/qa-checks.md does not exist in this repository).

Negative probes beyond the declared steps, targeted at this head's own delta over the last-verified SHA (6a5028e/b80ab70, where the unknown-effort warning was still the indented WARN: form): sync --compact on a project carrying the agent-hiigh.md fixture prints the unindented WARNING: .../agent-hiigh.md:4 effort "hiigh" is not one of... line (it is suppressed in no other way), confirming compact mode now surfaces it; intelligence init on a fresh directory pre-populated with an off-scale-effort agent prints the same WARNING: line during its initial sync. By contrast, an unrelated lint note this commit did not touch (a leading tab in frontmatter, WARN: .../agent-leadingtab.md:4 leading tab in frontmatter...) appears in a plain sync but is correctly absent from sync --compact output, confirming the fix promoted only the effort warning and left the other indented lint notes suppressed under --compact as before.

Failures

None.

@dzykovic dzykovic added the ai:verified Declared behavior verified on the reported head label Oct 7, 2026
@dzykovic

dzykovic commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

Code review - CLEAN

head: bd3b6cb - base: main - files 35, +562/-108

Critical

(none)

Warning

(none)

Suggestion

  • cli/tests/e2e-lifecycle.sh:753 - the "tools without a per-agent effort field never receive the key" loop checks .github/agents, .opencode/agents, .agents/agents (Antigravity) and .cursor/agents, but not .pi/prompts/intelligence-agent-<name>.md. Not a correctness gap - sync_pi_agents (engine/adapters/pi.sh:214) rebuilds the frontmatter from scratch like the other no-field tools, so a leak is architecturally impossible - but Pi is the one built-in target this test block doesn't assert against, for coverage symmetry with the other five.

@dzykovic dzykovic added the ai:reviewed Diff reviewed on the reported head label Oct 7, 2026
@dzykovic dzykovic removed ai:verified Declared behavior verified on the reported head ai:reviewed Diff reviewed on the reported head labels Oct 7, 2026
@dzykovic

dzykovic commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

Verification - PASS

head: fd34d94 - steps: PR body "## Verification" steps 1-6 (no qa_checks matched; docs/qa-checks.md absent) - env: local (empty git init'd dir outside any checkout; head tree extracted via git -C <workspace> archive fd34d94b64dac66ea0ec2fab85a738f06526298a | tar -x; CLI run as bash <head-tree>/cli/intelligence)

# Step Source Verdict Observed
1 Tier selects the model only: agents with tier: frontier, tier: heavy, no tier, none with effort:; run sync PR pass No .codex/agents/*.toml contains model_reasoning_effort (checked across all 5 generated files, including the package's intelligence-architect and intelligence-operator); no .claude/agents/*.md contains effort:. Models unchanged: agent-frontier.toml has model = "gpt-6-astra", agent-frontier.md has model: fable. Sync exit 0, no WARN.
2 Explicit effort reaches only the tools with a field: tier: light, effort: xhigh; run sync PR pass .claude/agents/agent-light-xhigh.md has effort: xhigh and model: sonnet. .codex/agents/agent-light-xhigh.toml has model_reasoning_effort = "xhigh" and model = "gpt-6-luna". .github/agents/agent-light-xhigh.agent.md and .opencode/agents/agent-light-xhigh.md contain no effort: key (confirmed by exact line match; an earlier substring grep hit was only the word "effort" inside the fixture's own description text). Sync exit 0, no WARN.
3 A level a tool lacks maps to its nearest lower level: effort: ultra; run sync PR pass .claude/agents/agent-ultra.md has effort: max. .codex/agents/agent-ultra.toml has model_reasoning_effort = "ultra".
4 A typo warns and does not block: effort: hiigh; run sync PR pass Exit 0. Output contains WARNING: .../intelligence/agents/agent-hiigh.md:4 effort "hiigh" is not one of low, medium, high, xhigh, max, ultra — ignored.... Neither .claude/agents/agent-hiigh.md nor .codex/agents/agent-hiigh.toml carries an effort key.
5 Skills follow the same rules: skill with effort: ultra, skill with effort: hiigh; run sync PR pass .claude/skills/skill-ultra/SKILL.md has effort: max; .agents/skills/skill-ultra/SKILL.md has effort: ultra. Neither .claude/skills/skill-hiigh/SKILL.md nor .agents/skills/skill-hiigh/SKILL.md contains effort:; output contains a WARNING line naming .../intelligence/skills/skill-hiigh/SKILL.md:4 and hiigh.
6 An empty effort is an absent effort: agent+skill with bare effort:, agent+skill with effort: ""; run sync PR pass Exit 0, no new WARN line beyond the two already present from steps 4-5 (diffed verbatim). None of .claude/agents/agent-empty-bare.md, .claude/agents/agent-empty-quoted.md, .codex/agents/agent-empty-bare.toml, .codex/agents/agent-empty-quoted.toml, .claude/skills/skill-empty-bare/SKILL.md, .claude/skills/skill-empty-quoted/SKILL.md, .agents/skills/skill-empty-bare/SKILL.md, .agents/skills/skill-empty-quoted/SKILL.md contains effort: or model_reasoning_effort.

No qa_checks entries matched (docs/qa-checks.md does not exist in this repository; profile does not override qa_checks).

Negative probes beyond the declared steps: an off-scale value differing only by case (effort: High) warned and was ignored exactly like hiigh, confirming the match against the documented scale is case-sensitive; neither the Claude nor the Codex file for that agent carried an effort key. agent-frontier.toml and agent-heavy.toml are byte-identical apart from name =/description =/body, confirming the two tiers render identical Codex agents now that neither derives model_reasoning_effort (the behavior change the PR declares).

This head is a merge of main into the feature branch since the last-verified head (bd3b6cb4); the merge brings in an unrelated PR (#55, status --check/.gitignore) and touches none of steps 1-6's files or behavior. All 6 steps were re-executed in full against this exact head rather than inferred from the prior report.

Failures

None.

@dzykovic dzykovic added the ai:verified Declared behavior verified on the reported head label Oct 7, 2026
@dzykovic

dzykovic commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

Code review - CLEAN

head: fd34d94 - base: main - files 19, +520/-68

Critical

(none)

Warning

(none)

Suggestion

  • cli/tests/e2e-lifecycle.sh:820 - the "tools without a per-agent effort field never receive the key" loop asserts .github/agents, .opencode/agents, .agents/agents (Antigravity) and .cursor/agents, but not .pi/prompts/intelligence-agent-<name>.md. Not a correctness gap - sync_pi_agents (engine/adapters/pi.sh) rebuilds frontmatter from scratch like the other no-field tools, so a leak there is architecturally impossible - but Pi is the one built-in target this block skips, for coverage symmetry with the other five. Carried forward unaddressed from the bd3b6cb review round on this same PR.

@dzykovic dzykovic added ai:reviewed Diff reviewed on the reported head ai:completed Agent work completed; required factors are fresh and removed ai:processing An agent is working on this pull request labels Oct 7, 2026
@dzykovic
dzykovic merged commit 8a8908a into main Oct 7, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai:completed Agent work completed; required factors are fresh ai:reviewed Diff reviewed on the reported head ai:verified Declared behavior verified on the reported head

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants