Skip to content

[COVAL-5819] feat: add Python metric draft commands and audit request fields - #141

Merged
callumreid merged 8 commits into
mainfrom
chore/weekly-api-parity
Nov 2, 2026
Merged

callumreid merged 8 commits into
mainfrom
chore/weekly-api-parity

Conversation

@callumreid

@callumreid callumreid commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor
  • Add the six immutable Python metric draft/preview/publication commands and the persona interruption-instructions flag, with source-file input and agent next actions.
  • Preserve the newly published request fields on existing commands: review batch mode/size, uploaded-conversation stereo channel and set-once agent assignment, and atomic test-set creation with inline cases. Preserve omitted fields, explicit batch resets, channel zero/null, and empty inline-case arrays; reject invalid null case lists before sending.
  • Reconcile the November 2 catalog: 125 of 199 operations have first-class commands, with 74 explicitly reviewed route gaps. All 472 top-level fields on covered request operations are modeled. The request-field audit also rejects body calls it cannot parse. New exploratory audio-analysis/publication and transcript-comment resources are explicitly deferred with contract-specific reasons in the manifest.
  • Keep this implementation PR at 0.9.2. The 0.10.0 bump will be a separate manifest-only PR after merge, followed by package/Homebrew verification and the prepared user-doc updates.

Validation: five new captured-wire regressions failed against 97eed40 before the November correction; all six final new tests pass. Python automation (65 tests), fmt, clippy, Python lint/format, release-manifest checks and the live 472/472 audit pass. The full local Rust run passed 226/227 tests, with the unchanged 500ms advisory update-notification test timing out once under host load; that isolated test then passed. The updater source is byte-identical to main. Fresh final-head CI is required before merge.

JSON numeric bounds and nested inline-case validation remain enforced by the API; CLI flags constrain channel 0-1 and batch size 1-50. No live conversation, review-project, test-set, draft, preview or publication writes were performed during validation.

Terminal preview follow-up (COVAL-7661)

Preserves the external COMPLETED/SUCCEEDED and queued-admission-warning corrections. An unavailable Python metric evaluation terminates as SKIPPED in the backend; it now retains its diagnostic without suggesting either polling or publishing. The captured-wire regression fails on ba2b63a before this correction and passes after it. All five Python metric CLI tests pass.

Local validation at 112ad1f: 9 unit tests passed; 219 of 220 integration tests passed in the full run. The unchanged 500ms update-notification advisory timed out under the full run and passed in isolation; no timeout or assertion was weakened. Clippy with warnings denied, formatting, and whitespace checks pass. Exact-head GitHub CI remains the required final source gate. This changes no database queries.

Final validation at 1104eb3 (additive merge of main's approved permission/Gate workflows; no source rewrite): all 11 GitHub checks passed, including the real Switchboard Gate from GitHub Actions app15368, all five platform builds, and CodeQL. CI run https://github.com/coval-ai/cli/actions/runs/41542332213 passed 67 automation tests, 9 Rust unit tests and all 220 integration tests. Local advisory timeout limitation above is retained for transparency; exact-head CI passed it. Existing human changes-requested review remains an independent merge gate.

callumreid and others added 4 commits October 26, 2026 10:10
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@callumreid callumreid changed the title chore(cli): reconcile weekly API parity [COVAL-5819] feat: add Python metric draft commands and audit request fields Oct 26, 2026

@borgesius borgesius left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes for the successful-preview status mismatch, which leaves the agent workflow polling after completion. The separate note about polling during uncertain admission is nonblocking. The existing Rust suite passes; correcting the success fixture to the API response status reproduces the mismatch.

Comment thread src/commands/python_metric_drafts.rs Resolved
Comment thread src/commands/python_metric_drafts.rs Resolved
@callumreid
callumreid requested a review from borgesius November 2, 2026 22:27
@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown

The weekly refresh left this branch unchanged because it has commits not made by the bot.
Merge this PR, or rebase it onto main and regenerate api-coverage-report.md, so the next run can pick up new API changes.

Run: https://github.com/coval-ai/cli/actions/runs/41541116967

@borgesius borgesius left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous preview-state findings are fixed on this head: COMPLETED offers publication, queued admission warnings retain polling, and SKIPPED is terminal with its diagnostic preserved. All 229 Rust tests and 67 automation tests pass locally. Looks good.

@callumreid
callumreid merged commit 1ed8849 into main Nov 2, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants