Repository navigation
Audit releases against a generated baseline for knowledge changes - #205
Conversation
The release audit enforced every issue against the committed release_baseline.yaml, so a knowledge change made right after an upstream release failed until someone advanced the adapter baseline in a separate PR. The knowledge service also could not load the auditor from an installed wheel: it lived under tools/, which the wheel does not ship. - Move the auditor to the adapter plugin adapters/vllm_omni/release_audit.py (packaged runtime data); tools/vllm_omni_release_audit.py re-exports it, so the CLI, CI workflow, imupdate skill and existing tests are unchanged. - Add --baseline-source generated (committed stays the default): the baseline is built in memory for the audited SHA pair. Only knowledge-document issues are enforced (stale_knowledge_source, knowledge_metadata_error, owner_document_missing). Pin and inventory drift, unrouted paths, stale source-map patterns and doc pins go to a `reconciliation` list, together with committed_baseline_behind / committed_inventory_differs: that list is what the adapter-baseline companion PR must fix. - Add knowledge_service.release_audit, a repo-neutral loader for a plugin's audit_for_knowledge() entry. It refuses absolute, escaping, missing or non-.py plugin paths. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2511f3447a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if not isinstance(baseline, dict) or not isinstance(baseline.get("upstream"), dict) \ | ||
| or baseline["upstream"].get("audited_sha") != shas[1]: | ||
| raise bad("generated_baseline.upstream.audited_sha must equal upstream.to.sha") |
There was a problem hiding this comment.
When a plugin omits generated_baseline.upstream.previous_audited_sha or returns a stale value, this validation still accepts the report and may mark it clean because it checks only audited_sha. The accepted baseline therefore does not necessarily represent the reported from/to pair and can give reconciliation or a companion baseline update the wrong starting commit; require previous_audited_sha to be a valid SHA equal to upstream.from.sha.
Useful? React with 👍 / 👎.
Third PR of the knowledge-base redesign. It lets the knowledge service and the knowledge gate audit an upstream release without waiting for the adapter baseline to be advanced.
What changes
adapters/vllm_omni/release_audit.py, which ships in the wheel as runtime data.tools/vllm_omni_release_audit.pyre-exports it, so the CLI, CI workflow, imupdate skill and existing tests are unchanged.--baseline-source generatedmode (committedstays the default). The baseline is built in memory for the audited SHA pair.stale_knowledge_source,knowledge_metadata_error,owner_document_missing).reconciliation, never blocking: pin and inventory drift, unrouted paths, stale source-map patterns, doc pins,committed_baseline_behind/committed_previous_differs/committed_inventory_differs. This list is what the adapter-baseline companion PR must fix.knowledge_service.release_audit: repo-neutral plugin loader. It calls the plugin'saudit_for_knowledge(), validates the report strictly (a malformed report can never read as clean), and refuses plugin paths that are absolute, escape the adapter, are missing, or aren't.py.Test Plan
test/test_release_audit.py:check_spec_freshness --strict,check_doc_links.Test Result
🤖 Generated with Claude Code