Repository navigation
feat: opt-in Muse Mesh usage collection and consent-scoped MCP reporting - #38
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
@codex review Please review head |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0f5da163be
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| evidence = {key: event[key] for key in ("model", "input_tokens", "input_uncached", "cache_read", "cache_write", "output", | ||
| "reasoning", "total", "cost_usd", "usage_source", "input_semantics", "capture_incomplete", | ||
| "charge_source", "estimated_usage")} |
There was a problem hiding this comment.
When the same native response is first captured as interrupted or failed with a complete usage envelope and later replayed as completed, both normalized events have the same request ID and evidence fingerprint even though usage_pricing_eligible changes from false to true. ingest_verified_events therefore silently skips the completed record instead of reporting a conflict, permanently retaining the unpriced/incomplete version and undercounting API-equivalent cost. Include status or the derived eligibility flag in the fingerprint so this evidence change cannot be treated as an identical replay.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 4e6fa0f149bf8478331c2a59153e6e26121c2f34. The accounting fingerprint now includes both status and usage_pricing_eligible. Two regression tests cover interrupted/failed/cancelled versus completed in both arrival orders, both within an initial batch and after reopening the persisted SQLite store. A mismatch now raises a conflict and rolls back unrelated new rows; exact replay still adds zero. All 31 ingestion/accounting tests and the real Muse/MCP fixture interaction pass locally after this fix. Native companion patch is unchanged.
|
@codex review Please re-review current head |
|
@codex security review Please confirm the security review on final head |
|
Codex Review: Didn't find any major issues. 🎉 Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Post-merge verification completeMerged normally as All five post-merge push workflows succeeded, as did all five workflows on the final PR head. Merged-commit checks. Final evidence: 31 ingestion/accounting tests, 6 MCP tests, 18 native adapter tests and 1 actual Muse/SDK/CLI/SQLite/MCP deterministic fixture test pass; native E2E has zero skips. Existing package, pricing, scoped reporting, security/store/parser, Goose and mini-SWE gates pass. Final-head automated code review reported no major issues after the completion-status fingerprint correction. A separate security review completed on earlier The numeric final-head CI receipt exactly equals the local post-fix receipt (SHA256 Companion delivery and remaining prerequisite
Muse main remains No user's collection, service or schedule was enabled. Temporary source-transfer/bootstrap work remains only on the separate unmerged branch, outside main and this PR. A final source-only delivery archive contains both complete patches, licenses, installation/verification docs, a manifest with exact commits/checks/review scope, passing test logs and the numeric CI receipt. |
1 participant
Change and affected skills
Implement both requested Muse Mesh paths without a second pricing/storage engine: versioned native usage export and local stdio MCP reporting. Ledger delivery is based on
05849f9dd595af420dddb2f4b1cf39b88dbf67b9(retains the concurrent mini-SWE-agent and Goose merges).scripts/muse_usage.py: explicit approved-source scanning, nullable evidence validation, stable native-response identity with persisted pre-request UUID fallback, strict allowlisted metadata, and bounded source-scoped reports using existing store/query/pricing code.SqliteStore.ingest_verified_events: additive atomic ingestion. Exact replay skips; conflicting evidence under the same identity rejects the transaction; concurrent scans cannot append duplicates. Legacy ingestion remains unchanged.scripts/ledger_mcp.py: six existing-capability report tools and an optional separately authorized scan tool through Muse's actual stdio MCP client. No caller-supplied paths, SQL, shell, configuration or consent-changing tool.usage.coststays distinct from API-equivalent list-price estimates; the existing pricer is reused.Muse companion and cross-repository prerequisite
Audited upstream:
muse-mesh/mu@e537b757c112557f5256b3397523a6772d768697.Companion local commit:
56cc31db555922dc92645c09b888984a65669c1aonfeat/usage-ledger-20261012(not pushed upstream).Companion resulting tree:
a5f1f7da1673127e5f685f81c4ea7b7d257d3c68.Patch:
integrations/muse-mesh/mu.patch.txt, SHA256da5b76b047ad7ccc3cbd369b6de55d6dd587f1317f597cc66a96636197ab0655.An actual attempt to create the Muse upstream branch returned HTTP 403,
Resource not accessible by integration. Therefore no upstream PR or merge is claimed. This ledger PR includes a fully tested patch installation path; upstream acceptance/publication remains a Muse maintainer prerequisite. Do not describe unmodified stock Muse as exporting this schema.The native change uses the supported AI SDK provider
fetchoption, preserving raw usage before SDK zero-filling. It covers CLI agent/tool loops, memory extraction, context compaction and the separate Web UI path. Collection is disabled by default, disabled fetch is the original function, and unavailable export storage does not reissue model calls or stop Muse. The SDK does not expose logical retry grouping, so captured transport attempts have an ordinal andretry_of: null, never an invented parent.Evidence
Completed before opening this PR:
validate_package.pypassed in the clean publication preflight on the latest base.git apply --checkon exact upstream base and its applied Git tree matches the local companion commit.Deterministic end-to-end accounting proof
The test executes the real Muse Agent and SDK using a local deterministic HTTP provider fixture; exports native usage; imports via the real Python CLI/SQLite store; loads reporting tools using Muse's actual
loadMCPTools; and queries through the stdio transport.First response: 100 input tokens (40 cached), 20 output (5 reasoning subset), 120 total. A fixture gateway-reported USD charge of 0.015 is reported separately from a hypothetical ledger-price estimate of 0.0003. Replaying that native response adds zero rows and leaves totals at 120. Additional memory, Web UI, compaction/reset and SDK rate-limit-retry paths reconcile 7 recorded attempts / 6 measured responses / 720 tokens, with the failed attempt's missing usage explicit. Pricing rates in this test are synthetic, not current commercial rate assertions.
No live model/provider, invoice reconciliation, user-log scan, deployment, service, schedule, or registry publication was performed. Native tests isolate HOME/workspace/config and use synthetic credentials and local fixtures.
Boundaries and security/privacy review
Collection, aggregate reporting disclosure, and MCP-triggered scanning each require explicit operator choices; the MCP process reloads authorization on every call. Reporting to a model may disclose aggregates to its configured provider, which is why its consent is separate. Source paths are fixed in private operator config, not accepted from tool arguments. Source records exclude prompts, completions, tool arguments, headers, credentials and error bodies. Session/run/response IDs use stable keyed hashes by default; raw identifiers are explicit opt-in. Source/key replacement, key truncation, symlinks, nonregular files and conflicting identity modes are rejected. Inputs/queries/output groups are bounded and MCP errors are sanitized.
Live imports require SQLite for atomic conflict checking. Existing file-store exports remain supported and tested; no false cross-backend live-transaction guarantee is made. A hard process kill before final metadata persistence can leave an explicit started marker/coverage gap; a failed disk plus process loss cannot be recovered without original provider evidence. Gateway usage and operator provider attribution are not independent billing/route attestation. Preserve the source identity/key across migrations; changing it can create a new deduplication namespace. The pinned legacy MCP handshake is supported, not an untested claim about future protocol versions.
Review also fixed a truncated HMAC-key caching/recovery edge case, covered by the 18th native test. No independent reviewer approval is claimed at PR creation; check current reviews/checks before merge.
Family consistency
Only the ledger-specific integration changes. Activation key, specialist CLI paths, consent gates, pricing snapshots and shared family contract remain intact. No sibling/orchestrator contract change is required. Other active integration branches and PRs were not edited. Temporary source-transport/bootstrap files are excluded from this branch and PR.
The nested
.gitattributesapplies only to the distributable unified patch, whose single-space blank context lines are required patch syntax. Actual changed Python/TypeScript code and the applied Muse diff are still whitespace-checked normally.