[chou-godaddy]

Summary

ans-ocsf-hydrator could report 100% policy adherence while an agent violated its policy. In OpenShell's audit mode, violations are logged with allow enums and only the message says audit. Connection errors counted as violations, session records counted as allowed, and one request produced several records. This PR corrects those counts in a new, opt-in openshell.behavior.activity output that agent-trust-discovery imports directly through its existing {score, explanation} signal format. The existing signals are unchanged.

Fixes #9

Changes

• Classification (internal/hydrator/activity_event.go): an ordered rule list routes each OCSF record. The five L7 formats (L7_REQUEST, GRAPHQL_L7_REQUEST, JSONRPC_L7_REQUEST, L7_REQUEST_TRANSFORMED, WEBSOCKET_L7_REQUEST) are parsed into allowed, denied, audited, or unknown. Connection errors, sessions, DNS refusals, middleware telemetry, passthrough HTTP, and coverage records become diagnostics instead of decisions. A record that states a decision but cannot be counted marks the agent incomplete, so it cannot hide a denial.
• Scoring (activity_score.go): connection, initial HTTP, post-rewrite, and WebSocket checks are counted separately. The score is the rounded minimum of 100 × allowed / (allowed + denied + audited) over groups that meet activity.min_decisions. Unknown decisions or exhausted deduplication budgets withhold only the affected agent's score.
• Evidence (activity.go, --evidence-output): per-agent JSONL with the window, counts per group, sample size, finding origins and match counts, policy hashes and versions, policyChanged, and diagnostics. Finding-only windows produce evidence without a score. Findings never change the score.
• Output (observation.go, run.go, cmd/ans-ocsf-hydrator): signals.activity enables the new observation; risk codes are omitted. Explanations summarize findings and name the policies behind the counts.
• Deploy and docs: deploy/examples/hydrator-activity.yaml, deploy/examples/trust-index-score-signals.yaml (registration with threshold: 0), the Helm window comment, and docs/hydrator-activity.md with migration steps and known limits.

Defensive checks beyond the issue

Audit records are partly written by the sandbox being measured, so the new path adds four checks that #9 does not ask for. Each one withholds a score rather than risk reporting a wrong one. They are covered by unit tests; normal traffic does not trigger them, so the live run does not exercise them.

• Total deduplication budget (activity.max_uids, default 1,000,000, next to the per-agent activity.max_uids_per_agent): when the total fills, the agent holding the most event IDs is marked dedup_saturated and its set is released, so later agents still fit in memory.
• DNS-to-TCP check: DNS refusals are diagnostics, not decisions. If a destination has more DNS refusals than TCP denials, the agent is marked incomplete (unrepresented_dns_refusals), so a refusal with no matching denial cannot disappear.
• Unreadable audit lines: an audit-shaped line that cannot be decoded or exceeds 1 MiB cannot be attributed to a sandbox, so it withholds every agent's new score (unreadable_audit_records) until the file rotates. An incomplete final line that is still being written only adds a diagnostic.
• Evidence file safety: --evidence-output creates the file exclusively with mode 0600 and refuses to overwrite an existing file. The Helm chart accepts only stdout (-) or disabled, because the CronJob reruns every schedule and would collide with its previous evidence file.

Per-agent maps and text fields from audit records are also bounded.

Testing

• go test -race ./..., go vet ./..., gofmt, and golangci-lint v2.10.1 with a clean cache: all pass, 0 issues. Hydrator coverage 97.8%.
• Table tests cover each audit format, connection errors, sessions, an allowed upstream 503, separate counters, connection-only traffic, deduplication isolation, finding-only windows, and legacy isolation. A fuzz test checks that a record stating a decision is never silently dropped.
• Live run on OpenShell v0.1.5 (Docker Desktop) with agent-trust-discovery@e94aca5:
  • Legacy output from main and this branch is byte-identical on the same captures.
  • Personas score 100 / 67 / 50 / 38, the reference values in hydrator: correct behavior observations and add Trust Index output #9. A persona whose allowed upstream returns 503 scores 100.
  • With enforcement: audit, two forbidden POSTs reached the upstream and were logged as action_id 1 / disposition_id 1 with L7_REQUEST audit. Legacy protocolAdherence reports 1.0; the new score is 67.
  • The Trust Index loads the registration and a profile with openshell.behavior.activity: 0, imports the output without conversion, and shows each score at weight 0 without changing the behavior dimension.
  • A mid-window openshell policy set yields policyChanged: true and an explanation naming both policy hashes.
  • The four copy-and-run inputs in hydrator: correct behavior observations and add Trust Index output #9 give the corrected results listed there.

Rollout criteria

Retained and superseded scores are consumer behavior, and this repository has no agent-trust-discovery dependency. They are covered by agent-trust-discovery tests (TestImportObservations_Supersede, TestAppendObservationDedupsIdenticalTriple) and were confirmed in the live run: a skipped report kept an earlier 100, and a later clean window replaced a 67. Re-posting the same window is ignored by the consumer, as documented.

Known limits

The signal stays at weight 0 until the consumer can expire stale scores. docs/hydrator-activity.md lists the remaining limits: a sandbox can withhold its own new score with conflicting records, or every agent's with one unreadable audit line until the file rotates; a single violation among 200 or more decisions still rounds to 100; and this measures policy adherence, not content safety.

AI assistance

Assisted-by: Cursor (Claude Opus 5.5)