audit: standardize operation correlation, identity provenance, and outcome semantics #341

Problem

An investigation should be able to answer who did what, when, where, and with what outcome across components that use audit.AuditEvent. We already have the basic structure, generated event IDs, UTC logging timestamps, MCP event constants, and extensible metadata. We should reuse those rather than have each consumer invent a different record format.

What is missing is a documented shared profile for correlating an operation across authorization, dispatch, and observation, and for distinguishing verified identities from reported context. Free-form maps can carry the data today, but independently chosen keys and meanings make cross-component investigations unreliable.

Existing code: https://github.com/stacklok/toolhive-core/blob/5ba2b36edf8051b47bab103aad93223cb3dcd0b3/audit/event.go

Requested enhancement

Define shared conventions, constants, and only the small helpers needed for:

• An operation identifier, an attempt identifier, and the existing event identifier. Multiple events can describe one attempt; a retry is a new attempt; resending the same audit record does not invent a new event.
• Distinct authorization-decision, dispatch-intent/attempt, and observed-outcome events. A permit is not proof of dispatch, and a response is not independent proof of a downstream effect.
• An explicit unknown or indeterminate outcome, distinct from denied, failed, and successful. In particular, a timeout after possible dispatch must not be reported as proof of nonexecution.
• Issuer/trust-domain-qualified identities and evidence provenance: what the recording component verified, what another component reported, and what is unavailable. Do not promote an unverified identity from a rejected request into an authenticated subject.
• Consistent action/target and recording component/location fields, plus namespaced optional references to policy revisions or other explanatory evidence. Keep the profile generic rather than adding product-specific session or agent models to the package.
• A precise definition of LoggedAt as recording time and a convention for occurrence time when it differs. Preserve UTC/offset semantics and identify whose clock supplied a timestamp.

Prefer existing AuditEvent, Subjects, Target, and metadata extension points where they suffice. Document a stable profile/version and compatibility policy rather than requiring a wholesale schema redesign. Document the JSON and slog projections explicitly; they need consistent meaning even where field names already differ.

Acceptance criteria

• Examples show two services recording a denied operation and a retried operation with an unknown first outcome. An investigator can distinguish events, attempts, sources, and verified versus reported subjects.
• Tests cover shared field semantics in both JSON serialization and LogTo output, including missing identities, unknown outcomes, and caller-supplied recording IDs.
• Existing event consumers and existing outcome values remain compatible, or any necessary breaking change is explicitly assessed before implementation.
• Examples use allowlisted metadata and references, not raw credentials, claims, request headers, or unrestricted request/response bodies.

Scope

This supports the record-content and timestamp goals of NIST SP 800-53 AU-3 and AU-8. It does not claim full NIST compliance or select an event-retention backend, authorization policy, exactly-once execution mechanism, or required event inventory for every consumer.

NIST reference: https://csrc.nist.gov/pubs/sp/800/53/r5/upd1/final

One motivating consumer is the agent-audit design in stacklok/mecatl#2121, but the profile should work for gateways, brokers, services, and workflows generally.