Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 64 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,70 @@ Before writing any new endpoint, making database changes, or updating routing, c
Endpoints that perform destructive actions, credential changes, or billing subscription alterations MUST reject API keys. Gating should explicitly check for JWT authentication.


## Audit Logging Is Automatic

Every state-changing API call is recorded in `audit_events` by a global Echo
middleware. You do not opt in to basic coverage; you opt in to *semantic
quality*. The framework lives in `internal/auditlog`, the read API in
`internal/api/audit_handlers.go`, and the UI at `/reports/audit`.

### Rules

1. **Register new mutating routes with `s.audited(...)`**, not
`group.POST(...)`. It registers the route and its `auditlog.Spec` together so
they cannot drift. Use an action constant from
`internal/auditlog/catalog.go` — never a string literal.
2. **Privileged handlers MUST call `s.auditRecorder.RecordTxEcho(c, tx, ...)` inside the
same transaction.** User/role, API-key, password, billing, organization, and
settings changes are fail-closed: if the audit row cannot be written, the
business change rolls back. Middleware records the failed attempt outside
the transaction.
3. **Never put bodies, passwords, tokens, API keys, authorization headers,
cookies, or raw query strings in `details`.** Use `auditlog.Details(...)`; the
sanitizer is a backstop, not permission. Raw prompts/responses are never
stored — use a prompt hash and byte size.
4. **Exports, file downloads, and audit reads MUST emit a sensitive-read
event** (`auditlog.Recorder.SensitiveRead` / the audit handlers' `recordAuditRead`).
Background jobs and webhooks use `s.auditRecorder.System(...)`; central LLM
call sites use `s.auditRecorder.AIInvocation(...)` with a prompt fingerprint.
5. **Audit reads MUST go through `auditlog.NewQuery(scope, filters)`** (built
from `PermissionContext`). Never hand-write an audit `WHERE org_id = ...`.
The query builder enforces tenant isolation plus the member/owner/super-admin
visibility rules in one place.
6. **Every new action must appear in `docs/audit-events.md`**; a test enforces
it. Action names are a public SIEM contract: rename = alias, never reuse.
7. **`go test ./internal/auditlog/...`** enforces the catalog/docs contract.
The HTTP middleware guarantees generic coverage for every non-GET route; run
the server in development to see which existing routes still need a
semantic spec.
8. **API keys:** only `kind='audit_reader'` keys may read audit data, and only
a session may read cross-org admin audit data or change retention settings.
An audit-only key must be rejected on every non-audit route.
9. **Append-only:** `audit_events` rejects `UPDATE` and `DELETE` (the retention
job opts in with `app.audit_purge=on`). Do not add code that edits history;
record a new event instead.
10. **Documentation:** update `internal/docindex/docs/routes_guide/reports/audit-log.md`
when the page, its filters, or its access rules change.

**Visibility contract:** every super-admin action is recorded with
`actor_capacity='admin'` (and usually `actor_kind='super_admin'`). Owners see
their organization and all member actions, but not super-admin-capacity work;
members see only their own actions; super admins see everything, in every org.
Never skip recording an event because the actor is a super admin.

### Example

```go
s.audited(orgGroup, echo.PUT, "/users/:user_id/role", auditlog.Spec{
Action: auditlog.ActionUserRoleChange,
Category: auditlog.CategoryUser,
ResourceType: "user",
ResourceIDParam: "user_id",
Severity: auditlog.SeverityWarning,
Sensitive: true,
}, s.userHandlers.ChangeUserRole)
```

## Porting from git-lrc

`git-lrc` (sibling repo, typically checked out at `../git-lrc`) is where LiveReview's
Expand Down
44 changes: 44 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
.PHONY: razorpay-webhook-ensure razorpay-webhook-ensure-dry razorpay-verify-plans razorpay-verify-plans-low-pricing
.PHONY: raw-deploy raw-deploy-low-pricing raw-deploy-backend raw-deploy-backend-low-pricing build-staging-with-ui raw-deploy-staging stop-staging ensure-seed-demo-env
.PHONY: dev dev-up dev-down dev-restart dev-status dev-attach
.PHONY: e2e e2e-up e2e-down e2e-auth e2e-preflight e2e-audit e2e-review e2e-urls e2e-visual e2e-clean e2e-shell e2e-doctor e2e-status

# ============================================================================
# Environment switching
Expand Down Expand Up @@ -1566,6 +1567,49 @@ dev-status:
dev-attach:
@scripts/dev.sh attach

# ============================================================================
# Local E2E harness (real accounts, isolated lrc, audit scoping)
# ============================================================================

e2e-up:
@scripts/e2e/bootstrap.sh up --interactive

e2e-down:
@scripts/e2e/bootstrap.sh down

e2e-auth:
@scripts/e2e/bootstrap.sh auth $(ARGS)

e2e-status:
@scripts/e2e/bootstrap.sh status

e2e-doctor:
@scripts/e2e/bootstrap.sh doctor

e2e-preflight:
@scripts/e2e/bootstrap.sh preflight $(ARGS)

e2e-audit:
@scripts/e2e/bootstrap.sh audit $(ARGS)

e2e-review:
@scripts/e2e/bootstrap.sh review $(ARGS)

e2e-urls:
@scripts/e2e/bootstrap.sh urls $(ARGS)

e2e:
@scripts/e2e/run.sh $(ARGS)

e2e-visual:
@scripts/e2e/run.sh --skip-api --skip-review --skip-urls $(ARGS)

e2e-clean:
@scripts/e2e/bootstrap.sh clean

e2e-shell:
@scripts/e2e/shell.sh

# ============================================================================
# CI/CD Gate demo (shrsv/claude-world) -- see demo/cicd/README.md
# ============================================================================
Expand Down
4 changes: 0 additions & 4 deletions config/osv-scanner.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,3 @@
[[ignoredVulns]]
id = "GO-2026-5932"
reason = "False positive — golang.org/x/crypto/openpgp sub-package is never imported by this project; only golang.org/x/crypto/bcrypt is used."

[[ignoredVulns]]
id = "GHSA-vfj7-8cjw-p6xm"
reason = "braces v3.0.3 is the latest version available and there is no patch. As a devDependency only invoked at build time, it is not exposed to untrusted user input."
182 changes: 182 additions & 0 deletions db/migrations/20261025120000_create_audit_events.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,182 @@
-- migrate:up

-- Unified, append-only audit trail for enterprise compliance.
--
-- Partitioned by month so retention is a cheap DROP PARTITION rather than a
-- mass DELETE. A DEFAULT partition is kept as a safety net: audit inserts must
-- never fail because a futuristic partition was not created yet. Monthly
-- partitions are created ahead of time by AuditMaintenanceManager.
CREATE TABLE audit_events (
id BIGSERIAL,
occurred_at TIMESTAMPTZ NOT NULL DEFAULT now(),

org_id BIGINT NOT NULL,
actor_user_id BIGINT NULL,
actor_email TEXT NULL,
actor_name TEXT NULL,
actor_kind TEXT NOT NULL DEFAULT 'system', -- member | api_key | system | webhook | super_admin | legacy
actor_capacity TEXT NOT NULL DEFAULT 'member', -- member | admin | system
auth_method TEXT NULL, -- session | api_key | none
api_key_id BIGINT NULL,

action TEXT NOT NULL, -- e.g. user.role_changed
category TEXT NOT NULL, -- auth | user | org | ...
resource_type TEXT NULL,
resource_id TEXT NULL,

outcome TEXT NOT NULL DEFAULT 'success', -- success | failure | denied | error
status_code INT NULL,
severity TEXT NOT NULL DEFAULT 'info', -- info | warning | critical

ip INET NULL,
user_agent TEXT NULL,
request_id TEXT NULL,
method TEXT NULL,
path TEXT NULL, -- route template, never a raw URL

details JSONB NOT NULL DEFAULT '{}',
source TEXT NOT NULL DEFAULT 'api', -- api | webhook | system | legacy
legacy_id BIGINT NULL,

-- Reserved for the per-org hash chain (enabled in phase B).
seq BIGINT NULL,
prev_hash TEXT NULL,
hash TEXT NULL,

PRIMARY KEY (id, occurred_at)
) PARTITION BY RANGE (occurred_at);

-- Safety net so an insert can never fail for lack of a monthly partition.
CREATE TABLE audit_events_default PARTITION OF audit_events DEFAULT;

-- Query indexes. Creating them on the partitioned parent propagates to every
-- existing and future partition.
CREATE INDEX idx_audit_events_id
ON audit_events (id);
CREATE INDEX idx_audit_events_org_time
ON audit_events (org_id, occurred_at DESC, id DESC);
CREATE INDEX idx_audit_events_org_category_time
ON audit_events (org_id, category, occurred_at DESC);
CREATE INDEX idx_audit_events_org_actor_time
ON audit_events (org_id, actor_user_id, occurred_at DESC);
CREATE INDEX idx_audit_events_org_action_time
ON audit_events (org_id, action, occurred_at DESC);
CREATE INDEX idx_audit_events_org_outcome_time
ON audit_events (org_id, outcome, occurred_at DESC);
CREATE INDEX idx_audit_events_org_severity_time
ON audit_events (org_id, severity, occurred_at DESC);
CREATE INDEX idx_audit_events_org_seq
ON audit_events (org_id, seq) WHERE seq IS NOT NULL;
CREATE UNIQUE INDEX uq_audit_events_legacy
ON audit_events (source, legacy_id, occurred_at) WHERE legacy_id IS NOT NULL;

-- Append-only guard. UPDATE is never allowed. DELETE is allowed only when the
-- retention job explicitly opts in for the current transaction, so normal
-- application code (and a compromised handler) cannot rewrite history.
CREATE OR REPLACE FUNCTION audit_events_append_only() RETURNS trigger AS $$
BEGIN
IF TG_OP = 'UPDATE' THEN
RAISE EXCEPTION 'audit_events is append-only: UPDATE is not permitted';
END IF;
IF TG_OP = 'DELETE'
AND COALESCE(current_setting('app.audit_purge', true), '') <> 'on' THEN
RAISE EXCEPTION 'audit_events is append-only: DELETE is not permitted';
END IF;
RETURN OLD;
END;
$$ LANGUAGE plpgsql;

CREATE TRIGGER audit_events_append_only_trigger
BEFORE UPDATE OR DELETE ON audit_events
FOR EACH ROW EXECUTE FUNCTION audit_events_append_only();

-- One row per archived (month, org). The object lives in the configured blob
-- store; this table is the index and the tamper-evidence digest.
CREATE TABLE audit_archives (
org_id BIGINT NOT NULL,
month TEXT NOT NULL, -- YYYY-MM
object_key TEXT NOT NULL,
row_count BIGINT NOT NULL,
sha256 TEXT NOT NULL,
archived_at TIMESTAMPTZ NOT NULL DEFAULT now(),
PRIMARY KEY (org_id, month)
);

-- Runtime policy. Code also has safe defaults if this row is absent.
INSERT INTO system_settings (name, data)
VALUES ('audit_settings', '{
"enabled": true,
"hot_days": 90,
"retention_days": 365,
"archive_enabled": true,
"anchor_enabled": true,
"hash_chain_enabled": true,
"cron_expression": "15 3 * * *"
}'::jsonb)
ON CONFLICT (name) DO NOTHING;

-- Import the pre-existing user-management audit trail as legacy history, so
-- the new page is populated from day one. These rows predate IP/device
-- capture, so those columns stay NULL and source='legacy' marks them.
INSERT INTO audit_events (
occurred_at, org_id,
actor_user_id, actor_email, actor_name, actor_kind, actor_capacity,
action, category, resource_type, resource_id,
outcome, severity, details, source, legacy_id
)
SELECT
uma.created_at AT TIME ZONE 'UTC',
uma.org_id,
uma.performed_by_user_id,
p.email,
COALESCE(NULLIF(TRIM(CONCAT_WS(' ', p.first_name, p.last_name)), ''), p.email),
CASE WHEN EXISTS (
SELECT 1 FROM user_roles ur
JOIN roles r ON r.id = ur.role_id
WHERE ur.user_id = uma.performed_by_user_id AND r.name = 'super_admin'
) THEN 'super_admin' ELSE 'legacy' END,
CASE WHEN EXISTS (
SELECT 1 FROM user_roles ur
JOIN roles r ON r.id = ur.role_id
WHERE ur.user_id = uma.performed_by_user_id AND r.name = 'super_admin'
) THEN 'admin' ELSE 'member' END,
CASE uma.action
WHEN 'created' THEN 'user.create'
WHEN 'updated' THEN 'user.update'
WHEN 'deactivated' THEN 'user.deactivate'
WHEN 'reactivated' THEN 'user.reactivate'
WHEN 'role_changed' THEN 'user.role.changed'
WHEN 'password_reset' THEN 'user.password.reset.force'
WHEN 'transferred' THEN 'user.transfer'
ELSE 'user.' || uma.action
END,
'user',
'user',
uma.target_user_id::text,
'success',
CASE WHEN uma.action IN ('role_changed', 'deactivated') THEN 'warning' ELSE 'info' END,
COALESCE(uma.details, '{}'::jsonb),
'legacy',
uma.id
FROM user_management_audit uma
LEFT JOIN users p ON p.id = uma.performed_by_user_id;

-- migrate:down

DROP INDEX IF EXISTS uq_audit_events_legacy;
DROP INDEX IF EXISTS idx_audit_events_org_seq;
DROP INDEX IF EXISTS idx_audit_events_org_severity_time;
DROP INDEX IF EXISTS idx_audit_events_org_outcome_time;
DROP INDEX IF EXISTS idx_audit_events_org_action_time;
DROP INDEX IF EXISTS idx_audit_events_org_actor_time;
DROP INDEX IF EXISTS idx_audit_events_org_category_time;
DROP INDEX IF EXISTS idx_audit_events_org_time;
DROP INDEX IF EXISTS idx_audit_events_id;

DROP TRIGGER IF EXISTS audit_events_append_only_trigger ON audit_events;
DROP FUNCTION IF EXISTS audit_events_append_only();

DROP TABLE IF EXISTS audit_archives;
DROP TABLE IF EXISTS audit_events;

DELETE FROM system_settings WHERE name = 'audit_settings';
29 changes: 29 additions & 0 deletions db/migrations/20261025120100_add_api_key_kind.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
-- migrate:up

-- API key kinds. 'standard' keys keep today's behaviour (inherit the
-- creator's live role). 'audit_reader' keys are SIEM-only: they may call
-- audit read/export endpoints and nothing else.
ALTER TABLE api_keys
ADD COLUMN IF NOT EXISTS kind VARCHAR(20) NOT NULL DEFAULT 'standard';

DO $$
BEGIN
IF NOT EXISTS (
SELECT 1 FROM pg_constraint WHERE conname = 'chk_api_keys_kind'
) THEN
ALTER TABLE api_keys
ADD CONSTRAINT chk_api_keys_kind
CHECK (kind IN ('standard', 'audit_reader'));
END IF;
END $$;

CREATE INDEX IF NOT EXISTS idx_api_keys_org_kind
ON api_keys (org_id, kind)
WHERE revoked_at IS NULL;

-- migrate:down

DROP INDEX IF EXISTS idx_api_keys_org_kind;

ALTER TABLE api_keys DROP CONSTRAINT IF EXISTS chk_api_keys_kind;
ALTER TABLE api_keys DROP COLUMN IF EXISTS kind;
32 changes: 32 additions & 0 deletions db/migrations/20261025130000_backfill_super_admin_visibility.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
-- migrate:up

-- Normalize historical rows created before the super-admin visibility rule was
-- introduced. Every super-admin action is visible only to super admins; org
-- owners see their org and all member actions. This is a one-time metadata
-- correction, so the append-only trigger is disabled for the duration of this
-- transactional migration and re-enabled immediately afterwards.
--
-- This takes an ACCESS EXCLUSIVE lock on the audit table/partitions. Run it in
-- a maintenance window on a busy production instance.
ALTER TABLE audit_events DISABLE TRIGGER audit_events_append_only_trigger;

UPDATE audit_events ae
SET actor_capacity = 'admin',
actor_kind = CASE
WHEN ae.actor_kind = 'member' THEN 'super_admin'
ELSE ae.actor_kind
END
WHERE ae.actor_capacity <> 'admin'
AND ae.actor_kind NOT IN ('system', 'webhook')
AND ae.actor_user_id IN (
SELECT ur.user_id
FROM user_roles ur
JOIN roles r ON r.id = ur.role_id
WHERE r.name = 'super_admin'
);

ALTER TABLE audit_events ENABLE TRIGGER audit_events_append_only_trigger;

-- migrate:down

-- No-op by design. Reverting would re-expose support actions to customers.
Loading
Loading