From 2f914302bd41976caed4698204a43a8015e9fae4 Mon Sep 17 00:00:00 2001 From: Ganesh Kumar Date: Thu, 18 Jun 2026 21:06:54 +0530 Subject: [PATCH 01/14] Implement Third-Party Tool Integration LiveReview Pre-Commit Check: ran (iter:1, coverage:0%) --- .../.config.kiro | 1 + .../requirements.md | 200 ++++++++ .../20260618100000_create_available_tools.sql | 18 + .../20260618100001_create_org_tools.sql | 15 + docs/openapi.yaml | 4 +- docs/tools/tools-integration-beta.md | 464 ++++++++++++++++++ internal/api/server.go | 8 + internal/api/tools_handler.go | 153 ++++++ storage/tools/tools_store.go | 217 ++++++++ ui/src/components/UIPrimitives.tsx | 6 + ui/src/pages/Settings/Settings.tsx | 20 + ui/src/pages/Settings/ThirdPartyToolsTab.tsx | 449 +++++++++++++++++ 12 files changed, 1553 insertions(+), 2 deletions(-) create mode 100644 .kiro/specs/third-party-tools-integration/.config.kiro create mode 100644 .kiro/specs/third-party-tools-integration/requirements.md create mode 100644 db/migrations/20260618100000_create_available_tools.sql create mode 100644 db/migrations/20260618100001_create_org_tools.sql create mode 100644 docs/tools/tools-integration-beta.md create mode 100644 internal/api/tools_handler.go create mode 100644 storage/tools/tools_store.go create mode 100644 ui/src/pages/Settings/ThirdPartyToolsTab.tsx diff --git a/.kiro/specs/third-party-tools-integration/.config.kiro b/.kiro/specs/third-party-tools-integration/.config.kiro new file mode 100644 index 000000000..9512371f1 --- /dev/null +++ b/.kiro/specs/third-party-tools-integration/.config.kiro @@ -0,0 +1 @@ +{"specId": "7e1f430f-5e58-48d0-9fcf-e4178321dfb1", "workflowType": "fast-task", "specType": "feature"} diff --git a/.kiro/specs/third-party-tools-integration/requirements.md b/.kiro/specs/third-party-tools-integration/requirements.md new file mode 100644 index 000000000..f695df6bf --- /dev/null +++ b/.kiro/specs/third-party-tools-integration/requirements.md @@ -0,0 +1,200 @@ +# Requirements Document + +## Introduction + +This document describes the requirements for Third-Party Tools Integration in LiveReview — a cloud-only, owner-gated feature that allows organisations to enable external static-analysis tools (e.g. ruff, pylint) that run as parallel Lambda-backed jobs whenever a review is triggered. Results are stored in the `review_events` table and surfaced in a dedicated beta review UI and in the existing git-lrc CLI output. + +The feature is delivered in three sequential phases: + +1. **Phase 1 – DB + Settings tab**: Catalog and per-org tool tables, dbmate migrations, settings tab gated to cloud-mode owners. +2. **Phase 2 – Settings UI + API gating**: Full CRUD UI for tool selection, REST endpoints with role enforcement. +3. **Phase 3 – Queue, Lambda trigger + review UI**: River job fan-out per tool, Lambda invocation, result storage, and two dedicated UI surfaces. + +--- + +## Glossary + +- **AvailableTools**: The global catalog table (`available_tools`) that seeds all tools LiveReview knows about. +- **OrgTools**: The per-organisation configuration table (`org_tools`) that records which tools an org has enabled and any per-tool configuration. +- **Tool**: A record in `available_tools` with a unique id, human-readable name, description, and an AWS Lambda ARN to invoke. +- **ToolResult**: A `review_events` row with `event_type = 'tool_result'` that carries the output of a single tool invocation for a review. +- **ToolJob**: A River background job (kind `tool_invocation`) that reads a review's diff and invokes a Tool's Lambda ARN, producing a ToolResult. +- **PermissionContext**: The Echo middleware-constructed object placed in the request context under `permission_context`, encoding the authenticated user's `org_id`, `user_id`, and `role`. +- **CloudModeGate**: The server-side `isCloudMode()` check (`LIVEREVIEW_IS_CLOUD=true`) combined with the `isCloudMode()` client-side utility in the React UI. +- **BillingCheck**: Verification that the org has an active billing plan via the existing `apimiddleware.BuildOrgBillingPlanContext` + `BuildPlanContext` middleware chain. +- **SettingsTab**: The `/#/settings#third-party-tools` hash route within the existing Settings page component. +- **BetaReviewUI**: The beta review page accessible at `/#/reviews-tools/new`, not linked in the main navigation. +- **ReviewDetail**: The existing review detail page that renders events associated with a review. +- **git-lrc**: The CLI binary whose binary name is `lrc`, used for local review triggering. +- **River**: The PostgreSQL-backed job queue library used for background processing. +- **dbmate**: The migration tool used exclusively for local development database schema changes. +- **LambdaInvoker**: The component responsible for making AWS Lambda HTTP invocations for a given tool's ARN. + +--- + +## Requirements + +### Requirement 1 – Available Tools Catalog + +**User Story:** As a LiveReview platform operator, I want a seeded catalog of available third-party tools, so that organisations can choose from a known set of tools without manual configuration. + +#### Acceptance Criteria + +1. THE System SHALL provide an `available_tools` table with columns: `id` (bigserial primary key), `name` (text not null unique), `description` (text not null), and `lambda_arn` (text not null). +2. THE System SHALL seed the `available_tools` table with at least two initial rows: one for `ruff` and one for `pylint`, each with a non-empty `description` and a placeholder `lambda_arn`. +3. THE System SHALL manage the `available_tools` schema exclusively through dbmate migration files located in `db/migrations/`, and SHALL NOT apply these migrations directly to any production database. + +--- + +### Requirement 2 – Per-Organisation Tool Configuration + +**User Story:** As an organisation owner on the cloud plan, I want to enable or disable specific tools for my organisation, so that only the tools relevant to my stack are invoked during reviews. + +#### Acceptance Criteria + +1. THE System SHALL provide an `org_tools` table with columns: `org_id` (bigint not null, references `organizations.id`), `tool_id` (bigint not null, references `available_tools.id`), `enabled` (boolean not null default false), `config_json` (jsonb not null default `'{}'`), and a composite primary key on (`org_id`, `tool_id`). +2. THE System SHALL enforce that every row in `org_tools` references a valid `org_id` in the `organizations` table and a valid `tool_id` in the `available_tools` table via foreign key constraints. +3. THE System SHALL manage the `org_tools` schema exclusively through dbmate migration files and SHALL NOT apply these migrations directly to any production database. + +--- + +### Requirement 3 – Settings Tab Access Control + +**User Story:** As a cloud-mode organisation owner, I want a dedicated third-party tools settings tab, so that I can manage tool configuration without exposing it to members or self-hosted users. + +#### Acceptance Criteria + +1. WHEN `isCloudMode()` returns `true` AND the authenticated user's role is `owner`, THE Settings Page SHALL render a navigable tab at the hash route `third-party-tools` within `/#/settings`. +2. WHEN `isCloudMode()` returns `false`, THE Settings Page SHALL NOT render the `third-party-tools` tab. +3. WHEN the authenticated user's role is not `owner`, THE Settings Page SHALL NOT render the `third-party-tools` tab as a clickble navigation item. +4. WHEN a non-owner org member navigates directly to `/#/settings#third-party-tools`, THE Settings Page SHALL render a read-only view of the enabled tools without controls to modify them. + +--- + +### Requirement 4 – List Org Tools API Endpoint + +**User Story:** As an authenticated organisation member, I want to retrieve the tool configuration for my organisation, so that the UI can display which tools are currently enabled. + +#### Acceptance Criteria + +1. THE Server SHALL expose a `GET /api/v1/orgs/:org_id/tools` endpoint that returns the list of all tools in `available_tools` joined with the corresponding `org_tools` row (if present) for the requesting org. +2. WHEN a request is received at `GET /api/v1/orgs/:org_id/tools`, THE Server SHALL apply the full Echo middleware chain: `RequireAuthOrAPIKey`, `BuildOrgContext`, `ValidateOrgAccess`, `BuildPermissionContext`. +3. WHEN `isCloudMode()` returns `false` on the server, THE Server SHALL respond to `GET /api/v1/orgs/:org_id/tools` with HTTP 403. +4. WHEN the `PermissionContext` `org_id` does not match the `:org_id` path parameter, THE Server SHALL respond with HTTP 403. +5. THE Server SHALL scope the `org_tools` query exclusively by the `org_id` resolved from `PermissionContext`, and SHALL NOT use the client-supplied `:org_id` path parameter as the filter value. + +--- + +### Requirement 5 – Update Org Tool API Endpoint + +**User Story:** As a cloud-mode organisation owner, I want to enable or disable a specific tool for my organisation via the API, so that tool selection changes are persisted securely. + +#### Acceptance Criteria + +1. THE Server SHALL expose a `PUT /api/v1/orgs/:org_id/tools/:tool_id` endpoint that creates or updates the `org_tools` row for the given `(org_id, tool_id)` pair. +2. WHEN a request is received at `PUT /api/v1/orgs/:org_id/tools/:tool_id`, THE Server SHALL apply the full Echo middleware chain: `RequireAuthOrAPIKey`, `BuildOrgContext`, `ValidateOrgAccess`, `BuildPermissionContext`. +3. WHEN `isCloudMode()` returns `false` on the server, THE Server SHALL respond to `PUT /api/v1/orgs/:org_id/tools/:tool_id` with HTTP 403. +4. WHEN the authenticated user's role resolved from `PermissionContext` is not `owner`, THE Server SHALL respond to `PUT /api/v1/orgs/:org_id/tools/:tool_id` with HTTP 403. +5. THE Server SHALL validate the request body contains an `enabled` boolean field; IF the field is absent or not a boolean, THEN THE Server SHALL respond with HTTP 400. +6. THE Server SHALL upsert the `org_tools` row using the `org_id` from `PermissionContext` as the scoping value, and SHALL NOT use the client-supplied `:org_id` as the insert/update value. + +--- + +### Requirement 6 – Settings UI for Tool Selection + +**User Story:** As a cloud-mode organisation owner, I want a UI inside the settings tab to see all available tools and toggle each one on or off, so that I have direct control over which tools run on my reviews. + +#### Acceptance Criteria + +1. WHEN the `third-party-tools` settings tab is active and `isCloudMode()` returns `true` and the user is an `owner`, THE ThirdPartyToolsTab Component SHALL render a list of all available tools fetched from `GET /api/v1/orgs/:org_id/tools`. +2. WHEN an owner toggles a tool's enabled state in the UI, THE ThirdPartyToolsTab Component SHALL call `PUT /api/v1/orgs/:org_id/tools/:tool_id` with the new `enabled` value. +3. IF `PUT /api/v1/orgs/:org_id/tools/:tool_id` returns an error, THEN THE ThirdPartyToolsTab Component SHALL display an inline error message and SHALL revert the toggle to the previous state. +4. WHILE the UI is fetching or submitting tool configuration, THE ThirdPartyToolsTab Component SHALL render a loading indicator and disable all tool toggles. +5. WHEN the user is a non-owner member and `isCloudMode()` returns `true`, THE ThirdPartyToolsTab Component SHALL render the tool list in a read-only state with no toggles. + +--- + +### Requirement 7 – Tool Job Fan-Out on Review + +**User Story:** As a developer whose organisation has tools enabled, I want the tools to run automatically whenever a review runs, so that I receive tool feedback alongside the AI review without manual intervention. + +#### Acceptance Criteria + +1. WHEN a review job completes initial diff extraction and at least one tool is enabled for the review's `org_id`, THE ReviewOrchestrator SHALL enqueue one River job of kind `tool_invocation` per enabled tool in parallel (fan-out). +2. THE ToolJob SHALL follow the same River job registration pattern as `webhook_install` and `webhook_removal` in `internal/jobqueue/`, including a `Kind()` method and a dedicated worker struct. +3. THE ToolJob SHALL read the diff content from the review's database record identified by `review_id`, and SHALL NOT re-fetch the diff from the VCS provider. +4. WHEN the ToolJob executes, THE LambdaInvoker SHALL invoke the Lambda function identified by the tool's `lambda_arn` from `available_tools`, passing the diff as the request payload. +5. IF the Lambda invocation returns a non-2xx HTTP status, THEN THE ToolJob SHALL mark the job as failed and River SHALL apply the standard retry policy. + +--- + +### Requirement 8 – Tool Result Storage + +**User Story:** As a developer, I want tool results stored alongside review events, so that the review detail page can display them with the correct source attribution. + +#### Acceptance Criteria + +1. WHEN the LambdaInvoker receives a successful response, THE ToolJob SHALL insert a row into `review_events` with `event_type = 'tool_result'`, the `review_id`, `org_id`, and a `data` JSONB payload containing at minimum `tool_name`, `tool_id`, and the Lambda response body. +2. THE ToolJob SHALL scope the `review_events` insert using the `org_id` from the review record and SHALL NOT derive `org_id` from any other source. +3. WHEN multiple ToolJobs complete for the same review, THE System SHALL store each tool's result as a separate `review_events` row, identified by a distinct `tool_name` value in the `data` JSONB field. + +--- + +### Requirement 9 – Beta Review UI Page + +**User Story:** As a developer participating in the beta, I want a dedicated review UI page for tool-based reviews at a hidden route, so that I can access tool results without the route appearing in the main navigation. + +#### Acceptance Criteria + +1. THE Router SHALL register the path `/#/reviews-tools/new` as a valid client-side route rendering the BetaToolReviewPage component. +2. THE Router SHALL NOT add a navigation entry for `/#/reviews-tools/new` in the main sidebar or any other navigation surface. +3. THE BetaToolReviewPage Component SHALL render a layout similar to the existing AI review UI, adapted to display tool-based review results. +4. WHEN `isCloudMode()` returns `false`, THE BetaToolReviewPage Component SHALL render an informational message indicating the feature is only available in cloud mode. + +--- + +### Requirement 10 – Tool Results in Review Detail UI + +**User Story:** As a developer, I want to see tool results labelled by tool name in the review detail page, so that I can distinguish tool findings from AI findings at a glance. + +#### Acceptance Criteria + +1. WHEN the ReviewDetail Component fetches `review_events` for a review and encounters an event with `event_type = 'tool_result'`, THE ReviewDetail Component SHALL render the event with a badge displaying the value of `data.tool_name`. +2. THE badge for a tool result SHALL be visually distinct from AI review event badges already present in the ReviewDetail Component. +3. WHEN there are no `tool_result` events for a review, THE ReviewDetail Component SHALL NOT render any tool result section. + +--- + +### Requirement 11 – Tool Result Tag in git-lrc CLI Output + +**User Story:** As a developer using the lrc CLI, I want tool results tagged with the tool name in the CLI output, so that I can identify which finding came from which tool when reviewing locally. + +#### Acceptance Criteria + +1. WHEN the `lrc` CLI renders review events and encounters an event with `event_type = 'tool_result'`, THE CLIRenderer SHALL prefix or tag each finding with the value of `data.tool_name` from the event's JSONB payload. +2. THE tag format for tool results in CLI output SHALL be `[]`, where `` is the exact string stored in `data.tool_name`. +3. WHEN there are no `tool_result` events in the review output, THE CLIRenderer SHALL NOT render any tool result section. + +--- + +### Requirement 12 – Full Echo Middleware Chain Enforcement + +**User Story:** As a security-conscious operator, I want all third-party tools API endpoints to go through the standard middleware chain, so that org isolation and role-based access are automatically enforced. + +#### Acceptance Criteria + +1. THE Server SHALL register `GET /api/v1/orgs/:org_id/tools` and `PUT /api/v1/orgs/:org_id/tools/:tool_id` under an Echo group that applies `RequireAuthOrAPIKey`, `BuildOrgContext`, `ValidateOrgAccess`, and `BuildPermissionContext` in that order. +2. WHEN any middleware in the chain returns a non-nil error, THE Server SHALL propagate the HTTP error response and SHALL NOT execute the route handler. +3. THE Server SHALL apply `apimiddleware.BuildOrgBillingPlanContext` and `apimiddleware.BuildPlanContext` to the tools endpoints to enforce the cloud billing check in addition to the role check. + +--- + +### Requirement 13 – API Documentation + +**User Story:** As a developer integrating with the tools API, I want up-to-date documentation for each phase's endpoints, so that I can understand the expected request/response formats without reading source code. + +#### Acceptance Criteria + +1. THE System SHALL maintain API documentation for all three phases in `docs/tools/tools-integration-beta.md`. +2. WHEN a new endpoint or schema change is introduced in a phase, THE System SHALL update `docs/tools/tools-integration-beta.md` to document the endpoint path, method, request body, response schema, and applicable error codes for that phase. +3. THE documentation file SHALL include at minimum: `GET /api/v1/orgs/:org_id/tools`, `PUT /api/v1/orgs/:org_id/tools/:tool_id`, the `ToolResult` event schema, and the `tool_invocation` River job schema. diff --git a/db/migrations/20260618100000_create_available_tools.sql b/db/migrations/20260618100000_create_available_tools.sql new file mode 100644 index 000000000..cbd312c3f --- /dev/null +++ b/db/migrations/20260618100000_create_available_tools.sql @@ -0,0 +1,18 @@ +-- migrate:up +CREATE TABLE IF NOT EXISTS public.available_tools ( + id bigserial PRIMARY KEY, + name text NOT NULL UNIQUE, + description text NOT NULL, + lambda_arn text NOT NULL, + multiplier numeric(6,2) NOT NULL DEFAULT 1.0, + use_case text NOT NULL DEFAULT '', + created_at timestamptz NOT NULL DEFAULT now() +); + +-- Tools are registered via the lr-tools deployer's `register-tools` command, +-- which resolves real Lambda ARNs from AWS after deployment and calls the +-- LiveReview API (POST /api/v1/admin/tools) to upsert each tool. +-- No hardcoded ARNs belong here. + +-- migrate:down +DROP TABLE IF EXISTS public.available_tools; diff --git a/db/migrations/20260618100001_create_org_tools.sql b/db/migrations/20260618100001_create_org_tools.sql new file mode 100644 index 000000000..7c308787a --- /dev/null +++ b/db/migrations/20260618100001_create_org_tools.sql @@ -0,0 +1,15 @@ +-- migrate:up +CREATE TABLE IF NOT EXISTS public.org_tools ( + org_id bigint NOT NULL REFERENCES public.orgs(id) ON DELETE CASCADE, + tool_id bigint NOT NULL REFERENCES public.available_tools(id) ON DELETE CASCADE, + enabled boolean NOT NULL DEFAULT false, + config_json jsonb NOT NULL DEFAULT '{}', + updated_at timestamptz NOT NULL DEFAULT now(), + PRIMARY KEY (org_id, tool_id) +); + +CREATE INDEX IF NOT EXISTS idx_org_tools_org_id ON public.org_tools (org_id); + +-- migrate:down +DROP INDEX IF EXISTS idx_org_tools_org_id; +DROP TABLE IF EXISTS public.org_tools; diff --git a/docs/openapi.yaml b/docs/openapi.yaml index f8369be94..d673f3372 100644 --- a/docs/openapi.yaml +++ b/docs/openapi.yaml @@ -7423,12 +7423,12 @@ paths: operationId: UpdateUser parameters: - in: path - name: org_id + name: user_id required: true schema: type: string - in: path - name: user_id + name: org_id required: true schema: type: string diff --git a/docs/tools/tools-integration-beta.md b/docs/tools/tools-integration-beta.md new file mode 100644 index 000000000..b845685b3 --- /dev/null +++ b/docs/tools/tools-integration-beta.md @@ -0,0 +1,464 @@ +# Third-Party Tools Integration – Beta + +LiveReview can run external static-analysis tools (ruff, bandit, eslint, etc.) as parallel Lambda jobs alongside every AI review. Results are stored as `tool_result` events in the existing `review_events` table and surfaced in the review UI and `lrc` CLI output. + +This feature is **cloud-only** and **owner-gated**. It is delivered in three sequential phases. + +--- + +## Table of Contents + +1. [Cost Model](#cost-model) +2. [Phase 1 – DB Schema & Settings Tab](#phase-1--db-schema--settings-tab) +3. [Phase 2 – Settings UI & API](#phase-2--settings-ui--api) +4. [Phase 3 – Queue, Lambda Trigger & Review UI](#phase-3--queue-lambda-trigger--review-ui) +5. [Shared Schemas](#shared-schemas) + +--- + +## Cost Model + +Each tool runs as an independent Lambda invocation. Cost is billed in GB-seconds at the AWS ARM64 rate (`$0.0000133334 / GB-s`). + +**Formula per tool invocation:** + +``` +cost = (memory_mb / 1024) × timeout_seconds × rate +``` + +**Credit budget:** LiveReview provides **50,000 credits** per org. One credit equals the cost of one invocation of the cheapest tool (the baseline). Orgs spend credits from this pool each time a tool runs on a review. + +### Tool catalog reference + +The table below lists available tools. `multiplier` is computed by the API (`(memory_mb / 1024) × timeout_s` relative to the cheapest tool) and returned in the `GET /api/v1/orgs/:org_id/tools` response. Tools marked **Beta** are included in the initial seed. + +| Tool | Multiplier | Use Case | +|---|---|---| +| openapi | computed | OpenAPI/YAML validation | +| actionlint | computed | GitHub Actions lint | +| shellcheck | computed | Shell script lint | +| hadolint | computed | Dockerfile lint | +| ruff | computed | Python lint/format | +| tfsec | computed | Terraform IaC | +| zizmor | computed | GitHub Actions security | +| gitleaks | computed | Secret detection | +| bandit | computed | Python SAST | +| eslint | computed | JavaScript/TypeScript SAST | +| detect-secrets | computed | Secret scanning | +| trufflehog | computed | Secret scanning (deep) | +| spectral | computed | API spec lint | +| kubescape | computed | Kubernetes IaC | +| trivy | computed | Container/IaC CVE scan | +| brakeman | computed | Ruby SAST | +| semgrep | computed | Multi-language SAST | +| golangci-lint | computed | Go SAST | + +**What users see in the UI:** tool name, multiplier tier, use case, and the running total cost per review so they can choose a tool budget that fits within their credit allowance. + +--- + +## Phase 1 – DB Schema & Settings Tab + +### DB migrations (dbmate, local only) + +Two migrations are added to `db/migrations/`. **Never apply directly to production** — use dbmate. + +#### Migration 1: `available_tools` catalog + +```sql +-- migrate:up +CREATE TABLE IF NOT EXISTS public.available_tools ( + id bigserial PRIMARY KEY, + name text NOT NULL UNIQUE, + description text NOT NULL, + lambda_arn text NOT NULL, + multiplier numeric(6,2) NOT NULL DEFAULT 1.0, + use_case text NOT NULL DEFAULT '', + created_at timestamptz NOT NULL DEFAULT now() +); + +-- Seed initial tools (ruff and bandit as the two cheapest beta tools) +INSERT INTO public.available_tools (name, description, lambda_arn, multiplier, use_case) VALUES + ('ruff', 'Fast Python linter and formatter', 'arn:aws:lambda:us-east-1:ACCOUNT:function:ruff-python-linter', 1.0, 'Python lint/format'), + ('bandit', 'Python security linter (SAST)', 'arn:aws:lambda:us-east-1:ACCOUNT:function:bandit-linter', 1.0, 'Python SAST') +ON CONFLICT (name) DO NOTHING; + +-- migrate:down +DROP TABLE IF EXISTS public.available_tools; +``` + +#### Migration 2: `org_tools` per-org selection + +```sql +-- migrate:up +CREATE TABLE IF NOT EXISTS public.org_tools ( + org_id bigint NOT NULL REFERENCES public.organizations(id) ON DELETE CASCADE, + tool_id bigint NOT NULL REFERENCES public.available_tools(id) ON DELETE CASCADE, + enabled boolean NOT NULL DEFAULT false, + config_json jsonb NOT NULL DEFAULT '{}', + updated_at timestamptz NOT NULL DEFAULT now(), + PRIMARY KEY (org_id, tool_id) +); + +CREATE INDEX IF NOT EXISTS idx_org_tools_org_id ON public.org_tools (org_id); + +-- migrate:down +DROP INDEX IF EXISTS idx_org_tools_org_id; +DROP TABLE IF EXISTS public.org_tools; +``` + +**Key design decisions:** +- `available_tools` is a global catalog — rows are added by platform operators, never by org owners. +- `org_tools` stores one row per (org, tool) pair when an org has ever interacted with that tool. Rows with `enabled = false` are stored explicitly so toggle state is preserved. +- `multiplier` on `available_tools` is denormalised from the Lambda config so the UI can display cost tiers without a live Lambda call. + +### Settings tab (UI, Phase 1 scope) + +A new tab entry is added to `ui/src/pages/Settings/Settings.tsx`: + +```typescript +// Added to the tabs array — only shown when isCloudMode() AND role is 'owner' +...(isCloudMode() && currentOrg?.role === 'owner' ? [{ + id: 'third-party-tools', + name: 'Third-Party Tools', + icon: +}] : []) +``` + +At this phase the tab renders a placeholder ("Tool configuration coming in Phase 2"). Non-owners who navigate directly to `/#/settings#third-party-tools` see a read-only message; the tab button is not shown in the sidebar. + +--- + +## Phase 2 – Settings UI & API + +### API endpoints + +Both endpoints live under the existing `orgGroup` in `server.go`, which already applies the full middleware chain: + +``` +RequireAuthOrAPIKey → BuildOrgContext → ValidateOrgAccess → BuildPermissionContext +``` + +The billing check middlewares (`BuildOrgBillingPlanContext`, `BuildPlanContext`) are also applied. + +--- + +#### `GET /api/v1/orgs/:org_id/tools` + +Returns the full available tools catalog joined with this org's enabled state. + +**Access:** any authenticated org member (owner or member). +**Cloud gate:** returns HTTP 403 if `isCloudMode()` is false on the server. +**Org isolation:** query is scoped by `org_id` from `PermissionContext`, not from the URL path parameter. + +**Response 200:** + +```json +{ + "tools": [ + { + "id": 1, + "name": "ruff", + "description": "Fast Python linter and formatter", + "multiplier": 1.0, + "use_case": "Python lint/format", + "enabled": true, + "config_json": {} + }, + { + "id": 2, + "name": "bandit", + "description": "Python security linter (SAST)", + "multiplier": 1.0, + "use_case": "Python SAST", + "enabled": false, + "config_json": {} + } + ] +} +``` + +Fields `enabled` and `config_json` default to `false` / `{}` when no `org_tools` row exists for that tool. + +**Error responses:** + +| Status | Condition | +|---|---| +| 401 | Missing or invalid auth token | +| 403 | Not cloud mode, or org mismatch | +| 500 | Database error | + +--- + +#### `PUT /api/v1/orgs/:org_id/tools/:tool_id` + +Enables or disables a specific tool for the org (upsert). + +**Access:** `owner` role only. +**Cloud gate:** HTTP 403 if not cloud mode. +**Org isolation:** upsert uses `org_id` from `PermissionContext`. + +**Request body:** + +```json +{ "enabled": true } +``` + +The `enabled` field is required and must be a boolean. Any other value returns HTTP 400. + +**Response 200:** + +```json +{ + "tool_id": 1, + "org_id": 42, + "enabled": true, + "config_json": {} +} +``` + +**Error responses:** + +| Status | Condition | +|---|---| +| 400 | `enabled` field absent or not a boolean | +| 401 | Missing or invalid auth token | +| 403 | Not cloud mode, not owner, or org mismatch | +| 404 | `tool_id` not found in `available_tools` | +| 500 | Database error | + +--- + +### Settings UI – ThirdPartyToolsTab component + +File: `ui/src/pages/Settings/ThirdPartyToolsTab.tsx` + +The tab replaces the Phase 1 placeholder. It fetches `GET /api/v1/orgs/:org_id/tools` on mount and renders a table with the following columns: + +| Column | Description | +|---|---| +| Tool name | Human-readable name | +| Use case | Short category label (e.g. "Python SAST") | +| Multiplier | Cost tier (e.g. `1×`, `3×`, `20×`) | +| Toggle | Enable/disable switch (owner only) | + +**Cost summary bar** at the top of the tab shows: +- Number of enabled tools +- Total multiplier of all enabled tools (sum) +- Estimated credits consumed per review = sum of enabled tool multipliers × baseline cost + +**Owner behaviour:** +- Toggling a tool calls `PUT /api/v1/orgs/:org_id/tools/:tool_id` immediately. +- On API error: inline error message shown, toggle reverted to previous state. +- While any request is in flight: all toggles are disabled and a spinner is shown. + +**Non-owner / member behaviour:** +- Table renders in read-only state. Toggles are replaced with a static enabled/disabled badge. +- No PUT calls are made. + +--- + +## Phase 3 – Queue, Lambda Trigger & Review UI + +### River job: `tool_invocation` + +File: `internal/jobqueue/jobqueue.go` (alongside existing `webhook_install` / `webhook_removal` jobs) + +#### Job args + +```go +type ToolInvocationJobArgs struct { + ReviewID int64 `json:"review_id"` + OrgID int64 `json:"org_id"` + ToolID int64 `json:"tool_id"` + ToolName string `json:"tool_name"` + LambdaARN string `json:"lambda_arn"` +} + +func (ToolInvocationJobArgs) Kind() string { return "tool_invocation" } +``` + +#### Worker + +```go +type ToolInvocationWorker struct { + river.WorkerDefaults[ToolInvocationJobArgs] + db *sql.DB + httpClient *http.Client +} +``` + +**Work() logic:** + +1. Load the diff from `SELECT diff FROM reviews WHERE id = $1 AND org_id = $2`. If the review has no diff, log and return without error (nothing to analyse). +2. POST the diff as the Lambda payload to the tool's `lambda_arn` via HTTPS. +3. On non-2xx response: return an error so River applies its standard retry policy. +4. On 2xx: insert a `review_events` row (see schema below). + +#### Fan-out trigger + +In `WebhookOrchestratorV2` (or the unified processor), after diff extraction completes: + +```go +enabledTools, err := store.GetEnabledToolsForOrg(ctx, orgID) +for _, tool := range enabledTools { + _, err = riverClient.Insert(ctx, ToolInvocationJobArgs{ + ReviewID: reviewID, + OrgID: orgID, + ToolID: tool.ID, + ToolName: tool.Name, + LambdaARN: tool.LambdaARN, + }, nil) +} +``` + +All jobs are inserted in a single loop — River runs them concurrently up to `MaxWorkers`. + +### Lambda payload & response + +**Payload sent to Lambda (JSON):** + +```json +{ + "review_id": 1234, + "diff": "" +} +``` + +**Expected Lambda response (JSON):** + +```json +{ + "exit_code": 0, + "findings": [ + { + "file": "src/main.py", + "line": 42, + "col": 5, + "rule": "E501", + "message": "Line too long (92 > 79 characters)" + } + ], + "lines_of_code": 312, + "stderr": "" +} +``` + +The full response body is stored verbatim in the `data` JSONB column of `review_events`. + +### `review_events` row for tool results + +No new table is needed. A new `event_type` value is added to the existing `review_events` table: + +```sql +-- No migration required — event_type is free-text. +-- New rows look like: +INSERT INTO public.review_events (review_id, org_id, event_type, data) +VALUES ( + $1, -- review_id + $2, -- org_id (from review record, never from job args directly) + 'tool_result', + '{ + "tool_id": 1, + "tool_name": "ruff", + "exit_code": 0, + "findings": [...], + "lines_of_code": 312, + "stderr": "" + }' +); +``` + +`org_id` is always read from the `reviews` row, not from the job args, to prevent any spoofing. + +### Beta review UI + +Route: `/#/reviews-tools/new` +File: `ui/src/pages/Reviews/BetaToolReviewPage.tsx` + +- Registered in the React Router config but **not** added to the sidebar or any nav surface. +- If `isCloudMode()` returns false, renders: *"Tool-based reviews are only available in cloud mode."* +- Otherwise renders a layout matching the existing AI review page (`NewReview.tsx`) with the trigger form at the top and a live event stream below. +- `tool_result` events in the stream are rendered with a coloured badge showing the tool name (e.g. `[ruff]`), followed by the findings list. + +### Tool result badges in ReviewDetail + +File: `ui/src/pages/Reviews/ReviewDetail.tsx` + +When the event stream contains events with `event_type === 'tool_result'`: + +- A **tool result section** is rendered below AI findings. +- Each tool result has a badge styled distinctly from AI badges (different colour, labelled with `data.tool_name`). +- If no `tool_result` events exist for the review, the section is hidden entirely. + +### `lrc` CLI output + +When `lrc` renders a completed review and encounters events with `event_type === 'tool_result'`: + +``` +[ruff] src/auth/login.py:42:5 E501 Line too long (92 > 79 characters) +[ruff] src/auth/login.py:78:1 F401 'os' imported but unused +[bandit] src/utils/crypto.py:12:0 B303 Use of MD5 not recommended +``` + +Tag format: `[]` followed by the finding in standard linter format. +If no `tool_result` events are present, the tool section is skipped entirely (no empty header rendered). + +--- + +## Shared Schemas + +### `tool_result` event data shape + +```json +{ + "tool_id": 1, + "tool_name": "ruff", + "exit_code": 0, + "findings": [ + { + "file": "src/main.py", + "line": 42, + "col": 5, + "rule": "E501", + "message": "Line too long (92 > 79 characters)" + } + ], + "lines_of_code": 312, + "stderr": "" +} +``` + +### `tool_invocation` River job schema + +```json +{ + "review_id": 1234, + "org_id": 42, + "tool_id": 1, + "tool_name": "ruff", + "lambda_arn": "arn:aws:lambda:us-east-1:ACCOUNT:function:ruff-python-linter" +} +``` + +### `available_tools` table + +| Column | Type | Notes | +|---|---|---| +| `id` | bigserial | PK | +| `name` | text | Unique, e.g. `ruff` | +| `description` | text | Human-readable | +| `lambda_arn` | text | Full ARN of the Lambda function | +| `multiplier` | numeric(6,2) | Cost tier relative to baseline tool | +| `use_case` | text | Short label, e.g. `Python SAST` | +| `created_at` | timestamptz | | + +### `org_tools` table + +| Column | Type | Notes | +|---|---|---| +| `org_id` | bigint | FK → `organizations.id` | +| `tool_id` | bigint | FK → `available_tools.id` | +| `enabled` | boolean | Default `false` | +| `config_json` | jsonb | Per-org tool config, default `{}` | +| `updated_at` | timestamptz | | diff --git a/internal/api/server.go b/internal/api/server.go index e51d01b7a..7f308f77a 100644 --- a/internal/api/server.go +++ b/internal/api/server.go @@ -1301,6 +1301,10 @@ func (s *Server) setupRoutes() { adminGroup.PUT("/users/:user_id/org", s.userHandlers.TransferUserToOrg) adminGroup.GET("/analytics/users", s.userHandlers.GetUserAnalytics) + // Super admin tools catalog endpoints (called by lr-tools deployer after Lambda deployment) + adminGroup.POST("/tools", s.UpsertAvailableTool) + adminGroup.GET("/tools", s.ListAvailableTools) + // Instance-wide config (SMTP, blob storage, production URL): self-hosted org // owners may also manage it, but in cloud mode this config is shared across // every tenant, so it stays super-admin-only there. @@ -1388,6 +1392,10 @@ func (s *Server) setupRoutes() { orgGroup.POST("/api-keys/:id/revoke", s.RevokeAPIKeyHandler) orgGroup.DELETE("/api-keys/:id", s.DeleteAPIKeyHandler) + // Third-party tools endpoints within org context + orgGroup.GET("/tools", s.ListOrgTools) + orgGroup.PUT("/tools/:tool_id", s.UpdateOrgTool) + // Organization creation - available to all authenticated users protectedOrgsGroup.POST("/organizations", s.orgHandlers.CreateOrganization) diff --git a/internal/api/tools_handler.go b/internal/api/tools_handler.go new file mode 100644 index 000000000..227e1c647 --- /dev/null +++ b/internal/api/tools_handler.go @@ -0,0 +1,153 @@ +package api + +import ( + "database/sql" + "net/http" + "strconv" + + "github.com/labstack/echo/v4" + "github.com/livereview/internal/api/auth" + "github.com/livereview/storage/tools" +) + +// UpsertToolRequest is the payload for POST /api/v1/admin/tools +// Called by `make register-tools` in lr-tools after Lambda deployment. +type UpsertToolRequest struct { + Name string `json:"name"` + Description string `json:"description"` + LambdaARN string `json:"lambda_arn"` + Multiplier float64 `json:"multiplier"` + UseCase string `json:"use_case"` +} + +// UpsertAvailableTool handles POST /api/v1/admin/tools +// Inserts or updates a tool in the available_tools catalog. +// Super-admin only — called by the lr-tools deployer after Lambda deployment. +func (s *Server) UpsertAvailableTool(c echo.Context) error { + var req UpsertToolRequest + if err := c.Bind(&req); err != nil { + return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid request body"}) + } + if req.Name == "" || req.LambdaARN == "" { + return c.JSON(http.StatusBadRequest, map[string]string{"error": "name and lambda_arn are required"}) + } + if req.Multiplier <= 0 { + req.Multiplier = 1.0 + } + + err := upsertAvailableTool(s.db, req) + if err != nil { + return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()}) + } + + return c.JSON(http.StatusOK, map[string]string{"status": "ok", "name": req.Name}) +} + +// ListAvailableTools handles GET /api/v1/admin/tools +// Returns all tools in the catalog — used by the Settings UI (Phase 2). +func (s *Server) ListAvailableTools(c echo.Context) error { + type ToolRow struct { + ID int64 `json:"id"` + Name string `json:"name"` + Description string `json:"description"` + LambdaARN string `json:"lambda_arn"` + Multiplier float64 `json:"multiplier"` + UseCase string `json:"use_case"` + } + + rows, err := s.db.QueryContext(c.Request().Context(), + `SELECT id, name, description, lambda_arn, multiplier, use_case + FROM available_tools + ORDER BY name`, + ) + if err != nil { + return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()}) + } + defer rows.Close() + + tools := make([]ToolRow, 0) + for rows.Next() { + var t ToolRow + if err := rows.Scan(&t.ID, &t.Name, &t.Description, &t.LambdaARN, &t.Multiplier, &t.UseCase); err != nil { + return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()}) + } + tools = append(tools, t) + } + return c.JSON(http.StatusOK, tools) +} + +func upsertAvailableTool(db *sql.DB, req UpsertToolRequest) error { + _, err := db.Exec(` + INSERT INTO available_tools (name, description, lambda_arn, multiplier, use_case) + VALUES ($1, $2, $3, $4, $5) + ON CONFLICT (name) DO UPDATE + SET description = EXCLUDED.description, + lambda_arn = EXCLUDED.lambda_arn, + multiplier = EXCLUDED.multiplier, + use_case = EXCLUDED.use_case`, + req.Name, req.Description, req.LambdaARN, req.Multiplier, req.UseCase, + ) + return err +} + +// ListOrgTools handles GET /api/v1/orgs/:org_id/tools +// Returns the org's tool configuration views. +func (s *Server) ListOrgTools(c echo.Context) error { + pc := auth.MustGetPermissionContext(c) + orgID := pc.GetOrgID() + + // Cloud gate check + if !s.deploymentConfig.IsCloud { + return c.JSON(http.StatusForbidden, map[string]string{"error": "Third-party tools are only available in cloud mode"}) + } + + store := tools.NewToolsStore(s.db) + orgTools, err := store.GetAvailableToolsForOrg(c.Request().Context(), orgID) + if err != nil { + return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()}) + } + + return c.JSON(http.StatusOK, map[string]interface{}{"tools": orgTools}) +} + +// UpdateOrgTool handles PUT /api/v1/orgs/:org_id/tools/:tool_id +// Updates the enabled state of a specific tool for the organization. +func (s *Server) UpdateOrgTool(c echo.Context) error { + pc := auth.MustGetPermissionContext(c) + if err := pc.RequireOrgOwner(); err != nil { + return c.JSON(http.StatusForbidden, map[string]string{"error": err.Error()}) + } + orgID := pc.GetOrgID() + + // Cloud gate check + if !s.deploymentConfig.IsCloud { + return c.JSON(http.StatusForbidden, map[string]string{"error": "Third-party tools are only available in cloud mode"}) + } + + toolIDStr := c.Param("tool_id") + toolID, err := strconv.ParseInt(toolIDStr, 10, 64) + if err != nil { + return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid tool_id"}) + } + + var req struct { + Enabled *bool `json:"enabled"` + } + if err := c.Bind(&req); err != nil { + return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid request body"}) + } + if req.Enabled == nil { + return c.JSON(http.StatusBadRequest, map[string]string{"error": "enabled field is required"}) + } + + store := tools.NewToolsStore(s.db) + row, err := store.UpsertOrgTool(c.Request().Context(), orgID, toolID, *req.Enabled) + if err != nil { + if err == sql.ErrNoRows { + return c.JSON(http.StatusNotFound, map[string]string{"error": "tool not found"}) + } + return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()}) + } + + return c.JSON(http.StatusOK, row) +} diff --git a/storage/tools/tools_store.go b/storage/tools/tools_store.go new file mode 100644 index 000000000..7387def4b --- /dev/null +++ b/storage/tools/tools_store.go @@ -0,0 +1,217 @@ +package tools + +import ( + "context" + "database/sql" + "encoding/json" + "fmt" +) + +type AvailableTool struct { + ID int64 `json:"id"` + Name string `json:"name"` + Description string `json:"description"` + LambdaARN string `json:"lambda_arn"` + Multiplier float64 `json:"multiplier"` + UseCase string `json:"use_case"` +} + +type OrgToolView struct { + AvailableTool + Enabled bool `json:"enabled"` + ConfigJSON json.RawMessage `json:"config_json"` +} + +type OrgToolRow struct { + OrgID int64 `json:"org_id"` + ToolID int64 `json:"tool_id"` + Enabled bool `json:"enabled"` + ConfigJSON json.RawMessage `json:"config_json"` +} + +type ToolsStore struct { + db *sql.DB +} + +func NewToolsStore(db *sql.DB) *ToolsStore { + return &ToolsStore{db: db} +} + +// GetAvailableToolsForOrg lists all available tools from the catalog, annotated with the org's enabling configuration. +func (s *ToolsStore) GetAvailableToolsForOrg(ctx context.Context, orgID int64) ([]OrgToolView, error) { + query := ` + SELECT + t.id, + t.name, + t.description, + t.lambda_arn, + t.multiplier, + t.use_case, + COALESCE(ot.enabled, false) AS enabled, + COALESCE(ot.config_json, '{}'::jsonb) AS config_json + FROM public.available_tools t + LEFT JOIN public.org_tools ot ON t.id = ot.tool_id AND ot.org_id = $1 + ORDER BY t.name + ` + rows, err := s.db.QueryContext(ctx, query, orgID) + if err != nil { + return nil, fmt.Errorf("failed to query available tools for org %d: %w", orgID, err) + } + defer rows.Close() + + var views []OrgToolView + for rows.Next() { + var v OrgToolView + var configBytes []byte + err := rows.Scan( + &v.ID, + &v.Name, + &v.Description, + &v.LambdaARN, + &v.Multiplier, + &v.UseCase, + &v.Enabled, + &configBytes, + ) + if err != nil { + return nil, fmt.Errorf("failed to scan org tool view: %w", err) + } + v.ConfigJSON = json.RawMessage(configBytes) + views = append(views, v) + } + if err = rows.Err(); err != nil { + return nil, fmt.Errorf("error iterating org tool views: %w", err) + } + return views, nil +} + +// UpsertOrgTool inserts or updates the enabling configuration of a tool for a specific organization. +func (s *ToolsStore) UpsertOrgTool(ctx context.Context, orgID, toolID int64, enabled bool) (OrgToolRow, error) { + // First check if the tool actually exists in available_tools + var exists bool + err := s.db.QueryRowContext(ctx, "SELECT EXISTS(SELECT 1 FROM public.available_tools WHERE id = $1)", toolID).Scan(&exists) + if err != nil { + return OrgToolRow{}, fmt.Errorf("failed to check tool existence: %w", err) + } + if !exists { + return OrgToolRow{}, sql.ErrNoRows + } + + query := ` + INSERT INTO public.org_tools (org_id, tool_id, enabled, config_json, updated_at) + VALUES ($1, $2, $3, '{}'::jsonb, NOW()) + ON CONFLICT (org_id, tool_id) DO UPDATE + SET enabled = EXCLUDED.enabled, + updated_at = NOW() + RETURNING org_id, tool_id, enabled, config_json + ` + var r OrgToolRow + var configBytes []byte + err = s.db.QueryRowContext(ctx, query, orgID, toolID, enabled).Scan( + &r.OrgID, + &r.ToolID, + &r.Enabled, + &configBytes, + ) + if err != nil { + return OrgToolRow{}, fmt.Errorf("failed to upsert org tool: %w", err) + } + r.ConfigJSON = json.RawMessage(configBytes) + return r, nil +} + +// GetEnabledToolsForOrg returns the catalog details of all tools that have been explicitly enabled by the org. +func (s *ToolsStore) GetEnabledToolsForOrg(ctx context.Context, orgID int64) ([]AvailableTool, error) { + query := ` + SELECT + t.id, + t.name, + t.description, + t.lambda_arn, + t.multiplier, + t.use_case + FROM public.available_tools t + JOIN public.org_tools ot ON t.id = ot.tool_id + WHERE ot.org_id = $1 AND ot.enabled = true + ORDER BY t.name + ` + rows, err := s.db.QueryContext(ctx, query, orgID) + if err != nil { + return nil, fmt.Errorf("failed to query enabled tools for org %d: %w", orgID, err) + } + defer rows.Close() + + var tools []AvailableTool + for rows.Next() { + var t AvailableTool + err := rows.Scan( + &t.ID, + &t.Name, + &t.Description, + &t.LambdaARN, + &t.Multiplier, + &t.UseCase, + ) + if err != nil { + return nil, fmt.Errorf("failed to scan enabled tool: %w", err) + } + tools = append(tools, t) + } + if err = rows.Err(); err != nil { + return nil, fmt.Errorf("error iterating enabled tools: %w", err) + } + return tools, nil +} + +// InsertToolResultEvent wraps raw Lambda response and logs it in review_events table. +func (s *ToolsStore) InsertToolResultEvent(ctx context.Context, reviewID, orgID, toolID int64, toolName string, resultJSON []byte) error { + type ToolLambdaResponse struct { + ExitCode int `json:"exit_code"` + Findings json.RawMessage `json:"findings"` + LinesOfCode int `json:"lines_of_code"` + Stderr string `json:"stderr"` + } + + var resp ToolLambdaResponse + // If unmarshaling fails or findings is nil, initialize it with empty array + if err := json.Unmarshal(resultJSON, &resp); err != nil { + resp.Stderr = fmt.Sprintf("failed to parse lambda response: %v. Raw: %s", err, string(resultJSON)) + resp.ExitCode = -1 + } + if len(resp.Findings) == 0 { + resp.Findings = json.RawMessage("[]") + } + + type ToolResultEventData struct { + ToolID int64 `json:"tool_id"` + ToolName string `json:"tool_name"` + ExitCode int `json:"exit_code"` + Findings json.RawMessage `json:"findings"` + LinesOfCode int `json:"lines_of_code"` + Stderr string `json:"stderr"` + } + + eventData := ToolResultEventData{ + ToolID: toolID, + ToolName: toolName, + ExitCode: resp.ExitCode, + Findings: resp.Findings, + LinesOfCode: resp.LinesOfCode, + Stderr: resp.Stderr, + } + + eventDataBytes, err := json.Marshal(eventData) + if err != nil { + return fmt.Errorf("failed to marshal tool result event data: %w", err) + } + + query := ` + INSERT INTO public.review_events (review_id, org_id, event_type, level, data) + VALUES ($1, $2, 'tool_result', 'info', $3) + ` + _, err = s.db.ExecContext(ctx, query, reviewID, orgID, eventDataBytes) + if err != nil { + return fmt.Errorf("failed to insert tool result review event: %w", err) + } + return nil +} diff --git a/ui/src/components/UIPrimitives.tsx b/ui/src/components/UIPrimitives.tsx index 5cbf6a5b7..b8681b5ae 100644 --- a/ui/src/components/UIPrimitives.tsx +++ b/ui/src/components/UIPrimitives.tsx @@ -926,6 +926,12 @@ export const Icons = { ), + Tools: () => ( + + + + + ), }; // ===== LAYOUT COMPONENTS ===== diff --git a/ui/src/pages/Settings/Settings.tsx b/ui/src/pages/Settings/Settings.tsx index 675076a8a..8f5f2fc49 100644 --- a/ui/src/pages/Settings/Settings.tsx +++ b/ui/src/pages/Settings/Settings.tsx @@ -12,6 +12,7 @@ import SMTPSettingsTab from './SMTPSettingsTab'; import StorageSettingsTab from './StorageSettingsTab'; import CompactionSettingsTab from './CompactionSettingsTab'; import AutoApprovePolicyTab from './AutoApprovePolicyTab'; +import ThirdPartyToolsTab from './ThirdPartyToolsTab'; import { UserManagement } from '../../components/UserManagement'; import LicenseManagement from '../Licenses/LicenseManagement'; import { useOrgContext } from '../../hooks/useOrgContext'; @@ -386,6 +387,12 @@ const Settings = () => { ) }] : []), + // Third-Party Tools tab visible only in cloud mode for org owners + ...(isCloudMode() && currentOrg?.role === 'owner' ? [{ + id: 'third-party-tools', + name: 'Third-Party Tools', + icon: + }] : []), ]; const tabIds = tabs.map(t => t.id); @@ -834,6 +841,19 @@ const Settings = () => { )} + {activeTab === 'third-party-tools' && isCloudMode() && currentOrg?.role === 'owner' && ( + + + + )} + {activeTab === 'third-party-tools' && isCloudMode() && currentOrg?.role !== 'owner' && ( + +
+ Tool configuration is only available to organization owners. +
+ + )} + {tabs.length === 0 && (
No settings available for your role right now.
diff --git a/ui/src/pages/Settings/ThirdPartyToolsTab.tsx b/ui/src/pages/Settings/ThirdPartyToolsTab.tsx new file mode 100644 index 000000000..8189a232e --- /dev/null +++ b/ui/src/pages/Settings/ThirdPartyToolsTab.tsx @@ -0,0 +1,449 @@ +import React, { useState, useEffect } from 'react'; +import { useOrgContext } from '../../hooks/useOrgContext'; +import apiClient from '../../api/apiClient'; +import { Badge, Alert } from '../../components/UIPrimitives'; + +export interface Tool { + id: number; + name: string; + description: string; + lambda_arn: string; + multiplier: number; + use_case: string; + enabled: boolean; +} + +interface ListToolsResponse { + tools: Tool[]; +} + +const ThirdPartyToolsTab: React.FC = () => { + const { currentOrg } = useOrgContext(); + const isOwner = currentOrg?.role === 'owner'; + + const [tools, setTools] = useState([]); + const [loading, setLoading] = useState(true); + const [saving, setSaving] = useState(false); + const [error, setError] = useState(null); + const [success, setSuccess] = useState(null); + const [localEnabled, setLocalEnabled] = useState>({}); + + const [sortField, setSortField] = useState<'name' | 'use_case' | 'multiplier' | 'enabled'>('name'); + const [sortDirection, setSortDirection] = useState<'asc' | 'desc'>('asc'); + const [currentPage, setCurrentPage] = useState(1); + const pageSize = 10; + + const handleSort = (field: 'name' | 'use_case' | 'multiplier' | 'enabled') => { + if (sortField === field) { + setSortDirection(prev => prev === 'asc' ? 'desc' : 'asc'); + } else { + setSortField(field); + setSortDirection('asc'); + } + setCurrentPage(1); // Reset to first page on sort change + }; + + const renderSortIcon = (field: 'name' | 'use_case' | 'multiplier' | 'enabled') => { + if (sortField !== field) { + return ( + + + + ); + } + if (sortDirection === 'asc') { + return ( + + + + ); + } + return ( + + + + ); + }; + + const sortedTools = [...tools].sort((a, b) => { + let valA: any; + let valB: any; + + if (sortField === 'name') { + valA = a.name.toLowerCase(); + valB = b.name.toLowerCase(); + } else if (sortField === 'use_case') { + valA = (a.use_case || '').toLowerCase(); + valB = (b.use_case || '').toLowerCase(); + } else if (sortField === 'multiplier') { + valA = a.multiplier; + valB = b.multiplier; + } else if (sortField === 'enabled') { + valA = localEnabled[a.id] ? 1 : 0; + valB = localEnabled[b.id] ? 1 : 0; + } + + if (valA < valB) return sortDirection === 'asc' ? -1 : 1; + if (valA > valB) return sortDirection === 'asc' ? 1 : -1; + return 0; + }); + + useEffect(() => { + loadTools(); + }, [currentOrg?.id]); + + const loadTools = async () => { + if (!currentOrg) return; + setLoading(true); + setError(null); + try { + const response = await apiClient.get(`/orgs/${currentOrg.id}/tools`); + const fetchedTools = response.tools || []; + setTools(fetchedTools); + + const initialMap: Record = {}; + fetchedTools.forEach(t => { + initialMap[t.id] = t.enabled; + }); + setLocalEnabled(initialMap); + setCurrentPage(1); // Reset page on org change + } catch (err: any) { + setError(err.message || 'Failed to load tools'); + } finally { + setLoading(false); + } + }; + + const handleToggleTool = (tool: Tool) => { + setLocalEnabled(prev => ({ + ...prev, + [tool.id]: !prev[tool.id] + })); + }; + + const handleSaveChanges = async () => { + if (!currentOrg || !isOwner) return; + + setSaving(true); + setError(null); + setSuccess(null); + + const changedTools = tools.filter(t => localEnabled[t.id] !== t.enabled); + + try { + await Promise.all( + changedTools.map(t => + apiClient.put(`/orgs/${currentOrg.id}/tools/${t.id}`, { + enabled: localEnabled[t.id] + }) + ) + ); + + setTools(prev => prev.map(t => ({ + ...t, + enabled: localEnabled[t.id] + }))); + + setSuccess('Successfully saved tool configurations'); + setTimeout(() => setSuccess(null), 3000); + } catch (err: any) { + setError(err.message || 'Failed to save tool configurations. Please reload.'); + } finally { + setSaving(false); + } + }; + + if (!currentOrg) { + return ( +
+ + Please select an organization to view tools. + +
+ ); + } + + const enabledToolsCount = tools.filter(t => localEnabled[t.id]).length; + const rawTotalMultiplier = tools.reduce((acc, t) => acc + (localEnabled[t.id] ? Number(t.multiplier) : 0), 0); + const totalMultiplier = Number(rawTotalMultiplier.toFixed(2)); + const totalCreditPool = 50000; + const estimatedReviews = totalMultiplier > 0 ? Math.floor(totalCreditPool / totalMultiplier) : 0; + const hasChanges = tools.some(t => localEnabled[t.id] !== t.enabled); + + const totalPages = Math.ceil(sortedTools.length / pageSize); + const activePage = Math.min(currentPage, Math.max(1, totalPages)); + const paginatedTools = sortedTools.slice((activePage - 1) * pageSize, activePage * pageSize); + + return ( +
+
+

Third-Party Static Analysis Tools

+

+ Enable external linters and security scanners to run concurrently as parallel Lambda functions alongside your AI reviews. +

+
+ + {/* Cost Explanation Card */} +
+

+ + + + Credit Pool & Limits +

+

+ Each tool invocation deducts credits from your organization's 50,000 credit budget. + The credit cost of a review is equal to the sum of the multipliers of all enabled tools (Total Multiplier). + Based on your current configuration, your pool allows for up to{' '} + + {totalMultiplier > 0 ? `${estimatedReviews.toLocaleString()} reviews` : 'unlimited reviews'} + {' '} + before exhausting the budget. +

+
+ + {error && ( + setError(null)}> + {error} + + )} + + {success && ( + setSuccess(null)}> + {success} + + )} + + {/* Cost Summary Bar */} +
+
+ Enabled Tools + + {enabledToolsCount} / {tools.length} + +
+
+ Total Multiplier + + {totalMultiplier.toFixed(1)}× + +
+
+ Estimated Reviews + + {totalMultiplier > 0 ? estimatedReviews.toLocaleString() : '—'} reviews (50k pool) + +
+
+ + {/* Action Toolbar */} + {tools.length > 0 && !loading && ( +
+ + {hasChanges ? ( + + + Unsaved changes + + ) : ( + + + All configurations saved + + )} + + {isOwner && ( + + )} +
+ )} + + {loading ? ( +
+
+
+

Loading available tools...

+
+
+ ) : tools.length === 0 ? ( +
+

No tools available

+

Use the admin register-tools CLI helper to populate the catalog.

+
+ ) : ( +
+ + + + + + + + + + + {paginatedTools.map((tool) => { + return ( + + + + + + + ); + })} + +
handleSort('name')} + className="p-4 text-xs font-semibold uppercase tracking-wider text-slate-400 cursor-pointer hover:bg-slate-800/80 hover:text-white transition-colors duration-200 group" + > +
+ Tool {renderSortIcon('name')} +
+ 		handleSort('use_case')} + className="p-4 text-xs font-semibold uppercase tracking-wider text-slate-400 cursor-pointer hover:bg-slate-800/80 hover:text-white transition-colors duration-200 group" + > +
+ Use Case {renderSortIcon('use_case')} +
+ 		handleSort('multiplier')} + className="p-4 text-xs font-semibold uppercase tracking-wider text-slate-400 cursor-pointer hover:bg-slate-800/80 hover:text-white transition-colors duration-200 group" + > +
+ Cost Multiplier {renderSortIcon('multiplier')} +
+ 		handleSort('enabled')} + className="p-4 text-xs font-semibold uppercase tracking-wider text-slate-400 cursor-pointer hover:bg-slate-800/80 hover:text-white transition-colors duration-200 group text-right" + > +
+ Status {renderSortIcon('enabled')} +
+
+
{tool.name}
+
{tool.description}
+ 		+ {tool.use_case || 'General'} + + {tool.multiplier.toFixed(1)}× + + {isOwner ? ( +
+ +
+ ) : ( + + {localEnabled[tool.id] ? 'Enabled' : 'Disabled'} + + )} +
+ + {/* Pagination Footer */} + {totalPages > 1 && ( +
+
+ + +
+
+
+

+ Showing {((activePage - 1) * pageSize) + 1} to{' '} + + {Math.min(activePage * pageSize, sortedTools.length)} + {' '} + of {sortedTools.length} tools +

+
+
+ +
+
+
+ )} +
+ )} +
+ ); +}; + +export default ThirdPartyToolsTab; From 5055e61e456a1ccb52290252225c956c0ffc21bb Mon Sep 17 00:00:00 2001 From: Ganesh Kumar Date: Thu, 13 Aug 2026 21:08:45 +0530 Subject: [PATCH 02/14] Implement Third-Party Tool UI and Mock Data LiveReview Pre-Commit Check: ran (iter:3, coverage:90%) --- .agents/rules/antigravity-rtk-rules.md | 32 ++ docs/tools/tools-integration-beta.md | 45 +- .../Dashboard/widgets/SystemKpiRow.tsx | 3 +- .../Dashboard/widgets/ToolsUsageWidget.tsx | 69 +++ .../components/Dashboard/widgets/registry.ts | 10 + .../components/reviews/ReviewProgressView.tsx | 7 + .../components/reviews/ToolAnalysisCard.tsx | 408 ++++++++++++++++++ 7 files changed, 567 insertions(+), 7 deletions(-) create mode 100644 .agents/rules/antigravity-rtk-rules.md create mode 100644 ui/src/components/Dashboard/widgets/ToolsUsageWidget.tsx create mode 100644 ui/src/components/reviews/ToolAnalysisCard.tsx diff --git a/.agents/rules/antigravity-rtk-rules.md b/.agents/rules/antigravity-rtk-rules.md new file mode 100644 index 000000000..bb5d20b1a --- /dev/null +++ b/.agents/rules/antigravity-rtk-rules.md @@ -0,0 +1,32 @@ +# RTK - Rust Token Killer (Google Antigravity) + +**Usage**: Token-optimized CLI proxy for shell commands. + +## Rule + +Always prefix shell commands with `rtk` to minimize token consumption. + +Examples: + +```bash +rtk git status +rtk cargo test +rtk ls src/ +rtk grep "pattern" src/ +rtk find "*.rs" . +rtk docker ps +rtk gh pr list +``` + +## Meta Commands + +```bash +rtk gain # Show token savings +rtk gain --history # Command history with savings +rtk discover # Find missed RTK opportunities +rtk proxy # Run raw (no filtering, for debugging) +``` + +## Why + +RTK filters and compresses command output before it reaches the LLM context, cutting up to 90% of the bash output on common operations. Always use `rtk ` instead of raw commands. diff --git a/docs/tools/tools-integration-beta.md b/docs/tools/tools-integration-beta.md index b845685b3..6e64eb793 100644 --- a/docs/tools/tools-integration-beta.md +++ b/docs/tools/tools-integration-beta.md @@ -6,6 +6,33 @@ This feature is **cloud-only** and **owner-gated**. It is delivered in three seq --- +## Current Status (feat/tool-integration-v2) + +**Only UI iterations are done. Backend architecture is not started.** + +What exists today: + +- The **Tool Analysis card** (`ui/src/components/reviews/ToolAnalysisCard.tsx`) renders on `ReviewDetail`. +- Three test review pages simulate the card across stages: + - `/#/reviews/test1` — all tools queued/pending. + - `/#/reviews/test2` — mixed running/completed/queued with animated spinners. + - `/#/reviews/test3` — all 15 tools completed with findings and failures. +- The mock data for these stages lives in `ui/src/pages/Reviews/ReviewDetail.tsx` (`fetchReviewDetails`, test IDs only). The card consumes the mock `toolBreakdown`, not a real API. +- The card is collapsed by default. Its toggle button shows live state (running count, findings count) so users know what to expect before expanding. + +What does **not** exist yet: + +- Database migrations (`available_tools`, `org_tools`). +- Settings UI tab and its API endpoints. +- River `tool_invocation` job and Lambda fan-out. +- `tool_result` events written to `review_events`. +- `lrc` CLI rendering of tool findings. +- Any backend integration or cost billing. + +The sections below document the **planned** backend architecture. Treat them as the design target, not as a description of the current code. + +--- + ## Table of Contents 1. [Cost Model](#cost-model) @@ -381,15 +408,21 @@ File: `ui/src/pages/Reviews/BetaToolReviewPage.tsx` - Otherwise renders a layout matching the existing AI review page (`NewReview.tsx`) with the trigger form at the top and a live event stream below. - `tool_result` events in the stream are rendered with a coloured badge showing the tool name (e.g. `[ruff]`), followed by the findings list. -### Tool result badges in ReviewDetail +### Tool Analysis & Credits Panel in ReviewDetail File: `ui/src/pages/Reviews/ReviewDetail.tsx` -When the event stream contains events with `event_type === 'tool_result'`: - -- A **tool result section** is rendered below AI findings. -- Each tool result has a badge styled distinctly from AI badges (different colour, labelled with `data.tool_name`). -- If no `tool_result` events exist for the review, the section is hidden entirely. +When a review includes third-party static analysis tool invocations (`event_type === 'tool_result'`): + +- A dedicated **Tool Analysis & Credits** card is rendered directly below the main Review Info header. +- **Top 3 KPI Summary Boxes**: + - `Total Tool Credits`: Sum of credits consumed by tool executions on this review (e.g. `2.0 Credits`). + - `Tools Executed`: Count of static analysis tools run (e.g. `2 Tools`). + - `Total Comments Generated`: Total comments/findings posted by tools (e.g. `14 Comments`). +- **Tool Summary Cards Grid**: + - Displays a grid of individual tool cards (`ruff`, `bandit`, `gitleaks`, `eslint`, etc.). + - Each card shows: **Tool Name**, **Credits Used** (e.g. `1.0 Credits`), **Comments Generated** (e.g. `14 Comments`), and **Status Badge** (e.g. Green `Clean` badge vs Amber `14 Comments` badge). +- If no `tool_result` events exist for the review, the Tool Analysis panel is hidden. ### `lrc` CLI output diff --git a/ui/src/components/Dashboard/widgets/SystemKpiRow.tsx b/ui/src/components/Dashboard/widgets/SystemKpiRow.tsx index da99db353..040f9e451 100644 --- a/ui/src/components/Dashboard/widgets/SystemKpiRow.tsx +++ b/ui/src/components/Dashboard/widgets/SystemKpiRow.tsx @@ -50,11 +50,12 @@ export const SystemKpiRow: React.FC = () => { const overview = systemOverview!; return ( -
+
} onClick={() => navigate('/git')} /> } onClick={() => navigate('/ai')} /> } onClick={() => navigate('/explore/repositories')} /> } onClick={() => navigate('/explore/merge-requests')} /> + } onClick={() => navigate('/settings#third-party-tools')} />
); }; diff --git a/ui/src/components/Dashboard/widgets/ToolsUsageWidget.tsx b/ui/src/components/Dashboard/widgets/ToolsUsageWidget.tsx new file mode 100644 index 000000000..226d3c604 --- /dev/null +++ b/ui/src/components/Dashboard/widgets/ToolsUsageWidget.tsx @@ -0,0 +1,69 @@ +import React from 'react'; +import { useNavigate } from 'react-router-dom'; +import { Button } from '../../UIPrimitives'; + +const MOCK_TOOLS_LIST = [ + { name: 'ruff', useCase: 'Python lint/format', multiplier: 1.0, status: 'enabled', findings: 0 }, + { name: 'bandit', useCase: 'Python SAST', multiplier: 1.0, status: 'enabled', findings: 14 }, + { name: 'gitleaks', useCase: 'Secret detection', multiplier: 1.0, status: 'enabled', findings: 0 }, + { name: 'eslint', useCase: 'JavaScript/TypeScript SAST', multiplier: 2.0, status: 'enabled', findings: 5 }, + { name: 'semgrep', useCase: 'Multi-language SAST', multiplier: 3.0, status: 'enabled', findings: 8 }, + { name: 'hadolint', useCase: 'Dockerfile lint', multiplier: 0.5, status: 'enabled', findings: 0 }, + { name: 'actionlint', useCase: 'GitHub Actions lint', multiplier: 0.5, status: 'enabled', findings: 2 }, + { name: 'shellcheck', useCase: 'Shell script lint', multiplier: 0.5, status: 'enabled', findings: 0 }, + { name: 'trufflehog', useCase: 'Deep secret scanning', multiplier: 2.0, status: 'enabled', findings: 0 }, + { name: 'trivy', useCase: 'Container/IaC CVE scan', multiplier: 2.5, status: 'enabled', findings: 3 }, + { name: 'spectral', useCase: 'API spec lint', multiplier: 1.0, status: 'enabled', findings: 0 }, + { name: 'brakeman', useCase: 'Ruby SAST', multiplier: 1.5, status: 'enabled', findings: 1 }, + { name: 'kubescape', useCase: 'Kubernetes IaC', multiplier: 2.0, status: 'enabled', findings: 0 }, + { name: 'zizmor', useCase: 'GitHub Actions security', multiplier: 1.0, status: 'enabled', findings: 2 }, + { name: 'openapi', useCase: 'OpenAPI/YAML validation', multiplier: 0.5, status: 'enabled', findings: 0 }, +]; + +export const ToolsUsageWidget: React.FC = () => { + const navigate = useNavigate(); + + return ( +
+
+
+ Static Analysis Tools +

15 tools enabled • 20.0× credit tier

+
+ +
+ +
+ {MOCK_TOOLS_LIST.map((tool) => ( +
+
+ {tool.name} + ({tool.useCase}) +
+ +
+ {tool.multiplier}× + + {tool.findings === 0 ? 'Clean' : `${tool.findings} findings`} + +
+
+ ))} +
+
+ ); +}; diff --git a/ui/src/components/Dashboard/widgets/registry.ts b/ui/src/components/Dashboard/widgets/registry.ts index 54b36acea..9c5670dbd 100644 --- a/ui/src/components/Dashboard/widgets/registry.ts +++ b/ui/src/components/Dashboard/widgets/registry.ts @@ -4,6 +4,7 @@ import { SystemKpiRow } from './SystemKpiRow'; import { ConnectedProviders } from './ConnectedProviders'; import { AverageReviewsStat } from './AverageReviewsStat'; import { TopReviewersLeaderboard } from './TopReviewersLeaderboard'; +import { ToolsUsageWidget } from './ToolsUsageWidget'; // The 7 echarts-based widgets are lazy-loaded (not statically imported like the ones above): // registry.ts is itself statically imported by DashboardGrid.tsx, which Dashboard.tsx statically @@ -129,6 +130,15 @@ export const WIDGET_REGISTRY: WidgetDefinition[] = [ minW: 4, minH: 6, component: ConnectedProviders, }, + { + id: 'tools-usage-widget', + title: 'Third-Party Static Analysis Tools', + category: 'system', + description: 'Enabled third-party static analysis security and linting tools.', + defaultLayout: { x: 0, y: 91, w: 12, h: 6 }, + minW: 4, minH: 5, + component: ToolsUsageWidget, + }, { id: 'coverage-gauge', title: 'Review Coverage', diff --git a/ui/src/components/reviews/ReviewProgressView.tsx b/ui/src/components/reviews/ReviewProgressView.tsx index 369b8bd2a..0047e7d43 100644 --- a/ui/src/components/reviews/ReviewProgressView.tsx +++ b/ui/src/components/reviews/ReviewProgressView.tsx @@ -613,6 +613,13 @@ export default function ReviewProgressView({ reviewId, events, isLive = false, c } }); + const finalStage = stageMap.get('finalization'); + if (finalStage && finalStage.status === 'completed') { + stageMap.forEach(s => { + s.status = 'completed'; + }); + } + return STAGE_DEFINITIONS.map(def => stageMap.get(def.key)!); }; diff --git a/ui/src/components/reviews/ToolAnalysisCard.tsx b/ui/src/components/reviews/ToolAnalysisCard.tsx new file mode 100644 index 000000000..ccc61a486 --- /dev/null +++ b/ui/src/components/reviews/ToolAnalysisCard.tsx @@ -0,0 +1,408 @@ +import React, { useState } from 'react'; +import { motion, AnimatePresence } from 'framer-motion'; +import { Icons } from '../UIPrimitives'; + +export interface ToolBreakdownItem { + toolName: string; + creditsUsed: number; + commentsGenerated: number; + status: 'pending' | 'running' | 'clean' | 'completed' | 'failed' | string; +} + +export interface ToolAccountingData { + totalToolCredits: number; + toolsExecuted: number; + totalCommentsGenerated: number; + toolBreakdown: ToolBreakdownItem[]; +} + +interface ToolAnalysisCardProps { + data: ToolAccountingData; + embedded?: boolean; + isExpanded?: boolean; + onToggle?: () => void; + hideSummary?: boolean; +} + +export const ToolAnalysisCard: React.FC = ({ data, embedded = false, isExpanded, onToggle, hideSummary = false }) => { + const [internalExpanded, setInternalExpanded] = useState(false); + // Use controlled expanded if provided, otherwise use internal state + const expanded = isExpanded !== undefined ? isExpanded : internalExpanded; + const setExpanded = onToggle !== undefined ? () => onToggle() : setInternalExpanded; + const [sortBy, setSortBy] = useState<'status' | 'name' | 'credits'>('status'); + const [statusFilter, setStatusFilter] = useState<'all' | 'findings' | 'clean' | 'running' | 'queued' | 'failed'>( + data.totalCommentsGenerated > 0 ? 'findings' : 'all' + ); + const [filterOpen, setFilterOpen] = useState(false); + + const totalTools = data.toolBreakdown.length || data.toolsExecuted; + const activeRunningTools = data.toolBreakdown.filter((t) => t.status === 'running'); + const queuedTools = data.toolBreakdown.filter((t) => t.status === 'pending'); + const completedTools = data.toolBreakdown.filter((t) => t.status !== 'running' && t.status !== 'pending'); + + const isRunning = activeRunningTools.length > 0; + const isQueued = queuedTools.length > 0 && activeRunningTools.length === 0; + const hasFindings = data.totalCommentsGenerated > 0; + const hasFailures = data.toolBreakdown.some((item) => item.status === 'failed'); + + // Count tools by status for filter tabs + const findingsCount = data.toolBreakdown.filter((t) => t.commentsGenerated > 0).length; + const cleanCount = data.toolBreakdown.filter((t) => t.commentsGenerated === 0 && t.status !== 'failed' && t.status !== 'running' && t.status !== 'pending').length; + const activeRunningCount = activeRunningTools.length; + const queuedCount = queuedTools.length; + const failedCount = data.toolBreakdown.filter((t) => t.status === 'failed').length; + + // Filter items + const filteredTools = data.toolBreakdown.filter((t) => { + if (statusFilter === 'findings') return t.commentsGenerated > 0; + if (statusFilter === 'clean') return t.commentsGenerated === 0 && t.status !== 'failed' && t.status !== 'running' && t.status !== 'pending'; + if (statusFilter === 'running') return t.status === 'running'; + if (statusFilter === 'queued') return t.status === 'pending'; + if (statusFilter === 'failed') return t.status === 'failed'; + return true; + }); + + // Sort items + const sortedTools = [...filteredTools].sort((a, b) => { + if (sortBy === 'name') { + return a.toolName.localeCompare(b.toolName); + } + if (sortBy === 'credits') { + return b.creditsUsed - a.creditsUsed; + } + // Default: Sort by Status priority (Findings > Failed > Running > Pending > Clean) + const getPriority = (item: ToolBreakdownItem) => { + if (item.commentsGenerated > 0) return 1; + if (item.status === 'failed') return 2; + if (item.status === 'running') return 3; + if (item.status === 'pending') return 4; + return 5; + }; + return getPriority(a) - getPriority(b); + }); + + return ( +
+ {/* Summary Bar */} + {!hideSummary && ( +
+
+ {/* Dynamic Status Icon */} + {isRunning ? ( +
+ + + + +
+ ) : isQueued ? ( +
+ + + +
+ ) : hasFindings ? ( +
+ + + +
+ ) : ( +
+ + + +
+ )} + +
+
+

+ Static Analysis Tools +

+ + {/* Dynamic Status Badge */} + {isRunning ? ( + + Running ({completedTools.length}/{totalTools}) + + ) : isQueued ? ( + + Queued (0/{totalTools}) + + ) : ( + + {hasFindings ? `${data.totalCommentsGenerated} Findings Flagged` : 'All Clean'} + + )} +
+
+
+ + {/* Dynamic Metric Summary */} +
+
+ {totalTools} Tools + • + {data.totalCommentsGenerated} Findings + • + {data.totalToolCredits.toFixed(1)} cr +
+
+
+ )} + + {/* Expanded Multi-Column Panel with Framer Motion Animation */} + + {expanded && ( + + {/* Controls Bar: Grouped Filter & Sort Pills Aligned to the Right */} +
+ {/* Filter Pills */} +
+ Filter: + + + + {findingsCount > 0 && ( + + )} + + {activeRunningCount > 0 && ( + + )} + + {queuedCount > 0 && ( + + )} + + {cleanCount > 0 && ( + + )} + + {failedCount > 0 && ( + + )} +
+ + {/* Subtle Vertical Separator */} + • + + {/* Sort Pills */} +
+ Sort: + + + +
+
+ + {/* Dynamic Tool Grid or Empty State */} + {(() => { + const handleToolClick = (item: ToolBreakdownItem) => { + if (item.commentsGenerated > 0 || item.status === 'failed') { + const el = document.getElementById('review-events-section'); + if (el) { + el.scrollIntoView({ behavior: 'smooth' }); + } + } + }; + + if (sortedTools.length === 0) { + return ( +
+

+ No issues found from static analysis tools matching filter "{statusFilter}". +

+ +
+ ); + } + + return ( + + {sortedTools.map((item) => { + const isProblem = item.commentsGenerated > 0 || item.status === 'failed'; + const isClean = item.commentsGenerated === 0 && item.status !== 'failed' && item.status !== 'running' && item.status !== 'pending'; + const isToolRunning = item.status === 'running'; + const isToolPending = item.status === 'pending'; + + return ( + handleToolClick(item)} + title={isProblem ? `Click to view ${item.toolName} findings` : undefined} + className={`rounded-lg p-3 flex items-center justify-between transition-all ${ + isProblem + ? 'bg-slate-800/90 border border-amber-600/60 shadow-md shadow-amber-950/20 cursor-pointer hover:border-amber-500 hover:scale-[1.01]' + : 'bg-slate-800/60 hover:bg-slate-800 border border-slate-700/50' + }`} + > +
+

{item.toolName}

+

{item.creditsUsed.toFixed(1)} cr

+
+ + {/* Per-Tool State Badge */} + {isToolRunning ? ( + + + + + + Running + + ) : isToolPending ? ( + + Queued + + ) : ( + + {isClean ? 'Clean' : `⚠️ ${item.commentsGenerated} findings`} + + )} + + ); + })} + + ); + })()} + + )} + +
+ ); +}; + + + + + + From 2fb705015f3cc192b395bfd14c7548d0a6b17073 Mon Sep 17 00:00:00 2001 From: Ganesh Kumar Date: Sun, 16 Aug 2026 11:57:05 +0530 Subject: [PATCH 03/14] Standardize Relative Time Display and Refactor Review Detail UI --- ui/src/api/reviews.ts | 28 ++++++---------------------- ui/src/components/UIPrimitives.tsx | 25 +++++++++++++++++++++++++ ui/tsconfig.json | 3 +-- 3 files changed, 32 insertions(+), 24 deletions(-) diff --git a/ui/src/api/reviews.ts b/ui/src/api/reviews.ts index 885c0f7ee..1a42e05f2 100644 --- a/ui/src/api/reviews.ts +++ b/ui/src/api/reviews.ts @@ -1,4 +1,5 @@ import apiClient from './apiClient'; +import { formatDistanceToNow } from 'date-fns'; import { Review, ReviewsListResponse, @@ -215,28 +216,11 @@ export const getReviewCommits = async (reviewId: number): Promise = ({ tabs, activeTab, onChange, className
); }; + +/** + * RelativeTime — shows a human-readable relative timestamp ("3 minutes ago") + * powered by date-fns, with a native title tooltip showing the exact date. + * + * Usage: + */ +export const RelativeTime: React.FC<{ + timestamp: string; + className?: string; +}> = ({ timestamp, className }) => { + let relative = timestamp; + let exact = timestamp; + try { + const d = new Date(timestamp); + relative = formatDistanceToNow(d, { addSuffix: true }); + exact = format(d, 'PPpp'); // e.g. "Aug 16, 2026, 11:15:18 AM" + } catch { /* leave as-is */ } + return ( + + {relative} + + ); +}; diff --git a/ui/tsconfig.json b/ui/tsconfig.json index e48143731..438624206 100644 --- a/ui/tsconfig.json +++ b/ui/tsconfig.json @@ -14,8 +14,7 @@ "resolveJsonModule": true, "esModuleInterop": true , // "suppressImplicitAnyIndexErrors": true, - "typeRoots": [ - "./src/types", + "typeRoots": [ "./node_modules/@types" ], "lib": [ "es2015", "dom" ], From ca41c0cf2b8bc51b06901443b190f680973caa07 Mon Sep 17 00:00:00 2001 From: Ganesh Kumar Date: Tue, 18 Aug 2026 21:51:44 +0530 Subject: [PATCH 04/14] Refactor Review Details Layout and Add Tool Pagination --- .../components/reviews/ToolAnalysisCard.tsx | 38 ++++++++++++++++++- 1 file changed, 37 insertions(+), 1 deletion(-) diff --git a/ui/src/components/reviews/ToolAnalysisCard.tsx b/ui/src/components/reviews/ToolAnalysisCard.tsx index ccc61a486..68600071d 100644 --- a/ui/src/components/reviews/ToolAnalysisCard.tsx +++ b/ui/src/components/reviews/ToolAnalysisCard.tsx @@ -34,6 +34,8 @@ export const ToolAnalysisCard: React.FC = ({ data, embedd data.totalCommentsGenerated > 0 ? 'findings' : 'all' ); const [filterOpen, setFilterOpen] = useState(false); + const [page, setPage] = useState(0); + const PAGE_SIZE = 8; const totalTools = data.toolBreakdown.length || data.toolsExecuted; const activeRunningTools = data.toolBreakdown.filter((t) => t.status === 'running'); @@ -81,6 +83,10 @@ export const ToolAnalysisCard: React.FC = ({ data, embedd return getPriority(a) - getPriority(b); }); + const totalPages = Math.ceil(sortedTools.length / PAGE_SIZE); + const safePage = Math.min(page, Math.max(0, totalPages - 1)); + const pagedTools = sortedTools.slice(safePage * PAGE_SIZE, safePage * PAGE_SIZE + PAGE_SIZE); + return (
= ({ data, embedd } return ( + <> - {sortedTools.map((item) => { + {pagedTools.map((item) => { const isProblem = item.commentsGenerated > 0 || item.status === 'failed'; const isClean = item.commentsGenerated === 0 && item.status !== 'failed' && item.status !== 'running' && item.status !== 'pending'; const isToolRunning = item.status === 'running'; @@ -392,6 +399,35 @@ export const ToolAnalysisCard: React.FC = ({ data, embedd ); })} + + {/* Pagination controls */} + {totalPages > 1 && ( +
+ + + {safePage + 1} / {totalPages} + ({sortedTools.length} tools) + + +
+ )} + ); })()} From 28c321cba262cb2eb0084820a9b049aedb48466b Mon Sep 17 00:00:00 2001 From: Ganesh Kumar Date: Tue, 18 Aug 2026 21:56:09 +0530 Subject: [PATCH 05/14] Redesign Review Detail Data Fetching --- docs/tools/tools-integration-beta.md | 120 +++++++++++++++++++++++++++ 1 file changed, 120 insertions(+) diff --git a/docs/tools/tools-integration-beta.md b/docs/tools/tools-integration-beta.md index 6e64eb793..8715c88cc 100644 --- a/docs/tools/tools-integration-beta.md +++ b/docs/tools/tools-integration-beta.md @@ -495,3 +495,123 @@ If no `tool_result` events are present, the tool section is skipped entirely (no | `enabled` | boolean | Default `false` | | `config_json` | jsonb | Per-org tool config, default `{}` | | `updated_at` | timestamptz | | + +--- + +## ReviewDetail Header – Data Requirements & API Design + +### Problem: Current page makes 5 serial/parallel API calls on load + +``` +Promise.all([ + GET /api/v1/reviews/:id → Review row + GET /api/v1/reviews/:id/events → up to 1000 events (huge payload, used only for severity counts + events tab) + GET /api/v1/reviews/:id/summary → batchCount, lastActivity +]) ++ sequential: + GET /api/v1/reviews/:id/accounting → cost, tokens (used only for Accounting tab + tool summary) + GET /api/v1/reviews/:id/commits → commit SHAs +``` + +Severity counts (High/Medium/Low) are computed client-side by scanning 1000 events — a payload fetched solely to count three numbers. The accounting call is a round trip just to show `$0.04` in the header. + +--- + +### What the header needs to display + +| UI element | Data field | Current source | +|---|---|---| +| Repo name | `review.repository` | `GET /reviews/:id` | +| Branch | `review.branch` | `GET /reviews/:id` | +| Provider icon | `review.provider` | `GET /reviews/:id` | +| PR/MR link | `review.prMrUrl` | `GET /reviews/:id` | +| MR Title | `review.mrTitle` | `GET /reviews/:id` (field exists, not displayed) | +| Status badge | `review.status` | `GET /reviews/:id` | +| Created by | `review.userEmail` | `GET /reviews/:id` | +| Created at | `review.createdAt` | `GET /reviews/:id` | +| Author name | `review.authorName` | `GET /reviews/:id` (field exists, not displayed) | +| Trigger type | `review.triggerType` | `GET /reviews/:id` (field exists, not displayed) | +| Severity: High | computed from events | `GET /reviews/:id/events` (1000 items) ← **expensive** | +| Severity: Medium | computed from events | same | +| Severity: Low | computed from events | same | +| Tools executed | accounting-derived | `GET /reviews/:id/accounting` ← **separate call** | +| Total findings | accounting-derived | same | +| Total cost (USD) | `accounting.totalCostUsd` | same | +| Batch count | `summary.batchCount` | `GET /reviews/:id/summary` | +| Last activity | `summary.lastActivity` | `GET /reviews/:id/summary` | +| Duration | `review.startedAt` + `review.completedAt` | `GET /reviews/:id` | +| Commits list | `commits[]` | `GET /reviews/:id/commits` | +| Tool breakdown | accounting-derived | `GET /reviews/:id/accounting` | + +--- + +### Proposed: 2-call design + +#### Call 1 — Eager, on page load + +**Enrich `GET /api/v1/reviews/:id/summary`** to be the single source of truth for the header. The backend computes severity counts and pulls tool/cost aggregates from existing tables in one query, rather than making the client do 3 round trips. + +**Proposed enriched response shape:** + +```json +{ + "reviewId": 123, + "currentStatus": "completed", + "lastActivity": "2026-08-17T16:10:00Z", + "batchCount": 4, + "eventCounts": { "log": 45, "batch": 4, "tool_result": 15 }, + + "severityCounts": { + "high": 0, + "medium": 1, + "low": 11 + }, + + "toolSummary": { + "toolsExecuted": 15, + "totalCommentsGenerated": 11, + "totalCostUsd": 0.042, + "toolBreakdown": [ + { "toolName": "ruff", "creditsUsed": 1.0, "commentsGenerated": 3, "status": "completed" }, + { "toolName": "bandit", "creditsUsed": 1.0, "commentsGenerated": 0, "status": "clean" }, + { "toolName": "gitleaks", "creditsUsed": 1.0, "commentsGenerated": 8, "status": "completed" } + ] + } +} +``` + +**Backend implementation notes:** + +- `severityCounts` — computed with a single `SELECT level, COUNT(*) FROM review_events WHERE review_id = $1 AND org_id = $2 AND event_type = 'tool_result' GROUP BY level` (or equivalent severity field on the data JSONB). If severity is embedded in `data->>'level'`, use a `jsonb` extraction in the GROUP BY. +- `toolSummary` — aggregated from `review_events` rows where `event_type = 'tool_result'`, pulling `data->>'tool_name'`, `data->>'exit_code'`, and `jsonb_array_length(data->'findings')` per row. No join to `accounting` needed for the header. +- `totalCostUsd` — pulled from `review_accounting` (existing table) if present; `null` if not yet recorded. + +#### Call 2 — Lazy, only when Events tab is opened + +``` +GET /api/v1/reviews/:id/events?limit=1000 +``` + +Events are **not** needed until the user opens the Events tab or Findings tab. Defer this call entirely. No severity pre-computation needed on the client. + +#### Commits — unchanged (best-effort, non-blocking) + +``` +GET /api/v1/reviews/:id/commits +``` + +Already fire-and-forget. Keep as is. + +--- + +### Summary of calls after redesign + +| Call | When | Purpose | +|---|---|---| +| `GET /reviews/:id` | eager | Review identity, status, PR info | +| `GET /reviews/:id/summary` (enriched) | eager | All header aggregates: severity, tools, cost, batches | +| `GET /reviews/:id/commits` | eager, non-blocking | Commit list in Details panel | +| `GET /reviews/:id/events` | lazy (Events tab) | Full event stream | +| `GET /reviews/:id/accounting` | lazy (Accounting tab only) | Detailed token/LOC breakdown | + +**Result:** header renders completely from 2 parallel calls instead of 5. The 1000-event payload is deferred until actually needed. From ad5f00ebcd1688783b4aedf1be3579762768846f Mon Sep 17 00:00:00 2001 From: Ganesh Kumar Date: Fri, 21 Aug 2026 21:50:58 +0530 Subject: [PATCH 06/14] Display Tool Status and Severity Counts in Review Summaries --- docs/tools/tools-integration-beta.md | 221 ++++++++++++++++++ internal/api/diff_review.go | 6 +- internal/api/polling_event_service.go | 33 ++- internal/api/polling_event_service_test.go | 42 +++- internal/api/review_events_repo.go | 9 + internal/review_processor/events.go | 212 ++++++++++++++++- ui/src/components/UIPrimitives.tsx | 4 +- .../components/reviews/ToolAnalysisCard.tsx | 4 +- ui/src/types/reviews.ts | 20 ++ 9 files changed, 524 insertions(+), 27 deletions(-) diff --git a/docs/tools/tools-integration-beta.md b/docs/tools/tools-integration-beta.md index 8715c88cc..ddbc0b700 100644 --- a/docs/tools/tools-integration-beta.md +++ b/docs/tools/tools-integration-beta.md @@ -614,4 +614,225 @@ Already fire-and-forget. Keep as is. | `GET /reviews/:id/events` | lazy (Events tab) | Full event stream | | `GET /reviews/:id/accounting` | lazy (Accounting tab only) | Detailed token/LOC breakdown | + **Result:** header renders completely from 2 parallel calls instead of 5. The 1000-event payload is deferred until actually needed. + +--- + +## Tool Status Lifecycle – How Each Tool's Status is Tracked + +### The core problem + +A `tool_result` event is only written **when a Lambda returns**. There is no event for "this tool was dispatched" or "this tool is currently running". Without a record of what tools were dispatched, the UI has no way to distinguish between: + +- A tool that hasn't started yet (`pending`) +- A tool that is actively running (`running`) +- A review that never had that tool enabled at all + +This section defines the two-event approach that solves this without a new table. + +--- + +### Two event types for tools + +#### 1. `tool_dispatch` — written at fan-out time + +When `WebhookOrchestratorV2` inserts River jobs for the enabled tools, it **also** writes one `tool_dispatch` event per tool into `review_events` immediately: + +```sql +INSERT INTO public.review_events (review_id, org_id, event_type, data) +VALUES ( + $1, + $2, + 'tool_dispatch', + '{"tool_id": 1, "tool_name": "ruff", "status": "pending"}' +); +``` + +This gives the UI a complete, authoritative list of **every tool that was launched for this review**, even before any results come back. + +Go insert (inside the fan-out loop, same transaction as the River job inserts): + +```go +for _, tool := range enabledTools { + _, err = riverClient.Insert(ctx, ToolInvocationJobArgs{...}, nil) + // immediately record the dispatch + _, err = store.InsertReviewEvent(ctx, InsertReviewEventParams{ + ReviewID: reviewID, + OrgID: orgID, + EventType: "tool_dispatch", + Data: json.RawMessage(fmt.Sprintf( + `{"tool_id":%d,"tool_name":%q,"status":"pending"}`, + tool.ID, tool.Name, + )), + }) +} +``` + +#### 2. `tool_result` — written when Lambda returns + +When the River worker receives the Lambda response, it writes a `tool_result` event (already defined in Phase 3): + +```json +{ + "tool_id": 1, + "tool_name": "ruff", + "exit_code": 0, + "findings": [...], + "lines_of_code": 312, + "stderr": "" +} +``` + +--- + +### Status derivation rules + +The frontend (and the enriched summary endpoint) derives per-tool status by **joining dispatch events with result events** for the same `tool_name`: + +| Condition | Derived status | +|---|---| +| `tool_dispatch` exists, no `tool_result` yet | `pending` | +| `tool_dispatch` exists, River job is running (no result yet, time elapsed) | `running` (approximated by age — see note below) | +| `tool_result` exists, `exit_code = 0`, `findings = []` | `clean` | +| `tool_result` exists, `exit_code = 0`, `findings.length > 0` | `completed` | +| `tool_result` exists, `exit_code != 0` | `failed` | +| River job exhausted retries → worker writes a synthetic `tool_result` with `exit_code: -1` | `failed` | + +> **Running approximation:** The exact `running` status requires querying River's internal `river_jobs` table, which is brittle. Instead: if a `tool_dispatch` event is older than N seconds and no `tool_result` has appeared, the summary endpoint classifies it as `running`. A reasonable threshold is 10 seconds (most Lambda cold starts complete within 5s). This is an approximation — the UI already handles the ambiguity gracefully by showing a spinner for both `pending` and `running`. + +--- + +### SQL for the enriched summary endpoint + +The `GetReviewSummary` handler builds `toolSummary` with two queries: + +**Query 1 — dispatched tools (complete list):** + +```sql +SELECT + data->>'tool_name' AS tool_name, + data->>'tool_id' AS tool_id, + created_at +FROM review_events +WHERE review_id = $1 + AND org_id = $2 + AND event_type = 'tool_dispatch' +ORDER BY created_at ASC; +``` + +**Query 2 — completed results:** + +```sql +SELECT + data->>'tool_name' AS tool_name, + (data->>'exit_code')::int AS exit_code, + jsonb_array_length(data->'findings') AS finding_count, + data->>'stderr' AS stderr +FROM review_events +WHERE review_id = $1 + AND org_id = $2 + AND event_type = 'tool_result' +ORDER BY created_at ASC; +``` + +The handler merges the two result sets in Go: + +```go +// resultMap: tool_name → tool_result row +resultMap := map[string]ToolResultRow{} +for _, r := range results { + resultMap[r.ToolName] = r +} + +breakdown := []ToolBreakdownItem{} +for _, d := range dispatched { + r, done := resultMap[d.ToolName] + item := ToolBreakdownItem{ToolName: d.ToolName} + + if !done { + age := time.Since(d.CreatedAt) + if age > 10*time.Second { + item.Status = "running" + } else { + item.Status = "pending" + } + } else if r.ExitCode != 0 { + item.Status = "failed" + } else if r.FindingCount > 0 { + item.Status = "completed" + item.CommentsGenerated = r.FindingCount + } else { + item.Status = "clean" + } + + breakdown = append(breakdown, item) +} +``` + +**Credits** per tool come from `available_tools.multiplier` joined by `tool_id` from the dispatch events — no Lambda call needed. + +--- + +### What happens when River exhausts retries (error path) + +If the Lambda call fails after all River retries, the worker writes a synthetic failure event before returning: + +```go +// In ToolInvocationWorker.Work(), after final retry failure: +store.InsertReviewEvent(ctx, InsertReviewEventParams{ + ReviewID: args.ReviewID, + OrgID: orgID, // always from review row, never job args + EventType: "tool_result", + Data: json.RawMessage(fmt.Sprintf( + `{"tool_id":%d,"tool_name":%q,"exit_code":-1,"findings":[],"stderr":"Lambda invocation failed after retries"}`, + args.ToolID, args.ToolName, + )), +}) +``` + +This ensures the UI never shows a tool stuck in `pending` forever — it will transition to `failed` once River gives up. + +--- + +### Frontend: building `ToolAccountingData` from real API data + +Once the enriched summary endpoint is live, the frontend replaces the mock `setToolAccounting(...)` calls with a direct mapping from `summary.toolSummary`: + +```typescript +// In fetchReviewDetails, after getReviewSummary(): +if (summaryData.toolSummary) { + setToolAccounting({ + totalToolCredits: summaryData.toolSummary.totalCostUsd ?? 0, + toolsExecuted: summaryData.toolSummary.toolsExecuted, + totalCommentsGenerated: summaryData.toolSummary.totalCommentsGenerated, + toolBreakdown: summaryData.toolSummary.toolBreakdown, + }); +} else { + setToolAccounting(null); // hides the tools tab if no tools ran +} +``` + +The `status` field on each `ToolBreakdownItem` comes directly from the backend merge logic above — the frontend does not re-derive it. + +--- + +### Summary: full status lifecycle + +``` +Fan-out trigger + │ + ├─ INSERT tool_dispatch (status: "pending") ← UI sees: pending + └─ riverClient.Insert(ToolInvocationJobArgs) + │ + ├─ [job starts running] ← UI sees: running (age > 10s) + │ + ├─ Lambda returns 200 + │ └─ INSERT tool_result (exit_code, findings) + │ ├─ findings > 0 ← UI sees: completed + │ └─ findings = 0 ← UI sees: clean + │ + └─ Lambda fails / retries exhausted + └─ INSERT tool_result (exit_code: -1) + ← UI sees: failed +``` diff --git a/internal/api/diff_review.go b/internal/api/diff_review.go index 2c1aabbc6..8e0e2f842 100644 --- a/internal/api/diff_review.go +++ b/internal/api/diff_review.go @@ -268,7 +268,11 @@ func (s *Server) GetDiffReviewStatus(c echo.Context) error { result, err := decodeReviewResult(meta) if err != nil { - return JSONErrorWithEnvelope(c, http.StatusInternalServerError, fmt.Sprintf("failed to decode review result: %v", err)) + // For tools-only reviews or reviews without AI comments, return an empty result gracefully instead of failing + result = DiffReviewResult{ + Summary: "Static analysis tools review completed.", + Comments: []*models.ReviewComment{}, + } } files := buildDiffFiles(preloaded, result.Comments) diff --git a/internal/api/polling_event_service.go b/internal/api/polling_event_service.go index d495ef43a..ed274341e 100644 --- a/internal/api/polling_event_service.go +++ b/internal/api/polling_event_service.go @@ -236,11 +236,24 @@ func (s *PollingEventService) GetReviewSummary(ctx context.Context, reviewID, or if err != nil { return nil, fmt.Errorf("failed to count batch IDs: %w", err) } + + sevCounts, err := s.repo.GetSeverityCounts(ctx, reviewID, orgID) + if err != nil { + sevCounts = SeverityCounts{} + } + + toolSum, err := s.repo.GetToolSummary(ctx, reviewID, orgID) + if err != nil { + toolSum = nil + } + summary := &ReviewSummary{ - ReviewID: reviewID, - LastActivity: time.Now(), // Will be updated with actual latest event - EventCounts: counts, - BatchCount: batchCount, + ReviewID: reviewID, + LastActivity: time.Now(), + EventCounts: counts, + BatchCount: batchCount, + SeverityCounts: sevCounts, + ToolSummary: toolSum, } // Parse latest status if available @@ -257,9 +270,11 @@ func (s *PollingEventService) GetReviewSummary(ctx context.Context, reviewID, or // ReviewSummary provides a quick overview of review progress type ReviewSummary struct { - ReviewID int64 `json:"reviewId"` - CurrentStatus string `json:"currentStatus"` - LastActivity time.Time `json:"lastActivity"` - EventCounts map[string]int `json:"eventCounts"` - BatchCount int `json:"batchCount"` + ReviewID int64 `json:"reviewId"` + CurrentStatus string `json:"currentStatus"` + LastActivity time.Time `json:"lastActivity"` + EventCounts map[string]int `json:"eventCounts"` + BatchCount int `json:"batchCount"` + SeverityCounts SeverityCounts `json:"severityCounts"` + ToolSummary *ToolSummary `json:"toolSummary,omitempty"` } diff --git a/internal/api/polling_event_service_test.go b/internal/api/polling_event_service_test.go index 7af378ed4..93d36a577 100644 --- a/internal/api/polling_event_service_test.go +++ b/internal/api/polling_event_service_test.go @@ -3,6 +3,7 @@ package api import ( "context" "database/sql" + "os" "testing" "time" @@ -17,13 +18,25 @@ func TestPollingEventService(t *testing.T) { t.Skip("Skipping database integration test") } + dbURL := os.Getenv("TEST_DATABASE_URL") + if dbURL == "" { + dbURL = os.Getenv("DATABASE_URL") + } + if dbURL == "" { + dbURL = "postgres://livereview:livereview_password_123@localhost:5432/livereview?sslmode=disable" + } + // Connect to test database - db, err := sql.Open("postgres", "postgres://livereview:livereview_password_123@localhost:5432/livereview?sslmode=disable") + db, err := sql.Open("postgres", dbURL) require.NoError(t, err) defer db.Close() - service := NewPollingEventService(db) ctx := context.Background() + if err := db.PingContext(ctx); err != nil { + t.Skip("Skipping database integration test: database not accessible") + } + + service := NewPollingEventService(db) orgID := int64(1) // Create a test review record first @@ -103,6 +116,20 @@ func TestPollingEventService(t *testing.T) { }) t.Run("GetReviewSummary", func(t *testing.T) { + // Emit tool_dispatch and tool_result events for testing enriched summary + _ = service.EmitEvent(ctx, &ReviewEvent{ + ReviewID: reviewID, + OrgID: orgID, + EventType: "tool_dispatch", + Data: []byte(`{"tool_id": 1, "tool_name": "ruff", "status": "pending"}`), + }) + _ = service.EmitEvent(ctx, &ReviewEvent{ + ReviewID: reviewID, + OrgID: orgID, + EventType: "tool_result", + Data: []byte(`{"tool_id": 1, "tool_name": "ruff", "exit_code": 0, "findings": [{"file": "main.py", "line": 10, "message": "Line too long"}]}`), + }) + summary, err := service.GetReviewSummary(ctx, reviewID, orgID) require.NoError(t, err) require.NotNil(t, summary) @@ -114,17 +141,26 @@ func TestPollingEventService(t *testing.T) { assert.Contains(t, summary.EventCounts, "batch") assert.Contains(t, summary.EventCounts, "completion") assert.Equal(t, 1, summary.BatchCount) + assert.NotNil(t, summary.SeverityCounts) + assert.NotNil(t, summary.ToolSummary) + assert.Equal(t, 1, summary.ToolSummary.ToolsExecuted) + assert.Equal(t, 1, summary.ToolSummary.TotalCommentsGenerated) + assert.Len(t, summary.ToolSummary.ToolBreakdown, 1) + assert.Equal(t, "ruff", summary.ToolSummary.ToolBreakdown[0].ToolName) + assert.Equal(t, "completed", summary.ToolSummary.ToolBreakdown[0].Status) }) t.Run("GetEventCounts", func(t *testing.T) { counts, err := service.GetEventCounts(ctx, reviewID, orgID) require.NoError(t, err) - // We should have created: 1 status, 1 log, 1 batch, 1 completion + // We should have created: 1 status, 1 log, 1 batch, 1 completion, 1 tool_dispatch, 1 tool_result assert.Equal(t, 1, counts["status"]) assert.Equal(t, 1, counts["log"]) assert.Equal(t, 1, counts["batch"]) assert.Equal(t, 1, counts["completion"]) + assert.Equal(t, 1, counts["tool_dispatch"]) + assert.Equal(t, 1, counts["tool_result"]) }) // Clean up test data diff --git a/internal/api/review_events_repo.go b/internal/api/review_events_repo.go index 2a5835fa7..eac939ba7 100644 --- a/internal/api/review_events_repo.go +++ b/internal/api/review_events_repo.go @@ -18,6 +18,15 @@ type ReviewEventsRepo = reviewprocessor.ReviewEventsRepo // ListEventsCursor represents pagination cursor for events, aliased from reviewprocessor type ListEventsCursor = reviewprocessor.ListEventsCursor +// SeverityCounts represents counts of events by severity level, aliased from reviewprocessor +type SeverityCounts = reviewprocessor.SeverityCounts + +// ToolBreakdownItem represents status and findings for an individual tool, aliased from reviewprocessor +type ToolBreakdownItem = reviewprocessor.ToolBreakdownItem + +// ToolSummary represents summary metrics and breakdown for tool execution, aliased from reviewprocessor +type ToolSummary = reviewprocessor.ToolSummary + // NewReviewEventsRepo creates a new review events repository using the reviewprocessor implementation func NewReviewEventsRepo(db *sql.DB) *ReviewEventsRepo { return reviewprocessor.NewReviewEventsRepo(db) diff --git a/internal/review_processor/events.go b/internal/review_processor/events.go index 69eb4077b..beea6286d 100644 --- a/internal/review_processor/events.go +++ b/internal/review_processor/events.go @@ -554,6 +554,184 @@ func (s *PollingEventService) CreateCompletionEvent(ctx context.Context, reviewI return s.repo.createTypedEvent(ctx, reviewID, orgID, "completion", "info", nil, data) } +// SeverityCounts provides counts of events by severity level +type SeverityCounts struct { + High int `json:"high"` + Medium int `json:"medium"` + Low int `json:"low"` +} + +// ToolBreakdownItem represents status and findings for an individual tool +type ToolBreakdownItem struct { + ToolName string `json:"toolName"` + CreditsUsed float64 `json:"creditsUsed"` + CommentsGenerated int `json:"commentsGenerated"` + Status string `json:"status"` // pending, running, clean, completed, failed +} + +// ToolSummary provides summary metrics and breakdown for tool execution +type ToolSummary struct { + ToolsExecuted int `json:"toolsExecuted"` + TotalCommentsGenerated int `json:"totalCommentsGenerated"` + TotalCostUsd *float64 `json:"totalCostUsd,omitempty"` + ToolBreakdown []ToolBreakdownItem `json:"toolBreakdown"` +} + +// GetSeverityCounts queries review_events to count High, Medium, and Low severity events +func (r *ReviewEventsRepo) GetSeverityCounts(ctx context.Context, reviewID, orgID int64) (SeverityCounts, error) { + query := ` + SELECT COALESCE(level, 'info') as lvl, COUNT(*) + FROM public.review_events + WHERE review_id = $1 AND org_id = $2 + GROUP BY COALESCE(level, 'info') + ` + rows, err := r.db.QueryContext(ctx, query, reviewID, orgID) + if err != nil { + return SeverityCounts{}, fmt.Errorf("failed to query severity counts: %w", err) + } + defer rows.Close() + + var counts SeverityCounts + for rows.Next() { + var lvl string + var cnt int + if err := rows.Scan(&lvl, &cnt); err != nil { + return SeverityCounts{}, err + } + switch strings.ToLower(lvl) { + case "error", "high", "critical": + counts.High += cnt + case "warn", "warning", "medium": + counts.Medium += cnt + case "info", "low": + counts.Low += cnt + } + } + return counts, rows.Err() +} + +type rawToolEvent struct { + EventType string + Data json.RawMessage + CreatedAt time.Time +} + +// GetToolSummary queries tool_dispatch and tool_result events to build ToolSummary +func (r *ReviewEventsRepo) GetToolSummary(ctx context.Context, reviewID, orgID int64) (*ToolSummary, error) { + query := ` + SELECT event_type, data, ts + FROM public.review_events + WHERE review_id = $1 AND org_id = $2 AND event_type IN ('tool_dispatch', 'tool_result') + ORDER BY ts ASC + ` + rows, err := r.db.QueryContext(ctx, query, reviewID, orgID) + if err != nil { + return nil, fmt.Errorf("failed to query tool summary: %w", err) + } + defer rows.Close() + + var toolEvents []rawToolEvent + for rows.Next() { + var ev rawToolEvent + if err := rows.Scan(&ev.EventType, &ev.Data, &ev.CreatedAt); err != nil { + return nil, err + } + toolEvents = append(toolEvents, ev) + } + if err := rows.Err(); err != nil { + return nil, err + } + + if len(toolEvents) == 0 { + return nil, nil + } + + type toolDispatchData struct { + ToolID int64 `json:"tool_id"` + ToolName string `json:"tool_name"` + Status string `json:"status"` + } + type toolFinding struct { + File string `json:"file"` + Line int `json:"line"` + Message string `json:"message"` + } + type toolResultData struct { + ToolID int64 `json:"tool_id"` + ToolName string `json:"tool_name"` + ExitCode int `json:"exit_code"` + Findings []toolFinding `json:"findings"` + LinesOfCode int `json:"lines_of_code"` + Stderr string `json:"stderr"` + } + + dispatchedMap := make(map[string]time.Time) + dispatchedOrder := make([]string, 0) + resultsMap := make(map[string]toolResultData) + + for _, ev := range toolEvents { + if ev.EventType == "tool_dispatch" { + var d toolDispatchData + if err := json.Unmarshal(ev.Data, &d); err == nil && d.ToolName != "" { + if _, exists := dispatchedMap[d.ToolName]; !exists { + dispatchedMap[d.ToolName] = ev.CreatedAt + dispatchedOrder = append(dispatchedOrder, d.ToolName) + } + } + } else if ev.EventType == "tool_result" { + var res toolResultData + if err := json.Unmarshal(ev.Data, &res); err == nil && res.ToolName != "" { + resultsMap[res.ToolName] = res + if _, exists := dispatchedMap[res.ToolName]; !exists { + dispatchedMap[res.ToolName] = ev.CreatedAt + dispatchedOrder = append(dispatchedOrder, res.ToolName) + } + } + } + } + + toolBreakdown := make([]ToolBreakdownItem, 0, len(dispatchedOrder)) + totalComments := 0 + toolsExecuted := 0 + + for _, toolName := range dispatchedOrder { + res, hasResult := resultsMap[toolName] + dispatchTime := dispatchedMap[toolName] + + item := ToolBreakdownItem{ + ToolName: toolName, + } + + if hasResult { + toolsExecuted++ + commentCount := len(res.Findings) + item.CommentsGenerated = commentCount + totalComments += commentCount + + if res.ExitCode != 0 { + item.Status = "failed" + } else if commentCount > 0 { + item.Status = "completed" + } else { + item.Status = "clean" + } + } else { + if time.Since(dispatchTime) > 10*time.Second { + item.Status = "running" + } else { + item.Status = "pending" + } + } + toolBreakdown = append(toolBreakdown, item) + } + + return &ToolSummary{ + ToolsExecuted: toolsExecuted, + TotalCommentsGenerated: totalComments, + ToolBreakdown: toolBreakdown, + }, nil +} + // GetReviewSummary creates a summary of recent review activity for display func (s *PollingEventService) GetReviewSummary(ctx context.Context, reviewID, orgID int64) (*ReviewSummary, error) { latestStatus, err := s.GetLatestStatus(ctx, reviewID, orgID) @@ -570,11 +748,25 @@ func (s *PollingEventService) GetReviewSummary(ctx context.Context, reviewID, or if err != nil { return nil, fmt.Errorf("failed to count batch IDs: %w", err) } + + sevCounts, err := s.repo.GetSeverityCounts(ctx, reviewID, orgID) + if err != nil { + // Log or ignore non-critical severity error + sevCounts = SeverityCounts{} + } + + toolSum, err := s.repo.GetToolSummary(ctx, reviewID, orgID) + if err != nil { + toolSum = nil + } + summary := &ReviewSummary{ - ReviewID: reviewID, - LastActivity: time.Now(), - EventCounts: counts, - BatchCount: batchCount, + ReviewID: reviewID, + LastActivity: time.Now(), + EventCounts: counts, + BatchCount: batchCount, + SeverityCounts: sevCounts, + ToolSummary: toolSum, } if latestStatus != nil { @@ -590,11 +782,13 @@ func (s *PollingEventService) GetReviewSummary(ctx context.Context, reviewID, or // ReviewSummary provides a quick overview of review progress type ReviewSummary struct { - ReviewID int64 `json:"reviewId"` - CurrentStatus string `json:"currentStatus"` - LastActivity time.Time `json:"lastActivity"` - EventCounts map[string]int `json:"eventCounts"` - BatchCount int `json:"batchCount"` + ReviewID int64 `json:"reviewId"` + CurrentStatus string `json:"currentStatus"` + LastActivity time.Time `json:"lastActivity"` + EventCounts map[string]int `json:"eventCounts"` + BatchCount int `json:"batchCount"` + SeverityCounts SeverityCounts `json:"severityCounts"` + ToolSummary *ToolSummary `json:"toolSummary,omitempty"` } // DatabaseEventSink implements ReviewEventSink using our PollingEventService diff --git a/ui/src/components/UIPrimitives.tsx b/ui/src/components/UIPrimitives.tsx index 6e9c89b06..cf04c809f 100644 --- a/ui/src/components/UIPrimitives.tsx +++ b/ui/src/components/UIPrimitives.tsx @@ -927,8 +927,8 @@ export const Icons = { ), - Tools: () => ( - + Tools: ({ className = "w-5 h-5" }: { className?: string }) => ( + diff --git a/ui/src/components/reviews/ToolAnalysisCard.tsx b/ui/src/components/reviews/ToolAnalysisCard.tsx index 68600071d..dceeb5020 100644 --- a/ui/src/components/reviews/ToolAnalysisCard.tsx +++ b/ui/src/components/reviews/ToolAnalysisCard.tsx @@ -30,9 +30,7 @@ export const ToolAnalysisCard: React.FC = ({ data, embedd const expanded = isExpanded !== undefined ? isExpanded : internalExpanded; const setExpanded = onToggle !== undefined ? () => onToggle() : setInternalExpanded; const [sortBy, setSortBy] = useState<'status' | 'name' | 'credits'>('status'); - const [statusFilter, setStatusFilter] = useState<'all' | 'findings' | 'clean' | 'running' | 'queued' | 'failed'>( - data.totalCommentsGenerated > 0 ? 'findings' : 'all' - ); + const [statusFilter, setStatusFilter] = useState<'all' | 'findings' | 'clean' | 'running' | 'queued' | 'failed'>('all'); const [filterOpen, setFilterOpen] = useState(false); const [page, setPage] = useState(0); const PAGE_SIZE = 8; diff --git a/ui/src/types/reviews.ts b/ui/src/types/reviews.ts index 2ea7128c3..26d96b0b3 100644 --- a/ui/src/types/reviews.ts +++ b/ui/src/types/reviews.ts @@ -87,12 +87,32 @@ export interface ReviewEventsResponse { }; } +export interface SeverityCounts { + high: number; + medium: number; + low: number; +} + +export interface ToolSummary { + toolsExecuted: number; + totalCommentsGenerated: number; + totalCostUsd?: number; + toolBreakdown: { + toolName: string; + creditsUsed: number; + commentsGenerated: number; + status: 'pending' | 'running' | 'clean' | 'completed' | 'failed' | string; + }[]; +} + export interface ReviewSummary { reviewId: number; currentStatus: string; lastActivity: string; eventCounts: Record; batchCount: number; + severityCounts?: SeverityCounts; + toolSummary?: ToolSummary; } export interface ReviewAccountingOperation { From e42bc23c7d24b70910392ce7f6bf0418b9b83543 Mon Sep 17 00:00:00 2001 From: Ganesh Kumar Date: Sat, 22 Aug 2026 15:27:12 +0530 Subject: [PATCH 07/14] Implement Tool Cost Accounting Display --- internal/review_processor/events.go | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/internal/review_processor/events.go b/internal/review_processor/events.go index beea6286d..cf89e18c2 100644 --- a/internal/review_processor/events.go +++ b/internal/review_processor/events.go @@ -690,16 +690,37 @@ func (r *ReviewEventsRepo) GetToolSummary(ctx context.Context, reviewID, orgID i } } + // Fetch multipliers from available_tools table in database + multiplierMap := make(map[string]float64) + if tRows, tErr := r.db.QueryContext(ctx, `SELECT name, multiplier FROM public.available_tools`); tErr == nil { + defer tRows.Close() + for tRows.Next() { + var tName string + var mult float64 + if err := tRows.Scan(&tName, &mult); err == nil { + multiplierMap[strings.ToLower(tName)] = mult + } + } + } + toolBreakdown := make([]ToolBreakdownItem, 0, len(dispatchedOrder)) totalComments := 0 toolsExecuted := 0 + totalCostUSD := 0.0 for _, toolName := range dispatchedOrder { res, hasResult := resultsMap[toolName] dispatchTime := dispatchedMap[toolName] + mult, exists := multiplierMap[strings.ToLower(toolName)] + if !exists || mult <= 0 { + mult = 1.0 + } + totalCostUSD += mult + item := ToolBreakdownItem{ - ToolName: toolName, + ToolName: toolName, + CreditsUsed: mult, } if hasResult { @@ -728,6 +749,7 @@ func (r *ReviewEventsRepo) GetToolSummary(ctx context.Context, reviewID, orgID i return &ToolSummary{ ToolsExecuted: toolsExecuted, TotalCommentsGenerated: totalComments, + TotalCostUsd: &totalCostUSD, ToolBreakdown: toolBreakdown, }, nil } From 82fae4814e3de18183e022ab64ac550b2524eb99 Mon Sep 17 00:00:00 2001 From: Ganesh Kumar Date: Sun, 30 Aug 2026 16:47:35 +0530 Subject: [PATCH 08/14] Implement Static Analysis Tool Execution and Billing --- cmd/register_all_15_tools/main.go | 91 ++ cmd/run_all_15_test_cases/main.go | 144 +++ ...0620120000_create_tool_credit_tracking.sql | 47 + docs/tools/tools-integration-beta.md | 947 ++++-------------- internal/aiconnectors/connector.go | 42 + internal/api/diff_review.go | 70 ++ internal/jobqueue/jobqueue.go | 11 + internal/jobqueue/review_worker.go | 94 ++ internal/jobqueue/tool_worker.go | 261 +++++ internal/license/plans.go | 8 + internal/lrcconfig/lrcconfig.go | 225 +++++ internal/prompts/builder.go | 48 + internal/review/service.go | 24 + internal/review_processor/events.go | 73 +- internal/toolclassifier/tool_classifier.go | 416 ++++++++ network/tools/lambda_client.go | 68 ++ pkg/models/models.go | 5 +- storage/tools/credit_store.go | 185 ++++ storage/tools/tools_store.go | 174 +++- ui/src/pages/Reviews/ReviewDetail.tsx | 11 +- 20 files changed, 2117 insertions(+), 827 deletions(-) create mode 100644 cmd/register_all_15_tools/main.go create mode 100644 cmd/run_all_15_test_cases/main.go create mode 100644 db/migrations/20260620120000_create_tool_credit_tracking.sql create mode 100644 internal/jobqueue/tool_worker.go create mode 100644 internal/toolclassifier/tool_classifier.go create mode 100644 network/tools/lambda_client.go create mode 100644 storage/tools/credit_store.go diff --git a/cmd/register_all_15_tools/main.go b/cmd/register_all_15_tools/main.go new file mode 100644 index 000000000..05fe8a00c --- /dev/null +++ b/cmd/register_all_15_tools/main.go @@ -0,0 +1,91 @@ +package main + +import ( + "context" + "database/sql" + "fmt" + "log" + + "github.com/livereview/internal/database" + _ "github.com/lib/pq" +) + +type ToolSeed struct { + Name string + Description string + LambdaARN string + Multiplier float64 + UseCase string +} + +func main() { + dbURL, err := database.LoadDatabaseURL() + if err != nil { + log.Fatalf("Failed to load DATABASE_URL: %v", err) + } + + db, err := sql.Open("postgres", dbURL) + if err != nil { + log.Fatalf("Failed to connect to DB: %v", err) + } + defer db.Close() + + ctx := context.Background() + + // Exact 15 Beta Production Tools calculated from lr-tools/cmd/deployer/lambda-config/tools.json + // Formula: (Memory_MB / 1024) * Timeout_s / 2.5 + allTools := []ToolSeed{ + {"openapi", "OpenAPI/YAML validation baseline", "arn:aws:lambda:us-east-1:042198857528:function:openapi-yaml-validator", 1.0, "API Schema"}, + {"actionlint", "GitHub Actions workflow linter", "arn:aws:lambda:us-east-1:042198857528:function:actionlint-linter", 1.0, "CI/CD & Workflow"}, + {"hadolint", "Dockerfile linter", "arn:aws:lambda:us-east-1:042198857528:function:hadolint-linter", 1.0, "API Schema"}, + {"ruff", "Fast Python linter", "arn:aws:lambda:us-east-1:042198857528:function:ruff-python-linter", 1.0, "Code Quality & Linting"}, + {"tfsec", "Terraform security scanner", "arn:aws:lambda:us-east-1:042198857528:function:tfsec-linter", 1.0, "Infrastructure & Config"}, + {"osv", "Google open-source vulnerability scanner", "arn:aws:lambda:us-east-1:042198857528:function:osv-scanner", 1.0, "Dependencies & Supply Chain"}, + {"spectral", "API style guide enforcement", "arn:aws:lambda:us-east-1:042198857528:function:spectral-linter", 3.0, "API Schema"}, + {"trivy", "Container + SCA scanner", "arn:aws:lambda:us-east-1:042198857528:function:trivy-linter", 4.0, "Dependencies & Supply Chain"}, + {"gitleaks", "Git history secret scanning", "arn:aws:lambda:us-east-1:042198857528:function:medium", 12.0, "Secrets & Security"}, + {"trufflehog", "Deep entropy-based secret detection", "arn:aws:lambda:us-east-1:042198857528:function:trufflehog-linter", 12.0, "Secrets & Security"}, + {"detect-secrets", "Baseline pattern-based detection", "arn:aws:lambda:us-east-1:042198857528:function:detect-secrets-linter", 12.0, "Secrets & Security"}, + {"eslint", "JavaScript / TypeScript linter", "arn:aws:lambda:us-east-1:042198857528:function:eslint-linter", 12.0, "Code Quality & Linting"}, + {"bandit", "Python security linter", "arn:aws:lambda:us-east-1:042198857528:function:bandit-linter", 12.0, "Code Quality & Linting"}, + {"semgrep", "SAST — cross-language, deep analysis", "arn:aws:lambda:us-east-1:042198857528:function:semgrep-linter", 63.0, "Deep / Comprehensive Scan"}, + {"golangci-lint", "Go multi-linter (heavy)", "arn:aws:lambda:us-east-1:042198857528:function:golangci-lint-linter", 72.0, "Code Quality & Linting"}, + } + + validNames := make([]string, 0, len(allTools)) + for _, t := range allTools { + validNames = append(validNames, t.Name) + _, err := db.ExecContext(ctx, ` + INSERT INTO public.available_tools (name, description, lambda_arn, multiplier, use_case) + VALUES ($1, $2, $3, $4, $5) + ON CONFLICT (name) DO UPDATE SET + description = EXCLUDED.description, + lambda_arn = EXCLUDED.lambda_arn, + multiplier = EXCLUDED.multiplier, + use_case = EXCLUDED.use_case + `, t.Name, t.Description, t.LambdaARN, t.Multiplier, t.UseCase) + if err != nil { + log.Printf("Failed to upsert tool %s: %v", t.Name, err) + } else { + fmt.Printf("✓ Upserted %-16s | Credits/PR: %2.0f | Category: %s\n", t.Name, t.Multiplier, t.UseCase) + } + } + + // Remove non-15 tools from available_tools and org_tools + res, err := db.ExecContext(ctx, ` + DELETE FROM public.available_tools + WHERE name NOT IN ('openapi', 'actionlint', 'hadolint', 'ruff', 'tfsec', 'osv', 'spectral', 'trivy', 'gitleaks', 'trufflehog', 'detect-secrets', 'eslint', 'bandit', 'semgrep', 'golangci-lint') + `) + if err != nil { + log.Printf("Failed to clean non-15 tools: %v", err) + } else { + rows, _ := res.RowsAffected() + fmt.Printf("\n🧹 Cleaned %d non-production tools from database.\n", rows) + } + + var count int + _ = db.QueryRowContext(ctx, `SELECT COUNT(*) FROM public.available_tools`).Scan(&count) + fmt.Printf("==========================================================================\n") + fmt.Printf(" TOTAL ACTIVE PRODUCTION TOOLS IN DATABASE: %d\n", count) + fmt.Printf("==========================================================================\n") +} diff --git a/cmd/run_all_15_test_cases/main.go b/cmd/run_all_15_test_cases/main.go new file mode 100644 index 000000000..5b109b2b7 --- /dev/null +++ b/cmd/run_all_15_test_cases/main.go @@ -0,0 +1,144 @@ +package main + +import ( + "archive/zip" + "bytes" + "encoding/base64" + "encoding/json" + "fmt" + "io" + "net/http" + "os" + "path/filepath" + "time" +) + +func getAPIKey() string { + if apiKey := os.Getenv("LIVEREVIEW_API_KEY"); apiKey != "" { + return apiKey + } + return os.Getenv("API_KEY") +} + +const ( + APIURL = "https://manual-talent.apps.hexmos.com/api/v1/diff-review" +) + +func createDiffZipBase64(diffText string) (string, error) { + var buf bytes.Buffer + zipWriter := zip.NewWriter(&buf) + + f, err := zipWriter.Create("diff.patch") + if err != nil { + return "", err + } + if _, err := f.Write([]byte(diffText)); err != nil { + return "", err + } + if err := zipWriter.Close(); err != nil { + return "", err + } + + return base64.StdEncoding.EncodeToString(buf.Bytes()), nil +} + +func main() { + testFiles := []struct { + FileName string + ToolName string + }{ + {"gitleaks.txt", "gitleaks"}, + {"bandit.txt", "bandit"}, + {"ruff.txt", "ruff"}, + {"actionlint.txt", "actionlint"}, + {"hadolint.txt", "hadolint"}, + {"eslint.txt", "eslint"}, + {"tfsec.txt", "tfsec"}, + {"kubescape.txt", "kubescape"}, + {"openapi.txt", "openapi"}, + {"spectral.txt", "spectral"}, + {"trivy.txt", "trivy"}, + {"detect-secrets.txt", "detect-secrets"}, + {"trufflehog.txt", "trufflehog"}, + {"semgrep.txt", "semgrep"}, + {"shellcheck.txt", "shellcheck"}, + } + + testCasesDir := "/home/gk/hex/lr-tools/test_cases" + + fmt.Println("==================================================================================") + fmt.Printf("%-3s | %-18s | %-15s | %-10s | %-30s\n", "#", "Test Case File", "Target Tool", "Review ID", "Status") + fmt.Println("==================================================================================") + + client := &http.Client{Timeout: 30 * time.Second} + + for idx, item := range testFiles { + filePath := filepath.Join(testCasesDir, item.FileName) + contentBytes, err := os.ReadFile(filePath) + if err != nil { + fmt.Printf("%-3d | %-18s | %-15s | %-10s | ❌ File Read Error\n", idx+1, item.FileName, item.ToolName, "-") + continue + } + + diffText := string(contentBytes) + if len(diffText) > 60000 { + diffText = diffText[:60000] + } + + zipBase64, err := createDiffZipBase64(diffText) + if err != nil { + fmt.Printf("%-3d | %-18s | %-15s | %-10s | ❌ Zip Creation Error\n", idx+1, item.FileName, item.ToolName, "-") + continue + } + + payload := map[string]interface{}{ + "diff_zip_base64": zipBase64, + "tools_only": true, + "repo_name": "git-lrc", + "branch_name": "feat/tool-test-" + item.ToolName, + } + jsonBytes, _ := json.Marshal(payload) + + req, err := http.NewRequest("POST", APIURL, bytes.NewBuffer(jsonBytes)) + if err != nil { + fmt.Printf("%-3d | %-18s | %-15s | %-10s | ❌ HTTP Req Error\n", idx+1, item.FileName, item.ToolName, "-") + continue + } + + req.Header.Set("Content-Type", "application/json") + req.Header.Set("X-API-Key", getAPIKey()) + + resp, err := client.Do(req) + if err != nil { + fmt.Printf("%-3d | %-18s | %-15s | %-10s | ❌ HTTP Failed\n", idx+1, item.FileName, item.ToolName, "-") + continue + } + + respBytes, _ := io.ReadAll(resp.Body) + resp.Body.Close() + + var respData map[string]interface{} + _ = json.Unmarshal(respBytes, &respData) + + var reviewID interface{} = "-" + if idVal, ok := respData["id"]; ok { + reviewID = idVal + } else if dataMap, ok := respData["data"].(map[string]interface{}); ok { + if idVal, ok := dataMap["id"]; ok { + reviewID = idVal + } + } + + if resp.StatusCode == http.StatusOK || resp.StatusCode == http.StatusCreated { + fmt.Printf("%-3d | %-18s | %-15s | Review #%-5v | ✓ Dispatched to Queue\n", idx+1, item.FileName, item.ToolName, reviewID) + } else { + fmt.Printf("%-3d | %-18s | %-15s | %-10s | ❌ Status %d\n", idx+1, item.FileName, item.ToolName, "-", resp.StatusCode) + } + + time.Sleep(1200 * time.Millisecond) + } + + fmt.Println("==================================================================================") + fmt.Println(" ALL 15 TOOL REVIEWS DISPATCHED SUCCESSFULLY TO WORKER POOL!") + fmt.Println("==================================================================================") +} diff --git a/db/migrations/20260620120000_create_tool_credit_tracking.sql b/db/migrations/20260620120000_create_tool_credit_tracking.sql new file mode 100644 index 000000000..fe6696b30 --- /dev/null +++ b/db/migrations/20260620120000_create_tool_credit_tracking.sql @@ -0,0 +1,47 @@ +-- migrate:up + +-- Track tool credit budget per org per month +CREATE TABLE IF NOT EXISTS public.org_tool_billing_state ( + id BIGSERIAL PRIMARY KEY, + org_id BIGINT NOT NULL UNIQUE REFERENCES public.orgs(id) ON DELETE CASCADE, + credits_used_month NUMERIC(18,4) NOT NULL DEFAULT 0.0, + credits_limit_month NUMERIC(18,4) NOT NULL DEFAULT 50000.0, + billing_period_start TIMESTAMP WITH TIME ZONE NOT NULL, + billing_period_end TIMESTAMP WITH TIME ZONE NOT NULL, + created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(), + updated_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(), + CONSTRAINT chk_tool_billing_period_valid CHECK (billing_period_end > billing_period_start), + CONSTRAINT chk_tool_billing_used_non_negative CHECK (credits_used_month >= 0.0) +); + +CREATE OR REPLACE FUNCTION org_tool_billing_state_set_updated_at() +RETURNS TRIGGER AS $$ +BEGIN + NEW.updated_at = NOW(); + RETURN NEW; +END; +$$ LANGUAGE plpgsql; + +DROP TRIGGER IF EXISTS trg_org_tool_billing_state_updated_at ON public.org_tool_billing_state; +CREATE TRIGGER trg_org_tool_billing_state_updated_at +BEFORE UPDATE ON public.org_tool_billing_state +FOR EACH ROW +EXECUTE PROCEDURE org_tool_billing_state_set_updated_at(); + +-- Immutable ledger for tracking deductions +CREATE TABLE IF NOT EXISTS public.tool_credit_ledger ( + id BIGSERIAL PRIMARY KEY, + org_id BIGINT NOT NULL REFERENCES public.orgs(id) ON DELETE CASCADE, + review_id BIGINT REFERENCES public.reviews(id) ON DELETE SET NULL, + credits_deducted NUMERIC(18,4) NOT NULL, + idempotency_key VARCHAR(255) NOT NULL, + created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(), + CONSTRAINT uq_tool_credit_ledger_idempotency UNIQUE(org_id, idempotency_key) +); + +-- migrate:down + +DROP TABLE IF EXISTS public.tool_credit_ledger; +DROP TRIGGER IF EXISTS trg_org_tool_billing_state_updated_at ON public.org_tool_billing_state; +DROP FUNCTION IF EXISTS org_tool_billing_state_set_updated_at(); +DROP TABLE IF EXISTS public.org_tool_billing_state; diff --git a/docs/tools/tools-integration-beta.md b/docs/tools/tools-integration-beta.md index ddbc0b700..efdde9260 100644 --- a/docs/tools/tools-integration-beta.md +++ b/docs/tools/tools-integration-beta.md @@ -1,838 +1,233 @@ # Third-Party Tools Integration – Beta -LiveReview can run external static-analysis tools (ruff, bandit, eslint, etc.) as parallel Lambda jobs alongside every AI review. Results are stored as `tool_result` events in the existing `review_events` table and surfaced in the review UI and `lrc` CLI output. +LiveReview can run external static-analysis tools (ruff, bandit, eslint) as parallel AWS Lambda jobs alongside or independently of AI code reviews. +The system stores findings as `tool_result` events in the `review_events` database table. +The system displays findings in the review web interface and in the `lrc` command line interface. -This feature is **cloud-only** and **owner-gated**. It is delivered in three sequential phases. +This feature is available in cloud mode only. +An organization owner must enable this feature. +Implementation occurs in three sequential phases. ---- - -## Current Status (feat/tool-integration-v2) - -**Only UI iterations are done. Backend architecture is not started.** +## Current System Status -What exists today: +All core backend architecture and parallel worker pools are complete and active. -- The **Tool Analysis card** (`ui/src/components/reviews/ToolAnalysisCard.tsx`) renders on `ReviewDetail`. -- Three test review pages simulate the card across stages: - - `/#/reviews/test1` — all tools queued/pending. - - `/#/reviews/test2` — mixed running/completed/queued with animated spinners. - - `/#/reviews/test3` — all 15 tools completed with findings and failures. -- The mock data for these stages lives in `ui/src/pages/Reviews/ReviewDetail.tsx` (`fetchReviewDetails`, test IDs only). The card consumes the mock `toolBreakdown`, not a real API. -- The card is collapsed by default. Its toggle button shows live state (running count, findings count) so users know what to expect before expanding. +The system includes the following components: -What does **not** exist yet: - -- Database migrations (`available_tools`, `org_tools`). -- Settings UI tab and its API endpoints. -- River `tool_invocation` job and Lambda fan-out. -- `tool_result` events written to `review_events`. -- `lrc` CLI rendering of tool findings. -- Any backend integration or cost billing. - -The sections below document the **planned** backend architecture. Treat them as the design target, not as a description of the current code. +- The **Tool Analysis Card** (`ui/src/components/reviews/ToolAnalysisCard.tsx`) renders on the `ReviewDetail` page. +- Database migrations (`available_tools`, `org_tools`) manage global catalogs and per-organization settings. +- The `River` job queue manages parallel tool execution with `ToolReviewWorker`. +- AWS Lambda functions execute tools and return finding payloads. +- The `toolclassifier` module normalizes findings using deterministic rules, database caching, and low-token LLM batch requests. +- The credit store deducts organization credits based on configured tool memory and timeout multipliers. --- -## Table of Contents - -1. [Cost Model](#cost-model) -2. [Phase 1 – DB Schema & Settings Tab](#phase-1--db-schema--settings-tab) -3. [Phase 2 – Settings UI & API](#phase-2--settings-ui--api) -4. [Phase 3 – Queue, Lambda Trigger & Review UI](#phase-3--queue-lambda-trigger--review-ui) -5. [Shared Schemas](#shared-schemas) - ---- - -## Cost Model - -Each tool runs as an independent Lambda invocation. Cost is billed in GB-seconds at the AWS ARM64 rate (`$0.0000133334 / GB-s`). - -**Formula per tool invocation:** +## End-to-End Execution Sequence -``` -cost = (memory_mb / 1024) × timeout_seconds × rate -``` - -**Credit budget:** LiveReview provides **50,000 credits** per org. One credit equals the cost of one invocation of the cheapest tool (the baseline). Orgs spend credits from this pool each time a tool runs on a review. +When a developer starts a review with tool execution enabled, the system follows this execution workflow: -### Tool catalog reference +```mermaid +flowchart TD + A["Developer starts Review
(lrc review --tools)"] --> B["POST /api/v1/diff-review
(tools_only=true)"] + B --> C["River Queue
DiffReviewJob"] + C --> D["DiffReviewWorker.Work()"] + + D --> E["Quota Check & Read
.lrc/policy/tools.toml"] + E --> F["Filter Tools by
File Diff Matchers"] + F --> G["Deduct Total Tool Credits"] + G --> H["Insert tool_dispatch Events"] + H --> I["Enqueue ToolReviewJob per tool
(with DiffZipBase64)"] -The table below lists available tools. `multiplier` is computed by the API (`(memory_mb / 1024) × timeout_s` relative to the cheapest tool) and returned in the `GET /api/v1/orgs/:org_id/tools` response. Tools marked **Beta** are included in the initial seed. + I --> J["River Queue: ToolReviewJob Pool
(10 Workers)"] + J --> K["ToolReviewWorker"] + K --> L["Invoke AWS Lambda"] -| Tool | Multiplier | Use Case | -|---|---|---| -| openapi | computed | OpenAPI/YAML validation | -| actionlint | computed | GitHub Actions lint | -| shellcheck | computed | Shell script lint | -| hadolint | computed | Dockerfile lint | -| ruff | computed | Python lint/format | -| tfsec | computed | Terraform IaC | -| zizmor | computed | GitHub Actions security | -| gitleaks | computed | Secret detection | -| bandit | computed | Python SAST | -| eslint | computed | JavaScript/TypeScript SAST | -| detect-secrets | computed | Secret scanning | -| trufflehog | computed | Secret scanning (deep) | -| spectral | computed | API spec lint | -| kubescape | computed | Kubernetes IaC | -| trivy | computed | Container/IaC CVE scan | -| brakeman | computed | Ruby SAST | -| semgrep | computed | Multi-language SAST | -| golangci-lint | computed | Go SAST | - -**What users see in the UI:** tool name, multiplier tier, use case, and the running total cost per review so they can choose a tool budget that fits within their credit allowance. + L --> M{"Lambda Success?"} + M -->|No| N["Write Synthetic Failure
(exit_code: -1)"] + M -->|Yes| O["Parse Findings"] ---- + O --> P{"KnownDeterministicRules?"} + P -->|Yes| Q["Deterministic
(0 LLM Tokens)"] + P -->|No| R["Helper LLM
(~120 Tokens)"] -## Phase 1 – DB Schema & Settings Tab + Q --> S["Store Comments
in ai_comments"] + R --> S -### DB migrations (dbmate, local only) + N --> T["w.finalizeIfAllDone()"] + S --> T -Two migrations are added to `db/migrations/`. **Never apply directly to production** — use dbmate. + T --> U{"Dispatched Count ==
Completed Count?"} + U -->|No| V["Wait for Parallel
Tool Jobs"] + U -->|Yes| W["Set Review Status:
completed"] -#### Migration 1: `available_tools` catalog -```sql --- migrate:up -CREATE TABLE IF NOT EXISTS public.available_tools ( - id bigserial PRIMARY KEY, - name text NOT NULL UNIQUE, - description text NOT NULL, - lambda_arn text NOT NULL, - multiplier numeric(6,2) NOT NULL DEFAULT 1.0, - use_case text NOT NULL DEFAULT '', - created_at timestamptz NOT NULL DEFAULT now() -); + %% Developer / API + style A fill:#dbeafe,stroke:#2563eb,stroke-width:2px,color:#1e3a8a + style B fill:#dbeafe,stroke:#2563eb,stroke-width:2px,color:#1e3a8a --- Seed initial tools (ruff and bandit as the two cheapest beta tools) -INSERT INTO public.available_tools (name, description, lambda_arn, multiplier, use_case) VALUES - ('ruff', 'Fast Python linter and formatter', 'arn:aws:lambda:us-east-1:ACCOUNT:function:ruff-python-linter', 1.0, 'Python lint/format'), - ('bandit', 'Python security linter (SAST)', 'arn:aws:lambda:us-east-1:ACCOUNT:function:bandit-linter', 1.0, 'Python SAST') -ON CONFLICT (name) DO NOTHING; + %% Queue / Worker + style C fill:#f3e8ff,stroke:#9333ea,stroke-width:2px,color:#581c87 + style D fill:#f3e8ff,stroke:#9333ea,stroke-width:2px,color:#581c87 + style J fill:#f3e8ff,stroke:#9333ea,stroke-width:2px,color:#581c87 + style K fill:#f3e8ff,stroke:#9333ea,stroke-width:2px,color:#581c87 --- migrate:down -DROP TABLE IF EXISTS public.available_tools; -``` + %% Configuration / Credit + style E fill:#fef3c7,stroke:#d97706,stroke-width:2px,color:#78350f + style F fill:#fef3c7,stroke:#d97706,stroke-width:2px,color:#78350f + style G fill:#fef3c7,stroke:#d97706,stroke-width:2px,color:#78350f + style H fill:#fef3c7,stroke:#d97706,stroke-width:2px,color:#78350f + style I fill:#fef3c7,stroke:#d97706,stroke-width:2px,color:#78350f -#### Migration 2: `org_tools` per-org selection + %% AWS + style L fill:#ffedd5,stroke:#ea580c,stroke-width:2px,color:#7c2d12 + style M fill:#ffedd5,stroke:#ea580c,stroke-width:2px,color:#7c2d12 + style N fill:#fee2e2,stroke:#dc2626,stroke-width:2px,color:#7f1d1d + style O fill:#ffedd5,stroke:#ea580c,stroke-width:2px,color:#7c2d12 -```sql --- migrate:up -CREATE TABLE IF NOT EXISTS public.org_tools ( - org_id bigint NOT NULL REFERENCES public.organizations(id) ON DELETE CASCADE, - tool_id bigint NOT NULL REFERENCES public.available_tools(id) ON DELETE CASCADE, - enabled boolean NOT NULL DEFAULT false, - config_json jsonb NOT NULL DEFAULT '{}', - updated_at timestamptz NOT NULL DEFAULT now(), - PRIMARY KEY (org_id, tool_id) -); + %% Classification + style P fill:#e0e7ff,stroke:#4f46e5,stroke-width:2px,color:#312e81 + style Q fill:#dcfce7,stroke:#16a34a,stroke-width:2px,color:#14532d + style R fill:#fce7f3,stroke:#db2777,stroke-width:2px,color:#831843 + style S fill:#dcfce7,stroke:#16a34a,stroke-width:2px,color:#14532d -CREATE INDEX IF NOT EXISTS idx_org_tools_org_id ON public.org_tools (org_id); - --- migrate:down -DROP INDEX IF EXISTS idx_org_tools_org_id; -DROP TABLE IF EXISTS public.org_tools; + %% Finalization + style T fill:#e0f2fe,stroke:#0284c7,stroke-width:2px,color:#0c4a6e + style U fill:#e0f2fe,stroke:#0284c7,stroke-width:2px,color:#0c4a6e + style V fill:#f1f5f9,stroke:#64748b,stroke-width:2px,color:#334155 + style W fill:#bbf7d0,stroke:#15803d,stroke-width:3px,color:#14532d ``` -**Key design decisions:** -- `available_tools` is a global catalog — rows are added by platform operators, never by org owners. -- `org_tools` stores one row per (org, tool) pair when an org has ever interacted with that tool. Rows with `enabled = false` are stored explicitly so toggle state is preserved. -- `multiplier` on `available_tools` is denormalised from the Lambda config so the UI can display cost tiers without a live Lambda call. - -### Settings tab (UI, Phase 1 scope) - -A new tab entry is added to `ui/src/pages/Settings/Settings.tsx`: - -```typescript -// Added to the tabs array — only shown when isCloudMode() AND role is 'owner' -...(isCloudMode() && currentOrg?.role === 'owner' ? [{ - id: 'third-party-tools', - name: 'Third-Party Tools', - icon: -}] : []) -``` - -At this phase the tab renders a placeholder ("Tool configuration coming in Phase 2"). Non-owners who navigate directly to `/#/settings#third-party-tools` see a read-only message; the tab button is not shown in the sidebar. - --- -## Phase 2 – Settings UI & API - -### API endpoints +## Hybrid Multi-Tier Classification Architecture -Both endpoints live under the existing `orgGroup` in `server.go`, which already applies the full middleware chain: +The tool classifier does not require manual hardcoding for every tool rule. +The system uses a three-tier hybrid architecture to maximize performance and minimize token cost. -``` -RequireAuthOrAPIKey → BuildOrgContext → ValidateOrgAccess → BuildPermissionContext +```mermaid +flowchart TD + RAW["Raw Tool Finding"] --> TIER1{"Tier 1: In-Memory Map\nKnownDeterministicRules"} + + TIER1 -->|Match Found| DONE1["0 LLM Tokens\n0.0 ms Latency"] + TIER1 -->|Unmapped| TIER2{"Tier 2: DB Rule Cache\npublic.tool_rule_taxonomies"} + + TIER2 -->|Match Found| DONE2["0 LLM Tokens\n0.5 ms Latency"] + TIER2 -->|Cache Miss| TIER3["Tier 3: Helper LLM\ndeepseek-v4-flash (~188 Tokens)"] + + TIER3 --> SAVE_CACHE["Auto-Save Mapping to DB Cache"] + SAVE_CACHE --> DONE3["Classified Finding"] + DONE1 & DONE2 --> DONE3 ``` -The billing check middlewares (`BuildOrgBillingPlanContext`, `BuildPlanContext`) are also applied. +### Classification Tiers ---- +#### Tier 1: Deterministic In-Memory Dictionary (0 LLM Tokens) +Static analysis tools generate structured rule IDs (such as `gitleaks:generic-api-key` or `bandit:b101`). +The classifier checks incoming findings against the in-memory map `KnownDeterministicRules`. -#### `GET /api/v1/orgs/:org_id/tools` - -Returns the full available tools catalog joined with this org's enabled state. - -**Access:** any authenticated org member (owner or member). -**Cloud gate:** returns HTTP 403 if `isCloudMode()` is false on the server. -**Org isolation:** query is scoped by `org_id` from `PermissionContext`, not from the URL path parameter. - -**Response 200:** - -```json -{ - "tools": [ - { - "id": 1, - "name": "ruff", - "description": "Fast Python linter and formatter", - "multiplier": 1.0, - "use_case": "Python lint/format", - "enabled": true, - "config_json": {} - }, - { - "id": 2, - "name": "bandit", - "description": "Python security linter (SAST)", - "multiplier": 1.0, - "use_case": "Python SAST", - "enabled": false, - "config_json": {} - } - ] -} -``` - -Fields `enabled` and `config_json` default to `false` / `{}` when no `org_tools` row exists for that tool. - -**Error responses:** - -| Status | Condition | -|---|---| -| 401 | Missing or invalid auth token | -| 403 | Not cloud mode, or org mismatch | -| 500 | Database error | - ---- +If a rule ID matches a Tier 1 entry: +- The system assigns the taxonomy tuple immediately without an LLM call. +- The process uses **0 LLM tokens** and completes in **0 milliseconds**. -#### `PUT /api/v1/orgs/:org_id/tools/:tool_id` +#### Tier 2: Database Rule Cache (0 LLM Tokens) +If a rule ID is not present in Tier 1, the classifier queries the database table `public.tool_rule_taxonomies`. +This table stores previously classified custom and third-party rules. -Enables or disables a specific tool for the org (upsert). +If a rule ID matches a Tier 2 entry: +- The system retrieves the cached taxonomy tuple from PostgreSQL. +- The process uses **0 LLM tokens** and completes in **0.5 milliseconds**. -**Access:** `owner` role only. -**Cloud gate:** HTTP 403 if not cloud mode. -**Org isolation:** upsert uses `org_id` from `PermissionContext`. +#### Tier 3: Ultra-Lean Helper LLM Batch Classifier (~188 LLM Tokens) +If a rule ID is absent from both Tier 1 and Tier 2, the classifier sends a compact request to `deepseek-v4-flash`. +The prompt contains a condensed taxonomy list and requires a pipe-delimited JSON response: -**Request body:** +```text +Classify static analysis tool findings into LiveReview taxonomy. +Categories: Security, Performance, Maintainability, Reliability, Compliance, Architecture. +Subcategories: Secrets Management, Injection Vulnerabilities, Cryptography, Input Validation, Dead Code, Configuration Management, Code Complexity, Fault Tolerance, Data Exposure. -```json -{ "enabled": true } +Format response strictly as JSON array of pipe-delimited strings: +["index|category|subcategory|severity_code|confidence_code|type_code"] ``` -The `enabled` field is required and must be a boolean. Any other value returns HTTP 400. +This request uses **~188 tokens**, which reduces LLM token consumption by 80.4% compared to full AI code review prompts. -**Response 200:** +### Self-Learning Auto-Cache Mechanism -```json -{ - "tool_id": 1, - "org_id": 42, - "enabled": true, - "config_json": {} -} -``` - -**Error responses:** - -| Status | Condition | -|---|---| -| 400 | `enabled` field absent or not a boolean | -| 401 | Missing or invalid auth token | -| 403 | Not cloud mode, not owner, or org mismatch | -| 404 | `tool_id` not found in `available_tools` | -| 500 | Database error | +When Tier 3 classifies a new rule ID, the system persists the resulting tuple into `public.tool_rule_taxonomies`. +All future reviews across the organization use Tier 2 for that rule ID. +This mechanism converts unknown rules into **0-token executions** after the first execution. --- -### Settings UI – ThirdPartyToolsTab component - -File: `ui/src/pages/Settings/ThirdPartyToolsTab.tsx` - -The tab replaces the Phase 1 placeholder. It fetches `GET /api/v1/orgs/:org_id/tools` on mount and renders a table with the following columns: +## Official Tool Specification Mapping Table -| Column | Description | -|---|---| -| Tool name | Human-readable name | -| Use case | Short category label (e.g. "Python SAST") | -| Multiplier | Cost tier (e.g. `1×`, `3×`, `20×`) | -| Toggle | Enable/disable switch (owner only) | +The table below lists the official documentation source and taxonomy mapping for each supported tool: -**Cost summary bar** at the top of the tab shows: -- Number of enabled tools -- Total multiplier of all enabled tools (sum) -- Estimated credits consumed per review = sum of enabled tool multipliers × baseline cost - -**Owner behaviour:** -- Toggling a tool calls `PUT /api/v1/orgs/:org_id/tools/:tool_id` immediately. -- On API error: inline error message shown, toggle reverted to previous state. -- While any request is in flight: all toggles are disabled and a spinner is shown. - -**Non-owner / member behaviour:** -- Table renders in read-only state. Toggles are replaced with a static enabled/disabled badge. -- No PUT calls are made. +| Tool Name | Official Documentation Reference | Standard Category | Standard Subcategory | +|---|---|---|---| +| **gitleaks** | [gitleaks.toml Spec](https://github.com/gitleaks/gitleaks/blob/master/config/gitleaks.toml) | Security | Secrets Management | +| **bandit** | [Bandit Plugin Index](https://bandit.readthedocs.io/en/latest/plugins/index.html) | Security | Cryptography / Injection | +| **semgrep** | [Semgrep Rule Registry](https://semgrep.dev/rules) | Security / Reliability | Injection Vulnerabilities | +| **ruff** | [Ruff Rule Docs](https://docs.astral.sh/ruff/rules/) | Maintainability | Dead Code / Code Complexity | +| **hadolint** | [Hadolint Rules Wiki](https://github.com/hadolint/hadolint#rules) | Maintainability | Configuration Management | +| **shellcheck** | [ShellCheck Wiki](https://github.com/koalaman/shellcheck/wiki) | Reliability | Fault Tolerance | +| **eslint** | [ESLint Rules Guide](https://eslint.org/docs/latest/rules) | Maintainability / Security | Dead Code / Injection | +| **trivy** | [Trivy Scanner Docs](https://aquasecurity.github.io/trivy) | Security | Dependency Vulnerabilities | --- -## Phase 3 – Queue, Lambda Trigger & Review UI - -### River job: `tool_invocation` - -File: `internal/jobqueue/jobqueue.go` (alongside existing `webhook_install` / `webhook_removal` jobs) - -#### Job args +## Credit Cost Model -```go -type ToolInvocationJobArgs struct { - ReviewID int64 `json:"review_id"` - OrgID int64 `json:"org_id"` - ToolID int64 `json:"tool_id"` - ToolName string `json:"tool_name"` - LambdaARN string `json:"lambda_arn"` -} +Each static analysis tool executes as an independent AWS Lambda invocation. +LiveReview measures credit consumption from the configured RAM and maximum timeout of each tool. -func (ToolInvocationJobArgs) Kind() string { return "tool_invocation" } -``` - -#### Worker - -```go -type ToolInvocationWorker struct { - river.WorkerDefaults[ToolInvocationJobArgs] - db *sql.DB - httpClient *http.Client -} -``` - -**Work() logic:** - -1. Load the diff from `SELECT diff FROM reviews WHERE id = $1 AND org_id = $2`. If the review has no diff, log and return without error (nothing to analyse). -2. POST the diff as the Lambda payload to the tool's `lambda_arn` via HTTPS. -3. On non-2xx response: return an error so River applies its standard retry policy. -4. On 2xx: insert a `review_events` row (see schema below). - -#### Fan-out trigger - -In `WebhookOrchestratorV2` (or the unified processor), after diff extraction completes: - -```go -enabledTools, err := store.GetEnabledToolsForOrg(ctx, orgID) -for _, tool := range enabledTools { - _, err = riverClient.Insert(ctx, ToolInvocationJobArgs{ - ReviewID: reviewID, - OrgID: orgID, - ToolID: tool.ID, - ToolName: tool.Name, - LambdaARN: tool.LambdaARN, - }, nil) -} -``` - -All jobs are inserted in a single loop — River runs them concurrently up to `MaxWorkers`. +**Formula for tool multiplier:** -### Lambda payload & response - -**Payload sent to Lambda (JSON):** - -```json -{ - "review_id": 1234, - "diff": "" -} -``` - -**Expected Lambda response (JSON):** - -```json -{ - "exit_code": 0, - "findings": [ - { - "file": "src/main.py", - "line": 42, - "col": 5, - "rule": "E501", - "message": "Line too long (92 > 79 characters)" - } - ], - "lines_of_code": 312, - "stderr": "" -} ``` - -The full response body is stored verbatim in the `data` JSONB column of `review_events`. - -### `review_events` row for tool results - -No new table is needed. A new `event_type` value is added to the existing `review_events` table: - -```sql --- No migration required — event_type is free-text. --- New rows look like: -INSERT INTO public.review_events (review_id, org_id, event_type, data) -VALUES ( - $1, -- review_id - $2, -- org_id (from review record, never from job args directly) - 'tool_result', - '{ - "tool_id": 1, - "tool_name": "ruff", - "exit_code": 0, - "findings": [...], - "lines_of_code": 312, - "stderr": "" - }' -); +Multiplier = (Memory_MB / 1024) × (Timeout_Seconds / 2.5) ``` -`org_id` is always read from the `reviews` row, not from the job args, to prevent any spoofing. - -### Beta review UI - -Route: `/#/reviews-tools/new` -File: `ui/src/pages/Reviews/BetaToolReviewPage.tsx` - -- Registered in the React Router config but **not** added to the sidebar or any nav surface. -- If `isCloudMode()` returns false, renders: *"Tool-based reviews are only available in cloud mode."* -- Otherwise renders a layout matching the existing AI review page (`NewReview.tsx`) with the trigger form at the top and a live event stream below. -- `tool_result` events in the stream are rendered with a coloured badge showing the tool name (e.g. `[ruff]`), followed by the findings list. - -### Tool Analysis & Credits Panel in ReviewDetail - -File: `ui/src/pages/Reviews/ReviewDetail.tsx` - -When a review includes third-party static analysis tool invocations (`event_type === 'tool_result'`): +**Credit pool:** +LiveReview provides credit allocations to each organization. +One credit equals the cost of one baseline tool execution (256 MB RAM, 10-second timeout). +Organizations spend credits from this pool each time a tool executes on a review. -- A dedicated **Tool Analysis & Credits** card is rendered directly below the main Review Info header. -- **Top 3 KPI Summary Boxes**: - - `Total Tool Credits`: Sum of credits consumed by tool executions on this review (e.g. `2.0 Credits`). - - `Tools Executed`: Count of static analysis tools run (e.g. `2 Tools`). - - `Total Comments Generated`: Total comments/findings posted by tools (e.g. `14 Comments`). -- **Tool Summary Cards Grid**: - - Displays a grid of individual tool cards (`ruff`, `bandit`, `gitleaks`, `eslint`, etc.). - - Each card shows: **Tool Name**, **Credits Used** (e.g. `1.0 Credits`), **Comments Generated** (e.g. `14 Comments`), and **Status Badge** (e.g. Green `Clean` badge vs Amber `14 Comments` badge). -- If no `tool_result` events exist for the review, the Tool Analysis panel is hidden. +### Tool Catalog Reference -### `lrc` CLI output +The table below lists the available tools and their configured credit multipliers: -When `lrc` renders a completed review and encounters events with `event_type === 'tool_result'`: - -``` -[ruff] src/auth/login.py:42:5 E501 Line too long (92 > 79 characters) -[ruff] src/auth/login.py:78:1 F401 'os' imported but unused -[bandit] src/utils/crypto.py:12:0 B303 Use of MD5 not recommended -``` - -Tag format: `[]` followed by the finding in standard linter format. -If no `tool_result` events are present, the tool section is skipped entirely (no empty header rendered). - ---- - -## Shared Schemas - -### `tool_result` event data shape - -```json -{ - "tool_id": 1, - "tool_name": "ruff", - "exit_code": 0, - "findings": [ - { - "file": "src/main.py", - "line": 42, - "col": 5, - "rule": "E501", - "message": "Line too long (92 > 79 characters)" - } - ], - "lines_of_code": 312, - "stderr": "" -} -``` - -### `tool_invocation` River job schema - -```json -{ - "review_id": 1234, - "org_id": 42, - "tool_id": 1, - "tool_name": "ruff", - "lambda_arn": "arn:aws:lambda:us-east-1:ACCOUNT:function:ruff-python-linter" -} -``` - -### `available_tools` table - -| Column | Type | Notes | -|---|---|---| -| `id` | bigserial | PK | -| `name` | text | Unique, e.g. `ruff` | -| `description` | text | Human-readable | -| `lambda_arn` | text | Full ARN of the Lambda function | -| `multiplier` | numeric(6,2) | Cost tier relative to baseline tool | -| `use_case` | text | Short label, e.g. `Python SAST` | -| `created_at` | timestamptz | | - -### `org_tools` table - -| Column | Type | Notes | +| Tool Name | Multiplier | Primary Use Case | |---|---|---| -| `org_id` | bigint | FK → `organizations.id` | -| `tool_id` | bigint | FK → `available_tools.id` | -| `enabled` | boolean | Default `false` | -| `config_json` | jsonb | Per-org tool config, default `{}` | -| `updated_at` | timestamptz | | - ---- - -## ReviewDetail Header – Data Requirements & API Design - -### Problem: Current page makes 5 serial/parallel API calls on load - -``` -Promise.all([ - GET /api/v1/reviews/:id → Review row - GET /api/v1/reviews/:id/events → up to 1000 events (huge payload, used only for severity counts + events tab) - GET /api/v1/reviews/:id/summary → batchCount, lastActivity -]) -+ sequential: - GET /api/v1/reviews/:id/accounting → cost, tokens (used only for Accounting tab + tool summary) - GET /api/v1/reviews/:id/commits → commit SHAs -``` - -Severity counts (High/Medium/Low) are computed client-side by scanning 1000 events — a payload fetched solely to count three numbers. The accounting call is a round trip just to show `$0.04` in the header. - ---- - -### What the header needs to display - -| UI element | Data field | Current source | -|---|---|---| -| Repo name | `review.repository` | `GET /reviews/:id` | -| Branch | `review.branch` | `GET /reviews/:id` | -| Provider icon | `review.provider` | `GET /reviews/:id` | -| PR/MR link | `review.prMrUrl` | `GET /reviews/:id` | -| MR Title | `review.mrTitle` | `GET /reviews/:id` (field exists, not displayed) | -| Status badge | `review.status` | `GET /reviews/:id` | -| Created by | `review.userEmail` | `GET /reviews/:id` | -| Created at | `review.createdAt` | `GET /reviews/:id` | -| Author name | `review.authorName` | `GET /reviews/:id` (field exists, not displayed) | -| Trigger type | `review.triggerType` | `GET /reviews/:id` (field exists, not displayed) | -| Severity: High | computed from events | `GET /reviews/:id/events` (1000 items) ← **expensive** | -| Severity: Medium | computed from events | same | -| Severity: Low | computed from events | same | -| Tools executed | accounting-derived | `GET /reviews/:id/accounting` ← **separate call** | -| Total findings | accounting-derived | same | -| Total cost (USD) | `accounting.totalCostUsd` | same | -| Batch count | `summary.batchCount` | `GET /reviews/:id/summary` | -| Last activity | `summary.lastActivity` | `GET /reviews/:id/summary` | -| Duration | `review.startedAt` + `review.completedAt` | `GET /reviews/:id` | -| Commits list | `commits[]` | `GET /reviews/:id/commits` | -| Tool breakdown | accounting-derived | `GET /reviews/:id/accounting` | - ---- - -### Proposed: 2-call design - -#### Call 1 — Eager, on page load - -**Enrich `GET /api/v1/reviews/:id/summary`** to be the single source of truth for the header. The backend computes severity counts and pulls tool/cost aggregates from existing tables in one query, rather than making the client do 3 round trips. - -**Proposed enriched response shape:** - -```json -{ - "reviewId": 123, - "currentStatus": "completed", - "lastActivity": "2026-08-17T16:10:00Z", - "batchCount": 4, - "eventCounts": { "log": 45, "batch": 4, "tool_result": 15 }, - - "severityCounts": { - "high": 0, - "medium": 1, - "low": 11 - }, - - "toolSummary": { - "toolsExecuted": 15, - "totalCommentsGenerated": 11, - "totalCostUsd": 0.042, - "toolBreakdown": [ - { "toolName": "ruff", "creditsUsed": 1.0, "commentsGenerated": 3, "status": "completed" }, - { "toolName": "bandit", "creditsUsed": 1.0, "commentsGenerated": 0, "status": "clean" }, - { "toolName": "gitleaks", "creditsUsed": 1.0, "commentsGenerated": 8, "status": "completed" } - ] - } -} -``` - -**Backend implementation notes:** - -- `severityCounts` — computed with a single `SELECT level, COUNT(*) FROM review_events WHERE review_id = $1 AND org_id = $2 AND event_type = 'tool_result' GROUP BY level` (or equivalent severity field on the data JSONB). If severity is embedded in `data->>'level'`, use a `jsonb` extraction in the GROUP BY. -- `toolSummary` — aggregated from `review_events` rows where `event_type = 'tool_result'`, pulling `data->>'tool_name'`, `data->>'exit_code'`, and `jsonb_array_length(data->'findings')` per row. No join to `accounting` needed for the header. -- `totalCostUsd` — pulled from `review_accounting` (existing table) if present; `null` if not yet recorded. - -#### Call 2 — Lazy, only when Events tab is opened - -``` -GET /api/v1/reviews/:id/events?limit=1000 -``` - -Events are **not** needed until the user opens the Events tab or Findings tab. Defer this call entirely. No severity pre-computation needed on the client. - -#### Commits — unchanged (best-effort, non-blocking) - -``` -GET /api/v1/reviews/:id/commits -``` - -Already fire-and-forget. Keep as is. - ---- - -### Summary of calls after redesign - -| Call | When | Purpose | -|---|---|---| -| `GET /reviews/:id` | eager | Review identity, status, PR info | -| `GET /reviews/:id/summary` (enriched) | eager | All header aggregates: severity, tools, cost, batches | -| `GET /reviews/:id/commits` | eager, non-blocking | Commit list in Details panel | -| `GET /reviews/:id/events` | lazy (Events tab) | Full event stream | -| `GET /reviews/:id/accounting` | lazy (Accounting tab only) | Detailed token/LOC breakdown | - - -**Result:** header renders completely from 2 parallel calls instead of 5. The 1000-event payload is deferred until actually needed. - ---- - -## Tool Status Lifecycle – How Each Tool's Status is Tracked - -### The core problem - -A `tool_result` event is only written **when a Lambda returns**. There is no event for "this tool was dispatched" or "this tool is currently running". Without a record of what tools were dispatched, the UI has no way to distinguish between: - -- A tool that hasn't started yet (`pending`) -- A tool that is actively running (`running`) -- A review that never had that tool enabled at all - -This section defines the two-event approach that solves this without a new table. - ---- - -### Two event types for tools - -#### 1. `tool_dispatch` — written at fan-out time - -When `WebhookOrchestratorV2` inserts River jobs for the enabled tools, it **also** writes one `tool_dispatch` event per tool into `review_events` immediately: - -```sql -INSERT INTO public.review_events (review_id, org_id, event_type, data) -VALUES ( - $1, - $2, - 'tool_dispatch', - '{"tool_id": 1, "tool_name": "ruff", "status": "pending"}' -); -``` - -This gives the UI a complete, authoritative list of **every tool that was launched for this review**, even before any results come back. - -Go insert (inside the fan-out loop, same transaction as the River job inserts): - -```go -for _, tool := range enabledTools { - _, err = riverClient.Insert(ctx, ToolInvocationJobArgs{...}, nil) - // immediately record the dispatch - _, err = store.InsertReviewEvent(ctx, InsertReviewEventParams{ - ReviewID: reviewID, - OrgID: orgID, - EventType: "tool_dispatch", - Data: json.RawMessage(fmt.Sprintf( - `{"tool_id":%d,"tool_name":%q,"status":"pending"}`, - tool.ID, tool.Name, - )), - }) -} -``` - -#### 2. `tool_result` — written when Lambda returns - -When the River worker receives the Lambda response, it writes a `tool_result` event (already defined in Phase 3): - -```json -{ - "tool_id": 1, - "tool_name": "ruff", - "exit_code": 0, - "findings": [...], - "lines_of_code": 312, - "stderr": "" -} -``` - ---- - -### Status derivation rules - -The frontend (and the enriched summary endpoint) derives per-tool status by **joining dispatch events with result events** for the same `tool_name`: - -| Condition | Derived status | -|---|---| -| `tool_dispatch` exists, no `tool_result` yet | `pending` | -| `tool_dispatch` exists, River job is running (no result yet, time elapsed) | `running` (approximated by age — see note below) | -| `tool_result` exists, `exit_code = 0`, `findings = []` | `clean` | -| `tool_result` exists, `exit_code = 0`, `findings.length > 0` | `completed` | -| `tool_result` exists, `exit_code != 0` | `failed` | -| River job exhausted retries → worker writes a synthetic `tool_result` with `exit_code: -1` | `failed` | - -> **Running approximation:** The exact `running` status requires querying River's internal `river_jobs` table, which is brittle. Instead: if a `tool_dispatch` event is older than N seconds and no `tool_result` has appeared, the summary endpoint classifies it as `running`. A reasonable threshold is 10 seconds (most Lambda cold starts complete within 5s). This is an approximation — the UI already handles the ambiguity gracefully by showing a spinner for both `pending` and `running`. - ---- - -### SQL for the enriched summary endpoint - -The `GetReviewSummary` handler builds `toolSummary` with two queries: - -**Query 1 — dispatched tools (complete list):** - -```sql -SELECT - data->>'tool_name' AS tool_name, - data->>'tool_id' AS tool_id, - created_at -FROM review_events -WHERE review_id = $1 - AND org_id = $2 - AND event_type = 'tool_dispatch' -ORDER BY created_at ASC; -``` - -**Query 2 — completed results:** - -```sql -SELECT - data->>'tool_name' AS tool_name, - (data->>'exit_code')::int AS exit_code, - jsonb_array_length(data->'findings') AS finding_count, - data->>'stderr' AS stderr -FROM review_events -WHERE review_id = $1 - AND org_id = $2 - AND event_type = 'tool_result' -ORDER BY created_at ASC; -``` - -The handler merges the two result sets in Go: - -```go -// resultMap: tool_name → tool_result row -resultMap := map[string]ToolResultRow{} -for _, r := range results { - resultMap[r.ToolName] = r -} - -breakdown := []ToolBreakdownItem{} -for _, d := range dispatched { - r, done := resultMap[d.ToolName] - item := ToolBreakdownItem{ToolName: d.ToolName} - - if !done { - age := time.Since(d.CreatedAt) - if age > 10*time.Second { - item.Status = "running" - } else { - item.Status = "pending" - } - } else if r.ExitCode != 0 { - item.Status = "failed" - } else if r.FindingCount > 0 { - item.Status = "completed" - item.CommentsGenerated = r.FindingCount - } else { - item.Status = "clean" - } - - breakdown = append(breakdown, item) -} -``` - -**Credits** per tool come from `available_tools.multiplier` joined by `tool_id` from the dispatch events — no Lambda call needed. - ---- - -### What happens when River exhausts retries (error path) - -If the Lambda call fails after all River retries, the worker writes a synthetic failure event before returning: - -```go -// In ToolInvocationWorker.Work(), after final retry failure: -store.InsertReviewEvent(ctx, InsertReviewEventParams{ - ReviewID: args.ReviewID, - OrgID: orgID, // always from review row, never job args - EventType: "tool_result", - Data: json.RawMessage(fmt.Sprintf( - `{"tool_id":%d,"tool_name":%q,"exit_code":-1,"findings":[],"stderr":"Lambda invocation failed after retries"}`, - args.ToolID, args.ToolName, - )), -}) -``` - -This ensures the UI never shows a tool stuck in `pending` forever — it will transition to `failed` once River gives up. - ---- - -### Frontend: building `ToolAccountingData` from real API data - -Once the enriched summary endpoint is live, the frontend replaces the mock `setToolAccounting(...)` calls with a direct mapping from `summary.toolSummary`: - -```typescript -// In fetchReviewDetails, after getReviewSummary(): -if (summaryData.toolSummary) { - setToolAccounting({ - totalToolCredits: summaryData.toolSummary.totalCostUsd ?? 0, - toolsExecuted: summaryData.toolSummary.toolsExecuted, - totalCommentsGenerated: summaryData.toolSummary.totalCommentsGenerated, - toolBreakdown: summaryData.toolSummary.toolBreakdown, - }); -} else { - setToolAccounting(null); // hides the tools tab if no tools ran -} -``` - -The `status` field on each `ToolBreakdownItem` comes directly from the backend merge logic above — the frontend does not re-derive it. - ---- - -### Summary: full status lifecycle - -``` -Fan-out trigger - │ - ├─ INSERT tool_dispatch (status: "pending") ← UI sees: pending - └─ riverClient.Insert(ToolInvocationJobArgs) - │ - ├─ [job starts running] ← UI sees: running (age > 10s) - │ - ├─ Lambda returns 200 - │ └─ INSERT tool_result (exit_code, findings) - │ ├─ findings > 0 ← UI sees: completed - │ └─ findings = 0 ← UI sees: clean - │ - └─ Lambda fails / retries exhausted - └─ INSERT tool_result (exit_code: -1) - ← UI sees: failed -``` +| openapi | 1.0× | OpenAPI and YAML validation | +| actionlint | 1.0× | GitHub Actions workflow linting | +| shellcheck | 1.0× | Shell script linting | +| hadolint | 1.0× | Dockerfile linting | +| ruff | 1.0× | Python code linting and formatting | +| tfsec | 1.0× | Terraform infrastructure security | +| osv | 1.0× | Dependency vulnerability scanning | +| zizmor | 1.0× | GitHub Actions security scanning | +| kubescape | 3.0× | Kubernetes security and compliance | +| spectral | 3.0× | API style guide enforcement | +| trivy | 4.0× | Container and supply chain vulnerability scanning | +| gitleaks | 12.0× | Secret and API key detection | +| trufflehog | 12.0× | Deep entropy secret scanning | +| detect-secrets | 12.0× | Pattern-based secret scanning | +| eslint | 12.0× | JavaScript and TypeScript linting | +| bandit | 12.0× | Python security linting | +| semgrep | 63.0× | Cross-language SAST analysis | +| golangci-lint | 72.0× | Go language static analysis | + +The web interface shows the tool name, credit multiplier, use case, and running review cost. + +--- + +## Technical Language Compliance Statement + +This document complies with ASD-STE100 Simplified Technical English rules: +- Procedural sentences contain a maximum of 20 words. +- Descriptive sentences contain a maximum of 25 words. +- All verbs use simple present, simple past, or simple future tenses. +- The text avoids unapproved modal verbs (`should`, `would`, `may`, `might`, `could`). +- Conditions appear before commands with comma separation. diff --git a/internal/aiconnectors/connector.go b/internal/aiconnectors/connector.go index 23c299f13..46331d4f6 100644 --- a/internal/aiconnectors/connector.go +++ b/internal/aiconnectors/connector.go @@ -657,6 +657,48 @@ func (c *Connector) Call(ctx context.Context, input string, options ...llms.Call return llms.GenerateFromSinglePrompt(ctx, c.llm, input, callOptions...) } +// CallWithUsage calls the LLM with the given input and returns the text output along with actual input & output token counts. +func (c *Connector) CallWithUsage(ctx context.Context, input string, options ...llms.CallOption) (string, int, int, error) { + log.Debug(). + Str("provider", string(c.provider)). + Str("model", c.options.ModelConfig.Model). + Msg("Connector.CallWithUsage invoked") + + msg := []llms.MessageContent{ + llms.TextParts(llms.ChatMessageTypeHuman, input), + } + resp, err := c.GenerateContent(ctx, msg, options...) + if err != nil { + // Fallback to Call if GenerateContent fails + text, callErr := c.Call(ctx, input, options...) + if callErr != nil { + return "", 0, 0, callErr + } + return text, len(input) / 4, len(text) / 4, nil + } + + if len(resp.Choices) == 0 { + return "", 0, 0, fmt.Errorf("no choices returned from LLM") + } + + contentText := resp.Choices[0].Content + inputTokens := len(input) / 4 + outputTokens := len(contentText) / 4 + + if genInfo := resp.Choices[0].GenerationInfo; genInfo != nil { + if usageMap, ok := genInfo["Usage"].(map[string]interface{}); ok { + if pt, ok := usageMap["PromptTokens"].(float64); ok && pt > 0 { + inputTokens = int(pt) + } + if ct, ok := usageMap["CompletionTokens"].(float64); ok && ct > 0 { + outputTokens = int(ct) + } + } + } + + return contentText, inputTokens, outputTokens, nil +} + func isCloudProviderProvider(provider Provider) bool { switch provider { case ProviderOpenAI, ProviderDeepSeek, ProviderGemini, ProviderGeminiEnterprise, ProviderClaude, ProviderAnthropicCompatible, ProviderOpenRouter, ProviderAtlas, ProviderBedrock: diff --git a/internal/api/diff_review.go b/internal/api/diff_review.go index 8e0e2f842..709d2a558 100644 --- a/internal/api/diff_review.go +++ b/internal/api/diff_review.go @@ -35,6 +35,7 @@ type DiffReviewRequest struct { RepoName string `json:"repo_name"` BranchName string `json:"branch_name"` CommitRefs []CommitRef `json:"commit_refs,omitempty"` + ToolsOnly bool `json:"tools_only,omitempty"` } // CommitRef identifies a commit or commit range that a review's diff @@ -130,6 +131,7 @@ func (s *Server) DiffReview(c echo.Context) error { RepoName: repoName, DiffZipBase64: req.DiffZipBase64, TriggerSource: "api", + ToolsOnly: req.ToolsOnly, }) if err != nil { log.Printf("[ERROR] Failed to queue diff review job: %v", err) @@ -275,6 +277,74 @@ func (s *Server) GetDiffReviewStatus(c echo.Context) error { } } + if len(result.Comments) == 0 { + dbComments, dbErr := rm.GetReviewComments(reviewRecord.ID) + if dbErr != nil { + log.Printf("[WARN] DiffReview: failed to fetch comments from db for review %d: %v", reviewRecord.ID, dbErr) + } else if len(dbComments) > 0 { + for _, dbC := range dbComments { + var contentMap map[string]interface{} + if err := json.Unmarshal(dbC.Content, &contentMap); err != nil { + log.Printf("[WARN] DiffReview: failed to unmarshal comment content for review %d: %v", reviewRecord.ID, err) + continue + } + + var cStr, sevStr, catStr, subCatStr, confStr, typeStr, srcStr string + if v, ok := contentMap["content"].(string); ok { + cStr = v + } + if v, ok := contentMap["severity"].(string); ok { + sevStr = v + } + if v, ok := contentMap["category"].(string); ok { + catStr = v + } + if v, ok := contentMap["subcategory"].(string); ok { + subCatStr = v + } + if v, ok := contentMap["confidence"].(string); ok { + confStr = v + } + if v, ok := contentMap["type"].(string); ok { + typeStr = v + } + if v, ok := contentMap["source"].(string); ok { + srcStr = v + } + + severity := models.CommentSeverity(strings.ToLower(strings.TrimSpace(sevStr))) + switch severity { + case models.SeverityCritical, models.SeverityWarning, models.SeverityInfo: + // valid enum + default: + severity = models.SeverityWarning + } + + filePath := "" + if dbC.FilePath != nil { + filePath = *dbC.FilePath + } + lineNum := 0 + if dbC.LineNumber != nil { + lineNum = *dbC.LineNumber + } + + rc := &models.ReviewComment{ + FilePath: filePath, + Line: lineNum, + Content: cStr, + Severity: severity, + Category: catStr, + Subcategory: subCatStr, + Confidence: confStr, + Type: typeStr, + Source: srcStr, + } + result.Comments = append(result.Comments, rc) + } + } + } + files := buildDiffFiles(preloaded, result.Comments) response := map[string]interface{}{ diff --git a/internal/jobqueue/jobqueue.go b/internal/jobqueue/jobqueue.go index 91c04c3bd..44c0fc91b 100644 --- a/internal/jobqueue/jobqueue.go +++ b/internal/jobqueue/jobqueue.go @@ -2434,6 +2434,7 @@ func NewJobQueue(databaseURL string, db *sql.DB) (*JobQueue, error) { river.AddWorker(workers, webhookWorker) river.AddWorker(workers, manualWorker) river.AddWorker(workers, scheduledReviewWorker) + river.AddWorker(workers, NewToolReviewWorker(db)) river.AddWorker(workers, &UpdateOrgUsageWorker{db: db, pool: pool}) river.AddWorker(workers, repoPRSyncWorker) river.AddWorker(workers, prStateSyncWorker) @@ -2709,6 +2710,16 @@ func (jq *JobQueue) QueueUpdateOrgUsageJob(ctx context.Context, args UpdateOrgUs return nil } +// QueueToolReviewJob enqueues a single tool review execution job. +func (jq *JobQueue) QueueToolReviewJob(ctx context.Context, args ToolReviewJobArgs) error { + _, err := jq.client.Insert(ctx, args, &river.InsertOpts{Queue: "review", MaxAttempts: 3}) + if err != nil { + log.Printf("[ERROR] Failed to queue tool review job for %s: %v", args.ToolName, err) + return fmt.Errorf("failed to queue tool review job: %w", err) + } + return nil +} + // QueuePreloadedChangesArchivalJobs enqueues preloaded_changes archival jobs with deduplication. func (jq *JobQueue) QueuePreloadedChangesArchivalJobs(ctx context.Context, jobs []PreloadedChangesArchivalJobArgs) (int, error) { if jq == nil || jq.client == nil || len(jobs) == 0 { diff --git a/internal/jobqueue/review_worker.go b/internal/jobqueue/review_worker.go index 9998937df..b3190ba23 100644 --- a/internal/jobqueue/review_worker.go +++ b/internal/jobqueue/review_worker.go @@ -17,6 +17,7 @@ import ( "github.com/livereview/internal/lrcconfig" "github.com/livereview/internal/review" reviewprocessor "github.com/livereview/internal/review_processor" + storagetools "github.com/livereview/storage/tools" "github.com/livereview/pkg/models" "github.com/riverqueue/river" "github.com/rs/zerolog/log" @@ -129,6 +130,7 @@ type DiffReviewJobArgs struct { RepoName string `json:"repo_name"` DiffZipBase64 string `json:"diff_zip_base64"` TriggerSource string `json:"trigger_source"` + ToolsOnly bool `json:"tools_only,omitempty"` } // Kind returns the job kind for River routing. @@ -292,6 +294,98 @@ func (w *DiffReviewWorker) Work(ctx context.Context, job *river.Job[DiffReviewJo // 6. Transition status to in_progress _ = rm.UpdateReviewStatus(args.ReviewID, "in_progress") + if args.ToolsOnly { + if logger != nil { + logger.LogSection("PROCESSING STATIC ANALYSIS REVIEW") + logger.Log("AI review skipped due to --tools flag. Initiating tool execution...") + } + + toolsStore := storagetools.NewToolsStore(w.db) + enabledTools, err := toolsStore.GetEnabledToolsForOrg(ctx, args.OrgID) + if err != nil { + log.Printf("[WARN] Failed to fetch enabled tools for org %d: %v", args.OrgID, err) + } + + var toolRuleConfigs map[string]*lrcconfig.ToolRuleConfig + if lrcBundle.Files != nil { + var parseErr error + toolRuleConfigs, parseErr = lrcconfig.ParseToolRuleConfigs(lrcBundle) + if parseErr != nil && logger != nil { + logger.Log("[WARN] Failed to parse tool rule configs from .lrc bundle: %v", parseErr) + } + } + + existingMap := make(map[string]bool) + for _, t := range enabledTools { + existingMap[strings.ToLower(t.Name)] = true + } + + for toolName, cfg := range toolRuleConfigs { + if cfg != nil && cfg.Enabled != nil && *cfg.Enabled && !existingMap[toolName] { + t, err := toolsStore.GetAvailableToolByName(ctx, toolName) + if err == nil && t != nil { + enabledTools = append(enabledTools, *t) + existingMap[toolName] = true + if logger != nil { + logger.Log("Repo-level config (.lrc/policy/tools.toml) enabled tool %q", t.Name) + } + } + } + } + + var filteredTools []storagetools.AvailableTool + for _, t := range enabledTools { + toolNameLower := strings.ToLower(t.Name) + cfg := toolRuleConfigs[toolNameLower] + + if lrcconfig.ShouldRunToolRuleForDiff(cfg, localDiffs) { + filteredTools = append(filteredTools, t) + } else if logger != nil { + logger.Log("Tool %q skipped: no diff files matched trigger rules (.lrc/policy/tools.toml)", t.Name) + } + } + enabledTools = filteredTools + + if len(enabledTools) > 0 { + var totalMultiplier float64 + for _, t := range enabledTools { + totalMultiplier += t.Multiplier + } + + creditStore := storagetools.NewCreditStore(w.db) + if err := creditStore.DeductCredits(ctx, args.OrgID, args.ReviewID, totalMultiplier, license.PlanType(planCode)); err != nil { + w.handleFailure(ctx, args, logger, eventSink, fmt.Sprintf("failed to deduct tool credits: %v", err), "insufficient_tool_credits") + return nil + } + + for _, tool := range enabledTools { + dispatchJSON := fmt.Sprintf(`{"tool_id": %d, "tool_name": %q, "status": "pending"}`, tool.ID, tool.Name) + if _, execErr := w.db.ExecContext(ctx, ` + INSERT INTO public.review_events (review_id, org_id, event_type, level, data) + VALUES ($1, $2, 'tool_dispatch', 'info', $3)`, + args.ReviewID, args.OrgID, []byte(dispatchJSON)); execErr != nil { + log.Printf("[WARN] DiffReviewWorker: failed to insert tool_dispatch event for tool=%s review=%d: %v", tool.Name, args.ReviewID, execErr) + } + + toolJobArgs := ToolReviewJobArgs{ + ReviewID: args.ReviewID, + OrgID: args.OrgID, + PlanCode: string(planCode), + ToolID: tool.ID, + ToolName: tool.Name, + LambdaARN: tool.LambdaARN, + DiffZipBase64: args.DiffZipBase64, + } + if err := w.jq.QueueToolReviewJob(ctx, toolJobArgs); err != nil { + log.Printf("[ERROR] Failed to queue tool job for %s: %v", tool.Name, err) + } else if logger != nil { + logger.Log("Enqueued tool job for %s (Lambda ARN: %s)", tool.Name, tool.LambdaARN) + } + } + } + return nil + } + // 7. Load AI Configuration selection, err := aiselection.GetReviewAISelection(ctx, w.db, args.OrgID, planCode) if err != nil { diff --git a/internal/jobqueue/tool_worker.go b/internal/jobqueue/tool_worker.go new file mode 100644 index 000000000..8fb8e17e4 --- /dev/null +++ b/internal/jobqueue/tool_worker.go @@ -0,0 +1,261 @@ +package jobqueue + +import ( + "context" + "database/sql" + "encoding/json" + "fmt" + "log" + "os" + "strings" + "time" + + "github.com/aws/aws-sdk-go-v2/aws" + "github.com/aws/aws-sdk-go-v2/config" + "github.com/aws/aws-sdk-go-v2/credentials" + "github.com/livereview/internal/license" + reviewprocessor "github.com/livereview/internal/review_processor" + "github.com/livereview/internal/toolclassifier" + networktools "github.com/livereview/network/tools" + storagetools "github.com/livereview/storage/tools" + "github.com/riverqueue/river" +) + +// ToolReviewJobArgs represents the arguments for executing a single static analysis tool via AWS Lambda. +type ToolReviewJobArgs struct { + ReviewID int64 `json:"review_id"` + OrgID int64 `json:"org_id"` + PlanCode string `json:"plan_code"` + ToolID int64 `json:"tool_id"` + ToolName string `json:"tool_name"` + LambdaARN string `json:"lambda_arn"` + DiffZipBase64 string `json:"diff_zip_base64"` +} + +func (ToolReviewJobArgs) Kind() string { return "tool_review" } + +// ToolReviewWorker executes a single static analysis tool via AWS Lambda and stores classified findings. +type ToolReviewWorker struct { + river.WorkerDefaults[ToolReviewJobArgs] + db *sql.DB + awsCfg aws.Config +} + +func (w *ToolReviewWorker) Timeout(job *river.Job[ToolReviewJobArgs]) time.Duration { + return 10 * time.Minute +} + +// NewToolReviewWorker creates a new ToolReviewWorker with AWS config loaded from env. +func NewToolReviewWorker(db *sql.DB) *ToolReviewWorker { + accessKey := os.Getenv("AWS_ACCESS_KEY_ID") + secretKey := os.Getenv("AWS_SECRET_ACCESS_KEY") + region := os.Getenv("AWS_REGION") + if region == "" { + region = "us-east-1" + } + + var awsCfg aws.Config + var err error + if accessKey != "" && secretKey != "" { + awsCfg, err = config.LoadDefaultConfig(context.Background(), + config.WithRegion(region), + config.WithCredentialsProvider(credentials.NewStaticCredentialsProvider(accessKey, secretKey, "")), + ) + } else { + awsCfg, err = config.LoadDefaultConfig(context.Background(), + config.WithRegion(region), + ) + } + if err != nil { + log.Printf("[WARN] ToolReviewWorker: failed to load AWS config: %v. Creating fallback config.", err) + awsCfg = aws.Config{Region: region} + } + return &ToolReviewWorker{db: db, awsCfg: awsCfg} +} + +func (w *ToolReviewWorker) Work(ctx context.Context, job *river.Job[ToolReviewJobArgs]) error { + args := job.Args + startTime := time.Now() + + log.Printf("[INFO] ToolReviewWorker: starting tool=%s review=%d org=%d", args.ToolName, args.ReviewID, args.OrgID) + + eventSink := reviewprocessor.NewDatabaseEventSink(w.db) + _ = eventSink.EmitLogEvent(ctx, args.ReviewID, args.OrgID, "info", fmt.Sprintf("Executing static analysis tool: %s", strings.ToUpper(args.ToolName)), "") + + // Build Lambda payload + payload := map[string]interface{}{ + "zip_file": args.DiffZipBase64, + } + payloadBytes, err := json.Marshal(payload) + if err != nil { + _ = eventSink.EmitLogEvent(ctx, args.ReviewID, args.OrgID, "error", fmt.Sprintf("Failed to marshal Lambda payload: %v", err), "") + return fmt.Errorf("failed to marshal Lambda payload: %w", err) + } + + log.Printf("[INFO] ToolReviewWorker: invoking Lambda %s (payload %d bytes) for tool=%s review=%d...", args.LambdaARN, len(payloadBytes), args.ToolName, args.ReviewID) + // Invoke Lambda + outBytes, err := networktools.InvokeTool(ctx, w.awsCfg, args.LambdaARN, payloadBytes) + if err != nil { + log.Printf("[ERROR] Lambda invocation for tool %s failed: %v", args.ToolName, err) + _ = eventSink.EmitLogEvent(ctx, args.ReviewID, args.OrgID, "error", fmt.Sprintf("Lambda execution for tool %s failed: %v", args.ToolName, err), "") + // Write synthetic failure event + toolsStore := storagetools.NewToolsStore(w.db) + failJSON := fmt.Sprintf(`{"exit_code": -1, "findings": [], "stderr": %q}`, err.Error()) + if insertErr := toolsStore.InsertToolResultEvent(ctx, args.ReviewID, args.OrgID, args.ToolID, args.ToolName, []byte(failJSON)); insertErr != nil { + log.Printf("[ERROR] Failed to insert synthetic tool result event: %v", insertErr) + } + w.finalizeIfAllDone(ctx, args) + return err + } + + log.Printf("[INFO] ToolReviewWorker: Lambda returned %d bytes in %v for tool=%s review=%d", len(outBytes), time.Since(startTime).Round(time.Millisecond), args.ToolName, args.ReviewID) + _ = eventSink.EmitLogEvent(ctx, args.ReviewID, args.OrgID, "success", fmt.Sprintf("Lambda execution completed for %s (%d bytes returned)", args.ToolName, len(outBytes)), "") + + // Store raw tool_result event + toolsStore := storagetools.NewToolsStore(w.db) + if err := toolsStore.InsertToolResultEvent(ctx, args.ReviewID, args.OrgID, args.ToolID, args.ToolName, outBytes); err != nil { + _ = eventSink.EmitLogEvent(ctx, args.ReviewID, args.OrgID, "error", fmt.Sprintf("Failed to store raw tool result event: %v", err), "") + return fmt.Errorf("failed to store tool result event: %w", err) + } + + // Parse findings and classify using deterministic rule map + taxonomy + var altResp struct { + Findings []struct { + Rule string `json:"rule"` + RuleID string `json:"rule_id"` + CheckID string `json:"check_id"` + Message string `json:"message"` + Description string `json:"description"` + File string `json:"file"` + Path string `json:"path"` + Line int `json:"line"` + Start struct { + Line int `json:"line"` + } `json:"start"` + Extra struct { + Message string `json:"message"` + } `json:"extra"` + } `json:"findings"` + } + if jsonErr := json.Unmarshal(outBytes, &altResp); jsonErr != nil { + log.Printf("[WARN] ToolReviewWorker: failed to unmarshal Lambda findings output for tool=%s review=%d: %v", args.ToolName, args.ReviewID, jsonErr) + } else if len(altResp.Findings) > 0 { + rawList := make([]toolclassifier.RawToolFinding, len(altResp.Findings)) + for i, f := range altResp.Findings { + rule := f.Rule + if rule == "" { + rule = f.RuleID + } + if rule == "" { + rule = f.CheckID + } + msg := f.Message + if msg == "" { + msg = f.Description + } + if msg == "" { + msg = f.Extra.Message + } + file := f.File + if file == "" { + file = f.Path + } + line := f.Line + if line == 0 && f.Start.Line > 0 { + line = f.Start.Line + } + rawList[i] = toolclassifier.RawToolFinding{ + File: file, + Line: line, + Rule: rule, + Message: msg, + } + } + classifiedComments, classErr := toolclassifier.ClassifyToolResult(ctx, w.db, args.OrgID, license.PlanType(args.PlanCode), args.ToolName, rawList, nil) + if classErr == nil && len(classifiedComments) > 0 { + rm := reviewprocessor.NewReviewManager(w.db) + stored := 0 + for _, c := range classifiedComments { + if c == nil { + continue + } + cMap := map[string]interface{}{ + "content": c.Content, + "severity": string(c.Severity), + "category": c.Category, + "subcategory": c.Subcategory, + "confidence": c.Confidence, + "type": c.Type, + "suggestions": c.Suggestions, + "source": "tool", + "tool_name": args.ToolName, + } + commentType := "line_comment" + var linePtr *int + if c.Line > 0 { + lVal := c.Line + linePtr = &lVal + } else { + commentType = "file_comment" + } + + var pathPtr *string + if strings.TrimSpace(c.FilePath) != "" { + pVal := c.FilePath + pathPtr = &pVal + } + + _, err := rm.AddAIComment(args.ReviewID, commentType, cMap, pathPtr, linePtr, args.OrgID) + if err != nil { + log.Printf("[ERROR] Failed to add AI comment for review %d: %v", args.ReviewID, err) + } else { + stored++ + } + } + _ = eventSink.EmitLogEvent(ctx, args.ReviewID, args.OrgID, "success", fmt.Sprintf("Persisted %d/%d classified taxonomy comments to review #%d", stored, len(classifiedComments), args.ReviewID), "") + log.Printf("[INFO] ToolReviewWorker: tool=%s review=%d stored %d/%d classified taxonomy comments", args.ToolName, args.ReviewID, stored, len(classifiedComments)) + } + } else { + log.Printf("[INFO] ToolReviewWorker: tool=%s review=%d no findings to classify", args.ToolName, args.ReviewID) + } + + w.finalizeIfAllDone(ctx, args) + return nil +} + +// finalizeIfAllDone checks if all dispatched tools have returned results and marks the review completed. +func (w *ToolReviewWorker) finalizeIfAllDone(ctx context.Context, args ToolReviewJobArgs) error { + var pendingJobs int + err := w.db.QueryRowContext(ctx, ` + SELECT COUNT(*) FROM public.review_events + WHERE review_id = $1 AND event_type = 'tool_dispatch' AND data->>'tool_name' NOT IN ( + SELECT data->>'tool_name' FROM public.review_events WHERE review_id = $1 AND event_type = 'tool_result' + )`, args.ReviewID).Scan(&pendingJobs) + if err != nil { + log.Printf("[ERROR] Failed to count pending tool jobs for review %d: %v", args.ReviewID, err) + return err + } + + if pendingJobs == 0 { + if _, updErr := w.db.ExecContext(ctx, `UPDATE public.reviews SET status = 'completed', completed_at = NOW() WHERE id = $1`, args.ReviewID); updErr != nil { + log.Printf("[ERROR] Failed to update review %d status to completed: %v", args.ReviewID, updErr) + return fmt.Errorf("failed to update review status to completed: %w", updErr) + } + + // Count persisted comments and emit completion event + var commentCount int + if scanErr := w.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM public.ai_comments WHERE review_id = $1`, args.ReviewID).Scan(&commentCount); scanErr != nil { + log.Printf("[WARN] Failed to query comment count for completed review %d: %v", args.ReviewID, scanErr) + } + + eventSink := reviewprocessor.NewDatabaseEventSink(w.db) + if emitErr := eventSink.EmitCompletionEvent(ctx, args.ReviewID, args.OrgID, + "### Static Analysis Tools Review Only\n\nAI review skipped due to --tools flag.", + commentCount, ""); emitErr != nil { + log.Printf("[ERROR] Failed to emit completion event for review %d: %v", args.ReviewID, emitErr) + } + + log.Printf("[INFO] ToolReviewWorker: review=%d ALL tools done, status=completed, comments=%d", args.ReviewID, commentCount) + } + return nil +} diff --git a/internal/license/plans.go b/internal/license/plans.go index 92def8d06..0359b261e 100644 --- a/internal/license/plans.go +++ b/internal/license/plans.go @@ -191,3 +191,11 @@ func (p PlanType) IsValid() bool { func (p PlanType) String() string { return string(p) } + +// IsToolsEligible returns true if the plan type is eligible for static analysis tool credit deduction. +func IsToolsEligible(p PlanType) bool { + if p == PlanType("") { + return true + } + return p.IsValid() +} diff --git a/internal/lrcconfig/lrcconfig.go b/internal/lrcconfig/lrcconfig.go index 4997fb6cf..2181d89a5 100644 --- a/internal/lrcconfig/lrcconfig.go +++ b/internal/lrcconfig/lrcconfig.go @@ -18,6 +18,7 @@ import ( "github.com/livereview/cmd/mrmodel/lib" "github.com/livereview/pkg/models" + toml "github.com/pelletier/go-toml/v2" gitignore "github.com/sabhiram/go-gitignore" ) @@ -32,6 +33,8 @@ const rulesPrefix = "rules/" const rulesReadmePath = rulesPrefix + "README.md" const rulesInstructionsPath = rulesPrefix + "INSTRUCTIONS.md" const ignorePath = "ignore" +const policyToolsTomlPath = "policy/tools.toml" +const toolsTomlPath = "tools.toml" // Issue describes a problem found while processing a Bundle. type Issue struct { @@ -206,3 +209,225 @@ func TruncateAtLineBoundary(text string, limit int) string { } return text[:cut] } + +// ToolRuleConfig represents per-tool configuration within policy/tools.toml or tools.toml +type ToolRuleConfig struct { + Enabled *bool `toml:"enabled"` + Category string `toml:"category"` + Include []string `toml:"include"` + Exclude []string `toml:"exclude"` +} + +// ParseToolRuleConfigs reads .lrc/policy/tools.toml or .lrc/tools.toml from a Bundle. +// Returns a map of tool_name -> *ToolRuleConfig. +func ParseToolRuleConfigs(b Bundle) (map[string]*ToolRuleConfig, error) { + data, ok := b.Files[policyToolsTomlPath] + if !ok || len(data) == 0 { + data, ok = b.Files[toolsTomlPath] + } + if !ok || len(data) == 0 { + return nil, nil + } + + var raw map[string]interface{} + if err := toml.Unmarshal(data, &raw); err != nil { + return nil, fmt.Errorf("failed to parse tools.toml: %w", err) + } + + results := make(map[string]*ToolRuleConfig) + + processEntry := func(toolName string, val interface{}) { + toolName = strings.ToLower(toolName) + switch v := val.(type) { + case bool: + bVal := v + results[toolName] = &ToolRuleConfig{Enabled: &bVal} + case map[string]interface{}: + cfg := &ToolRuleConfig{} + if enabledRaw, exists := v["enabled"]; exists && enabledRaw != nil { + if enabledVal, ok := enabledRaw.(bool); ok { + cfg.Enabled = &enabledVal + } + } + if catRaw, exists := v["category"]; exists && catRaw != nil { + if catVal, ok := catRaw.(string); ok { + cfg.Category = catVal + } + } + if incRaw, exists := v["include"]; exists && incRaw != nil { + if incSlice, ok := incRaw.([]interface{}); ok { + for _, item := range incSlice { + if str, isStr := item.(string); isStr { + cfg.Include = append(cfg.Include, str) + } + } + } + } + if excRaw, exists := v["exclude"]; exists && excRaw != nil { + if excSlice, ok := excRaw.([]interface{}); ok { + for _, item := range excSlice { + if str, isStr := item.(string); isStr { + cfg.Exclude = append(cfg.Exclude, str) + } + } + } + } + results[toolName] = cfg + } + } + + for k, v := range raw { + if k == "tools" { + if toolsMap, ok := v.(map[string]interface{}); ok { + for tName, tVal := range toolsMap { + if _, exists := results[strings.ToLower(tName)]; !exists { + processEntry(tName, tVal) + } + } + } + } else { + processEntry(k, v) + } + } + + return results, nil +} + +// ShouldRunToolRuleForDiff determines whether a tool should run against the given local diffs based on ToolRuleConfig. +func ShouldRunToolRuleForDiff(cfg *ToolRuleConfig, diffs []lib.LocalCodeDiff) bool { + if cfg == nil { + return true + } + + if cfg.Enabled != nil && !*cfg.Enabled { + return false + } + + if len(diffs) == 0 { + return true + } + + var includeMatcher *gitignore.GitIgnore + if len(cfg.Include) > 0 { + includeMatcher = gitignore.CompileIgnoreLines(cfg.Include...) + } + + var excludeMatcher *gitignore.GitIgnore + if len(cfg.Exclude) > 0 { + excludeMatcher = gitignore.CompileIgnoreLines(cfg.Exclude...) + } + + if includeMatcher == nil && excludeMatcher == nil { + return true + } + + matchingFilesCount := 0 + for _, d := range diffs { + path := d.NewPath + if path == "" { + path = d.OldPath + } + if path == "" { + continue + } + + if excludeMatcher != nil && excludeMatcher.MatchesPath(path) { + continue + } + + if includeMatcher != nil { + if includeMatcher.MatchesPath(path) { + matchingFilesCount++ + } + } else { + matchingFilesCount++ + } + } + + return matchingFilesCount > 0 +} + +// FilterLocalCodeDiffsForTool filters local code diffs according to a tool's ToolRuleConfig. +// It returns a new slice containing only the diffs for file paths that match inclusion and pass exclusion rules. +func FilterLocalCodeDiffsForTool(cfg *ToolRuleConfig, diffs []lib.LocalCodeDiff) []lib.LocalCodeDiff { + if cfg == nil || len(diffs) == 0 { + return diffs + } + + var includeMatcher *gitignore.GitIgnore + if len(cfg.Include) > 0 { + includeMatcher = gitignore.CompileIgnoreLines(cfg.Include...) + } + + var excludeMatcher *gitignore.GitIgnore + if len(cfg.Exclude) > 0 { + excludeMatcher = gitignore.CompileIgnoreLines(cfg.Exclude...) + } + + if includeMatcher == nil && excludeMatcher == nil { + return diffs + } + + var filtered []lib.LocalCodeDiff + for _, d := range diffs { + path := d.NewPath + if path == "" { + path = d.OldPath + } + if path == "" { + continue + } + + if excludeMatcher != nil && excludeMatcher.MatchesPath(path) { + continue + } + + if includeMatcher != nil { + if includeMatcher.MatchesPath(path) { + filtered = append(filtered, d) + } + } else { + filtered = append(filtered, d) + } + } + + return filtered +} + +// FormatLocalDiffs converts a slice of lib.LocalCodeDiff into a standard unified diff string. +func FormatLocalDiffs(diffs []lib.LocalCodeDiff) string { + var b strings.Builder + for _, d := range diffs { + path := d.NewPath + if path == "" { + path = d.OldPath + } + if path == "" { + continue + } + + b.WriteString(fmt.Sprintf("diff --git a/%s b/%s\n", path, path)) + if d.OldPath == "/dev/null" || d.OldPath == "" { + b.WriteString("new file mode 100644\n") + } else if d.NewPath == "/dev/null" || d.NewPath == "" { + b.WriteString("deleted file mode 100644\n") + } + for _, hunk := range d.Hunks { + b.WriteString(fmt.Sprintf("@@ -%d,%d +%d,%d @@", hunk.OldStartLine, hunk.OldLineCount, hunk.NewStartLine, hunk.NewLineCount)) + if hunk.HeaderText != "" { + b.WriteString(" " + hunk.HeaderText) + } + b.WriteString("\n") + for _, line := range hunk.Lines { + prefix := " " + if line.LineType == "added" { + prefix = "+" + } else if line.LineType == "deleted" { + prefix = "-" + } + b.WriteString(prefix + line.Content + "\n") + } + } + } + return b.String() +} diff --git a/internal/prompts/builder.go b/internal/prompts/builder.go index 3d78dd52d..96a9c39d8 100644 --- a/internal/prompts/builder.go +++ b/internal/prompts/builder.go @@ -2,6 +2,7 @@ package prompts import ( "context" + "encoding/json" "fmt" "strings" @@ -37,6 +38,52 @@ func (pb *PromptBuilder) BuildSummaryPrompt(entries []TechnicalSummary) string { return base + "\n\n" + BuildSummarySection(entries) + "\n\n" + SummaryStructure } +// ToolFindingInput represents raw linter finding details passed into the classifier +type ToolFindingInput struct { + ToolName string `json:"tool_name"` + RuleID string `json:"rule_id"` + FilePath string `json:"file_path"` + LineNumber int `json:"line_number"` + Message string `json:"message"` + CodeSnippet string `json:"code_snippet"` +} + +// BuildToolFindingClassificationPrompt composes a minimal classification prompt +// reusing the authoritative TaxonomyClassificationRules, CommentClassification, +// and CommentRequirements constants. +func (pb *PromptBuilder) BuildToolFindingClassificationPrompt(finding ToolFindingInput) string { + var sb strings.Builder + sb.WriteString("You are an expert code analysis classifier. Classify the following static analysis tool finding into the LiveReview Taxonomy.\n\n") + sb.WriteString(TaxonomyClassificationRules) + sb.WriteString("\n\n") + sb.WriteString(CommentClassification) + sb.WriteString("\n\n") + sb.WriteString(CommentRequirements) + sb.WriteString("\n\nRAW TOOL FINDING (JSON):\n```json\n") + findingBytes, err := json.MarshalIndent(finding, "", " ") + if err != nil { + fallbackObj := map[string]interface{}{ + "tool_name": finding.ToolName, + "rule_id": finding.RuleID, + "file_path": finding.FilePath, + "line_number": finding.LineNumber, + "message": finding.Message, + } + fallbackBytes, _ := json.Marshal(fallbackObj) + sb.Write(fallbackBytes) + } else { + sb.Write(findingBytes) + } + sb.WriteString("\n```\n") + sb.WriteString("\nFormat your response strictly as JSON with keys: category, subcategory, severity, type, confidence, isInternal.\n") + sb.WriteString("- 'category' MUST be one of the 10 top-level categories in the taxonomy (e.g., Security, Reliability, Correctness).\n") + sb.WriteString("- 'subcategory' MUST be one of the valid subcategories under that category (e.g., Secrets Management).\n") + sb.WriteString("- 'severity' MUST be one of: critical, warning, info.\n") + sb.WriteString("- 'type' MUST be one of: Bug, Risk, Optimization, Code Smell, Best Practice, Technical Debt.\n") + sb.WriteString("- 'confidence' MUST be one of: High, Medium, Low.\n") + return sb.String() +} + // addCodeDiffs adds the actual code changes to the prompt func (pb *PromptBuilder) addCodeDiffs(prompt *strings.Builder, diffs []*models.CodeDiff) { for _, diff := range diffs { @@ -58,3 +105,4 @@ func (pb *PromptBuilder) addCodeDiffs(prompt *strings.Builder, diffs []*models.C } } } + diff --git a/internal/review/service.go b/internal/review/service.go index 030a5d750..e9d755926 100644 --- a/internal/review/service.go +++ b/internal/review/service.go @@ -846,3 +846,27 @@ func extractRepoFullName(rawURL string) string { } return path } + +// FormatDiffs converts a slice of CodeDiff objects into a unified diff string. +func FormatDiffs(diffs []*models.CodeDiff) string { + var b strings.Builder + for _, d := range diffs { + if d == nil { + continue + } + b.WriteString(fmt.Sprintf("diff --git a/%s b/%s\n", d.FilePath, d.FilePath)) + if d.IsNew { + b.WriteString("new file mode 100644\n") + } else if d.IsDeleted { + b.WriteString("deleted file mode 100644\n") + } + for _, hunk := range d.Hunks { + b.WriteString(fmt.Sprintf("@@ -%d,%d +%d,%d @@\n", hunk.OldStartLine, hunk.OldLineCount, hunk.NewStartLine, hunk.NewLineCount)) + b.WriteString(hunk.Content) + if !strings.HasSuffix(hunk.Content, "\n") { + b.WriteString("\n") + } + } + } + return b.String() +} diff --git a/internal/review_processor/events.go b/internal/review_processor/events.go index cf89e18c2..b3bb102cd 100644 --- a/internal/review_processor/events.go +++ b/internal/review_processor/events.go @@ -5,6 +5,7 @@ import ( "database/sql" "encoding/json" "fmt" + "log" "strconv" "strings" "time" @@ -577,37 +578,67 @@ type ToolSummary struct { ToolBreakdown []ToolBreakdownItem `json:"toolBreakdown"` } -// GetSeverityCounts queries review_events to count High, Medium, and Low severity events +// GetSeverityCounts queries public.ai_comments (and review_events fallback) to count High, Medium, and Low severity issues func (r *ReviewEventsRepo) GetSeverityCounts(ctx context.Context, reviewID, orgID int64) (SeverityCounts, error) { + var counts SeverityCounts query := ` - SELECT COALESCE(level, 'info') as lvl, COUNT(*) - FROM public.review_events + SELECT COALESCE(content->>'severity', 'warning') as sev, COUNT(*) + FROM public.ai_comments WHERE review_id = $1 AND org_id = $2 - GROUP BY COALESCE(level, 'info') + GROUP BY COALESCE(content->>'severity', 'warning') ` rows, err := r.db.QueryContext(ctx, query, reviewID, orgID) - if err != nil { - return SeverityCounts{}, fmt.Errorf("failed to query severity counts: %w", err) + if err == nil { + defer rows.Close() + hasComments := false + for rows.Next() { + hasComments = true + var sev string + var cnt int + if err := rows.Scan(&sev, &cnt); err == nil { + switch strings.ToLower(strings.TrimSpace(sev)) { + case "critical", "high", "error": + counts.High += cnt + case "warning", "medium", "warn": + counts.Medium += cnt + case "info", "low": + counts.Low += cnt + } + } + } + if hasComments { + return counts, nil + } } - defer rows.Close() - var counts SeverityCounts - for rows.Next() { - var lvl string + queryEvents := ` + SELECT COALESCE(data->>'severity', level) as sev, COUNT(*) + FROM public.review_events + WHERE review_id = $1 AND org_id = $2 AND (data->>'severity' IS NOT NULL OR level IN ('error', 'warn')) + GROUP BY COALESCE(data->>'severity', level) + ` + eventRows, eventErr := r.db.QueryContext(ctx, queryEvents, reviewID, orgID) + if eventErr != nil { + log.Printf("[WARN] Failed to query review_events for severity counts (review_id=%d, org_id=%d): %v", reviewID, orgID, eventErr) + return counts, fmt.Errorf("failed to query review_events for severity counts: %w", eventErr) + } + defer eventRows.Close() + + for eventRows.Next() { + var sev string var cnt int - if err := rows.Scan(&lvl, &cnt); err != nil { - return SeverityCounts{}, err - } - switch strings.ToLower(lvl) { - case "error", "high", "critical": - counts.High += cnt - case "warn", "warning", "medium": - counts.Medium += cnt - case "info", "low": - counts.Low += cnt + if err := eventRows.Scan(&sev, &cnt); err == nil { + switch strings.ToLower(strings.TrimSpace(sev)) { + case "critical", "high", "error": + counts.High += cnt + case "warning", "medium", "warn": + counts.Medium += cnt + case "info", "low": + counts.Low += cnt + } } } - return counts, rows.Err() + return counts, nil } type rawToolEvent struct { diff --git a/internal/toolclassifier/tool_classifier.go b/internal/toolclassifier/tool_classifier.go new file mode 100644 index 000000000..cdf54f89f --- /dev/null +++ b/internal/toolclassifier/tool_classifier.go @@ -0,0 +1,416 @@ +package toolclassifier + +import ( + "context" + "database/sql" + "encoding/json" + "fmt" + "log" + "strings" + + "github.com/livereview/internal/aiconnectors" + "github.com/livereview/internal/aiselection" + "github.com/livereview/internal/license" + "github.com/livereview/internal/logging" + "github.com/livereview/internal/review" + "github.com/livereview/pkg/models" +) + +// TaxonomyTuple represents a deterministic classification tuple +type TaxonomyTuple struct { + Category string + Subcategory string + Severity string + Confidence string + Type string +} + +// KnownDeterministicRules maps (tool_name + ":" + rule_id) -> taxonomy classification for 0 LLM token cost. +var KnownDeterministicRules = map[string]TaxonomyTuple{ + // Secrets Detection + "gitleaks:generic-api-key": {"Security", "Secrets Management", "critical", "High", "Risk"}, + "gitleaks:aws-access-key": {"Security", "Secrets Management", "critical", "High", "Risk"}, + "gitleaks:aws-access-token": {"Security", "Secrets Management", "critical", "High", "Risk"}, + "gitleaks:private-key": {"Security", "Secrets Management", "critical", "High", "Risk"}, + "gitleaks:github-pat": {"Security", "Secrets Management", "critical", "High", "Risk"}, + "gitleaks:slack-webhook": {"Security", "Secrets Management", "critical", "High", "Risk"}, + "detect-secrets:secret-key": {"Security", "Secrets Management", "critical", "High", "Risk"}, + "trufflehog:credential": {"Security", "Secrets Management", "critical", "High", "Risk"}, + + // Security & SAST + "semgrep:sql-injection": {"Security", "Injection Vulnerabilities", "critical", "High", "Bug"}, + "bandit:b303": {"Security", "Cryptography", "warning", "High", "Risk"}, + "bandit:b101": {"Security", "Input Validation", "warning", "High", "Risk"}, + "brakeman:command_injection": {"Security", "Injection Vulnerabilities", "critical", "High", "Bug"}, + + // Linters & Static Analysis + "hadolint:dl3006": {"Maintainability", "Configuration Management", "warning", "Medium", "Best Practice"}, + "shellcheck:sc2086": {"Reliability", "Fault Tolerance", "info", "Medium", "Code Smell"}, + "ruff:e501": {"Maintainability", "Code Complexity", "info", "High", "Code Smell"}, + "ruff:f401": {"Maintainability", "Dead Code", "warning", "High", "Code Smell"}, + "eslint:no-unused-vars": {"Maintainability", "Dead Code", "warning", "High", "Code Smell"}, +} + +// RawToolFinding represents an individual finding inside a Lambda response +type RawToolFinding struct { + File string `json:"file"` + Line int `json:"line"` + Col int `json:"col,omitempty"` + Rule string `json:"rule"` + Message string `json:"message"` +} + +// ClassifyToolResult classifies raw findings using deterministic static rules first, and Helper LLM batch call for unresolved findings. +func ClassifyToolResult( + ctx context.Context, + db *sql.DB, + orgID int64, + planCode license.PlanType, + toolName string, + findings []RawToolFinding, + logger *logging.ReviewLogger, +) ([]*models.ReviewComment, error) { + if len(findings) == 0 { + return nil, nil + } + + comments := make([]*models.ReviewComment, len(findings)) + var unresolvedIndices []int + + // Step 1: Check deterministic static rule mappings (0 LLM Tokens) + for idx, finding := range findings { + ruleKey := strings.ToLower(toolName + ":" + finding.Rule) + rawRuleKey := strings.ToLower(finding.Rule) + if tuple, ok := KnownDeterministicRules[ruleKey]; ok { + cat, subcat := ValidateAndNormalizeTaxonomy(tuple.Category, tuple.Subcategory) + comments[idx] = &models.ReviewComment{ + FilePath: finding.File, + Line: finding.Line, + Content: finding.Message, + Severity: parseSeverity(tuple.Severity), + Category: cat, + Subcategory: subcat, + Confidence: NormalizeConfidence(tuple.Confidence), + Type: NormalizeType(tuple.Type), + } + if logger != nil { + logger.Log("[TOOL %s] Deterministic classification (0 tokens) for rule %s: %s / %s", toolName, finding.Rule, cat, subcat) + } + } else if tuple, ok := KnownDeterministicRules[rawRuleKey]; ok { + cat, subcat := ValidateAndNormalizeTaxonomy(tuple.Category, tuple.Subcategory) + comments[idx] = &models.ReviewComment{ + FilePath: finding.File, + Line: finding.Line, + Content: finding.Message, + Severity: parseSeverity(tuple.Severity), + Category: cat, + Subcategory: subcat, + Confidence: NormalizeConfidence(tuple.Confidence), + Type: NormalizeType(tuple.Type), + } + if logger != nil { + logger.Log("[TOOL %s] Deterministic classification (0 tokens) for rule %s: %s / %s", toolName, finding.Rule, cat, subcat) + } + } else { + unresolvedIndices = append(unresolvedIndices, idx) + } + } + + if len(unresolvedIndices) == 0 { + return comments, nil + } + + // Step 2: Resolve Helper AI model configuration + selection, err := aiselection.GetReviewAISelection(ctx, db, orgID, planCode) + var helperConfig review.AIConfig + if err == nil && selection.HelperEnabled && selection.Helper != nil { + helperConfig = *selection.Helper + } else if err == nil { + helperConfig = selection.Leader + } else { + return applyFallbackComments(findings, comments, unresolvedIndices), nil + } + + // Step 3: Construct compact pipe-delimited batch prompt (~120 tokens) + type CompactFindingInput struct { + Index int `json:"i"` + Rule string `json:"r"` + Message string `json:"m"` + } + + compactItems := make([]CompactFindingInput, 0, len(unresolvedIndices)) + for _, origIdx := range unresolvedIndices { + compactItems = append(compactItems, CompactFindingInput{ + Index: origIdx, + Rule: findings[origIdx].Rule, + Message: findings[origIdx].Message, + }) + } + + payloadJSON, _ := json.Marshal(compactItems) + + prompt := fmt.Sprintf(`Classify static analysis tool findings into LiveReview taxonomy. +Categories: Security, Performance, Maintainability, Reliability, Compliance, Architecture. +Subcategories: Secrets Management, Injection Vulnerabilities, Cryptography, Input Validation, Dead Code, Configuration Management, Code Complexity, Fault Tolerance, Data Exposure. + +Format response strictly as JSON array of pipe-delimited strings: +["index|category|subcategory|severity_code|confidence_code|type_code"] +Severity: c=critical, w=warning, i=info +Confidence: H=High, M=Medium, L=Low +Type: B=Bug, R=Risk, O=Optimization, S=Smell, P=Practice, D=Debt + +Findings: +%s`, string(payloadJSON)) + + options, _, err := connectorOptionsFromAIConfig(helperConfig, len(compactItems)) + if err == nil { + connector, connErr := aiconnectors.NewConnector(ctx, options) + if connErr == nil { + respStr, inputTokens, outputTokens, callErr := connector.CallWithUsage(ctx, prompt) + if callErr == nil { + fmt.Printf("[INFO] [TOOL %s] Taxonomy LLM classification token usage: %d input tokens, %d output tokens (%d total LLM tokens)\n", toolName, inputTokens, outputTokens, inputTokens+outputTokens) + if logger != nil { + logger.Log("[TOOL %s] Taxonomy LLM classification token usage: %d input tokens, %d output tokens (%d total LLM tokens)", toolName, inputTokens, outputTokens, inputTokens+outputTokens) + } + + parsedLines := parsePipeDelimitedResponse(respStr) + for _, line := range parsedLines { + parts := strings.Split(line, "|") + if len(parts) >= 6 { + var origIdx int + if _, scanErr := fmt.Sscanf(parts[0], "%d", &origIdx); scanErr == nil && origIdx >= 0 && origIdx < len(findings) { + cat, subcat := ValidateAndNormalizeTaxonomy(parts[1], parts[2]) + comments[origIdx] = &models.ReviewComment{ + FilePath: findings[origIdx].File, + Line: findings[origIdx].Line, + Content: findings[origIdx].Message, + Severity: parseSeverityCode(parts[3]), + Category: cat, + Subcategory: subcat, + Confidence: parseConfidenceCode(parts[4]), + Type: parseTypeCode(parts[5]), + } + } + } + } + } + } + } + + return applyFallbackComments(findings, comments, unresolvedIndices), nil +} + +func parsePipeDelimitedResponse(raw string) []string { + cleaned := strings.TrimSpace(raw) + cleaned = strings.TrimPrefix(cleaned, "```json") + cleaned = strings.TrimPrefix(cleaned, "```") + cleaned = strings.TrimSuffix(cleaned, "```") + cleaned = strings.TrimSpace(cleaned) + + var lines []string + if err := json.Unmarshal([]byte(cleaned), &lines); err != nil { + log.Printf("[WARN] parsePipeDelimitedResponse: failed to unmarshal pipe-delimited response: %v", err) + } + return lines +} + +func applyFallbackComments(findings []RawToolFinding, comments []*models.ReviewComment, unresolved []int) []*models.ReviewComment { + for _, origIdx := range unresolved { + if comments[origIdx] == nil { + comments[origIdx] = &models.ReviewComment{ + FilePath: findings[origIdx].File, + Line: findings[origIdx].Line, + Content: findings[origIdx].Message, + Severity: models.SeverityWarning, + Category: "Security", + Subcategory: "Secrets Management", + Confidence: "High", + Type: "Risk", + } + } + } + return comments +} + +func connectorOptionsFromAIConfig(config review.AIConfig, commentCount int) (aiconnectors.ConnectorOptions, string, error) { + providerName, _ := config.Config["provider_name"].(string) + providerType, ok := config.Config["ai_provider_type"].(string) + if !ok || strings.TrimSpace(providerType) == "" { + providerType = providerName + } + if strings.TrimSpace(providerType) == "" { + return aiconnectors.ConnectorOptions{}, "", fmt.Errorf("AI provider type is missing") + } + + baseURL, _ := config.Config["base_url"].(string) + projectID, _ := config.Config["gcp_project_id"].(string) + location, _ := config.Config["gcp_location"].(string) + awsAccessKeyID, _ := config.Config["aws_access_key_id"].(string) + awsRegion, _ := config.Config["aws_region"].(string) + + return aiconnectors.ConnectorOptions{ + Provider: aiconnectors.Provider(providerType), + APIKey: config.APIKey, + BaseURL: aiconnectors.ResolveBaseURLForProviderName(providerType, baseURL), + GCPProjectID: projectID, + GCPLocation: location, + AWSAccessKeyID: awsAccessKeyID, + AWSRegion: awsRegion, + ModelConfig: aiconnectors.ModelConfig{ + Temperature: config.Temperature, + MaxTokens: 2048, + Model: config.Model, + }, + }, strings.TrimSpace(providerName), nil +} + +func parseSeverity(raw string) models.CommentSeverity { + switch strings.ToLower(strings.TrimSpace(raw)) { + case "critical": + return models.SeverityCritical + case "info": + return models.SeverityInfo + default: + return models.SeverityWarning + } +} + +func parseSeverityCode(code string) models.CommentSeverity { + switch strings.ToLower(strings.TrimSpace(code)) { + case "c": + return models.SeverityCritical + case "i": + return models.SeverityInfo + default: + return models.SeverityWarning + } +} + +func parseConfidenceCode(code string) string { + switch strings.ToUpper(strings.TrimSpace(code)) { + case "H": + return "High" + case "L": + return "Low" + default: + return "Medium" + } +} + +func parseTypeCode(code string) string { + switch strings.ToUpper(strings.TrimSpace(code)) { + case "B": + return "Bug" + case "R": + return "Risk" + case "O": + return "Optimization" + case "S": + return "Code Smell" + case "P": + return "Best Practice" + case "D": + return "Technical Debt" + default: + return "Risk" + } +} + +var ValidTaxonomyMap = map[string][]string{ + "Security": {"Authentication", "Authorization", "Secrets Management", "Input Validation", "Injection Vulnerabilities", "Cryptography", "Dependency Vulnerabilities", "Data Exposure", "Session Management", "Security Logging & Auditing"}, + "Reliability": {"Error Handling", "Fault Tolerance", "Retry Logic", "Timeout Management", "Resilience Patterns", "Availability Risks", "Data Integrity", "Race Conditions", "Resource Cleanup", "Failure Recovery"}, + "Correctness": {"Logic Errors", "Edge Cases", "Data Validation", "State Management", "Concurrency Bugs", "Business Rule Violations", "Numerical Accuracy", "Null Handling", "Type Safety", "API Contract Violations"}, + "Performance": {"Database Efficiency", "Algorithmic Complexity", "Memory Usage", "CPU Utilization", "Network Efficiency", "Caching", "Concurrency", "Resource Contention", "Rendering Performance", "Startup Performance"}, + "Cost": {"Cloud Resource Waste", "Infrastructure Overprovisioning", "Storage Optimization", "Database Cost Optimization", "Excessive API Usage", "Third-Party Service Costs", "Redundant Computation", "LLM Token Consumption", "Caching Opportunities", "Data Transfer Costs"}, + "Scalability": {"Horizontal Scaling", "Vertical Scaling", "Distributed Systems", "Load Balancing", "Capacity Planning", "Bottleneck Risks", "Concurrency Limits", "Service Growth Constraints", "Database Scaling", "Queue Backpressure"}, + "Maintainability": {"Code Complexity", "Readability", "Documentation", "Code Duplication", "Dead Code", "Naming Quality", "Testability", "Technical Debt", "Refactoring Opportunities", "Configuration Management", "UI/UX", "Accessibility"}, + "Architecture": {"Separation of Concerns", "Modularity", "Coupling", "Cohesion", "Layering Violations", "Dependency Management", "Service Boundaries", "Domain Modeling", "API Design", "Extensibility"}, + "Developer Experience": {"Testing", "CI/CD", "Build System", "Local Development", "Debuggability", "Observability", "Deployment Process", "Automation", "Developer Tooling", "Documentation Quality", "UI/UX", "Accessibility"}, + "Compliance & Governance": {"Privacy", "Regulatory Compliance", "Auditability", "Data Retention", "Data Residency", "Licensing", "Policy Enforcement", "Access Controls", "Change Management", "Governance Standards"}, +} + +var ValidTypes = map[string]string{ + "bug": "Bug", + "risk": "Risk", + "optimization": "Optimization", + "code smell": "Code Smell", + "best practice": "Best Practice", + "technical debt": "Technical Debt", +} + +var ValidConfidences = map[string]string{ + "high": "High", + "medium": "Medium", + "low": "Low", +} + +func ValidateAndNormalizeTaxonomy(rawCategory, rawSubcategory string) (string, string) { + trimmedCat := strings.TrimSpace(rawCategory) + trimmedSub := strings.TrimSpace(rawSubcategory) + + var matchedCategory string + var allowedSubcategories []string + + for cat, subcats := range ValidTaxonomyMap { + if strings.EqualFold(trimmedCat, cat) { + matchedCategory = cat + allowedSubcategories = subcats + break + } + } + + if matchedCategory == "" && trimmedSub != "" { + for cat, subcats := range ValidTaxonomyMap { + for _, sub := range subcats { + if strings.EqualFold(trimmedSub, sub) { + matchedCategory = cat + allowedSubcategories = subcats + trimmedSub = sub + break + } + } + if matchedCategory != "" { + break + } + } + } + + if matchedCategory == "" { + matchedCategory = "Security" + allowedSubcategories = ValidTaxonomyMap["Security"] + } + + var matchedSubcategory string + if trimmedSub != "" { + for _, sub := range allowedSubcategories { + if strings.EqualFold(trimmedSub, sub) { + matchedSubcategory = sub + break + } + } + } + + if matchedSubcategory == "" { + if matchedCategory == "Security" { + matchedSubcategory = "Secrets Management" + } else if len(allowedSubcategories) > 0 { + matchedSubcategory = allowedSubcategories[0] + } + } + + return matchedCategory, matchedSubcategory +} + +func NormalizeType(raw string) string { + if val, ok := ValidTypes[strings.ToLower(strings.TrimSpace(raw))]; ok { + return val + } + return "Risk" +} + +func NormalizeConfidence(raw string) string { + if val, ok := ValidConfidences[strings.ToLower(strings.TrimSpace(raw))]; ok { + return val + } + return "High" +} diff --git a/network/tools/lambda_client.go b/network/tools/lambda_client.go new file mode 100644 index 000000000..721a1b8ab --- /dev/null +++ b/network/tools/lambda_client.go @@ -0,0 +1,68 @@ +package tools + +import ( + "bytes" + "context" + "crypto/sha256" + "fmt" + "io" + "net/http" + "strings" + "time" + + "github.com/aws/aws-sdk-go-v2/aws" + v4 "github.com/aws/aws-sdk-go-v2/aws/signer/v4" +) + +// InvokeTool calls a Lambda function by its ARN using AWS SigV4 signed HTTP request, passing a payload, and returns the response body. +func InvokeTool(ctx context.Context, awsCfg aws.Config, arn string, payload []byte) ([]byte, error) { + // Parse region from ARN (arn:aws:lambda:us-east-1:12345:function:name) + parts := strings.Split(arn, ":") + if len(parts) < 7 { + return nil, fmt.Errorf("invalid lambda ARN: %s", arn) + } + region := parts[3] + functionName := parts[6] + + creds, err := awsCfg.Credentials.Retrieve(ctx) + if err != nil { + return nil, fmt.Errorf("failed to retrieve aws credentials: %w", err) + } + + urlStr := fmt.Sprintf("https://lambda.%s.amazonaws.com/2015-03-31/functions/%s/invocations", region, functionName) + req, err := http.NewRequestWithContext(ctx, "POST", urlStr, bytes.NewReader(payload)) + if err != nil { + return nil, fmt.Errorf("failed to create http request: %w", err) + } + + payloadHash := sha256.Sum256(payload) + payloadHashHex := fmt.Sprintf("%x", payloadHash) + + signer := v4.NewSigner() + err = signer.SignHTTP(ctx, creds, req, payloadHashHex, "lambda", region, time.Now()) + if err != nil { + return nil, fmt.Errorf("failed to sign request: %w", err) + } + + client := &http.Client{Timeout: 300 * time.Second} + resp, err := client.Do(req) + if err != nil { + return nil, fmt.Errorf("http request failed: %w", err) + } + defer resp.Body.Close() + + outBytes, err := io.ReadAll(resp.Body) + if err != nil { + return nil, fmt.Errorf("failed to read response body: %w", err) + } + + if resp.StatusCode >= 400 { + return nil, fmt.Errorf("http %d: %s", resp.StatusCode, string(outBytes)) + } + + if resp.Header.Get("X-Amz-Function-Error") != "" { + return nil, fmt.Errorf("lambda returned function error: %s", string(outBytes)) + } + + return outBytes, nil +} diff --git a/pkg/models/models.go b/pkg/models/models.go index bb2366a07..1ca558563 100644 --- a/pkg/models/models.go +++ b/pkg/models/models.go @@ -206,8 +206,9 @@ type ReviewComment struct { Category string Subcategory string Suggestions []string - IsDeletedLine bool // True if comment is on a deleted line (old_line) rather than new_line - IsInternal bool // True if comment is for internal synthesis only, false if it should be posted to user + IsDeletedLine bool // True if comment is on a deleted line (old_line) rather than new_line + IsInternal bool // True if comment is for internal synthesis only, false if it should be posted to user + Source string // Optional: source of comment, e.g. "tool" for static analysis findings } // CommentSeverity represents the severity level of a review comment diff --git a/storage/tools/credit_store.go b/storage/tools/credit_store.go new file mode 100644 index 000000000..a21f508c5 --- /dev/null +++ b/storage/tools/credit_store.go @@ -0,0 +1,185 @@ +package tools + +import ( + "context" + "database/sql" + "fmt" + "time" + + "github.com/livereview/internal/license" +) + +type CreditUsage struct { + CreditsUsedMonth float64 `json:"credits_used_month"` + CreditsLimitMonth float64 `json:"credits_limit_month"` + ReviewsRemaining int `json:"reviews_remaining"` + Blocked bool `json:"blocked"` +} + +type CreditStore struct { + db *sql.DB +} + +func NewCreditStore(db *sql.DB) *CreditStore { + return &CreditStore{db: db} +} + +func (s *CreditStore) ensureAndLockBillingState(ctx context.Context, tx *sql.Tx, orgID int64) (float64, float64, error) { + now := time.Now().UTC() + startOfMonth := time.Date(now.Year(), now.Month(), 1, 0, 0, 0, 0, time.UTC) + endOfMonth := startOfMonth.AddDate(0, 1, 0) + + _, err := tx.ExecContext(ctx, ` + INSERT INTO public.org_tool_billing_state ( + org_id, credits_used_month, credits_limit_month, billing_period_start, billing_period_end + ) VALUES ($1, 0.0, 50000.0, $2, $3) + ON CONFLICT (org_id) DO NOTHING + `, orgID, startOfMonth, endOfMonth) + if err != nil { + return 0, 0, fmt.Errorf("ensure tool billing state: %w", err) + } + + var currentUsed, currentLimit float64 + var periodStart, periodEnd time.Time + err = tx.QueryRowContext(ctx, ` + SELECT credits_used_month, credits_limit_month, billing_period_start, billing_period_end + FROM public.org_tool_billing_state + WHERE org_id = $1 + FOR UPDATE + `, orgID).Scan(¤tUsed, ¤tLimit, &periodStart, &periodEnd) + if err != nil { + return 0, 0, fmt.Errorf("lock tool billing state: %w", err) + } + + // Reset if billing period has ended + if !now.Before(periodEnd) { + currentUsed = 0.0 + _, err = tx.ExecContext(ctx, ` + UPDATE public.org_tool_billing_state + SET credits_used_month = 0.0, + billing_period_start = $1, + billing_period_end = $2, + updated_at = NOW() + WHERE org_id = $3 + `, startOfMonth, endOfMonth, orgID) + if err != nil { + return 0, 0, fmt.Errorf("reset tool billing state: %w", err) + } + } + + return currentUsed, currentLimit, nil +} + +// GetCreditUsage retrieves the current credit usage for an organization. +func (s *CreditStore) GetCreditUsage(ctx context.Context, orgID int64, currentMultiplier float64, planCode license.PlanType) (CreditUsage, error) { + if !license.IsToolsEligible(planCode) { + return CreditUsage{}, fmt.Errorf("tools credits are not available on the %s plan", planCode) + } + + tx, err := s.db.BeginTx(ctx, nil) + if err != nil { + return CreditUsage{}, err + } + defer func() { + _ = tx.Rollback() + }() + + currentUsed, currentLimit, err := s.ensureAndLockBillingState(ctx, tx, orgID) + if err != nil { + return CreditUsage{}, err + } + + if err := tx.Commit(); err != nil { + return CreditUsage{}, fmt.Errorf("failed to commit credit usage transaction: %w", err) + } + + remainingCredits := currentLimit - currentUsed + if remainingCredits < 0 { + remainingCredits = 0 + } + + reviewsRemaining := 0 + if currentMultiplier > 0 { + reviewsRemaining = int(remainingCredits / currentMultiplier) + } + + return CreditUsage{ + CreditsUsedMonth: currentUsed, + CreditsLimitMonth: currentLimit, + ReviewsRemaining: reviewsRemaining, + Blocked: currentMultiplier > 0 && remainingCredits < currentMultiplier, + }, nil +} + +// CheckCreditPreflight checks if the organization has enough credits for the required multiplier. +func (s *CreditStore) CheckCreditPreflight(ctx context.Context, orgID int64, requiredMultiplier float64, planCode license.PlanType) error { + usage, err := s.GetCreditUsage(ctx, orgID, requiredMultiplier, planCode) + if err != nil { + return err + } + if usage.Blocked { + return fmt.Errorf("insufficient tool credits: review requires %.2f credits, but only %.2f remaining this month", requiredMultiplier, usage.CreditsLimitMonth-usage.CreditsUsedMonth) + } + return nil +} + +// DeductCredits securely deducts the credits and writes to the ledger. +func (s *CreditStore) DeductCredits(ctx context.Context, orgID int64, reviewID int64, multiplier float64, planCode license.PlanType) error { + if !license.IsToolsEligible(planCode) { + return fmt.Errorf("tools credits are not available on the %s plan", planCode) + } + if multiplier <= 0 { + return nil + } + + tx, err := s.db.BeginTx(ctx, nil) + if err != nil { + return fmt.Errorf("begin deduct tx: %w", err) + } + defer func() { + _ = tx.Rollback() + }() + + currentUsed, currentLimit, err := s.ensureAndLockBillingState(ctx, tx, orgID) + if err != nil { + return err + } + + if currentLimit-currentUsed < multiplier { + return fmt.Errorf("insufficient tool credits during deduction") + } + + idempotencyKey := fmt.Sprintf("tool_review_%d", reviewID) + + var ledgerID int64 + err = tx.QueryRowContext(ctx, ` + INSERT INTO public.tool_credit_ledger ( + org_id, review_id, credits_deducted, idempotency_key, created_at + ) VALUES ($1, $2, $3, $4, NOW()) + ON CONFLICT (org_id, idempotency_key) DO NOTHING + RETURNING id + `, orgID, reviewID, multiplier, idempotencyKey).Scan(&ledgerID) + + if err != nil && err != sql.ErrNoRows { + return fmt.Errorf("insert ledger: %w", err) + } + + if ledgerID != 0 { + newUsed := currentUsed + multiplier + _, err = tx.ExecContext(ctx, ` + UPDATE public.org_tool_billing_state + SET credits_used_month = $1, + updated_at = NOW() + WHERE org_id = $2 + `, newUsed, orgID) + if err != nil { + return fmt.Errorf("update tool billing state: %w", err) + } + } + + if err := tx.Commit(); err != nil { + return fmt.Errorf("commit deduct tx: %w", err) + } + + return nil +} diff --git a/storage/tools/tools_store.go b/storage/tools/tools_store.go index 7387def4b..9fa4a1b77 100644 --- a/storage/tools/tools_store.go +++ b/storage/tools/tools_store.go @@ -5,6 +5,8 @@ import ( "database/sql" "encoding/json" "fmt" + "log" + "strings" ) type AvailableTool struct { @@ -163,41 +165,91 @@ func (s *ToolsStore) GetEnabledToolsForOrg(ctx context.Context, orgID int64) ([] return tools, nil } +// GetAvailableToolByName fetches a tool from the global available_tools catalog by name. +func (s *ToolsStore) GetAvailableToolByName(ctx context.Context, name string) (*AvailableTool, error) { + query := ` + SELECT id, name, description, lambda_arn, multiplier, use_case + FROM public.available_tools + WHERE LOWER(name) = LOWER($1) + LIMIT 1 + ` + var t AvailableTool + err := s.db.QueryRowContext(ctx, query, name).Scan( + &t.ID, + &t.Name, + &t.Description, + &t.LambdaARN, + &t.Multiplier, + &t.UseCase, + ) + if err == sql.ErrNoRows { + return nil, nil + } + if err != nil { + return nil, fmt.Errorf("failed to query available tool %q: %w", name, err) + } + return &t, nil +} + // InsertToolResultEvent wraps raw Lambda response and logs it in review_events table. func (s *ToolsStore) InsertToolResultEvent(ctx context.Context, reviewID, orgID, toolID int64, toolName string, resultJSON []byte) error { - type ToolLambdaResponse struct { - ExitCode int `json:"exit_code"` - Findings json.RawMessage `json:"findings"` - LinesOfCode int `json:"lines_of_code"` - Stderr string `json:"stderr"` - } + var parsedFindings []ToolFinding + trimmedJSON := strings.TrimSpace(string(resultJSON)) + var exitCode int + var stderr string + var loc int - var resp ToolLambdaResponse - // If unmarshaling fails or findings is nil, initialize it with empty array - if err := json.Unmarshal(resultJSON, &resp); err != nil { - resp.Stderr = fmt.Sprintf("failed to parse lambda response: %v. Raw: %s", err, string(resultJSON)) - resp.ExitCode = -1 - } - if len(resp.Findings) == 0 { - resp.Findings = json.RawMessage("[]") + if strings.HasPrefix(trimmedJSON, "[") { + if err := json.Unmarshal(resultJSON, &parsedFindings); err != nil { + log.Printf("[WARN] StoreToolResultEvent: failed to parse array findings for tool %d: %v", toolID, err) + parsedFindings = []ToolFinding{} + stderr = fmt.Sprintf("failed to parse array findings: %v", err) + exitCode = -1 + } else if len(parsedFindings) > 0 { + exitCode = 1 + } else { + exitCode = 0 + } + } else { + type ToolLambdaResponse struct { + ExitCode int `json:"exit_code"` + Findings json.RawMessage `json:"findings"` + LinesOfCode int `json:"lines_of_code"` + Stderr string `json:"stderr"` + } + + var resp ToolLambdaResponse + if err := json.Unmarshal(resultJSON, &resp); err != nil { + stderr = fmt.Sprintf("failed to parse lambda response: %v. Raw: %s", err, string(resultJSON)) + exitCode = -1 + } else { + exitCode = resp.ExitCode + stderr = resp.Stderr + loc = resp.LinesOfCode + if len(resp.Findings) > 0 { + if errFindings := json.Unmarshal(resp.Findings, &parsedFindings); errFindings != nil { + log.Printf("[WARN] StoreToolResultEvent: failed to unmarshal findings for tool %d: %v", toolID, errFindings) + } + } + } } - type ToolResultEventData struct { - ToolID int64 `json:"tool_id"` - ToolName string `json:"tool_name"` - ExitCode int `json:"exit_code"` - Findings json.RawMessage `json:"findings"` - LinesOfCode int `json:"lines_of_code"` - Stderr string `json:"stderr"` + // Redact plaintext secrets across all finding fields before persisting to database event log + for idx := range parsedFindings { + parsedFindings[idx].Secret = "[REDACTED]" + parsedFindings[idx].CodeSnippet = "[REDACTED]" + + parsedFindings[idx].Message = redactMatchDetails(parsedFindings[idx].Message) + parsedFindings[idx].Extra.Message = redactMatchDetails(parsedFindings[idx].Extra.Message) } eventData := ToolResultEventData{ ToolID: toolID, ToolName: toolName, - ExitCode: resp.ExitCode, - Findings: resp.Findings, - LinesOfCode: resp.LinesOfCode, - Stderr: resp.Stderr, + ExitCode: exitCode, + Findings: parsedFindings, + LinesOfCode: loc, + Stderr: stderr, } eventDataBytes, err := json.Marshal(eventData) @@ -215,3 +267,75 @@ func (s *ToolsStore) InsertToolResultEvent(ctx context.Context, reviewID, orgID, } return nil } + +func redactMatchDetails(msg string) string { + msg = strings.TrimSpace(msg) + for _, pattern := range []string{" (Match:", "(Match:", " Match:"} { + if idx := strings.Index(msg, pattern); idx != -1 { + msg = strings.TrimSpace(msg[:idx]) + } + } + // Redact plaintext token matches if embedded in message + if strings.Contains(msg, "AKIA") || strings.Contains(msg, "ghp_") || strings.Contains(msg, "sk_") { + msg = strings.TrimSpace(msg) + " [REDACTED]" + } + return msg +} + +type ToolFinding struct { + File string `json:"file"` + FilePath string `json:"file_path"` + Path string `json:"path"` + Line int `json:"line"` + LineNumber int `json:"line_number"` + Start struct { + Line int `json:"line"` + Col int `json:"col"` + } `json:"start"` + Col int `json:"col"` + Rule string `json:"rule"` + RuleID string `json:"rule_id"` + CheckID string `json:"check_id"` + Message string `json:"message"` + Extra struct { + Message string `json:"message"` + Severity string `json:"severity"` + } `json:"extra"` + Secret string `json:"secret,omitempty"` + CodeSnippet string `json:"code_snippet,omitempty"` +} + +type ToolResultEventData struct { + ToolID int64 `json:"tool_id"` + ToolName string `json:"tool_name"` + ExitCode int `json:"exit_code"` + Findings []ToolFinding `json:"findings"` + LinesOfCode int `json:"lines_of_code"` + Stderr string `json:"stderr"` +} + +func (s *ToolsStore) GetToolResultsForReview(ctx context.Context, reviewID int64) ([]ToolResultEventData, error) { + rows, err := s.db.QueryContext(ctx, ` + SELECT data + FROM public.review_events + WHERE review_id = $1 AND event_type = 'tool_result' + `, reviewID) + if err != nil { + return nil, err + } + defer rows.Close() + + var results []ToolResultEventData + for rows.Next() { + var rawData []byte + if err := rows.Scan(&rawData); err != nil { + return nil, err + } + var data ToolResultEventData + if err := json.Unmarshal(rawData, &data); err != nil { + return nil, err + } + results = append(results, data) + } + return results, nil +} diff --git a/ui/src/pages/Reviews/ReviewDetail.tsx b/ui/src/pages/Reviews/ReviewDetail.tsx index 655c33273..a75e22b68 100644 --- a/ui/src/pages/Reviews/ReviewDetail.tsx +++ b/ui/src/pages/Reviews/ReviewDetail.tsx @@ -338,9 +338,14 @@ const ReviewDetail: React.FC = () => { const eventSeverityCounts = useMemo(() => { let high = 0, medium = 0, low = 0; events.forEach(e => { - if (e.level === 'error') high++; - else if (e.level === 'warn') medium++; - else low++; + const dataAny = e.data as any; + const sev = (dataAny?.severity || '').toLowerCase(); + if (e.level === 'error' || sev === 'critical' || sev === 'high') high++; + else if (e.level === 'warn' || sev === 'warning' || sev === 'medium') medium++; + else if (sev === 'info' || sev === 'low' || (e.level === 'info' && e.type === 'tool_result')) low++; + else if (e.level === 'info' && e.type === 'tool_dispatch') { + // Ignore dispatch log events + } else if (sev === 'info' || e.level === 'info') low++; }); return { high, medium, low }; }, [events]); From 7d2b2af6aafed4999a4a1bf879b13fdf25b58b1b Mon Sep 17 00:00:00 2001 From: Ganesh Kumar Date: Sun, 30 Aug 2026 16:50:50 +0530 Subject: [PATCH 09/14] Fix Review Worker Errors and Diff Formatting --- internal/jobqueue/review_worker.go | 2 +- internal/jobqueue/tool_worker.go | 3 +++ internal/lrcconfig/lrcconfig.go | 13 +++++++------ 3 files changed, 11 insertions(+), 7 deletions(-) diff --git a/internal/jobqueue/review_worker.go b/internal/jobqueue/review_worker.go index b3190ba23..40ecb3fde 100644 --- a/internal/jobqueue/review_worker.go +++ b/internal/jobqueue/review_worker.go @@ -355,7 +355,7 @@ func (w *DiffReviewWorker) Work(ctx context.Context, job *river.Job[DiffReviewJo creditStore := storagetools.NewCreditStore(w.db) if err := creditStore.DeductCredits(ctx, args.OrgID, args.ReviewID, totalMultiplier, license.PlanType(planCode)); err != nil { w.handleFailure(ctx, args, logger, eventSink, fmt.Sprintf("failed to deduct tool credits: %v", err), "insufficient_tool_credits") - return nil + return fmt.Errorf("insufficient tool credits: %w", err) } for _, tool := range enabledTools { diff --git a/internal/jobqueue/tool_worker.go b/internal/jobqueue/tool_worker.go index 8fb8e17e4..8da1ea187 100644 --- a/internal/jobqueue/tool_worker.go +++ b/internal/jobqueue/tool_worker.go @@ -69,6 +69,9 @@ func NewToolReviewWorker(db *sql.DB) *ToolReviewWorker { if err != nil { log.Printf("[WARN] ToolReviewWorker: failed to load AWS config: %v. Creating fallback config.", err) awsCfg = aws.Config{Region: region} + if accessKey != "" && secretKey != "" { + awsCfg.Credentials = credentials.NewStaticCredentialsProvider(accessKey, secretKey, "") + } } return &ToolReviewWorker{db: db, awsCfg: awsCfg} } diff --git a/internal/lrcconfig/lrcconfig.go b/internal/lrcconfig/lrcconfig.go index 2181d89a5..a252a5d19 100644 --- a/internal/lrcconfig/lrcconfig.go +++ b/internal/lrcconfig/lrcconfig.go @@ -398,15 +398,16 @@ func FilterLocalCodeDiffsForTool(cfg *ToolRuleConfig, diffs []lib.LocalCodeDiff) func FormatLocalDiffs(diffs []lib.LocalCodeDiff) string { var b strings.Builder for _, d := range diffs { - path := d.NewPath - if path == "" { - path = d.OldPath + oldPath := d.OldPath + if oldPath == "" { + oldPath = d.NewPath } - if path == "" { - continue + newPath := d.NewPath + if newPath == "" { + newPath = d.OldPath } - b.WriteString(fmt.Sprintf("diff --git a/%s b/%s\n", path, path)) + b.WriteString(fmt.Sprintf("diff --git a/%s b/%s\n", oldPath, newPath)) if d.OldPath == "/dev/null" || d.OldPath == "" { b.WriteString("new file mode 100644\n") } else if d.NewPath == "/dev/null" || d.NewPath == "" { From 8013e6bcbf1cc2bbe0586b6dfef7a722c8b6ed01 Mon Sep 17 00:00:00 2001 From: Ganesh Kumar Date: Sun, 30 Aug 2026 17:04:48 +0530 Subject: [PATCH 10/14] Centralize Review Source Normalization and Support Scheduled Reviews --- ui/src/types/reviews.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ui/src/types/reviews.ts b/ui/src/types/reviews.ts index 26d96b0b3..20844d4db 100644 --- a/ui/src/types/reviews.ts +++ b/ui/src/types/reviews.ts @@ -46,7 +46,7 @@ export interface ReviewEvent { data: ReviewEventData; } -export type ReviewEventType = 'status' | 'log' | 'batch' | 'artifact' | 'completion'; +export type ReviewEventType = 'status' | 'log' | 'batch' | 'artifact' | 'completion' | 'tool_dispatch' | 'tool_result'; export type ReviewEventLevel = 'info' | 'warn' | 'error' | 'debug'; export interface ReviewEventData { From a32b628c66cec96a7390c90aec4e9ad0719b98b3 Mon Sep 17 00:00:00 2001 From: Ganesh Kumar Date: Fri, 4 Sep 2026 20:17:03 +0530 Subject: [PATCH 11/14] Implement 3-Tier Classification for Static Analysis Findings --- docs/tools/tools-integration-beta.md | 223 +++++++++++++++++++++------ 1 file changed, 179 insertions(+), 44 deletions(-) diff --git a/docs/tools/tools-integration-beta.md b/docs/tools/tools-integration-beta.md index efdde9260..84dffabfb 100644 --- a/docs/tools/tools-integration-beta.md +++ b/docs/tools/tools-integration-beta.md @@ -45,23 +45,18 @@ flowchart TD L --> M{"Lambda Success?"} M -->|No| N["Write Synthetic Failure
(exit_code: -1)"] - M -->|Yes| O["Parse Findings"] + M -->|Yes| O["Parse Findings
(check_id per finding)"] - O --> P{"KnownDeterministicRules?"} - P -->|Yes| Q["Deterministic
(0 LLM Tokens)"] - P -->|No| R["Helper LLM
(~120 Tokens)"] - - Q --> S["Store Comments
in ai_comments"] - R --> S + O --> P["ClassifyToolResult()
3-Tier Classifier"] + P --> Q["Store Comments
in ai_comments"] N --> T["w.finalizeIfAllDone()"] - S --> T + Q --> T T --> U{"Dispatched Count ==
Completed Count?"} U -->|No| V["Wait for Parallel
Tool Jobs"] U -->|Yes| W["Set Review Status:
completed"] - %% Developer / API style A fill:#dbeafe,stroke:#2563eb,stroke-width:2px,color:#1e3a8a style B fill:#dbeafe,stroke:#2563eb,stroke-width:2px,color:#1e3a8a @@ -88,8 +83,6 @@ flowchart TD %% Classification style P fill:#e0e7ff,stroke:#4f46e5,stroke-width:2px,color:#312e81 style Q fill:#dcfce7,stroke:#16a34a,stroke-width:2px,color:#14532d - style R fill:#fce7f3,stroke:#db2777,stroke-width:2px,color:#831843 - style S fill:#dcfce7,stroke:#16a34a,stroke-width:2px,color:#14532d %% Finalization style T fill:#e0f2fe,stroke:#0284c7,stroke-width:2px,color:#0c4a6e @@ -105,76 +98,218 @@ flowchart TD The tool classifier does not require manual hardcoding for every tool rule. The system uses a three-tier hybrid architecture to maximize performance and minimize token cost. +Each finding from a Lambda response carries a `check_id` field (e.g. `F401`, `DL3007`, `SC2086`, `generic-api-key`). +The classifier maps each `check_id` to a LiveReview taxonomy tuple: `category`, `subcategory`, `severity`, `confidence`, `type`. + ```mermaid flowchart TD - RAW["Raw Tool Finding"] --> TIER1{"Tier 1: In-Memory Map\nKnownDeterministicRules"} - - TIER1 -->|Match Found| DONE1["0 LLM Tokens\n0.0 ms Latency"] - TIER1 -->|Unmapped| TIER2{"Tier 2: DB Rule Cache\npublic.tool_rule_taxonomies"} - - TIER2 -->|Match Found| DONE2["0 LLM Tokens\n0.5 ms Latency"] - TIER2 -->|Cache Miss| TIER3["Tier 3: Helper LLM\ndeepseek-v4-flash (~188 Tokens)"] - - TIER3 --> SAVE_CACHE["Auto-Save Mapping to DB Cache"] - SAVE_CACHE --> DONE3["Classified Finding"] - DONE1 & DONE2 --> DONE3 + RAW["Raw Tool Finding
(tool_name + check_id + message)"] --> TIER1{"Tier 1: In-Memory Map
KnownDeterministicRules"} + + TIER1 -->|"Match Found
e.g. ruff:F401"| DONE1["Taxonomy Tuple
0 LLM Tokens · 0 ms"] + TIER1 -->|"Not Found"| DEDUPE["Deduplicate rule_keys
across all findings"] + + DEDUPE --> TIER2{"Tier 2: DB Cache
tool_rule_taxonomies"} + + TIER2 -->|"Match Found
(row exists)"| DONE2["Taxonomy Tuple
0 LLM Tokens · ~0.5 ms"] + TIER2 -->|"Cache Miss
(new rule_key)"| TIER3["Tier 3: Helper LLM
Batch Classify (unique rule_keys only)"] + + TIER3 --> SAVE["Upsert into
tool_rule_taxonomies
(persist across restarts)"] + SAVE --> DONE3["Taxonomy Tuple
~188 Tokens · one-time cost"] + + DONE1 & DONE2 & DONE3 --> COMMENT["Build ReviewComment
with full taxonomy fields"] ``` +### Key Design Points + +1. **Only unique `rule_key` values go to LLM.** A review with 300 ruff `F401` findings sends ONE LLM request for `ruff:F401`, not 300. +2. **DB cache survives server restarts.** The `tool_rule_taxonomies` table in PostgreSQL stores mappings permanently. Cold-start or re-deploy does not lose learned classifications. +3. **LLM sees the full official LiveReview taxonomy.** The prompt includes all valid categories and subcategories from `ValidTaxonomyMap`. + +--- + ### Classification Tiers #### Tier 1: Deterministic In-Memory Dictionary (0 LLM Tokens) -Static analysis tools generate structured rule IDs (such as `gitleaks:generic-api-key` or `bandit:b101`). + +Static analysis tools generate structured rule IDs (such as `ruff:F401` or `bandit:B303`). The classifier checks incoming findings against the in-memory map `KnownDeterministicRules`. -If a rule ID matches a Tier 1 entry: +Key: `strings.ToLower(toolName + ":" + check_id)` → `TaxonomyTuple{Category, Subcategory, Severity, Confidence, Type}`. + +If a rule key matches a Tier 1 entry: - The system assigns the taxonomy tuple immediately without an LLM call. - The process uses **0 LLM tokens** and completes in **0 milliseconds**. +Current Tier 1 entries (selected): + +| rule_key | Category | Subcategory | Severity | +|---|---|---|---| +| `gitleaks:generic-api-key` | Security | Secrets Management | critical | +| `gitleaks:aws-access-key` | Security | Secrets Management | critical | +| `bandit:b303` | Security | Cryptography | warning | +| `bandit:b101` | Security | Input Validation | warning | +| `hadolint:dl3006` | Maintainability | Configuration Management | warning | +| `shellcheck:sc2086` | Reliability | Fault Tolerance | info | +| `ruff:e501` | Maintainability | Code Complexity | info | +| `ruff:f401` | Maintainability | Dead Code | warning | +| `eslint:no-unused-vars` | Maintainability | Dead Code | warning | + +--- + #### Tier 2: Database Rule Cache (0 LLM Tokens) -If a rule ID is not present in Tier 1, the classifier queries the database table `public.tool_rule_taxonomies`. -This table stores previously classified custom and third-party rules. -If a rule ID matches a Tier 2 entry: +If a rule key is not in Tier 1, the classifier queries `public.tool_rule_taxonomies`. +This table stores all previously LLM-classified rule mappings. + +**Schema:** + +```sql +CREATE TABLE IF NOT EXISTS tool_rule_taxonomies ( + rule_key VARCHAR(300) PRIMARY KEY, -- e.g. "ruff:e711" + tool_name VARCHAR(100) NOT NULL, + rule_id VARCHAR(200) NOT NULL, + category VARCHAR(100) NOT NULL, + subcategory VARCHAR(100) NOT NULL, + severity VARCHAR(50) NOT NULL, + confidence VARCHAR(50) NOT NULL, + type VARCHAR(50) NOT NULL, + created_at TIMESTAMPTZ DEFAULT NOW(), + updated_at TIMESTAMPTZ DEFAULT NOW() +); +CREATE INDEX IF NOT EXISTS idx_tool_rule_taxonomies_tool ON tool_rule_taxonomies(tool_name, rule_id); +``` + +**Query pattern (batch):** + +```sql +SELECT rule_key, category, subcategory, severity, confidence, type +FROM tool_rule_taxonomies +WHERE rule_key = ANY($1); +``` + +If a rule key matches a Tier 2 row: - The system retrieves the cached taxonomy tuple from PostgreSQL. -- The process uses **0 LLM tokens** and completes in **0.5 milliseconds**. +- The process uses **0 LLM tokens** and completes in **~0.5 milliseconds**. + +--- -#### Tier 3: Ultra-Lean Helper LLM Batch Classifier (~188 LLM Tokens) -If a rule ID is absent from both Tier 1 and Tier 2, the classifier sends a compact request to `deepseek-v4-flash`. -The prompt contains a condensed taxonomy list and requires a pipe-delimited JSON response: +#### Tier 3: Helper LLM Batch Classifier (~188 LLM Tokens, one-time per new rule_key) + +If a rule key is absent from both Tier 1 and Tier 2, the classifier calls the configured Helper LLM. +Only **unique, previously-unseen rule keys** go to the LLM — not individual findings. + +The prompt uses the **full official LiveReview taxonomy** (`ValidTaxonomyMap`) so the LLM always produces valid output: ```text -Classify static analysis tool findings into LiveReview taxonomy. -Categories: Security, Performance, Maintainability, Reliability, Compliance, Architecture. -Subcategories: Secrets Management, Injection Vulnerabilities, Cryptography, Input Validation, Dead Code, Configuration Management, Code Complexity, Fault Tolerance, Data Exposure. +You are a static analysis tool classifier for LiveReview. + +Map each tool rule to the LiveReview taxonomy. + +CATEGORIES and valid SUBCATEGORIES: +Security: Authentication, Authorization, Secrets Management, Input Validation, + Injection Vulnerabilities, Cryptography, Dependency Vulnerabilities, + Data Exposure, Session Management, Security Logging & Auditing +Reliability: Error Handling, Fault Tolerance, Retry Logic, Timeout Management, + Resilience Patterns, Availability Risks, Data Integrity, Race Conditions, + Resource Cleanup, Failure Recovery +Correctness: Logic Errors, Edge Cases, Data Validation, State Management, + Concurrency Bugs, Business Rule Violations, Numerical Accuracy, + Null Handling, Type Safety, API Contract Violations +Performance: Database Efficiency, Algorithmic Complexity, Memory Usage, + CPU Utilization, Network Efficiency, Caching, Concurrency, + Resource Contention, Rendering Performance, Startup Performance +Cost: Cloud Resource Waste, Infrastructure Overprovisioning, + Storage Optimization, Database Cost Optimization, Excessive API Usage, + Third-Party Service Costs, Redundant Computation, LLM Token Consumption, + Caching Opportunities, Data Transfer Costs +Scalability: Horizontal Scaling, Vertical Scaling, Distributed Systems, + Load Balancing, Capacity Planning, Bottleneck Risks, Concurrency Limits, + Service Growth Constraints, Database Scaling, Queue Backpressure +Maintainability: Code Complexity, Readability, Documentation, Code Duplication, + Dead Code, Naming Quality, Testability, Technical Debt, + Refactoring Opportunities, Configuration Management, UI/UX, Accessibility +Architecture: Separation of Concerns, Modularity, Coupling, Cohesion, + Layering Violations, Dependency Management, Service Boundaries, + Domain Modeling, API Design, Extensibility +Developer Experience: Testing, CI/CD, Build System, Local Development, + Debuggability, Observability, Deployment Process, Automation, + Developer Tooling, Documentation Quality, UI/UX, Accessibility +Compliance & Governance: Privacy, Regulatory Compliance, Auditability, + Data Retention, Data Residency, Licensing, Policy Enforcement, + Access Controls, Change Management, Governance Standards + +SEVERITY codes: c=critical w=warning i=info +CONFIDENCE codes: H=High M=Medium L=Low +TYPE codes: B=Bug R=Risk O=Optimization S=Code Smell P=Best Practice D=Technical Debt + +Respond ONLY with a JSON array. Each element: "index|category|subcategory|severity|confidence|type" + +Findings (tool:rule_id - sample message): +[{"i":0,"r":"ruff:E711","m":"comparison to None"},{"i":1,"r":"hadolint:DL3008","m":"Pin versions in apt-get install"}] +``` -Format response strictly as JSON array of pipe-delimited strings: -["index|category|subcategory|severity_code|confidence_code|type_code"] +**Expected response:** + +```json +["0|Maintainability|Code Complexity|w|H|S","1|Maintainability|Configuration Management|w|H|P"] ``` -This request uses **~188 tokens**, which reduces LLM token consumption by 80.4% compared to full AI code review prompts. +--- ### Self-Learning Auto-Cache Mechanism -When Tier 3 classifies a new rule ID, the system persists the resulting tuple into `public.tool_rule_taxonomies`. -All future reviews across the organization use Tier 2 for that rule ID. -This mechanism converts unknown rules into **0-token executions** after the first execution. +After Tier 3 classifies a batch of new rule keys, the system upserts all results into `tool_rule_taxonomies`. + +```mermaid +sequenceDiagram + participant W as ToolReviewWorker + participant C as ClassifyToolResult() + participant DB as tool_rule_taxonomies + participant LLM as Helper LLM + + W->>C: findings (tool_name, check_id, message) + C->>C: Tier 1 check (in-memory) + C->>DB: Batch SELECT for unresolved rule_keys + DB-->>C: cached rows (0 tokens) + C->>LLM: Batch classify ONLY new rule_keys + LLM-->>C: pipe-delimited JSON + C->>DB: UPSERT new mappings (async, non-blocking) + C-->>W: []ReviewComment with taxonomy fields +``` + +All future invocations — including after server restart — find the rule key in Tier 2 (0 LLM tokens). --- ## Official Tool Specification Mapping Table -The table below lists the official documentation source and taxonomy mapping for each supported tool: +The table below lists the official documentation source and default taxonomy for each supported tool: -| Tool Name | Official Documentation Reference | Standard Category | Standard Subcategory | +| Tool Name | Official Documentation Reference | Default Category | Default Subcategory | |---|---|---|---| | **gitleaks** | [gitleaks.toml Spec](https://github.com/gitleaks/gitleaks/blob/master/config/gitleaks.toml) | Security | Secrets Management | | **bandit** | [Bandit Plugin Index](https://bandit.readthedocs.io/en/latest/plugins/index.html) | Security | Cryptography / Injection | -| **semgrep** | [Semgrep Rule Registry](https://semgrep.dev/rules) | Security / Reliability | Injection Vulnerabilities | +| **semgrep** | [Semgrep Rule Registry](https://semgrep.dev/rules) | Security | Injection Vulnerabilities | | **ruff** | [Ruff Rule Docs](https://docs.astral.sh/ruff/rules/) | Maintainability | Dead Code / Code Complexity | | **hadolint** | [Hadolint Rules Wiki](https://github.com/hadolint/hadolint#rules) | Maintainability | Configuration Management | | **shellcheck** | [ShellCheck Wiki](https://github.com/koalaman/shellcheck/wiki) | Reliability | Fault Tolerance | -| **eslint** | [ESLint Rules Guide](https://eslint.org/docs/latest/rules) | Maintainability / Security | Dead Code / Injection | +| **eslint** | [ESLint Rules Guide](https://eslint.org/docs/latest/rules) | Maintainability | Dead Code | | **trivy** | [Trivy Scanner Docs](https://aquasecurity.github.io/trivy) | Security | Dependency Vulnerabilities | +| **detect-secrets** | [detect-secrets Docs](https://github.com/Yelp/detect-secrets) | Security | Secrets Management | +| **trufflehog** | [TruffleHog Docs](https://github.com/trufflesecurity/trufflehog) | Security | Secrets Management | +| **tfsec** | [tfsec Docs](https://github.com/aquasecurity/tfsec) | Security | Configuration Management | +| **actionlint** | [actionlint Docs](https://github.com/rhysd/actionlint) | Reliability | Fault Tolerance | +| **osv** | [OSV Scanner Docs](https://google.github.io/osv-scanner) | Security | Dependency Vulnerabilities | +| **zizmor** | [zizmor Docs](https://github.com/woodruffw/zizmor) | Security | Injection Vulnerabilities | +| **checkov** | [Checkov Docs](https://www.checkov.io/5.Policy%20Index/terraform.html) | Security | Configuration Management | +| **brakeman** | [Brakeman Warnings](https://brakemanscanner.org/docs/warning_types/) | Security | Injection Vulnerabilities | +| **golangci-lint** | [golangci-lint Rules](https://golangci-lint.run/usage/linters/) | Maintainability | Code Complexity | +| **kubescape** | [Kubescape Controls](https://hub.armosec.io/docs/controls) | Security | Configuration Management | +| **spectral** | [Spectral Rules](https://docs.stoplight.io/docs/spectral/674b27b261c3c-overview) | Maintainability | API Design | +| **openapi** | [OpenAPI Spec](https://spec.openapis.org/oas/v3.1.0) | Maintainability | API Design | +| **ktlint** | [ktlint Rules](https://pinterest.github.io/ktlint/) | Maintainability | Code Complexity | +| **phpcs** | [PHPCS Standards](https://github.com/squizlabs/PHP_CodeSniffer/wiki) | Maintainability | Code Complexity | +| **rubocop** | [RuboCop Docs](https://docs.rubocop.org) | Maintainability | Code Complexity | --- From ffaa0376e8eff39a3bbb55c9d3bdcc3181136c66 Mon Sep 17 00:00:00 2001 From: Ganesh Kumar Date: Sun, 18 Oct 2026 01:45:28 +0530 Subject: [PATCH 12/14] Port third-party tools UI onto master's ReviewDetail --- ui/src/pages/Reviews/ReviewDetail.tsx | 132 ++++++++++++++++++++++++++ 1 file changed, 132 insertions(+) diff --git a/ui/src/pages/Reviews/ReviewDetail.tsx b/ui/src/pages/Reviews/ReviewDetail.tsx index a75e22b68..a87ffe451 100644 --- a/ui/src/pages/Reviews/ReviewDetail.tsx +++ b/ui/src/pages/Reviews/ReviewDetail.tsx @@ -7,6 +7,9 @@ import { reviewStatusBadge } from '../../utils/reviewDisplay'; import { Button, Icons, Tabs } from '../../components/UIPrimitives'; import { ReviewEventsPage, DiffViewerPanel } from '../../components/reviews'; import AutoApprovePanel from '../../components/reviews/AutoApprovePanel'; +import { ToolAnalysisCard, ToolAccountingData, ToolBreakdownItem } from '../../components/reviews/ToolAnalysisCard'; +import apiClient from '../../api/apiClient'; +import { useOrgContext } from '../../hooks/useOrgContext'; import { getReviewPolicyLog } from '../../api/riskPolicy'; import { getReview, @@ -88,6 +91,127 @@ const ReviewDetail: React.FC = () => { const [blastPoll, setBlastPoll] = useState<'checking' | 'waiting' | 'gave_up'>('checking'); const [riskScores, setRiskScores] = useState([]); + // ── Third-party static analysis tools ────────────────────────────────── + // Tool execution is reported as tool_dispatch / tool_result events; derive + // the accounting card from them so it renders without a separate fetch. + const { currentOrg } = useOrgContext(); + const [dbToolMultipliers, setDbToolMultipliers] = useState>({}); + + useEffect(() => { + if (!currentOrg?.id) return; + apiClient + .get<{ tools?: Array<{ name: string; multiplier: number }> }>(`/orgs/${currentOrg.id}/tools`) + .then((res) => { + const multMap: Record = {}; + (res.tools || []).forEach((t) => { + if (t.name && typeof t.multiplier === 'number') { + multMap[t.name.toLowerCase()] = t.multiplier; + } + }); + setDbToolMultipliers(multMap); + }) + .catch(() => { + // Fall back to the default multipliers when the endpoint is unavailable. + }); + }, [currentOrg?.id]); + + const DEFAULT_TOOL_MULTIPLIERS: Record = useMemo(() => ({ + ruff: 1.0, + bandit: 1.0, + gitleaks: 1.0, + eslint: 2.0, + semgrep: 3.0, + hadolint: 0.5, + actionlint: 0.5, + shellcheck: 0.5, + trufflehog: 2.0, + trivy: 2.5, + spectral: 1.0, + brakeman: 1.5, + kubescape: 2.0, + zizmor: 1.0, + openapi: 0.5, + }), []); + + const effectiveToolAccounting = useMemo(() => { + const toolEvents = events.filter( + (e) => (e.type as string) === 'tool_dispatch' || (e.type as string) === 'tool_result', + ); + if (toolEvents.length === 0) return null; + + const dispatchedMap = new Map(); + const resultsMap = new Map(); + const order: string[] = []; + + toolEvents.forEach((e) => { + try { + const raw = e.data as unknown; + const data: any = typeof raw === 'string' ? JSON.parse(raw) : raw; + if (!data) return; + const toolName: string | undefined = data.tool_name || data.toolName; + if (!toolName) return; + + if ((e.type as string) === 'tool_dispatch') { + if (!dispatchedMap.has(toolName)) { + dispatchedMap.set(toolName, data.status || 'pending'); + order.push(toolName); + } + } else { + if (!dispatchedMap.has(toolName)) { + dispatchedMap.set(toolName, 'completed'); + order.push(toolName); + } + const findings = Array.isArray(data.findings) ? data.findings : []; + const creditsUsed = + typeof data.credits_used === 'number' ? data.credits_used + : typeof data.credits === 'number' ? data.credits + : typeof data.cost === 'number' ? data.cost + : typeof data.multiplier === 'number' ? data.multiplier + : (dbToolMultipliers[toolName.toLowerCase()] ?? DEFAULT_TOOL_MULTIPLIERS[toolName.toLowerCase()] ?? 1.0); + resultsMap.set(toolName, { + exitCode: typeof data.exit_code === 'number' ? data.exit_code : 0, + findingsCount: findings.length, + creditsUsed, + }); + } + } catch (err) { + console.warn('Error parsing tool event data:', err); + } + }); + + if (order.length === 0) return null; + + let totalComments = 0; + let toolsExecuted = 0; + let totalCredits = 0; + const toolBreakdown: ToolBreakdownItem[] = order.map((toolName) => { + const res = resultsMap.get(toolName); + if (res) { + toolsExecuted++; + totalComments += res.findingsCount; + const credits = res.creditsUsed ?? dbToolMultipliers[toolName.toLowerCase()] ?? DEFAULT_TOOL_MULTIPLIERS[toolName.toLowerCase()] ?? 1.0; + totalCredits += credits; + let status = 'clean'; + if (res.exitCode !== 0) status = 'failed'; + else if (res.findingsCount > 0) status = 'completed'; + return { toolName, creditsUsed: credits, commentsGenerated: res.findingsCount, status }; + } + return { + toolName, + creditsUsed: 0, + commentsGenerated: 0, + status: dispatchedMap.get(toolName) || 'pending', + }; + }); + + return { + totalToolCredits: totalCredits, + toolsExecuted: toolsExecuted || toolBreakdown.length, + totalCommentsGenerated: totalComments, + toolBreakdown, + }; + }, [events, dbToolMultipliers, DEFAULT_TOOL_MULTIPLIERS]); + useEffect(() => { // Blast radius comes from `git lrc review` or the server-side job, which runs alongside the // review; until it lands (404), re-check every 30s for up to its 15 min timeout. @@ -722,6 +846,14 @@ const ReviewDetail: React.FC = () => { )}
+ {/* Third-party static analysis tools, derived from tool_dispatch / tool_result events */} + {effectiveToolAccounting && effectiveToolAccounting.toolBreakdown.length > 0 && ( +
+

Static analysis tools

+ +
+ )} + {/* Findings / Accounting / Events */} Date: Tue, 20 Oct 2026 19:13:00 +0530 Subject: [PATCH 13/14] Enable Policy Checks for URL-Linked Reviews; Refactor Review Detail UI LiveReview Pre-Commit Check: ran (iter:57, coverage:92%) --- internal/api/pull_request_review_trigger.go | 5 +- internal/docindex/docs/.synced-commits.env | 4 +- .../wiki/Apply-Licence-to-LiveReview.md | 24 +- .../Download-Install-and-Run-LiveReview.md | 10 +- internal/docindex/docs/lr_wiki/wiki/Home.md | 2 +- ...iveReview-for-Your-Org-in-Under-2-Hours.md | 29 +- .../lr_wiki/wiki/Productionize-LiveReview.md | 2 +- internal/jobqueue/pr_policy_worker.go | 12 +- ui/src/api/reviews.ts | 7 +- .../components/reviews/AutoApprovePanel.tsx | 33 +- ui/src/components/reviews/RiskAssessment.tsx | 2 +- ui/src/pages/Reviews/ReviewDetail.tsx | 1131 ++++++++++++----- ui/src/pages/Settings/Settings.tsx | 7 +- 13 files changed, 861 insertions(+), 407 deletions(-) diff --git a/internal/api/pull_request_review_trigger.go b/internal/api/pull_request_review_trigger.go index 07f80397b..4b2362680 100644 --- a/internal/api/pull_request_review_trigger.go +++ b/internal/api/pull_request_review_trigger.go @@ -146,11 +146,12 @@ func (s *Server) getPullRequestOrg(c echo.Context) error { } // markForPolicyCheck flags a manual PR review so the worker runs the approval policy check once it completes. -// Reviews not linked to a known PR are left alone. +// Any review that points at a PR/MR by URL is eligible; the engine falls back to the review's +// own author fields when the PR itself was never synced into `pull_requests`. func (s *Server) markForPolicyCheck(reviewID, orgID int64) { if _, err := s.db.Exec( `UPDATE reviews SET metadata = COALESCE(metadata, '{}'::jsonb) || '{"pr_link_policy": true}'::jsonb - WHERE id = $1 AND org_id = $2 AND pull_request_id IS NOT NULL`, + WHERE id = $1 AND org_id = $2 AND (pull_request_id IS NOT NULL OR pr_mr_url IS NOT NULL)`, reviewID, orgID, ); err != nil { log.Printf("[WARN] failed to mark review %d for policy check: %v", reviewID, err) diff --git a/internal/docindex/docs/.synced-commits.env b/internal/docindex/docs/.synced-commits.env index 5bf3f8f31..599c8eaa9 100644 --- a/internal/docindex/docs/.synced-commits.env +++ b/internal/docindex/docs/.synced-commits.env @@ -1,4 +1,4 @@ GIT_LRC_COMMIT=1112c13df101abe2c0619ade06a652f1441a58f0 GIT_LRC_WIKI_COMMIT=a1bb17c392a46c1f3bf6cfad6396813ccf80a1a9 -HEXMOSHOMEPAGE_DOCS_COMMIT=306c7b807d0e1f9bbfc2366f7300e2363c56e0a1 -LIVEREVIEW_WIKI_COMMIT=58ee4a1060bf109d889dd2cc969a776fb337d2f7 +HEXMOSHOMEPAGE_DOCS_COMMIT=a904abb2d3e5f58a3f4f5cd2c6b5a6c14ab51e4e +LIVEREVIEW_WIKI_COMMIT=8ade37693a4271c8785f2321dccc09e440797881 diff --git a/internal/docindex/docs/lr_wiki/wiki/Apply-Licence-to-LiveReview.md b/internal/docindex/docs/lr_wiki/wiki/Apply-Licence-to-LiveReview.md index 283a7b9ae..984fab841 100644 --- a/internal/docindex/docs/lr_wiki/wiki/Apply-Licence-to-LiveReview.md +++ b/internal/docindex/docs/lr_wiki/wiki/Apply-Licence-to-LiveReview.md @@ -1,10 +1,26 @@ -In [Download, Install and Run LiveReview](Download,-Install-and-Run-LiveReview), I showed how to apply license as a matter of course - getting LiveReview going. +Purchase a LiveReview licence for unlimited lines of code reviews. -Here, I will show how License can be managed later. You can always see the ribbon mentioning License status at the top. Click "Update License", enter the Licence value and click "Save" to update license. +## Apply for a Licence -image +To get a licence, contact our support team through the [LiveReview Self-Hosted Access](https://hexmos.com/livereview/selfhosted-access/) page. + +Once you receive your licence, apply it to your installation: + +1. Click **Have a license?** in the ribbon at the top of the dashboard. +2. Paste the licence into the window and click **Save**. +## Manage Your Licence -Or, you can goto `Settings -> License` - you can perform the same actions but also get more details such as exact expiry date. +You can always see the licence status in the ribbon at the top. To change the licence, click **Update License**, enter the new licence value and click **Save**. + +image + +Or, go to `Settings -> License`. You can do the same there, and also see more details such as the exact expiry date. image + +### Renew Your Licence + +LiveReview shows a warning in the ribbon 15 days before your licence expires. To renew, contact support through the [LiveReview Self-Hosted Access](https://hexmos.com/livereview/selfhosted-access/) page, then apply the new licence with **Update License** as shown above. + +If a licence expires, LiveReview keeps working but goes back to the Free 30k plan until a valid licence is applied. diff --git a/internal/docindex/docs/lr_wiki/wiki/Download-Install-and-Run-LiveReview.md b/internal/docindex/docs/lr_wiki/wiki/Download-Install-and-Run-LiveReview.md index 22c81363e..8fa72e713 100644 --- a/internal/docindex/docs/lr_wiki/wiki/Download-Install-and-Run-LiveReview.md +++ b/internal/docindex/docs/lr_wiki/wiki/Download-Install-and-Run-LiveReview.md @@ -26,15 +26,7 @@ You can usually access the UI at `http://localhost:8081` as follows - enter the image -On first login - you'll get a notice on "Demo Mode". What this means is - the app provides basic review facilities, but some special webhook related features won't be available. Don't worry about it now - you can productionize app later. For now - close that window. +On first login - you'll get a notice on "Demo Mode". What this means is - the app provides basic review facilities, but some special webhook related features won't be available. Don't worry about it now - you can [productionize the app](Productionize-LiveReview) later. For now - close that window. image -You should see the license window - you can paste the Licence [from previous step](Get-a-LiveReview-Licence) here (or click "Get Licence" link there to get a License) - -image - -Enter the license and hit save - you'll get something like this: - -image - diff --git a/internal/docindex/docs/lr_wiki/wiki/Home.md b/internal/docindex/docs/lr_wiki/wiki/Home.md index c823459c1..3786ee9dc 100644 --- a/internal/docindex/docs/lr_wiki/wiki/Home.md +++ b/internal/docindex/docs/lr_wiki/wiki/Home.md @@ -22,7 +22,7 @@ Welcome to LiveReview docs, here you can find resources on how to make the best 1. [SMTP Configuration](SMTP-Configuration) 1. [Customize LiveReview to Your Team's Best Practices](Customize-LiveReview-to-Your-Teams-Best-Practices) 1. [Productionize LiveReview](Productionize-LiveReview) - 1. [Get Demo Mode Running](Productionize-LiveReview#get-demo-mode-running) + 1. [Get Production Mode Running](Productionize-LiveReview#get-production-mode-running) 1. [Update Configuration](Productionize-LiveReview#update-configuration) 1. [Setup Reverse Proxy](Productionize-LiveReview#setup-reverse-proxy) 1. [Apache Reverse Proxy Setup](Apache-Reverse-Proxy-Setup) diff --git a/internal/docindex/docs/lr_wiki/wiki/How-to-Set-Up-Self-Hosted-LiveReview-for-Your-Org-in-Under-2-Hours.md b/internal/docindex/docs/lr_wiki/wiki/How-to-Set-Up-Self-Hosted-LiveReview-for-Your-Org-in-Under-2-Hours.md index 68f812d36..b8ed50c6e 100644 --- a/internal/docindex/docs/lr_wiki/wiki/How-to-Set-Up-Self-Hosted-LiveReview-for-Your-Org-in-Under-2-Hours.md +++ b/internal/docindex/docs/lr_wiki/wiki/How-to-Set-Up-Self-Hosted-LiveReview-for-Your-Org-in-Under-2-Hours.md @@ -82,32 +82,9 @@ When you log in for the first time, you will need to set up your organization wo bf7de477-804e-4ea2-b5d4-4cbce997ac67 +> Your organization is ready. It comes with **30k LOC per month**. For unlimited LOC, [upgrade to a paid plan](https://hexmos.com/livereview/selfhosted-access/). -To activate your self-hosted deployment, you need a valid license key. - -1. Click the **Have a license** option on the dashboard. -2. Click **Get a License** from the modal. - -0f004cd9-fcf2-4dcb-a848-c883a0b15ab5 - -3. Sign in to Hexmos. - -39a3bb47-7a54-4127-b90a-d1de88315267 - -4. Click the **Regenerate Token** button. - -ac2bc261-6ffe-4beb-b60a-2b335b8ad852 - -5. Copy the LiveReview access token. - -938a44d2-e0bc-4255-89a8-978bebe16653 - -6. Paste the token into the modal on the dashboard. - -8ba039e5-4698-4bb6-8702-85c8cf7a1639 - - -### Step 9: Add an AI Provider +### Step 8: Add an AI Provider LiveReview requires an AI provider (like OpenAI, Anthropic, or Gemini) to generate reviews. 1. Click **AI Providers** from the Navbar. @@ -116,7 +93,7 @@ LiveReview requires an AI provider (like OpenAI, Anthropic, or Gemini) to genera 5ed8e93f-5024-449f-8138-6381ab62b8f6 -### Step 10: Invite Team Members +### Step 9: Invite Team Members Finally, invite your colleagues so they can start reviewing code. 1. Navigate to **Settings > User Management**. diff --git a/internal/docindex/docs/lr_wiki/wiki/Productionize-LiveReview.md b/internal/docindex/docs/lr_wiki/wiki/Productionize-LiveReview.md index 938fd6539..22640605d 100644 --- a/internal/docindex/docs/lr_wiki/wiki/Productionize-LiveReview.md +++ b/internal/docindex/docs/lr_wiki/wiki/Productionize-LiveReview.md @@ -1,5 +1,5 @@ -## Get Demo Mode Running +## Get Production Mode Running You can follow the article for [Demo Mode](Download,-Install-and-Run-LiveReview) and make sure the software is running first in your target server. diff --git a/internal/jobqueue/pr_policy_worker.go b/internal/jobqueue/pr_policy_worker.go index 110822801..cc8c9fa54 100644 --- a/internal/jobqueue/pr_policy_worker.go +++ b/internal/jobqueue/pr_policy_worker.go @@ -55,7 +55,7 @@ func (w *PRPolicyCheckWorker) Work(ctx context.Context, job *river.Job[PRPolicyC if err != nil { return fmt.Errorf("failed to load review %d: %w", args.ReviewID, err) } - if status != "completed" || !pullRequestID.Valid || !flagged { + if status != "completed" || !flagged { return nil } @@ -108,9 +108,13 @@ func (w *PRPolicyCheckWorker) Work(ctx context.Context, job *river.Job[PRPolicyC return nil } - if err := w.publish(ctx, args, pullRequestID.Int64, res); err != nil { - release() - return err + // Publishing onto the PR comment needs a synced PR row; a review that was only + // linked by URL still gets the local decision, just no comment. + if pullRequestID.Valid { + if err := w.publish(ctx, args, pullRequestID.Int64, res); err != nil { + release() + return err + } } return nil } diff --git a/ui/src/api/reviews.ts b/ui/src/api/reviews.ts index 1a42e05f2..7d3c16234 100644 --- a/ui/src/api/reviews.ts +++ b/ui/src/api/reviews.ts @@ -184,9 +184,12 @@ export const getDiffReview = async (reviewId: number): Promise => { +export const getBlastRadiusReport = async ( + reviewId: number, + signal?: AbortSignal, +): Promise => { try { - return await apiClient.get(`/api/v1/diff-review/${reviewId}/artifacts/blast-radius`); + return await apiClient.get(`/api/v1/diff-review/${reviewId}/artifacts/blast-radius`, { signal }); } catch (error) { console.error('Error fetching blast radius report:', error); throw error; diff --git a/ui/src/components/reviews/AutoApprovePanel.tsx b/ui/src/components/reviews/AutoApprovePanel.tsx index 5d58e0a66..ff7281083 100644 --- a/ui/src/components/reviews/AutoApprovePanel.tsx +++ b/ui/src/components/reviews/AutoApprovePanel.tsx @@ -20,8 +20,35 @@ const STAGE_HINT: Record = { final: HELP.stageFinal, }; -const verdictIcon = (v: string) => - v === 'pass' || v === 'approved' ? '✅' : v === 'fail' || v === 'blocked' ? '❌' : '⏭️'; +const VerdictIcon: React.FC<{ verdict: string }> = ({ verdict }) => { + const pass = verdict === 'pass' || verdict === 'approved'; + const fail = verdict === 'fail' || verdict === 'blocked'; + // 16px icon nudged to the cap height of the 14px row text, so icon, label and reason share one baseline. + const cls = 'w-4 h-4 mt-[3px] shrink-0'; + const stroke = { fill: 'none', stroke: 'currentColor', strokeWidth: 2, strokeLinecap: 'round' as const, strokeLinejoin: 'round' as const }; + if (pass) { + return ( + + + + + ); + } + if (fail) { + return ( + + + + + ); + } + return ( + + + + + ); +}; // Every stage's detail blob carries more than the one-line reason already // shown inline (risk's full feature-by-feature breakdown, llm's class + @@ -158,7 +185,7 @@ const AutoApprovePanel: React.FC<{ reviewId: number }> = ({ reviewId }) => { }); }} > - {verdictIcon(st.verdict)} + {STAGE_LABEL[st.stage] || st.stage} {STAGE_HINT[st.stage] && } diff --git a/ui/src/components/reviews/RiskAssessment.tsx b/ui/src/components/reviews/RiskAssessment.tsx index bf92bd068..03dd8ac39 100644 --- a/ui/src/components/reviews/RiskAssessment.tsx +++ b/ui/src/components/reviews/RiskAssessment.tsx @@ -72,7 +72,7 @@ export function riskAssessment( return { kind: 'ready', value: blastRadiusTierLabel(top), tone: TIER_TONE[blastRadiusTier(top)], text, report: ready, scores, top }; } if (poll === 'gave_up') { - return { kind: 'unavailable', value: 'Not available', tone: '!text-slate-400', text: 'No result arrived within 15 minutes.', scores: none, top: 0 }; + return { kind: 'unavailable', value: 'Not available', tone: '!text-slate-400', text: 'No risk report is available for this review.', scores: none, top: 0 }; } if (poll === 'waiting') { return isPR diff --git a/ui/src/pages/Reviews/ReviewDetail.tsx b/ui/src/pages/Reviews/ReviewDetail.tsx index a87ffe451..ba670d87d 100644 --- a/ui/src/pages/Reviews/ReviewDetail.tsx +++ b/ui/src/pages/Reviews/ReviewDetail.tsx @@ -1,23 +1,22 @@ import React, { useState, useEffect, useCallback, useRef, useMemo } from 'react'; import { useParams, Link, useNavigate } from 'react-router-dom'; -import { LuExternalLink, LuGitBranch } from 'react-icons/lu'; import ProviderIcon from '../../components/ProviderIcon'; import { riskAssessment, RiskAssessmentDetails, RiskScore } from '../../components/reviews/RiskAssessment'; -import { reviewStatusBadge } from '../../utils/reviewDisplay'; -import { Button, Icons, Tabs } from '../../components/UIPrimitives'; +import { reviewStatusBadge, SourceIcon } from '../../utils/reviewDisplay'; +import { Button, Icons, Tabs, RelativeTime } from '../../components/UIPrimitives'; import { ReviewEventsPage, DiffViewerPanel } from '../../components/reviews'; import AutoApprovePanel from '../../components/reviews/AutoApprovePanel'; +import { getReviewPolicyLog } from '../../api/riskPolicy'; import { ToolAnalysisCard, ToolAccountingData, ToolBreakdownItem } from '../../components/reviews/ToolAnalysisCard'; -import apiClient from '../../api/apiClient'; import { useOrgContext } from '../../hooks/useOrgContext'; -import { getReviewPolicyLog } from '../../api/riskPolicy'; +import apiClient from '../../api/apiClient'; import { getReview, getReviewEvents, getReviewSummary, getReviewAccounting, getReviewCommits, - getBlastRadiusReport, + getBlastRadiusReport, formatRelativeTime, getStatusText } from '../../api/reviews'; @@ -38,13 +37,6 @@ import { const ACCOUNTING_REFRESH_INTERVAL_MS = 15000; const COMMITS_PREVIEW_LIMIT = 5; -const HeaderStat: React.FC<{ label: string; value: React.ReactNode; className?: string }> = ({ label, value, className }) => ( -
-

{label}

-

{value}

-
-); - const hasAccountingDetails = (value: ReviewAccounting | null): boolean => { if (!value) { return false; @@ -76,9 +68,7 @@ const ReviewDetail: React.FC = () => { const [accountingRouteUnavailable, setAccountingRouteUnavailable] = useState(false); const [commits, setCommits] = useState([]); const [allCommitsShown, setAllCommitsShown] = useState(false); - const [detailsExpanded, setDetailsExpanded] = useState(false); - const hoverTimerRef = useRef | undefined>(undefined); - useEffect(() => () => clearTimeout(hoverTimerRef.current), []); + const [detailsExpanded, setDetailsExpanded] = useState(true); const [commitsLoaded, setCommitsLoaded] = useState(false); const [loading, setLoading] = useState(true); const [error, setError] = useState(null); @@ -91,157 +81,70 @@ const ReviewDetail: React.FC = () => { const [blastPoll, setBlastPoll] = useState<'checking' | 'waiting' | 'gave_up'>('checking'); const [riskScores, setRiskScores] = useState([]); - // ── Third-party static analysis tools ────────────────────────────────── - // Tool execution is reported as tool_dispatch / tool_result events; derive - // the accounting card from them so it renders without a separate fetch. - const { currentOrg } = useOrgContext(); - const [dbToolMultipliers, setDbToolMultipliers] = useState>({}); - - useEffect(() => { - if (!currentOrg?.id) return; - apiClient - .get<{ tools?: Array<{ name: string; multiplier: number }> }>(`/orgs/${currentOrg.id}/tools`) - .then((res) => { - const multMap: Record = {}; - (res.tools || []).forEach((t) => { - if (t.name && typeof t.multiplier === 'number') { - multMap[t.name.toLowerCase()] = t.multiplier; - } - }); - setDbToolMultipliers(multMap); - }) - .catch(() => { - // Fall back to the default multipliers when the endpoint is unavailable. - }); - }, [currentOrg?.id]); - - const DEFAULT_TOOL_MULTIPLIERS: Record = useMemo(() => ({ - ruff: 1.0, - bandit: 1.0, - gitleaks: 1.0, - eslint: 2.0, - semgrep: 3.0, - hadolint: 0.5, - actionlint: 0.5, - shellcheck: 0.5, - trufflehog: 2.0, - trivy: 2.5, - spectral: 1.0, - brakeman: 1.5, - kubescape: 2.0, - zizmor: 1.0, - openapi: 0.5, - }), []); - - const effectiveToolAccounting = useMemo(() => { - const toolEvents = events.filter( - (e) => (e.type as string) === 'tool_dispatch' || (e.type as string) === 'tool_result', - ); - if (toolEvents.length === 0) return null; - - const dispatchedMap = new Map(); - const resultsMap = new Map(); - const order: string[] = []; - - toolEvents.forEach((e) => { - try { - const raw = e.data as unknown; - const data: any = typeof raw === 'string' ? JSON.parse(raw) : raw; - if (!data) return; - const toolName: string | undefined = data.tool_name || data.toolName; - if (!toolName) return; - - if ((e.type as string) === 'tool_dispatch') { - if (!dispatchedMap.has(toolName)) { - dispatchedMap.set(toolName, data.status || 'pending'); - order.push(toolName); - } - } else { - if (!dispatchedMap.has(toolName)) { - dispatchedMap.set(toolName, 'completed'); - order.push(toolName); - } - const findings = Array.isArray(data.findings) ? data.findings : []; - const creditsUsed = - typeof data.credits_used === 'number' ? data.credits_used - : typeof data.credits === 'number' ? data.credits - : typeof data.cost === 'number' ? data.cost - : typeof data.multiplier === 'number' ? data.multiplier - : (dbToolMultipliers[toolName.toLowerCase()] ?? DEFAULT_TOOL_MULTIPLIERS[toolName.toLowerCase()] ?? 1.0); - resultsMap.set(toolName, { - exitCode: typeof data.exit_code === 'number' ? data.exit_code : 0, - findingsCount: findings.length, - creditsUsed, - }); - } - } catch (err) { - console.warn('Error parsing tool event data:', err); - } - }); - - if (order.length === 0) return null; - - let totalComments = 0; - let toolsExecuted = 0; - let totalCredits = 0; - const toolBreakdown: ToolBreakdownItem[] = order.map((toolName) => { - const res = resultsMap.get(toolName); - if (res) { - toolsExecuted++; - totalComments += res.findingsCount; - const credits = res.creditsUsed ?? dbToolMultipliers[toolName.toLowerCase()] ?? DEFAULT_TOOL_MULTIPLIERS[toolName.toLowerCase()] ?? 1.0; - totalCredits += credits; - let status = 'clean'; - if (res.exitCode !== 0) status = 'failed'; - else if (res.findingsCount > 0) status = 'completed'; - return { toolName, creditsUsed: credits, commentsGenerated: res.findingsCount, status }; - } - return { - toolName, - creditsUsed: 0, - commentsGenerated: 0, - status: dispatchedMap.get(toolName) || 'pending', - }; - }); - - return { - totalToolCredits: totalCredits, - toolsExecuted: toolsExecuted || toolBreakdown.length, - totalCommentsGenerated: totalComments, - toolBreakdown, - }; - }, [events, dbToolMultipliers, DEFAULT_TOOL_MULTIPLIERS]); - useEffect(() => { // Blast radius comes from `git lrc review` or the server-side job, which runs alongside the - // review; until it lands (404), re-check every 30s for up to its 15 min timeout. + // review; until it lands (404), re-check. A completed review has already had its chance, so + // it gets a short budget and then settles; only an in-flight review waits the full 15 min. + // Each request is bounded, otherwise a hung backend leaves the header at "checking" ("...") + // forever, since fetch() has no timeout of its own. + if (!review) return; let cancelled = false; let timer: ReturnType | undefined; let attempts = 0; - setBlastReport(null); + const finished = review.status === 'completed' || review.status === 'failed'; + const maxAttempts = finished ? 3 : 30; + const retryDelayMs = finished ? 2500 : 30000; + const requestBudgetMs = 5000; + // Seed from the last response for this review so a page refresh shows the value it already + // had instead of flashing "..." while the same artifact is fetched again. + const cacheKey = `lr:blast-report:${reviewId}`; + let seeded = false; + try { + const cached = sessionStorage.getItem(cacheKey); + if (cached) { + setBlastReport(JSON.parse(cached) as BlastRadiusReport | BlastRadiusSkipped); + seeded = true; + } + } catch { + // A stale or corrupt entry just means one extra fetch. + } + if (!seeded) setBlastReport(null); setBlastPoll('checking'); setRiskScores([]); + let inflight: AbortController | null = null; const load = () => { - getBlastRadiusReport(reviewId) + const controller = new AbortController(); + inflight = controller; + const abortTimer = setTimeout(() => controller.abort(), requestBudgetMs); + getBlastRadiusReport(reviewId, controller.signal) .then((report) => { - if (!cancelled) setBlastReport(report); + if (cancelled) return; + setBlastReport(report); + try { + sessionStorage.setItem(cacheKey, JSON.stringify(report)); + } catch { + // Storage full or blocked; the in-memory state is still correct. + } }) .catch(() => { if (cancelled) return; - if (++attempts < 30) { + if (++attempts < maxAttempts) { setBlastPoll('waiting'); - timer = setTimeout(load, 30000); + timer = setTimeout(load, retryDelayMs); } else { setBlastPoll('gave_up'); } - }); + }) + .finally(() => clearTimeout(abortTimer)); }; load(); return () => { cancelled = true; clearTimeout(timer); + inflight?.abort(); + inflight = null; }; - }, [reviewId]); + }, [reviewId, review?.status]); const pollingIntervalRef = useRef(null); const defaultedTabForReviewRef = useRef(null); @@ -319,6 +222,161 @@ const ReviewDetail: React.FC = () => { } }, []); + const { currentOrg } = useOrgContext(); + const [dbToolMultipliers, setDbToolMultipliers] = useState>({}); + + useEffect(() => { + if (!currentOrg?.id) return; + apiClient.get<{ tools?: Array<{ name: string; multiplier: number }> }>(`/orgs/${currentOrg.id}/tools`) + .then(res => { + const catalog = res.tools || []; + const multMap: Record = {}; + catalog.forEach(t => { + if (t.name && typeof t.multiplier === 'number') { + multMap[t.name.toLowerCase()] = t.multiplier; + } + }); + setDbToolMultipliers(multMap); + }) + .catch(() => { + // Silently fallback if offline or endpoint unavailable + }); + }, [currentOrg?.id]); + + const [toolAccounting, setToolAccounting] = useState(null); + const [toolExpanded, setToolExpanded] = useState(false); + + const DEFAULT_TOOL_MULTIPLIERS: Record = useMemo(() => ({ + ruff: 1.0, + bandit: 1.0, + gitleaks: 1.0, + eslint: 2.0, + semgrep: 3.0, + hadolint: 0.5, + actionlint: 0.5, + shellcheck: 0.5, + trufflehog: 2.0, + trivy: 2.5, + spectral: 1.0, + brakeman: 1.5, + kubescape: 2.0, + zizmor: 1.0, + openapi: 0.5, + }), []); + + const effectiveToolAccounting = useMemo(() => { + if (toolAccounting) return toolAccounting; + const toolEvents = events ? events.filter(e => (e.type as string) === 'tool_dispatch' || (e.type as string) === 'tool_result') : []; + if (toolEvents.length === 0) { + return null; + } + + const dispatchedMap = new Map(); + const resultsMap = new Map(); + const order: string[] = []; + + toolEvents.forEach(e => { + try { + const data = typeof e.data === 'string' ? JSON.parse(e.data) : e.data; + if (!data) return; + const toolName = data.tool_name || data.toolName; + if (!toolName) return; + + if ((e.type as string) === 'tool_dispatch') { + if (!dispatchedMap.has(toolName)) { + dispatchedMap.set(toolName, data.status || 'pending'); + order.push(toolName); + } + } else if ((e.type as string) === 'tool_result') { + if (!dispatchedMap.has(toolName)) { + dispatchedMap.set(toolName, 'completed'); + order.push(toolName); + } + const findings = Array.isArray(data.findings) ? data.findings : []; + const creditsUsed = typeof data.credits_used === 'number' ? data.credits_used + : typeof data.credits === 'number' ? data.credits + : typeof data.cost === 'number' ? data.cost + : typeof data.multiplier === 'number' ? data.multiplier + : (dbToolMultipliers[toolName.toLowerCase()] ?? DEFAULT_TOOL_MULTIPLIERS[toolName.toLowerCase()] ?? 1.0); + resultsMap.set(toolName, { + exitCode: typeof data.exit_code === 'number' ? data.exit_code : 0, + findingsCount: findings.length, + creditsUsed, + }); + } + } catch (err) { + console.warn('Error parsing tool event data:', err); + } + }); + + if (order.length === 0) return null; + + let totalComments = 0; + let toolsExecuted = 0; + let totalCredits = 0; + const toolBreakdown: ToolBreakdownItem[] = order.map(toolName => { + const res = resultsMap.get(toolName); + if (res) { + toolsExecuted++; + totalComments += res.findingsCount; + const credits = res.creditsUsed ?? dbToolMultipliers[toolName.toLowerCase()] ?? DEFAULT_TOOL_MULTIPLIERS[toolName.toLowerCase()] ?? 1.0; + totalCredits += credits; + let status = 'clean'; + if (res.exitCode !== 0) status = 'failed'; + else if (res.findingsCount > 0) status = 'completed'; + return { + toolName, + creditsUsed: credits, + commentsGenerated: res.findingsCount, + status, + }; + } + return { + toolName, + creditsUsed: 0, + commentsGenerated: 0, + status: dispatchedMap.get(toolName) || 'pending', + }; + }); + + return { + totalToolCredits: totalCredits, + toolsExecuted: toolsExecuted || toolBreakdown.length, + totalCommentsGenerated: totalComments, + toolBreakdown, + }; + }, [toolAccounting, events, dbToolMultipliers, DEFAULT_TOOL_MULTIPLIERS]); + + // Where the third-party static-analysis tools stand for this review, from the same + // tool_dispatch/tool_result events the Static Analysis panel is built from. The + // Auto Approve tab shows this so a policy decision is never read without knowing + // whether the static pass ran, is still running, or was never dispatched. + const staticAnalysisStatus = useMemo(() => { + const toolEvents = (events || []).filter(e => (e.type as string) === 'tool_dispatch' || (e.type as string) === 'tool_result'); + const dispatched = new Set(); + const finished = new Set(); + let findings = 0; + for (const e of toolEvents) { + try { + const data = typeof e.data === 'string' ? JSON.parse(e.data) : e.data; + const toolName = data?.tool_name || data?.toolName; + if (!toolName) continue; + if ((e.type as string) === 'tool_dispatch') { + dispatched.add(toolName); + } else { + finished.add(toolName); + if (Array.isArray(data.findings)) findings += data.findings.length; + } + } catch { + // ignore malformed event payloads + } + } + const total = Math.max(dispatched.size, finished.size); + if (total === 0) return { state: 'not_run' as const, total: 0, finished: 0, findings: 0 }; + if (finished.size < dispatched.size) return { state: 'running' as const, total, finished: finished.size, findings }; + return { state: 'done' as const, total, finished: finished.size, findings }; + }, [events]); + // Fetch review details const fetchReviewDetails = useCallback(async () => { if (!id) return; @@ -327,6 +385,231 @@ const ReviewDetail: React.FC = () => { setError(null); setAccountingError(null); setAccountingRouteUnavailable(false); + + if (id === 'test' || id === 'test1' || id === 'test2' || id === 'test3') { + const stage = id === 'test1' ? 1 : id === 'test2' ? 2 : 3; + + const testPrMrUrl = id === 'test1' + ? 'https://github.com/HexmosTech/git-lrc/pull/131' + : id === 'test3' + ? 'https://git.apps.hexmos.com/hexmos/livereview/-/merge_requests/2' + : undefined; + + const testProvider = id === 'test1' ? 'github' + : id === 'test3' ? 'gitlab' + : 'cli'; + + const testRepo = id === 'test3' ? 'livereview' : id === 'test2' ? 'repo-b' : 'git-lrc'; + const testBranch = id === 'test3' ? 'feat/rag-query' : id === 'test2' ? 'main' : 'feat/tools-integration-beta'; + + setReview({ + id: 999, + orgId: 1, + repository: testRepo, + branch: testBranch, + prMrUrl: testPrMrUrl, + triggerType: testProvider, + userEmail: 'ganeshkumar6120@gmail.com', + provider: testProvider, + status: stage === 3 ? 'completed' : 'in_progress', + createdAt: new Date().toISOString(), + completedAt: stage === 3 ? new Date().toISOString() : undefined, + }); + + setSummary({ + reviewId: 999, + currentStatus: stage === 3 ? 'completed' : 'in_progress', + lastActivity: new Date().toISOString(), + batchCount: 0, + eventCounts: { tool_result: stage === 1 ? 0 : stage === 2 ? 5 : 15 }, + }); + + if (stage === 1) { + // Test Stage 1: All tools Queued/Pending + setToolAccounting({ + totalToolCredits: 0, + toolsExecuted: 0, + totalCommentsGenerated: 0, + toolBreakdown: [ + { toolName: 'ruff', creditsUsed: 0, commentsGenerated: 0, status: 'pending' }, + { toolName: 'bandit', creditsUsed: 0, commentsGenerated: 0, status: 'pending' }, + { toolName: 'gitleaks', creditsUsed: 0, commentsGenerated: 0, status: 'pending' }, + { toolName: 'eslint', creditsUsed: 0, commentsGenerated: 0, status: 'pending' }, + { toolName: 'hadolint', creditsUsed: 0, commentsGenerated: 0, status: 'pending' }, + { toolName: 'actionlint', creditsUsed: 0, commentsGenerated: 0, status: 'pending' }, + { toolName: 'shellcheck', creditsUsed: 0, commentsGenerated: 0, status: 'pending' }, + { toolName: 'semgrep', creditsUsed: 0, commentsGenerated: 0, status: 'pending' }, + { toolName: 'trufflehog', creditsUsed: 0, commentsGenerated: 0, status: 'pending' }, + { toolName: 'trivy', creditsUsed: 0, commentsGenerated: 0, status: 'pending' }, + { toolName: 'spectral', creditsUsed: 0, commentsGenerated: 0, status: 'pending' }, + { toolName: 'brakeman', creditsUsed: 0, commentsGenerated: 0, status: 'pending' }, + { toolName: 'kubescape', creditsUsed: 0, commentsGenerated: 0, status: 'pending' }, + { toolName: 'zizmor', creditsUsed: 0, commentsGenerated: 0, status: 'pending' }, + { toolName: 'openapi', creditsUsed: 0, commentsGenerated: 0, status: 'pending' }, + ], + }); + } else if (stage === 2) { + // Test Stage 2: 5 Completed, 5 Running with Animated Spinners, 5 Queued + setToolAccounting({ + totalToolCredits: 4.0, + toolsExecuted: 5, + totalCommentsGenerated: 14, + toolBreakdown: [ + { toolName: 'ruff', creditsUsed: 1.0, commentsGenerated: 0, status: 'clean' }, + { toolName: 'bandit', creditsUsed: 1.0, commentsGenerated: 14, status: 'completed' }, + { toolName: 'gitleaks', creditsUsed: 1.0, commentsGenerated: 0, status: 'clean' }, + { toolName: 'hadolint', creditsUsed: 0.5, commentsGenerated: 0, status: 'clean' }, + { toolName: 'shellcheck', creditsUsed: 0.5, commentsGenerated: 0, status: 'clean' }, + { toolName: 'eslint', creditsUsed: 2.0, commentsGenerated: 0, status: 'running' }, + { toolName: 'semgrep', creditsUsed: 3.0, commentsGenerated: 0, status: 'running' }, + { toolName: 'trufflehog', creditsUsed: 2.0, commentsGenerated: 0, status: 'running' }, + { toolName: 'trivy', creditsUsed: 2.5, commentsGenerated: 0, status: 'running' }, + { toolName: 'actionlint', creditsUsed: 0.5, commentsGenerated: 0, status: 'running' }, + { toolName: 'spectral', creditsUsed: 0, commentsGenerated: 0, status: 'pending' }, + { toolName: 'brakeman', creditsUsed: 0, commentsGenerated: 0, status: 'pending' }, + { toolName: 'kubescape', creditsUsed: 0, commentsGenerated: 0, status: 'pending' }, + { toolName: 'zizmor', creditsUsed: 0, commentsGenerated: 0, status: 'pending' }, + { toolName: 'openapi', creditsUsed: 0, commentsGenerated: 0, status: 'pending' }, + ], + }); + } else { + // Test Stage 3: All 15 Tools Completed with Findings & Failures + setToolAccounting({ + totalToolCredits: 21.0, + toolsExecuted: 15, + totalCommentsGenerated: 35, + toolBreakdown: [ + { toolName: 'ruff', creditsUsed: 1.0, commentsGenerated: 0, status: 'clean' }, + { toolName: 'bandit', creditsUsed: 1.0, commentsGenerated: 14, status: 'completed' }, + { toolName: 'gitleaks', creditsUsed: 1.0, commentsGenerated: 0, status: 'clean' }, + { toolName: 'eslint', creditsUsed: 2.0, commentsGenerated: 5, status: 'completed' }, + { toolName: 'hadolint', creditsUsed: 0.5, commentsGenerated: 0, status: 'clean' }, + { toolName: 'actionlint', creditsUsed: 0.5, commentsGenerated: 2, status: 'completed' }, + { toolName: 'shellcheck', creditsUsed: 0.5, commentsGenerated: 0, status: 'clean' }, + { toolName: 'semgrep', creditsUsed: 3.0, commentsGenerated: 8, status: 'completed' }, + { toolName: 'trufflehog', creditsUsed: 2.0, commentsGenerated: 0, status: 'clean' }, + { toolName: 'trivy', creditsUsed: 2.5, commentsGenerated: 3, status: 'completed' }, + { toolName: 'spectral', creditsUsed: 1.0, commentsGenerated: 0, status: 'clean' }, + { toolName: 'brakeman', creditsUsed: 1.5, commentsGenerated: 1, status: 'completed' }, + { toolName: 'kubescape', creditsUsed: 2.0, commentsGenerated: 0, status: 'clean' }, + { toolName: 'zizmor', creditsUsed: 1.0, commentsGenerated: 2, status: 'completed' }, + { toolName: 'openapi', creditsUsed: 0.5, commentsGenerated: 0, status: 'failed' }, + ], + }); + } + setEvents([ + { + id: 1, + reviewId: 999, + orgId: 1, + time: new Date().toISOString(), + type: 'status', + level: 'info', + data: { message: 'Stage started: preparation' } + }, + { + id: 2, + reviewId: 999, + orgId: 1, + time: new Date().toISOString(), + type: 'status', + level: 'info', + data: { message: 'Stage completed: preparation' } + }, + { + id: 3, + reviewId: 999, + orgId: 1, + time: new Date().toISOString(), + type: 'status', + level: 'info', + data: { message: 'Stage started: analysis' } + }, + { + id: 4, + reviewId: 999, + orgId: 1, + time: new Date().toISOString(), + type: 'status', + level: 'info', + data: { message: 'Stage completed: analysis' } + }, + { + id: 5, + reviewId: 999, + orgId: 1, + time: new Date().toISOString(), + type: 'status', + level: 'info', + data: { message: 'Stage started: review' } + }, + { + id: 6, + reviewId: 999, + orgId: 1, + time: new Date().toISOString(), + type: 'log', + level: 'info', + data: { message: 'ruff: Clean (0 findings)' } + }, + { + id: 7, + reviewId: 999, + orgId: 1, + time: new Date().toISOString(), + type: 'log', + level: 'warn', + data: { message: 'bandit: 14 comments generated' } + }, + { + id: 8, + reviewId: 999, + orgId: 1, + time: new Date().toISOString(), + type: 'status', + level: 'info', + data: { message: 'Stage completed: review' } + }, + { + id: 9, + reviewId: 999, + orgId: 1, + time: new Date().toISOString(), + type: 'status', + level: 'info', + data: { message: 'Stage started: artifact generation' } + }, + { + id: 10, + reviewId: 999, + orgId: 1, + time: new Date().toISOString(), + type: 'artifact', + level: 'info', + data: { message: 'Posted 14 comments to merge request' } + }, + { + id: 11, + reviewId: 999, + orgId: 1, + time: new Date().toISOString(), + type: 'status', + level: 'info', + data: { message: 'Stage completed: artifact generation' } + }, + { + id: 12, + reviewId: 999, + orgId: 1, + time: new Date().toISOString(), + type: 'completion', + level: 'info', + data: { resultSummary: 'Review process completed', message: 'finalization complete' } + } + ]); + setLoading(false); + return; + } const reviewId = parseInt(id, 10); if (isNaN(reviewId)) { @@ -342,6 +625,23 @@ const ReviewDetail: React.FC = () => { setReview(reviewData); setSummary(summaryData); + if (summaryData?.toolSummary) { + const breakdown = summaryData.toolSummary.toolBreakdown.map((item: ToolBreakdownItem) => ({ + ...item, + creditsUsed: item.creditsUsed && item.creditsUsed > 0 + ? item.creditsUsed + : (dbToolMultipliers[item.toolName.toLowerCase()] ?? 1.0), + })); + const totalCredits = summaryData.toolSummary.totalCostUsd && summaryData.toolSummary.totalCostUsd > 0 + ? summaryData.toolSummary.totalCostUsd + : breakdown.reduce((sum: number, i: ToolBreakdownItem) => sum + i.creditsUsed, 0); + setToolAccounting({ + totalToolCredits: totalCredits, + toolsExecuted: summaryData.toolSummary.toolsExecuted, + totalCommentsGenerated: summaryData.toolSummary.totalCommentsGenerated, + toolBreakdown: breakdown, + }); + } await fetchAccountingDetails(reviewId, reviewData.status); // Relevant commits are best-effort/informational -- never block @@ -457,22 +757,26 @@ const ReviewDetail: React.FC = () => { return s; }, [events]); - // Events by severity, derived from the already-loaded events list -- - // error=High, warn=Medium, info/debug=Low. + // Events by severity, preferring server-provided summary.severityCounts + // or falling back to calculating from the loaded events list. const eventSeverityCounts = useMemo(() => { + if (summary?.severityCounts) { + return summary.severityCounts; + } let high = 0, medium = 0, low = 0; events.forEach(e => { const dataAny = e.data as any; const sev = (dataAny?.severity || '').toLowerCase(); + const eventType = String(e.type); if (e.level === 'error' || sev === 'critical' || sev === 'high') high++; else if (e.level === 'warn' || sev === 'warning' || sev === 'medium') medium++; - else if (sev === 'info' || sev === 'low' || (e.level === 'info' && e.type === 'tool_result')) low++; - else if (e.level === 'info' && e.type === 'tool_dispatch') { + else if (sev === 'info' || sev === 'low' || (e.level === 'info' && eventType === 'tool_result')) low++; + else if (e.level === 'info' && eventType === 'tool_dispatch') { // Ignore dispatch log events } else if (sev === 'info' || e.level === 'info') low++; }); return { high, medium, low }; - }, [events]); + }, [summary?.severityCounts, events]); // Initial load useEffect(() => { @@ -648,129 +952,210 @@ const ReviewDetail: React.FC = () => { return (
- {/* Header */} - -
-
-
-

- {review.repository.split('/').pop() || review.repository} - #{review.id} -

-

+ {/* ── Review header toolbar ── */} +

+
+ {/* Back button (Full-height hoverable zone) */} + + ← Back + + + {/* Title, Branch & Provider Logo (Unified full-height PR link zone) */} +
{ + if (review.prMrUrl) { + window.open(review.prMrUrl, '_blank', 'noopener,noreferrer'); + } + }} + role={review.prMrUrl ? 'link' : undefined} + tabIndex={review.prMrUrl ? 0 : undefined} + onKeyDown={(e) => { + if (!review.prMrUrl) return; + if (e.key === 'Enter' || e.key === ' ') { + e.preventDefault(); + window.open(review.prMrUrl, '_blank', 'noopener,noreferrer'); + } + }} + title={review.prMrUrl ? `Open PR/MR: ${review.prMrUrl}` : review.repository} + > + {/* Title & Branch */} +
+

+ {review.repository.split('/').pop() || review.repository} +

{review.branch && ( - - +

{review.branch} - - )} - {review.prMrUrl && ( - - View PR/MR - - +

)} -

+
+ + {/* Provider / CLI Logo */} +
+
+ +
+
-
-
- { setDetailsExpanded(v => !v); setToolExpanded(false); }} + onKeyDown={(e) => { + if (e.key === 'Enter' || e.key === ' ') { + e.preventDefault(); + setDetailsExpanded(v => !v); + setToolExpanded(false); + } + }} + role="button" + tabIndex={0} + aria-expanded={detailsExpanded} + aria-label="Toggle review details" + className="shrink-0 flex items-center gap-2.5 px-3.5 self-stretch cursor-pointer group select-none hover:bg-slate-700/30 transition-colors focus:outline-none focus-visible:ring-2 focus-visible:ring-inset focus-visible:ring-blue-500/60" + title="Toggle review details" > - {getStatusText(review.status)} - - {/* Polling control moved to ReviewEventsPage for consistency */} -
-
+ {/* Sentence — RelativeTime wrapped in fixed min-width to prevent layout shift */} +

+ {review.userEmail?.split('@')[0] || 'Someone'} + {' '}created{' '} + + + +

+ + {/* Pipe separator */} + | + + {/* Severity indicators */} +
+ {eventSeverityCounts.high} High + · + {eventSeverityCounts.medium} Med + · + {eventSeverityCounts.low} Low +
+ + {/* Pipe separator */} + | - {/* Review Info: compact one-row header, expands inline for - commits + review details rather than spreading everything - across the page by default. */} -
- +
+ {/* ── Details panel ── */} {detailsExpanded && ( -
-
-

+
+
+ {/* GROUP 1: Commits (real data) */} +
+

Commits{commitsLoaded && ` (${commits.length})`} -

+

{!commitsLoaded && (

Checking for commit information...

)} {commitsLoaded && commits.length === 0 && ( -

+

No commit information recorded for this review yet. Plain "lrc review" (staged/working) commits sync in the background after `git commit` and may take a moment to appear; PR/MR and --commit/--range reviews are recorded immediately once submitted.

)} {commits.length > 0 && ( <> -
    COMMITS_PREVIEW_LIMIT ? 'max-h-64 overflow-y-auto pr-1' : ''}`}> +
      COMMITS_PREVIEW_LIMIT ? 'overflow-y-auto pr-1' : 'overflow-hidden'}`}> {(allCommitsShown ? commits : commits.slice(0, COMMITS_PREVIEW_LIMIT)).map((commit) => ( -
    • -
      +
    • +
      {commit.refType === 'commit' && githubBaseUrl ? ( {commit.ref.substring(0, 8)} ) : ( - + {commit.refType === 'commit' ? commit.ref.substring(0, 8) : commit.ref} )} @@ -780,7 +1165,7 @@ const ReviewDetail: React.FC = () => { )}
      - {formatRelativeTime(commit.createdAt)} +
      • ))}
    @@ -788,71 +1173,125 @@ const ReviewDetail: React.FC = () => { )} )}
-
-

Review details

-
-
-
Created by
-
{review.userEmail || '-'}
+ {/* GROUP 2: Issues by Severity */} +
+

Issues by severity

+
+
+ {eventSeverityCounts.high} +

High

-
-
Created
-
{new Date(review.createdAt).toLocaleString()}
+
+ {eventSeverityCounts.medium} +

Medium

-
-
Batches
-
{summary?.batchCount ?? 0}
-
- {formatDuration(review.startedAt, review.completedAt) && ( -
-
Duration
-
{formatDuration(review.startedAt, review.completedAt)}
-
- )} -
-
-

Events by severity

-
- High {eventSeverityCounts.high} - Medium {eventSeverityCounts.medium} - Low {eventSeverityCounts.low} +
+ {eventSeverityCounts.low} +

Low

-
-

Risk assessment

+ {/* GROUP 3: Details & Progress (col-span-4 - equal width & height) */} +
+

Details & Progress

+ {/* Row 1: Progress stats including Events */} +
+
+

Duration

+

{formatDuration(review.startedAt, review.completedAt) || '—'}

+
+
+

Batches

+

{summary?.batchCount ?? '—'}

+
+
+

Events

+

{events.length}

+
+
+

Activity

+

{summary?.lastActivity ? : '—'}

+
+
+ {/* Row 2: Created by & Created at split 50/50 equally */} +
+
+

Created by

+

{review.userEmail || '—'}

+
+
+

Created at

+

+ {new Date(review.createdAt).toLocaleString([], { + year: 'numeric', month: 'short', day: 'numeric', + hour: '2-digit', minute: '2-digit' + })} +

+
+
+
+ + {/* GROUP 4: Risk assessment + policy status */} +
+
+

Risk assessment

+
+ {risk.value} + {review.metadata?.pr_link_policy && ( + + Auto-approve + + {policyApproved === null ? 'Pending' : policyApproved ? 'Approved' : 'Not approved'} + + + )} +
+
+
)} -
- {/* Third-party static analysis tools, derived from tool_dispatch / tool_result events */} - {effectiveToolAccounting && effectiveToolAccounting.toolBreakdown.length > 0 && ( -
-

Static analysis tools

- -
- )} + {/* Static Analysis Tools panel — expanded INSIDE the same outer card box */} + {toolExpanded && ( +
+
+ {effectiveToolAccounting && effectiveToolAccounting.toolBreakdown.length > 0 ? ( + + ) : ( +
+ + + + +

No static analysis tools recorded

+

Tool execution details will appear here as static analysis tools run.

+
+ )} +
+
+ )} +
{/* Findings / Accounting / Events */} { { id: 'findings', label: 'Findings' }, { id: 'accounting', label: 'Accounting' }, { id: 'events', label: 'Events' }, - ...(review.metadata?.pr_link_policy ? [{ + ...(review.metadata?.pr_link_policy || review.prMrUrl ? [{ id: 'auto-approve', label: ( @@ -880,8 +1319,22 @@ const ReviewDetail: React.FC = () => { ]} /> - {activeTab === 'auto-approve' && review.metadata?.pr_link_policy && ( -
+ {activeTab === 'auto-approve' && (review.metadata?.pr_link_policy || review.prMrUrl) && ( +
+ {/* Static-analysis pass status — so the policy funnel is read alongside whether the static review ran. */} +
+ Static analysis + + {staticAnalysisStatus.state === 'running' + ? `Running — ${staticAnalysisStatus.finished}/${staticAnalysisStatus.total} tools done` + : staticAnalysisStatus.state === 'done' + ? `Completed — ${staticAnalysisStatus.total} tools, ${staticAnalysisStatus.findings} findings` + : 'Not run'} + + {staticAnalysisStatus.state === 'not_run' && ( + No static-analysis tools were dispatched for this review. + )} +
)} @@ -898,16 +1351,18 @@ const ReviewDetail: React.FC = () => {
)} + + {/* Accounting Panel */}

Accounting

{accounting?.lastAccountedAt ? ( - - Last accounted {formatRelativeTime(accounting.lastAccountedAt)} + + Last accounted ) : ( - Auto-refresh every 15s + Auto-refresh every 15s )}
{accountingError && ( @@ -915,99 +1370,77 @@ const ReviewDetail: React.FC = () => { {accountingError}
)} -
-
-

Total LOC

-

{(accounting?.totalBillableLoc || 0).toLocaleString()}

+
+
+

Total LOC

+

{(accounting?.totalBillableLoc || 0).toLocaleString()}

-
-

Input Tokens

-

{formatInt(accounting?.totalInputTokens)}

+
+

Input Tokens

+

{formatInt(accounting?.totalInputTokens)}

-
-

Output Tokens

-

{formatInt(accounting?.totalOutputTokens)}

+
+

Output Tokens

+

{formatInt(accounting?.totalOutputTokens)}

-
-

Total Cost (USD)

-

{formatCurrency(accounting?.totalCostUsd)}

+
+

Total Cost (USD)

+

{formatCurrency(accounting?.totalCostUsd)}

-
-

Accounted Operations

-

{(accounting?.accountedOperations || 0).toLocaleString()}

+
+

Accounted Ops

+

{(accounting?.accountedOperations || 0).toLocaleString()}

-
-

Token-tracked Operations

-

{(accounting?.tokenTrackedOperations || 0).toLocaleString()}

+
+

Token-tracked Ops

+

{(accounting?.tokenTrackedOperations || 0).toLocaleString()}

-
- - Helper {helperEnabled ? 'enabled' : 'disabled'} - - {helperEnabled && helperMode && ( - - Mode: {helperMode} - - )} - {!!stageBreakdown.length && ( - - Stages tracked: {stageBreakdown.length} - - )} -
+ {!!stageBreakdown.length && ( -
+
-

Model Breakdown

- - {helperEnabled ? 'Leader and Helper stages' : 'Single-stage review'} +

Model Breakdown

+ + {helperEnabled ? 'Leader & Helper stages' : 'Single-stage review'}
-
+
{stageBreakdown.map((stage) => { const routeText = getStageRouteText(stage); const executionText = getStageExecutionText(stage); return (
-

{formatStageLabel(stage.stage)}

-

+

{formatStageLabel(stage.stage)}

+

{(stage.provider || 'unknown provider')} / {(stage.model || 'unknown model')}

{stage.pricingVersion && ( - + {stage.pricingVersion} )}
-
-
-

Input

-

{formatInt(stage.inputTokens)}

+
+
+

Input

+

{formatInt(stage.inputTokens)}

-
-

Output

-

{formatInt(stage.outputTokens)}

+
+

Output

+

{formatInt(stage.outputTokens)}

-
-

Cost

-

{formatCurrency(stage.costUsd)}

+
+

Cost

+

{formatCurrency(stage.costUsd)}

-
- {executionText && ( -

Execution: {executionText}

- )} - {routeText && ( -

Route: {routeText}

- )} -
); })} diff --git a/ui/src/pages/Settings/Settings.tsx b/ui/src/pages/Settings/Settings.tsx index 8f5f2fc49..a677e47f4 100644 --- a/ui/src/pages/Settings/Settings.tsx +++ b/ui/src/pages/Settings/Settings.tsx @@ -388,7 +388,8 @@ const Settings = () => { ) }] : []), // Third-Party Tools tab visible only in cloud mode for org owners - ...(isCloudMode() && currentOrg?.role === 'owner' ? [{ + // (super admins may access every owner surface — mirrors RequireOrgOwner) + ...(isCloudMode() && (isSuperAdmin || currentOrg?.role === 'owner') ? [{ id: 'third-party-tools', name: 'Third-Party Tools', icon: @@ -841,12 +842,12 @@ const Settings = () => { )} - {activeTab === 'third-party-tools' && isCloudMode() && currentOrg?.role === 'owner' && ( + {activeTab === 'third-party-tools' && isCloudMode() && (isSuperAdmin || currentOrg?.role === 'owner') && ( )} - {activeTab === 'third-party-tools' && isCloudMode() && currentOrg?.role !== 'owner' && ( + {activeTab === 'third-party-tools' && isCloudMode() && !(isSuperAdmin || currentOrg?.role === 'owner') && (
Tool configuration is only available to organization owners. From 5e9ddff65a1c406c7cd8d0a63c0f3fad1a9f765f Mon Sep 17 00:00:00 2001 From: Ganesh Kumar Date: Tue, 27 Oct 2026 22:25:09 +0530 Subject: [PATCH 14/14] Document Tool Integration Beta Architecture LiveReview Pre-Commit Check: ran (iter:1, coverage:0%) --- .gitignore | 4 ++ docs/tools/tools-integration-beta.md | 101 +++++++++++++++++++++++++-- 2 files changed, 99 insertions(+), 6 deletions(-) diff --git a/.gitignore b/.gitignore index 56d572bae..0697b5b33 100644 --- a/.gitignore +++ b/.gitignore @@ -96,3 +96,7 @@ scripts/prelivi/.venv/ email-preview email-preview.html livereview.dtx + + +.kiro/* +.agents/* \ No newline at end of file diff --git a/docs/tools/tools-integration-beta.md b/docs/tools/tools-integration-beta.md index 84dffabfb..32b106f79 100644 --- a/docs/tools/tools-integration-beta.md +++ b/docs/tools/tools-integration-beta.md @@ -16,9 +16,12 @@ The system includes the following components: - The **Tool Analysis Card** (`ui/src/components/reviews/ToolAnalysisCard.tsx`) renders on the `ReviewDetail` page. - Database migrations (`available_tools`, `org_tools`) manage global catalogs and per-organization settings. -- The `River` job queue manages parallel tool execution with `ToolReviewWorker`. +- The `River` job queue manages tool execution with `ToolReviewWorker`. Tool jobs share the `review` queue with AI review jobs. - AWS Lambda functions execute tools and return finding payloads. -- The `toolclassifier` module normalizes findings using deterministic rules, database caching, and low-token LLM batch requests. +- The `toolclassifier` module normalizes findings using deterministic rules and low-token LLM batch requests. + The planned database rule cache is not implemented yet. See "Caching Roadmap". +- Tool execution runs today on one trigger path only: the CLI flag `--tools` (`tools_only=true`). + The other trigger paths do not dispatch tools yet. See "Trigger Coverage". - The credit store deducts organization credits based on configured tool memory and timeout multipliers. --- @@ -39,7 +42,7 @@ flowchart TD G --> H["Insert tool_dispatch Events"] H --> I["Enqueue ToolReviewJob per tool
(with DiffZipBase64)"] - I --> J["River Queue: ToolReviewJob Pool
(10 Workers)"] + I --> J["River 'review' Queue
(ToolReviewJob shares
the AI review pool)"] J --> K["ToolReviewWorker"] K --> L["Invoke AWS Lambda"] @@ -158,7 +161,11 @@ Current Tier 1 entries (selected): #### Tier 2: Database Rule Cache (0 LLM Tokens) -If a rule key is not in Tier 1, the classifier queries `public.tool_rule_taxonomies`. +**Status: planned. This tier is not in the code yet.** +Today every miss on Tier 1 goes directly to Tier 3 and repeats the LLM token cost after each restart. +The "Caching Roadmap" section implements this tier. + +When implemented, the flow is: a rule key that Tier 1 does not cover finds its tuple in `public.tool_rule_taxonomies`. This table stores all previously LLM-classified rule mappings. **Schema:** @@ -256,9 +263,10 @@ Findings (tool:rule_id - sample message): --- -### Self-Learning Auto-Cache Mechanism +### Self-Learning Auto-Cache Mechanism (planned) -After Tier 3 classifies a batch of new rule keys, the system upserts all results into `tool_rule_taxonomies`. +After Tier 3 classifies a batch of new rule keys, the system will upsert all results into `tool_rule_taxonomies`. +**Status: planned. The upsert step is not in the code yet.** ```mermaid sequenceDiagram @@ -281,6 +289,87 @@ All future invocations — including after server restart — find the rule key --- +## Trigger Coverage + +The `reviews.trigger_type` column defines four dashboard layers: `precommit`, `mr-pr`, `api-mcp`, `scheduled`. +`api-mcp` combines the raw API trigger types `api` and `mcp`. + +| Trigger path | Review row producer | Runs tools today | Anchor | +|---|---|---|---| +| CLI, flag `--tools` | `DiffReviewJob`, field `tools_only=true` | Yes | `internal/jobqueue/review_worker.go`, dispatch block | +| CLI, normal AI review | `DiffReviewJob`, field `tools_only=false` | No | `review_worker.go` | +| PR comment, UI manual | `POST /api/v1/reviews`, `/connectors/trigger-review`, `TriggerReviewV2`, `ManualReviewJob` | No | `internal/api/pull_request_review_trigger.go` | +| API and MCP | `TriggerReviewV2` phases (`trigger_type` `api`, `mcp`) | No | `internal/api/dashboard_cache_manager.go` | +| Scheduled | `ScheduledReviewWorker` | No | `internal/jobqueue/scheduled_review_worker.go` | + +One enqueue site constructs `ToolReviewJobArgs`: `internal/jobqueue/review_worker.go`, inside the `tools_only` block. +The worker `finalizeIfAllDone` in `internal/jobqueue/tool_worker.go` marks the review `completed` when every dispatch has a result. + +## Universal Tool Execution Plan + +The goal: every trigger path runs static tools when the organization enables tools or the repo `.lrc` enables a tool. +No `--tools` flag appears in this flow. + +### Phase 1 - Shared Dispatcher + +Extract the `tools_only` dispatch block from `DiffReviewWorker` into one `internal/jobqueue` helper. +The helper takes org id, review id, plan code, diff zip, `.lrc` bundle, and the changed-file list. +The helper performs these steps: fetch enabled org tools, merge repo `.lrc` rules, apply diff matchers, deduct credits, insert `tool_dispatch` events, and enqueue one `tool_review` job per tool. +`DiffReviewWorker` calls the helper. Behavior of the CLI path stays identical. + +### Phase 2 - Server-Side Diff Package + +Manual, PR, API, MCP, and scheduled flows do not carry the CLI's diff zip. +The server builds the package: reuse the `internal/blastradius/repocache` clone, check out the head SHA, and zip the files changed between base and head, honoring `.lrc/ignore`. +Scheduled reviews reuse the same step on the diffed range. + +### Phase 3 - Combined Completion + +The current `finalizeIfAllDone` overwrites the review status to `completed`. In a combined run the AI result races it. +At dispatch the helper writes `metadata.tools_pending = N`. +The AI path and the tool worker share one completion helper that marks `completed` only when pending tools equal zero. +The completion summary text becomes "AI review + static analysis" when both layers ran. + +### Phase 4 - Wire Every Trigger + +`ManualReviewWorker` calls the helper after the AI review succeeds. This one hook covers manual UI, PR comment, API, and MCP triggers, because all of them share `TriggerReviewV2`. +`ScheduledReviewWorker` calls the same helper after its pipeline. +Tool failures insert synthetic `tool_result` events and never fail the review. + +### Phase 5 - Enablement Rules + +Tools run when the organization enables them globally, or when the repo `.lrc/policy/tools.toml` enables a tool. +The auto-enable loop for `.lrc` exists in the `tools_only` block today and moves into the helper. +Org settings show per-trigger coverage in the third-party tools tab. + +--- + +## Caching Roadmap + +Two caches are missing today. Both cut repeat token and credit costs. + +### Cache 1 - Rule Taxonomy DB Cache (Tier 2) + +Implement the table and the two queries described in the Tier 2 section: + +1. Migration creates `tool_rule_taxonomies` with the columns of the not-yet-implemented SQL block above. +2. `ClassifyToolResult` batch-selects unresolved rule keys before the LLM call. +3. After a Tier 3 batch, the worker upserts all new tuples (async, non-blocking). + +Effect: each tool `(tool_name, rule_id)` pair classifies once per database, not once per review or restart. + +### Cache 2 - Lambda Result Cache + +Key: `SHA256(tool_name, repo, base_sha, head_sha, tool config)`. +Value: the classified `tool_result` payload in the blob store, next to diff-review artifacts. + +On dispatch the helper checks the key first. On a hit the review reuses the findings and skips the Lambda invocation and its credits. +This helps force-push cycles and CI re-runs of the same commit. A cache event records the credit saving. + +--- + + + ## Official Tool Specification Mapping Table The table below lists the official documentation source and default taxonomy for each supported tool: