diff --git a/CHANGELOG.md b/CHANGELOG.md index ef535366..9877f34f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,12 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## [Unreleased] +### Added + +- **Explicit source freshness**: Add opt-in `index_status({ checkFreshness: true })` and `ocbi status --check-freshness --json` across supported hosts. Keep stored catalog readiness, source freshness, and coordinator state distinct; expose shared structured status without refreshing indexes. +- **Source-linked pre-edit tests**: Add opt-in `includeTests` to `codebase_edit_context`, reusing current-source JS/TS discovery for direct target and one-hop caller references. Return complete budgeted citations and selected-only structured evidence across hosts, distinguish discovery caps from response omissions, and preserve explicit non-coverage labels. +- **Working-tree impact**: Add `pr_impact({ workingTree: true })` and `ocbi impact --working-tree [--json]` for current indexed staged, unstaged, and untracked file-level graph evidence. Require explicit incremental refresh for stale bytes, preserve local Git/index state, disclose path/provenance/consistency limits, and report UNKNOWN for deleted, unindexed, symbol-free, or capped coverage. + ### Changed - **Unified codebase explorer**: Replace separate architecture/network/symbol views with a categorized module directory and one module → exact file → symbol drill-down. Stable caller → focus → callee layouts use paged neighbors, direct navigation, and inspectable original callsites, including self-calls. Refine typography, category cards, snapshot totals, metadata density, and evidence panels; show module-anchored display paths while preserving original locations and add sticky narrow-layout section jumps. Keep whole-snapshot Changes, Hotspots, and Cycles in Analyze without losing exploration state. Preserve offline HTML, public export/tool arguments, safe snapshot data, search/history, responsive dialogs, and dark/light themes; remove draggable-network layout code. @@ -16,6 +22,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed +- **Consumer-visible evidence correctness**: Count only retained source and graph rows in pre-edit details, close truncated source fences at whole lines, and reserve outer-heading/final-truncation overhead before source fitting. Credit source only when its complete fitted block survives final composition. Allow structural-mode architecture maps to read branch files from SQLite without a vector store. Add native-backed citation, Unicode/CRLF, scope, and token-budget acceptance. +- **Pre-edit source identity and cancellation**: Resolve canonical native symbol IDs before fitting target source, including large fragmented declarations, instead of accepting ranked body slices, suffix paths, or enclosing declarations. Cap source blocks at 512 tokens, withhold missing source, preserve default host roots, and share the exact lookup with evaluation. Keep selection-hook test references metadata-only and honor cancellation after empty lookups. +- **Working-tree snapshot integrity**: Detect concurrent database publication even when source hashes and catalog counts are unchanged. Reject lossy UTF-8 Git paths and retain only project-local endpoints for cross-boundary renames. + - **Concurrent branch materialization**: Use invocation-unique worktree registration names and a common-Git-directory lease around registration/deregistration, preventing overlapping calls from reading or recycling partially initialized peer metadata. Keep fetches and indexing callbacks concurrent, preserve cancellation cleanup and fail-closed retention, and cover peer preservation plus real multiprocess overlap. - **Directory watcher permission recovery**: Recover readable, policy-selected directory and project-root registrations rejected with `EPERM` or `EACCES`, including empty directories that later gain source files. Reuse the one-shot polling cutover and catch-up reconciliation without broadening ignored, excluded, restricted, protected-index, unreadable, or external-configuration scope. Exercise actual Chokidar 6 directory resources in the watcher/Indexer acceptance checks. - **Visualizer cyclic groups and graph scale**: Detect module and symbol strongly connected components iteratively, including direct recursion and loops longer than five symbols. Preserve distinct symbol IDs and page search, hotspot, relationship, and cyclic-group lists instead of silently capping results. diff --git a/README.md b/README.md index 70cc730c..915274a4 100644 --- a/README.md +++ b/README.md @@ -20,7 +20,7 @@ ocbi status --project /path/to/repo --host jcode ocbi search "retry recovery" --project /path/to/repo ``` -`cbi` remains available as a compatibility alias. `ocbi` provides status, indexing, search, definition lookup, and direct caller or callee inspection. See [Installation and host setup](docs/installation.md#human-cli) for the full command reference. +`cbi` remains available as a compatibility alias. `ocbi` provides status, indexing, search, definition lookup, direct caller/callee inspection, and current indexed working-tree impact. See [Installation and host setup](docs/installation.md#human-cli) for the full command reference. ## Highlights @@ -105,10 +105,10 @@ See [Installation and host setup](docs/installation.md) for complete instruction ## Recommended workflow -1. **Check readiness when unknown** with `index_status` or `/status`; index only when needed. +1. **Check stored readiness when unknown** with `index_status` or `/status`; use `index_status({checkFreshness:true})` when source/HEAD freshness matters. Ordinary status does not scan source; index only when needed. 2. **Orient in unfamiliar code** with one compact `codebase_context` first pass (`tokenBudget: 600`, `limit: 5`), then inspect the evidence. Skip this step when the task is already scoped. 3. **Go directly to known definitions or relationships** with `implementation_lookup`, `call_graph`, or `call_graph_path`. -4. **Optionally prepare a known edit target** with `codebase_edit_context` for bounded source and direct graph evidence. +4. **Optionally prepare a known edit target** with `codebase_edit_context` for bounded source and direct graph evidence; `includeTests:true` adds source-linked JS/TS test references, not runtime coverage. 5. **Choose metadata or matching source** with `codebase_peek` or `codebase_search`; do not repeat context calls unnecessarily. 6. **Read known paths or match literals directly** with targeted reads or `grep`. @@ -123,7 +123,7 @@ See [Installation and host setup](docs/installation.md) for complete instruction | Find analogous implementations or duplicates | `find_similar` | | Find direct callers or callees | `call_graph` | | Find a path between two symbols | `call_graph_path` | -| Analyze a branch or pull request blast radius | `pr_impact` | +| Analyze branch/PR impact, or current indexed work with `workingTree:true` | `pr_impact` | See [Tools and commands](docs/tools.md) for host availability, tool details, MCP prompts, and slash commands. diff --git a/docs/installation.md b/docs/installation.md index d306de7c..7f6105af 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -207,18 +207,28 @@ The concise `ocbi` command is for terminal use. `cbi` remains available as a com ```bash # Inspect readiness, then create or refresh the index ocbi status --project /path/to/repo --host jcode +# Optional read-only source/HEAD freshness check with machine-readable status +ocbi status --check-freshness --json --project /path/to/repo --host jcode ocbi index --project /path/to/repo --host jcode # Search and inspect code from the terminal ocbi search "retry recovery" --project /path/to/repo --limit 5 ocbi definition Indexer --project /path/to/repo ocbi graph callers Indexer --project /path/to/repo + +# Refresh edited source, then inspect the current indexed dirty tree without committing +ocbi index --project /path/to/repo --host omp +ocbi impact --working-tree --json --project /path/to/repo --host omp ``` Use `ocbi index --dry-run` for a parse-only embedding-token total, `--estimate-only` for an estimate, and `ocbi --help` for command usage. +Ordinary status reports stored catalog facts with freshness explicitly not checked. `--check-freshness` compares current selected source/configuration and HEAD with the index; it does not refresh the index or request embeddings. A successfully inspected missing/stale index still exits successfully; operational failures exit nonzero. The `--check-freshness` and status `--json` flags apply only to status (workspace status retains its separate JSON contract). + Without `--project`, the CLI discovers the nearest containing Git checkout and reuses its project configuration and branch catalog, even from a nested package directory. MCP startup uses the same rule. Nested repositories, submodules, and linked worktrees retain their own checkout roots. Non-Git directories keep the current directory as their scope. An explicit `--project` is never widened, so use `--project .` to deliberately index only a child directory. +`ocbi impact` requires `--working-tree` and accepts optional `--json` plus project/host/config selection. It reports the union of staged, unstaged, and untracked changes against the current indexed files, not staged source or deleted historical declarations. Refresh stale source with ordinary `ocbi index` first; impact inspection itself does not index or contact GitHub. Missing graph coverage or the 200-path cap produces UNKNOWN risk. See [Impact evidence](tools.md#pr_impact) for provenance and consistency limits. + ### Private history on GitHub.com and Enterprise Enable sources in the **project-local** host configuration using `repositoryHistory.repositories` (see [Configuration](configuration.md#private-github-history)). Selectors accept `[HOST/]OWNER/REPO`: bare `owner/repo` defaults to GitHub.com, and explicit `github.com/OWNER/REPO` normalizes to that same lowercase identity. Enterprise selectors normalize to lowercase `hostname/owner/repo`. Use a DNS hostname only, with no URL scheme, credentials, port, or traversal components. diff --git a/docs/pre-edit-context-design.md b/docs/pre-edit-context-design.md index 5a7bd117..6aafdd3e 100644 --- a/docs/pre-edit-context-design.md +++ b/docs/pre-edit-context-design.md @@ -14,15 +14,19 @@ The tool accepts: - `symbol` and optional `filePath`: an authoritative target when known - `tokenBudget`: total returned context budget - `callerLimit` and `calleeLimit`: direct graph-edge limits +- `includeTests` (default false): current-source JS/TS test-reference evidence +- `includeApiImpact` (default false): bounded Express/fetch associations It returns a single token-bounded evidence pack containing: -1. The authoritative target implementation source, when a symbol resolves. +1. A bounded excerpt of the authoritative target implementation source, when a symbol resolves. 2. Direct callers and callees for that resolved symbol, with locations and edge types. 3. A short risk note when the target is ambiguous, unresolved, or graph data is unavailable. 4. Conceptual retrieval evidence when no target can be resolved. -The first release must not claim test coverage or related-test certainty. The current index has test-path-aware ranking, but no durable test-to-symbol relationship. Test discovery is a separate, measurable follow-up. +The initial release did not claim test coverage. The opt-in test section now reuses native AST-backed `discoverSymbolTests`: direct target references precede graph neighbors, and one-hop caller-test references follow them. Target `filePath` disambiguation leaves test-directory scope independent. Returned callback citations, evidence labels, selected-only structured details, discovery caps, and response omissions remain inspectable; no static link claims runtime coverage or passing tests. See [Tools and commands](tools.md#codebase_edit_context) for the current contract. + +Resolved targets use canonical native symbol IDs and declaration coordinates rather than ranked body fragments. The shared per-indexer lookup serves both production and evaluation. Source blocks are capped at 512 tokens, preserve declaration starts and whole-line fences, and leave response space for relationships and test references. Unreadable source retains canonical metadata but receives no source-included credit. ## Integration boundaries diff --git a/docs/tools.md b/docs/tools.md index 88fbfb2a..586e318c 100644 --- a/docs/tools.md +++ b/docs/tools.md @@ -170,6 +170,20 @@ Do not repeat context calls or broad reads when existing evidence already answer ## Core retrieval tools +### `codebase_edit_context` + +Use for a known or suspected edit target. `query`, optional `symbol`/`filePath`, caller/callee limits, and a 128–4000-token budget produce target source and direct graph evidence, or a risk-marked conceptual fallback. `filePath` disambiguates the target; it does not restrict related test locations to the implementation directory. Details count only source and graph evidence retained in the response. + +Target source comes from the resolved native symbol ID, not ranked body chunks. Selection checks the exact project path, name, kind, and declaration start; a same-named suffix path or enclosing declaration is not target evidence. Large declarations start at their canonical signature. Source blocks are capped at 512 tokens to reserve space for relationships and test references, while the complete response still honors the requested budget. Missing or nonunique implementation source is explicitly withheld rather than replaced with another body or an inaccessible-file marker. + +Source fitting reserves the outer heading and final truncation notice before graph/API sections are appended. Source receives credit only when the complete fitted block, including its closing fence, survives final composition; tight budgets do not authorize a clipped line or dangling source fence. + +Set `includeTests: true` to add current-source, AST-verified JS/TS test references using the same bounded discovery engine as `codebase_context`. `direct_target_test_reference` identifies a call/construction of the target inside a supported test callback; `related_caller_test_reference` identifies a verified one-hop caller reference. Both are static source links, **not runtime coverage or proof that a test passes**. Test rows contain exact callback citations, the canonical referenced symbol, and the reference line; drill into those locations for assertions. + +Direct test rows precede graph neighbors; related caller-test rows follow them. Every complete row, source excerpt, label, and notice shares the existing token budget. MCP `structuredContent.testDiscovery` and Pi/omp details contain only rendered `results`, discovery resolution/counters/`capped`, and `omittedReferenceCount` for proved references lost to response fitting. Empty, ambiguous, unsupported, capped, or unavailable discovery is not proof that no tests exist. Omitted/false `includeTests` does not run discovery. + +The existing `includeApiImpact: true` option remains separate: Express/fetch `candidate_test_file` matches are candidates, not verified test-reference labels. See [API impact](api-impact.md). MCP and Pi/omp expose structured pre-edit details; OpenCode retains its text response. + ### `architecture_context` Use `architecture_context` when an agent needs a concise repository map before focused retrieval or edits. Each module includes a responsibility excerpt derived from readable source and exact symbol/file/line citations. Cross-module boundaries include representative source and target symbols, while missing or sparse graph coverage is reported explicitly instead of inferred. Coverage includes branch-aware resolved and unresolved edge totals plus a deterministic per-language breakdown. @@ -242,9 +256,11 @@ Finds code analogous to a supplied snippet. Useful for duplicate detection, patt ### `index_status` -Reports readiness, chunk counts, compatibility, current provider/model, and index health information. +Reports stored catalog state, chunk counts, compatibility, current provider/model, and auto-index coordination separately from source freshness. Ordinary calls return `freshness: null` and `Source freshness: not checked`; a populated catalog is not proof that current source bytes or HEAD were indexed. + +Pass `checkFreshness: true` to compare the selected source/configuration hashes and HEAD with the index using the existing freshness engine. Structured status returns `{ readable, current, reason }` under `freshness`, with the same facts shown in text. The check does not index, retry failed batches, request embeddings, or start watchers; explicitly enabled SCIP validation may use its configured bounded decoder. This is a point-in-time check, not an atomic snapshot. Cancelled or failed checks do not become current results. -Catalog presence and provider/model compatibility are separate from pending storage migrations. An active catalog can be `ready` and compatible while its parser, symbol, or call-graph migration markers are outdated. Shared status results expose `migrationRequired`; public status reports pending migrations and recommends ordinary `index_codebase` with `force: false`. This refresh upgrades affected cached declarations without deleting the index. With automatic indexing disabled, existing declarations can remain available while newly supported declarations are absent; with readiness-gated retrieval, pending migrations can block search. Status only checks stored migration markers, not source-file or HEAD freshness, and never starts a refresh. +Catalog presence and provider/model compatibility remain separate from pending storage migrations. An active catalog can be `ready` and compatible while parser, symbol, or call-graph migration markers are outdated. `migrationRequired` and a noncurrent freshness reason recommend ordinary `index_codebase` with `force: false`, not index deletion. Resolve unreadable source/index access before indexing. The CLI equivalents are `ocbi status --check-freshness` and `ocbi status --check-freshness --json` (`cbi` is equivalent). JSON contains the shared status object; MCP adds the diagnostics below, while Pi/omp return the same facts as details. Workspace status remains a bounded catalog inspection with freshness explicitly not checked. MCP responses add `structuredContent.mcpDiagnostics` with schema version `1`. `activeOperations` reports each operation, its current phase, start and last-activity timestamps, and `active` or `suspected_stall` status. The current `index_status` call is excluded. `latestInterruptedOperation` is included only after an ordered shutdown marks an active call or a later local process confirms that the recorded PID is absent. A stale heartbeat alone is never classified as an interruption. @@ -319,6 +335,14 @@ Analyzes changed files, affected symbols, transitive dependencies, graph communi Branch materialization uses invocation-unique temporary worktrees. A cross-process lease scoped to the common Git directory protects registration and deregistration, including linked-worktree callers; fetches and indexing callbacks remain concurrent. Cancellation while waiting for registration removes the unused temporary checkout, and unverified cleanup preserves its path rather than deleting potentially registered data. +Set `workingTree: true` to inspect the union of staged, unstaged, and untracked path changes. First run ordinary incremental `index_codebase` after editing: this mode uses the **current indexed dirty tree**, not the old committed graph. A stale, mismatched, missing, or migration-pending index is rejected with a refresh instruction; inspection does not refresh it automatically. + +Working-tree analysis is file-level static graph reachability, not changed-line analysis, a staged-blob comparison, or a historical deletion graph. It reads local Git status with no optional locks/fsmonitor hook, preserves literal XY states and rename origins, and never fetches, invokes `gh`, creates a worktree, or stages/resets files. It is mutually exclusive with explicit `pr`, `branch`, and `checkConflicts: true`. The existing branch/PR preparation path is unchanged when the flag is false. + +MCP structured content and Pi/omp details include `workingTree.headCommit`, `scope: "current_indexed_files"`, selected `entries`, `omittedFileCount`, and `unassessed` paths/reasons. Deleted files, rename source paths, unindexed or symbol-free files, non-regular/outside-project targets, and changes beyond the 200-path cap yield `riskLevel: "UNKNOWN"` rather than a false LOW. Known graph counts remain available but are partial. A clean tree reports no working-tree changes; an empty list never implies runtime safety. + +Before/after status, HEAD, indexed changed-file hashes, freshness, and database publication fingerprints reject concurrent changes without automatic retries. These are bounded observations, not an atomic filesystem snapshot. Nested-project analysis retains only local rename endpoints. Unmerged status, invalid UTF-8 paths, malformed/overflowing Git output (1 MiB bound), and operational failures are explicit errors. The CLI equivalent is `ocbi impact --working-tree --json` (`cbi` is equivalent); see [Human CLI](installation.md#human-cli). + ## OpenCode-native tools OpenCode registers the portable tools, the three shared knowledge-base tools, and one OpenCode-only tool: diff --git a/src/adapters/cbi.ts b/src/adapters/cbi.ts index 515866ce..a374df25 100644 --- a/src/adapters/cbi.ts +++ b/src/adapters/cbi.ts @@ -3,6 +3,7 @@ import * as path from "node:path"; import { fileURLToPath } from "node:url"; import type { HostMode } from "../config/host.js"; +import type { executeIndexStatus } from "../tools/execute-common.js"; import { parseHostMode } from "../config/host.js"; import { loadConfigFile } from "../config/merger.js"; @@ -19,7 +20,7 @@ type GraphDirection = "callers" | "callees"; export interface CbiDeps { runIndex?: typeof handleIndexCommand; - runStatus?: (projectRoot: string | undefined, host: HostMode) => Promise; + runStatus?: typeof executeIndexStatus; runDefinition?: (projectRoot: string | undefined, host: HostMode, query: string) => Promise; runSearch?: (projectRoot: string | undefined, host: HostMode, query: string, limit: number) => Promise; runCallGraph?: ( @@ -43,13 +44,17 @@ export interface CbiCommandArgs { limit: number; filePath?: string; positionals: string[]; + checkFreshness: boolean; + workingTree: boolean; + json: boolean; } function printUsage(output: TextSink): void { output(`Usage: ocbi [options] Commands: - status Show index status + status [--check-freshness] [--json] Show stored status; optionally check source freshness + impact --working-tree [--json] Inspect current indexed working-tree file impact index [options] Create or refresh the index search [--limit ] Full-content semantic search definition Find authoritative definitions @@ -68,7 +73,8 @@ Global options: function printCommandUsage(output: TextSink, command: string): void { const usage: Record = { - status: "Usage: ocbi status [--project ] [--host ] [--config ]", + status: "Usage: ocbi status [--check-freshness] [--json] [--project ] [--host ] [--config ]", + impact: "Usage: ocbi impact --working-tree [--json] [--project ] [--host ] [--config ]", index: "Usage: ocbi index [--project ] [--host ] [--config ] [--force] [--estimate-only] [--dry-run] [--verbose]", search: "Usage: ocbi search [--limit ] [--project ] [--host ] [--config ]", definition: "Usage: ocbi definition [--project ] [--host ] [--config ]", @@ -93,11 +99,26 @@ export function parseCbiCommandArgs(command: string, args: string[], cwd: string let config: string | undefined; let limit = 5; let filePath: string | undefined; + let checkFreshness = false; + let workingTree = false; + let json = false; const positionals: string[] = []; for (let index = 0; index < args.length; index += 1) { const arg = args[index]; if (arg === "--help" || arg === "-h") throw new Error("help-requested"); + if (command === "status" && arg === "--check-freshness") { + checkFreshness = true; + continue; + } + if ((command === "status" || command === "impact") && arg === "--json") { + json = true; + continue; + } + if (command === "impact" && arg === "--working-tree") { + workingTree = true; + continue; + } if (arg === "--project" || arg.startsWith("--project=")) { const parsed = optionValue(args, index, "project"); @@ -134,7 +155,7 @@ export function parseCbiCommandArgs(command: string, args: string[], cwd: string positionals.push(arg); } - return { project: project ?? resolveDefaultProjectRoot(cwd), host, config, limit, filePath, positionals }; + return { project: project ?? resolveDefaultProjectRoot(cwd), host, config, limit, filePath, positionals, checkFreshness, workingTree, json }; } async function initializeFromConfig(args: CbiCommandArgs, deps: CbiDeps): Promise { @@ -158,7 +179,10 @@ const defaultSearch = async (projectRoot: string | undefined, host: HostMode, qu function requirePositionals(args: CbiCommandArgs, command: string, count: number): string[] { if (args.positionals.length !== count) { - throw new Error(`${command} requires ${count === 1 ? "exactly one argument" : "a direction and a symbol"}.`); + let expected = "a direction and a symbol"; + if (count === 0) expected = "no positional arguments"; + else if (count === 1) expected = "exactly one argument"; + throw new Error(`${command} requires ${expected}.`); } return args.positionals; } @@ -174,7 +198,7 @@ export async function runCbiCli(argv: string[], cwd: string, deps: CbiDeps = {}) return 0; } - if (!["status", "index", "search", "definition", "graph", "workspace"].includes(command)) { + if (!["status", "impact", "index", "search", "definition", "graph", "workspace"].includes(command)) { stderr(`Unknown command: ${command}`); printUsage(stderr); return 1; @@ -230,13 +254,26 @@ export async function runCbiCli(argv: string[], cwd: string, deps: CbiDeps = {}) } const args = parseCbiCommandArgs(command, argv.slice(3), cwd); + if (command === "impact") { + requirePositionals(args, command, 0); + if (!args.workingTree) throw new Error("impact requires --working-tree."); + } await initializeFromConfig(args, deps); if (command === "status") { requirePositionals(args, command, 0); - const result = await (deps.runStatus ?? executeIndexStatus)(args.project, args.host); + const result = await (deps.runStatus ?? executeIndexStatus)(args.project, args.host, { checkFreshness: args.checkFreshness }); if (result.isError) { stderr(result.text); return 1; } - stdout(result.text); + if (args.json && result.details === undefined) throw new Error("Status did not return structured details."); + stdout(args.json ? JSON.stringify(result.details, null, 2) : result.text); + return 0; + } + if (command === "impact") { + // Static imports would load the native code runtime for help/history-only CLI use. + const { getPrImpact } = await import("../tools/operations.js"); + const { formatPrImpact } = await import("../tools/format-pr-impact.js"); + const result = await getPrImpact(args.project, args.host, { workingTree: true }); + stdout(args.json ? JSON.stringify(result, null, 2) : formatPrImpact(result)); return 0; } if (command === "search") { diff --git a/src/adapters/mcp/register-tools.ts b/src/adapters/mcp/register-tools.ts index 30eb582d..7bf7bd79 100644 --- a/src/adapters/mcp/register-tools.ts +++ b/src/adapters/mcp/register-tools.ts @@ -148,10 +148,12 @@ export function registerMcpTools(server: McpServer, runtime: McpServerRuntime): .default(DEFAULT_CONTEXT_PACK_TOKEN_BUDGET)), includeApiImpact: z.boolean().optional().default(false) .describe("Include bounded syntactic Express route to exact relative fetch evidence. Matches are not call edges; tests are candidates only."), + includeTests: z.boolean().optional().default(false) + .describe("Include bounded JS/TS static source-linked test references, not test execution or runtime coverage."), }, async (args, control) => { const result = await executeCodebaseEditContext(runtime.projectRoot, runtime.host, args, control); - return { content: [{ type: "text", text: result.text }] }; + return { content: [{ type: "text", text: result.text }], structuredContent: result.details }; }, ); @@ -243,11 +245,13 @@ export function registerMcpTools(server: McpServer, runtime: McpServerRuntime): registerMcpTool(server, runtime, TOOL_NAME.INDEX_STATUS, - "Check index readiness or freshness when unknown. Reports indexed chunks, compatibility, and embedding configuration. If ready, use the tool appropriate to the task; otherwise run index_codebase.", - {}, - async (_args, control) => { - const result = await executeIndexStatus(runtime.projectRoot, runtime.host, control); - return { content: [{ type: "text", text: result.text }] }; + "Inspect stored catalog readiness, compatibility, and embedding configuration. Source freshness is not checked unless checkFreshness=true; that read-only check does not refresh the index.", + { + checkFreshness: allowNullAsUndefined(z.boolean().optional().default(false)).describe("Check whether selected source matches the index at this point in time without indexing"), + }, + async (args, control) => { + const result = await executeIndexStatus(runtime.projectRoot, runtime.host, args, control); + return { content: [{ type: "text", text: result.text }], structuredContent: result.details }; }, ); @@ -378,10 +382,11 @@ export function registerMcpTools(server: McpServer, runtime: McpServerRuntime): ); registerMcpTool(server, runtime, TOOL_NAME.PR_IMPACT, - "FIRST TOOL for pull-request or branch blast-radius questions. Analyzes changed files, affected symbols, transitive dependencies, communities, hub nodes, conflicts, and risk before merging.", + "FIRST TOOL for pull-request, branch, or working-tree blast-radius questions. Working-tree mode inspects current indexed files across staged, unstaged, and untracked changes without refreshing the index.", { pr: allowNullAsUndefined(z.number().optional()).describe("Pull request number to analyze"), branch: allowNullAsUndefined(z.string().optional()).describe("Branch name to analyze (defaults to current branch)"), + workingTree: allowNullAsUndefined(z.boolean().optional().default(false)).describe("Inspect staged, unstaged, and untracked paths using current indexed file contents; requires a current index and cannot combine with pr, branch, or checkConflicts"), maxDepth: allowNullAsUndefined(z.number().optional().default(5)).describe("Maximum traversal depth for transitive callers (default: 5)"), hubThreshold: allowNullAsUndefined(z.number().optional().default(10)).describe("Minimum caller count to flag a symbol as a hub node (default: 10)"), checkConflicts: allowNullAsUndefined(z.boolean().optional().default(false)).describe("Check for conflicting open PRs touching the same communities (default: false)"), @@ -393,12 +398,13 @@ export function registerMcpTools(server: McpServer, runtime: McpServerRuntime): const result = await getPrImpact(runtime.projectRoot, runtime.host, { pr: args.pr, branch: args.branch, + workingTree: args.workingTree, maxDepth: args.maxDepth, hubThreshold: args.hubThreshold, checkConflicts: args.checkConflicts, direction: args.direction, }, control); - return { content: [{ type: "text", text: formatPrImpact(result) }] }; + return { content: [{ type: "text", text: formatPrImpact(result) }], structuredContent: { ...result } }; }, ); diff --git a/src/adapters/mcp/server.ts b/src/adapters/mcp/server.ts index b7e06978..3a00bda3 100644 --- a/src/adapters/mcp/server.ts +++ b/src/adapters/mcp/server.ts @@ -61,7 +61,7 @@ async function releaseMcpBackgroundWorker(projectRoot: string, host: HostMode): function getServerInstructions(host: string): string { const hostText = `host ${host}`; - return `This MCP server is the preferred codebase-understanding path for ${hostText}. Check index_status when index readiness or freshness is unknown. Use codebase_context only when repository orientation is needed (layout, relevant symbols, or cross-file intent), not mechanically before every task or search. Request a compact first pass (tokenBudget: 600, limit: 5) and inspect returned evidence before broad reads or searches. Use implementation_lookup directly for known definitions and call_graph or call_graph_path directly for relationships once endpoints are identified. For a change with a known or suspected symbol, optionally use codebase_edit_context for bounded target source plus direct callers and callees. Use codebase_peek for metadata-only semantic locations, codebase_search when matching source content is needed, targeted file reads for known paths, and grep for exact or exhaustive literal matches. Do not repeat retrieval or broad reads when existing evidence already answers the question.`; + return `This MCP server is the preferred codebase-understanding path for ${hostText}. Check index_status when index readiness is unknown; pass checkFreshness:true only when a point-in-time source freshness check is needed. Ordinary status reports stored catalog facts without checking source freshness. Use codebase_context only when repository orientation is needed (layout, relevant symbols, or cross-file intent), not mechanically before every task or search. Request a compact first pass (tokenBudget: 600, limit: 5) and inspect returned evidence before broad reads or searches. Use implementation_lookup directly for known definitions and call_graph or call_graph_path directly for relationships once endpoints are identified. For a change with a known or suspected symbol, optionally use codebase_edit_context for bounded target source plus direct callers and callees. Use codebase_peek for metadata-only semantic locations, codebase_search when matching source content is needed, targeted file reads for known paths, and grep for exact or exhaustive literal matches. Do not repeat retrieval or broad reads when existing evidence already answers the question.`; } interface McpBackgroundWorkerConfiguration { diff --git a/src/adapters/opencode/pr-impact.ts b/src/adapters/opencode/pr-impact.ts index 9e2de8e0..214d3d3c 100644 --- a/src/adapters/opencode/pr-impact.ts +++ b/src/adapters/opencode/pr-impact.ts @@ -2,6 +2,7 @@ import type { PrImpactResult } from "../../indexer/pr-impact-types.js"; import { tool, type ToolDefinition } from "@opencode-ai/plugin"; import { getIndexerForProject } from "./tools.js"; import { formatPrImpact } from "../../tools/format-pr-impact.js"; +import { getPrImpact } from "../../tools/operations.js"; import { calculatePercentage, formatProgressTitle } from "../../tools/utils.js"; import { isOperationInterruption, throwIfOperationAborted } from "../../utils/operation-control.js"; @@ -12,12 +13,13 @@ const AUTOMATIC_PREPARATION_POLICY = export const pr_impact: ToolDefinition = tool({ description: - "Analyze the impact of a pull request or branch by examining changed files, " + - "affected symbols, transitive callers, communities touched, hub nodes, and risk level. " + - "Use to understand blast radius before merging.", + "Analyze a pull request, branch, or working tree through changed files, " + + "current indexed symbols, transitive callers, communities, hubs, and risk. " + + "Working-tree mode inspects staged, unstaged, and untracked paths without refreshing the index.", args: { pr: z.number().optional().describe("Pull request number to analyze"), branch: z.string().optional().describe("Branch name to analyze (defaults to current branch)"), + workingTree: z.boolean().optional().default(false).describe("Inspect current indexed file contents for staged, unstaged, and untracked paths; cannot combine with pr, branch, or checkConflicts"), maxDepth: z.number().optional().default(5).describe("Maximum traversal depth for transitive callers (default: 5)"), hubThreshold: z.number().optional().default(10).describe("Minimum caller count to flag a symbol as a hub node (default: 10)"), checkConflicts: z.boolean().optional().default(false).describe("Check for conflicting open PRs touching the same communities (default: false)"), @@ -25,8 +27,20 @@ export const pr_impact: ToolDefinition = tool({ }, async execute(args, context) { throwIfOperationAborted(context?.abort); - const indexer = getIndexerForProject(context?.worktree); try { + if (args.workingTree) { + const result = await getPrImpact(context?.worktree, "opencode", args, { + signal: context?.abort, + setPhase: (phase) => { + context.metadata?.({ + title: "Inspecting current indexed working-tree files", + metadata: { phase }, + }); + }, + }); + return formatPrImpact(result); + } + const indexer = getIndexerForProject(context?.worktree); context.metadata?.({ title: "Preparing impact analysis: resolving authoritative branch head", metadata: { diff --git a/src/adapters/opencode/tools.ts b/src/adapters/opencode/tools.ts index 5560cd38..e421a085 100644 --- a/src/adapters/opencode/tools.ts +++ b/src/adapters/opencode/tools.ts @@ -155,6 +155,8 @@ export const codebase_edit_context: ToolDefinition = tool({ .nullable().optional().default(DEFAULT_CONTEXT_PACK_TOKEN_BUDGET), includeApiImpact: z.boolean().optional().default(false) .describe("Include bounded syntactic Express route to exact relative fetch evidence. Matches are not call edges; tests are candidates only."), + includeTests: z.boolean().optional().default(false) + .describe("Include bounded JS/TS static source-linked test references, not test execution or runtime coverage."), }, async execute(args, context) { return (await executeCodebaseEditContext(context?.worktree, DEFAULT_HOST, args, { signal: context?.abort })).text; @@ -211,10 +213,12 @@ export const index_codebase: ToolDefinition = tool({ export const index_status: ToolDefinition = tool({ description: - "Check index readiness or freshness when unknown. Reports indexed chunks, compatibility, and embedding configuration. A ready index does not require another indexing call before retrieval.", - args: {}, - async execute(_args, context) { - return (await executeIndexStatus(context?.worktree, DEFAULT_HOST, { signal: context?.abort })).text; + "Inspect stored catalog readiness, compatibility, and embedding configuration. Source freshness is not checked unless checkFreshness=true; that read-only check does not refresh the index.", + args: { + checkFreshness: z.boolean().optional().default(false).describe("Check whether selected source matches the index at this point in time without indexing"), + }, + async execute(args, context) { + return (await executeIndexStatus(context?.worktree, DEFAULT_HOST, args, { signal: context?.abort })).text; }, }); diff --git a/src/adapters/shared/extension-core.ts b/src/adapters/shared/extension-core.ts index 5e0568d2..77302f14 100644 --- a/src/adapters/shared/extension-core.ts +++ b/src/adapters/shared/extension-core.ts @@ -229,10 +229,14 @@ export function registerCodebaseIndexTools(pi: CodebaseIndexExtensionAPI, schema default: false, description: "Include bounded syntactic Express route to exact relative fetch evidence. Matches are not call edges; tests are candidates only.", })), + includeTests: schema.Optional(schema.Boolean({ + default: false, + description: "Include bounded JS/TS static source-linked test references, not test execution or runtime coverage.", + })), }), async execute(_toolCallId, params, signal, _onUpdate, ctx) { const result = await resolveCodebaseEditContext(projectRoot(ctx), HOST, params, { signal }); - return text(result.text); + return text(result.text, result.details); }, }); @@ -338,7 +342,7 @@ export function registerCodebaseIndexTools(pi: CodebaseIndexExtensionAPI, schema pi.registerTool({ name: TOOL_NAME.INDEX_CODEBASE, label: "Index Codebase", - description: "Build or refresh the semantic codebase index. Run index_status when freshness is unknown.", + description: "Build or refresh the semantic codebase index. Use index_status with checkFreshness=true when source freshness is needed.", parameters: schema.Object({ force: schema.Optional(schema.Boolean({ default: false })), estimateOnly: schema.Optional(schema.Boolean({ default: false })), @@ -371,10 +375,12 @@ export function registerCodebaseIndexTools(pi: CodebaseIndexExtensionAPI, schema pi.registerTool({ name: TOOL_NAME.INDEX_STATUS, label: "Index Status", - description: "Check index health and current status.", - parameters: schema.Object({}), - async execute(_toolCallId, _params, signal, _onUpdate, ctx) { - const status = await getIndexStatus(projectRoot(ctx), HOST, { signal }); + description: "Inspect stored catalog readiness and compatibility. Source freshness is not checked unless checkFreshness=true; that read-only check does not refresh the index.", + parameters: schema.Object({ + checkFreshness: schema.Optional(schema.Boolean({ default: false, description: "Check selected source against the index at this point in time without indexing" })), + }), + async execute(_toolCallId, params, signal, _onUpdate, ctx) { + const status = await getIndexStatus(projectRoot(ctx), HOST, params, { signal }); return text(formatStatus(status), status); }, }); @@ -436,10 +442,11 @@ export function registerCodebaseIndexTools(pi: CodebaseIndexExtensionAPI, schema pi.registerTool({ name: TOOL_NAME.PR_IMPACT, label: "PR Impact", - description: "Analyze PR or branch impact through changed symbols and call graph neighborhoods.", + description: "Analyze PR, branch, or working-tree impact through current indexed files and call graph neighborhoods. Working-tree mode includes staged, unstaged, and untracked paths without refreshing the index.", parameters: schema.Object({ pr: schema.Optional(schema.Number()), branch: schema.Optional(schema.String()), + workingTree: schema.Optional(schema.Boolean({ default: false })), maxDepth: schema.Optional(schema.Number({ default: 5 })), hubThreshold: schema.Optional(schema.Number({ default: 10 })), checkConflicts: schema.Optional(schema.Boolean({ default: false })), diff --git a/src/eval/runner.ts b/src/eval/runner.ts index 37cc199b..d0fb8f40 100644 --- a/src/eval/runner.ts +++ b/src/eval/runner.ts @@ -10,6 +10,7 @@ import { resolveCodebaseEditContextWithDependencies, type CodebaseEditContextEvi import { getCallGraphDataForIndexer, getArchitectureContextForIndexer, + implementationLookupForIndexer, type CallGraphDataResult, } from "../tools/operations.js"; import { DEFAULT_CONTEXT_PACK_TOKEN_BUDGET } from "../tools/utils.js"; @@ -220,7 +221,7 @@ async function runEditContextQuery( omittedCount: number; }; }> { - let selectedEvidence: CodebaseEditContextEvidence = { sources: [], callers: [], callees: [] }; + let selectedEvidence: CodebaseEditContextEvidence = { sources: [], callers: [], callees: [], testReferences: [] }; let callers: CallGraphDataResult | undefined; const editContext = await resolveCodebaseEditContextWithDependencies({ @@ -236,10 +237,8 @@ async function runEditContextQuery( searchCodebase: (searchQuery, options) => indexer.search(searchQuery, options?.limit, { filterByBranch: !!query.expected.branch, }), - implementationLookup: (symbol, options) => indexer.search(symbol, options?.limit, { - filterByBranch: !!query.expected.branch, - definitionIntent: true, - }), + implementationLookup: (symbol, options, control) => + implementationLookupForIndexer(indexer, projectRoot, symbol, options, control), getCallGraphData: async (params) => { const result = await getCallGraphDataForIndexer(indexer, projectRoot, params); if (params.direction === "callers") callers = result; diff --git a/src/indexer/index.ts b/src/indexer/index.ts index a1381005..c924b48f 100644 --- a/src/indexer/index.ts +++ b/src/indexer/index.ts @@ -1,3 +1,4 @@ +import type { Stats } from "fs"; import { existsSync, readFileSync, statSync, writeFileSync, renameSync, unlinkSync, mkdirSync, promises as fsPromises } from "fs"; import * as path from "path"; import { performance } from "perf_hooks"; @@ -38,7 +39,8 @@ import { getBranchOrDefault, getBaseBranch, isGitRepo } from "../git/index.js"; import { isFullGitCommit, resolveLocalGitCommit, withMaterializedBranch } from "../git/branch-materialization.js"; import type { HostMode } from "../config/host.js"; import { isProjectIndexPathOwnedByProject, resolveProjectIndexPath } from "../config/paths.js"; -import { getChangedFiles } from "../tools/changed-files.js"; +import { getChangedFiles, getWorkingTreeChanges } from "../tools/changed-files.js"; +import type { SharedPrImpactArgs } from "../tools/contracts.js"; import type { PrImpactResult } from "./pr-impact-types.js"; import { UnsupportedIndexOperationError } from "./errors.js"; export { UnsupportedIndexOperationError } from "./errors.js"; @@ -8184,134 +8186,210 @@ export class Indexer { return result; } - async getPrImpact(opts: { - pr?: number; - branch?: string; - maxDepth?: number; - hubThreshold?: number; - checkConflicts?: boolean; - direction?: "callers" | "callees" | "both"; - }, onPreparationProgress?: ProgressCallback, options: IndexOperationOptions = {}): Promise { + async getPrImpact(opts: SharedPrImpactArgs, onPreparationProgress?: ProgressCallback, options: IndexOperationOptions = {}): Promise { + if (opts.workingTree && (opts.pr !== undefined || opts.branch !== undefined || opts.checkConflicts === true)) { + throw new Error("workingTree cannot be combined with pr, branch, or checkConflicts"); + } throwIfOperationAborted(options.signal); const initialState = await this.ensureInitialized(); throwIfOperationAborted(options.signal); let database = initialState.database; const { readIssues } = initialState; - this.requireReadableComponents(readIssues, "database"); + if (!opts.workingTree) this.requireReadableComponents(readIssues, "database"); const execFileAsync = promisify(execFile); - const changedFilesResult = await getChangedFiles({ - pr: opts.pr, - branch: opts.branch, - projectRoot: this.projectRoot, - baseBranch: this.baseBranch, - signal: options.signal, - }); - const changedFiles = changedFilesResult.files; - const headRefName = changedFilesResult.headRefName; - const expectedCommit = changedFilesResult.headRef; - throwIfOperationAborted(options.signal); - - if (opts.pr !== undefined && headRefName === undefined) { - throw new Error( - `Could not resolve head branch for PR #${opts.pr}. Run index_codebase on the PR branch first.`, - ); - } - if (!expectedCommit || !isFullGitCommit(expectedCommit)) { - throw new Error("Could not resolve an authoritative full commit OID for impact analysis."); - } - - const resolvedBranch = opts.pr !== undefined - ? headRefName - : opts.branch || this.currentBranch; - const catalogIdentity = changedFilesResult.catalogIdentity; - const branchKey = this.getBranchCatalogKeyFor(catalogIdentity); - - let branchSymbols = database.getSymbolsForBranch(branchKey); - let indexPreparation: NonNullable = { - prepared: false, - branch: resolvedBranch || "default", - }; - const requestedRef = opts.pr !== undefined - ? expectedCommit - : headRefName ?? resolvedBranch; - const storedCommit = this.getStoredBranchCommit(database, catalogIdentity); - const catalogIdentityMatches = storedCommit === expectedCommit; - - const migrationsCurrent = this.areBranchMigrationVersionsCurrent(database, catalogIdentity); - - if (branchSymbols.length === 0 || !catalogIdentityMatches || !migrationsCurrent) { - if (!resolvedBranch || resolvedBranch === "default") { - throw new Error("Run index_codebase first to build the call graph and symbol index for this project."); - } - - onPreparationProgress?.({ - phase: "scanning", - filesProcessed: 0, - totalFiles: 0, - chunksProcessed: 0, - totalChunks: 0, - }); - this.resetLoadedIndexState(); - throwIfOperationAborted(options.signal); - const materialized = await withMaterializedBranch( - { - projectRoot: this.projectRoot, - branch: resolvedBranch, - ref: requestedRef, - expectedCommit, - pr: opts.pr, - repository: changedFilesResult.baseRepository, - signal: options.signal, - }, - async (worktreePath, info) => { - const branchIndexer = new Indexer(this.projectRoot, this.config, this.host, { - materializedProjectRoot: worktreePath, - branchName: resolvedBranch, - catalogIdentity, - expectedCommit, - indexPath: this.indexPath, - }); + let changedFiles: string[]; + let analysisFiles: string[]; + let branchKey: string; + let indexPreparation: PrImpactResult["indexPreparation"]; + let workingTree: PrImpactResult["workingTree"]; + let checkWorkingTreeConsistency: (() => Promise) | undefined; + + if (opts.workingTree) { + const currentIndexError = "Working-tree impact requires a current index. Run index_codebase, then retry."; + const changedDuringAnalysis = "Working tree changed during impact analysis; retry on a stable tree."; + const graphFingerprint = this.captureReaderArtifactFingerprint().database; + const status = await this.getStatus(); + if (!status.indexed || status.checkoutBranch === undefined + || status.checkoutBranch !== status.currentBranch + || status.branchReadiness?.state !== "ready" || status.migrationRequired !== false) { + throw new Error(currentIndexError); + } + const changes = await getWorkingTreeChanges(this.projectRoot, options.signal); + if (!(await this.getIndexFreshness(options)).current) throw new Error(currentIndexError); + const canonicalRoot = await fsPromises.realpath(this.projectRoot); + const captureFiles = async () => { + const indexedFiles = new Set((await this.getIndexedFilePathsForActiveBranch()).map((file) => path.resolve(file))); + const files: string[] = []; + const hashes: Array<[string, string]> = []; + const unassessed: NonNullable["unassessed"] = []; + for (const entry of changes.entries) { + throwIfOperationAborted(options.signal); + if (entry.previousPath && (entry.indexStatus === "R" || entry.worktreeStatus === "R")) { + unassessed.push({ path: entry.previousPath, reason: "renamed_from" }); + } + const file = path.resolve(this.projectRoot, entry.path); + if (!isPathWithinRoot(file, this.projectRoot)) { + unassessed.push({ path: entry.path, reason: "outside_project" }); + continue; + } + let stats: Stats; + let realFile: string; try { - return await branchIndexer.indexBranchIfMissing( - resolvedBranch, - info.commit, - onPreparationProgress, - options, - ); - } finally { - await branchIndexer.close(); + // Resolve containment before hashing, including symlinked parent directories. + stats = await fsPromises.lstat(file); + realFile = await fsPromises.realpath(file); + } catch (error) { + if (error instanceof Error && "code" in error && (error.code === "ENOENT" || error.code === "ENOTDIR")) { + unassessed.push({ path: entry.path, reason: entry.indexStatus === "D" || entry.worktreeStatus === "D" ? "deleted" : "not_regular" }); + continue; + } + throw error; } - }, - ); - - indexPreparation = { - prepared: materialized.value.prepared, - branch: resolvedBranch, - commit: materialized.info.commit, - source: materialized.info.source, + if (!isPathWithinRoot(realFile, canonicalRoot)) { + unassessed.push({ path: entry.path, reason: "outside_project" }); + } else if (!stats.isFile()) { + unassessed.push({ path: entry.path, reason: "not_regular" }); + } else if (!indexedFiles.has(file)) { + unassessed.push({ path: entry.path, reason: "not_indexed" }); + } else { + files.push(entry.path); + hashes.push([entry.path, hashFile(file)]); + } + await options.heartbeat?.(); + } + throwIfOperationAborted(options.signal); + return { files, hashes, unassessed }; }; - - const refreshedState = await this.ensureInitialized(); + const before = await captureFiles(); + changedFiles = changes.entries.map((entry) => entry.path); + analysisFiles = before.files; + branchKey = this.getBranchCatalogKey(); + workingTree = { ...changes, scope: "current_indexed_files", unassessed: [...before.unassessed] }; + checkWorkingTreeConsistency = async () => { + // Bounded before/after observations, not an atomic filesystem snapshot. + try { + const afterChanges = await getWorkingTreeChanges(this.projectRoot, options.signal); + const afterStatus = await this.getStatus(); + const freshness = await this.getIndexFreshness(options); + const afterFiles = await captureFiles(); + if (!freshness.current + || graphFingerprint !== this.captureReaderArtifactFingerprint().database + || JSON.stringify(status) !== JSON.stringify(afterStatus) + || JSON.stringify(changes) !== JSON.stringify(afterChanges) + || JSON.stringify(before) !== JSON.stringify(afterFiles)) { + throw new Error(changedDuringAnalysis); + } + } catch (error) { + if (isOperationInterruption(error)) throw error; + throwIfOperationAborted(options.signal); + throw new Error(changedDuringAnalysis, { cause: error }); + } + }; + } else { + const changedFilesResult = await getChangedFiles({ + pr: opts.pr, + branch: opts.branch, + projectRoot: this.projectRoot, + baseBranch: this.baseBranch, + signal: options.signal, + }); + changedFiles = changedFilesResult.files; + analysisFiles = changedFiles; + const headRefName = changedFilesResult.headRefName; + const expectedCommit = changedFilesResult.headRef; throwIfOperationAborted(options.signal); - this.requireReadableComponents(refreshedState.readIssues, "database"); - database = refreshedState.database; - branchSymbols = database.getSymbolsForBranch(branchKey); - if (branchSymbols.length === 0) { + + if (opts.pr !== undefined && headRefName === undefined) { throw new Error( - `Branch ${JSON.stringify(resolvedBranch)} (catalog ${JSON.stringify(catalogIdentity)}) was indexed but produced no call-graph symbols. ` - + `Available branch catalogs: ${database.getAllBranches().join(", ") || "none"}; ` - + `${database.getBranchChunkIds(branchKey).length} chunks, ${database.getBranchSymbolIds(branchKey).length} symbol IDs. ` - + "Ensure the branch contains a supported source language and is included by the index configuration.", + `Could not resolve head branch for PR #${opts.pr}. Run index_codebase on the PR branch first.`, ); } + if (!expectedCommit || !isFullGitCommit(expectedCommit)) { + throw new Error("Could not resolve an authoritative full commit OID for impact analysis."); + } + + const resolvedBranch = opts.pr !== undefined + ? headRefName + : opts.branch || this.currentBranch; + const catalogIdentity = changedFilesResult.catalogIdentity; + branchKey = this.getBranchCatalogKeyFor(catalogIdentity); + let branchSymbols = database.getSymbolsForBranch(branchKey); + indexPreparation = { prepared: false, branch: resolvedBranch || "default" }; + const requestedRef = opts.pr !== undefined ? expectedCommit : headRefName ?? resolvedBranch; + const storedCommit = this.getStoredBranchCommit(database, catalogIdentity); + const catalogIdentityMatches = storedCommit === expectedCommit; + const migrationsCurrent = this.areBranchMigrationVersionsCurrent(database, catalogIdentity); + + if (branchSymbols.length === 0 || !catalogIdentityMatches || !migrationsCurrent) { + if (!resolvedBranch || resolvedBranch === "default") { + throw new Error("Run index_codebase first to build the call graph and symbol index for this project."); + } + onPreparationProgress?.({ + phase: "scanning", filesProcessed: 0, totalFiles: 0, chunksProcessed: 0, totalChunks: 0, + }); + this.resetLoadedIndexState(); + throwIfOperationAborted(options.signal); + const materialized = await withMaterializedBranch( + { + projectRoot: this.projectRoot, + branch: resolvedBranch, + ref: requestedRef, + expectedCommit, + pr: opts.pr, + repository: changedFilesResult.baseRepository, + signal: options.signal, + }, + async (worktreePath, info) => { + const branchIndexer = new Indexer(this.projectRoot, this.config, this.host, { + materializedProjectRoot: worktreePath, + branchName: resolvedBranch, + catalogIdentity, + expectedCommit, + indexPath: this.indexPath, + }); + try { + return await branchIndexer.indexBranchIfMissing(resolvedBranch, info.commit, onPreparationProgress, options); + } finally { + await branchIndexer.close(); + } + }, + ); + indexPreparation = { + prepared: materialized.value.prepared, + branch: resolvedBranch, + commit: materialized.info.commit, + source: materialized.info.source, + }; + const refreshedState = await this.ensureInitialized(); + throwIfOperationAborted(options.signal); + this.requireReadableComponents(refreshedState.readIssues, "database"); + database = refreshedState.database; + branchSymbols = database.getSymbolsForBranch(branchKey); + if (branchSymbols.length === 0) { + throw new Error( + `Branch ${JSON.stringify(resolvedBranch)} (catalog ${JSON.stringify(catalogIdentity)}) was indexed but produced no call-graph symbols. ` + + `Available branch catalogs: ${database.getAllBranches().join(", ") || "none"}; ` + + `${database.getBranchChunkIds(branchKey).length} chunks, ${database.getBranchSymbolIds(branchKey).length} symbol IDs. ` + + "Ensure the branch contains a supported source language and is included by the index configuration.", + ); + } + } } const toStoredChangedFiles = (filePaths: readonly string[]): string[] => filePaths.map((filePath) => this.toStoredFilePath(path.resolve(this.projectRoot, filePath))); - const storedChangedFiles = toStoredChangedFiles(changedFiles); + const storedChangedFiles = toStoredChangedFiles(analysisFiles); throwIfOperationAborted(options.signal); const directSymbols = database.getSymbolsForFiles(storedChangedFiles, branchKey); + if (workingTree) { + const symbolFiles = new Set(directSymbols.map((symbol) => symbol.filePath)); + for (const file of analysisFiles) { + if (!symbolFiles.has(this.toStoredFilePath(path.resolve(this.projectRoot, file)))) { + workingTree.unassessed.push({ path: file, reason: "no_symbols" }); + } + } + } throwIfOperationAborted(options.signal); const directIds = directSymbols.map((s) => s.id); @@ -8382,7 +8460,7 @@ export class Indexer { } const totalAffected = allAffectedIds.length; - let riskLevel: "LOW" | "MEDIUM" | "HIGH"; + let riskLevel: PrImpactResult["riskLevel"]; let riskReason: string; if (totalAffected < 5 && hubNodes.length === 0) { @@ -8395,6 +8473,15 @@ export class Indexer { riskLevel = "MEDIUM"; riskReason = `Moderate impact: ${totalAffected} affected symbols${hubNodes.length === 1 ? ", 1 hub node touched" : ""}.`; } + if (workingTree) { + if (workingTree.unassessed.length > 0 || workingTree.omittedFileCount > 0) { + riskLevel = "UNKNOWN"; + riskReason = `Partial current-graph impact: ${totalAffected} known affected symbols; unassessed or omitted files remain.`; + } else if (changedFiles.length === 0) { + riskReason = "No working-tree changes."; + } + riskReason += " Static file-level analysis of current indexed contents, not staged blobs or historical deletions."; + } let conflictingPRs: PrImpactResult["conflictingPRs"]; if (opts.checkConflicts) { @@ -8475,9 +8562,12 @@ export class Indexer { } } + throwIfOperationAborted(options.signal); + await checkWorkingTreeConsistency?.(); throwIfOperationAborted(options.signal); return { indexPreparation, + ...(workingTree ? { workingTree } : {}), changedFiles, directSymbols: directSymbols.map((s) => ({ id: s.id, @@ -8511,9 +8601,9 @@ export class Indexer { edges: CallEdgeData[]; }> { throwIfOperationAborted(options.signal); - const { database, store, readIssues } = await this.ensureInitialized(); + const { database, readIssues } = await this.ensureInitialized(); throwIfOperationAborted(options.signal); - this.requireReadableComponents(readIssues, "vectors", "database"); + this.requireReadableComponents(readIssues, "database"); const seenSymbols = new Map(); const seenEdges = new Map(); @@ -8523,14 +8613,7 @@ export class Indexer { const symbolIds = database.getBranchSymbolIds(branchKey); const symbolIdSet = new Set(symbolIds); - // Get unique file paths from branch chunks' metadata - const chunkIds = database.getBranchChunkIds(branchKey); - const metadataMap = chunkIds.length > 0 ? requireLoadedCapability(store, "vector store").getMetadataBatch(chunkIds) : new Map(); - const filePaths = new Set(); - for (const [, meta] of metadataMap) { - throwIfOperationAborted(options.signal); - if (meta.filePath) filePaths.add(meta.filePath); - } + const filePaths = database.getBranchFilePaths(branchKey); const directory = options?.directory?.replace(/\/$/, ""); const absoluteDirectoryFilter = directory diff --git a/src/indexer/pr-impact-types.ts b/src/indexer/pr-impact-types.ts index 65659372..ff5423f2 100644 --- a/src/indexer/pr-impact-types.ts +++ b/src/indexer/pr-impact-types.ts @@ -1,3 +1,5 @@ +import type { WorkingTreeChanges } from "../tools/changed-files.js"; + export interface PrImpactResult { indexPreparation?: { prepared: boolean; @@ -5,6 +7,16 @@ export interface PrImpactResult { commit?: string; source?: "local" | "remote-fetch" | "pull-ref" | "gh"; }; + workingTree?: { + headCommit: string | null; + scope: "current_indexed_files"; + entries: WorkingTreeChanges["entries"]; + omittedFileCount: number; + unassessed: Array<{ + path: string; + reason: "deleted" | "renamed_from" | "not_indexed" | "no_symbols" | "outside_project" | "not_regular"; + }>; + }; changedFiles: string[]; directSymbols: Array<{ id: string; @@ -30,7 +42,7 @@ export interface PrImpactResult { callerCount: number; filePath: string; }>; - riskLevel: "LOW" | "MEDIUM" | "HIGH"; + riskLevel: "LOW" | "MEDIUM" | "HIGH" | "UNKNOWN"; riskReason: string; conflictingPRs?: Array<{ pr: number; diff --git a/src/tools/changed-files.ts b/src/tools/changed-files.ts index b1d70fe3..34369462 100644 --- a/src/tools/changed-files.ts +++ b/src/tools/changed-files.ts @@ -7,6 +7,7 @@ import { assertValidGitRefName, isFullGitCommit, resolveGitCommit, + resolveLocalGitCommit, resolveLocalPullRequestRefs, } from "../git/branch-materialization.js"; import { throwIfOperationAborted } from "../utils/operation-control.js"; @@ -28,6 +29,96 @@ function getErrorMessage(error: unknown): string { return String(error); } +export interface WorkingTreeChanges { + headCommit: string | null; + entries: Array<{ + path: string; + previousPath?: string; + indexStatus: string; + worktreeStatus: string; + }>; + omittedFileCount: number; +} + +/** Local path changes only; this does not read staged blobs or refresh the index. */ +export async function getWorkingTreeChanges( + projectRoot: string, + signal?: AbortSignal, +): Promise { + throwIfOperationAborted(signal); + const decodePaths = (output: Buffer): string => { + try { + return new TextDecoder("utf-8", { fatal: true, ignoreBOM: true }).decode(output); + } catch (error) { + throw new Error("Working-tree paths must be valid UTF-8; impact analysis cannot represent these filenames safely.", { cause: error }); + } + }; + const { stdout: rootBytes } = await execFileAsync("git", [ + "--no-optional-locks", "-c", "core.fsmonitor=false", "rev-parse", "--show-toplevel", + ], { cwd: projectRoot, timeout: 30000, maxBuffer: 1024 * 1024, encoding: "buffer", signal }); + throwIfOperationAborted(signal); + // Remove Git's terminating newline only; directory names may contain whitespace. + const rootOutput = decodePaths(rootBytes); + const gitRoot = rootOutput.endsWith("\n") ? rootOutput.slice(0, -1) : rootOutput; + if (!path.isAbsolute(gitRoot)) throw new Error("Could not resolve the working-tree root."); + const canonicalProjectRoot = realpathSync.native(projectRoot); + const { stdout: statusBytes } = await execFileAsync("git", [ + "--no-optional-locks", "-c", "core.fsmonitor=false", "status", + "--porcelain=v1", "-z", "--untracked-files=all", "--", ".", + ], { cwd: projectRoot, timeout: 30000, maxBuffer: 1024 * 1024, encoding: "buffer", signal }); + throwIfOperationAborted(signal); + const records = decodePaths(statusBytes).split("\0"); + if (records.pop() !== "") throw new Error("Malformed working-tree status output."); + const entries: WorkingTreeChanges["entries"] = []; + const unmerged: Record = { DD: true, AU: true, UD: true, UA: true, DU: true, AA: true, UU: true }; + const validStatus = /^[ MTADRCU?!]{2}$/u; + const normalized = (filePath: string): string | undefined => { + if (!filePath || path.isAbsolute(filePath)) throw new Error("Malformed working-tree status path."); + const repositoryPath = normalizeFiles([filePath], gitRoot)[0]; + const relative = path.relative(canonicalProjectRoot, path.resolve(gitRoot, repositoryPath)); + if (relative === ".." || relative.startsWith(`..${path.sep}`) || path.isAbsolute(relative)) return undefined; + return relative.split(path.sep).join("/"); + }; + for (let index = 0; index < records.length; index++) { + throwIfOperationAborted(signal); + const record = records[index]; + const status = record.slice(0, 2); + if (record[2] !== " " || !validStatus.test(status)) { + throw new Error("Malformed working-tree status record."); + } + const destination = normalized(record.slice(3)); + let previousPath: string | undefined; + if (status.includes("R") || status.includes("C")) { + const previousRecord = records[++index]; + if (previousRecord === undefined) throw new Error("Malformed working-tree rename record."); + previousPath = normalized(previousRecord); + } + if (destination === undefined) { + if (previousPath !== undefined && status.includes("R")) { + // Only the deletion belongs to this project; do not expose the sibling destination. + entries.push({ + path: previousPath, + indexStatus: status[0] === "R" ? "D" : " ", + worktreeStatus: status[1] === "R" ? "D" : " ", + }); + } + continue; + } + if (unmerged[status]) throw new Error("Resolve unmerged files before analyzing working-tree impact."); + const crossesBoundary = previousPath === undefined && (status.includes("R") || status.includes("C")); + entries.push({ + path: destination, + ...(previousPath === undefined ? {} : { previousPath }), + indexStatus: crossesBoundary && (status[0] === "R" || status[0] === "C") ? "A" : status[0], + worktreeStatus: crossesBoundary && (status[1] === "R" || status[1] === "C") ? "A" : status[1], + }); + } + entries.sort((left, right) => left.path < right.path ? -1 : left.path > right.path ? 1 : 0); + const headCommit = await resolveLocalGitCommit(projectRoot, "HEAD", signal); + throwIfOperationAborted(signal); + return { headCommit, entries: entries.slice(0, 200), omittedFileCount: Math.max(0, entries.length - 200) }; +} + export interface ChangedFilesResult { files: string[]; baseBranch: string; diff --git a/src/tools/context.ts b/src/tools/context.ts index 9a2f5f84..69f74752 100644 --- a/src/tools/context.ts +++ b/src/tools/context.ts @@ -2,14 +2,11 @@ import type { HostMode } from "../config/host.js"; import type { OperationControl } from "../utils/operation-control.js"; import * as path from "node:path"; import { canonicalizePathForComparison } from "../utils/canonical-path.js"; -import { throwIfOperationAborted } from "../utils/operation-control.js"; -import { getProjectRoot } from "./operation-runtime.js"; -import { discoverSymbolTests, TEST_DISCOVERY_MAX_DEFINITIONS, TEST_DISCOVERY_MAX_FILES } from "./test-discovery.js"; import { loadRuntimeConfig } from "./config-state.js"; import { getCallGraphData, + getSymbolTestEvidence, getCallGraphPath, - getIndexerForProject, implementationLookup, isToolEffectivenessEnabled, recordToolEffectiveness, @@ -168,24 +165,7 @@ async function resolveCodebaseContextUnmeasured( ? searchCodebase(projectRoot, host, queryText, options, control) : searchCodebase(projectRoot, host, queryText, options); }, - discoverTests: (testSymbol, scope) => { - const root = getProjectRoot(projectRoot, host); - return discoverSymbolTests(root, testSymbol, scope, { - lookup: (name) => implementationLookup(root, host, name, { exactSymbol: true, limit: TEST_DISCOVERY_MAX_DEFINITIONS }, control), - graph: (name, filePath) => getCallGraphData(root, host, { name, filePath, direction: "callers" }, control), - indexedFiles: async () => { - throwIfOperationAborted(control?.signal); - const files = await getIndexerForProject(root, host).getIndexedFilePathsForActiveBranch(); - throwIfOperationAborted(control?.signal); - return files; - }, - lexicalFiles: (identifiers, allowedFiles) => getIndexerForProject(root, host).getKeywordFileCandidates( - identifiers, allowedFiles.map((filePath) => path.resolve(root, filePath)), TEST_DISCOVERY_MAX_FILES, { - signal: control?.signal, heartbeat: control?.heartbeat, setPhase: control?.setPhase, - }, - ), - }, control); - }, + discoverTests: (testSymbol, scope) => getSymbolTestEvidence(projectRoot, host, testSymbol, scope, control), }, projectRoot ? configuredOrigins(projectRoot, host) : undefined); } diff --git a/src/tools/contracts.ts b/src/tools/contracts.ts index bc4842c1..cbde3ce3 100644 --- a/src/tools/contracts.ts +++ b/src/tools/contracts.ts @@ -58,6 +58,7 @@ export interface SharedCodebaseEditContextArgs { calleeLimit?: number | null; tokenBudget?: number | null; includeApiImpact?: boolean; + includeTests?: boolean; } export const MIN_CODEBASE_EDIT_CONTEXT_EDGE_LIMIT = 1 as const; @@ -71,6 +72,10 @@ export interface SharedIndexCodebaseArgs { verbose?: boolean; } +export interface SharedIndexStatusArgs { + checkFreshness?: boolean; +} + export interface SharedIndexMetricsArgs { reset?: boolean; } @@ -105,6 +110,16 @@ export interface SharedCallGraphPathArgs { maxDepth?: number; } +export interface SharedPrImpactArgs { + pr?: number; + branch?: string; + maxDepth?: number; + hubThreshold?: number; + checkConflicts?: boolean; + direction?: "callers" | "callees" | "both"; + workingTree?: boolean; +} + export interface SharedArchitectureContextArgs { query?: string | null; directory?: string | null; diff --git a/src/tools/edit-context.ts b/src/tools/edit-context.ts index 856c0ff7..782c5a20 100644 --- a/src/tools/edit-context.ts +++ b/src/tools/edit-context.ts @@ -1,11 +1,15 @@ import type { HostMode } from "../config/host.js"; import type { OperationControl } from "../utils/operation-control.js"; +import * as path from "node:path"; import { isOperationInterruption, throwIfOperationAborted, } from "../utils/operation-control.js"; +import { getProjectRoot } from "./operation-runtime.js"; import type { SearchResult } from "../indexer/index.js"; import type { CallEdgeData } from "../native/index.js"; +import { isJavaScriptRuntimeSourceFilePath } from "../indexer/local-module-resolution.js"; +import type { TestDiscoveryResult, TestReferenceResult } from "./test-discovery.js"; import { DEFAULT_CODEBASE_EDIT_CONTEXT_EDGE_LIMIT, MAX_CODEBASE_EDIT_CONTEXT_EDGE_LIMIT, @@ -15,19 +19,24 @@ import { import { getCallGraphData, getApiImpactEvidence, + getSymbolTestEvidence, implementationLookup, searchCodebase, type CallGraphDataResult, type CallGraphSymbolResolution, } from "./operations.js"; import type { ApiImpactEvidence, ApiImpactTarget } from "./api-impact.js"; +import type { BudgetedTextResult } from "./context-pack.js"; import { buildContextPack, DEFAULT_CONTEXT_PACK_TOKEN_BUDGET, fitTextToContextBudget, + countContextTokens, MIN_CONTEXT_PACK_TOKEN_BUDGET, } from "./utils.js"; +const MAX_TARGET_SOURCE_TOKENS = 512; + export interface CodebaseEditContextResult { text: string; details: { @@ -40,6 +49,29 @@ export interface CodebaseEditContextResult { calleeCount: number; apiImpactIncluded?: boolean; apiImpactTruncated?: boolean; + testDiscovery?: CodebaseEditTestDiscovery; + }; +} + +export type CodebaseEditTestReference = Pick; + +export interface CodebaseEditTestDiscovery extends Omit { + resolution: TestDiscoveryResult["resolution"] | "not_requested" | "unsupported" | "unavailable"; + results: CodebaseEditTestReference[]; + omittedReferenceCount: number; +} + +interface PendingTestDiscovery extends Omit { + results: TestReferenceResult[]; +} + +const TEST_REFERENCE_NOTICE = "Static source references; not runtime coverage."; + +function emptyTestDiscovery(resolution: CodebaseEditTestDiscovery["resolution"]): PendingTestDiscovery { + return { + resolution, results: [], scannedFileCount: 0, candidateFileCount: 0, + verifiedCallerCount: 0, capped: false, omittedReferenceCount: 0, }; } @@ -47,6 +79,7 @@ export interface CodebaseEditContextEvidence { sources: SearchResult[]; callers: CallEdgeData[]; callees: CallEdgeData[]; + testReferences: CodebaseEditTestReference[]; } export interface CodebaseEditContextDependencies { @@ -60,7 +93,7 @@ export interface CodebaseEditContextDependencies { ) => Promise; implementationLookup: ( query: string, - options?: { limit?: number }, + options?: Parameters[3], control?: OperationControl, ) => Promise; getCallGraphData: (params: { @@ -72,6 +105,10 @@ export interface CodebaseEditContextDependencies { target: ApiImpactTarget, control?: OperationControl, ) => Promise; + getTestEvidence?: ( + target: ApiImpactTarget, + control?: OperationControl, + ) => Promise; } function edgeLimit(value: number | null | undefined): number { @@ -84,41 +121,58 @@ function edgeLimit(value: number | null | undefined): number { ); } -function normalizedPath(value: string): string { - return value.replaceAll("\\", "/").replace(/^\.\//, ""); -} - -function pathsMatch(left: string, right: string): boolean { - const normalizedLeft = normalizedPath(left); - const normalizedRight = normalizedPath(right); - return normalizedLeft === normalizedRight - || normalizedLeft.endsWith(`/${normalizedRight}`) - || normalizedRight.endsWith(`/${normalizedLeft}`); -} - function targetSource( results: SearchResult[], resolution: Extract, + projectRoot: string | undefined, signal?: AbortSignal, ): SearchResult | undefined { - const containsLine = results.find((result) => { - throwIfOperationAborted(signal); - return pathsMatch(result.filePath, resolution.filePath) - && result.startLine <= resolution.startLine - && result.endLine >= resolution.startLine; - }); - if (containsLine) return containsLine; - return results.find((result) => { + const targetPath = path.resolve(projectRoot ?? ".", resolution.filePath); + const matches = results.filter((result) => { throwIfOperationAborted(signal); - return pathsMatch(result.filePath, resolution.filePath) && result.name === resolution.name; + return path.resolve(projectRoot ?? ".", result.filePath) === targetPath + && result.name === resolution.name + && result.chunkType === resolution.kind + && result.startLine === resolution.startLine + && result.content.length > 0; }); + return matches.length === 1 ? matches[0] : undefined; } -function formatSource(result: SearchResult, onHeader?: (endOffset: number) => void): string { +function targetSourceTokenBudget(tokenBudget: number): number { + return Math.max( + MIN_CONTEXT_PACK_TOKEN_BUDGET, + Math.min(MAX_TARGET_SOURCE_TOKENS, Math.floor(tokenBudget * 0.6)), + ); +} + +function fitSource( + result: SearchResult, + tokenBudget: number, + onSource: (endOffset: number) => void, + fitsOverall: (text: string) => boolean = () => true, +): BudgetedTextResult { const name = result.name ? ` ${result.name}` : ""; const header = `## Target implementation\n${result.filePath}:${result.startLine}-${result.endLine} (${result.chunkType}${name})`; - onHeader?.(header.length); - return [header, "```", result.content, "```"].join("\n"); + const lines = result.content.replaceAll("\r\n", "\n").trimEnd().split("\n"); + if (lines[0]) onSource(header.length + "\n```\n".length + lines[0].length); + const full = [header, "```", lines.join("\n"), "```"].join("\n"); + const fitted = fitTextToContextBudget(full, tokenBudget); + if (!fitted.truncated && fitsOverall(full)) return fitted; + const render = (count: number): string => + [header, "```", lines.slice(0, count).join("\n"), "```", "...[truncated to context token budget]"].join("\n"); + let low = 0; + let high = lines.length; + while (low < high) { + const middle = Math.ceil((low + high) / 2); + if (countContextTokens(render(middle)) <= fitted.tokenBudget && fitsOverall(render(middle))) low = middle; + else high = middle - 1; + } + if (low === 0) { + return { ...fitTextToContextBudget(`${header}\nSource body omitted: token budget.`, tokenBudget), truncated: true, retainedLength: 0 }; + } + const text = render(low); + return { text, tokenBudget: fitted.tokenBudget, tokenEstimate: countContextTokens(text), truncated: true, retainedLength: text.length }; } function formatEdges( @@ -164,6 +218,113 @@ function formatResolutionRisk(resolution: Exclude tokenBudget) { + header = "# Pre-edit context\nRisk: target identity exceeds the response budget."; + truncated = true; + } + let sourceEnd = Infinity; + const sourceFit = source ? fitSource( + source, + targetSourceTokenBudget(tokenBudget), + (endOffset) => { sourceEnd = endOffset; }, + (text) => countContextTokens(`${header}\n\n${text}\n\n${notice}`) <= tokenBudget, + ) : undefined; + const sourceText = sourceFit?.text + ?? `## Target implementation\nRisk: no implementation source matched the resolved target at ${sourceFilePath}.`; + const sourceFits = countContextTokens(`${header}\n\n${sourceText}\n\n${notice}`) <= tokenBudget; + let text = sourceFits + ? `${header}\n\n${sourceText}\n\n${notice}` + : `${header}\n\n${notice}`; + truncated ||= sourceFit?.truncated === true || !sourceFits; + const selected: CodebaseEditContextEvidence = { + sources: source && sourceFits && sourceEnd <= (sourceFit?.retainedLength ?? 0) ? [source] : [], + callers: [], callees: [], testReferences: [], + }; + const append = (section: string): boolean => { + const candidate = `${text}\n\n${section}`; + if (countContextTokens(candidate) > tokenBudget) { + truncated = true; + return false; + } + text = candidate; + return true; + }; + const appendTests = (kind: TestReferenceResult["evidenceKind"], heading?: string): void => { + let headingIncluded = false; + for (const reference of discovery.results) { + if (reference.evidenceKind !== kind) continue; + const row = `- ${reference.filePath}:${reference.startLine}-${reference.endLine} [${reference.evidenceKind}] referencedSymbol=${reference.referencedSymbol} referenceLine=${reference.referenceLine}`; + if (append(heading && !headingIncluded ? `${heading}\n${row}` : row)) { + const { filePath, startLine, endLine, evidenceKind, referencedSymbol, referenceLine } = reference; + selected.testReferences.push({ filePath, startLine, endLine, evidenceKind, referencedSymbol, referenceLine }); + headingIncluded = true; + } + } + }; + appendTests("direct_target_test_reference"); + const appendEdges = ( + edges: CallEdgeData[], + render: (edges: CallEdgeData[]) => string, + retained: CallEdgeData[], + ): void => { + if (edges.length === 0) { + append(render([])); + return; + } + let headingIncluded = false; + for (const edge of edges) { + const section = render([edge]); + const row = headingIncluded ? section.slice(section.indexOf("\n") + 1) : section; + if (append(row)) { + retained.push(edge); + headingIncluded = true; + } + } + }; + appendEdges(callers, formatCallers, selected.callers); + appendEdges(callees, (edges) => formatCallees(edges, sourceFilePath), selected.callees); + appendTests("related_caller_test_reference", "## Related caller-test references"); + const apiImpactIncluded = apiImpact ? append(apiImpact.text) : false; + const testDiscovery: CodebaseEditTestDiscovery = { + ...discovery, + results: selected.testReferences, + omittedReferenceCount: discovery.results.length - selected.testReferences.length, + }; + dependencies.onSelectedEvidence?.(selected); + return { + text, + details: { + resolution: "resolved", + tokenBudget, + tokenEstimate: countContextTokens(text), + truncated, + sourceIncluded: selected.sources.length > 0, + callerCount: selected.callers.length, + calleeCount: selected.callees.length, + testDiscovery, + ...(apiImpact ? { + apiImpactIncluded, + apiImpactTruncated: apiImpact.truncated || !apiImpactIncluded, + } : {}), + }, + }; +} + async function fallbackPack( dependencies: CodebaseEditContextDependencies, query: string, @@ -172,6 +333,7 @@ async function fallbackPack( tokenBudget: number | null | undefined, candidateSource: SearchResult[] = [], control?: OperationControl, + includeTests = false, ): Promise { throwIfOperationAborted(control?.signal); const conceptual = await dependencies.searchCodebase(query, { limit: 5 }, control); @@ -186,13 +348,35 @@ async function fallbackPack( projectRoot: dependencies.projectRoot, onSelectedEvidence: dependencies.onSelectedEvidence ? (evidence) => { selectedEvidence = evidence; } : undefined, }); - const fitted = fitTextToContextBudget(`${risk}\n\n${pack.text}`, tokenBudget ?? undefined); + const testDiscovery = includeTests ? emptyTestDiscovery( + resolution === "ambiguous" || resolution === "not_found" ? resolution : "not_requested", + ) : undefined; + let prefix = risk; + let riskTruncated = false; + if (testDiscovery) { + const notice = `## Source-linked tests\n${TEST_REFERENCE_NOTICE}\nDiscovery: ${testDiscovery.resolution}.`; + const budget = fitTextToContextBudget("", tokenBudget ?? undefined).tokenBudget; + const riskCharacters = Array.from(risk); + let low = 0; + let high = riskCharacters.length; + const render = (length: number): string => + `${riskCharacters.slice(0, length).join("")}${length < riskCharacters.length ? "…" : ""}\n\n${notice}`; + while (low < high) { + const middle = Math.ceil((low + high) / 2); + if (countContextTokens(`${render(middle)}\n\n...[truncated to context token budget]`) <= budget) low = middle; + else high = middle - 1; + } + prefix = render(low); + riskTruncated = low < riskCharacters.length; + } + const fitted = fitTextToContextBudget(`${prefix}\n\n${pack.text}`, tokenBudget ?? undefined); dependencies.onSelectedEvidence?.({ sources: (selectedEvidence ?? []) - .filter((entry) => risk.length + 2 + entry.endOffset <= fitted.retainedLength) + .filter((entry) => prefix.length + 2 + entry.endOffset <= fitted.retainedLength) .map((entry) => entry.result), callers: [], callees: [], + testReferences: [], }); return { text: fitted.text, @@ -200,10 +384,11 @@ async function fallbackPack( resolution, tokenBudget: fitted.tokenBudget, tokenEstimate: fitted.tokenEstimate, - truncated: fitted.truncated, - sourceIncluded: candidateSource.length > 0, + truncated: fitted.truncated || riskTruncated, + sourceIncluded: false, callerCount: 0, calleeCount: 0, + ...(testDiscovery ? { testDiscovery } : {}), }, }; } @@ -224,6 +409,7 @@ export async function resolveCodebaseEditContextWithDependencies( input.tokenBudget, [], control, + input.includeTests, ); } @@ -247,6 +433,7 @@ export async function resolveCodebaseEditContextWithDependencies( input.tokenBudget, candidates, control, + input.includeTests, ); } @@ -259,12 +446,13 @@ export async function resolveCodebaseEditContextWithDependencies( input.tokenBudget, [], control, + input.includeTests, ); } const resolution = callersResult.resolution; const [definitionsResult, calleesResult] = await Promise.allSettled([ - dependencies.implementationLookup(symbol, { limit: 10 }, control), + dependencies.implementationLookup(resolution.symbolId, { limit: 1, exactSymbol: true }, control), dependencies.getCallGraphData({ name: symbol, filePath: input.filePath ?? resolution.filePath, @@ -286,17 +474,16 @@ export async function resolveCodebaseEditContextWithDependencies( callees = calleesResult.value.callees.slice(0, edgeLimit(input.calleeLimit)); } - const source = targetSource(definitionsResult.value, resolution, control?.signal); + const source = targetSource(definitionsResult.value, resolution, dependencies.projectRoot, control?.signal); const callers = callersResult.callers.slice(0, edgeLimit(input.callerLimit)); - const sourceBudget = Math.max( - MIN_CONTEXT_PACK_TOKEN_BUDGET, - Math.floor((input.tokenBudget ?? DEFAULT_CONTEXT_PACK_TOKEN_BUDGET) * 0.6), - ); + const sourceBudget = targetSourceTokenBudget(input.tokenBudget ?? DEFAULT_CONTEXT_PACK_TOKEN_BUDGET); + const header = [`# Pre-edit context for ${resolution.name}`, graphRisk] + .filter((section) => section !== undefined).join("\n\n"); + const responseBudget = fitTextToContextBudget("", input.tokenBudget ?? undefined).tokenBudget; let sourceHeaderEnd = Infinity; - const sourceFit = source - ? fitTextToContextBudget(formatSource(source, dependencies.onSelectedEvidence - ? (endOffset) => { sourceHeaderEnd = endOffset; } - : undefined), sourceBudget) + const sourceFit = source && !input.includeTests + ? fitSource(source, sourceBudget, (endOffset) => { sourceHeaderEnd = endOffset; }, + (text) => countContextTokens(`${header}\n\n${text}\n\n...[truncated to context token budget]`) <= responseBudget) : undefined; const sourceText = sourceFit?.text ?? `## Target implementation\nRisk: no implementation source matched the resolved target at ${resolution.filePath}:${resolution.startLine}.`; @@ -321,27 +508,44 @@ export async function resolveCodebaseEditContextWithDependencies( }; } } - const header = [`# Pre-edit context for ${resolution.name}`, graphRisk] - .filter((section) => section !== undefined).join("\n\n"); + let testDiscovery: PendingTestDiscovery | undefined; + if (input.includeTests) { + testDiscovery = emptyTestDiscovery("unsupported"); + if (isJavaScriptRuntimeSourceFilePath(resolution.filePath)) { + testDiscovery = emptyTestDiscovery("unavailable"); + if (dependencies.getTestEvidence) { + try { + const discovered = await dependencies.getTestEvidence({ + symbol: resolution.name, filePath: resolution.filePath, startLine: resolution.startLine, + }, control); + testDiscovery = { ...discovered, omittedReferenceCount: 0 }; + } catch (error) { + if (isOperationInterruption(error)) throw error; + throwIfOperationAborted(control?.signal); + } + } + } + } + throwIfOperationAborted(control?.signal); + if (testDiscovery) { + return packTestContext(input, dependencies, header, source, callers, callees, resolution.filePath, testDiscovery, apiImpact); + } const sourceOffset = header.length + 2; const callerOffset = sourceOffset + sourceText.length + 2; - const callerEnds: number[] | undefined = dependencies.onSelectedEvidence ? [] : undefined; - const callerText = formatCallers(callers, callerEnds - ? (endOffset) => { callerEnds.push(callerOffset + endOffset); } - : undefined); + const callerEnds: number[] = []; + const callerText = formatCallers(callers, (endOffset) => { callerEnds.push(callerOffset + endOffset); }); const calleeOffset = callerOffset + callerText.length + 2; - const calleeEnds: number[] | undefined = dependencies.onSelectedEvidence ? [] : undefined; - const calleeText = formatCallees(callees, resolution.filePath, calleeEnds - ? (endOffset) => { calleeEnds.push(calleeOffset + endOffset); } - : undefined); + const calleeEnds: number[] = []; + const calleeText = formatCallees(callees, resolution.filePath, (endOffset) => { calleeEnds.push(calleeOffset + endOffset); }); const precedingText = [header, sourceText, callerText, calleeText].join("\n\n"); const completeText = apiImpact ? `${precedingText}\n\n${apiImpact.text}` : precedingText; const fitted = fitTextToContextBudget(completeText, input.tokenBudget ?? undefined); + const selectedSources = source && sourceFit && sourceHeaderEnd <= sourceFit.retainedLength + && sourceOffset + sourceFit.text.length <= fitted.retainedLength ? [source] : []; + const selectedCallers = callers.filter((_edge, index) => callerEnds[index] <= fitted.retainedLength); + const selectedCallees = callees.filter((_edge, index) => calleeEnds[index] <= fitted.retainedLength); dependencies.onSelectedEvidence?.({ - sources: source && sourceHeaderEnd <= (sourceFit?.retainedLength ?? 0) - && sourceOffset + sourceHeaderEnd <= fitted.retainedLength ? [source] : [], - callers: callers.filter((_edge, index) => (callerEnds?.[index] ?? Infinity) <= fitted.retainedLength), - callees: callees.filter((_edge, index) => (calleeEnds?.[index] ?? Infinity) <= fitted.retainedLength), + sources: selectedSources, callers: selectedCallers, callees: selectedCallees, testReferences: [], }); const apiImpactOffset = Array.from(`${precedingText}\n\n`).length; const renderedApiTail = Array.from(fitted.text).slice(apiImpactOffset).join(""); @@ -354,10 +558,10 @@ export async function resolveCodebaseEditContextWithDependencies( resolution: "resolved", tokenBudget: fitted.tokenBudget, tokenEstimate: fitted.tokenEstimate, - truncated: fitted.truncated, - sourceIncluded: source !== undefined, - callerCount: callers.length, - calleeCount: callees.length, + truncated: fitted.truncated || sourceFit?.truncated === true, + sourceIncluded: selectedSources.length > 0, + callerCount: selectedCallers.length, + calleeCount: selectedCallees.length, ...(apiImpact ? { apiImpactIncluded: apiImpactRendered, apiImpactTruncated: apiImpact.truncated || !apiImpactFullyRendered, @@ -372,33 +576,37 @@ export async function resolveCodebaseEditContext( input: SharedCodebaseEditContextArgs, control?: OperationControl, ): Promise { + const root = getProjectRoot(projectRoot, host); return resolveCodebaseEditContextWithDependencies(input, { - projectRoot, + projectRoot: root, searchCodebase: (query, options, operationControl) => searchCodebase( - projectRoot, + root, host, query, options, operationControl, ), implementationLookup: (query, options, operationControl) => implementationLookup( - projectRoot, + root, host, query, options, operationControl, ), getCallGraphData: (params, operationControl) => getCallGraphData( - projectRoot, + root, host, params, operationControl, ), getApiImpactEvidence: (target, operationControl) => getApiImpactEvidence( - projectRoot, + root, host, target, operationControl, ), + getTestEvidence: (target, operationControl) => getSymbolTestEvidence( + root, host, target.symbol, { targetFilePath: target.filePath }, operationControl, + ), }, control); } diff --git a/src/tools/execute-common.ts b/src/tools/execute-common.ts index bf0e8ba1..88aacfc2 100644 --- a/src/tools/execute-common.ts +++ b/src/tools/execute-common.ts @@ -9,6 +9,7 @@ import type { SharedIndexCodebaseArgs, SharedIndexLogsArgs, SharedIndexMetricsArgs, + SharedIndexStatusArgs, SharedImplementationLookupArgs, } from "./contracts.js"; import { @@ -64,7 +65,8 @@ export async function executeCodebaseEditContext( args: SharedCodebaseEditContextArgs, control?: OperationControl, ): Promise { - return { text: (await resolveCodebaseEditContext(projectRoot, host, args, control)).text }; + const result = await resolveCodebaseEditContext(projectRoot, host, args, control); + return { text: result.text, details: result.details }; } export async function executeIndexCodebase( @@ -88,9 +90,11 @@ export async function executeIndexCodebase( export async function executeIndexStatus( projectRoot: string | undefined, host: HostMode, + args: SharedIndexStatusArgs = {}, control?: OperationControl, ): Promise { - return { text: formatStatus(await getIndexStatus(projectRoot, host, control)) }; + const status = await getIndexStatus(projectRoot, host, args, control); + return { text: formatStatus(status), details: { ...status } }; } export async function executeIndexHealthCheck( diff --git a/src/tools/format-pr-impact.ts b/src/tools/format-pr-impact.ts index ac86d756..6931a9f5 100644 --- a/src/tools/format-pr-impact.ts +++ b/src/tools/format-pr-impact.ts @@ -3,6 +3,40 @@ import type { PrImpactResult } from "../indexer/pr-impact-types.js"; export function formatPrImpact(result: PrImpactResult): string { const lines: string[] = []; + const workingTree = result.workingTree; + const partial = workingTree !== undefined + && (workingTree.unassessed.length > 0 || workingTree.omittedFileCount > 0); + if (workingTree) { + lines.push("→ Working-tree impact: union of staged, unstaged, and untracked path changes."); + lines.push(" Scope: current indexed files; static file-level graph analysis, not changed-line or staged-source analysis."); + lines.push(" Consistency: bounded before/after checks, not an atomic filesystem snapshot."); + lines.push(` HEAD: ${workingTree.headCommit ?? "unborn repository"}`); + if (workingTree.entries.length === 0 && workingTree.omittedFileCount === 0) { + lines.push("No working-tree changes."); + return lines.join("\n"); + } + lines.push("→ Path provenance (XY: index/worktree status; space means unchanged):"); + for (const entry of workingTree.entries) { + const origin = entry.previousPath === undefined ? "" : ` (from ${JSON.stringify(entry.previousPath)})`; + lines.push(` [${entry.indexStatus}${entry.worktreeStatus}] ${JSON.stringify(entry.path)}${origin}`); + } + if (workingTree.unassessed.length > 0) { + const reasons: Record<(typeof workingTree.unassessed)[number]["reason"], string> = { + deleted: "deleted; historical declarations not assessed", + renamed_from: "rename source; historical declarations not assessed", + not_indexed: "not indexed; source not assessed", + no_symbols: "no indexed symbols; graph impact unknown", + outside_project: "outside project; target not read", + not_regular: "not a regular file; target not assessed", + }; + lines.push("→ Unassessed paths (coverage unknown):"); + for (const entry of workingTree.unassessed) { + lines.push(` - ${JSON.stringify(entry.path)}: ${reasons[entry.reason]}`); + } + } + lines.push(`→ Omitted paths: ${workingTree.omittedFileCount}${workingTree.omittedFileCount > 0 ? " (200-path cap; coverage unknown)" : ""}`); + if (partial) lines.push(" Graph counts below are partial; missing coverage is not low risk."); + } if (result.indexPreparation) { const status = result.indexPreparation.prepared ? "prepared automatically" : "already available"; const commit = result.indexPreparation.commit ? ` at ${result.indexPreparation.commit.slice(0, 12)}` : ""; @@ -11,14 +45,18 @@ export function formatPrImpact(result: PrImpactResult): string { lines.push(`→ Files changed: ${result.changedFiles.length}`); for (const file of result.changedFiles) { - lines.push(` - ${file}`); + lines.push(` - ${workingTree ? JSON.stringify(file) : file}`); } const directCount = result.directSymbols.length; const transitiveCount = result.transitiveCallers.length; const directionLabel = result.direction === 'callees' ? 'callees' : result.direction === 'both' ? 'reachable (callers + callees)' : 'callers'; + let symbolLabel = "Symbols affected"; + if (workingTree) { + symbolLabel = partial ? "Current indexed graph symbols affected (partial)" : "Current indexed graph symbols affected"; + } lines.push( - `\u2192 Symbols affected: ${result.totalAffected} (${directCount} direct, ${transitiveCount} transitive ${directionLabel})`, + `\u2192 ${symbolLabel}: ${result.totalAffected} (${directCount} direct, ${transitiveCount} transitive ${directionLabel})`, ); if (directCount > 0) { @@ -45,7 +83,7 @@ export function formatPrImpact(result: PrImpactResult): string { lines.push("→ Communities touched: none"); } - lines.push(`→ Risk: ${result.riskLevel} — ${result.riskReason}`); + lines.push(`→ Risk: ${result.riskLevel} — ${result.riskReason}${workingTree ? " (static, file-level current-graph heuristic; not runtime impact)" : ""}`); if (result.hubNodes.length > 0) { lines.push(" Hub nodes in change scope:"); for (const hub of result.hubNodes) { diff --git a/src/tools/operations.ts b/src/tools/operations.ts index c7a9ac8b..2352509c 100644 --- a/src/tools/operations.ts +++ b/src/tools/operations.ts @@ -16,6 +16,13 @@ import { import { getRecentGitActivityAbortable } from "./visualize/activity.js"; import { transformForVisualization } from "./visualize/transform.js"; import { buildApiImpactEvidence, type ApiImpactEvidence, type ApiImpactTarget } from "./api-impact.js"; +import { + discoverSymbolTests, + TEST_DISCOVERY_MAX_DEFINITIONS, + TEST_DISCOVERY_MAX_FILES, + type TestDiscoveryResult, + type TestDiscoveryScope, +} from "./test-discovery.js"; import { CODE_COMMUNITIES_DEFAULT_COUPLING_LIMIT, CODE_COMMUNITIES_DEFAULT_HUB_THRESHOLD, @@ -25,13 +32,13 @@ import { CODE_COMMUNITIES_MIN_COUPLING, CODE_COMMUNITIES_MIN_SIZE, } from "./contracts.js"; -import type { SharedArchitectureContextArgs, SharedCodeCommunitiesArgs } from "./contracts.js"; +import type { SharedArchitectureContextArgs, SharedCodeCommunitiesArgs, SharedIndexStatusArgs, SharedPrImpactArgs } from "./contracts.js"; import { calculatePercentage, formatProgressTitle, formatStatus } from "./utils.js"; import type { LogLevel } from "../config/schema.js"; import type { LogEntry } from "../utils/logger.js"; import type { CostEstimate, DryRunEstimate } from "../utils/cost.js"; import type { AutoIndexStatusSnapshot } from "../utils/auto-index.js"; -import type { SearchTrace } from "../indexer/index.js"; +import type { IndexFreshnessResult, SearchTrace } from "../indexer/index.js"; import { formatEffectivenessMetrics, getProcessEffectivenessMetrics, @@ -72,7 +79,10 @@ import { type SearchResult = Awaited>[number]; type IndexStats = Awaited>; type StatusResult = Awaited>; -export type IndexStatusResult = StatusResult & { autoIndex: AutoIndexStatusSnapshot }; +export type IndexStatusResult = StatusResult & { + autoIndex: AutoIndexStatusSnapshot; + freshness: IndexFreshnessResult | null; +}; type HealthCheckResult = Awaited>; type PrImpactResult = Awaited>; type IndexMessageResult = { kind: "message"; text: string }; @@ -200,7 +210,7 @@ function exactSymbolSearchResults( throwIfOperationAborted(signal); const matching = symbols.filter((symbol) => { throwIfOperationAborted(signal); - return symbolNameMatches(symbol, name) && exactSymbolMatchesScope(symbol, projectRoot, options); + return (symbol.id === name || symbolNameMatches(symbol, name)) && exactSymbolMatchesScope(symbol, projectRoot, options); }); throwIfOperationAborted(signal); return matching @@ -208,14 +218,14 @@ function exactSymbolSearchResults( .slice(0, limit) .map((symbol) => { throwIfOperationAborted(signal); - let content = "[File not accessible]"; + let content = ""; try { content = readFileSync(symbol.filePath, "utf-8") .split("\n") .slice(symbol.startLine - 1, symbol.endLine) .join("\n"); } catch { - // Preserve the normal search contract when a stale indexed file is unavailable. + // Keep canonical coordinates, but never substitute a marker for source bytes. } return { filePath: symbol.filePath, @@ -445,9 +455,19 @@ export async function implementationLookup( ): Promise { await runOperationPhase(control, "waiting_for_index"); await ensureAutoIndexReadyForRetrieval(projectRoot, host, control); - await runOperationPhase(control, options.exactSymbol ? "resolving_symbol" : "embedding_query"); const root = getProjectRoot(projectRoot, host); const indexer = getIndexerForProject(root, host); + return implementationLookupForIndexer(indexer, root, query, options, control); +} + +export async function implementationLookupForIndexer( + indexer: Indexer, + projectRoot: string, + query: string, + options: NonNullable[3]> = {}, + control?: OperationControl, +): Promise { + await runOperationPhase(control, options.exactSymbol ? "resolving_symbol" : "embedding_query"); if (options.exactSymbol) { const results = exactSymbolSearchResults( await indexer.getCallGraphSymbols({ @@ -455,7 +475,7 @@ export async function implementationLookup( setPhase: control?.setPhase, heartbeat: control?.heartbeat, }), - root, + projectRoot, query, options, control?.signal, @@ -496,6 +516,33 @@ export async function getCallGraphData( return result; } +export async function getSymbolTestEvidence( + projectRoot: string | undefined, + host: HostMode, + symbol: string, + scope: TestDiscoveryScope, + control?: OperationControl, +): Promise { + const root = getProjectRoot(projectRoot, host); + return discoverSymbolTests(root, symbol, scope, { + lookup: (name) => implementationLookup(root, host, name, { + exactSymbol: true, limit: TEST_DISCOVERY_MAX_DEFINITIONS, + }, control), + graph: (name, filePath) => getCallGraphData(root, host, { name, filePath, direction: "callers" }, control), + indexedFiles: async () => { + throwIfOperationAborted(control?.signal); + const files = await getIndexerForProject(root, host).getIndexedFilePathsForActiveBranch(); + throwIfOperationAborted(control?.signal); + return files; + }, + lexicalFiles: (identifiers, allowedFiles) => getIndexerForProject(root, host).getKeywordFileCandidates( + identifiers, allowedFiles.map((filePath) => path.resolve(root, filePath)), TEST_DISCOVERY_MAX_FILES, { + signal: control?.signal, heartbeat: control?.heartbeat, setPhase: control?.setPhase, + }, + ), + }, control); +} + export async function getApiImpactEvidence( projectRoot: string | undefined, host: HostMode, @@ -719,6 +766,7 @@ export async function runIndexCodebase( export async function getIndexStatus( projectRoot: string | undefined, host: HostMode, + args: SharedIndexStatusArgs = {}, control?: OperationControl, ): Promise { await runOperationPhase(control, "reading_status"); @@ -726,8 +774,17 @@ export async function getIndexStatus( const indexer = getIndexerForProject(root, host); const status = await indexer.getStatus(); throwIfOperationAborted(control?.signal); + const freshness = args.checkFreshness + ? await indexer.getIndexFreshness({ + signal: control?.signal, + setPhase: control?.setPhase, + heartbeat: control?.heartbeat, + }) + : null; + throwIfOperationAborted(control?.signal); return { ...status, + freshness, autoIndex: getAutoIndexStatus(root, host), }; } @@ -765,29 +822,20 @@ export async function runIndexHealthCheck( export async function getPrImpact( projectRoot: string | undefined, host: HostMode, - params: { - pr?: number; - branch?: string; - maxDepth?: number; - hubThreshold?: number; - checkConflicts?: boolean; - direction?: "callers" | "callees" | "both"; - }, + params: SharedPrImpactArgs, control?: OperationControl, ): Promise { - await runOperationPhase(control, "waiting_for_index"); - await ensureAutoIndexReadyForRetrieval(projectRoot, host, control); + if (params.workingTree && (params.pr !== undefined || params.branch !== undefined || params.checkConflicts === true)) { + throw new Error("workingTree cannot be combined with pr, branch, or checkConflicts"); + } + if (!params.workingTree) { + await runOperationPhase(control, "waiting_for_index"); + await ensureAutoIndexReadyForRetrieval(projectRoot, host, control); + } await runOperationPhase(control, "analyzing_impact"); const indexer = getIndexerForProject(projectRoot, host); const result = await indexer.getPrImpact( - { - pr: params.pr, - branch: params.branch, - maxDepth: params.maxDepth, - hubThreshold: params.hubThreshold, - checkConflicts: params.checkConflicts, - direction: params.direction, - }, + params, (progress) => { const percentage = calculatePercentage(progress); void control?.reportProgress?.(percentage, progress.phase); diff --git a/src/tools/test-discovery.ts b/src/tools/test-discovery.ts index c9e07a0d..4afeb5ab 100644 --- a/src/tools/test-discovery.ts +++ b/src/tools/test-discovery.ts @@ -48,6 +48,8 @@ export interface TestDiscoveryOperations { export interface TestDiscoveryScope { fileType?: string; directory?: string; + /** Selects the implementation only; returned tests retain their own scope. */ + targetFilePath?: string; } function relativePath(root: string, filePath: string): string | undefined { @@ -242,6 +244,18 @@ export async function discoverSymbolTests( }; const markCapped = (): void => { result.capped = true; }; throwIfOperationAborted(control?.signal); + let targetFilePath: string | undefined; + if (scope.targetFilePath !== undefined) { + const canonicalRoot = await fs.realpath(root); + throwIfOperationAborted(control?.signal); + const requestedPath = path.resolve(root, scope.targetFilePath); + const requestedRelative = relativePath(root, requestedPath); + const canonicalTarget = await fs.realpath(requestedPath).catch(() => + requestedRelative ? path.resolve(canonicalRoot, requestedRelative) : requestedPath); + throwIfOperationAborted(control?.signal); + targetFilePath = relativePath(canonicalRoot, canonicalTarget); + if (!targetFilePath) throw new Error("Test discovery targetFilePath must remain inside the project root."); + } const sources = new Map>(); const currentSource = (filePath: string): Promise => { let pending = sources.get(filePath); @@ -254,6 +268,7 @@ export async function discoverSymbolTests( }; const runtimeDefinitions = async (name: string, expectedPath?: string): Promise<{ candidates: RuntimeDefinition[]; complete: boolean }> => { const definitions = await operations.lookup(name); + throwIfOperationAborted(control?.signal); const executable: RuntimeDefinition[] = []; let complete = definitions.length < TEST_DISCOVERY_MAX_DEFINITIONS; if (!complete) { @@ -291,7 +306,7 @@ export async function discoverSymbolTests( `${candidate.filePath}:${candidate.declaration.startLine}:${candidate.declaration.startCol}`, candidate, ])).values()], complete }; }; - const targetLookup = await runtimeDefinitions(symbol); + const targetLookup = await runtimeDefinitions(symbol, targetFilePath); const unique = targetLookup.candidates; if (!targetLookup.complete || unique.length !== 1) { result.resolution = !targetLookup.complete || unique.length > 1 ? "ambiguous" : "not_found"; diff --git a/src/tools/utils.ts b/src/tools/utils.ts index ee56929d..cc412099 100644 --- a/src/tools/utils.ts +++ b/src/tools/utils.ts @@ -103,6 +103,18 @@ export function formatIndexStats(stats: IndexStats, verbose: boolean = false): s export function formatStatus(status: StatusResult | IndexStatusResult): string { const autoIndex = "autoIndex" in status ? status.autoIndex : undefined; const autoIndexLines = autoIndex ? formatAutoIndexStatus(autoIndex) : []; + const freshness = "freshness" in status ? status.freshness : null; + const freshnessLines = ["Source freshness: not checked"]; + if (freshness === null) { + freshnessLines.push("If source freshness is needed, run index_status({checkFreshness:true})."); + } else { + freshnessLines[0] = `Source freshness: ${freshness.current ? "current" : "not current"} (reason: ${freshness.reason})`; + if (!freshness.current) { + freshnessLines.push(freshness.reason === "unreadable" + ? "Resolve source/index access and readability before running index_codebase." + : "Run index_codebase normally to create or refresh the index, then check freshness again."); + } + } const countLine = status.mode === "structural" ? `Stored chunks (structural, all branches): ${(status.indexedChunkCount ?? 0).toLocaleString()}` : `${status.branchReadiness ? "Indexed chunks (all branches)" : "Indexed chunks"}: ${status.vectorCount.toLocaleString()}`; @@ -120,7 +132,7 @@ export function formatStatus(status: StatusResult | IndexStatusResult): string { const unindexedDetails = status.branchReadiness ? [countLine, `Location: ${status.indexPath}`, ...branchLines, ...migrationLines] : []; if (!status.indexed) { if (status.warning) { - return [...autoIndexLines, ...unindexedDetails, status.warning].join("\n"); + return [...autoIndexLines, ...freshnessLines, ...unindexedDetails, status.warning].join("\n"); } if (status.failedBatchesCount > 0) { @@ -133,14 +145,15 @@ export function formatStatus(status: StatusResult | IndexStatusResult): string { lines.push(`Failed batches: ${status.failedBatchesPath}`); } - return [...autoIndexLines, ...unindexedDetails, ...lines].join("\n"); + return [...autoIndexLines, ...freshnessLines, ...unindexedDetails, ...lines].join("\n"); } - return [...autoIndexLines, ...unindexedDetails, "Codebase is not indexed. Run index_codebase to create an index."].join("\n"); + return [...autoIndexLines, ...freshnessLines, ...unindexedDetails, "Codebase is not indexed. Run index_codebase to create an index."].join("\n"); } const lines = [ ...autoIndexLines, + ...freshnessLines, ...("mode" in status && status.mode ? [`Indexing mode: ${status.mode}`] : []), countLine, `Provider: ${status.provider}`, diff --git a/tests/cbi-cli.test.ts b/tests/cbi-cli.test.ts index a198276e..5909e775 100644 --- a/tests/cbi-cli.test.ts +++ b/tests/cbi-cli.test.ts @@ -4,7 +4,7 @@ import { parseCbiCommandArgs, runCbiCli } from "../src/adapters/cbi.js"; describe("ocbi CLI (with cbi compatibility alias)", () => { it.each(["ocbi", "cbi"])("prints primary help through the %s alias", async (binary) => { - for (const command of [undefined, "status", "index", "search", "definition", "graph", "workspace"]) { + for (const command of [undefined, "status", "impact", "index", "search", "definition", "graph", "workspace"]) { const output: string[] = []; const args = command === "workspace" ? ["workspace", "status"] : command ? [command] : []; const exitCode = await runCbiCli(["node", binary, ...args, "--help"], "/tmp", { @@ -22,16 +22,46 @@ describe("ocbi CLI (with cbi compatibility alias)", () => { .toMatchObject({ positionals: ["retry logic"], limit: 3, project: "/tmp/repo", host: "jcode" }); }); - it.each(["ocbi", "cbi"])("prints status through the shared status operation via %s", async (binary) => { + it.each([ + ["search", "signalGate", "--check-freshness"], + ["search", "signalGate", "--json"], + ["definition", "signalGate", "--check-freshness"], + ["definition", "signalGate", "--json"], + ["graph", "callers", "signalGate", "--check-freshness"], + ["graph", "callers", "signalGate", "--json"], + ["index", "--check-freshness"], + ["index", "--json"], + ["workspace", "status", "--repo", "fixture=/tmp", "--check-freshness"], + ])("rejects status-only flags on %s", async (...args) => { const stdout: string[] = []; - const exitCode = await runCbiCli(["node", binary, "status", "--project", "/repo", "--host", "jcode"], "/tmp", { - runStatus: async () => ({ text: "Indexed chunks: 10" }), + const stderr: string[] = []; + expect(await runCbiCli(["node", "ocbi", ...args], "/tmp", { printStdout: (text) => stdout.push(text), - printStderr: () => undefined, - }); + printStderr: (text) => stderr.push(text), + })).toBe(1); + expect(stdout).toEqual([]); + expect(stderr.join("\n")).toMatch(/Unknown (?:index )?option/); + }); - expect(exitCode).toBe(0); - expect(stdout).toEqual(["Indexed chunks: 10"]); + it.each([ + ["impact"], + ["impact", "--json"], + ["impact", "--working-tree", "extra"], + ["impact", "--working-tree", "--pr", "1"], + ["impact", "--working-tree", "--branch", "main"], + ["impact", "--working-tree", "--check-conflicts"], + ["impact", "--working-tree", "--check-freshness"], + ["status", "--working-tree"], + ["search", "signalGate", "--working-tree"], + ])("rejects unsupported impact CLI invocations: %s", async (...args) => { + const stdout: string[] = []; + const stderr: string[] = []; + expect(await runCbiCli(["node", "ocbi", ...args], "/tmp", { + printStdout: (text) => stdout.push(text), + printStderr: (text) => stderr.push(text), + })).toBe(1); + expect(stdout).toEqual([]); + expect(stderr.join("\n")).toMatch(/requires --working-tree|requires no positional arguments|Unknown option/); }); it("dispatches search, definition, and graph commands with parsed inputs", async () => { @@ -120,7 +150,7 @@ describe("ocbi CLI (with cbi compatibility alias)", () => { }); const exitCode = await runCbiCli(["node", "cbi", "status", "--project", "/repo", "--host", "jcode"], "/tmp", { - runStatus: async () => ({ text: "apiKey=top-secret-value", isError: true }), + runStatus: async () => { throw new Error("apiKey=top-secret-value"); }, printStdout: () => undefined, printStderr: undefined, }); diff --git a/tests/context-evidence-acceptance.test.ts b/tests/context-evidence-acceptance.test.ts new file mode 100644 index 00000000..1a393c2d --- /dev/null +++ b/tests/context-evidence-acceptance.test.ts @@ -0,0 +1,370 @@ +import type { CodebaseContextResult } from "../src/tools/context.js"; +import type { CodebaseEditContextResult } from "../src/tools/edit-context.js"; +import type { ArchitectureContextResult } from "../src/tools/architecture-context.js"; +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import * as fs from "node:fs/promises"; +import * as os from "node:os"; +import * as path from "node:path"; +import { parseConfig } from "../src/config/schema.js"; +import { Indexer } from "../src/indexer/index.js"; +import { resolveCodebaseContext } from "../src/tools/context.js"; +import { resolveCodebaseEditContext } from "../src/tools/edit-context.js"; +import { getArchitectureContext } from "../src/tools/operations.js"; +import { configCache, getIndexerCacheKey, indexerCache } from "../src/tools/operation-runtime.js"; +import { countContextTokens } from "../src/tools/context-pack.js"; +import { estimateTokens } from "../src/utils/cost.js"; + +const TARGET = "signalGateForUnicodeEnvelopeValidationWithPreservedSourceCoordinatesAndExplicitEvidenceIdentity"; +const TARGET_DIRECTORY = "src/envelopes/coordinate-preserving-transport/evidence-with-a-deliberately-long-directory-name"; +const TARGET_PATH = `${TARGET_DIRECTORY}/unicode-envelope-validation-with-a-long-source-file-name.ts`; +const DUPLICATE_PATH = "src/foreign/gate.ts"; +const CALLER_PATH = "src/transport.ts"; +const HELPER_PATH = "src/normalize.ts"; +const BUDGETS = [128, 600, 1200] as const; + +interface Citation { + filePath: string; + startLine: number; + endLine: number; +} + +interface SourceCredit { + citation: Citation; + completeDeclaration: boolean; +} + +let root: string; +let indexer: Indexer | undefined; +let sources: Map; + +function normalizeEol(value: string): string { + return value.replaceAll("\r\n", "\n"); +} + +// Resolve compact display paths against this fixture only, never the host filesystem. +function fixturePath(displayPath: string): string { + const candidates = [...sources.keys()].filter((relative) => { + const absolute = path.join(root, relative); + return displayPath.startsWith("…") + ? displayPath.length > 1 && absolute.endsWith(displayPath.slice(1)) + : displayPath === relative || displayPath === absolute; + }); + expect(candidates, `fixture identity for ${displayPath}`).toHaveLength(1); + return candidates[0]; +} + +function sourceLines(citation: Citation): string[] { + const relative = fixturePath(citation.filePath); + const lines = normalizeEol(sources.get(relative)!).trimEnd().split("\n"); + expect(Number.isInteger(citation.startLine)).toBe(true); + expect(Number.isInteger(citation.endLine)).toBe(true); + expect(citation.startLine).toBeGreaterThanOrEqual(1); + expect(citation.endLine).toBeGreaterThanOrEqual(citation.startLine); + expect(citation.endLine).toBeLessThanOrEqual(lines.length); + return lines.slice(citation.startLine - 1, citation.endLine); +} + +function identity(citation: Citation): string { + sourceLines(citation); + return `${fixturePath(citation.filePath)}:${citation.startLine}-${citation.endLine}`; +} + +function outsideSourceLines(text: string): string[] { + let inSource = false; + return text.split("\n").filter((line) => { + if (line.startsWith("```")) { + inSource = !inSource; + return false; + } + return !inSource; + }); +} + +function contextCitations(text: string): Citation[] { + return outsideSourceLines(text).flatMap((line) => { + const match = /^\[\d+\] \S+(?: "(?:\\.|[^"\\])*")? in (.+?):(\d+)-(\d+)(?: \[origin: [^\]]+\])? \(score [^)]+\)/u.exec(line); + return match ? [{ filePath: match[1], startLine: Number(match[2]), endLine: Number(match[3]) }] : []; + }); +} + +function assertContextEvidence(response: CodebaseContextResult): Citation[] { + const citations = contextCitations(response.text); + const selected = response.details?.results ?? []; + expect(citations.map(identity)).toEqual(selected.map(identity)); + expect(response.details?.selectedCount).toBe(citations.length); + expect(countContextTokens(response.text)).toBeLessThanOrEqual(response.details!.tokenBudget); + // Context packs expose locations, not source bodies: no declaration credit here. + return citations; +} + +function requireRetainedCitation(citations: Citation[], expected: Citation): void { + expect(citations.map(identity), "required evidence must survive rendering").toContain(identity(expected)); +} + +function targetSourceCredit(text: string): SourceCredit | undefined { + const section = /(?:^|\n)## Target implementation\n([^\n]+):(\d+)-(\d+) \([^\n]*\)\n```\n/u.exec(text); + if (!section) return undefined; + const citation = { filePath: section[1], startLine: Number(section[2]), endLine: Number(section[3]) }; + const expected = sourceLines(citation).join("\n").trim(); + const tail = normalizeEol(text.slice(section.index + section[0].length)); + const closing = tail.indexOf("\n```"); + const truncation = tail.startsWith("...[truncated to context token budget]") + ? 0 : tail.indexOf("\n...[truncated to context token budget]"); + const completeBody = closing >= 0 && (truncation < 0 || closing < truncation); + const rawBody = completeBody ? tail.slice(0, closing) : tail.slice(0, truncation < 0 ? tail.length : truncation); + expect(expected.startsWith(rawBody.trim()), `source excerpt at ${identity(citation)}`).toBe(true); + // A clipped final line is not a full source line; retain only verifiable lines. + const body = completeBody ? rawBody.trim() : rawBody.slice(0, rawBody.lastIndexOf("\n") + 1).trim(); + if (!body) return undefined; + if (completeBody && truncation < 0) expect(body).toBe(expected); + const declaration = normalizeEol(sources.get(fixturePath(citation.filePath))!).trim(); + return { + citation, + completeDeclaration: completeBody && body === declaration, + }; +} + +function editNeighbors(text: string): { callers: Citation[]; callees: Citation[] } { + const callers: Citation[] = []; + const callees: Citation[] = []; + let section = ""; + for (const line of outsideSourceLines(text)) { + if (line.startsWith("## ")) section = line; + const match = /^- (\S+) (?:at|from) (.+?):(\d+) \(([^,]+), (?:resolved|unresolved)\)$/u.exec(line); + if (!match || !["## Direct callers", "## Direct callees"].includes(section)) continue; + const citation = { filePath: match[2], startLine: Number(match[3]), endLine: Number(match[3]) }; + const source = sourceLines(citation).join("\n"); + expect(source).toContain(section === "## Direct callers" ? `${TARGET}(` : `${match[1]}(`); + (section === "## Direct callers" ? callers : callees).push(citation); + } + return { callers, callees }; +} + +function assertEditEvidence(response: CodebaseEditContextResult): SourceCredit | undefined { + const source = targetSourceCredit(response.text); + const neighbors = editNeighbors(response.text); + expect(countContextTokens(response.text)).toBeLessThanOrEqual(response.details.tokenBudget); + expect(response.details.sourceIncluded, "sourceIncluded credits at least one complete source line, not a header").toBe(source !== undefined); + expect(response.details.callerCount, "callerCount credits only complete visible rows").toBe(neighbors.callers.length); + expect(response.details.calleeCount, "calleeCount credits only complete visible rows").toBe(neighbors.callees.length); + return source; +} + +function architectureCitations(text: string): Citation[] { + return outsideSourceLines(text).flatMap((line) => { + const match = /^- Evidence: .+ \((.+):(\d+)\)$/u.exec(line); + return match ? [{ filePath: match[1], startLine: Number(match[2]), endLine: Number(match[2]) }] : []; + }); +} + +function assertArchitectureEvidence(response: ArchitectureContextResult): Citation[] { + const selected = response.modules.flatMap((module) => module.evidence); + const citations = architectureCitations(response.text); + expect(citations.map(identity)).toEqual(selected.map((entry) => identity({ + filePath: entry.filePath, startLine: entry.line, endLine: entry.line, + }))); + for (const entry of selected) { + const declaration = sourceLines({ filePath: entry.filePath, startLine: entry.line, endLine: entry.line })[0]; + expect(declaration).toContain(`function ${entry.symbol}(`); + if (entry.excerpt) { + expect(declaration.replace(/\s+/gu, " ").trim().startsWith(entry.excerpt.replace(/\s+/gu, " ").trim())).toBe(true); + } + } + const responsibilities = [...response.text.matchAll(/^- Source-backed responsibility: (.+) \[(.+) at (.+):(\d+)\]$/gmu)]; + for (const match of responsibilities) { + const entry = selected.find((candidate) => candidate.symbol === match[2] + && candidate.filePath === match[3] && candidate.line === Number(match[4])); + expect(entry, "responsibility must cite retained evidence").toBeDefined(); + expect(match[1]).toBe(entry!.excerpt); + } + expect(responsibilities.length).toBe(response.modules.filter((module) => module.evidence.some((entry) => entry.excerpt)).length); + expect(estimateTokens(response.text)).toBeLessThanOrEqual(response.tokenBudget); + for (const boundary of response.boundaries) { + for (const edge of boundary.evidence) { + fixturePath(edge.fromFilePath); + fixturePath(edge.toFilePath); + expect(response.text).toContain(` - ${edge.fromSymbol} (${edge.fromFilePath}) -> ${edge.toSymbol} (${edge.toFilePath})`); + } + } + for (const hub of response.hubs) { + fixturePath(hub.filePath); + expect(response.text).toContain(`- ${hub.symbol} (${hub.filePath}), ${hub.connections} graph connections`); + } + return citations; +} + +async function writeSource(relative: string, content: string): Promise { + sources.set(relative, content); + await fs.mkdir(path.dirname(path.join(root, relative)), { recursive: true }); + await fs.writeFile(path.join(root, relative), content); +} + +beforeEach(async () => { + root = await fs.mkdtemp(path.join(os.tmpdir(), "context-evidence-acceptance-")); + sources = new Map(); + const config = parseConfig({ + indexing: { mode: "structural", autoIndex: false, watchFiles: false, requireProjectMarker: false }, + search: { minScore: 0 }, + }); + indexer = new Indexer(root, config, "opencode"); + const key = getIndexerCacheKey(root, "opencode"); + configCache.set(key, config); + indexerCache.set(key, indexer); + await writeSource(TARGET_PATH, [ + `export function ${TARGET}(input: string): string {`, + " let envelope = input;", + ...Array.from({ length: 12 }, (_, index) => ` envelope = envelope.replace('é雪🙂-${index}', 'validated-${index}');`), + " return envelope;", + "}", + "", + ].join("\r\n")); + await writeSource(DUPLICATE_PATH, `export function ${TARGET}(input: string): string {\n return 'foreign:' + input;\n}\n`); + await writeSource(HELPER_PATH, "export function normalizeEnvelope(input: string): string {\n return input.trim();\n}\n"); + await writeSource(CALLER_PATH, [ + `import { ${TARGET} } from './${TARGET_PATH.slice(4).replace(/\.ts$/u, ".js")}';`, + "import { normalizeEnvelope } from './normalize.js';", + "export function createTransport(input: string): string {", + ` return ${TARGET}(normalizeEnvelope(input));`, + "}", + "", + ].join("\n")); + await indexer.index(); +}); + +afterEach(async () => { + try { + await indexer?.close(); + } finally { + const key = getIndexerCacheKey(root, "opencode"); + indexerCache.delete(key); + configCache.delete(key); + indexer = undefined; + await fs.rm(root, { recursive: true, force: true }); + } +}); + +describe("native-backed consumer evidence acceptance", () => { + it.each(BUDGETS)("retains only fixture citations in %i-token metadata context", async (tokenBudget) => { + const response = await resolveCodebaseContext(root, "opencode", { + query: TARGET, symbol: TARGET, directory: TARGET_DIRECTORY, tokenBudget, + }); + expect(response.details?.route).toBe("definition"); + const citations = assertContextEvidence(response); + expect(citations.map((citation) => fixturePath(citation.filePath))).toEqual([TARGET_PATH]); + expect(citations[0].startLine).toBe(1); + expect(response.text).not.toContain("envelope.replace("); + }); + + it.each(BUDGETS)("checks CRLF and Unicode source against retained %i-token pre-edit evidence", async (tokenBudget) => { + const response = await resolveCodebaseEditContext(root, "opencode", { + query: `change ${TARGET}`, symbol: TARGET, filePath: TARGET_PATH, tokenBudget, + }); + expect(response.details.resolution).toBe("resolved"); + const source = assertEditEvidence(response); + if (source) { + expect(fixturePath(source.citation.filePath)).toBe(TARGET_PATH); + expect(source.citation.startLine).toBe(1); + } + if (tokenBudget === 128) expect(source?.completeDeclaration ?? false).toBe(false); + if (tokenBudget === 1200) { + expect(source?.completeDeclaration).toBe(true); + expect(response.text).toContain("é雪🙂-0"); + expect(editNeighbors(response.text).callers.map((citation) => fixturePath(citation.filePath))).toEqual([CALLER_PATH]); + } + }); + + it.each(BUDGETS)("fits source and whole test-reference rows with CRLF, Unicode, and long identities at %i tokens", async (tokenBudget) => { + const testPath = "tests/coordinate-preserving-transport/source-linked-envelope-validation-with-a-long-test-file-name.test.ts"; + await writeSource(testPath, [ + `import { ${TARGET} } from '../../${TARGET_PATH.replace(/\.ts$/u, ".js")}';`, + "it('preserves é雪🙂 source coordinates', () => {", + ` expect(${TARGET}('é雪🙂')).toBe('é雪🙂');`, + "});", + "", + ].join("\r\n")); + await indexer!.index(); + const response = await resolveCodebaseEditContext(root, "opencode", { + query: `change ${TARGET}`, symbol: TARGET, filePath: TARGET_PATH, includeTests: true, tokenBudget, + }); + assertEditEvidence(response); + const rows = [...response.text.matchAll(/^- (.+):(\d+)-(\d+) \[(direct_target_test_reference|related_caller_test_reference)\] referencedSymbol=(\S+) referenceLine=(\d+)$/gmu)]; + expect(rows.map((row) => ({ + filePath: row[1], startLine: Number(row[2]), endLine: Number(row[3]), + evidenceKind: row[4], referencedSymbol: row[5], referenceLine: Number(row[6]), + }))).toEqual(response.details.testDiscovery?.results); + for (const reference of response.details.testDiscovery!.results) { + expect(fixturePath(reference.filePath)).toBe(testPath); + expect([reference.startLine, reference.endLine, reference.referenceLine]).toEqual([2, 4, 3]); + expect(sourceLines(reference)[1]).toContain(`${TARGET}('é雪🙂')`); + expect(reference).not.toHaveProperty("content"); + } + expect(response.text).not.toContain("it('preserves"); + expect(response.text.match(/```/gu)?.length ?? 0).toBe(response.details.sourceIncluded ? 2 : 0); + expect(response.details.testDiscovery?.resolution).toBe("resolved"); + expect(response.details.testDiscovery?.omittedReferenceCount).toBe(1 - rows.length); + if (tokenBudget === 1200) expect(rows.map((row) => fixturePath(row[1]))).toEqual([testPath]); + }); + + it.each(BUDGETS)("validates selected architecture locations and normalized excerpts at %i tokens", async (tokenBudget) => { + const response = await getArchitectureContext(root, "opencode", { directory: "src", tokenBudget }); + const citations = assertArchitectureEvidence(response); + if (tokenBudget === 1200) { + expect(citations.map((citation) => fixturePath(citation.filePath))).toContain(TARGET_PATH); + expect(citations.map((citation) => fixturePath(citation.filePath))).toContain(CALLER_PATH); + } + }); + + it("keeps duplicate symbol identity explicit and does not recover outside a strict missing directory", async () => { + const duplicate = await resolveCodebaseContext(root, "opencode", { query: TARGET, symbol: TARGET, tokenBudget: 1200 }); + expect(duplicate.details?.resolution).toBe("ambiguous"); + expect(assertContextEvidence(duplicate).map((citation) => fixturePath(citation.filePath)).sort()).toEqual([TARGET_PATH, DUPLICATE_PATH].sort()); + const edit = await resolveCodebaseEditContext(root, "opencode", { query: `change ${TARGET}`, symbol: TARGET, tokenBudget: 1200 }); + expect(edit.details.resolution).toBe("ambiguous"); + expect(targetSourceCredit(edit.text)).toBeUndefined(); + for (const tokenBudget of BUDGETS) { + const missed = await resolveCodebaseContext(root, "opencode", { + query: TARGET, symbol: TARGET, directory: "src/missing-scope", allowUnscopedRecovery: false, tokenBudget, + }); + expect(missed.details?.results ?? []).toEqual([]); + expect(contextCitations(missed.text)).toEqual([]); + expect(countContextTokens(missed.text)).toBeLessThanOrEqual(tokenBudget); + expect(missed.details?.recovery?.attempts.flatMap((attempt) => attempt.relaxedFields)).toEqual([]); + const architecture = await getArchitectureContext(root, "opencode", { directory: "src/missing-scope", tokenBudget }); + expect(architecture.modules).toEqual([]); + expect(architecture.boundaries).toEqual([]); + expect(architecture.hubs).toEqual([]); + expect(assertArchitectureEvidence(architecture)).toEqual([]); + } + }); + + it("rejects a real graph neighbor omitted from the small response", async () => { + const request = { query: `change ${TARGET}`, symbol: TARGET, filePath: TARGET_PATH }; + const larger = await resolveCodebaseEditContext(root, "opencode", { ...request, tokenBudget: 1200 }); + const smaller = await resolveCodebaseEditContext(root, "opencode", { ...request, tokenBudget: 128 }); + const caller = { filePath: CALLER_PATH, startLine: 4, endLine: 4 }; + requireRetainedCitation(editNeighbors(larger.text).callers, caller); + expect(() => requireRetainedCitation(editNeighbors(smaller.text).callers, caller)).toThrow(); + }); + + it("rejects wrong-range excerpts and paths echoed only in prose or source", async () => { + const response = await resolveCodebaseEditContext(root, "opencode", { + query: `change ${TARGET}`, symbol: TARGET, filePath: TARGET_PATH, tokenBudget: 1200, + }); + const selected = targetSourceCredit(response.text); + expect(selected?.completeDeclaration).toBe(true); + const citation = selected!.citation; + // Keep the real path and body; move to another valid line within the same file. + const wrongRange = response.text.replace(`${citation.filePath}:${citation.startLine}-${citation.endLine}`, `${citation.filePath}:2-${citation.endLine}`); + expect(() => targetSourceCredit(wrongRange)).toThrow(); + const context = await resolveCodebaseContext(root, "opencode", { + query: TARGET, symbol: TARGET, directory: TARGET_DIRECTORY, tokenBudget: 1200, + }); + const actual = contextCitations(context.text)[0]; + requireRetainedCitation(contextCitations(context.text), actual); + const row = outsideSourceLines(context.text).find((line) => line.startsWith("[1] "))!; + const prose = `Related source: ${row}`; + const quotedSource = `\`\`\`ts\n${row}\n\`\`\``; + expect(() => requireRetainedCitation(contextCitations(prose), actual)).toThrow(); + expect(() => requireRetainedCitation(contextCitations(quotedSource), actual)).toThrow(); + }); +}); diff --git a/tests/edit-context-test-evidence.test.ts b/tests/edit-context-test-evidence.test.ts new file mode 100644 index 00000000..5ab89c9b --- /dev/null +++ b/tests/edit-context-test-evidence.test.ts @@ -0,0 +1,464 @@ +import type { CodebaseEditContextEvidence, CodebaseEditContextResult } from "../src/tools/edit-context.js"; +import type { SharedCodebaseEditContextArgs } from "../src/tools/contracts.js"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import * as fs from "node:fs/promises"; +import * as os from "node:os"; +import * as path from "node:path"; +import { Client } from "@modelcontextprotocol/sdk/client/index.js"; +import { InMemoryTransport } from "@modelcontextprotocol/sdk/inMemory.js"; +import { parseConfig } from "../src/config/schema.js"; +import { Indexer } from "../src/indexer/index.js"; +import { createMcpServer } from "../src/mcp-server.js"; +import { codebase_edit_context } from "../src/adapters/opencode/tools.js"; +import { resolveCodebaseContext } from "../src/tools/context.js"; +import { resolveCodebaseEditContext, resolveCodebaseEditContextWithDependencies } from "../src/tools/edit-context.js"; +import { executeCodebaseEditContext } from "../src/tools/execute-common.js"; +import { getCallGraphData, getIndexerForProject, getSymbolTestEvidence, implementationLookup, searchCodebase } from "../src/tools/operations.js"; +import { configCache, defaultProjectRoots, getIndexerCacheKey, indexerCache } from "../src/tools/operation-runtime.js"; +import { countContextTokens } from "../src/tools/context-pack.js"; +import { discoverSymbolTests, TEST_DISCOVERY_MAX_FILE_BYTES } from "../src/tools/test-discovery.js"; +import { OperationCancelledError } from "../src/utils/operation-control.js"; + +const config = parseConfig({ + indexing: { mode: "structural", autoIndex: false, watchFiles: false, requireProjectMarker: false }, + search: { minScore: 0 }, +}); +let root: string; +let indexer: Indexer; + +async function writeSource(relative: string, content: string): Promise { + await fs.mkdir(path.dirname(path.join(root, relative)), { recursive: true }); + await fs.writeFile(path.join(root, relative), content); +} + +beforeEach(async () => { + root = await fs.mkdtemp(path.join(os.tmpdir(), "edit-test-evidence-")); + indexer = new Indexer(root, config, "opencode"); + const key = getIndexerCacheKey(root, "opencode"); + configCache.set(key, config); + indexerCache.set(key, indexer); + await writeSource("src/gate.ts", `export function signalGate(inputs: boolean[]) { + return inputs.every(Boolean); +} +`); + await writeSource("src/transport.ts", `import { signalGate } from './gate.js'; +export function createTransport(inputs: boolean[]) { + return signalGate(inputs); +} +`); + await writeSource("src/foreign.ts", "export function foreignGate(inputs: boolean[]) { return false; }\n"); + await writeSource("tests/z-direct.test.ts", `import { signalGate as aliasGate } from '../src/gate.js'; +it('rejects a false input', () => { + expect(aliasGate([true, false])).toBe(false); +}); +`); + await writeSource("tests/a-related.test.ts", `import { createTransport } from '../src/transport.js'; +it('forwards gate behavior', () => { + expect(createTransport([true, false])).toBe(false); +}); +`); + await writeSource("tests/decoy.test.ts", `import { foreignGate as signalGate } from '../src/foreign.js'; +it('uses a foreign import', () => { + expect(signalGate([false])).toBe(false); +}); +it('contains only spelling', () => { + // signalGate([false]); + expect('signalGate([false])').toContain('signalGate'); +}); +`); +}); + +afterEach(async () => { + await indexer.close(); + const key = getIndexerCacheKey(root, "opencode"); + indexerCache.delete(key); + configCache.delete(key); + await fs.rm(root, { recursive: true, force: true }); +}); + +function request(args: Partial = {}): Promise { + return resolveCodebaseEditContext(root, "opencode", { + query: "change signalGate", symbol: "signalGate", filePath: "src/gate.ts", + includeTests: true, tokenBudget: 1200, ...args, + }); +} + +async function assertRenderedReferences(response: CodebaseEditContextResult): Promise { + const rows = [...response.text.matchAll(/^- (.+):(\d+)-(\d+) \[(direct_target_test_reference|related_caller_test_reference)\] referencedSymbol=(\S+) referenceLine=(\d+)$/gmu)]; + const references = response.details.testDiscovery!.results; + expect(rows.map((row) => ({ + filePath: row[1], startLine: Number(row[2]), endLine: Number(row[3]), + evidenceKind: row[4], referencedSymbol: row[5], referenceLine: Number(row[6]), + }))).toEqual(references.map(({ filePath, startLine, endLine, evidenceKind, referencedSymbol, referenceLine }) => ({ + filePath, startLine, endLine, evidenceKind, referencedSymbol, referenceLine, + }))); + for (const reference of references) { + const lines = (await fs.readFile(reference.filePath, "utf8")).split("\n"); + expect(reference.startLine).toBe(2); + expect(reference.endLine).toBe(4); + expect(reference.referenceLine).toBe(3); + expect(lines[reference.referenceLine - 1]).toMatch(/\b(?:aliasGate|createTransport)\(/u); + expect(reference).not.toHaveProperty("content"); + } + expect(countContextTokens(response.text)).toBeLessThanOrEqual(response.details.tokenBudget); +} + +describe("native-backed pre-edit test evidence", () => { + it("renders alias-proved direct tests before graph and related tests, excluding name-only and foreign decoys", async () => { + await indexer.index(); + const response = await request(); + expect(response.details.testDiscovery?.resolution).toBe("resolved"); + expect(response.details.testDiscovery?.results.map((result) => [path.relative(root, result.filePath), result.evidenceKind])).toEqual([ + ["tests/z-direct.test.ts", "direct_target_test_reference"], + ["tests/a-related.test.ts", "related_caller_test_reference"], + ]); + expect(response.details.sourceIncluded).toBe(true); + expect(response.text).toContain("Static source references; not runtime coverage."); + expect(response.text.indexOf("direct_target_test_reference")).toBeLessThan(response.text.indexOf("## Direct callers")); + expect(response.text.indexOf("## Direct callees")).toBeLessThan(response.text.indexOf("related_caller_test_reference")); + expect(response.text).not.toContain("decoy.test.ts"); + await assertRenderedReferences(response); + }); + + it.each([128, 600, 1200])("credits only whole rendered rows at %i tokens", async (tokenBudget) => { + await indexer.index(); + const proved = await getSymbolTestEvidence(root, "opencode", "signalGate", { targetFilePath: "src/gate.ts" }); + let selected: CodebaseEditContextEvidence | undefined; + const response = await resolveCodebaseEditContextWithDependencies({ + query: "change signalGate", symbol: "signalGate", filePath: "src/gate.ts", includeTests: true, tokenBudget, + }, { + projectRoot: root, + onSelectedEvidence: (evidence) => { selected = evidence; }, + searchCodebase: (query, options, control) => searchCodebase(root, "opencode", query, options, control), + implementationLookup: (query, options, control) => implementationLookup(root, "opencode", query, options, control), + getCallGraphData: (params, control) => getCallGraphData(root, "opencode", params, control), + getTestEvidence: (target, control) => getSymbolTestEvidence(root, "opencode", target.symbol, { targetFilePath: target.filePath }, control), + }); + await assertRenderedReferences(response); + expect(selected?.testReferences).toEqual(response.details.testDiscovery?.results); + for (const reference of selected!.testReferences) { + expect(reference).not.toHaveProperty("content"); + } + expect(response.details.testDiscovery?.omittedReferenceCount).toBe(proved.results.length - selected!.testReferences.length); + expect(response.details.callerCount).toBe(selected!.callers.length); + expect(response.details.calleeCount).toBe(selected!.callees.length); + expect(response.details.sourceIncluded).toBe(selected!.sources.length > 0); + if (tokenBudget === 128) expect(response.details.testDiscovery?.omittedReferenceCount).toBeGreaterThan(0); + expect(response.text.match(/```/gu)?.length ?? 0).toBe(response.details.sourceIncluded ? 2 : 0); + }); + + it("preserves the omitted/false route and performs no test-discovery inventory work", async () => { + await indexer.index(); + const inventory = vi.spyOn(indexer, "getIndexedFilePathsForActiveBranch"); + try { + const omitted = await request({ includeTests: undefined }); + const disabled = await request({ includeTests: false }); + expect(omitted).toEqual(disabled); + expect(disabled.details.sourceIncluded).toBe(true); + expect(disabled.details.testDiscovery).toBeUndefined(); + expect(disabled.text).not.toContain("Source-linked tests"); + expect(inventory).not.toHaveBeenCalled(); + } finally { + inventory.mockRestore(); + } + }); + + it("uses the selected host project root when the caller omits projectRoot", async () => { + await indexer.index(); + const previous = defaultProjectRoots.get("opencode"); + defaultProjectRoots.set("opencode", root); + try { + const response = await resolveCodebaseEditContext(undefined, "opencode", { + query: "change signalGate", symbol: "signalGate", filePath: "src/gate.ts", tokenBudget: 1200, + }); + expect(response.details.sourceIncluded).toBe(true); + expect(response.text).toContain("return inputs.every(Boolean);"); + } finally { + if (previous === undefined) defaultProjectRoots.delete("opencode"); + else defaultProjectRoots.set("opencode", previous); + } + }); + + it.each([ + { file: "src/multiline.ts", symbol: "multilineGate", source: "export\nfunction multilineGate() {\n return 1;\n}\n" }, + { file: "src/decorated.py", symbol: "decorated_gate", source: "@decorator\ndef decorated_gate():\n return 1\n" }, + ])("accepts native export/decorator declaration evidence for $symbol", async ({ file, symbol, source }) => { + await writeSource(file, source); + await indexer.index(); + const response = await request({ symbol, filePath: file }); + expect(response.details.resolution).toBe("resolved"); + expect(response.details.sourceIncluded).toBe(true); + expect(response.text).toContain("return 1"); + }); + + it("starts a large, fragmented declaration at its canonical signature rather than a ranked body slice", async () => { + await writeSource("src/large-gate.ts", [ + "export function largeGate() {", + " let total = 0;", + ...Array.from({ length: 240 }, (_, index) => ` total += ${index};`), + " return total;", + "}", + ].join("\n")); + await indexer.index(); + const response = await request({ symbol: "largeGate", filePath: "src/large-gate.ts" }); + expect(response.details.sourceIncluded).toBe(true); + expect(response.text).toContain("export function largeGate()"); + expect(response.text).toContain(" let total = 0;"); + expect(countContextTokens(response.text)).toBeLessThanOrEqual(1200); + }); + + it("withholds missing canonical source without treating an inaccessible-file marker as implementation", async () => { + await indexer.index(); + await fs.rm(path.join(root, "src/gate.ts")); + const response = await request(); + expect(response.details.sourceIncluded).toBe(false); + expect(response.text).not.toContain("[File not accessible]"); + }); + + it.each([false, true])("selects the exact source file rather than a suffix match with includeTests=%s", async (includeTests) => { + await writeSource("nested/src/gate.ts", "export function signalGate(inputs: boolean[]) { return 'foreign-source'; }\n"); + await indexer.index(); + const definitions = await implementationLookup(root, "opencode", "signalGate", { limit: 10, exactSymbol: true }); + const foreign = definitions.find((definition) => definition.filePath.endsWith("/nested/src/gate.ts"))!; + const target = definitions.find((definition) => definition.filePath === path.join(root, "src/gate.ts"))!; + expect(foreign).toBeDefined(); + expect(target).toBeDefined(); + const response = await resolveCodebaseEditContextWithDependencies({ + query: "change signalGate", symbol: "signalGate", filePath: path.join(root, "src/gate.ts"), includeTests, tokenBudget: 1200, + }, { + projectRoot: root, + searchCodebase: (query, options, control) => searchCodebase(root, "opencode", query, options, control), + implementationLookup: async () => [foreign, target], + getCallGraphData: (params, control) => getCallGraphData(root, "opencode", params, control), + getTestEvidence: (selected, control) => getSymbolTestEvidence(root, "opencode", selected.symbol, { targetFilePath: selected.filePath }, control), + }); + expect(response.details.sourceIncluded).toBe(true); + expect(response.text).toContain("return inputs.every(Boolean);"); + expect(response.text).not.toContain("foreign-source"); + }); + + it("withholds source credit when lookup returns only an enclosing declaration", async () => { + await writeSource("src/wrapped.ts", [ + "export class SignalOwner {", + " signalGate(inputs: boolean[]) {", + " return inputs.every(Boolean);", + " }", + "}", + "", + ].join("\n")); + await indexer.index(); + const enclosing = await implementationLookup(root, "opencode", "SignalOwner", { limit: 10 }); + expect(enclosing.map((definition) => definition.name)).toContain("SignalOwner"); + const response = await resolveCodebaseEditContextWithDependencies({ + query: "change signalGate", symbol: "signalGate", filePath: "src/wrapped.ts", tokenBudget: 1200, + }, { + projectRoot: root, + searchCodebase: (query, options, control) => searchCodebase(root, "opencode", query, options, control), + implementationLookup: async () => enclosing, + getCallGraphData: (params, control) => getCallGraphData(root, "opencode", params, control), + }); + expect(response.details.resolution).toBe("resolved"); + expect(response.details.sourceIncluded).toBe(false); + expect(response.text).not.toContain("export class SignalOwner"); + expect(response.text).not.toContain("return inputs.every(Boolean);"); + }); + + it("uses targetFilePath only to disambiguate the target, not to restrict external test directories", async () => { + await writeSource("src/duplicate.ts", "export function signalGate(inputs: boolean[]) { return true; }\n"); + await indexer.index(); + const response = await request(); + expect(response.details.testDiscovery?.results.map((result) => path.relative(root, result.filePath))).toEqual([ + "tests/z-direct.test.ts", "tests/a-related.test.ts", + ]); + const ambiguous = await request({ filePath: undefined }); + expect(ambiguous.details.resolution).toBe("ambiguous"); + expect(ambiguous.details.testDiscovery?.resolution).toBe("ambiguous"); + expect(ambiguous.details.testDiscovery?.results).toEqual([]); + const scoped = await getSymbolTestEvidence(root, "opencode", "signalGate", { + targetFilePath: path.join(root, "src/gate.ts"), directory: "tests", fileType: "ts", + }); + expect(scoped.results.map((result) => result.evidenceKind)).toEqual([ + "direct_target_test_reference", "related_caller_test_reference", + ]); + const ordinary = await resolveCodebaseContext(root, "opencode", { + query: "tests for signalGate", directory: "tests", tokenBudget: 1200, + }); + expect(ordinary.details?.testDiscovery?.resolution).toBe("ambiguous"); + await expect(getSymbolTestEvidence(root, "opencode", "signalGate", { targetFilePath: "../escape.ts" })) + .rejects.toThrow("inside the project root"); + }); + + it.skipIf(process.platform === "win32")("normalizes canonical target paths and rejects a symlink escape before discovery", async () => { + await indexer.index(); + const canonical = await getSymbolTestEvidence(root, "opencode", "signalGate", { + targetFilePath: await fs.realpath(path.join(root, "src/gate.ts")), + }); + expect(canonical.resolution).toBe("resolved"); + expect(canonical.results.map((result) => result.evidenceKind)).toEqual([ + "direct_target_test_reference", "related_caller_test_reference", + ]); + const outside = await fs.mkdtemp(path.join(os.tmpdir(), "edit-test-outside-")); + try { + const externalTarget = path.join(outside, "gate.ts"); + await fs.writeFile(externalTarget, "export function signalGate() { return false; }\n"); + await fs.symlink(externalTarget, path.join(root, "src/escape.ts")); + await expect(getSymbolTestEvidence(root, "opencode", "signalGate", { targetFilePath: "src/escape.ts" })) + .rejects.toThrow("inside the project root"); + } finally { + await fs.rm(outside, { recursive: true, force: true }); + } + }); + + it("fails closed on stale target coordinates rather than crediting a new same-name declaration", async () => { + await indexer.index(); + await writeSource("src/gate.ts", "// coordinate drift\nexport function signalGate(inputs: boolean[]) { return false; }\n"); + const response = await request(); + expect(response.details.testDiscovery?.resolution).toBe("ambiguous"); + expect(response.details.testDiscovery?.results).toEqual([]); + expect(response.details.testDiscovery?.capped).toBe(true); + }); + + it("reports unsupported and non-authoritative targets without claiming tests are absent", async () => { + await writeSource("src/math.py", "def calculate(value):\n return value + 1\n"); + await indexer.index(); + const unsupported = await request({ symbol: "calculate", filePath: "src/math.py" }); + expect(unsupported.details.resolution).toBe("resolved"); + expect(unsupported.details.testDiscovery?.resolution).toBe("unsupported"); + expect(unsupported.details.testDiscovery?.results).toEqual([]); + const conceptual = await request({ symbol: undefined }); + expect(conceptual.details.testDiscovery?.resolution).toBe("not_requested"); + expect(conceptual.details.testDiscovery?.results).toEqual([]); + expect(unsupported.text).toContain("Static source references; not runtime coverage."); + expect(unsupported.text).not.toContain("No tests"); + }); + + it("retains the 20-reference discovery cap separately from response omissions", async () => { + await writeSource("tests/z-direct.test.ts", `import { signalGate } from '../src/gate.js';\n${Array.from({ length: 24 }, (_, index) => `it('case ${index}', () => {\n expect(signalGate([false])).toBe(false);\n});\n`).join("")}`); + await indexer.index(); + const proved = await getSymbolTestEvidence(root, "opencode", "signalGate", { targetFilePath: "src/gate.ts" }); + expect(proved.results).toHaveLength(20); + expect(proved.capped).toBe(true); + const response = await request({ tokenBudget: 600 }); + expect(response.details.testDiscovery?.capped).toBe(true); + expect(response.details.testDiscovery?.omittedReferenceCount).toBe(20 - response.details.testDiscovery!.results.length); + expect(response.details.testDiscovery?.omittedReferenceCount).toBeGreaterThan(0); + expect(countContextTokens(response.text)).toBeLessThanOrEqual(600); + }); + + it("does not reinterpret a saturated canonical lookup as unique after file filtering", async () => { + for (let index = 0; index < 99; index += 1) { + await writeSource(`duplicates/gate-${index}.ts`, "export function signalGate(inputs: boolean[]) { return false; }\n"); + } + await indexer.index(); + const response = await request(); + expect(response.details.resolution).toBe("resolved"); + expect(response.details.testDiscovery?.resolution).toBe("ambiguous"); + expect(response.details.testDiscovery?.capped).toBe(true); + expect(response.details.testDiscovery?.results).toEqual([]); + }); + + it("keeps oversized test files out of evidence and exposes discovery incompleteness", async () => { + await indexer.index(); + await writeSource("tests/z-direct.test.ts", `//${"x".repeat(TEST_DISCOVERY_MAX_FILE_BYTES)}\n`); + const response = await request(); + expect(response.details.testDiscovery?.capped).toBe(true); + expect(response.details.testDiscovery?.results.map((result) => path.relative(root, result.filePath))).toEqual(["tests/a-related.test.ts"]); + expect(response.details.testDiscovery?.omittedReferenceCount).toBe(0); + }); + + it("propagates cancellation from inside discovery rather than returning partial selected evidence", async () => { + await indexer.index(); + const controller = new AbortController(); + const inventory = vi.spyOn(indexer, "getIndexedFilePathsForActiveBranch").mockImplementation(async () => { + controller.abort(); + return []; + }); + try { + await expect(resolveCodebaseEditContext(root, "opencode", { + query: "change signalGate", symbol: "signalGate", filePath: "src/gate.ts", includeTests: true, + }, { signal: controller.signal })).rejects.toBeInstanceOf(OperationCancelledError); + } finally { + inventory.mockRestore(); + } + }); + + it("rejects cancellation during an empty definition lookup instead of returning not_found", async () => { + await indexer.index(); + const controller = new AbortController(); + await expect(discoverSymbolTests(root, "missingGate", {}, { + lookup: async (symbol) => { + const definitions = await implementationLookup(root, "opencode", symbol, { limit: 100 }); + controller.abort(); + return definitions; + }, + graph: (symbol, filePath) => getCallGraphData(root, "opencode", { name: symbol, filePath, direction: "callers" }), + indexedFiles: () => indexer.getIndexedFilePathsForActiveBranch(), + lexicalFiles: async () => { throw new Error("Cancellation must precede test inventory ranking."); }, + }, { signal: controller.signal })).rejects.toBeInstanceOf(OperationCancelledError); + }); + + it("returns unavailable, not fabricated empty resolved discovery, after an operational failure", async () => { + await indexer.index(); + const inventory = vi.spyOn(indexer, "getIndexedFilePathsForActiveBranch").mockRejectedValue(new Error("inventory unavailable")); + try { + const response = await request(); + expect(response.details.testDiscovery?.resolution).toBe("unavailable"); + expect(response.details.testDiscovery?.results).toEqual([]); + expect(response.details.sourceIncluded).toBe(true); + expect(response.text).toContain("Discovery: unavailable"); + } finally { + inventory.mockRestore(); + } + }); + + it("keeps the OpenCode host text-valued while rendering the same native-proved test citations", async () => { + await indexer.index(); + const args = { + query: "change signalGate", symbol: "signalGate", filePath: "src/gate.ts", + includeTests: true, tokenBudget: 600, includeApiImpact: false, + callerLimit: null, calleeLimit: null, + }; + const shared = await resolveCodebaseEditContext(root, "opencode", args); + const context = { + worktree: root, abort: new AbortController().signal, + } as Parameters[1]; + const text = await codebase_edit_context.execute(args, context); + expect(typeof text).toBe("string"); + expect(text).toBe(shared.text); + expect(text).toContain("direct_target_test_reference"); + await assertRenderedReferences({ text, details: shared.details }); + }); + + it("preserves selected shared details through the real MCP text and structured-content boundary", async () => { + await indexer.index(); + const shared = await executeCodebaseEditContext(root, "opencode", { + query: "change signalGate", symbol: "signalGate", filePath: "src/gate.ts", includeTests: true, tokenBudget: 600, + }); + expect(shared.details?.testDiscovery).toEqual(expect.objectContaining({ resolution: "resolved" })); + await indexer.close(); + const server = createMcpServer(root, config, "opencode"); + indexer = getIndexerForProject(root, "opencode"); + const client = new Client({ name: "edit-test-evidence", version: "1.0.0" }); + const [clientTransport, serverTransport] = InMemoryTransport.createLinkedPair(); + try { + await server.connect(serverTransport); + await client.connect(clientTransport); + const tools = await client.listTools(); + expect(tools.tools.find((tool) => tool.name === "codebase_edit_context")?.inputSchema.properties?.includeTests) + .toEqual(expect.objectContaining({ type: "boolean", default: false })); + const result = await client.callTool({ + name: "codebase_edit_context", + arguments: { query: "change signalGate", symbol: "signalGate", filePath: "src/gate.ts", includeTests: true, tokenBudget: 600 }, + }); + expect(result.isError).not.toBe(true); + const content = result.content as Array<{ type: string; text: string }>; + const details = result.structuredContent as CodebaseEditContextResult["details"]; + expect(content[0].text).toBe(shared.text); + expect(details.testDiscovery).toEqual(shared.details?.testDiscovery); + await assertRenderedReferences({ text: content[0].text, details }); + } finally { + await client.close(); + await server.close(); + } + }); +}); diff --git a/tests/edit-context.test.ts b/tests/edit-context.test.ts index f135f743..290a76ad 100644 --- a/tests/edit-context.test.ts +++ b/tests/edit-context.test.ts @@ -240,7 +240,7 @@ describe("codebase edit context", () => { expect(result.text).toContain("Risk: graph data is unavailable"); expect(result.text).toContain("src/auth.ts"); expect(result.details.resolution).toBe("graph_unavailable"); - expect(result.details.sourceIncluded).toBe(true); + expect(result.details.sourceIncluded).toBe(false); }); it("returns a risk note and conceptual evidence when the symbol is unresolved", async () => { @@ -266,6 +266,40 @@ describe("codebase edit context", () => { expect(result.details.resolution).toBe("not_found"); }); + it.each([16, 64, 96])("keeps a closed, whole-line source block after final fitting with a %i-character identity", async (length) => { + const name = `gate${"x".repeat(length)}`; + const content = [ + `function ${name}(envelope: string) {`, + ...Array.from({ length: 20 }, (_, index) => ` const value${index} = "é雪🙂-${index}";`), + " return envelope;", + "}", + ].join("\r\n"); + operationMocks.getCallGraphData.mockImplementation(async (_root, _host, args) => ({ + direction: args.direction, resolution: { ...resolved, name }, + callers: [{ + id: "caller", fromSymbolId: "sym_caller", fromSymbolName: "handleRequest", + fromSymbolFilePath: "src/handler.ts", targetName: name, + callType: "Call", confidence: "Direct", line: 42, col: 2, isResolved: true, + }], + callees: [{ + id: "callee", fromSymbolId: resolved.symbolId, targetName: "parseToken", toSymbolId: "sym_parse", + callType: "Call", confidence: "Direct", line: 14, col: 2, isResolved: true, + }], + })); + operationMocks.implementationLookup.mockResolvedValue([{ ...source, name, content }]); + const response = await resolveCodebaseEditContext("/repo", "jcode", { + query: `change ${name}`, symbol: name, tokenBudget: 128, + }); + const block = /\n```\n([\s\S]*?)\n```/u.exec(response.text); + expect(response.text.match(/```/gu)?.length ?? 0).toBe(response.details.sourceIncluded ? 2 : 0); + expect(response.details.sourceIncluded).toBe(Boolean(block?.[1])); + if (block) { + const lines = block[1].split("\n"); + expect(lines).toEqual(content.split("\r\n").slice(0, lines.length)); + } + expect(countContextTokens(response.text)).toBeLessThanOrEqual(128); + }); + it("does not format resolved evidence after the operation is cancelled", async () => { const controller = new AbortController(); operationMocks.getCallGraphData.mockImplementation(async (_root, _host, args) => ({ diff --git a/tests/eval-runner.test.ts b/tests/eval-runner.test.ts index cf2c4ab7..ce261c85 100644 --- a/tests/eval-runner.test.ts +++ b/tests/eval-runner.test.ts @@ -391,9 +391,14 @@ describe("eval runner", () => { }); it("grades only graph neighbors retained after the final source and graph budget", async () => { + // This identity alone exceeds 128 tokens, independent of the temporary root's length. + const callerName = `evaluateRanking${"Linked".repeat(128)}`; writeFileSync(path.join(tempDir, "src", "indexer", "index.ts"), [ - `export function rankHybridResults(query: string) { /* ${"ranking ".repeat(250)} */ return query.length; }`, - "export function evaluateRanking() { return rankHybridResults('query'); }", + "export function rankHybridResults(query: string) {", + ` /* ${"ranking ".repeat(250)} */`, + " return query.length;", + "}", + `export function ${callerName}() { return rankHybridResults('query'); }`, ].join("\n")); writeFileSync(path.join(tempDir, "benchmarks", "golden", "edit-graph-budget.json"), JSON.stringify({ version: "1.0.0", name: "edit-graph-budget", @@ -404,7 +409,7 @@ describe("eval runner", () => { args: { symbol: "rankHybridResults", tokenBudget }, expected: { filePath: "src/indexer/index.ts", symbol: "rankHybridResults", - graphNeighbor: { direction: "caller", filePath: "src/indexer/index.ts", symbol: "evaluateRanking" }, + graphNeighbor: { direction: "caller", filePath: "src/indexer/index.ts", symbol: callerName }, }, })), })); @@ -417,7 +422,7 @@ describe("eval runner", () => { expect(result.perQuery[0].responseTokens).toBeLessThanOrEqual(128); expect(result.perQuery[1]).toMatchObject({ hitAt1: true, graphNeighborMatched: true }); expect(result.perQuery[1].results).toEqual(expect.arrayContaining([ - expect.objectContaining({ name: "evaluateRanking", graphDirection: "caller" }), + expect.objectContaining({ name: callerName, graphDirection: "caller" }), ])); }); diff --git a/tests/index-status-freshness.test.ts b/tests/index-status-freshness.test.ts new file mode 100644 index 00000000..5046f9d1 --- /dev/null +++ b/tests/index-status-freshness.test.ts @@ -0,0 +1,324 @@ +import type { HostMode } from "../src/config/host.js"; +import type { ExtensionAPI } from "@earendil-works/pi-coding-agent"; +import { execFileSync } from "node:child_process"; +import { createHash } from "node:crypto"; +import * as fs from "node:fs"; +import * as os from "node:os"; +import * as path from "node:path"; +import { Client } from "@modelcontextprotocol/sdk/client/index.js"; +import { InMemoryTransport } from "@modelcontextprotocol/sdk/inMemory.js"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; + +import { runCbiCli } from "../src/adapters/cbi.js"; +import { index_status } from "../src/adapters/opencode/tools.js"; +import { createMcpServer } from "../src/mcp-server.js"; +import { parseConfig } from "../src/config/schema.js"; +import { Indexer } from "../src/indexer/index.js"; +import { Database, hashContent } from "../src/native/index.js"; +import codebaseIndexOmpExtension from "../src/omp-extension.js"; +import codebaseIndexPiExtension from "../src/pi-extension.js"; +import { executeIndexStatus } from "../src/tools/execute-common.js"; +import { getIndexStatus } from "../src/tools/operations.js"; +import { + configCache, + defaultProjectRoots, + getIndexerCacheKey, + getIndexerForProject, + indexerCache, + initializeTools, +} from "../src/tools/operation-runtime.js"; +import { OperationCancelledError } from "../src/utils/operation-control.js"; +import { stopAutoIndex } from "../src/utils/auto-index.js"; + +const structuralConfig = () => parseConfig({ + indexing: { mode: "structural", autoIndex: false, watchFiles: false, autoGc: false, requireProjectMarker: false }, + include: ["src/**/*.ts"], + exclude: [], + search: { minScore: 0 }, +}); +const canRestrictAccess = process.platform !== "win32" && process.getuid?.() !== 0; + +describe("read-only shared status freshness", () => { + let root: string; + const readers = new Set(); + const hosts = new Set(); + const git = (...args: string[]): string => execFileSync("git", args, { cwd: root, encoding: "utf8", stdio: ["ignore", "pipe", "pipe"] }).trim(); + const sourcePath = () => path.join(root, "src", "gate.ts"); + function database(indexer: Indexer): Database { + // Fixture-only access permits corrupting persisted migration metadata without faking status. + const internals = indexer as unknown as { database: Database }; + return internals.database; + } + + function setup(host: HostMode = "pi", config = structuralConfig()): Indexer { + initializeTools(root, config, host); + hosts.add(host); + const indexer = getIndexerForProject(root, host); + readers.add(indexer); + return indexer; + } + + function artifactHashes(indexPath: string): Record { + const result: Record = {}; + for (const name of fs.readdirSync(indexPath).sort()) { + // Runtime diagnostics/logs are not indexed source artifacts; SQLite SHM is transient reader state. + if (name.endsWith("-shm") || name.includes("log") || name.startsWith("mcp-")) continue; + const file = path.join(indexPath, name); + if (fs.statSync(file).isFile()) result[name] = createHash("sha256").update(fs.readFileSync(file)).digest("hex"); + } + return result; + } + + beforeEach(() => { + root = fs.mkdtempSync(path.join(os.tmpdir(), "ocbi-status-freshness-")); + git("init", "-q", "-b", "main"); + git("config", "user.email", "status@example.com"); + git("config", "user.name", "Status Test"); + git("config", "gc.auto", "0"); + git("config", "maintenance.auto", "false"); + fs.mkdirSync(path.join(root, "src")); + fs.writeFileSync(sourcePath(), "export function signalGate() {\n return 'original';\n}\n"); + fs.writeFileSync(path.join(root, "src", "transport.ts"), "import { signalGate } from './gate.js';\nexport function createTransport() { return signalGate(); }\n"); + git("add", "src"); + git("commit", "-q", "-m", "fixture"); + }); + + afterEach(async () => { + vi.restoreAllMocks(); + for (const host of hosts) { + await stopAutoIndex(root, host); + const key = getIndexerCacheKey(root, host); + const cached = indexerCache.get(key); + if (cached) readers.add(cached); + indexerCache.delete(key); + configCache.delete(key); + if (defaultProjectRoots.get(host) === root) defaultProjectRoots.delete(host); + } + for (const indexer of readers) await indexer.close(); + readers.clear(); + hosts.clear(); + fs.rmSync(root, { recursive: true, force: true }); + }); + + it.each(["opencode", "codex", "claude", "pi", "jcode"] as const)("separates catalog readiness from edited source without refreshing %s artifacts", async (host) => { + const indexer = setup(host); + await indexer.index(); + const freshnessCheck = vi.spyOn(indexer, "getIndexFreshness"); + const before = await getIndexStatus(root, host); + const hashes = artifactHashes(before.indexPath); + expect(before).toMatchObject({ indexed: true, mode: "structural", provider: "none", freshness: null }); + expect(freshnessCheck).not.toHaveBeenCalled(); + const checked = await executeIndexStatus(root, host, { checkFreshness: true }); + expect(checked.details).toMatchObject({ freshness: { readable: true, current: true, reason: "current" } }); + expect(freshnessCheck).toHaveBeenCalledTimes(1); + + fs.writeFileSync(sourcePath(), "export function signalGate() {\n return 'changed-source';\n}\n"); + const plain = await executeIndexStatus(root, host, { checkFreshness: false }); + expect(plain.details).toMatchObject({ indexed: true, freshness: null }); + expect(plain.text).toContain("Source freshness: not checked"); + expect(freshnessCheck).toHaveBeenCalledTimes(1); + const stale = await executeIndexStatus(root, host, { checkFreshness: true }); + expect(stale.details).toMatchObject({ indexed: true, freshness: { readable: true, current: false, reason: "files-changed" } }); + expect(stale.text).toContain("Source freshness: not current (reason: files-changed)"); + expect(artifactHashes(before.indexPath)).toEqual(hashes); + + await indexer.index(); + expect((await getIndexStatus(root, host, { checkFreshness: true })).freshness?.reason).toBe("current"); + expect((await indexer.search("signalGate", 5, { definitionIntent: true }))[0]?.content).toContain("changed-source"); + }); + + it("reports a missing catalog without treating not-checked as current", async () => { + setup(); + expect(await getIndexStatus(root, "pi")).toMatchObject({ indexed: false, freshness: null }); + expect(await getIndexStatus(root, "pi", { checkFreshness: true })).toMatchObject({ + indexed: false, freshness: { readable: false, current: false, reason: "missing" }, + }); + }); + + it("keeps stopped coordination separate from current bytes and detects module configuration changes", async () => { + const indexer = setup(); + await indexer.index(); + await stopAutoIndex(root, "pi"); + expect(await getIndexStatus(root, "pi", { checkFreshness: true })).toMatchObject({ + indexed: true, autoIndex: { state: "stopped" }, freshness: { readable: true, current: true, reason: "current" }, + }); + fs.writeFileSync(path.join(root, "tsconfig.json"), JSON.stringify({ + compilerOptions: { baseUrl: ".", paths: { "@fixture/*": ["src/*"] } }, + })); + expect(await getIndexStatus(root, "pi", { checkFreshness: true })).toMatchObject({ + autoIndex: { state: "stopped" }, freshness: { readable: true, current: false, reason: "metadata-changed" }, + }); + }); + + it("retains migration and failed-batch facts without repairing or retrying them", async () => { + const indexer = setup(); + await indexer.index(); + const initial = await getIndexStatus(root, "pi"); + const marker = `index.callGraphResolutionVersion.${hashContent("main").slice(0, 24)}`; + const storedVersion = database(indexer).getMetadata(marker)!; + database(indexer).deleteMetadata(marker); + const migration = await getIndexStatus(root, "pi", { checkFreshness: true }); + expect(migration).toMatchObject({ indexed: true, migrationRequired: true, freshness: { readable: true, current: false, reason: "migration-required" } }); + expect(database(indexer).getMetadata(marker)).toBeNull(); + const writer = new Database(path.join(initial.indexPath, "codebase.db")); + try { writer.setMetadata(marker, storedVersion); } + finally { writer.close(); } + + // Fixture-only access targets the branch-scoped retry artifact without changing production visibility. + const internals = indexer as unknown as { failedBatchesPath: string }; + const failedPath = internals.failedBatchesPath; + const failedBytes = JSON.stringify([{ chunks: [{ id: "pending-gate", metadata: { filePath: sourcePath() } }], error: "fixture provider failure", attemptCount: 1, lastAttempt: "2026-01-01T00:00:00.000Z" }]); + fs.writeFileSync(failedPath, failedBytes); + expect(await getIndexStatus(root, "pi", { checkFreshness: true })).toMatchObject({ + indexed: true, failedBatchesCount: 1, freshness: { readable: true, current: false, reason: "failed-batches" }, + }); + expect(fs.readFileSync(failedPath, "utf8")).toBe(failedBytes); + }); + + it("checks commit identity and never substitutes another branch's source", async () => { + const indexer = setup(); + await indexer.index(); + git("commit", "-q", "--allow-empty", "-m", "new head"); + expect((await getIndexStatus(root, "pi", { checkFreshness: true })).freshness?.reason).toBe("branch-changed"); + git("checkout", "-q", "-b", "unindexed-feature"); + const missing = await getIndexStatus(root, "pi", { checkFreshness: true }); + expect(missing).toMatchObject({ indexed: false, currentBranch: "unindexed-feature", branchReadiness: { state: "missing" }, freshness: { current: false, reason: "missing" } }); + expect(await indexer.search("signalGate", 5, { definitionIntent: true })).toEqual([]); + }); + + it.runIf(canRestrictAccess)("reports unreadable source without calling it current or rebuilding", async () => { + const indexer = setup(); + await indexer.index(); + fs.chmodSync(sourcePath(), 0o000); + try { + const result = await executeIndexStatus(root, "pi", { checkFreshness: true }); + expect(result.details).toMatchObject({ freshness: { readable: false, current: false, reason: "unreadable" } }); + expect(result.text).toContain("Resolve source/index access and readability before running index_codebase"); + } finally { + fs.chmodSync(sourcePath(), 0o600); + } + }); + + it("reports incompatible hybrid metadata without requesting embeddings during inspection", async () => { + const base = structuralConfig(); + const config = parseConfig({ + ...base, + embeddingProvider: "custom", + customProvider: { baseUrl: "http://127.0.0.1:1/v1", model: "status-model-original", dimensions: 8 }, + indexing: { ...base.indexing, mode: "hybrid" }, + }); + const fetch = vi.spyOn(globalThis, "fetch").mockImplementation(async (_url, init) => { + const { input } = JSON.parse(String(init?.body)) as { input: string[] }; + return new Response(JSON.stringify({ + data: input.map(() => ({ embedding: [1, 0, 0, 0, 0, 0, 0, 0] })), + usage: { total_tokens: input.length }, + })); + }); + const writer = setup("pi", config); + await writer.index(); + await writer.close(); + readers.delete(writer); + const changedConfig = parseConfig({ ...config, customProvider: { ...config.customProvider, model: "status-model-changed" } }); + setup("pi", changedConfig); + fetch.mockClear(); + const status = await getIndexStatus(root, "pi", { checkFreshness: true }); + expect(status).toMatchObject({ + indexed: true, mode: "hybrid", compatibility: { compatible: false }, + freshness: { readable: true, current: false, reason: "incompatible" }, + }); + expect(fetch).not.toHaveBeenCalled(); + }); + + it("keeps OpenCode text-valued while honoring the explicit freshness option", async () => { + const indexer = setup("opencode"); + await indexer.index(); + // Only the context fields read by the real status adapter are needed for this host invocation. + const context = { worktree: root, abort: new AbortController().signal } as Parameters[1]; + const plain = await index_status.execute({ checkFreshness: false }, context); + expect(typeof plain).toBe("string"); + expect(plain).toContain("Source freshness: not checked"); + fs.appendFileSync(sourcePath(), "\nexport const openCodeChange = true;\n"); + const checked = await index_status.execute({ checkFreshness: true }, context); + expect(checked).toContain("Source freshness: not current (reason: files-changed)"); + }); + + it("interrupts a checked status during scanning and propagates unexpected check failures", async () => { + const indexer = setup(); + await indexer.index(); + const abort = new AbortController(); + await expect(executeIndexStatus(root, "pi", { checkFreshness: true }, { + signal: abort.signal, + setPhase: (phase) => { if (phase === "scanning") abort.abort(new OperationCancelledError()); }, + })).rejects.toBeInstanceOf(OperationCancelledError); + await expect(executeIndexStatus(root, "pi", { checkFreshness: true }, { + setPhase: (phase) => { if (phase === "scanning") throw new Error("check failed unexpectedly"); }, + })).rejects.toThrow("check failed unexpectedly"); + }); + + it.each(["ocbi", "cbi"])("prints only shared status JSON and returns success for stale/missing catalogs via %s", async (binary) => { + const indexer = setup(); + const stdout: string[] = []; + const stderr: string[] = []; + const run = (...flags: string[]) => runCbiCli(["node", binary, "status", "--project", root, "--host", "omp", ...flags], root, { + printStdout: (value) => stdout.push(value), printStderr: (value) => stderr.push(value), + }); + expect(await run("--check-freshness", "--json")).toBe(0); + expect(JSON.parse(stdout.pop()!)).toMatchObject({ indexed: false, freshness: { reason: "missing" } }); + await indexer.index(); + fs.appendFileSync(sourcePath(), "\nexport const newerSource = true;\n"); + expect(await run("--check-freshness", "--json")).toBe(0); + const stale = JSON.parse(stdout.pop()!); + expect(stale).toMatchObject({ indexed: true, freshness: { current: false, reason: "files-changed" } }); + expect(stale).not.toHaveProperty("text"); + expect(stale).not.toHaveProperty("details"); + expect(await run("--json")).toBe(0); + expect(JSON.parse(stdout.pop()!).freshness).toBeNull(); + expect(stderr).toEqual([]); + const invalidConfig = path.join(root, "invalid-config.json"); + fs.writeFileSync(invalidConfig, "{"); + expect(await run("--check-freshness", "--json", "--config", invalidConfig)).toBe(1); + expect(stdout).toEqual([]); + }); + + it.each(["pi", "omp"] as const)("exposes real checked status and default not-checked details through %s", async (host) => { + const indexer = setup(); + await indexer.index(); + type Tool = { name: string; execute: (id: string, args: Record, signal: AbortSignal, update: undefined, ctx: { cwd: string }) => Promise<{ content: Array<{ text: string }>; details?: unknown }> }; + const tools = new Map(); + const api = { + registerTool(tool) { tools.set(tool.name, tool as unknown as Tool); }, + on() {}, + } satisfies Pick; + (host === "omp" ? codebaseIndexOmpExtension : codebaseIndexPiExtension)(api); + const status = tools.get("index_status")!; + expect((await status.execute("plain", {}, new AbortController().signal, undefined, { cwd: root })).details).toMatchObject({ indexed: true, freshness: null }); + fs.appendFileSync(sourcePath(), "\nexport const staleMarker = true;\n"); + const checked = await status.execute("checked", { checkFreshness: true }, new AbortController().signal, undefined, { cwd: root }); + expect(checked.details).toMatchObject({ indexed: true, freshness: { current: false, reason: "files-changed" } }); + expect(checked.content[0].text).toContain("reason: files-changed"); + }); + + it("retains status facts alongside MCP runtime diagnostics through the public protocol", async () => { + const server = createMcpServer(root, structuralConfig(), "pi"); + hosts.add("pi"); + const indexer = getIndexerForProject(root, "pi"); + readers.add(indexer); + await indexer.index(); + const client = new Client({ name: "status-freshness-test", version: "1.0.0" }); + const [clientTransport, serverTransport] = InMemoryTransport.createLinkedPair(); + try { + await server.connect(serverTransport); + await client.connect(clientTransport); + const plain = await client.callTool({ name: "index_status", arguments: {} }); + expect(plain.structuredContent).toMatchObject({ indexed: true, freshness: null, mcpDiagnostics: { schemaVersion: 1, activeOperations: [] } }); + fs.appendFileSync(sourcePath(), "\nexport const mcpChange = true;\n"); + const checked = await client.callTool({ name: "index_status", arguments: { checkFreshness: true } }); + expect(checked.isError).not.toBe(true); + expect(checked.structuredContent).toMatchObject({ indexed: true, freshness: { readable: true, current: false, reason: "files-changed" }, mcpDiagnostics: { schemaVersion: 1, activeOperations: [] } }); + expect(checked.content).toEqual([expect.objectContaining({ type: "text", text: expect.stringContaining("reason: files-changed") })]); + } finally { + await client.close(); + await server.close(); + } + }); +}); diff --git a/tests/mcp-server.test.ts b/tests/mcp-server.test.ts index 311a7910..56b915c3 100644 --- a/tests/mcp-server.test.ts +++ b/tests/mcp-server.test.ts @@ -521,6 +521,7 @@ describe("MCP server tools and prompts", () => { expect(properties).toBeDefined(); expect(properties).toHaveProperty("includeApiImpact"); expect(properties?.includeApiImpact).toEqual(expect.objectContaining({ type: "boolean", default: false })); + expect(properties?.includeTests).toEqual(expect.objectContaining({ type: "boolean", default: false })); }); it("should register all 5 prompts", async () => { diff --git a/tests/pi-conformance.test.ts b/tests/pi-conformance.test.ts index f001a40f..a9dd2a0a 100644 --- a/tests/pi-conformance.test.ts +++ b/tests/pi-conformance.test.ts @@ -21,6 +21,7 @@ const operationMocks = vi.hoisted(() => ({ searchCodebase: vi.fn(), searchCodebaseWithEffectiveness: vi.fn(), implementationLookup: vi.fn(), + getSymbolTestEvidence: vi.fn(), recordToolEffectiveness: vi.fn(), getIndexMetrics: vi.fn(() => ({ text: "" })), isToolEffectivenessEnabled: vi.fn(() => true), @@ -60,6 +61,7 @@ vi.mock("../src/tools/operations.js", () => ({ getCodeCommunities: operationMocks.getCodeCommunities, getArchitectureContext: operationMocks.getArchitectureContext, implementationLookup: operationMocks.implementationLookup, + getSymbolTestEvidence: operationMocks.getSymbolTestEvidence, listKnowledgeBases: vi.fn(() => "No knowledge bases configured."), removeKnowledgeBase: vi.fn(() => "Removed knowledge base"), runIndexHealthCheck: operationMocks.runIndexHealthCheck, @@ -189,6 +191,7 @@ describe("Pi adapter conformance", () => { }, ); operationMocks.implementationLookup.mockReset(); + operationMocks.getSymbolTestEvidence.mockReset(); operationMocks.recordToolEffectiveness.mockReset(); operationMocks.isToolEffectivenessEnabled.mockReset(); operationMocks.isToolEffectivenessEnabled.mockReturnValue(true); @@ -366,6 +369,60 @@ describe("Pi adapter conformance", () => { expect(tools.get("codebase_edit_context")?.parameters?.properties?.includeApiImpact).toEqual( expect.objectContaining({ default: false, type: "boolean" }), ); + expect(tools.get("codebase_edit_context")?.parameters?.properties?.includeTests).toEqual( + expect.objectContaining({ default: false, type: "boolean" }), + ); + }); + + it.each(["pi", "omp"] as const)("returns only fully rendered static test references in %s details", async (host) => { + const source = { + filePath: "src/gate.ts", startLine: 1, endLine: 3, + chunkType: "function", name: "signalGate", score: 1, + content: "export function signalGate(value: boolean) {\n return value;\n}", + }; + operationMocks.implementationLookup.mockResolvedValue([source]); + operationMocks.getCallGraphData.mockImplementation(async (_root, _host, params) => ({ + direction: params.direction, + resolution: { + status: "resolved", name: "signalGate", symbolId: "gate", + filePath: "src/gate.ts", startLine: 1, kind: "function", matchedBy: "name", + }, + callers: [], callees: [], + })); + const references = Array.from({ length: 20 }, (_, index) => ({ + filePath: `tests/gate-${index}.test.ts`, startLine: 2, endLine: 4, + chunkType: "expression_statement", score: 1, name: "signalGate", + content: "it('gate', () => {\n expect(signalGate(false)).toBe(false);\n});", + evidenceKind: "direct_target_test_reference", referencedSymbol: "signalGate", referenceLine: 3, + })); + operationMocks.getSymbolTestEvidence.mockResolvedValue({ + resolution: "resolved", results: references, scannedFileCount: 20, + candidateFileCount: 20, verifiedCallerCount: 0, capped: true, + }); + const { tools } = await registerPiTools(host); + const tool = tools.get("codebase_edit_context")!; + const disabled = await tool.execute("without-tests", { + query: "change signalGate", symbol: "signalGate", includeTests: false, tokenBudget: 600, + }, new AbortController().signal, () => {}, { cwd: "/repo" }); + expect(disabled.content[0].text).not.toContain("direct_target_test_reference"); + expect(operationMocks.getSymbolTestEvidence).not.toHaveBeenCalled(); + const response = await tool.execute("with-tests", { + query: "change signalGate", symbol: "signalGate", includeTests: true, tokenBudget: 600, + }, new AbortController().signal, () => {}, { cwd: "/repo" }); + const details = response.details as { + sourceIncluded: boolean; + testDiscovery: { results: typeof references; capped: boolean; omittedReferenceCount: number }; + }; + const text = response.content[0].text; + const rendered = [...text.matchAll(/^- (.+):2-4 \[direct_target_test_reference\] referencedSymbol=signalGate referenceLine=3$/gmu)]; + expect(details.sourceIncluded).toBe(true); + expect(rendered.map((row) => row[1])).toEqual(details.testDiscovery.results.map((row) => row.filePath)); + for (const reference of details.testDiscovery.results) expect(reference).not.toHaveProperty("content"); + expect(details.testDiscovery.omittedReferenceCount).toBe(20 - rendered.length); + expect(details.testDiscovery.omittedReferenceCount).toBeGreaterThan(0); + expect(details.testDiscovery.capped).toBe(true); + expect(text).toContain("Static source references; not runtime coverage."); + expect(countContextTokens(text)).toBeLessThanOrEqual(600); }); it("keeps Pi normal output unchanged when diagnostic is false", async () => { diff --git a/tests/working-tree-impact-surfaces.test.ts b/tests/working-tree-impact-surfaces.test.ts new file mode 100644 index 00000000..08a42b45 --- /dev/null +++ b/tests/working-tree-impact-surfaces.test.ts @@ -0,0 +1,265 @@ +import type { ExtensionAPI } from "@earendil-works/pi-coding-agent"; +import type { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js"; +import type { HostMode } from "../src/config/host.js"; +import type { PrImpactResult } from "../src/indexer/pr-impact-types.js"; +import { execFileSync } from "node:child_process"; +import * as fs from "node:fs"; +import * as os from "node:os"; +import * as path from "node:path"; +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import { Client } from "@modelcontextprotocol/sdk/client/index.js"; +import { InMemoryTransport } from "@modelcontextprotocol/sdk/inMemory.js"; +import { runCbiCli } from "../src/adapters/cbi.js"; +import { createMcpServer } from "../src/adapters/mcp/server.js"; +import { pr_impact } from "../src/adapters/opencode/pr-impact.js"; +import codebaseIndexPiExtension from "../src/pi-extension.js"; +import codebaseIndexOmpExtension from "../src/omp-extension.js"; +import { parseConfig } from "../src/config/schema.js"; +import { getPrImpact } from "../src/tools/operations.js"; +import { formatPrImpact } from "../src/tools/format-pr-impact.js"; +import { + configCache, + getIndexerCacheKey, + getIndexerForProject, + indexerCache, + initializeTools, +} from "../src/tools/operation-runtime.js"; +import { stopAutoIndex } from "../src/utils/auto-index.js"; +import { stopBackgroundWorker } from "../src/utils/background-worker.js"; + +const config = parseConfig({ + indexing: { mode: "structural", autoIndex: false, watchFiles: false, requireProjectMarker: false }, + include: ["**/*.ts"], + exclude: ["excluded/**"], + search: { minScore: 0 }, +}); +const currentGate = "export function signalGate() { return 'current'; }\nexport function currentOnlyGate() { return signalGate(); }\n"; +const untrackedPath = process.platform === "win32" ? "src/new Ω file.ts" : "src/new Ω\nfile.ts"; + +describe("working-tree impact host and CLI behavior", () => { + let root: string; + let activeHost: HostMode; + let client: Client | undefined; + let server: McpServer | undefined; + + function git(...args: string[]): string { + return execFileSync("git", args, { + cwd: root, + encoding: "utf8", + env: { ...process.env, GIT_CONFIG_NOSYSTEM: "1", GIT_CONFIG_GLOBAL: os.devNull }, + }); + } + + function write(relative: string, source: string): void { + const file = path.join(root, relative); + fs.mkdirSync(path.dirname(file), { recursive: true }); + fs.writeFileSync(file, source); + } + + async function prepare(host: HostMode = "pi", mcp = false): Promise { + activeHost = host; + if (mcp) { + server = createMcpServer(root, config, host); + client = new Client({ name: "working-tree-impact-test", version: "1.0.0" }); + const [clientTransport, serverTransport] = InMemoryTransport.createLinkedPair(); + await Promise.all([server.connect(serverTransport), client.connect(clientTransport)]); + } else { + initializeTools(root, config, host); + } + await getIndexerForProject(root, host).index(); + } + + function editTree(): void { + write("src/gate.ts", "export function stagedOnlyGate() { return 'staged'; }\n"); + git("add", "src/gate.ts"); + write("src/gate.ts", currentGate); + write("src/transport.ts", "import { signalGate } from './gate.js';\nexport function createTransport() { return signalGate() + 'unstaged'; }\n"); + write(untrackedPath, "export function untrackedGate() { return 42; }\n"); + } + + function assertCurrentFiles(result: PrImpactResult): void { + expect(result.workingTree?.scope).toBe("current_indexed_files"); + expect(result.workingTree?.entries).toEqual([ + { path: "src/gate.ts", indexStatus: "M", worktreeStatus: "M" }, + { path: untrackedPath, indexStatus: "?", worktreeStatus: "?" }, + { path: "src/transport.ts", indexStatus: " ", worktreeStatus: "M" }, + ]); + expect(result.changedFiles).toEqual(["src/gate.ts", untrackedPath, "src/transport.ts"]); + expect(result.directSymbols.map((symbol) => symbol.name)).toEqual(expect.arrayContaining(["signalGate", "currentOnlyGate", "createTransport", "untrackedGate"])); + expect(result.directSymbols.map((symbol) => symbol.name)).not.toContain("stagedOnlyGate"); + expect(result.transitiveCallers.map((symbol) => symbol.name)).toContain("serveRequest"); + expect(result.workingTree?.unassessed).toEqual([]); + expect(result.workingTree?.omittedFileCount).toBe(0); + } + + beforeEach(() => { + root = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), "ocbi-working-tree-surfaces-"))); + activeHost = "pi"; + git("init", "--initial-branch=main"); + write(".gitignore", ".codebase-index/\n.opencode/\nruntime-config.json\n"); + write("src/gate.ts", "export function signalGate() { return 'base'; }\n"); + write("src/transport.ts", "import { signalGate } from './gate.js';\nexport function createTransport() { return signalGate(); }\n"); + write("src/server.ts", "import { createTransport } from './transport.js';\nexport function serveRequest() { return createTransport(); }\n"); + git("add", "."); + git("-c", "user.name=Fixture", "-c", "user.email=fixture@example.invalid", "-c", "commit.gpgsign=false", "commit", "-m", "fixture"); + }); + + afterEach(async () => { + await client?.close(); + await server?.close(); + client = undefined; + server = undefined; + await stopBackgroundWorker(root, activeHost); + await stopAutoIndex(root, activeHost); + const key = getIndexerCacheKey(root, activeHost); + await indexerCache.get(key)?.close(); + indexerCache.delete(key); + configCache.delete(key); + fs.rmSync(root, { recursive: true, force: true }); + }); + + it.each(["ocbi", "cbi"])("%s prints real current-file impact JSON and qualified text", async (binary) => { + editTree(); + // An explicit config exercises the CLI's actual runtime/config boundary. + write("runtime-config.json", JSON.stringify(config)); + await prepare(); + // The CLI replaces its runtime for --config; release the fixture's writer first. + await getIndexerForProject(root, "pi").close(); + indexerCache.delete(getIndexerCacheKey(root, "pi")); + const output: string[] = []; + const errors: string[] = []; + const argv = ["node", binary, "impact", "--working-tree", "--project", root, "--host", "omp"]; + expect(await runCbiCli([...argv, "--json", "--config", path.join(root, "runtime-config.json")], root, { + printStdout: (text) => output.push(text), printStderr: (text) => errors.push(text), + })).toBe(0); + expect(errors).toEqual([]); + assertCurrentFiles(JSON.parse(output[0]) as PrImpactResult); + output.length = 0; + expect(await runCbiCli(argv, root, { + printStdout: (text) => output.push(text), printStderr: (text) => errors.push(text), + })).toBe(0); + expect(output[0]).toContain("current indexed files"); + expect(output[0]).toContain("not changed-line or staged-source analysis"); + expect(output[0]).toContain('[MM] "src/gate.ts"'); + expect(output[0]).toContain(`[??] ${JSON.stringify(untrackedPath)}`); + expect(output[0]).toContain("serveRequest"); + }); + + it("MCP returns native-backed impact details with matching text", async () => { + editTree(); + await prepare("pi", true); + const response = await client!.callTool({ name: "pr_impact", arguments: { workingTree: true } }); + expect(response.isError).not.toBe(true); + assertCurrentFiles(response.structuredContent as unknown as PrImpactResult); + expect(response.content).toEqual(expect.arrayContaining([ + expect.objectContaining({ type: "text", text: expect.stringContaining('[MM] "src/gate.ts"') }), + ])); + }); + + it.each(["pi", "omp"] as const)("%s exposes real current graph evidence as details", async (host) => { + editTree(); + await prepare(); + const tools = new Map[0]>(); + const api = { + registerTool(tool) { tools.set(tool.name, tool); }, + on() {}, + } satisfies Pick; + (host === "pi" ? codebaseIndexPiExtension : codebaseIndexOmpExtension)(api); + const tool = tools.get("pr_impact")!; + const response = await tool.execute("impact", { workingTree: true }, new AbortController().signal, () => {}, + { cwd: root } as Parameters[4]); + assertCurrentFiles(response.details as PrImpactResult); + expect(response.content).toEqual(expect.arrayContaining([ + expect.objectContaining({ type: "text", text: expect.stringContaining("static file-level graph analysis") }), + ])); + }); + + it("OpenCode inspects the tree without announcing branch preparation", async () => { + editTree(); + await prepare("opencode"); + const titles: string[] = []; + const response = await pr_impact.execute({ workingTree: true }, { + worktree: root, + metadata: (metadata: { title?: string }) => { if (metadata.title) titles.push(metadata.title); }, + }); + expect(response).toContain('[MM] "src/gate.ts"'); + expect(response).toContain("currentOnlyGate"); + expect(response).toContain("serveRequest"); + expect(titles).toContain("Inspecting current indexed working-tree files"); + expect(titles.join("\n")).not.toMatch(/Preparing|branch index|resolving authoritative branch head/); + }); + + it("shared and CLI stale inspection refuses refresh and leaves the stale state visible", async () => { + await prepare(); + write("src/gate.ts", currentGate); + const before = git("status", "--porcelain=v1"); + await expect(getPrImpact(root, "pi", { workingTree: true })).rejects.toThrow("Working-tree impact requires a current index. Run index_codebase, then retry."); + const errors: string[] = []; + const output: string[] = []; + expect(await runCbiCli(["node", "ocbi", "impact", "--working-tree", "--project", root, "--host", "omp"], root, { + printStdout: (text) => output.push(text), printStderr: (text) => errors.push(text), + })).toBe(1); + expect(output).toEqual([]); + expect(errors.join("\n")).toContain("Working-tree impact requires a current index"); + expect((await getIndexerForProject(root, "pi").getIndexFreshness()).current).toBe(false); + expect(git("status", "--porcelain=v1")).toBe(before); + }); + + it.each([{ pr: 1 }, { branch: "main" }, { checkConflicts: true }])("rejects conflicting shared arguments before phases or runtime initialization: %j", async (conflict) => { + const phases: string[] = []; + await expect(getPrImpact(root, "pi", { workingTree: true, ...conflict }, { + setPhase: (phase) => { phases.push(phase); }, + })).rejects.toThrow("workingTree cannot be combined with pr, branch, or checkConflicts"); + expect(phases).toEqual([]); + expect(indexerCache.has(getIndexerCacheKey(root, "pi"))).toBe(false); + expect(fs.existsSync(path.join(root, ".codebase-index"))).toBe(false); + }); + + it("reports clean trees explicitly and excluded paths as UNKNOWN rather than zero LOW", async () => { + await prepare(); + const clean = await getPrImpact(root, "pi", { workingTree: true }); + expect(clean.changedFiles).toEqual([]); + expect(formatPrImpact(clean)).toContain("No working-tree changes."); + write("excluded/opaque.ts", "export function excludedGate() { return 'unknown'; }\n"); + const result = await getPrImpact(root, "pi", { workingTree: true }); + expect(result.riskLevel).toBe("UNKNOWN"); + expect(result.workingTree?.unassessed).toContainEqual({ path: "excluded/opaque.ts", reason: "not_indexed" }); + const text = formatPrImpact(result); + expect(text).toContain("not indexed; source not assessed"); + expect(text).toContain("Graph counts below are partial"); + expect(text).toContain("Risk: UNKNOWN"); + expect(text).not.toContain("Risk: LOW"); + }); + + it("renders rename origins and deleted coverage before UNKNOWN risk", async () => { + git("mv", "src/gate.ts", "src/renamed gate.ts"); + fs.rmSync(path.join(root, "src/transport.ts")); + await prepare(); + const result = await getPrImpact(root, "pi", { workingTree: true }); + expect(result.riskLevel).toBe("UNKNOWN"); + expect(result.workingTree?.entries).toContainEqual({ path: "src/renamed gate.ts", previousPath: "src/gate.ts", indexStatus: "R", worktreeStatus: " " }); + expect(result.workingTree?.unassessed).toEqual(expect.arrayContaining([ + { path: "src/gate.ts", reason: "renamed_from" }, + { path: "src/transport.ts", reason: "deleted" }, + ])); + const text = formatPrImpact(result); + expect(text).toContain('[R ] "src/renamed gate.ts" (from "src/gate.ts")'); + expect(text.indexOf("historical declarations not assessed")).toBeLessThan(text.indexOf("Risk: UNKNOWN")); + }); + + it("reports capped path coverage before UNKNOWN risk", async () => { + await prepare(); + for (let index = 0; index < 201; index += 1) { + write(`excluded/opaque-${String(index).padStart(3, "0")}.ts`, "export const opaque = true;\n"); + } + const result = await getPrImpact(root, "pi", { workingTree: true }); + expect(result.riskLevel).toBe("UNKNOWN"); + expect(result.workingTree?.omittedFileCount).toBe(1); + expect(result.workingTree?.entries[0].path).toBe("excluded/opaque-000.ts"); + expect(result.workingTree?.entries.at(-1)?.path).toBe("excluded/opaque-199.ts"); + const text = formatPrImpact(result); + expect(text).toContain("Omitted paths: 1 (200-path cap; coverage unknown)"); + expect(text.indexOf("200-path cap")).toBeLessThan(text.indexOf("Risk: UNKNOWN")); + expect(text).not.toContain("Risk: LOW"); + }); +}); diff --git a/tests/working-tree-impact.test.ts b/tests/working-tree-impact.test.ts new file mode 100644 index 00000000..2e651850 --- /dev/null +++ b/tests/working-tree-impact.test.ts @@ -0,0 +1,440 @@ +import { execFileSync } from "node:child_process"; +import { createHash } from "node:crypto"; +import * as fs from "node:fs/promises"; +import * as os from "node:os"; +import * as path from "node:path"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import { parseConfig } from "../src/config/schema.js"; +import { Indexer } from "../src/indexer/index.js"; +import { getWorkingTreeChanges } from "../src/tools/changed-files.js"; + +let root: string; +function git(...args: string[]): string { + return execFileSync("git", args, { cwd: root, encoding: "utf8", env: { ...process.env, GIT_CONFIG_NOSYSTEM: "1" } }).trim(); +} +async function source(filePath: string, content: string): Promise { + await fs.mkdir(path.dirname(path.join(root, filePath)), { recursive: true }); + await fs.writeFile(path.join(root, filePath), content); +} +async function commit(): Promise { + git("add", "--all"); + git("-c", "commit.gpgsign=false", "commit", "-m", "fixture"); +} +beforeEach(async () => { + root = await fs.mkdtemp(path.join(os.tmpdir(), "working-tree-impact-")); + git("init", "-b", "main"); + git("config", "user.email", "fixture@example.com"); + git("config", "user.name", "Fixture"); +}); +afterEach(async () => { await fs.rm(root, { recursive: true, force: true }); }); + +describe("working-tree change collection", () => { + it("reports current staged and unstaged paths without changing the Git index", async () => { + await source("staged.ts", "export const value = 1;\n"); + await source("unstaged.ts", "export const value = 1;\n"); + await commit(); + await source("staged.ts", "export const value = 2;\n"); + git("add", "--", "staged.ts"); + await source("staged.ts", "export const value = 3;\n"); + await source("unstaged.ts", "export const value = 4;\n"); + await source("new λ file.ts", "export const value = 5;\n"); + const before = await fs.readFile(path.join(root, ".git/index")); + const changes = await getWorkingTreeChanges(root); + expect(changes).toEqual({ headCommit: git("rev-parse", "HEAD"), omittedFileCount: 0, entries: [ + { path: "new λ file.ts", indexStatus: "?", worktreeStatus: "?" }, + { path: "staged.ts", indexStatus: "M", worktreeStatus: "M" }, + { path: "unstaged.ts", indexStatus: " ", worktreeStatus: "M" }, + ] }); + expect(await fs.readFile(path.join(root, ".git/index"))).toEqual(before); + expect(await fs.readFile(path.join(root, "staged.ts"), "utf8")).toContain("= 3"); + }); + + it("preserves both paths of renames and literal newline names", async () => { + await source("old.ts", "export function original() { return 1; }\n"); + await source("deleted.ts", "export const removed = 1;\n"); + await commit(); + git("mv", "--", "old.ts", "new name.ts"); + await fs.unlink(path.join(root, "deleted.ts")); + const newlineName = process.platform === "win32" ? "unicode-λ.ts" : "line\nbreak.ts"; + await source(newlineName, "export const fresh = 1;\n"); + const changes = await getWorkingTreeChanges(root); + expect(changes.entries).toContainEqual({ path: "new name.ts", previousPath: "old.ts", indexStatus: "R", worktreeStatus: " " }); + expect(changes.entries).toContainEqual({ path: "deleted.ts", indexStatus: " ", worktreeStatus: "D" }); + expect(changes.entries).toContainEqual({ path: newlineName, indexStatus: "?", worktreeStatus: "?" }); + }); + + // APFS and Windows reject these bytes before Git can report them; Linux preserves byte filenames. + it.runIf(process.platform === "linux")("rejects byte filenames that would alias a valid replacement-character filename", async () => { + await source("invalid-\uFFFD.ts", "export function validUnicodePath() { return 1; }\n"); + expect((await getWorkingTreeChanges(root)).entries).toEqual([ + { path: "invalid-\uFFFD.ts", indexStatus: "?", worktreeStatus: "?" }, + ]); + const invalidPath = Buffer.concat([Buffer.from(path.join(root, "invalid-")), Buffer.from([0xff]), Buffer.from(".ts")]); + await fs.writeFile(invalidPath, "export function differentBytePath() { return 2; }\n"); + await expect(getWorkingTreeChanges(root)).rejects.toThrow("Working-tree paths must be valid UTF-8"); + }); + + it("distinguishes an unborn repository, a clean tree, and ignored files", async () => { + expect(await getWorkingTreeChanges(root)).toEqual({ headCommit: null, entries: [], omittedFileCount: 0 }); + await source(".gitignore", "ignored.ts\n"); + await commit(); + await source("ignored.ts", "ignored\n"); + expect(await getWorkingTreeChanges(root)).toEqual({ headCommit: git("rev-parse", "HEAD"), entries: [], omittedFileCount: 0 }); + }); + + it("discloses deterministic path truncation rather than silently dropping files", async () => { + for (let index = 0; index < 203; index++) await source(`file-${String(index).padStart(3, "0")}.ts`, "export const x = 1;\n"); + const changes = await getWorkingTreeChanges(root); + expect(changes.entries.map((entry) => entry.path)).toEqual(Array.from({ length: 200 }, (_, index) => `file-${String(index).padStart(3, "0")}.ts`)); + expect(changes.omittedFileCount).toBe(3); + }); + + it("rejects unmerged index entries rather than assigning a risk", async () => { + await source("conflict.ts", "export const value = 1;\n"); + await commit(); + const blob = git("rev-parse", "HEAD:conflict.ts"); + execFileSync("git", ["update-index", "--index-info"], { + cwd: root, input: `0 ${"0".repeat(40)}\tconflict.ts\n100644 ${blob} 1\tconflict.ts\n100644 ${blob} 2\tconflict.ts\n100644 ${blob} 3\tconflict.ts\n`, + }); + await expect(getWorkingTreeChanges(root)).rejects.toThrow("Resolve unmerged files"); + }); + + it("rejects non-Git directories and cancellation", async () => { + const outside = await fs.mkdtemp(path.join(os.tmpdir(), "no-working-tree-")); + try { await expect(getWorkingTreeChanges(outside)).rejects.toThrow(); } + finally { await fs.rm(outside, { recursive: true, force: true }); } + await expect(getWorkingTreeChanges(root, AbortSignal.abort())).rejects.toThrow(); + }); + + it("scopes nested projects and converts repository-root paths into project-relative paths", async () => { + await source("nested project/src/gate.ts", "export function signalGate() { return true; }\n"); + await source("outside.ts", "export const other = 1;\n"); + await commit(); + await source("nested project/src/gate.ts", "export function signalGate() { return false; }\n"); + await source("nested project/new λ.ts", "export function fresh() { return true; }\n"); + await source("outside.ts", "export const other = 2;\n"); + const before = await fs.readFile(path.join(root, ".git/index")); + const changes = await getWorkingTreeChanges(path.join(root, "nested project")); + expect(changes.entries).toEqual([ + { path: "new λ.ts", indexStatus: "?", worktreeStatus: "?" }, + { path: "src/gate.ts", indexStatus: " ", worktreeStatus: "M" }, + ]); + expect(await fs.readFile(path.join(root, ".git/index"))).toEqual(before); + }); + + it.each(["inbound", "outbound"])("keeps a %s rename across a child project boundary scoped to its local endpoint", async (direction) => { + const local = "nested/gate.ts"; + const sibling = "sibling.ts"; + await source(direction === "inbound" ? sibling : local, "export function boundaryGate() { return true; }\n"); + await fs.mkdir(path.join(root, "nested"), { recursive: true }); + await commit(); + git("mv", "--", direction === "inbound" ? sibling : local, direction === "inbound" ? local : sibling); + const before = await fs.readFile(path.join(root, ".git/index")); + const changes = await getWorkingTreeChanges(path.join(root, "nested")); + expect(changes.entries).toEqual([ + { path: "gate.ts", indexStatus: direction === "inbound" ? "A" : "D", worktreeStatus: " " }, + ]); + expect(changes.omittedFileCount).toBe(0); + expect(await fs.readFile(path.join(root, ".git/index"))).toEqual(before); + }); +}); + +describe("working-tree impact on the current native graph", () => { + let indexer: Indexer; + let indexRoot: string; + + beforeEach(async () => { + indexRoot = await fs.mkdtemp(path.join(os.tmpdir(), "working-tree-impact-index-")); + await source("src/gate.ts", "export function signalGate(value: boolean) { return value; }\n"); + await source("src/transport.ts", "import { signalGate } from './gate.js';\nexport function createTransport(value: boolean) { return signalGate(value); }\n"); + await commit(); + indexer = new Indexer(root, parseConfig({ + indexing: { mode: "structural", autoIndex: false, watchFiles: false, requireProjectMarker: false, maxFilesPerDirectory: 1000 }, + include: ["**/*.ts", "**/*.md"], exclude: ["excluded/**"], search: { minScore: 0 }, + }), "omp", { indexPath: indexRoot }); + }); + + afterEach(async () => { + vi.restoreAllMocks(); + await indexer.close(); + await fs.rm(indexRoot, { recursive: true, force: true }); + }); + + async function snapshot(directory: string): Promise> { + const contents: Record = {}; + async function visit(current: string): Promise { + for (const entry of (await fs.readdir(current, { withFileTypes: true })).sort((a, b) => a.name.localeCompare(b.name))) { + // SQLite readers update transient shared-memory coordination, not persisted index evidence. + if (entry.name.endsWith(".db-shm")) continue; + const file = path.join(current, entry.name); + if (entry.isDirectory()) await visit(file); + else contents[path.relative(directory, file)] = entry.isSymbolicLink() + ? `symlink:${await fs.readlink(file)}` + : createHash("sha256").update(await fs.readFile(file)).digest("hex"); + } + } + await visit(directory); + return contents; + } + + it("rejects stale edits without mutation, then accepts refreshed dirty staged/unstaged/untracked contents and reachability", async () => { + await indexer.index(); + await source("src/gate.ts", "export function signalGate(value: boolean) { return !value; }\n"); + git("add", "--", "src/gate.ts"); + await source("src/gate.ts", "export function signalGate(value: boolean) { return dirtyGate(value); }\nexport function dirtyGate(value: boolean) { return value; }\n"); + await source("src/transport.ts", "import { signalGate } from './gate.js';\nexport function createTransport(value: boolean) { return signalGate(Boolean(value)); }\n"); + await source("src/new λ file.ts", "import { createTransport } from './transport.js';\nexport function untrackedConsumer() { return createTransport(true); }\n"); + const beforeGit = await snapshot(root); + const beforeIndex = await snapshot(indexRoot); + const fetchSpy = vi.spyOn(globalThis, "fetch").mockRejectedValue(new Error("Network is forbidden")); + const indexSpy = vi.spyOn(indexer, "index"); + await expect(indexer.getPrImpact({ workingTree: true })).rejects.toThrow("Working-tree impact requires a current index. Run index_codebase, then retry."); + expect(await snapshot(root)).toEqual(beforeGit); + expect(await snapshot(indexRoot)).toEqual(beforeIndex); + expect(indexSpy).not.toHaveBeenCalled(); + + await indexer.index(); + expect(await indexer.getIndexFreshness()).toMatchObject({ current: true, reason: "current" }); + const indexedGit = await snapshot(root); + const indexedArtifacts = await snapshot(indexRoot); + const progress = vi.fn(); + const result = await indexer.getPrImpact({ workingTree: true }, progress); + expect(result.workingTree).toEqual({ + headCommit: git("rev-parse", "HEAD"), scope: "current_indexed_files", omittedFileCount: 0, unassessed: [], + entries: [ + { path: "src/gate.ts", indexStatus: "M", worktreeStatus: "M" }, + { path: "src/new λ file.ts", indexStatus: "?", worktreeStatus: "?" }, + { path: "src/transport.ts", indexStatus: " ", worktreeStatus: "M" }, + ], + }); + expect(result.changedFiles).toEqual(["src/gate.ts", "src/new λ file.ts", "src/transport.ts"]); + expect(result.directSymbols.map((symbol) => symbol.name)).toEqual(expect.arrayContaining(["signalGate", "dirtyGate", "createTransport", "untrackedConsumer"])); + expect(result.riskLevel).not.toBe("UNKNOWN"); + expect(result.riskReason).toContain("current indexed contents, not staged blobs"); + expect(result.indexPreparation).toBeUndefined(); + expect(progress).not.toHaveBeenCalled(); + expect(fetchSpy).not.toHaveBeenCalled(); + expect(await snapshot(root)).toEqual(indexedGit); + expect(await snapshot(indexRoot)).toEqual(indexedArtifacts); + }); + + it("traverses unchanged callers of an indexed changed file", async () => { + await source("src/gate.ts", "export function signalGate(value: boolean) { return !value; }\n"); + await indexer.index(); + const result = await indexer.getPrImpact({ workingTree: true, direction: "callers" }); + expect(result.directSymbols.map((symbol) => symbol.name)).toEqual(["signalGate"]); + expect(result.transitiveCallers).toContainEqual(expect.objectContaining({ + name: "createTransport", filePath: path.join(root, "src/transport.ts"), depth: 1, + })); + }); + + it("reports deletions and rename sources as UNKNOWN without manufacturing historical symbols", async () => { + git("mv", "--", "src/gate.ts", "src/renamed gate.ts"); + await fs.unlink(path.join(root, "src/transport.ts")); + await indexer.index(); + const result = await indexer.getPrImpact({ workingTree: true }); + expect(result.workingTree?.entries).toContainEqual({ + path: "src/renamed gate.ts", previousPath: "src/gate.ts", indexStatus: "R", worktreeStatus: " ", + }); + expect(result.workingTree?.unassessed).toEqual(expect.arrayContaining([ + { path: "src/gate.ts", reason: "renamed_from" }, { path: "src/transport.ts", reason: "deleted" }, + ])); + expect(result.riskLevel).toBe("UNKNOWN"); + expect(result.directSymbols.map((symbol) => symbol.filePath)).toEqual([path.join(root, "src/renamed gate.ts")]); + expect(result.transitiveCallers.some((symbol) => symbol.name === "createTransport")).toBe(false); + }); + + it("retains UNKNOWN coverage gaps for unindexed scoped exclusions and files without graph symbols", async () => { + await source("excluded/new.ts", "export function notInScope() { return 1; }\n"); + await source("src/no-symbols.md", "# Operational notes\n\nDocumentation without call-graph declarations.\n"); + await indexer.index(); + const result = await indexer.getPrImpact({ workingTree: true }); + expect(result.workingTree?.unassessed).toEqual([ + { path: "excluded/new.ts", reason: "not_indexed" }, + { path: "src/no-symbols.md", reason: "no_symbols" }, + ]); + expect(result.riskLevel).toBe("UNKNOWN"); + expect(result.directSymbols).toEqual([]); + }); + + it.skipIf(process.platform === "win32")("never reads outside-project symlink targets and identifies nonregular local links", async () => { + const outside = path.join(indexRoot, "outside.ts"); + await fs.writeFile(outside, "export function outsideSentinel() { return 1; }\n"); + await fs.symlink(outside, path.join(root, "outside-link.ts")); + await fs.symlink(path.join(root, "src/gate.ts"), path.join(root, "inside-link.ts")); + await indexer.index(); + const before = await fs.readFile(outside); + const result = await indexer.getPrImpact({ workingTree: true }); + expect(result.workingTree?.unassessed).toEqual([ + { path: "inside-link.ts", reason: "not_regular" }, + { path: "outside-link.ts", reason: "outside_project" }, + ]); + expect(result.riskLevel).toBe("UNKNOWN"); + expect(result.directSymbols).toEqual([]); + expect(await fs.readFile(outside)).toEqual(before); + }); + + it("returns an empty current-tree result rather than guessing a committed comparison", async () => { + await indexer.index(); + const result = await indexer.getPrImpact({ workingTree: true }); + expect(result.changedFiles).toEqual([]); + expect(result.directSymbols).toEqual([]); + expect(result.transitiveCallers).toEqual([]); + expect(result.workingTree).toMatchObject({ entries: [], unassessed: [], omittedFileCount: 0 }); + expect(result.riskReason).toContain("No working-tree changes"); + expect(result.indexPreparation).toBeUndefined(); + }); + + it("analyzes an indexed unborn repository without inventing a HEAD commit", async () => { + await indexer.close(); + git("update-ref", "-d", "refs/heads/main"); + await fs.unlink(path.join(root, ".git/index")); + indexer = new Indexer(root, parseConfig({ + indexing: { mode: "structural", autoIndex: false, watchFiles: false, requireProjectMarker: false }, + include: ["**/*.ts"], exclude: [], + }), "omp", { indexPath: indexRoot }); + await indexer.index(); + const result = await indexer.getPrImpact({ workingTree: true }); + expect(result.workingTree?.headCommit).toBeNull(); + expect(result.workingTree?.entries).toEqual([ + { path: "src/gate.ts", indexStatus: "?", worktreeStatus: "?" }, + { path: "src/transport.ts", indexStatus: "?", worktreeStatus: "?" }, + ]); + expect(result.directSymbols.map((symbol) => symbol.name)).toEqual(expect.arrayContaining(["signalGate", "createTransport"])); + expect(result.indexPreparation).toBeUndefined(); + }); + + it("does not report LOW when the path cap leaves unassessed files", async () => { + for (let index = 0; index < 203; index++) await source(`excluded/${String(index).padStart(3, "0")}.ts`, "export function omitted() { return 1; }\n"); + await indexer.index(); + const result = await indexer.getPrImpact({ workingTree: true }); + expect(result.workingTree?.omittedFileCount).toBe(3); + expect(result.workingTree?.entries.map((entry) => entry.path)).toEqual( + Array.from({ length: 200 }, (_, index) => `excluded/${String(index).padStart(3, "0")}.ts`), + ); + expect(result.workingTree?.unassessed.every((entry) => entry.reason === "not_indexed")).toBe(true); + expect(result.riskLevel).toBe("UNKNOWN"); + }); + + it.each([{ pr: 1 }, { branch: "" }, { checkConflicts: true }])("rejects incompatible options before initializing or inspecting Git: %j", async (conflict) => { + const initializeSpy = vi.spyOn(indexer, "getStatus"); + const before = await snapshot(root); + const beforeIndex = await snapshot(indexRoot); + await expect(indexer.getPrImpact({ workingTree: true, ...conflict })).rejects.toThrow( + "workingTree cannot be combined with pr, branch, or checkConflicts", + ); + expect(initializeSpy).not.toHaveBeenCalled(); + expect(await snapshot(root)).toEqual(before); + expect(await snapshot(indexRoot)).toEqual(beforeIndex); + }); + + it("rejects cancellation before inspection and during the bounded consistency check", async () => { + await indexer.index(); + await expect(indexer.getPrImpact({ workingTree: true }, undefined, { signal: AbortSignal.abort() })).rejects.toThrow(); + const controller = new AbortController(); + const original = indexer.getIndexFreshness.bind(indexer); + let checks = 0; + vi.spyOn(indexer, "getIndexFreshness").mockImplementation(async (options) => { + const result = await original(options); + if (++checks === 2) controller.abort(); + return result; + }); + await expect(indexer.getPrImpact({ workingTree: true }, undefined, { signal: controller.signal })).rejects.toThrow(); + expect(checks).toBe(2); + }); + + it("invalidates concurrent dirty-byte changes deterministically even when Git XY status is unchanged", async () => { + await source("src/gate.ts", "export function signalGate(value: boolean) { return !value; }\n"); + await indexer.index(); + const original = indexer.getIndexFreshness.bind(indexer); + let checks = 0; + vi.spyOn(indexer, "getIndexFreshness").mockImplementation(async (options) => { + const result = await original(options); + if (++checks === 2) await source("src/gate.ts", "export function signalGate(value: boolean) { return Boolean(value); }\n"); + return result; + }); + await expect(indexer.getPrImpact({ workingTree: true })).rejects.toThrow( + "Working tree changed during impact analysis; retry on a stable tree.", + ); + expect(checks).toBe(2); + }); + + it("rejects a concurrent graph publication even when source bytes and graph counts remain current", async () => { + await source("src/gate.ts", "export function signalGate(value: boolean) { return !value; }\n"); + await indexer.index(); + const beforeStatus = await indexer.getStatus(); + const original = indexer.getIndexFreshness.bind(indexer); + let checks = 0; + vi.spyOn(indexer, "getIndexFreshness").mockImplementation(async (options) => { + if (++checks === 2) { + await indexer.index(); + expect(await indexer.getStatus()).toEqual(beforeStatus); + } + const freshness = await original(options); + expect(freshness.current).toBe(true); + return freshness; + }); + await expect(indexer.getPrImpact({ workingTree: true })).rejects.toThrow( + "Working tree changed during impact analysis; retry on a stable tree.", + ); + expect(checks).toBe(2); + }); + + it("rejects branch changes and pending migrations instead of preparing a different graph", async () => { + await indexer.index(); + git("checkout", "-b", "other"); + await expect(indexer.getPrImpact({ workingTree: true })).rejects.toThrow("Working-tree impact requires a current index."); + git("checkout", "main"); + const status = await indexer.getStatus(); + vi.spyOn(indexer, "getStatus").mockResolvedValue({ ...status, migrationRequired: true }); + await expect(indexer.getPrImpact({ workingTree: true })).rejects.toThrow("Working-tree impact requires a current index."); + }); + + it("rejects missing index data without preparing or publishing a branch", async () => { + const before = await snapshot(root); + const beforeIndex = await snapshot(indexRoot); + await expect(indexer.getPrImpact({ workingTree: true })).rejects.toThrow("Working-tree impact requires a current index."); + expect(await snapshot(root)).toEqual(before); + expect(await snapshot(indexRoot)).toEqual(beforeIndex); + }); + + it("propagates an unmerged-index error without producing a current impact result", async () => { + await indexer.index(); + const blob = git("rev-parse", "HEAD:src/gate.ts"); + execFileSync("git", ["update-index", "--index-info"], { + cwd: root, input: `0 ${"0".repeat(40)}\tsrc/gate.ts\n100644 ${blob} 1\tsrc/gate.ts\n100644 ${blob} 2\tsrc/gate.ts\n100644 ${blob} 3\tsrc/gate.ts\n`, + }); + const before = await snapshot(root); + await expect(indexer.getPrImpact({ workingTree: true })).rejects.toThrow("Resolve unmerged files"); + expect(await snapshot(root)).toEqual(before); + }); + + it("assesses literal spaces, newline and Unicode paths using their indexed current symbols", async () => { + const file = process.platform === "win32" ? "src/new λ space.ts" : "src/new λ\nspace.ts"; + await source(file, "export function literalPathFunction() { return true; }\n"); + await indexer.index(); + const result = await indexer.getPrImpact({ workingTree: true }); + expect(result.changedFiles).toEqual([file]); + expect(result.workingTree?.unassessed).toEqual([]); + expect(result.directSymbols).toContainEqual(expect.objectContaining({ + name: "literalPathFunction", filePath: path.join(root, file), + })); + }); + + it("uses the selected child project's indexed graph without assessing dirty siblings", async () => { + await indexer.close(); + await source("nested/src/gate.ts", "export function nestedGate() { return true; }\n"); + await commit(); + await source("nested/src/gate.ts", "export function nestedGate() { return false; }\n"); + await source("src/gate.ts", "export function outsideScope() { return false; }\n"); + indexer = new Indexer(path.join(root, "nested"), parseConfig({ + indexing: { mode: "structural", autoIndex: false, watchFiles: false, requireProjectMarker: false }, + include: ["**/*.ts"], exclude: [], + }), "omp", { indexPath: indexRoot }); + await indexer.index(); + const result = await indexer.getPrImpact({ workingTree: true }); + expect(result.changedFiles).toEqual(["src/gate.ts"]); + expect(result.directSymbols.map((symbol) => symbol.name)).toEqual(["nestedGate"]); + expect(result.workingTree?.unassessed).toEqual([]); + }); +}); diff --git a/tests/workspace-status.test.ts b/tests/workspace-status.test.ts index 9840f47a..a578b781 100644 --- a/tests/workspace-status.test.ts +++ b/tests/workspace-status.test.ts @@ -49,6 +49,7 @@ function status(currentBranch: string, overrides: Partial = { baseBranch: "main", compatibility: { compatible: true }, failedBatchesCount: 0, + freshness: null, autoIndex: { enabled: false, state: "idle", updatedAt: new Date(0).toISOString() }, branchReadiness: { state: "ready", activeCatalogChunkCount: 12, registeredCatalog: true }, ...overrides, @@ -258,15 +259,4 @@ describe("workspace status", () => { expect(exitCode).toBe(1); expect(JSON.parse(stdout[0])).toEqual(result); }); - - it("preserves existing status command dispatch", async () => { - const stdout: string[] = []; - const exitCode = await runCbiCli(["node", "cbi", "status", "--project", "/repo"], "/tmp", { - runStatus: async () => ({ text: "existing status" }), - printStdout: (text) => stdout.push(text), - printStderr: () => undefined, - }); - expect(exitCode).toBe(0); - expect(stdout).toEqual(["existing status"]); - }); });