Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ IngestionPipeline (pipeline/__init__.py)

**Node FQN format**: `module.ClassName.method_name` — fully dot-separated, derived from the file path (e.g., `src/cgis/pipeline/__init__.py` → `src.cgis.pipeline.IngestionPipeline.run`). Use `cgis.languages.scope_of(path, source_root)` to convert paths (it picks the file's own pack; the Python rule lives in `languages/python/extractor.py`). `__init__.py` strips the `/__init__` suffix.

**Raw call convention**: Extractors emit edges with `target = "raw_call:<name>"`. The `ResolverEngine` then resolves these to actual FQNs, or leaves them unresolved: the target becomes the bare raw name (or a pack fallback such as `js_builtins.<name>`), a virtual boundary node classified STDLIB, EXTERNAL or UNKNOWN (#661).
**Raw call convention**: Extractors emit edges with `target = "raw_call:<name>"`. The `ResolverEngine` then resolves these to actual FQNs, or leaves them unresolved: the target becomes the bare raw name (or a pack fallback such as `js_builtins.<name>`), a virtual boundary node classified STDLIB, EXTERNAL or UNKNOWN. A raw name equal to the id of a real node keeps the `raw_call:` prefix, so the edge never joins that node (#661).

**`ResolverEngine` resolution order** for a call to `name`:
1. `self.method` → look up `method` on the enclosing class via `_class_methods` index
Expand Down
2 changes: 1 addition & 1 deletion docs/architecture/HOW_IT_WORKS.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ The resolver operates on the complete set of raw edges emitted by Phase 1. It bu

1. If `name` starts with `self.`, look up the method on the enclosing class via `_class_methods`
2. Look up `name` in `_global_symbols`, among the caller's own language only; if ambiguous, prefer the same-scope candidate (a file, or a Go package) via `_file_global_symbols`
3. A resolved call scores `confidence=1.0` in every language. If unresolved, the target becomes the bare `<name>` (or a pack fallback such as `js_builtins.<name>`), a virtual boundary node classified STDLIB, EXTERNAL or UNKNOWN, at `confidence=0.8`
3. A resolved call scores `confidence=1.0` in every language (a Go builtin resolves to `builtin.<name>`), capped at 0.8 when its target is an UNKNOWN boundary node. If unresolved, the target becomes the bare `<name>` (or a pack fallback such as `js_builtins.<name>`), a virtual boundary node classified STDLIB, EXTERNAL or UNKNOWN, at `confidence=0.8`. A name equal to the id of a real node keeps the `raw_call:` prefix instead, so the edge cannot join that node (#661)

Unresolved calls are **first-class citizens** — they appear in the graph explicitly rather than being silently discarded.

Expand Down
6 changes: 3 additions & 3 deletions docs/architecture/ONTOLOGY.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ Produced by `ResolverEngine` after FQN resolution. These edges represent runtime
| `CALLS` | caller → callee | A resolved function/method invocation |
| `REFERENCES` | node → node | A non-call symbolic reference |

**Unresolved calls** keep `target = "raw_call:<name>"` with `confidence = 0.1` and are stored explicitly so gaps in resolution are visible rather than hidden.
**Unresolved calls** are stored explicitly, so gaps in resolution are visible rather than hidden. The target becomes the bare `<name>` (or a pack fallback such as `js_builtins.<name>`), a virtual boundary node classified STDLIB, EXTERNAL or UNKNOWN, at `confidence=0.8`. A name equal to the id of a real node keeps the `raw_call:` prefix instead, so the edge cannot join that node (#661). A resolved call scores 1.0 in every language (#674), capped at 0.8 when its target is an UNKNOWN boundary node (#493); a Go builtin resolves to `builtin.<name>`.

---

Expand Down Expand Up @@ -104,10 +104,10 @@ Node
Edge
├── id str — unique edge identifier (typically "source->target:type")
├── source str — FQN of origin node
├── target str — FQN of target node (or "raw_call:<name>" if unresolved)
├── target str — FQN of target node; unresolved: a virtual boundary node (see above)
├── type EdgeType — L1/L2/L3 classification
├── weight float — traversal weight (default 1.0)
└── confidence float — resolution confidence; 0.1 for unresolved raw calls
└── confidence float — CALLS: 1.0 resolved, 0.8 unresolved; DISPATCHES 0.5
```

---
Expand Down
4 changes: 2 additions & 2 deletions src/cgis/api/mcp_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
from cgis.api.tool_allowlist import TOOLS_ENV, CgisServer
from cgis.core.coverage import TraversalCoverage, UnresolvedSite
from cgis.core.freshness import Freshness, FreshnessState
from cgis.core.models import Edge, EdgeType, Node, NodeType
from cgis.core.models import RAW_CALL_PREFIX, Edge, EdgeType, Node, NodeType
from cgis.languages import extractors, impact_caveat
from cgis.pipeline import IngestionPipeline
from cgis.query.analysis.overview import DEFAULT_DEPTH, DEFAULT_LIMIT, build_overview
Expand Down Expand Up @@ -48,7 +48,7 @@
from cgis.query.render.mermaid import MermaidCompiler
from cgis.query.render.metrics import DuckDBAnalyzer
from cgis.query.render.text import graph_to_text
from cgis.storage.sqlite_store import RAW_CALL_PREFIX, SQLiteStore
from cgis.storage.sqlite_store import SQLiteStore

print("CGIS MCP Server starting…", file=sys.stderr)
# stdout is the JSON-RPC channel; the SDK diverts fd 1 only during a stdio session (#619).
Expand Down
12 changes: 10 additions & 2 deletions src/cgis/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,15 @@
from cgis import __app_name__, __version__, log_to_stderr
from cgis.core.coverage import TraversalCoverage
from cgis.core.freshness import FreshnessState
from cgis.core.models import VIRTUAL_FILE_PATH, Edge, EdgeType, Node, NodeNamespace, NodeType
from cgis.core.models import (
RAW_CALL_PREFIX,
VIRTUAL_FILE_PATH,
Edge,
EdgeType,
Node,
NodeNamespace,
NodeType,
)
from cgis.guardian.metrics import load_reviews, rate_review
from cgis.languages import extractors, file_node_id, is_supported
from cgis.pipeline import IngestionPipeline
Expand All @@ -56,7 +64,7 @@
from cgis.query.render.graph_json import graph_to_json
from cgis.query.render.mermaid import MermaidCompiler
from cgis.query.render.metrics import ArchitectureReport, DuckDBAnalyzer
from cgis.storage.sqlite_store import RAW_CALL_PREFIX, SQLiteStore
from cgis.storage.sqlite_store import SQLiteStore

_DEFAULT_DB = "graph.db"
_DEFAULT_DB_HELP = "Path to the SQLite database"
Expand Down
4 changes: 4 additions & 0 deletions src/cgis/core/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,10 @@ class NodeNamespace(StrEnum):
# These mirror the raw_call: / raw_dep: prefixes used by extractors and the
# resolver, but apply to self-method dispatch and class-inheritance edges.
SELF_PREFIX = "self."
#: What an extractor puts before every call's raw name, for the resolver to replace.
#: After resolution it stays only on an unresolved name that equals a real node's
#: id, which no real id can hold (#661).
RAW_CALL_PREFIX = "raw_call:"
RAW_CLASS_PREFIX = "raw_class:"


Expand Down
4 changes: 2 additions & 2 deletions src/cgis/guardian/chunker.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,12 @@
import structlog
from pydantic import BaseModel

from cgis.core.models import EdgeType
from cgis.core.models import RAW_CALL_PREFIX, EdgeType

# Re-exported: the per-file split moved to the pure diff leaf next to the other
# unified-diff parsers, since the skeptic pass needs it too (#246 plan T3).
from cgis.guardian.diff_index import split_diff_by_file as split_diff_by_file # noqa: PLC0414
from cgis.storage.sqlite_store import RAW_CALL_PREFIX, SQLiteStore
from cgis.storage.sqlite_store import SQLiteStore

log = structlog.getLogger(__name__)

Expand Down
3 changes: 3 additions & 0 deletions src/cgis/languages/go/_syntax.py
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,9 @@
}
)
BUILTINS = _BUILTIN_FUNCTIONS | PREDECLARED
#: Go's pseudo-package for its predeclared identifiers: a resolved builtin call
#: targets `builtin.<name>`, never the bare name (#661).
BUILTIN_PACKAGE = "builtin"
_MAJOR_VERSION = re.compile(r"v\d+")
#: Import names that bring a package in without a name to call it by.
UNNAMED_IMPORTS = frozenset({"_", "."})
Expand Down
11 changes: 8 additions & 3 deletions src/cgis/languages/go/index.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@

from cgis.core.index import ClassifyContext
from cgis.core.models import Node, NodeNamespace, NodeType
from cgis.languages.go._syntax import BUILTINS, PREDECLARED, dotted
from cgis.languages.go._syntax import BUILTIN_PACKAGE, BUILTINS, PREDECLARED, dotted
from cgis.languages.go.naming import GO_LANG
from cgis.languages.go.project import GO_MODULE_KEY

Expand Down Expand Up @@ -140,13 +140,18 @@ def classify(fqn: str, ctx: ClassifyContext) -> NodeNamespace:
facts = cast("GoIndexFacts", ctx.facts)
if _under(fqn, facts.module) or any(_under(fqn, m) for m in facts.replaces):
return NodeNamespace.INTERNAL
if fqn in BUILTINS:
if (
fqn.startswith(f"{BUILTIN_PACKAGE}.")
and fqn.removeprefix(f"{BUILTIN_PACKAGE}.") in BUILTINS
):
# Only the qualified form: a resolved builtin is `builtin.len` (#661). A bare
# `len` is an unresolved call, to a local named like a builtin, and stays UNKNOWN.
return NodeNamespace.STDLIB
imported = facts.file_imports.get(ctx.source_file or "", {})
under = [d for d in imported if _under(fqn, d)]
if under:
return import_namespace(imported[max(under, key=len)], facts)
if fqn.split(".", maxsplit=1)[0] in PREDECLARED:
if "." in fqn and fqn.split(".", maxsplit=1)[0] in PREDECLARED:
return NodeNamespace.STDLIB # a method of a predeclared type: `error.Error` (#608)
if fqn.split(".", maxsplit=1)[0] in ctx.internal_roots:
return NodeNamespace.INTERNAL
Expand Down
18 changes: 16 additions & 2 deletions src/cgis/languages/go/resolution.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,16 @@

from cgis.core.models import Node
from cgis.core.resolution import AFTER_IMPORT_MAP, ResolutionRules, ResolutionView
from cgis.languages.go._syntax import ANY_OF, BUILTINS, CALL_RESULT, ELEM, FIELD, GLOBAL, TAKE
from cgis.languages.go._syntax import (
ANY_OF,
BUILTIN_PACKAGE,
BUILTINS,
CALL_RESULT,
ELEM,
FIELD,
GLOBAL,
TAKE,
)
from cgis.languages.go.dispatch import interface_dispatch
from cgis.languages.go.index import GoIndexFacts
from cgis.languages.go.naming import GO_LANG
Expand Down Expand Up @@ -70,12 +79,17 @@ def builtin_call(name: str, file_path: str, view: ResolutionView) -> str | None:
name shadows it, as in Go: left to the same-scope tiebreak. Checked for
shadowing, it is a resolved call and scores 1.0, unlike a Python or TS builtin,
which is matched by name alone and keeps the unresolved 0.8 (#674 review).
The target is qualified, `builtin.len`, after Go's own pseudo-package, as a TS
builtin is `js_builtins.<name>`: a bare `len` would be shared with an unresolved
call to a local closure named `len`, and the two edges would merge (#661).
A module with a real top-level package `builtin` (GOROOT's own stubs) would see
these calls join it; accepted, as TS accepts it for `js_builtins`.
"""
if name not in BUILTINS:
return None
facts = view.facts.get(GO_LANG)
scope = cast("GoIndexFacts", facts).file_scopes.get(file_path, "") if facts else ""
return None if f"{scope}.{name}" in view.nodes else name
return None if f"{scope}.{name}" in view.nodes else f"{BUILTIN_PACKAGE}.{name}"


#: How deep one marker may expand: they nest only as far as the extractor bounds
Expand Down
12 changes: 9 additions & 3 deletions src/cgis/pipeline/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@

from cgis.core.extractor import BaseExtractor
from cgis.core.generated import is_generated_source
from cgis.core.models import Edge, EdgeType, Node, NodeType
from cgis.core.models import VIRTUAL_FILE_PATH, Edge, EdgeType, Node, NodeType
from cgis.core.paths import EXCLUDED_DIRS
from cgis.import_names import ImportNameCollector
from cgis.languages import cross_file_metadata, skips
Expand Down Expand Up @@ -445,8 +445,14 @@ def _cross_file_inputs_changed(
old = _resolution_signature(old_nodes, store.get_outgoing_edges_batch(old_ids))
fresh = new_by_file.get(file_path, {})
# An id another file's row holds (`m.py` beside `m/__init__.py`) was
# never this file's in the store, so it cannot be in `old` either.
owner = {n.id: n.file_path for n in store.get_nodes(list(fresh))}
# never this file's in the store, so it cannot be in `old` either. A
# virtual boundary row is no file's: a real node taking a once-virtual
# id is new, and unresolved edges that named it must re-resolve (#661).
owner = {
n.id: n.file_path
for n in store.get_nodes(list(fresh))
if n.file_path != VIRTUAL_FILE_PATH
}
new_nodes = [n for i, n in fresh.items() if owner.get(i, file_path) == file_path]
new_edges = [e for n in new_nodes for e in new_extends.get(n.id, {}).values()]
if _resolution_signature(new_nodes, new_edges) != old:
Expand Down
4 changes: 3 additions & 1 deletion src/cgis/query/analysis/cohesion.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
from dataclasses import dataclass
from typing import TYPE_CHECKING, Literal

from cgis.core.models import Edge, EdgeType, Node, NodeType
from cgis.core.models import RAW_CALL_PREFIX, Edge, EdgeType, Node, NodeType

if TYPE_CHECKING:
from collections.abc import Hashable, Mapping
Expand Down Expand Up @@ -67,6 +67,8 @@ def _resolve_to_file(node_id: str, file_set: set[str], suffix_index: dict[str, s
"""
if node_id in file_set:
return node_id
if node_id.startswith(RAW_CALL_PREFIX):
return None # an unresolved call whose name collided with a real id (#661)
parts = node_id.split(".")
for i in range(len(parts)):
hit = suffix_index.get(".".join(parts[i:]))
Expand Down
4 changes: 2 additions & 2 deletions src/cgis/query/context/context_service.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,11 @@
enclosing class is recovered separately from the structural layer.
"""

from cgis.core.models import EdgeType, Node, NodeNamespace, NodeType
from cgis.core.models import RAW_CALL_PREFIX, EdgeType, Node, NodeNamespace, NodeType
from cgis.query.context.prompt import compile_context
from cgis.query.context.snippet import extract_snippet, resolve_source_path
from cgis.query.engine import QueryEngine, is_unresolved
from cgis.storage.sqlite_store import RAW_CALL_PREFIX, SQLiteStore
from cgis.storage.sqlite_store import SQLiteStore

_CALLS: frozenset[EdgeType] = frozenset({EdgeType.CALLS})
_STRUCTURAL: frozenset[EdgeType] = frozenset({EdgeType.CONTAINS, EdgeType.DECLARES})
Expand Down
16 changes: 11 additions & 5 deletions src/cgis/query/engine.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
from typing import NamedTuple

from cgis.core.coverage import TraversalCoverage, rank_unresolved, unresolved_sites
from cgis.core.models import Edge, EdgeType, Node, NodeNamespace, NodeType
from cgis.storage.sqlite_store import RAW_CALL_PREFIX, SQLiteStore
from cgis.core.models import RAW_CALL_PREFIX, Edge, EdgeType, Node, NodeNamespace, NodeType
from cgis.storage.sqlite_store import SQLiteStore

STRUCTURAL_EDGE_TYPES: frozenset[EdgeType] = frozenset({EdgeType.CONTAINS, EdgeType.DECLARES})
BEHAVIORAL_EDGE_TYPES: frozenset[EdgeType] = frozenset(
Expand Down Expand Up @@ -101,9 +101,15 @@ def _admit(
Screen = Callable[[list[str]], _Verdict]


def _shown(target: str) -> str:
"""An unresolved target as a reader names it: without the `raw_call:` prefix a
name colliding with a real node keeps in storage (#661)."""
return target.removeprefix(RAW_CALL_PREFIX)


def _call_sites(calls: list[Edge]) -> list[tuple[str, str, str | None, int | None]]:
"""(target, caller, file, line) of each call, as `unresolved_sites` takes them."""
return [(e.target, e.source, e.file_path, e.line_number) for e in calls]
return [(_shown(e.target), e.source, e.file_path, e.line_number) for e in calls]


def agent_screen(
Expand Down Expand Up @@ -354,7 +360,7 @@ def get_impact_result(
left_out = screen(callers).dropped
examined = [e for e in walk.calls.values() if e.source not in left_out]
hidden = [e for e in hidden_calls if e.source not in left_out]
top = rank_unresolved(Counter(e.target for e in hidden))
top = rank_unresolved(Counter(_shown(e.target) for e in hidden))
coverage = TraversalCoverage(
basis="unresolved_calls_by_name",
calls_examined=len(examined) + len(hidden),
Expand Down Expand Up @@ -428,7 +434,7 @@ def get_flow_result(
known = {n.id: n for n in self.store.get_nodes([*walk.discovered, *call_targets])}
nodes = [n for n in known.values() if n.id in walk.discovered]
unresolved_calls = [e for e in walk.calls.values() if is_unresolved(e.target, known)]
top = rank_unresolved(Counter(e.target for e in unresolved_calls))
top = rank_unresolved(Counter(_shown(e.target) for e in unresolved_calls))
coverage = TraversalCoverage(
basis="unresolved_calls_made",
calls_examined=len(call_targets),
Expand Down
19 changes: 14 additions & 5 deletions src/cgis/resolver/engine.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

from cgis.core.models import (
DEFAULT_LANGUAGE,
RAW_CALL_PREFIX,
RAW_CLASS_PREFIX,
SELF_PREFIX,
VIRTUAL_FILE_PATH,
Expand All @@ -17,7 +18,6 @@
from cgis.languages import pack_for_path, rules_for
from cgis.resolver.symbols import SymbolResolver, dispatch_edges

RAW_CALL_PREFIX = "raw_call:"
RAW_DEP_PREFIX = "raw_dep:"
RAW_IMPORT_PREFIX = "raw_import:"

Expand All @@ -29,6 +29,14 @@
RESOLVED_CALL_CONFIDENCE = 1.0


def _unresolved_target(view: ResolutionView, raw_name: str) -> str:
"""What an unresolved call or class reference targets: its bare raw name, unless
that is the id of a real node, which the edge would join and read as resolved
(#661). Such a name takes the `raw_call:` prefix, which no real id holds and
consumers already read as unresolved; a class reference gives up `raw_class:`."""
return f"{RAW_CALL_PREFIX}{raw_name}" if raw_name in view.nodes else raw_name


def _virtual_rank(node: Node) -> tuple[bool, str]:
"""Which of two readings of one virtual id wins: a known namespace, then the
language that sorts first — never the order edges happen to arrive in."""
Expand Down Expand Up @@ -187,7 +195,7 @@ def _resolved_class_edge(self, edge: Edge) -> Edge:
"""
raw = edge.target.removeprefix(RAW_CLASS_PREFIX)
resolved = self._resolver.resolve_class_ref(raw, edge.source, edge.file_path)
final_target = resolved or raw
final_target = resolved or _unresolved_target(self._index, raw)
confidence = 1.0 if resolved else 0.5
return edge.model_copy(update={"target": final_target, "confidence": confidence})

Expand Down Expand Up @@ -218,7 +226,7 @@ def _resolved_call_edge(self, edge: Edge) -> Edge:
hit = _fallback(self._index, raw_name, file_path)
if hit is not None:
return edge.model_copy(update={"target": hit[0], "confidence": hit[1]})
final_target = new_target or raw_name
final_target = new_target or _unresolved_target(self._index, raw_name)
confidence = RESOLVED_CALL_CONFIDENCE if new_target else _UNRESOLVED_CONFIDENCE
return edge.model_copy(update={"target": final_target, "confidence": confidence})

Expand Down Expand Up @@ -294,7 +302,8 @@ def _ensure_virtual_node(self, edge: Edge, virtual_nodes: dict[str, Node]) -> No
language = source.language if source is not None else _pack_name(source_file)
if existing is not None and _virtual_rank(existing) <= (False, language):
return # nothing this caller can classify would outrank it
namespace = self._index.classify_fqn(target, source_file)
# A collision kept its `raw_call:` prefix (#661); the name itself is classified.
namespace = self._index.classify_fqn(target.removeprefix(RAW_CALL_PREFIX), source_file)
if namespace is NodeNamespace.INTERNAL:
# An internal-root FQN with no node is one of four things, measured on
# owner-api's 681 such nodes (#459, and the review of #493):
Expand All @@ -321,7 +330,7 @@ def _make_virtual_node(self, fqn: str, namespace: NodeNamespace, language: str)
return Node(
id=fqn,
type=NodeType.FUNCTION,
name=fqn.rsplit(".", maxsplit=1)[-1],
name=fqn.removeprefix(RAW_CALL_PREFIX).rsplit(".", maxsplit=1)[-1],
file_path=VIRTUAL_FILE_PATH,
start_line=0,
end_line=0,
Expand Down
2 changes: 1 addition & 1 deletion src/cgis/storage/sqlite_store.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@

from cgis.core.freshness import Freshness, FreshnessState
from cgis.core.models import (
RAW_CALL_PREFIX,
VIRTUAL_FILE_PATH,
Edge,
EdgeType,
Expand Down Expand Up @@ -48,7 +49,6 @@
}
)

RAW_CALL_PREFIX = "raw_call:"
_DELETE_ALL_NODES = "DELETE FROM nodes"
_DELETE_ALL_EDGES = "DELETE FROM edges"

Expand Down
Loading
Loading