Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,8 @@ Prefer the CLI when the connected MCP server is still `python3 -m wikifier.mcp.s
- Do not hand-edit `library.md`, `file_health.md`, `file_health.json`, or `pending_updates.md`. `update-maps` rewrites `library.md`. Health and pending change only through the commands. They take `.wikifier_staging/.lock`.
- `init` copies `index.html` only when it is missing, and it does not clobber an existing path list. `diagnostics.html` stays in this repo. `index.v2.html` and `diagnostics.v2.html` are comparison pages and are not deployed.
- This repo's map roots are `src/` and `tests/` (`map_paths.txt`). `monitored_paths.txt` is the extra watch list for docs and `wikifier.sh`.
- The import scan is comment-masked regex, plus named JS/TS re-exports to depth 8 (`parser: "regex+reexport"`). An indented Python import is conditional. `import()` is dynamic. Rust resolves `mod`, `crate::`, `self::`, `super::`, and the root `Cargo.toml` package or lib name (`-` to `_`). `crate::` stays in that file's crate (`tests/*.rs` and `src/bin/*.rs` included). The package or lib name points at the library, which is how `src/main.rs` and `tests/*.rs` show up as dependents. The accuracy gate is `accuracy_fixtures_match_frozen_edges` (the frozen layouts, including a Cargo crate). Do not describe the scan as a CPython `ast` walk, and do not claim it has been run on Babylon.js. It does not read tsconfig paths, package exports, or workspaces.
- `mark_green` checks Python, JS/TS, and Rust public names against the wiki. Rust names are top-level `pub` items, not `pub(crate)` and not methods inside `impl`. Go, C/C++, C#, Java, and Markdown come back `unchecked`. That is expected.
- The import scan is comment-masked regex, plus named JS/TS re-exports to depth 8 (`parser: "regex+reexport"`). An indented Python import is conditional. `import()` is dynamic. Rust resolves `mod`, `crate::`, `self::`, `super::`, and package or lib names from the root `Cargo.toml` and from `[workspace] members` (globs included, `-` to `_`). `crate::` stays in that file's crate (`tests/*.rs`, `src/bin/*.rs`, and each workspace member). A package or lib name points at that member's library, which is how `src/main.rs` and `tests/*.rs` show up as dependents. The accuracy gate is `accuracy_fixtures_match_frozen_edges` (the frozen layouts, including a Cargo crate). Do not describe the scan as a CPython `ast` walk, and do not claim it has been run on Babylon.js. It does not read tsconfig paths, package exports, or npm workspaces.
- `mark_green` checks Python, JS/TS, and Rust public names against the wiki. Rust names are unrestricted `pub` items, `pub` methods and associated items on a public type, and trait-impl items for a public type. `pub(crate)`, `pub(super)`, `pub(self)`, and `pub(in ...)` stay out. Go, C/C++, C#, Java, and Markdown come back `unchecked`. That is expected.
- `wikifier serve` binds `127.0.0.1:8787`. Do not bind port 8080.
- Do not set the project root to a parent that contains several repos.
- Git: honor hooks. Do not use `--no-verify`. Do not commit `.cursor/`, `target/`, or gitignored leftovers (`wikifier/*.pyc`, `build/`, `dist/`, `.venv-build/`).
Expand Down
2 changes: 1 addition & 1 deletion AGENTS.md.wiki.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@

**Role:** Entry point for any coding agent in this repository.

It points at `skills/run.md` (protocol v0.7) and names the live Rust binary, the daily loop, and the Python-era records kept under `Findings/archive/`. Map roots stay `src/` and `tests/`. The import scan stays comment-masked regex plus named re-exports. Rust also resolves `crate::`, `self::`, `super::`, and the root Cargo.toml package or lib name, so `src/main.rs` and `tests/*.rs` are dependents of the library modules they import. `mark-green` checks Rust top-level `pub` names.
It points at `skills/run.md` (protocol v0.7) and names the live Rust binary, the daily loop, and the Python-era records kept under `Findings/archive/`. Map roots stay `src/` and `tests/`. The import scan stays comment-masked regex plus named re-exports. Rust also resolves `crate::`, `self::`, `super::`, root crate names, and `[workspace] members`, so `src/main.rs` and `tests/*.rs` are dependents of the library modules they import. `mark-green` checks Rust `pub` names, impl methods, and trait-impl items.
3 changes: 2 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- This repo's import-map roots are `src/` and `tests/`.
- The import scan remains comment-masked regex plus named JS/TS re-exports to depth 8 (`parser: "regex+reexport"`). It is not a CPython `ast` walk, and it has not been run on Babylon.js. The accuracy gate is `accuracy_fixtures_match_frozen_edges`, now including a Cargo crate layout.
- Rust `mod`, `crate::`, `self::`, `super::`, and the root `Cargo.toml` package or lib name resolve to files. Hyphens in the package name become underscores. `crate::` stays inside the file's crate (`tests/*.rs` and `src/bin/*.rs` are separate crates). The package or lib name points at the library, so binaries and integration tests are dependents of the library modules they import.
- `mark-green` checks Rust top-level `pub` items (`fn`, `struct`, `enum`, `trait`, `mod`, `type`, `const`, `static`, and `pub use` names) the same way it checks Python and JS/TS names. `pub(crate)` and items inside `impl` stay out of that set. A Rust file can be `verified` Green. Go, C/C++, C#, and Java stay `unchecked`.
- `mark-green` checks Rust unrestricted `pub` items, including methods and associated items inside inherent impls and the items written in a trait impl for a public type. `pub(crate)`, `pub(super)`, `pub(self)`, and `pub(in ...)` stay out of that set. A Rust file can be `verified` Green. Go, C/C++, C#, and Java stay `unchecked`.
- Rust import names come from the root package and from `[workspace] members`, including globs. `use member_crate::...` points at that member. `crate::` stays inside the member that owns the file.
- Tests cover `WIKIFIER_MCP_DEADLINE_S`, `WIKIFIER_CHECK_CHANGES_MAX`, an `import_cache.sqlite` import (including `sqlite_present_unread`), and `actionable_low_conf_edges`.
- CI pins Rust 1.83.0 (`dtolnay/rust-toolchain@1.83.0`). Ubuntu, macOS, and Windows all run `cargo test --locked`, and a Windows failure fails the workflow.

Expand Down
2 changes: 1 addition & 1 deletion CHANGELOG.md.wiki.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@

**Role:** Keep a Changelog history for package 4.7.x.

The 4.7.0 section is the record of that release, including the Python package that shipped then. Unreleased notes describe the Rust binary: map artifacts, dependency flags, wiki checks, deletion, background commands, the dashboard serve routes, MCP error reporting, `heal-stubs --dry-run`, and `init DIR`. The import scan is recorded as `regex+reexport`. Unreleased notes that Rust crate-name imports resolve to the library, that the unreproducible lookup benchmark was removed, that Rust top-level `pub` names can be verified Green, and that the MCP deadline, check-changes cap, sqlite import, and low-confidence edges have tests. CI pins Rust 1.83.0 and requires Ubuntu, macOS, and Windows. CI checks the empty dependency list without Python.
The 4.7.0 section is the record of that release, including the Python package that shipped then. Unreleased notes describe the Rust binary: map artifacts, dependency flags, wiki checks, deletion, background commands, the dashboard serve routes, MCP error reporting, `heal-stubs --dry-run`, and `init DIR`. The import scan is recorded as `regex+reexport`. Unreleased notes that Rust crate-name imports resolve to the library and to `[workspace] members`, that the unreproducible lookup benchmark was removed, that Rust `pub` names including impl methods can be verified Green, and that the MCP deadline, check-changes cap, sqlite import, and low-confidence edges have tests. CI pins Rust 1.83.0 and requires Ubuntu, macOS, and Windows. CI checks the empty dependency list without Python.
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ Pipeline: **scan → parse → resolve → cache → health → artifacts**, imp

- `src/api.rs` — the library surface: `session_bootstrap`, `check_changes`, `prepare_edit`, `suggest_next_actions`, `record_change`, `record_changes`, `record_deletion`, `mark_green`, `verify_wikis`, `update_maps` / `update_maps_run`, `dependency_report`, `load_dependents`, `cycles`, `init_project`, `serve_loop`, `monitor_loop`, `daemon_ctl`. `check_changes` watches mapped files, Green files, and `monitored_paths.txt`. `record_changes` reads `git status` (git is optional).
- `src/scope.rs` — project root, `map_paths.txt`, `monitored_paths.txt`, excludes, and the file walk. A missing configured map directory is a scope error. Three or more child projects make the root a container and the walk stops.
- `src/parse.rs` — comment-masked regex scans for Python, JS/TS, Rust, Go, C/C++, C#, and Java. Named JS/TS re-exports are followed to depth 8 (`via_barrel`). An indented Python import is `conditional`. `import()` is `dynamic`. Rust resolves `mod`, `crate::`, `self::`, `super::`, and the root `Cargo.toml` package or lib name (`-` to `_`). `crate::` stays in the file's crate; the package or lib name points at the library. This is not a CPython `ast` parser and it does not read tsconfig paths, package `exports`, or workspaces. The accuracy gate is `accuracy_fixtures_match_frozen_edges`.
- `src/parse.rs` — comment-masked regex scans for Python, JS/TS, Rust, Go, C/C++, C#, and Java. Named JS/TS re-exports are followed to depth 8 (`via_barrel`). An indented Python import is `conditional`. `import()` is `dynamic`. Rust resolves `mod`, `crate::`, `self::`, `super::`, and package or lib names from the root `Cargo.toml` and `[workspace] members` (`-` to `_`, globs included). `crate::` stays in the file's crate; a member name points at that member's library. This is not a CPython `ast` parser and it does not read tsconfig paths, package `exports`, or npm workspaces. The accuracy gate is `accuracy_fixtures_match_frozen_edges`.
- `src/cache.rs` + `src/cache_commit.rs` — text index at `.wikifier_staging/wk_cache` (per-file payload, `by_src`, `by_dst`, notes, barrels). One commit per `update-maps`. Size and mtime skip unchanged files. `--full` rehashes. `src/legacy_cache.rs` imports an empty cache from `import_cache.sqlite` (the `sqlite3` CLI) or `import_cache.json`.
- `src/artifacts.rs` — `file_health.json` (schema version 2) plus the `file_health.md` view, pending lines, and the journal. Identical journal retries within 120 seconds are skipped. `mark_green` stores `public_symbols`, `wiki_hash`, and `source_mtime_ns`, and refuses a Python or JS/TS wiki that misses the current public names, still mentions a removed name in backticks, or is unchanged after the public surface changed.
- `src/maps_out.rs` — `library.md` (a file tree, a Mermaid block capped at 80 edges, and the dependency table) and sharded folder cards at `.wikifier_staging/maps/_root.json` and `maps/{prefix}/_index.json`.
Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md.wiki.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@

**Role:** Repo guidance for an agent working in this tree.

Hard constraints: empty Cargo dependencies, no hand-edits of `library.md` or the health and pending files, and the human dashboard stays secondary. The Python-era plan logs live under `Findings/archive/`. It points agents at `AGENTS.md` and `skills/run.md`. The architecture section names the Rust modules under `src/`, the library file tree, and this repo's map roots `src/` and `tests/`. The parse note includes Rust `crate::`, `self::`, `super::`, the Cargo.toml crate name, and `rust_public_symbols` for wiki checks. JSON failures use `success: false`.
Hard constraints: empty Cargo dependencies, no hand-edits of `library.md` or the health and pending files, and the human dashboard stays secondary. The Python-era plan logs live under `Findings/archive/`. It points agents at `AGENTS.md` and `skills/run.md`. The architecture section names the Rust modules under `src/`, the library file tree, and this repo's map roots `src/` and `tests/`. The parse note includes Rust `crate::`, `self::`, `super::`, Cargo.toml crate names, `[workspace] members`, and `rust_public_symbols` for wiki checks. JSON failures use `success: false`.
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Works from small scripts to large monorepos. **Deep import/include maps** (zero-
|----------|------------|--------|
| Python | `.py` | comment-masked regex; relative imports and same-tree module paths; an indented import is flagged conditional |
| JavaScript / TypeScript | `.js` `.ts` `.jsx` `.tsx` `.mjs` `.cjs` `.mts` `.cts` | comment-masked regex; named re-exports followed to depth 8; `import()` flagged dynamic |
| Rust | `.rs` | `mod`, `use`, `crate::`, `self::`, `super::`, and the root `Cargo.toml` package or lib name (`-` becomes `_`). A name points at the library; `crate::` stays inside the file's own crate, including `tests/` and `src/bin/` |
| Rust | `.rs` | `mod`, `use`, `crate::`, `self::`, `super::`, and package or lib names from the root `Cargo.toml` and from `[workspace] members` (globs included). `-` becomes `_`. A member name points at that member's library. `crate::` stays inside the crate that owns the file, including `tests/` and `src/bin/` |
| Go | `.go` | `import` / import blocks |
| C / C++ | `.c` `.h` `.cpp` `.cc` `.cxx` `.hpp` `.hh` | `#include` (local + system) |
| C# | `.cs` | `using` namespaces |
Expand Down Expand Up @@ -104,7 +104,7 @@ Advanced intel as needed: `list_paths` (names-only folder expand), `get_dependen
- **Import analysis** — comment-masked regex for Python, JS/TS, Rust, Go, C/C++, C#, and Java. Named JS/TS re-exports are followed to depth 8. Edges carry conditional, dynamic, and via-barrel flags, and a confidence read from those flags.
- **Incremental pipeline** — `update-maps` parses files whose size or mtime changed, then writes the `wk_cache` text index, reverse edges, `library.md`, and sharded folder cards.
- **Two path lists** — `map_paths.txt` = map package roots; `monitored_paths.txt` = extra watch list (docs, scripts). `check-changes` always watches mapped files and anything ever marked Green, so a narrow list never hides edits.
- **Green you can trust** — `mark-green` checks the wiki against Python, JS/TS, and Rust public symbols and refuses when a public name is missing, a removed name is still in backticks, or the public surface changed while the wiki file did not. Rust names are top-level `pub` items: `fn`, `struct`, `enum`, `trait`, `mod`, `type`, `const`, `static`, and the names a `pub use` exports. `pub(crate)` and items inside `impl` are not in that set. Go, C/C++, C#, and Java are recorded as unchecked. `verify-wikis` re-checks every Green wiki. `health --summary` includes `green_breakdown`.
- **Green you can trust** — `mark-green` checks the wiki against Python, JS/TS, and Rust public symbols and refuses when a public name is missing, a removed name is still in backticks, or the public surface changed while the wiki file did not. Rust names are unrestricted `pub` items, including methods and associated items on a public type and the items written in a trait impl for that type. `pub(crate)`, `pub(super)`, `pub(self)`, and `pub(in ...)` stay out of that set. Go, C/C++, C#, and Java are recorded as unchecked. `verify-wikis` re-checks every Green wiki. `health --summary` includes `green_breakdown`.
- **Partial-map honesty** — `--max-files` returns `map_complete=false` and `files_remaining_dirty`. Run the same command again to resume. Already-parsed files are stat-skipped.
- **Cache** — `.wikifier_staging/wk_cache`. `wikifier cache-status` reports the backend and the file count. An empty cache imports `import_cache.sqlite` through the `sqlite3` CLI, or `import_cache.json`.
- **Selective agent work** — health and `suggest-next` stay on 🔴 and actionable 🟡. Diagnostics list ACS reason codes and `actionable_low_conf_edges`.
Expand Down
2 changes: 1 addition & 1 deletion README.md.wiki.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@

**Role:** The public entry point for the std-only Rust program.

It tells agents to build with `cargo build --release`, start in this repo at `AGENTS.md`, and follow `skills/run.md`. Bootstrap a target with `wikifier init`. The language table describes the comment-masked regex scans, including Rust crate-name imports. The lookup benchmark that cited a missing `scripts/benchmark_lookup.py` is gone. The performance table that remains says its numbers are from the Python package. `mark-green` checks Rust top-level `pub` names as well as Python and JS/TS. MCP is `wikifier mcp`.
It tells agents to build with `cargo build --release`, start in this repo at `AGENTS.md`, and follow `skills/run.md`. Bootstrap a target with `wikifier init`. The language table describes the comment-masked regex scans, including Rust crate-name imports. The lookup benchmark that cited a missing `scripts/benchmark_lookup.py` is gone. The performance table that remains says its numbers are from the Python package. `mark-green` checks Rust `pub` items, impl methods, and trait-impl items, as well as Python and JS/TS. Rust also reads `[workspace] members`. MCP is `wikifier mcp`.
Loading
Loading