diff --git a/AGENTS.md b/AGENTS.md index d116569..aca9fad 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -30,10 +30,10 @@ Prefer the CLI when the connected MCP server is still `python3 -m wikifier.mcp.s - `Cargo.toml` keeps an empty `[dependencies]` list. Do not add a crates.io crate, a Python package, or a Node module. - Do not hand-edit `library.md`, `file_health.md`, `file_health.json`, or `pending_updates.md`. `update-maps` rewrites `library.md`. Health and pending change only through the commands. They take `.wikifier_staging/.lock`. -- `init` copies `index.html` only when it is missing, and it does not clobber an existing path list. `diagnostics.html` stays in this repo. `index.v2.html` and `diagnostics.v2.html` are comparison pages and are not deployed. +- `init` copies `index.html` only when it is missing (`--no-copy` skips it), and it does not clobber an existing path list. It also creates `exclude_patterns.txt`, empty health, pending, and library files, `.wikifier/config`, `journal/`, and `Logged_issues/` when those paths are absent. `diagnostics.html` stays in this repo. `index.v2.html` and `diagnostics.v2.html` are comparison pages and are not deployed. - This repo's map roots are `src/` and `tests/` (`map_paths.txt`). `monitored_paths.txt` is the extra watch list for docs and `wikifier.sh`. -- The import scan is comment-masked regex, plus named JS/TS re-exports to depth 8 (`parser: "regex+reexport"`). An indented Python import is conditional. `import()` is dynamic. Rust resolves `mod`, `crate::`, `self::`, `super::`, and package or lib names from the root `Cargo.toml` and from `[workspace] members` (globs included, `-` to `_`). `crate::` stays in that file's crate (`tests/*.rs`, `src/bin/*.rs`, and each workspace member). A package or lib name points at that member's library, which is how `src/main.rs` and `tests/*.rs` show up as dependents. The accuracy gate is `accuracy_fixtures_match_frozen_edges` (the frozen layouts, including a Cargo crate). Do not describe the scan as a CPython `ast` walk, and do not claim it has been run on Babylon.js. It does not read tsconfig paths, package exports, or npm workspaces. -- `mark_green` checks Python, JS/TS, and Rust public names against the wiki. Rust names are unrestricted `pub` items, `pub` methods and associated items on a public type, and trait-impl items for a public type. `pub(crate)`, `pub(super)`, `pub(self)`, and `pub(in ...)` stay out. Go, C/C++, C#, Java, and Markdown come back `unchecked`. That is expected. +- The import scan is comment-masked regex, plus named JS/TS re-exports to depth 8 (`parser: "regex+reexport"`). An indented Python import is conditional. `import()` is dynamic. JS/TS resolution reads `tsconfig.json` `paths`, `package.json` `exports`, and npm `workspaces` (`*` directory globs). Rust resolves `mod`, `crate::`, `self::`, `super::`, and package or lib names from the root `Cargo.toml` and from `[workspace] members` (globs included, `-` to `_`). `crate::` stays in that file's crate (`tests/*.rs`, `src/bin/*.rs`, and each workspace member). A package or lib name points at that member's library, which is how `src/main.rs` and `tests/*.rs` show up as dependents. The accuracy gate is `accuracy_fixtures_match_frozen_edges` (the frozen layouts, including a Cargo crate). Do not describe the scan as a CPython `ast` walk, and do not claim it has been run on Babylon.js. +- `mark_green` checks Python, JS/TS, and Rust public names against the wiki. Python names come from a tokenizer (`__all__`, top-level `def`/`class`, `.pyi`). JS/TS names are exports after comments and strings are masked. Rust names are unrestricted `pub` items, `pub` methods and associated items on a public type, and trait-impl items for a public type. `pub(crate)`, `pub(super)`, `pub(self)`, and `pub(in ...)` stay out. Go, C/C++, C#, Java, and Markdown come back `unchecked`. That is expected. Wiki lookup is the sibling `.wiki.md`, then `.md`, then `docs/wiki`, `docs`, `wiki`, `documentation`, and `.wiki`. - `wikifier serve` binds `127.0.0.1:8787`. Do not bind port 8080. - Do not set the project root to a parent that contains several repos. - Git: honor hooks. Do not use `--no-verify`. Do not commit `.cursor/`, `target/`, or gitignored leftovers (`wikifier/*.pyc`, `build/`, `dist/`, `.venv-build/`). diff --git a/CHANGELOG.md b/CHANGELOG.md index 54983b0..1a3b983 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -29,7 +29,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - `heal-stubs --dry-run` counts the stubs it would mark and does not write health. `seed-health` and `seed-source-hashes` return `HealthError` when the health write fails. - `init DIR` uses that directory as the project root unless `--target` is set. `verify-wikis DIR` limits the check to that file or directory. `search-journal --file` keeps only that path. - The CI empty-dependency check reads `Cargo.toml` with `awk`. The workflow does not run Python. -- The live daemon actions are `start`, `stop`, `status`, and `run`. +- Daemon actions are `start`, `stop`, `status`, `run`, `logs`, `restart`, `install-service`, and `uninstall-service`. The loop checks every 30 seconds and refreshes maps at start and then at most every 600 seconds. `start` refuses only readiness `blocked` unless `--force`. +- `init --no-copy` skips the dashboard. `init` writes `exclude_patterns.txt`, empty health, pending, and library files, `.wikifier/config`, `journal/`, and `Logged_issues/` when missing. +- Wiki lookup falls back from `.wiki.md` through `docs/wiki`, `docs`, `wiki`, `documentation`, and `.wiki`. Python and JS/TS public names ignore comments and strings. Python `__all__` may span lines. `.pyi` is checked. +- `record-changes --reasons` accepts a JSON object or `PATH=REASON` lines. `health --summary` omits the entry list. `list-paths` honors `--depth`, `--recursive`, and `--max-files`. `cycles` includes severity and a reverse-dependency ranking. `--text` prints prose; the default remains JSON. +- `suggest-next` emits `review_low_confidence_edges` when a non-dynamic resolved edge is below 0.65. `seed-source-hashes` records hashes on Green rows and does not change status. `heal-stubs` defaults to a 350-character wiki. `update-maps` seeds health rows only for paths in `monitored_paths.txt` when that file lists real paths. +- Pending bullets use `- path: message`. - This repo's import-map roots are `src/` and `tests/`. - The import scan remains comment-masked regex plus named JS/TS re-exports to depth 8 (`parser: "regex+reexport"`). It is not a CPython `ast` walk, and it has not been run on Babylon.js. The accuracy gate is `accuracy_fixtures_match_frozen_edges`, now including a Cargo crate layout. - Rust `mod`, `crate::`, `self::`, `super::`, and the root `Cargo.toml` package or lib name resolve to files. Hyphens in the package name become underscores. `crate::` stays inside the file's crate (`tests/*.rs` and `src/bin/*.rs` are separate crates). The package or lib name points at the library, so binaries and integration tests are dependents of the library modules they import. diff --git a/CLAUDE.md b/CLAUDE.md index 3872190..f8179d4 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -60,7 +60,7 @@ Pipeline: **scan → parse → resolve → cache → health → artifacts**, imp - `src/api.rs` — the library surface: `session_bootstrap`, `check_changes`, `prepare_edit`, `suggest_next_actions`, `record_change`, `record_changes`, `record_deletion`, `mark_green`, `verify_wikis`, `update_maps` / `update_maps_run`, `dependency_report`, `load_dependents`, `cycles`, `init_project`, `serve_loop`, `monitor_loop`, `daemon_ctl`. `check_changes` watches mapped files, Green files, and `monitored_paths.txt`. `record_changes` reads `git status` (git is optional). - `src/scope.rs` — project root, `map_paths.txt`, `monitored_paths.txt`, excludes, and the file walk. A missing configured map directory is a scope error. Three or more child projects make the root a container and the walk stops. -- `src/parse.rs` — comment-masked regex scans for Python, JS/TS, Rust, Go, C/C++, C#, and Java. Named JS/TS re-exports are followed to depth 8 (`via_barrel`). An indented Python import is `conditional`. `import()` is `dynamic`. Rust resolves `mod`, `crate::`, `self::`, `super::`, and package or lib names from the root `Cargo.toml` and `[workspace] members` (`-` to `_`, globs included). `crate::` stays in the file's crate; a member name points at that member's library. This is not a CPython `ast` parser and it does not read tsconfig paths, package `exports`, or npm workspaces. The accuracy gate is `accuracy_fixtures_match_frozen_edges`. +- `src/parse.rs` — comment-masked regex scans for Python, JS/TS, Rust, Go, C/C++, C#, and Java. Named JS/TS re-exports are followed to depth 8 (`via_barrel`). An indented Python import is `conditional`. `import()` is `dynamic`. JS/TS resolution reads `tsconfig.json` `paths`, `package.json` `exports`, and npm `workspaces`. Rust resolves `mod`, `crate::`, `self::`, `super::`, and package or lib names from the root `Cargo.toml` and `[workspace] members` (`-` to `_`, globs included). `crate::` stays in the file's crate; a member name points at that member's library. This is not a CPython `ast` parser. The accuracy gate is `accuracy_fixtures_match_frozen_edges`. - `src/cache.rs` + `src/cache_commit.rs` — text index at `.wikifier_staging/wk_cache` (per-file payload, `by_src`, `by_dst`, notes, barrels). One commit per `update-maps`. Size and mtime skip unchanged files. `--full` rehashes. `src/legacy_cache.rs` imports an empty cache from `import_cache.sqlite` (the `sqlite3` CLI) or `import_cache.json`. - `src/artifacts.rs` — `file_health.json` (schema version 2) plus the `file_health.md` view, pending lines, and the journal. Identical journal retries within 120 seconds are skipped. `mark_green` stores `public_symbols`, `wiki_hash`, and `source_mtime_ns`, and refuses a Python or JS/TS wiki that misses the current public names, still mentions a removed name in backticks, or is unchanged after the public surface changed. - `src/maps_out.rs` — `library.md` (a file tree, a Mermaid block capped at 80 edges, and the dependency table) and sharded folder cards at `.wikifier_staging/maps/_root.json` and `maps/{prefix}/_index.json`. diff --git a/README.md b/README.md index c8b0713..e282623 100644 --- a/README.md +++ b/README.md @@ -32,7 +32,7 @@ Works from small scripts to large monorepos. **Deep import/include maps** (zero- | C# | `.cs` | `using` namespaces | | Java | `.java` | `import` / `import static` | -Health/journal still work for any monitored path. The non-Python parsers are pragmatic regex (not full cargo/`go.mod`/classpath/`-I` resolution). On huge monorepos, split scope deliberately: +JavaScript and TypeScript resolution also reads `tsconfig.json` `paths`, `package.json` `exports`, and npm `workspaces`. Health/journal still work for any monitored path. The non-Python parsers are pragmatic regex (not full cargo/`go.mod`/classpath/`-I` resolution). On huge monorepos, split scope deliberately: | File | Surface | |------|---------| @@ -135,6 +135,25 @@ Warm **0-dirty** re-runs after **4.6.7** (same machine class; scoped; candidates Residual floor on large scopes is mtime/stat + live count under MapScope (not full JSON re-walk). Sub-100ms is not a hard SLA on every 1k+ tree. +## Benchmarks + +Python 4.7 at `24812ef` versus this Rust release build, 5 runs, median milliseconds (min–max). Machine: Intel Xeon, 4 cores, 15.6 GiB RAM, Linux 6.12.94+, Python 3.12.3, rustc 1.83.0. Cold deletes `.wikifier_staging` first. Warm is the next `update-maps`. Full notes, peak RSS, and commands: [`scripts/benchmark_results.md`](scripts/benchmark_results.md). Reproduce with `scripts/benchmark_parity.sh`. + +| Command | small Python | small Rust | medium Python | medium Rust | large Python | large Rust | +|---|---:|---:|---:|---:|---:|---:| +| startup (`--version`) | 63 (61–66) | 2 (1–2) | same pair | same pair | same pair | same pair | +| cold update-maps | 123 (121–126) | 8 (7–13) | 167 (163–168) | 62 (59–73) | 12675 (12540–13094) | 3142 (3114–3305) | +| warm update-maps | 69 (67–72) | 4 (3–5) | 73 (72–74) | 4 (4–5) | 422 (417–433) | 246 (243–253) | +| session-bootstrap | 66 (64–67) | 2 (1–2) | 65 (64–67) | 2 (1–2) | 270 (269–278) | 21 (21–21) | +| check-changes | 64 (62–65) | 3 (2–3) | 70 (69–75) | 5 (5–6) | 1232 (1217–1266) | 74 (73–76) | +| health --summary | 60 (60–62) | 2 (1–2) | 60 (59–63) | 1 (1–2) | 109 (108–114) | 8 (8–9) | +| verify-wikis | 60 (60–61) | 2 (1–2) | 62 (60–67) | 1 (1–2) | 106 (103–108) | 8 (7–8) | +| cycles | 60 (60–61) | 2 (1–2) | 61 (60–63) | 2 (2–2) | 63 (61–64) | 54 (53–55) | + +Resolved file edges after a warm map: small 2 and 2, medium (this repo) 44 Python and 98 Rust, large (Django `django/` + `tests/`, 2939 Python files) 9074 and 9074. Rust cold `update-maps` peak RSS was 3384 / 4424 / 40012 kbytes; Python was 27544 / 28392 / 97564 kbytes. The release binary is 2197456 bytes. + +Small is three modules under `src/pkg`. Medium is `src/` and `tests/`. Large is a shallow clone of Django. + ### Parser accuracy `cargo test` checks internal edges on the frozen accuracy layouts: @@ -163,13 +182,13 @@ Tests: `cargo test` (std-only Rust; accuracy fixtures, cache lookups, and the ag | `wikifier verify-wikis [dir]` | Re-check every Green wiki against its code (exit 1 on failures) | | `wikifier record-deletion "reason"` | Mark removed paths 🔴, drop them from the graph, prune barrel refs | | `wikifier dependencies [--full]` / `dependents ` | What a file imports (compact; `--full` for per-edge confidence records) / who imports it (JSON) | -| `wikifier suggest-next` | Next actions (🔴/actionable 🟡 only; `--json` for `actions[]`) | +| `wikifier suggest-next` | Next actions (JSON `actions[]`; `--text` for prose). Includes low-confidence edges when any exist | | `wikifier update-maps [--directory=src/] [--max-files=N]` | Rebuild graph + human `library.md` + agent folder maps (warm 0-dirty is fast; honors `map_paths.txt`) | | `wikifier cache-status` | `wk_cache` backend and mapped file count | -| `wikifier health [--summary\|--json]` | Health matrix (machine-friendly flags) | +| `wikifier health [--summary\|--format text]` | Health matrix. Default is JSON. `--summary` is counts only. `--text` or `--format text` is prose | | `wikifier validate` | Missing wiki rows + ghost paths | -| `wikifier cycles [--json]` | Circular deps + break hints | -| `wikifier heal-stubs [--dry-run]` | Promote Initial stubs that now have a real wiki | +| `wikifier cycles [--no-analysis] [--max-items N]` | Circular deps, break hints, severity, and who is most depended on | +| `wikifier heal-stubs [--dry-run]` | Promote Initial stubs whose wiki is at least 350 characters | | `wikifier monitor` / `daemon` | Background maintenance (`WIKIFIER_DAEMON_MAPS=0` for check-only) | | `wikifier serve` | Localhost dashboard with Run/Stop | diff --git a/file_health.json b/file_health.json index 64c196a..5ac356a 100644 --- a/file_health.json +++ b/file_health.json @@ -1 +1 @@ -{"entries":{"AGENTS.md":{"last_updated":"2026-11-03 05:49:16","reason":"Check impl items and resolve workspace member crates (subid=impl-workspace)","source_content_hash":"sha256:4c3c9a11f3c42c26325dfcb0f5b907814fc536165abd8f8c4139684077445151","source_mtime_ns":"1793684917066518484","status":"🟢 Green","verification":"unchecked","wiki_hash":"sha256:d5907471b6831952e34e64a033d471bb0264ef4dee92151ca2c0eaf40b1d16ef"},"CHANGELOG.md":{"last_updated":"2026-11-03 05:49:16","reason":"Check impl items and resolve workspace member crates (subid=impl-workspace)","source_content_hash":"sha256:c4d7b852c1656257acf0a79ba7623bf69d58b8517a4a3e2d6b0649b55d41c9ff","source_mtime_ns":"1793684916746519688","status":"🟢 Green","verification":"unchecked","wiki_hash":"sha256:43a7a238cc155970024675795f4d1f1fc0000e22cd22898ca7206c57dc111e67"},"CLAUDE.md":{"last_updated":"2026-11-03 05:49:16","reason":"Check impl items and resolve workspace member crates (subid=impl-workspace)","source_content_hash":"sha256:a3e8e8efe945e955c91fe596e51ecd0dcec709be5e9f397134abf46dab29a3bd","source_mtime_ns":"1793684916630520123","status":"🟢 Green","verification":"unchecked","wiki_hash":"sha256:afc87f10c92c7cf033d4a5af38ff318ccc1d8ff069b88cab22b3563552ea071b"},"CONTRIBUTING.md":{"last_updated":"2026-11-03 05:34:35","reason":"Address the 3 Nov review findings (subid=review-2026-11-03)","source_content_hash":"sha256:d090280cd3d1c3ab1cf342d69392471ed3835512ff7381bf4409bada57bbe89c","source_mtime_ns":"1793683924316411256","status":"🟢 Green","verification":"unchecked","wiki_hash":"sha256:ec0bd9342b8e9dfeefaabc8dcddeb577f6bd4f1c760d7286b21d709c2a372b29"},"Findings/M5-Dogfood-Assessment-Report.md":{"last_updated":"2026-07-09 05:02:27","reason":"kept; stale yellow cleared","source_content_hash":"sha256:238435188b1699e5bcf4753b62a6fe55cb064413a291259f0b696b5bd3c1692f","source_mtime_ns":"1780519956000000000","status":"🟢 Green"},"Findings/M5-Dogfood-Progress.md":{"last_updated":"2026-11-03 08:06:46","reason":"Wiki marks this file as the historical M5 log (subid=protocol-v07)","source_content_hash":"sha256:ef55167e819806060dfd7dfe7cdfd4d02024ca97706a7faa243710e2630b65fe","source_mtime_ns":"1793408707259550076","status":"🟢 Green","verification":"unchecked","wiki_hash":"sha256:b6b1fa4bbee0c7e8c4035cd925f6d07e1221ba483e3472878d5b88236c5cf7f9"},"Findings/M5.1-cross-hardening-analysis.md":{"last_updated":"2026-07-09 05:02:27","reason":"kept; stale yellow cleared","source_content_hash":"sha256:a2c2414e5b6c9b1a802bec8734a7924850d546cd7746a8087216f3685ca36cb4","source_mtime_ns":"1780519762000000000","status":"🟢 Green"},"Findings/Milestones-Overview.md":{"last_updated":"2026-07-09 05:02:27","reason":"kept; stale yellow cleared","source_content_hash":"sha256:f8b7030ec8980774cf3a2560ad7aee6537263eadf8374e650718f7c6793c6d50","source_mtime_ns":"1780800604000000000","status":"🟢 Green"},"Findings/p6_real_world_validation_report.md":{"last_updated":"2026-07-09 05:02:27","reason":"kept; stale yellow cleared","source_content_hash":"sha256:0468224675888f2747674324726c0ea9a5dddded39d9a1a50f22fc3357dafc0f","source_mtime_ns":"1780520008000000000","status":"🟢 Green"},"README.md":{"last_updated":"2026-11-03 05:49:16","reason":"Check impl items and resolve workspace member crates (subid=impl-workspace)","source_content_hash":"sha256:f27b8eaecf2f176d90593fb971d8b4fd6bce24128317136840e5dfdc9ed67d9f","source_mtime_ns":"1793684917046518560","status":"🟢 Green","verification":"unchecked","wiki_hash":"sha256:725a9cdfc3d48d9a343a319b192be15d107543a64246438ef72e83e710cb0667"},"docs/spec.md":{"last_updated":"2026-11-03 08:00:11","reason":"Name the Rust binary and protocol v0.7 in the living principles (subid=protocol-v07)","source_content_hash":"sha256:31fe6b3b3d705628692eb0512cf44cbb21861ccfc5bb02afdd46d796ec066c6e","source_mtime_ns":"1793653008580824356","status":"🟢 Green","verification":"unchecked","wiki_hash":"sha256:a58b929d25d510f15f9fe176d7823e852f6d089e398accc2e22e1ed189f260ac"},"skills/run.md":{"last_updated":"2026-11-03 05:49:16","reason":"Check impl items and resolve workspace member crates (subid=impl-workspace)","source_content_hash":"sha256:9ba8d1ee8bd5bc3552a6d27a30c8c1f46253599fa6cecac1fdca44c0cfc7c1a0","source_mtime_ns":"1793684917474516945","status":"🟢 Green","verification":"unchecked","wiki_hash":"sha256:5f6e7868ee9adee68bd634352413338e9a367eac640c623545bcad60f64b6389"},"src/api.rs":{"last_updated":"2026-11-03 05:34:35","public_symbols":"MapRun\nVERSION\nMCP_TOOLS\nupdate_maps\nupdate_maps_opts\nupdate_maps_run\nmapped_edges\nload_dependencies\ndependency_report\nload_dependents\nlist_paths\nsession_bootstrap\nsuggest_next_actions\ncheck_changes\nprepare_edit\nrecord_change\nmark_green\nhealth\nread_health\nread_journal\ncache_status\nvalidate\ncycles\nverify_wikis\nrecord_changes\nrecord_changes_opts\nrecord_deletion\nheal_stubs\nprune_health\nprune_health_opts\nprune_pending\nstale_wikis\nprune_barrels\nmetrics_snapshot\nhealing_stats\nresolution_diagnostics\nbarrel_reports\nincremental_status\nserve_once\nserve_loop\nmonitor_loop\ndaemon_ctl\ndispatch\nCLI_COMMANDS\nensure_root","reason":"Address the 3 Nov review findings (subid=review-2026-11-03)","source_content_hash":"sha256:bb9cda81c00a44543c1888c3b0e7f493390d9baee9b8675ad4b1580799e53ca8","source_mtime_ns":"1793684003575439733","status":"🟢 Green","verification":"verified","wiki_hash":"sha256:69ac1101639d23ac6893adf37e501a820e277ace0ddeaf3c2c80afba28557e54"},"src/artifacts.rs":{"last_updated":"2026-11-03 05:49:23","public_symbols":"HealthRow\nread_health\nwrite_health\nupsert_health_row\nJournalEntry\njournal_path\nread_journal\nparse_journal\nJournalWrite\nappend_journal\nread_pending\nfolder_card\nnew\nkeep_baseline_from","reason":"Wiki names impl items (subid=impl-workspace)","source_content_hash":"sha256:37b29254a15e6c09e76a54d1c5a7d0aa7fc54b25044e17db84f345e55cbfa790","source_mtime_ns":"1793678007374624809","status":"🟢 Green","verification":"verified","wiki_hash":"sha256:75bf0f72cc4bd57b6178dc6a0a58b8bf97bc3e7265485a28102093b967f20181"},"src/cache.rs":{"last_updated":"2026-11-03 05:49:23","public_symbols":"check_consistency\nrebuild\nConsistency\nSession\npayload_path\nby_src_path\ncommit_pending_path\nload_dependencies\nInternalEdge\nconfidence_of\nformat_edge_line\nDependencyView\nload_dependency_view\nload_dependents\nlist_prefix\nall_paths\nFileUpdate\nupsert_file\nremove_file\nindexed_rows\npayload_bytes\nstat_ns\nedges\nrel_string\nwrite_barrel_record\nprune_barrel_files\nconfidence\nacs_reason","reason":"Wiki names impl items (subid=impl-workspace)","source_content_hash":"sha256:79faa2b50bc95a573a1bd3555bcafbca271221af61849201df999cb229b4be63","source_mtime_ns":"1793678007370625780","status":"🟢 Green","verification":"verified","wiki_hash":"sha256:afb7493ff8742682f5f3c743da8919408502e6a6e75bba9998eb59cbbaf1c02a"},"src/cache_commit.rs":{"last_updated":"2026-11-03 05:49:23","public_symbols":"Consistency\nSession\ncheck_consistency\nrebuild\nopen\nknown\ndsts_of\nnote_stat\nrels\nupsert\nremove\nstage_barrel\ncommit","reason":"Wiki names impl items (subid=impl-workspace)","source_content_hash":"sha256:71ce0df880b4ba70abac47509ee22f03a4835d51f4ca91cd9249720abd5833e8","source_mtime_ns":"1793684429116993727","status":"🟢 Green","verification":"verified","wiki_hash":"sha256:c6f3fbdcd6aafe62de119c302e09e11ae0ea208681c52166cfcf79de80bb5f6c"},"src/jsonutil.rs":{"last_updated":"2026-11-03 05:49:23","public_symbols":"Json\nparse\nparse_jsonc\nstrip_json_comments\nobj\nfrom_pairs\nget\nas_str\nas_bool\nas_array\nas_object\ninsert\nfmt","reason":"Wiki names impl items (subid=impl-workspace)","source_content_hash":"sha256:18a53e0aca984c6ba4e58132ec21a194cc08733a5e7ad4fc22c003c04a5174fd","source_mtime_ns":"1793684429116993727","status":"🟢 Green","verification":"verified","wiki_hash":"sha256:fb4cffba8b14a179dd90f201b61df2608ca78a9775e9646e7f011f1b9fa07c63"},"src/legacy_cache.rs":{"last_updated":"2026-11-03 05:34:35","public_symbols":"import_if_empty","reason":"Address the 3 Nov review findings (subid=review-2026-11-03)","source_content_hash":"sha256:5bfe43e2a43b138f28d5daeecb8576f90a9c82bf01569f4d9d3e84339f1a2507","source_mtime_ns":"1793683981803718735","status":"🟢 Green","verification":"verified","wiki_hash":"sha256:a6fe6c79185a46cc4b30d215475bab28f517f6e5a6fd7bce6423c2792814aac8"},"src/lib.rs":{"last_updated":"2026-11-03 05:35:24","public_symbols":"api\nartifacts\ncache\njsonutil\nmaps_out\nmcp\nparse\nscope\nsha256\ntimeutil\ncache_status\ncheck_changes\nhealth\nlist_paths\nload_dependencies\nload_dependents\nmapped_edges\nmark_green\nprepare_edit\nread_health\nread_journal\nrecord_change\nsession_bootstrap\nsuggest_next_actions\nupdate_maps\nVERSION\nJson","reason":"Wiki names the crate re-exports (subid=review-w3)","source_content_hash":"sha256:d3afbbbe2faf0f68c7e4c4148f8aec42a728b1c529c8615689798a1a9f9b039d","source_mtime_ns":"1793678007370625780","status":"🟢 Green","verification":"verified","wiki_hash":"sha256:d7e771a3ff63ab55a7967195e3b403dd75d5ae020a383adc56f0e5c09d7b03a0"},"src/main.rs":{"last_updated":"2026-11-03 05:34:35","reason":"Address the 3 Nov review findings (subid=review-2026-11-03)","source_content_hash":"sha256:c29545879d40673183e8e9477dc2f80ec56326a1f33f19a640350f207a7a94e6","source_mtime_ns":"1793683990351610198","status":"🟢 Green","verification":"verified","wiki_hash":"sha256:db6e1cd5beeb90e5a65b3e00a37ee2c73ea4c53750771151e655459a1b823ff5"},"src/maps_out.rs":{"last_updated":"2026-11-03 05:35:24","public_symbols":"write\nwrite_status\nread_status","reason":"Wiki names read_status (subid=review-w3)","source_content_hash":"sha256:6f4523bfae50356ce8c90f2c840f04a79eb99c4ce889e985dd80432cba35d410","source_mtime_ns":"1793678007370625780","status":"🟢 Green","verification":"verified","wiki_hash":"sha256:d441c395fe565828a87c54d5545e4c89e4ec4cfc49308a08525e6dcf3c744e75"},"src/mcp.rs":{"last_updated":"2026-11-03 05:34:35","public_symbols":"tool_schema\nhandle_message\nserve_stdio","reason":"Address the 3 Nov review findings (subid=review-2026-11-03)","source_content_hash":"sha256:2b6a92af8522c7a5047fe64beee86bf1301992ec07d2e65263fe93e593bfc3c5","source_mtime_ns":"1793683982335712018","status":"🟢 Green","verification":"verified","wiki_hash":"sha256:8414b1757bf3acc39a66a52443e9625cb027173b2783d996e39e9f707a9a1933"},"src/parse.rs":{"last_updated":"2026-11-03 05:49:16","public_symbols":"RawImport\nKind\nScan\nlanguage\nread_utf8\nscan_file_warned\nscan_file\nresolve_project\nrust_public_symbols\nresolve_python_spec\nFactEdge\nFileFacts\nAnalysis\nanalyze_files\nanalyze_changed\nedges_for_files","reason":"Check impl items and resolve workspace member crates (subid=impl-workspace)","source_content_hash":"sha256:a188a97f91505c56c5c7a30ef0cab6ab3d627905a7a6aea76c077897a97bd969","source_mtime_ns":"1793684824322776381","status":"🟢 Green","verification":"verified","wiki_hash":"sha256:c70af52c81a4dfffb364cba13ca21873491e83e1c9c5cee31ab2bab2bc379fbb"},"src/proc.rs":{"last_updated":"2026-11-03 07:58:59","reason":"Initial stub","source_content_hash":"sha256:a4751d64c912e3eb9b5608410b1555a0495ee7114174ea03500fffd5336ea776","status":"🟡 Yellow","verification":"no_wiki"},"src/project_lock.rs":{"last_updated":"2026-11-03 05:49:23","public_symbols":"ProjectLock\ndrop\nacquire","reason":"Wiki names impl items (subid=impl-workspace)","source_content_hash":"sha256:d7f0448fccc96b9f9f4294f0abadb9044ee24551418180530eafd331b71b4ab5","source_mtime_ns":"1793678007370625780","status":"🟢 Green","verification":"verified","wiki_hash":"sha256:fccc8d547f30262ff64d506d842472623e828ebe39fde71532a683c6956329d2"},"src/scope.rs":{"last_updated":"2026-11-03 07:58:59","reason":"Initial stub","source_content_hash":"sha256:4d80b5432823864d0b95db68a558cd25aa0fe7d35b3f5796fbbdf107bee709aa","status":"🟡 Yellow","verification":"no_wiki"},"src/sha256.rs":{"last_updated":"2026-11-03 07:58:59","reason":"Initial stub","source_content_hash":"sha256:a68691e08d8260937e0f6c2534faf02ef58bb8332efa2cdd1b0a59f8fdf0d77d","status":"🟡 Yellow","verification":"no_wiki"},"src/timeutil.rs":{"last_updated":"2026-11-03 07:58:59","reason":"Initial stub","source_content_hash":"sha256:a90359c6ab1c816b7d7b2f9ce12c799d3f6c8d560de0ed491c7f069be6fbaa14","status":"🟡 Yellow","verification":"no_wiki"},"tests/behaviour_gaps.rs":{"last_updated":"2026-11-03 05:49:16","reason":"Check impl items and resolve workspace member crates (subid=impl-workspace)","source_content_hash":"sha256:e77dbec8b9cfcc9f81edcfc6937f407a80a1cd054899bb54c61680444497892a","source_mtime_ns":"1793684893298602299","status":"🟢 Green","verification":"verified","wiki_hash":"sha256:b9d20a310ca48ffd858e6d0d4e49b5e0568ecb57dc5ad981eb54398562864691"},"tests/cache_and_artifacts.rs":{"last_updated":"2026-11-03 07:58:59","reason":"Initial stub","source_content_hash":"sha256:e147bf88f37a4748bde1c20657b911da6ca329b99867d854eea6f30a7562fd30","status":"🟡 Yellow","verification":"no_wiki"},"tests/cli_surface.rs":{"last_updated":"2026-11-03 07:58:59","reason":"Initial stub","source_content_hash":"sha256:c88cd894134edacd5cfca1895d38fc25b1f12ebfd86d7be98033863238b5414c","status":"🟡 Yellow","verification":"no_wiki"},"tests/contract_gaps.rs":{"last_updated":"2026-11-03 07:58:59","reason":"Initial stub","source_content_hash":"sha256:ece7c1aa0d6c311280d6f93a902d8faa9e4645295730299598cabfec469ed56d","status":"🟡 Yellow","verification":"no_wiki"},"tests/dogfood.rs":{"last_updated":"2026-11-03 05:34:35","reason":"Address the 3 Nov review findings (subid=review-2026-11-03)","source_content_hash":"sha256:4efe688e0e9b58c8f0b0ca986dad90dfd837c4e36816579353023ede19044d3a","source_mtime_ns":"1793683337392806763","status":"🟢 Green","verification":"verified","wiki_hash":"sha256:9546cc2062f19584c348419322f07d25d353c7f8ba0257e992372f63d798c020"},"tests/parity.rs":{"last_updated":"2026-11-03 05:34:35","reason":"Address the 3 Nov review findings (subid=review-2026-11-03)","source_content_hash":"sha256:4eb986e4a3f922d917672f2df45644726f1f3a8cd132b0bf089740c91678a6d0","source_mtime_ns":"1793683314773282057","status":"🟢 Green","verification":"verified","wiki_hash":"sha256:32753c244642df6e08f8578c6203f77225363783c15bc57c33a84f03989f2378"},"tests/protocol_v07.rs":{"last_updated":"2026-11-03 05:49:16","reason":"Check impl items and resolve workspace member crates (subid=impl-workspace)","source_content_hash":"sha256:401a1187e9f8e7322ebf3658985fea0912d907959a3c28b00642a65ea875340c","source_mtime_ns":"1793684781738827745","status":"🟢 Green","verification":"verified","wiki_hash":"sha256:b38f4a13e316cefd56070f822980b4f83d9cdacb850ab1978de48ed35c630ec8"},"tests/robustness.rs":{"last_updated":"2026-11-03 07:58:59","reason":"Initial stub","source_content_hash":"sha256:1216cf8aa62c31215bba2107580e3c6824bed736ebd212670f68d8a33ecf0103","status":"🟡 Yellow","verification":"no_wiki"},"tests/skip_maps.rs":{"last_updated":"2026-11-03 07:58:59","reason":"Initial stub","source_content_hash":"sha256:4c5706c7ce384e330517848e51c690369d71e00b22b25ddfa19c9485f9988c43","status":"🟡 Yellow","verification":"no_wiki"},"wikifier.sh":{"last_updated":"2026-11-03 08:06:46","reason":"Wiki describes the Rust binary launcher (subid=protocol-v07)","source_content_hash":"sha256:4f467e2117e8c361882054998cb0506717d9555e2aa570f0e176fc18d9c2f4aa","source_mtime_ns":"1793410024331635951","status":"🟢 Green","verification":"unchecked","wiki_hash":"sha256:3876708eaec60862b9b8b6f73e2ae87e4abca2f228c456f973ddb7c9ee36854a"}},"version":2} \ No newline at end of file +{"entries":{"AGENTS.md":{"last_updated":"2026-11-03 08:56:14","reason":"Correct the scanner and wiki-lookup notes.","source_content_hash":"sha256:42429e681e0de3c1e95b8a98c2dd3eccf8ac60f79da7ed55224102934b8d00d2","source_mtime_ns":"1793684917066518484","status":"🟡 Yellow","wiki_hash":"sha256:d5907471b6831952e34e64a033d471bb0264ef4dee92151ca2c0eaf40b1d16ef"},"CHANGELOG.md":{"last_updated":"2026-11-03 08:56:14","reason":"Record the parity work under Unreleased.","source_content_hash":"sha256:be95d797bbc840cf053f5ce7623f9907d67798dc5ea2092c2ff60d1189fdd889","source_mtime_ns":"1793684916746519688","status":"🟡 Yellow","wiki_hash":"sha256:43a7a238cc155970024675795f4d1f1fc0000e22cd22898ca7206c57dc111e67"},"CLAUDE.md":{"last_updated":"2026-11-03 08:56:14","reason":"Correct the tsconfig note.","source_content_hash":"sha256:87053cbd72c291e0057cd438bdb57f3fba43fec214b326d9b944520db82483fa","source_mtime_ns":"1793684916630520123","status":"🟡 Yellow","wiki_hash":"sha256:afc87f10c92c7cf033d4a5af38ff318ccc1d8ff069b88cab22b3563552ea071b"},"CONTRIBUTING.md":{"last_updated":"2026-11-03 05:34:35","reason":"Address the 3 Nov review findings (subid=review-2026-11-03)","source_content_hash":"sha256:d090280cd3d1c3ab1cf342d69392471ed3835512ff7381bf4409bada57bbe89c","source_mtime_ns":"1793683924316411256","status":"🟢 Green","verification":"unchecked","wiki_hash":"sha256:ec0bd9342b8e9dfeefaabc8dcddeb577f6bd4f1c760d7286b21d709c2a372b29"},"Findings/M5-Dogfood-Assessment-Report.md":{"last_updated":"2026-07-09 05:02:27","reason":"kept; stale yellow cleared","source_content_hash":"sha256:238435188b1699e5bcf4753b62a6fe55cb064413a291259f0b696b5bd3c1692f","source_mtime_ns":"1780519956000000000","status":"🟢 Green"},"Findings/M5-Dogfood-Progress.md":{"last_updated":"2026-11-03 08:06:46","reason":"Wiki marks this file as the historical M5 log (subid=protocol-v07)","source_content_hash":"sha256:ef55167e819806060dfd7dfe7cdfd4d02024ca97706a7faa243710e2630b65fe","source_mtime_ns":"1793408707259550076","status":"🟢 Green","verification":"unchecked","wiki_hash":"sha256:b6b1fa4bbee0c7e8c4035cd925f6d07e1221ba483e3472878d5b88236c5cf7f9"},"Findings/M5.1-cross-hardening-analysis.md":{"last_updated":"2026-07-09 05:02:27","reason":"kept; stale yellow cleared","source_content_hash":"sha256:a2c2414e5b6c9b1a802bec8734a7924850d546cd7746a8087216f3685ca36cb4","source_mtime_ns":"1780519762000000000","status":"🟢 Green"},"Findings/Milestones-Overview.md":{"last_updated":"2026-07-09 05:02:27","reason":"kept; stale yellow cleared","source_content_hash":"sha256:f8b7030ec8980774cf3a2560ad7aee6537263eadf8374e650718f7c6793c6d50","source_mtime_ns":"1780800604000000000","status":"🟢 Green"},"Findings/p6_real_world_validation_report.md":{"last_updated":"2026-07-09 05:02:27","reason":"kept; stale yellow cleared","source_content_hash":"sha256:0468224675888f2747674324726c0ea9a5dddded39d9a1a50f22fc3357dafc0f","source_mtime_ns":"1780520008000000000","status":"🟢 Green"},"README.md":{"last_updated":"2026-11-03 08:56:14","reason":"Command notes and the measured benchmark table.","source_content_hash":"sha256:3edc4babf7fe44662dd94e8b784ef87cf845d1130f04720552daf67fc77673d8","source_mtime_ns":"1793684917046518560","status":"🟡 Yellow","wiki_hash":"sha256:725a9cdfc3d48d9a343a319b192be15d107543a64246438ef72e83e710cb0667"},"docs/spec.md":{"last_updated":"2026-11-03 08:00:11","reason":"Name the Rust binary and protocol v0.7 in the living principles (subid=protocol-v07)","source_content_hash":"sha256:31fe6b3b3d705628692eb0512cf44cbb21861ccfc5bb02afdd46d796ec066c6e","source_mtime_ns":"1793653008580824356","status":"🟢 Green","verification":"unchecked","wiki_hash":"sha256:a58b929d25d510f15f9fe176d7823e852f6d089e398accc2e22e1ed189f260ac"},"scripts/benchmark_parity.sh":{"last_updated":"2026-11-03 08:56:15","reason":"Benchmark script and measured results.","source_content_hash":"sha256:b8f72e3245e77b9df17b303293d8cbe4a93cc36023c41b89a794b77b01daef87","status":"🟡 Yellow"},"scripts/benchmark_results.md":{"last_updated":"2026-11-03 08:56:15","reason":"Benchmark script and measured results.","source_content_hash":"sha256:8bf9342fce85f126a37f812b03b6f6997f1fe8e7c5176d30301c1a4a15aeee29","status":"🟡 Yellow"},"skills/run.md":{"last_updated":"2026-11-03 08:56:14","reason":"Document the parity behaviour.","source_content_hash":"sha256:1bc5a678c5a2dff041ab80f88b314ca69e6d233a143ded039632469a081bbbde","source_mtime_ns":"1793684917474516945","status":"🟡 Yellow","wiki_hash":"sha256:5f6e7868ee9adee68bd634352413338e9a367eac640c623545bcad60f64b6389"},"src/api.rs":{"last_updated":"2026-11-03 08:56:14","public_symbols":"MapRun\nVERSION\nMCP_TOOLS\nupdate_maps\nupdate_maps_opts\nupdate_maps_run\nmapped_edges\nload_dependencies\ndependency_report\nload_dependents\nlist_paths\nsession_bootstrap\nsuggest_next_actions\ncheck_changes\nprepare_edit\nrecord_change\nmark_green\nhealth\nread_health\nread_journal\ncache_status\nvalidate\ncycles\nverify_wikis\nrecord_changes\nrecord_changes_opts\nrecord_deletion\nheal_stubs\nprune_health\nprune_health_opts\nprune_pending\nstale_wikis\nprune_barrels\nmetrics_snapshot\nhealing_stats\nresolution_diagnostics\nbarrel_reports\nincremental_status\nserve_once\nserve_loop\nmonitor_loop\ndaemon_ctl\ndispatch\nCLI_COMMANDS\nensure_root","reason":"Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions.","source_content_hash":"sha256:8b23c099bfab315e96c055c0bcaf7d7edceb9e40420adc2d21a8ba930abbf1ef","source_mtime_ns":"1793684003575439733","status":"🟡 Yellow","wiki_hash":"sha256:69ac1101639d23ac6893adf37e501a820e277ace0ddeaf3c2c80afba28557e54"},"src/artifacts.rs":{"last_updated":"2026-11-03 08:56:14","public_symbols":"HealthRow\nread_health\nwrite_health\nupsert_health_row\nJournalEntry\njournal_path\nread_journal\nparse_journal\nJournalWrite\nappend_journal\nread_pending\nfolder_card\nnew\nkeep_baseline_from","reason":"Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions.","source_content_hash":"sha256:6a7ca10120cbb4a37de62999330a92b181f3ebd7210f92209b280ba5282d8544","source_mtime_ns":"1793678007374624809","status":"🟡 Yellow","wiki_hash":"sha256:75bf0f72cc4bd57b6178dc6a0a58b8bf97bc3e7265485a28102093b967f20181"},"src/cache.rs":{"last_updated":"2026-11-03 08:56:14","public_symbols":"check_consistency\nrebuild\nConsistency\nSession\npayload_path\nby_src_path\ncommit_pending_path\nload_dependencies\nInternalEdge\nconfidence_of\nformat_edge_line\nDependencyView\nload_dependency_view\nload_dependents\nlist_prefix\nall_paths\nFileUpdate\nupsert_file\nremove_file\nindexed_rows\npayload_bytes\nstat_ns\nedges\nrel_string\nwrite_barrel_record\nprune_barrel_files\nconfidence\nacs_reason","reason":"Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions.","source_content_hash":"sha256:fb8a06997639e0f1556cc64ce5792f1525b0cb3c4afab5f9cbfcaeb9c809adf4","source_mtime_ns":"1793678007370625780","status":"🟡 Yellow","wiki_hash":"sha256:afb7493ff8742682f5f3c743da8919408502e6a6e75bba9998eb59cbbaf1c02a"},"src/cache_commit.rs":{"last_updated":"2026-11-03 08:56:14","public_symbols":"Consistency\nSession\ncheck_consistency\nrebuild\nopen\nknown\ndsts_of\nnote_stat\nrels\nupsert\nremove\nstage_barrel\ncommit","reason":"Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions.","source_content_hash":"sha256:c987ccdccdd0c980c0af64a2dcb349cd3b4ed1de7070a481dba146782f40a745","source_mtime_ns":"1793684429116993727","status":"🟡 Yellow","wiki_hash":"sha256:c6f3fbdcd6aafe62de119c302e09e11ae0ea208681c52166cfcf79de80bb5f6c"},"src/cycle_report.rs":{"last_updated":"2026-11-03 08:56:15","reason":"Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions.","source_content_hash":"sha256:db35b610d9a9f9917805da9383e61872e66e640482f57fc9e17ae91daf77fe3d","status":"🟡 Yellow"},"src/daemon_svc.rs":{"last_updated":"2026-11-03 08:56:15","reason":"Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions.","source_content_hash":"sha256:0887baeb3ad7f2fca851fccc0d9335dafd4b42f3e54e236175a13d39c5bcd063","status":"🟡 Yellow"},"src/jsonutil.rs":{"last_updated":"2026-11-03 08:56:14","public_symbols":"Json\nparse\nparse_jsonc\nstrip_json_comments\nobj\nfrom_pairs\nget\nas_str\nas_bool\nas_array\nas_object\ninsert\nfmt","reason":"Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions.","source_content_hash":"sha256:cb71e00e352df010928a42323d6d87bbe67348834a209348fc174b9d00489676","source_mtime_ns":"1793684429116993727","status":"🟡 Yellow","wiki_hash":"sha256:fb4cffba8b14a179dd90f201b61df2608ca78a9775e9646e7f011f1b9fa07c63"},"src/legacy_cache.rs":{"last_updated":"2026-11-03 08:56:14","public_symbols":"import_if_empty","reason":"Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions.","source_content_hash":"sha256:a127d0b1742d69b736194ca98579f5fc0611fc51a291d691aef6fe586fd68ba3","source_mtime_ns":"1793683981803718735","status":"🟡 Yellow","wiki_hash":"sha256:a6fe6c79185a46cc4b30d215475bab28f517f6e5a6fd7bce6423c2792814aac8"},"src/lib.rs":{"last_updated":"2026-11-03 08:56:14","public_symbols":"api\nartifacts\ncache\njsonutil\nmaps_out\nmcp\nparse\nscope\nsha256\ntimeutil\ncache_status\ncheck_changes\nhealth\nlist_paths\nload_dependencies\nload_dependents\nmapped_edges\nmark_green\nprepare_edit\nread_health\nread_journal\nrecord_change\nsession_bootstrap\nsuggest_next_actions\nupdate_maps\nVERSION\nJson","reason":"Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions.","source_content_hash":"sha256:9c27c83b58ff7f1acf62d5d19ee31e2070daa06499e27141cc7071fae44e26b8","source_mtime_ns":"1793678007370625780","status":"🟡 Yellow","wiki_hash":"sha256:d7e771a3ff63ab55a7967195e3b403dd75d5ae020a383adc56f0e5c09d7b03a0"},"src/low_conf.rs":{"last_updated":"2026-11-03 08:56:15","reason":"Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions.","source_content_hash":"sha256:3ade5fc70ce585a48b9acf6a2b48753e58e80f63281a31c8b8453816700b375e","status":"🟡 Yellow"},"src/main.rs":{"last_updated":"2026-11-03 08:56:14","reason":"Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions.","source_content_hash":"sha256:2b3cebdafca3ca3da7831101a63b36f72260b3b2879871e5364b427428e782e0","source_mtime_ns":"1793683990351610198","status":"🟡 Yellow","wiki_hash":"sha256:db6e1cd5beeb90e5a65b3e00a37ee2c73ea4c53750771151e655459a1b823ff5"},"src/maps_out.rs":{"last_updated":"2026-11-03 08:56:14","public_symbols":"write\nwrite_status\nread_status","reason":"Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions.","source_content_hash":"sha256:c018687f1e511e23bc92a3f771a1f2bf5e693d513d8e4bcad958d9908f0af09c","source_mtime_ns":"1793678007370625780","status":"🟡 Yellow","wiki_hash":"sha256:d441c395fe565828a87c54d5545e4c89e4ec4cfc49308a08525e6dcf3c744e75"},"src/mcp.rs":{"last_updated":"2026-11-03 08:56:14","public_symbols":"tool_schema\nhandle_message\nserve_stdio","reason":"Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions.","source_content_hash":"sha256:22c88894fc72f3fad68a31c1582a01772a8409483d3a3127730eccce63067119","source_mtime_ns":"1793683982335712018","status":"🟡 Yellow","wiki_hash":"sha256:8414b1757bf3acc39a66a52443e9625cb027173b2783d996e39e9f707a9a1933"},"src/parse.rs":{"last_updated":"2026-11-03 08:56:14","public_symbols":"RawImport\nKind\nScan\nlanguage\nread_utf8\nscan_file_warned\nscan_file\nresolve_project\nrust_public_symbols\nresolve_python_spec\nFactEdge\nFileFacts\nAnalysis\nanalyze_files\nanalyze_changed\nedges_for_files","reason":"Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions.","source_content_hash":"sha256:5607acdac0b08854951700a9d2187a9f849d2fdb4febc2959fe065bcf778963a","source_mtime_ns":"1793684824322776381","status":"🟡 Yellow","wiki_hash":"sha256:c70af52c81a4dfffb364cba13ca21873491e83e1c9c5cee31ab2bab2bc379fbb"},"src/pending_line.rs":{"last_updated":"2026-11-03 08:56:15","reason":"Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions.","source_content_hash":"sha256:c941813474dbb2cfc1ffed8c7bc9791d08eb718d92d336e1f42d1c1b0d67bcdc","status":"🟡 Yellow"},"src/proc.rs":{"last_updated":"2026-11-03 08:56:14","reason":"Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions.","source_content_hash":"sha256:8f13798a8b9289d78913becc626056c8b016838570e356e09bfa3333e800351c","status":"🟡 Yellow"},"src/project_lock.rs":{"last_updated":"2026-11-03 08:56:14","public_symbols":"ProjectLock\ndrop\nacquire","reason":"Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions.","source_content_hash":"sha256:194e329f35ff59b11fc734b980a1ee33f8e464bcdb5eae899486e487daef3a25","source_mtime_ns":"1793678007370625780","status":"🟡 Yellow","wiki_hash":"sha256:fccc8d547f30262ff64d506d842472623e828ebe39fde71532a683c6956329d2"},"src/scope.rs":{"last_updated":"2026-11-03 08:56:14","reason":"Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions.","source_content_hash":"sha256:6dec4152e1ae1657661bd9f504877c02dd70bdb97c39173a952bea2851bedb4c","status":"🟡 Yellow"},"src/sha256.rs":{"last_updated":"2026-11-03 08:56:14","reason":"Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions.","source_content_hash":"sha256:d16c7732fbdd20b3697b78129e8052f676c42bc862065f2bfe2b16fae1b7df30","status":"🟡 Yellow"},"src/symbols.rs":{"last_updated":"2026-11-03 08:56:15","reason":"Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions.","source_content_hash":"sha256:4a150ba04e9ea26020c874c5e256a9ec5acc3ef4d621fb6ff03b21935c3f8046","status":"🟡 Yellow"},"src/text_out.rs":{"last_updated":"2026-11-03 08:56:15","reason":"Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions.","source_content_hash":"sha256:8e0edc1141a2c1745c0e6fdd7a7645c7332365fac532d52fb3e188f4ef06148e","status":"🟡 Yellow"},"src/timeutil.rs":{"last_updated":"2026-11-03 08:56:14","reason":"Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions.","source_content_hash":"sha256:9921127144d9d40619e260753e700e9c8a47135ed95b2ad8791afb7fe78afd75","status":"🟡 Yellow"},"src/wiki_files.rs":{"last_updated":"2026-11-03 08:56:15","reason":"Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions.","source_content_hash":"sha256:2612539ce9092a3d126d80ce2947f6b4555ddb0bb335769b1619ec67d08a5e84","status":"🟡 Yellow"},"tests/behaviour_gaps.rs":{"last_updated":"2026-11-03 08:56:14","reason":"Tests for those parity fixes.","source_content_hash":"sha256:8c8d669dae2dbe39b88c2223d20fb2fca51ac1605c62edd108d6ef224223402b","source_mtime_ns":"1793684893298602299","status":"🟡 Yellow","wiki_hash":"sha256:b9d20a310ca48ffd858e6d0d4e49b5e0568ecb57dc5ad981eb54398562864691"},"tests/cache_and_artifacts.rs":{"last_updated":"2026-11-03 08:56:14","reason":"Tests for those parity fixes.","source_content_hash":"sha256:b9483666c1e19f90d4db1af718b08beebadfbcba66152497b7694d8e757a5ec1","status":"🟡 Yellow"},"tests/cli_surface.rs":{"last_updated":"2026-11-03 08:56:14","reason":"Tests for those parity fixes.","source_content_hash":"sha256:c6d2ec49ab47aac39a17e6d39fb7eae08d4b27935a9d68bc02a238000646c739","status":"🟡 Yellow"},"tests/contract_gaps.rs":{"last_updated":"2026-11-03 08:56:14","reason":"Tests for those parity fixes.","source_content_hash":"sha256:dc91a239bbc615524b4ac2c58d06515183dfc4981309906003f551e6721d82ae","status":"🟡 Yellow"},"tests/dogfood.rs":{"last_updated":"2026-11-03 08:56:15","reason":"Tests for those parity fixes.","source_content_hash":"sha256:b596d9aa2a19c0e030e8d0c02613d1ced0ddf026c976bfcb3c563758bce57fb6","source_mtime_ns":"1793683337392806763","status":"🟡 Yellow","wiki_hash":"sha256:9546cc2062f19584c348419322f07d25d353c7f8ba0257e992372f63d798c020"},"tests/parity.rs":{"last_updated":"2026-11-03 08:56:15","reason":"Tests for those parity fixes.","source_content_hash":"sha256:0246038a1cc01bf66da2fd186ede61f2dcd2c98da9c0c90dd7f3b2cbab1036db","source_mtime_ns":"1793683314773282057","status":"🟡 Yellow","wiki_hash":"sha256:32753c244642df6e08f8578c6203f77225363783c15bc57c33a84f03989f2378"},"tests/parity_fixes.rs":{"last_updated":"2026-11-03 08:56:15","reason":"Tests for those parity fixes.","source_content_hash":"sha256:ac0fa6064323885f6e7bd767557fe3721723c8d20d60e36c7cc5265b85747e31","status":"🟡 Yellow"},"tests/protocol_v07.rs":{"last_updated":"2026-11-03 08:56:15","reason":"Tests for those parity fixes.","source_content_hash":"sha256:bb7cffecabc3fb51ebcc77c64f7fd78f5cd15c445d7e13493ed491227280f37d","source_mtime_ns":"1793684781738827745","status":"🟡 Yellow","wiki_hash":"sha256:b38f4a13e316cefd56070f822980b4f83d9cdacb850ab1978de48ed35c630ec8"},"tests/robustness.rs":{"last_updated":"2026-11-03 08:56:15","reason":"Tests for those parity fixes.","source_content_hash":"sha256:fad304193a47baa8cce21cff8786b35974c7c7f6f71b3318edeb68bd60121681","status":"🟡 Yellow"},"tests/skip_maps.rs":{"last_updated":"2026-11-03 08:56:15","reason":"Tests for those parity fixes.","source_content_hash":"sha256:2eed0c6bc97d91a4be3f2b6ab08c78c62a3a05725a503496e3df91abad6b2130","status":"🟡 Yellow"},"wikifier.sh":{"last_updated":"2026-11-03 08:06:46","reason":"Wiki describes the Rust binary launcher (subid=protocol-v07)","source_content_hash":"sha256:4f467e2117e8c361882054998cb0506717d9555e2aa570f0e176fc18d9c2f4aa","source_mtime_ns":"1793410024331635951","status":"🟢 Green","verification":"unchecked","wiki_hash":"sha256:3876708eaec60862b9b8b6f73e2ae87e4abca2f228c456f973ddb7c9ee36854a"}},"version":2} \ No newline at end of file diff --git a/file_health.md b/file_health.md index 8a00024..29a02d3 100644 --- a/file_health.md +++ b/file_health.md @@ -2,41 +2,51 @@ | File | Status | Last Updated | Reason / Intent | |------|--------|--------------|-----------------| -| AGENTS.md | 🟢 Green | 2026-11-03 05:49:16 | Check impl items and resolve workspace member crates (subid=impl-workspace) | -| CHANGELOG.md | 🟢 Green | 2026-11-03 05:49:16 | Check impl items and resolve workspace member crates (subid=impl-workspace) | -| CLAUDE.md | 🟢 Green | 2026-11-03 05:49:16 | Check impl items and resolve workspace member crates (subid=impl-workspace) | +| AGENTS.md | 🟡 Yellow | 2026-11-03 08:56:14 | Correct the scanner and wiki-lookup notes. | +| CHANGELOG.md | 🟡 Yellow | 2026-11-03 08:56:14 | Record the parity work under Unreleased. | +| CLAUDE.md | 🟡 Yellow | 2026-11-03 08:56:14 | Correct the tsconfig note. | | CONTRIBUTING.md | 🟢 Green | 2026-11-03 05:34:35 | Address the 3 Nov review findings (subid=review-2026-11-03) | | Findings/M5-Dogfood-Assessment-Report.md | 🟢 Green | 2026-07-09 05:02:27 | kept; stale yellow cleared | | Findings/M5-Dogfood-Progress.md | 🟢 Green | 2026-11-03 08:06:46 | Wiki marks this file as the historical M5 log (subid=protocol-v07) | | Findings/M5.1-cross-hardening-analysis.md | 🟢 Green | 2026-07-09 05:02:27 | kept; stale yellow cleared | | Findings/Milestones-Overview.md | 🟢 Green | 2026-07-09 05:02:27 | kept; stale yellow cleared | | Findings/p6_real_world_validation_report.md | 🟢 Green | 2026-07-09 05:02:27 | kept; stale yellow cleared | -| README.md | 🟢 Green | 2026-11-03 05:49:16 | Check impl items and resolve workspace member crates (subid=impl-workspace) | +| README.md | 🟡 Yellow | 2026-11-03 08:56:14 | Command notes and the measured benchmark table. | | docs/spec.md | 🟢 Green | 2026-11-03 08:00:11 | Name the Rust binary and protocol v0.7 in the living principles (subid=protocol-v07) | -| skills/run.md | 🟢 Green | 2026-11-03 05:49:16 | Check impl items and resolve workspace member crates (subid=impl-workspace) | -| src/api.rs | 🟢 Green | 2026-11-03 05:34:35 | Address the 3 Nov review findings (subid=review-2026-11-03) | -| src/artifacts.rs | 🟢 Green | 2026-11-03 05:49:23 | Wiki names impl items (subid=impl-workspace) | -| src/cache.rs | 🟢 Green | 2026-11-03 05:49:23 | Wiki names impl items (subid=impl-workspace) | -| src/cache_commit.rs | 🟢 Green | 2026-11-03 05:49:23 | Wiki names impl items (subid=impl-workspace) | -| src/jsonutil.rs | 🟢 Green | 2026-11-03 05:49:23 | Wiki names impl items (subid=impl-workspace) | -| src/legacy_cache.rs | 🟢 Green | 2026-11-03 05:34:35 | Address the 3 Nov review findings (subid=review-2026-11-03) | -| src/lib.rs | 🟢 Green | 2026-11-03 05:35:24 | Wiki names the crate re-exports (subid=review-w3) | -| src/main.rs | 🟢 Green | 2026-11-03 05:34:35 | Address the 3 Nov review findings (subid=review-2026-11-03) | -| src/maps_out.rs | 🟢 Green | 2026-11-03 05:35:24 | Wiki names read_status (subid=review-w3) | -| src/mcp.rs | 🟢 Green | 2026-11-03 05:34:35 | Address the 3 Nov review findings (subid=review-2026-11-03) | -| src/parse.rs | 🟢 Green | 2026-11-03 05:49:16 | Check impl items and resolve workspace member crates (subid=impl-workspace) | -| src/proc.rs | 🟡 Yellow | 2026-11-03 07:58:59 | Initial stub | -| src/project_lock.rs | 🟢 Green | 2026-11-03 05:49:23 | Wiki names impl items (subid=impl-workspace) | -| src/scope.rs | 🟡 Yellow | 2026-11-03 07:58:59 | Initial stub | -| src/sha256.rs | 🟡 Yellow | 2026-11-03 07:58:59 | Initial stub | -| src/timeutil.rs | 🟡 Yellow | 2026-11-03 07:58:59 | Initial stub | -| tests/behaviour_gaps.rs | 🟢 Green | 2026-11-03 05:49:16 | Check impl items and resolve workspace member crates (subid=impl-workspace) | -| tests/cache_and_artifacts.rs | 🟡 Yellow | 2026-11-03 07:58:59 | Initial stub | -| tests/cli_surface.rs | 🟡 Yellow | 2026-11-03 07:58:59 | Initial stub | -| tests/contract_gaps.rs | 🟡 Yellow | 2026-11-03 07:58:59 | Initial stub | -| tests/dogfood.rs | 🟢 Green | 2026-11-03 05:34:35 | Address the 3 Nov review findings (subid=review-2026-11-03) | -| tests/parity.rs | 🟢 Green | 2026-11-03 05:34:35 | Address the 3 Nov review findings (subid=review-2026-11-03) | -| tests/protocol_v07.rs | 🟢 Green | 2026-11-03 05:49:16 | Check impl items and resolve workspace member crates (subid=impl-workspace) | -| tests/robustness.rs | 🟡 Yellow | 2026-11-03 07:58:59 | Initial stub | -| tests/skip_maps.rs | 🟡 Yellow | 2026-11-03 07:58:59 | Initial stub | +| scripts/benchmark_parity.sh | 🟡 Yellow | 2026-11-03 08:56:15 | Benchmark script and measured results. | +| scripts/benchmark_results.md | 🟡 Yellow | 2026-11-03 08:56:15 | Benchmark script and measured results. | +| skills/run.md | 🟡 Yellow | 2026-11-03 08:56:14 | Document the parity behaviour. | +| src/api.rs | 🟡 Yellow | 2026-11-03 08:56:14 | Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. | +| src/artifacts.rs | 🟡 Yellow | 2026-11-03 08:56:14 | Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. | +| src/cache.rs | 🟡 Yellow | 2026-11-03 08:56:14 | Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. | +| src/cache_commit.rs | 🟡 Yellow | 2026-11-03 08:56:14 | Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. | +| src/cycle_report.rs | 🟡 Yellow | 2026-11-03 08:56:15 | Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. | +| src/daemon_svc.rs | 🟡 Yellow | 2026-11-03 08:56:15 | Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. | +| src/jsonutil.rs | 🟡 Yellow | 2026-11-03 08:56:14 | Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. | +| src/legacy_cache.rs | 🟡 Yellow | 2026-11-03 08:56:14 | Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. | +| src/lib.rs | 🟡 Yellow | 2026-11-03 08:56:14 | Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. | +| src/low_conf.rs | 🟡 Yellow | 2026-11-03 08:56:15 | Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. | +| src/main.rs | 🟡 Yellow | 2026-11-03 08:56:14 | Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. | +| src/maps_out.rs | 🟡 Yellow | 2026-11-03 08:56:14 | Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. | +| src/mcp.rs | 🟡 Yellow | 2026-11-03 08:56:14 | Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. | +| src/parse.rs | 🟡 Yellow | 2026-11-03 08:56:14 | Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. | +| src/pending_line.rs | 🟡 Yellow | 2026-11-03 08:56:15 | Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. | +| src/proc.rs | 🟡 Yellow | 2026-11-03 08:56:14 | Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. | +| src/project_lock.rs | 🟡 Yellow | 2026-11-03 08:56:14 | Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. | +| src/scope.rs | 🟡 Yellow | 2026-11-03 08:56:14 | Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. | +| src/sha256.rs | 🟡 Yellow | 2026-11-03 08:56:14 | Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. | +| src/symbols.rs | 🟡 Yellow | 2026-11-03 08:56:15 | Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. | +| src/text_out.rs | 🟡 Yellow | 2026-11-03 08:56:15 | Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. | +| src/timeutil.rs | 🟡 Yellow | 2026-11-03 08:56:14 | Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. | +| src/wiki_files.rs | 🟡 Yellow | 2026-11-03 08:56:15 | Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. | +| tests/behaviour_gaps.rs | 🟡 Yellow | 2026-11-03 08:56:14 | Tests for those parity fixes. | +| tests/cache_and_artifacts.rs | 🟡 Yellow | 2026-11-03 08:56:14 | Tests for those parity fixes. | +| tests/cli_surface.rs | 🟡 Yellow | 2026-11-03 08:56:14 | Tests for those parity fixes. | +| tests/contract_gaps.rs | 🟡 Yellow | 2026-11-03 08:56:14 | Tests for those parity fixes. | +| tests/dogfood.rs | 🟡 Yellow | 2026-11-03 08:56:15 | Tests for those parity fixes. | +| tests/parity.rs | 🟡 Yellow | 2026-11-03 08:56:15 | Tests for those parity fixes. | +| tests/parity_fixes.rs | 🟡 Yellow | 2026-11-03 08:56:15 | Tests for those parity fixes. | +| tests/protocol_v07.rs | 🟡 Yellow | 2026-11-03 08:56:15 | Tests for those parity fixes. | +| tests/robustness.rs | 🟡 Yellow | 2026-11-03 08:56:15 | Tests for those parity fixes. | +| tests/skip_maps.rs | 🟡 Yellow | 2026-11-03 08:56:15 | Tests for those parity fixes. | | wikifier.sh | 🟢 Green | 2026-11-03 08:06:46 | Wiki describes the Rust binary launcher (subid=protocol-v07) | diff --git a/journal/2026/11/03.md b/journal/2026/11/03.md index 565d1bd..7c9a360 100644 --- a/journal/2026/11/03.md +++ b/journal/2026/11/03.md @@ -286,3 +286,163 @@ **File:** tests/protocol_v07.rs **Reason:** Check impl items and resolve workspace member crates (subid=impl-workspace) +## [2026-11-03 08:56:14] record-change +**File:** AGENTS.md +**Reason:** Correct the scanner and wiki-lookup notes. + +## [2026-11-03 08:56:14] record-change +**File:** CHANGELOG.md +**Reason:** Record the parity work under Unreleased. + +## [2026-11-03 08:56:14] record-change +**File:** CLAUDE.md +**Reason:** Correct the tsconfig note. + +## [2026-11-03 08:56:14] record-change +**File:** README.md +**Reason:** Command notes and the measured benchmark table. + +## [2026-11-03 08:56:14] record-change +**File:** skills/run.md +**Reason:** Document the parity behaviour. + +## [2026-11-03 08:56:14] record-change +**File:** src/api.rs +**Reason:** Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. + +## [2026-11-03 08:56:14] record-change +**File:** src/artifacts.rs +**Reason:** Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. + +## [2026-11-03 08:56:14] record-change +**File:** src/cache.rs +**Reason:** Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. + +## [2026-11-03 08:56:14] record-change +**File:** src/cache_commit.rs +**Reason:** Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. + +## [2026-11-03 08:56:14] record-change +**File:** src/jsonutil.rs +**Reason:** Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. + +## [2026-11-03 08:56:14] record-change +**File:** src/legacy_cache.rs +**Reason:** Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. + +## [2026-11-03 08:56:14] record-change +**File:** src/lib.rs +**Reason:** Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. + +## [2026-11-03 08:56:14] record-change +**File:** src/main.rs +**Reason:** Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. + +## [2026-11-03 08:56:14] record-change +**File:** src/maps_out.rs +**Reason:** Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. + +## [2026-11-03 08:56:14] record-change +**File:** src/mcp.rs +**Reason:** Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. + +## [2026-11-03 08:56:14] record-change +**File:** src/parse.rs +**Reason:** Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. + +## [2026-11-03 08:56:14] record-change +**File:** src/proc.rs +**Reason:** Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. + +## [2026-11-03 08:56:14] record-change +**File:** src/project_lock.rs +**Reason:** Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. + +## [2026-11-03 08:56:14] record-change +**File:** src/scope.rs +**Reason:** Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. + +## [2026-11-03 08:56:14] record-change +**File:** src/sha256.rs +**Reason:** Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. + +## [2026-11-03 08:56:14] record-change +**File:** src/timeutil.rs +**Reason:** Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. + +## [2026-11-03 08:56:14] record-change +**File:** tests/behaviour_gaps.rs +**Reason:** Tests for those parity fixes. + +## [2026-11-03 08:56:14] record-change +**File:** tests/cache_and_artifacts.rs +**Reason:** Tests for those parity fixes. + +## [2026-11-03 08:56:14] record-change +**File:** tests/cli_surface.rs +**Reason:** Tests for those parity fixes. + +## [2026-11-03 08:56:14] record-change +**File:** tests/contract_gaps.rs +**Reason:** Tests for those parity fixes. + +## [2026-11-03 08:56:15] record-change +**File:** tests/dogfood.rs +**Reason:** Tests for those parity fixes. + +## [2026-11-03 08:56:15] record-change +**File:** tests/parity.rs +**Reason:** Tests for those parity fixes. + +## [2026-11-03 08:56:15] record-change +**File:** tests/protocol_v07.rs +**Reason:** Tests for those parity fixes. + +## [2026-11-03 08:56:15] record-change +**File:** tests/robustness.rs +**Reason:** Tests for those parity fixes. + +## [2026-11-03 08:56:15] record-change +**File:** tests/skip_maps.rs +**Reason:** Tests for those parity fixes. + +## [2026-11-03 08:56:15] record-change +**File:** scripts/benchmark_parity.sh +**Reason:** Benchmark script and measured results. + +## [2026-11-03 08:56:15] record-change +**File:** scripts/benchmark_results.md +**Reason:** Benchmark script and measured results. + +## [2026-11-03 08:56:15] record-change +**File:** src/cycle_report.rs +**Reason:** Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. + +## [2026-11-03 08:56:15] record-change +**File:** src/daemon_svc.rs +**Reason:** Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. + +## [2026-11-03 08:56:15] record-change +**File:** src/low_conf.rs +**Reason:** Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. + +## [2026-11-03 08:56:15] record-change +**File:** src/pending_line.rs +**Reason:** Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. + +## [2026-11-03 08:56:15] record-change +**File:** src/symbols.rs +**Reason:** Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. + +## [2026-11-03 08:56:15] record-change +**File:** src/text_out.rs +**Reason:** Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. + +## [2026-11-03 08:56:15] record-change +**File:** src/wiki_files.rs +**Reason:** Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. + +## [2026-11-03 08:56:15] record-change +**File:** tests/parity_fixes.rs +**Reason:** Tests for those parity fixes. + diff --git a/pending_updates.md b/pending_updates.md index 314a830..04bc97d 100644 --- a/pending_updates.md +++ b/pending_updates.md @@ -1,3 +1,43 @@ # Pending Updates (no pending items — run check-changes after making edits) +- AGENTS.md: LLM/agent edit — Correct the scanner and wiki-lookup notes. +- CHANGELOG.md: LLM/agent edit — Record the parity work under Unreleased. +- CLAUDE.md: LLM/agent edit — Correct the tsconfig note. +- README.md: LLM/agent edit — Command notes and the measured benchmark table. +- skills/run.md: LLM/agent edit — Document the parity behaviour. +- src/api.rs: LLM/agent edit — Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. +- src/artifacts.rs: LLM/agent edit — Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. +- src/cache.rs: LLM/agent edit — Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. +- src/cache_commit.rs: LLM/agent edit — Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. +- src/jsonutil.rs: LLM/agent edit — Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. +- src/legacy_cache.rs: LLM/agent edit — Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. +- src/lib.rs: LLM/agent edit — Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. +- src/main.rs: LLM/agent edit — Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. +- src/maps_out.rs: LLM/agent edit — Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. +- src/mcp.rs: LLM/agent edit — Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. +- src/parse.rs: LLM/agent edit — Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. +- src/proc.rs: LLM/agent edit — Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. +- src/project_lock.rs: LLM/agent edit — Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. +- src/scope.rs: LLM/agent edit — Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. +- src/sha256.rs: LLM/agent edit — Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. +- src/timeutil.rs: LLM/agent edit — Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. +- tests/behaviour_gaps.rs: LLM/agent edit — Tests for those parity fixes. +- tests/cache_and_artifacts.rs: LLM/agent edit — Tests for those parity fixes. +- tests/cli_surface.rs: LLM/agent edit — Tests for those parity fixes. +- tests/contract_gaps.rs: LLM/agent edit — Tests for those parity fixes. +- tests/dogfood.rs: LLM/agent edit — Tests for those parity fixes. +- tests/parity.rs: LLM/agent edit — Tests for those parity fixes. +- tests/protocol_v07.rs: LLM/agent edit — Tests for those parity fixes. +- tests/robustness.rs: LLM/agent edit — Tests for those parity fixes. +- tests/skip_maps.rs: LLM/agent edit — Tests for those parity fixes. +- scripts/benchmark_parity.sh: LLM/agent edit — Benchmark script and measured results. +- scripts/benchmark_results.md: LLM/agent edit — Benchmark script and measured results. +- src/cycle_report.rs: LLM/agent edit — Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. +- src/daemon_svc.rs: LLM/agent edit — Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. +- src/low_conf.rs: LLM/agent edit — Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. +- src/pending_line.rs: LLM/agent edit — Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. +- src/symbols.rs: LLM/agent edit — Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. +- src/text_out.rs: LLM/agent edit — Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. +- src/wiki_files.rs: LLM/agent edit — Parity with the Python package: wiki lookup, symbols, daemon, init, flags, MCP, cycles, and low-confidence actions. +- tests/parity_fixes.rs: LLM/agent edit — Tests for those parity fixes. diff --git a/scripts/benchmark_parity.sh b/scripts/benchmark_parity.sh new file mode 100755 index 0000000..8366712 --- /dev/null +++ b/scripts/benchmark_parity.sh @@ -0,0 +1,270 @@ +#!/usr/bin/env bash +# Compare Python Wikifier at commit 24812ef with the Rust release binary. +# Stdlib and POSIX tools only. No extra crates. +# Usage: scripts/benchmark_parity.sh [results.md] +set -euo pipefail + +root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +out="${1:-$root/scripts/benchmark_results.md}" +py_src="${WIKIFIER_PY_SRC:-/tmp/wikifier-py}" +rs_bin="${WIKIFIER_RS_BIN:-$root/target/release/wikifier}" +runs="${WIKIFIER_BENCH_RUNS:-5}" +django_src="${WIKIFIER_DJANGO:-/tmp/bench-django}" + +if [[ ! -x "$rs_bin" ]]; then + echo "missing rust binary: $rs_bin (cargo build --release)" >&2 + exit 1 +fi +if [[ ! -f "$py_src/wikifier/__init__.py" ]]; then + git -C "$root" archive 24812eff13cf61cbf23ac5361a778b0013df6a5d | tar -x -C "$(dirname "$py_src")" +fi +if [[ ! -d "$django_src/django" ]]; then + git clone --depth 1 https://github.com/django/django.git "$django_src" +fi + +export PYTHONPATH="$py_src" +export PYTHONDONTWRITEBYTECODE=1 + +now_ns() { date +%s%N; } + +median_of() { + printf '%s\n' "$@" | sort -n | awk ' + { a[NR] = $1 } + END { + if (NR == 0) { print 0; exit } + if (NR % 2 == 1) printf "%.0f\n", a[(NR + 1) / 2] + else printf "%.0f\n", (a[NR / 2] + a[NR / 2 + 1]) / 2 + }' +} + +min_of() { printf '%s\n' "$@" | sort -n | head -1; } +max_of() { printf '%s\n' "$@" | sort -n | tail -1; } + +time_cmd() { + local start end + start="$(now_ns)" + "$@" >/tmp/bench-cmd.out 2>/tmp/bench-cmd.err || true + end="$(now_ns)" + echo $(( (end - start) / 1000000 )) +} + +rss_of() { + /usr/bin/time -v "$@" >/tmp/bench-rss.out 2>/tmp/bench-rss.err || true + awk -F: '/Maximum resident set size/ { gsub(/^[ \t]+/, "", $2); print $2; exit }' /tmp/bench-rss.err +} + +prepare_small() { + local dest="$1" + rm -rf "$dest" + mkdir -p "$dest/src/pkg" + printf 'src\n' >"$dest/map_paths.txt" + printf 'src\n' >"$dest/monitored_paths.txt" + printf 'from pkg.core import run\n' >"$dest/src/pkg/__init__.py" + printf 'from pkg.utils import helper\n\ndef run():\n return helper()\n' >"$dest/src/pkg/core.py" + printf 'def helper():\n return 1\n' >"$dest/src/pkg/utils.py" +} + +prepare_medium() { + local dest="$1" + rm -rf "$dest" + mkdir -p "$dest" + cp -a "$root/src" "$root/tests" "$root/Cargo.toml" "$root/map_paths.txt" "$root/monitored_paths.txt" "$dest/" +} + +prepare_large() { + local dest="$1" + rm -rf "$dest" + mkdir -p "$dest" + cp -a "$django_src/django" "$django_src/tests" "$dest/" + printf 'django\ntests\n' >"$dest/map_paths.txt" + printf 'django\ntests\n' >"$dest/monitored_paths.txt" +} + +wipe_cache() { + local dest="$1" + rm -rf "$dest/.wikifier_staging" "$dest/library.md" "$dest/file_health.json" "$dest/file_health.md" "$dest/pending_updates.md" +} + +py_cmd() { python3 -m wikifier --target "$1" "${@:2}"; } +rs_cmd() { "$rs_bin" --target "$1" "${@:2}"; } + +edge_count_py() { + python3 - "$1" <<'PY' +import sqlite3, sys +from pathlib import Path +db = Path(sys.argv[1]) / ".wikifier_staging" / "import_cache.sqlite" +con = sqlite3.connect(db) +rows = {(a, b) for a, b in con.execute("select src, dst from edges") if a and b} +print(len(rows)) +PY +} + +edge_count_rs() { + python3 - "$1" <<'PY' +import sys +from pathlib import Path +root = Path(sys.argv[1]) / ".wikifier_staging" / "wk_cache" / "by_src" +edges = set() +if root.is_dir(): + for path in root.glob("*.edges"): + lines = path.read_text(errors="replace").splitlines() + if not lines or not lines[0].startswith("@\t"): + continue + src = lines[0][2:].strip() + for line in lines[1:]: + dst = line.split("\t", 1)[0].strip() + if dst and dst != src: + edges.add((src, dst)) +print(len(edges)) +PY +} + +declare -A CELL +measure_size() { + local label="$1" prep="$2" + local py_dir="/tmp/bench-work/${label}-py" + local rs_dir="/tmp/bench-work/${label}-rs" + "$prep" "$py_dir" + "$prep" "$rs_dir" + local -a py_cold rs_cold py_warm rs_warm py_boot rs_boot py_check rs_check py_health rs_health py_verify rs_verify py_cycles rs_cycles + local i + for ((i = 1; i <= runs; i++)); do + wipe_cache "$py_dir" + wipe_cache "$rs_dir" + py_cold+=("$(time_cmd py_cmd "$py_dir" update-maps)") + rs_cold+=("$(time_cmd rs_cmd "$rs_dir" update-maps)") + py_warm+=("$(time_cmd py_cmd "$py_dir" update-maps)") + rs_warm+=("$(time_cmd rs_cmd "$rs_dir" update-maps)") + py_boot+=("$(time_cmd py_cmd "$py_dir" session-bootstrap)") + rs_boot+=("$(time_cmd rs_cmd "$rs_dir" session-bootstrap)") + py_check+=("$(time_cmd py_cmd "$py_dir" check-changes)") + rs_check+=("$(time_cmd rs_cmd "$rs_dir" check-changes)") + py_health+=("$(time_cmd py_cmd "$py_dir" health --summary)") + rs_health+=("$(time_cmd rs_cmd "$rs_dir" health --summary)") + py_verify+=("$(time_cmd py_cmd "$py_dir" verify-wikis)") + rs_verify+=("$(time_cmd rs_cmd "$rs_dir" verify-wikis)") + py_cycles+=("$(time_cmd py_cmd "$py_dir" cycles --json)") + rs_cycles+=("$(time_cmd rs_cmd "$rs_dir" cycles)") + echo "$label run $i done" >&2 + done + CELL["${label}|py|cold"]="$(median_of "${py_cold[@]}") $(min_of "${py_cold[@]}") $(max_of "${py_cold[@]}")" + CELL["${label}|rs|cold"]="$(median_of "${rs_cold[@]}") $(min_of "${rs_cold[@]}") $(max_of "${rs_cold[@]}")" + CELL["${label}|py|warm"]="$(median_of "${py_warm[@]}") $(min_of "${py_warm[@]}") $(max_of "${py_warm[@]}")" + CELL["${label}|rs|warm"]="$(median_of "${rs_warm[@]}") $(min_of "${rs_warm[@]}") $(max_of "${rs_warm[@]}")" + CELL["${label}|py|boot"]="$(median_of "${py_boot[@]}") $(min_of "${py_boot[@]}") $(max_of "${py_boot[@]}")" + CELL["${label}|rs|boot"]="$(median_of "${rs_boot[@]}") $(min_of "${rs_boot[@]}") $(max_of "${rs_boot[@]}")" + CELL["${label}|py|check"]="$(median_of "${py_check[@]}") $(min_of "${py_check[@]}") $(max_of "${py_check[@]}")" + CELL["${label}|rs|check"]="$(median_of "${rs_check[@]}") $(min_of "${rs_check[@]}") $(max_of "${rs_check[@]}")" + CELL["${label}|py|health"]="$(median_of "${py_health[@]}") $(min_of "${py_health[@]}") $(max_of "${py_health[@]}")" + CELL["${label}|rs|health"]="$(median_of "${rs_health[@]}") $(min_of "${rs_health[@]}") $(max_of "${rs_health[@]}")" + CELL["${label}|py|verify"]="$(median_of "${py_verify[@]}") $(min_of "${py_verify[@]}") $(max_of "${py_verify[@]}")" + CELL["${label}|rs|verify"]="$(median_of "${rs_verify[@]}") $(min_of "${rs_verify[@]}") $(max_of "${rs_verify[@]}")" + CELL["${label}|py|cycles"]="$(median_of "${py_cycles[@]}") $(min_of "${py_cycles[@]}") $(max_of "${py_cycles[@]}")" + CELL["${label}|rs|cycles"]="$(median_of "${rs_cycles[@]}") $(min_of "${rs_cycles[@]}") $(max_of "${rs_cycles[@]}")" + wipe_cache "$py_dir" + wipe_cache "$rs_dir" + CELL["${label}|py|rss_cold"]="$(rss_of python3 -m wikifier --target "$py_dir" update-maps)" + CELL["${label}|rs|rss_cold"]="$(rss_of "$rs_bin" --target "$rs_dir" update-maps)" + CELL["${label}|py|rss_warm"]="$(rss_of python3 -m wikifier --target "$py_dir" update-maps)" + CELL["${label}|rs|rss_warm"]="$(rss_of "$rs_bin" --target "$rs_dir" update-maps)" + CELL["${label}|py|edges"]="$(edge_count_py "$py_dir")" + CELL["${label}|rs|edges"]="$(edge_count_rs "$rs_dir")" +} + +declare -a py_start rs_start +for ((i = 1; i <= runs; i++)); do + py_start+=("$(time_cmd python3 -m wikifier --version)") + rs_start+=("$(time_cmd "$rs_bin" --version)") +done + +measure_size small prepare_small +measure_size medium prepare_medium +measure_size large prepare_large + +cpu="$(lscpu | awk -F: '/Model name/ { gsub(/^[ \t]+/, "", $2); print $2; exit }')" +ncpu="$(nproc)" +mem="$(awk '/MemTotal/ { printf "%.1f GiB", $2/1024/1024 }' /proc/meminfo)" +py_ver="$(python3 --version 2>&1)" +rs_ver="$("$rs_bin" --version)" +rustc_ver="$(rustc --version)" +bin_bytes="$(stat -c %s "$rs_bin")" +py_bytes="$(du -sb "$py_src/wikifier" | awk '{print $1}')" + +fmt_cell() { + local key="$1" + local value="${CELL[$key]}" + local med min max + read -r med min max <<<"$value" + printf '%s (%s–%s)' "$med" "$min" "$max" +} + +{ + echo "# Wikifier Python vs Rust benchmark" + echo + echo "Measured by \`scripts/benchmark_parity.sh\` on $(date -u +%Y-%m-%dT%H:%M:%SZ)." + echo + echo "## Machine" + echo + echo "- CPU: ${cpu} (${ncpu} cores)" + echo "- RAM: ${mem}" + echo "- Kernel: $(uname -sr)" + echo "- Python: ${py_ver} (commit 24812eff13cf61cbf23ac5361a778b0013df6a5d)" + echo "- Rust binary: ${rs_ver} (${rustc_ver})" + echo "- Runs per cell: ${runs}. Times are milliseconds, median (min–max)." + echo "- Peak RSS is one extra cold and warm \`update-maps\`, from \`/usr/bin/time -v\`, in kilobytes." + echo "- Rust binary size: ${bin_bytes} bytes. Python package tree \`wikifier/\`: ${py_bytes} bytes." + echo + echo "## Trees" + echo + echo "| Size | Tree |" + echo "|---|---|" + echo "| small | three Python modules under \`src/pkg\` |" + echo "| medium | this repo's \`src/\` and \`tests/\` |" + echo "| large | Django shallow clone, map roots \`django/\` and \`tests/\` (${django_src}) |" + echo + echo "## Milliseconds" + echo + echo "| Command | small Python | small Rust | medium Python | medium Rust | large Python | large Rust |" + echo "|---|---:|---:|---:|---:|---:|---:|" + for pair in "startup|start" "cold update-maps|cold" "warm update-maps|warm" "session-bootstrap|boot" "check-changes|check" "health --summary|health" "verify-wikis|verify" "cycles|cycles"; do + name="${pair%%|*}" + key="${pair##*|}" + if [[ "$key" == "start" ]]; then + printf '| %s | %s | %s | %s | %s | %s | %s |\n' "$name" \ + "$(printf '%s (%s–%s)' "$(median_of "${py_start[@]}")" "$(min_of "${py_start[@]}")" "$(max_of "${py_start[@]}")")" \ + "$(printf '%s (%s–%s)' "$(median_of "${rs_start[@]}")" "$(min_of "${rs_start[@]}")" "$(max_of "${rs_start[@]}")")" \ + "$(printf '%s (%s–%s)' "$(median_of "${py_start[@]}")" "$(min_of "${py_start[@]}")" "$(max_of "${py_start[@]}")")" \ + "$(printf '%s (%s–%s)' "$(median_of "${rs_start[@]}")" "$(min_of "${rs_start[@]}")" "$(max_of "${rs_start[@]}")")" \ + "$(printf '%s (%s–%s)' "$(median_of "${py_start[@]}")" "$(min_of "${py_start[@]}")" "$(max_of "${py_start[@]}")")" \ + "$(printf '%s (%s–%s)' "$(median_of "${rs_start[@]}")" "$(min_of "${rs_start[@]}")" "$(max_of "${rs_start[@]}")")" + continue + fi + printf '| %s | %s | %s | %s | %s | %s | %s |\n' "$name" \ + "$(fmt_cell "small|py|${key}")" "$(fmt_cell "small|rs|${key}")" \ + "$(fmt_cell "medium|py|${key}")" "$(fmt_cell "medium|rs|${key}")" \ + "$(fmt_cell "large|py|${key}")" "$(fmt_cell "large|rs|${key}")" + done + echo + echo "## Peak RSS (kbytes) and resolved file edges" + echo + echo "| | small Python | small Rust | medium Python | medium Rust | large Python | large Rust |" + echo "|---|---:|---:|---:|---:|---:|---:|" + printf '| cold update-maps RSS | %s | %s | %s | %s | %s | %s |\n' \ + "${CELL[small|py|rss_cold]}" "${CELL[small|rs|rss_cold]}" \ + "${CELL[medium|py|rss_cold]}" "${CELL[medium|rs|rss_cold]}" \ + "${CELL[large|py|rss_cold]}" "${CELL[large|rs|rss_cold]}" + printf '| warm update-maps RSS | %s | %s | %s | %s | %s | %s |\n' \ + "${CELL[small|py|rss_warm]}" "${CELL[small|rs|rss_warm]}" \ + "${CELL[medium|py|rss_warm]}" "${CELL[medium|rs|rss_warm]}" \ + "${CELL[large|py|rss_warm]}" "${CELL[large|rs|rss_warm]}" + printf '| resolved file edges | %s | %s | %s | %s | %s | %s |\n' \ + "${CELL[small|py|edges]}" "${CELL[small|rs|edges]}" \ + "${CELL[medium|py|edges]}" "${CELL[medium|rs|edges]}" \ + "${CELL[large|py|edges]}" "${CELL[large|rs|edges]}" + echo + echo "Startup is \`wikifier --version\` and does not depend on the tree, so that row repeats the same pair." + echo "A cold run deletes \`.wikifier_staging\` and the generated health and library files first." + echo "The warm run is the next \`update-maps\` on that same tree." +} >"$out" + +echo "wrote $out" diff --git a/scripts/benchmark_results.md b/scripts/benchmark_results.md new file mode 100644 index 0000000..cf5a887 --- /dev/null +++ b/scripts/benchmark_results.md @@ -0,0 +1,47 @@ +# Wikifier Python vs Rust benchmark + +Measured by `scripts/benchmark_parity.sh` on 2026-11-03T08:54:31Z. + +## Machine + +- CPU: Intel(R) Xeon(R) Processor (4 cores) +- RAM: 15.6 GiB +- Kernel: Linux 6.12.94+ +- Python: Python 3.12.3 (commit 24812eff13cf61cbf23ac5361a778b0013df6a5d) +- Rust binary: wikifier 4.7.0 (rustc 1.83.0 (90b35a623 2024-11-26)) +- Runs per cell: 5. Times are milliseconds, median (min–max). +- Peak RSS is one extra cold and warm `update-maps`, from `/usr/bin/time -v`, in kilobytes. +- Rust binary size: 2197456 bytes. Python package tree `wikifier/`: 1560419 bytes. + +## Trees + +| Size | Tree | +|---|---| +| small | three Python modules under `src/pkg` | +| medium | this repo's `src/` and `tests/` | +| large | Django shallow clone, map roots `django/` and `tests/` (/tmp/bench-django) | + +## Milliseconds + +| Command | small Python | small Rust | medium Python | medium Rust | large Python | large Rust | +|---|---:|---:|---:|---:|---:|---:| +| startup | 63 (61–66) | 2 (1–2) | 63 (61–66) | 2 (1–2) | 63 (61–66) | 2 (1–2) | +| cold update-maps | 123 (121–126) | 8 (7–13) | 167 (163–168) | 62 (59–73) | 12675 (12540–13094) | 3142 (3114–3305) | +| warm update-maps | 69 (67–72) | 4 (3–5) | 73 (72–74) | 4 (4–5) | 422 (417–433) | 246 (243–253) | +| session-bootstrap | 66 (64–67) | 2 (1–2) | 65 (64–67) | 2 (1–2) | 270 (269–278) | 21 (21–21) | +| check-changes | 64 (62–65) | 3 (2–3) | 70 (69–75) | 5 (5–6) | 1232 (1217–1266) | 74 (73–76) | +| health --summary | 60 (60–62) | 2 (1–2) | 60 (59–63) | 1 (1–2) | 109 (108–114) | 8 (8–9) | +| verify-wikis | 60 (60–61) | 2 (1–2) | 62 (60–67) | 1 (1–2) | 106 (103–108) | 8 (7–8) | +| cycles | 60 (60–61) | 2 (1–2) | 61 (60–63) | 2 (2–2) | 63 (61–64) | 54 (53–55) | + +## Peak RSS (kbytes) and resolved file edges + +| | small Python | small Rust | medium Python | medium Rust | large Python | large Rust | +|---|---:|---:|---:|---:|---:|---:| +| cold update-maps RSS | 27544 | 3384 | 28392 | 4424 | 97564 | 40012 | +| warm update-maps RSS | 24584 | 3424 | 24248 | 3476 | 30520 | 13120 | +| resolved file edges | 2 | 2 | 44 | 98 | 9074 | 9074 | + +Startup is `wikifier --version` and does not depend on the tree, so that row repeats the same pair. +A cold run deletes `.wikifier_staging` and the generated health and library files first. +The warm run is the next `update-maps` on that same tree. diff --git a/skills/run.md b/skills/run.md index 22ee8b0..05e7e7a 100644 --- a/skills/run.md +++ b/skills/run.md @@ -156,13 +156,17 @@ The program is a std-only Rust binary under `src/`. `Cargo.toml` has an empty `[ - A resolved edge line is `dst`, `line`, `names`, then optional flag letters `c` (conditional), `d` (dynamic), and `b` (via barrel). Confidence is assigned when the edge is read: dynamic 0.4, conditional 0.55, via barrel 0.7, static 0.9. ACS reason codes are `dynamic_literal`, `conditional_guard`, `via_barrel`, and `resolved_static`. Unresolved notes use `unresolved_relative` (the specifier starts with `.`), `dynamic_literal`, or `unresolved`. - `dependencies --full` returns `imports[]` with `confidence_score`, `confidence_reasons`, `acs`, `is_dynamic`, `is_conditional`, and `in_cycle`. Compact rows always include `conditional`, `dynamic`, `via_barrel`, and `confidence`. - `record_deletion` appends the journal line, marks the health row Red, drops the file from the graph, and removes its pending line. The file does not have to exist. -- `record_changes` matches reasons by exact path, then the longest `dir/` prefix, then the longest glob (`*` and `?`). `--only`, `--green`, `--dry-run`, and deletions are applied. `--green` lists verification failures and leaves those files Yellow. -- `mark_green` extracts Python top-level `def`/`class` names (or `__all__`), JS/TS `export` names, and Rust unrestricted `pub` items (`fn`, `struct`, `enum`, `trait`, `mod`, `type`, `const`, `static`, and `pub use` names), plus `pub` methods and associated items on a public type and the items written in a trait impl for that type. Each current name must appear in the wiki. `pub(crate)`, `pub(super)`, `pub(self)`, and `pub(in ...)` are not Rust public names. With a previous baseline, a removed name still wrapped in backticks is `wiki_mentions_removed_symbols`, and a changed public surface with an unchanged wiki hash is `wiki_not_updated_for_api_change`. Go, C/C++, C#, and Java have no symbol extractor, so their rows are `unchecked`. `health --summary` includes `green_breakdown`. A successful mark removes that file's pending line. -- The scan ignores comments and strings. It does not use CPython `ast`. It does not read tsconfig `paths`, package `exports`, or npm workspace maps. Named re-exports are the barrel follow. Rust reads the root `Cargo.toml` `[package].name`, `[lib].name`, and `[workspace] members` (globs included; hyphens become underscores). It resolves `crate::`, `self::`, `super::`, those crate names, and `mod` to files. `crate::` stays in the file's crate (`tests/*.rs` and `src/bin/*.rs` are their own crates, and a workspace member does not share `crate::` with another member). A package or lib name points at that member's library. Accuracy is `accuracy_fixtures_match_frozen_edges`, including the Rust crate layout. This parser has not been run on Babylon.js. -- MCP `tools/call` sets `isError` when the payload has `success: false`. `resources/read` and `prompts/get` answer. A call waits at most `WIKIFIER_MCP_DEADLINE_S` seconds (default 60) and then returns `timed_out` and `may_still_complete`. An identical journal action, file, and reason inside 120 seconds sets `duplicate_skipped` and does not append another line. +- `record_changes` matches reasons by exact path, then the longest `dir/` prefix, then the longest glob (`*` and `?`). `--reasons` accepts a JSON object (`{"path": "why"}`) or `PATH=REASON` lines (`-` reads stdin). `--only`, `--green`, `--dry-run`, and deletions are applied. `--green` lists verification failures and leaves those files Yellow. MCP `reasons` may be that object. +- `mark_green` extracts Python public names with a tokenizer (multi-line `__all__` lists, tuples, and `+=`; comments and strings ignored; top-level `def`, `async def`, and `class`, including decorated ones; `.py` and `.pyi`), JS/TS `export` names after comments and strings are masked, and Rust unrestricted `pub` items (`fn`, `struct`, `enum`, `trait`, `mod`, `type`, `const`, `static`, and `pub use` names), plus `pub` methods and associated items on a public type and the items written in a trait impl for that type. Each current name must appear in the wiki. `pub(crate)`, `pub(super)`, `pub(self)`, and `pub(in ...)` are not Rust public names. With a previous baseline, a removed name still wrapped in backticks is `wiki_mentions_removed_symbols`, and a changed public surface with an unchanged wiki hash is `wiki_not_updated_for_api_change`. Go, C/C++, C#, and Java have no symbol extractor, so their rows are `unchecked`. The wiki is `.wiki.md`, then `.md`, the stem beside the source, then the same names under `docs/wiki`, `docs`, `wiki`, `documentation`, and `.wiki`. `health --summary` returns `health_summary` (including `green_breakdown`) without the entry list. `health --format healing-stats` is the stub counts. `--directory` limits health, bootstrap, and suggest-next. A successful mark removes that file's pending line. Pending bullets are `- path: message` (`LLM/agent edit — …` after `record-change`). +- `suggest-next` adds `review_low_confidence_edges` when a mapped file has a resolved, non-dynamic edge below confidence 0.65. The count is stored while parsing (`low_conf.tsv`) so a warm `update-maps` does not rescan it. +- `list-paths --depth 0` or `--recursive` lists every file under the prefix. `--max-files` caps that list. `verify-wikis --limit` and `stale-wikis --limit` cap the rows returned. `cycles` adds `severity`, `score`, `external_blast_radius`, `recommendation`, and `reverse_dependencies` (top 20 by inbound edges). `--no-analysis` omits those. `--max-items` caps the cycle list. +- `seed-source-hashes` writes the current hash onto Green rows and does not change status. `--force` replaces a hash that is already stored. `seed-health` still adds Initial-stub rows. `heal-stubs` promotes a stub whose wiki is at least 350 characters (`--min-wiki-length` overrides). +- Health stubs from `update-maps` follow `monitored_paths.txt` when that file lists real paths. If the file is absent, empty, or only a bare `.`, every mapped file is seeded. +- The scan ignores comments and strings. It does not use CPython `ast`. It does read `tsconfig.json` `paths`, `package.json` `exports`, and npm `workspaces` (`*` directory globs). Named re-exports are the barrel follow. Rust reads the root `Cargo.toml` `[package].name`, `[lib].name`, and `[workspace] members` (globs included; hyphens become underscores). It resolves `crate::`, `self::`, `super::`, those crate names, and `mod` to files. `crate::` stays in the file's crate (`tests/*.rs` and `src/bin/*.rs` are their own crates, and a workspace member does not share `crate::` with another member). A package or lib name points at that member's library. Accuracy is `accuracy_fixtures_match_frozen_edges`, including the Rust crate layout. This parser has not been run on Babylon.js. +- MCP `tools/call` sets `isError` when the payload has `success: false`. `resources/read` and `prompts/get` answer. Prompts name the tools to call. Optional arguments include `format` (`text` or `json`), `timeout_s`, `directory`, `detail`, `low_confidence_only`, `unresolved_only`, `depth`, `recursive`, `max_files`, `limit`, `max_items`, and a `reasons` object. A call waits at most `timeout_s` or `WIKIFIER_MCP_DEADLINE_S` seconds (default 60) and then returns `timed_out` and `may_still_complete`. An identical journal action, file, and reason inside 120 seconds sets `duplicate_skipped` and does not append another line. The CLI default is JSON. `--text` prints prose. `health --format text` does the same for health. - Commands that write health, pending, the map, or the cache take `.wikifier_staging/.lock` before the cache lock. The lock is re-entrant on the same thread, waits 30 seconds, and is stolen when the recorded pid is dead. A timeout returns `error_type: "LockTimeoutError"` and `timed_out: true`. -- `init` keeps a path list that already has a non-comment entry. It seeds `src/` only when a list is empty and `src/` exists. It copies `index.html` only when that file is absent, and it writes `.wikifier/marker`. -- `serve` listens on `127.0.0.1:8787` until stopped. It answers `/`, `/__wikifier/api/{bootstrap,file,dependencies,dependents,cycles,journal,diagnostics}`, `/__wikifier/status`, and the project files the dashboard reads (`file_health.json`, `library.md`). `POST /__wikifier/run` accepts `update-maps`, `update-maps --full`, and `check-changes`. `POST /__wikifier/shutdown` stops the process. `monitor` repeats `check-changes` (default every 30 seconds). `daemon run` runs `check-changes` and `update-maps` until the pid file disappears. `WIKIFIER_DAEMON_MAPS=0` skips the map. `daemon start` refuses readiness `blocked` or `not_ready` unless `--force`. +- `init` keeps a path list that already has a non-comment entry. When a list is missing it writes a `src/` template. It also writes `exclude_patterns.txt`, empty `file_health.md`, `pending_updates.md`, `library.md`, `.wikifier/config`, `.wikifier/marker`, and the `journal/` and `Logged_issues/` directories when those paths are absent. `--no-copy` skips `index.html`. Otherwise `index.html` is copied only when it is missing. +- `serve` listens on `127.0.0.1:8787` until stopped. It answers `/`, `/__wikifier/api/{bootstrap,file,dependencies,dependents,cycles,journal,diagnostics}`, `/__wikifier/status`, and the project files the dashboard reads (`file_health.json`, `library.md`). `POST /__wikifier/run` accepts `update-maps`, `update-maps --full`, and `check-changes`. `POST /__wikifier/shutdown` stops the process. `monitor` repeats `check-changes` (default every 30 seconds). `daemon run` checks every `WIKIFIER_POLL_INTERVAL` seconds (default 30) and runs `update-maps` at start and then at most every `WIKIFIER_DAEMON_MAPS_INTERVAL` seconds (default 600). `WIKIFIER_DAEMON_MAPS=0` skips the map. State is `.wikifier_staging/wikifier.pid`, `daemon.log`, and `daemon_heartbeat.json`. `daemon logs`, `restart`, `install-service`, and `uninstall-service` are supported (systemd user unit on Linux, launchd plist on macOS, Task Scheduler on Windows). `daemon start` refuses readiness `blocked` unless `--force`. `not_ready` may start. `issues` is the same `actions[]` list as `suggest-next`. `library.md` stays the short tree, Mermaid block, and dependency table; cycle severity is on `cycles`, not in that file. ```bash wikifier --target /abs/project session-bootstrap diff --git a/src/api.rs b/src/api.rs index 9bb9e67..769ea0b 100644 --- a/src/api.rs +++ b/src/api.rs @@ -21,7 +21,11 @@ pub struct MapRun { fn lock_or_fail(root: &Path) -> Result { crate::project_lock::ProjectLock::acquire(root).map_err(|err| { - let kind = if err.contains("locked") { "LockTimeoutError" } else { "LockError" }; + let kind = if err.contains("locked") { + "LockTimeoutError" + } else { + "LockError" + }; let mut value = fail(kind, err); if kind == "LockTimeoutError" { value.insert("timed_out", Json::Bool(true)); @@ -86,7 +90,10 @@ fn root_or_err(root: Option<&Path>) -> Result { None => std::env::current_dir().map_err(|e| fail("ProjectRootError", e.to_string()))?, }; if !root.is_dir() { - return Err(fail("ProjectRootError", format!("project root does not exist: {}", root.display()))); + return Err(fail( + "ProjectRootError", + format!("project root does not exist: {}", root.display()), + )); } Ok(root) } @@ -96,12 +103,38 @@ fn scope_json(scope: &Scope) -> Json { ("ok", Json::Bool(scope.ok)), ("container", Json::Bool(scope.container)), ("child_project_count", Json::Int(scope.child_count as i64)), - ("child_project_sample", Json::Array(scope.child_sample.iter().cloned().map(Json::String).collect())), + ( + "child_project_sample", + Json::Array( + scope + .child_sample + .iter() + .cloned() + .map(Json::String) + .collect(), + ), + ), ("bare_dot_monitor", Json::Bool(scope.bare_dot)), ("full_tree", Json::Bool(scope.full_tree)), - ("missing_dirs", Json::Array(scope.missing_dirs.iter().cloned().map(Json::String).collect())), - ("prefixes", Json::Array(scope.prefixes.iter().cloned().map(Json::String).collect())), - ("warnings", Json::Array(scope.warnings.iter().cloned().map(Json::String).collect())), + ( + "missing_dirs", + Json::Array( + scope + .missing_dirs + .iter() + .cloned() + .map(Json::String) + .collect(), + ), + ), + ( + "prefixes", + Json::Array(scope.prefixes.iter().cloned().map(Json::String).collect()), + ), + ( + "warnings", + Json::Array(scope.warnings.iter().cloned().map(Json::String).collect()), + ), ]) } @@ -121,7 +154,12 @@ fn health_file_present(root: &Path) -> bool { root.join("file_health.json").is_file() || root.join("file_health.md").is_file() } -fn readiness_of(scope: &Scope, has_map: bool, rows: &[crate::artifacts::HealthRow], has_health_file: bool) -> &'static str { +fn readiness_of( + scope: &Scope, + has_map: bool, + rows: &[crate::artifacts::HealthRow], + has_health_file: bool, +) -> &'static str { // A container or a missing configured directory is a scope risk even when // there is no health matrix yet. Do not call that ready_for_daemon. if scope.container || !scope.missing_dirs.is_empty() { @@ -136,7 +174,10 @@ fn readiness_of(scope: &Scope, has_map: bool, rows: &[crate::artifacts::HealthRo if rows.iter().any(|r| is_red(&r.status)) { return "not_ready"; } - if rows.iter().any(|r| is_yellow(&r.status) && r.reason != "Initial stub") { + if rows + .iter() + .any(|r| is_yellow(&r.status) && r.reason != "Initial stub") + { return "ready_with_agent_wiki_work"; } "ready_for_daemon" @@ -149,7 +190,16 @@ pub fn update_maps(root: &Path) -> Json { /// `full` rehashes every file, including ones whose size and mtime still match. /// `quiet` keeps `edges_persisted` and omits the edge list. pub fn update_maps_opts(root: &Path, full: bool, quiet: bool) -> Json { - update_maps_run(root, &MapRun { full, quiet, directory: None, max_files: None, stream: false }) + update_maps_run( + root, + &MapRun { + full, + quiet, + directory: None, + max_files: None, + stream: false, + }, + ) } pub fn update_maps_run(root: &Path, run: &MapRun) -> Json { @@ -170,10 +220,18 @@ pub fn update_maps_run(root: &Path, run: &MapRun) -> Json { ("files_remaining_dirty", Json::Int(0)), ("edges", Json::Array(vec![])), ("scope", scope_json(&scope)), - ("message", Json::String("container root refused; target one child project".into())), + ( + "message", + Json::String("container root refused; target one child project".into()), + ), ]); } - if let Some(dir) = run.directory.as_ref().map(|s| s.trim().trim_matches('/')).filter(|s| !s.is_empty()) { + if let Some(dir) = run + .directory + .as_ref() + .map(|s| s.trim().trim_matches('/')) + .filter(|s| !s.is_empty()) + { let path = root.join(dir); if !path.is_dir() { return fail("ScopeError", format!("missing map directory: {dir}")); @@ -184,9 +242,18 @@ pub fn update_maps_run(root: &Path, run: &MapRun) -> Json { scope.ok = true; } if !scope.missing_dirs.is_empty() { - return fail("ScopeError", format!("missing map directory: {}", scope.missing_dirs.join(", "))); + return fail( + "ScopeError", + format!("missing map directory: {}", scope.missing_dirs.join(", ")), + ); } - stream_event(run.stream, Json::obj(vec![("event", Json::String("update_maps".into())), ("phase", Json::String("start".into()))])); + stream_event( + run.stream, + Json::obj(vec![ + ("event", Json::String("update_maps".into())), + ("phase", Json::String("start".into())), + ]), + ); let files = match scope::collect(root, &scope) { Ok(f) => f, Err(e) => return fail("ScopeError", e), @@ -194,7 +261,11 @@ pub fn update_maps_run(root: &Path, run: &MapRun) -> Json { let mut session = match Session::open(root) { Ok(session) => session, Err(err) => { - let kind = if err.contains("locked") { "CacheLocked" } else { "CacheError" }; + let kind = if err.contains("locked") { + "CacheLocked" + } else { + "CacheError" + }; return fail_warnings(kind, err, &[]); } }; @@ -207,9 +278,15 @@ pub fn update_maps_run(root: &Path, run: &MapRun) -> Json { seen.insert(rel.clone()); let (mtime, size) = cache::stat_ns(file).unwrap_or((0, 0)); let known = session.known(&rel); - let stat_hit = !full && known.as_ref().is_some_and(|(old_mtime, old_size, old_hash)| { - *old_mtime != 0 && *old_mtime == mtime && *old_size == size && old_hash.starts_with("sha256:") - }); + let stat_hit = !full + && known + .as_ref() + .is_some_and(|(old_mtime, old_size, old_hash)| { + *old_mtime != 0 + && *old_mtime == mtime + && *old_size == size + && old_hash.starts_with("sha256:") + }); if stat_hit { skipped += 1; continue; @@ -237,34 +314,81 @@ pub fn update_maps_run(root: &Path, run: &MapRun) -> Json { to_parse.truncate(limit); } } - let changed_paths: Vec = to_parse.iter().map(|(path, _, _, _, _)| path.clone()).collect(); + let changed_paths: Vec = to_parse + .iter() + .map(|(path, _, _, _, _)| path.clone()) + .collect(); let analysis = if to_parse.is_empty() { - crate::parse::Analysis { facts: BTreeMap::new(), scans: std::collections::HashMap::new(), warnings: Vec::new() } + crate::parse::Analysis { + facts: BTreeMap::new(), + scans: std::collections::HashMap::new(), + warnings: Vec::new(), + } } else if changed_paths.len() == files.len() { crate::parse::analyze_files(root, &files) } else { crate::parse::analyze_changed(root, &files, &changed_paths) }; let mut warnings = analysis.warnings.clone(); + let mut low_conf_updates = Vec::new(); for (path, rel, mtime, size, hash) in &to_parse { let _ = path; let fact = analysis.facts.get(rel); let dsts = fact.map(|f| f.dsts.clone()).unwrap_or_default(); let scan = analysis.scans.get(rel); - let barrel = scan.map(|s| s.imports.iter().any(|i| matches!(i.kind, crate::parse::Kind::ExportFrom | crate::parse::Kind::ExportStar))).unwrap_or(false); - let imports: Vec = scan.map(|s| s.imports.iter().map(|i| i.spec.clone()).collect()).unwrap_or_default(); + let barrel = scan + .map(|s| { + s.imports.iter().any(|i| { + matches!( + i.kind, + crate::parse::Kind::ExportFrom | crate::parse::Kind::ExportStar + ) + }) + }) + .unwrap_or(false); + let imports: Vec = scan + .map(|s| s.imports.iter().map(|i| i.spec.clone()).collect()) + .unwrap_or_default(); let barrel_dsts: Vec = dsts.iter().cloned().collect(); - let edge_lines = fact.map(|f| { - f.internal.iter().map(|edge| cache::format_edge_line(&edge.dst, edge.line, &edge.names, edge.conditional, edge.dynamic, edge.via_barrel)).collect() - }).unwrap_or_default(); + let edge_lines = fact + .map(|f| { + f.internal + .iter() + .map(|edge| { + cache::format_edge_line( + &edge.dst, + edge.line, + &edge.names, + edge.conditional, + edge.dynamic, + edge.via_barrel, + ) + }) + .collect() + }) + .unwrap_or_default(); + let low_count = fact + .map(|f| { + let flags: Vec<(bool, bool, bool)> = f + .internal + .iter() + .map(|edge| (edge.conditional, edge.dynamic, edge.via_barrel)) + .collect(); + crate::low_conf::count_edges(&flags) + }) + .unwrap_or(0); + low_conf_updates.push((rel.clone(), low_count)); let external = fact.map(|f| f.external.clone()).unwrap_or_default(); let unresolved = fact.map(|f| f.unresolved.clone()).unwrap_or_default(); if run.stream { - stream_event(true, Json::obj(vec![ - ("event", Json::String("update_maps".into())), - ("phase", Json::String("parsed".into())), - ("file", Json::String(rel.clone())), - ])); + stream_event( + true, + Json::obj(vec![ + ("event", Json::String("update_maps".into())), + ("phase", Json::String("parsed".into())), + ("file", Json::String(rel.clone())), + ]), + ); } session.upsert(&FileUpdate { rel: rel.clone(), @@ -282,10 +406,15 @@ pub fn update_maps_run(root: &Path, run: &MapRun) -> Json { } } let parsed = to_parse.len() as i64; - let stale: Vec = session.rels().into_iter().filter(|rel| !seen.contains(rel)).collect(); + let stale: Vec = session + .rels() + .into_iter() + .filter(|rel| !seen.contains(rel)) + .collect(); for rel in &stale { session.remove(rel); } + crate::low_conf::store_updates(root, &low_conf_updates, &stale); let mut edge_pairs: Vec<(String, String)> = Vec::new(); if !quiet { for rel in &seen { @@ -295,12 +424,18 @@ pub fn update_maps_run(root: &Path, run: &MapRun) -> Json { } } let edge_n: i64 = if quiet { - seen.iter().map(|rel| session.dsts_of(rel).len() as i64).sum() + seen.iter() + .map(|rel| session.dsts_of(rel).len() as i64) + .sum() } else { edge_pairs.len() as i64 }; if let Err(err) = session.commit() { - let kind = if err.contains("locked") { "CacheLocked" } else { "CacheError" }; + let kind = if err.contains("locked") { + "CacheLocked" + } else { + "CacheError" + }; return fail_warnings(kind, err, &warnings); } let _ = seed_health_from_map(root); @@ -313,21 +448,32 @@ pub fn update_maps_run(root: &Path, run: &MapRun) -> Json { ("files_remaining_dirty", Json::Int(files_remaining_dirty)), ("files_parsed", Json::Int(parsed)), ("files_skipped", Json::Int(skipped)), - ("directory", Json::String(run.directory.clone().unwrap_or_default())), + ( + "directory", + Json::String(run.directory.clone().unwrap_or_default()), + ), ("parser", Json::String("regex+reexport".into())), ]); if let Err(err) = crate::maps_out::write_status(root, &status) { warnings.push(format!("incremental status: {err}")); } - let edge_json = Json::Array(edge_pairs.into_iter().map(|(src, dst)| { - Json::obj(vec![("src", Json::String(src)), ("dst", Json::String(dst))]) - }).collect()); - stream_event(run.stream, Json::obj(vec![ - ("event", Json::String("update_maps".into())), - ("phase", Json::String("done".into())), - ("map_complete", Json::Bool(map_complete)), - ("files_remaining_dirty", Json::Int(files_remaining_dirty)), - ])); + let edge_json = Json::Array( + edge_pairs + .into_iter() + .map(|(src, dst)| { + Json::obj(vec![("src", Json::String(src)), ("dst", Json::String(dst))]) + }) + .collect(), + ); + stream_event( + run.stream, + Json::obj(vec![ + ("event", Json::String("update_maps".into())), + ("phase", Json::String("done".into())), + ("map_complete", Json::Bool(map_complete)), + ("files_remaining_dirty", Json::Int(files_remaining_dirty)), + ]), + ); ok(vec![ ("map_complete", Json::Bool(map_complete)), ("files_remaining_dirty", Json::Int(files_remaining_dirty)), @@ -343,7 +489,10 @@ pub fn update_maps_run(root: &Path, run: &MapRun) -> Json { ("scope", scope_json(&scope)), ("cache_backend", Json::String("wk_cache".into())), ("parser", Json::String("regex+reexport".into())), - ("warnings", Json::Array(warnings.into_iter().map(Json::String).collect())), + ( + "warnings", + Json::Array(warnings.into_iter().map(Json::String).collect()), + ), ]) } @@ -355,7 +504,10 @@ fn stream_event(stream: bool, value: Json) { fn fail_warnings(error_type: &str, error: impl Into, warnings: &[String]) -> Json { let mut value = fail(error_type, error); - value.insert("warnings", Json::Array(warnings.iter().cloned().map(Json::String).collect())); + value.insert( + "warnings", + Json::Array(warnings.iter().cloned().map(Json::String).collect()), + ); value } @@ -371,12 +523,25 @@ fn seed_health_from_map(root: &Path) -> Result<(), String> { let known: BTreeSet = rows.iter().map(|r| r.file.clone()).collect(); let stamp = local_stamp(); let mut changed = false; + let scoped = monitored_seed_prefixes(root); for rel in cache::all_paths(root) { + if let Some(prefixes) = &scoped { + if !prefixes.iter().any(|prefix| under_prefix(prefix, &rel)) { + continue; + } + } if known.contains(&rel) { continue; } let hash = file_sha256_prefixed(&root.join(&rel)).unwrap_or_default(); - rows.push(HealthRow::new(rel, "🟡 Yellow", stamp.clone(), "Initial stub", hash, "no_wiki")); + rows.push(HealthRow::new( + rel, + "🟡 Yellow", + stamp.clone(), + "Initial stub", + hash, + "no_wiki", + )); changed = true; } if changed || !root.join("file_health.json").is_file() && !rows.is_empty() { @@ -385,6 +550,77 @@ fn seed_health_from_map(root: &Path) -> Result<(), String> { Ok(()) } +/// `None` seeds every mapped file: `monitored_paths.txt` is absent, empty, or only a bare `.`. +/// A list of real paths seeds only those files and directories. +fn monitored_seed_prefixes(root: &Path) -> Option> { + if !root.join("monitored_paths.txt").is_file() { + return None; + } + let entries = scope::read_path_list(root, "monitored_paths.txt"); + if entries.is_empty() { + None + } else { + Some(entries) + } +} + +pub fn seed_source_hashes(root: &Path, force: bool, directory: Option<&str>) -> Json { + let _lock = match lock_or_fail(root) { + Ok(lock) => lock, + Err(err) => return err, + }; + let mut rows = artifacts::read_health(root); + let mut seeded = 0i64; + let mut skipped_has_hash = 0i64; + let mut skipped_not_green = 0i64; + let mut skipped_missing = 0i64; + for row in &mut rows { + if let Some(dir) = directory + .map(|d| d.trim_matches('/')) + .filter(|d| !d.is_empty()) + { + if !under_prefix(dir, &row.file) { + continue; + } + } + if !is_green(&row.status) { + skipped_not_green += 1; + continue; + } + if row.source_hash.starts_with("sha256:") && !force { + skipped_has_hash += 1; + continue; + } + let path = root.join(&row.file); + if !path.is_file() { + skipped_missing += 1; + continue; + } + let hash = file_sha256_prefixed(&path).unwrap_or_default(); + if hash.is_empty() { + skipped_missing += 1; + continue; + } + row.source_hash = hash; + row.source_mtime_ns = cache::stat_ns(&path) + .map(|(mtime, _)| mtime.to_string()) + .unwrap_or_default(); + seeded += 1; + } + if seeded > 0 { + if let Err(err) = artifacts::write_health(root, &rows) { + return fail("HealthError", err); + } + } + ok(vec![ + ("seeded", Json::Int(seeded)), + ("skipped_has_hash", Json::Int(skipped_has_hash)), + ("skipped_not_green", Json::Int(skipped_not_green)), + ("skipped_missing", Json::Int(skipped_missing)), + ("force", Json::Bool(force)), + ]) +} + pub fn mapped_edges(root: &Path) -> BTreeSet<(String, String)> { cache::edges(root) } @@ -398,18 +634,49 @@ pub fn dependency_report(root: &Path, rel: &str, full: bool) -> Json { let rel = rel.trim_start_matches("./"); match cache::load_dependency_view(root, rel) { Ok(view) => { - let cycled = if full { cycle_pairs(root) } else { BTreeSet::new() }; - let internal: Vec = view.internal.iter().map(|edge| edge_record(edge, full, cycled.contains(&(rel.to_string(), edge.path.clone())))).collect(); + let cycled = if full { + cycle_pairs(root) + } else { + BTreeSet::new() + }; + let internal: Vec = view + .internal + .iter() + .map(|edge| { + edge_record( + edge, + full, + cycled.contains(&(rel.to_string(), edge.path.clone())), + ) + }) + .collect(); let mut pairs = vec![ ("file", Json::String(rel.into())), - ("detail", Json::String(if full { "full" } else { "compact" }.into())), + ( + "detail", + Json::String(if full { "full" } else { "compact" }.into()), + ), ("internal", Json::Array(internal.clone())), - ("external", Json::Array(view.external.iter().cloned().map(Json::String).collect())), - ("needs_review", Json::Array(view.unresolved.iter().cloned().map(|(raw, line, reason)| Json::obj(vec![ - ("raw", Json::String(raw)), - ("line", Json::Int(line)), - ("reason", Json::String(reason)), - ])).collect())), + ( + "external", + Json::Array(view.external.iter().cloned().map(Json::String).collect()), + ), + ( + "needs_review", + Json::Array( + view.unresolved + .iter() + .cloned() + .map(|(raw, line, reason)| { + Json::obj(vec![ + ("raw", Json::String(raw)), + ("line", Json::Int(line)), + ("reason", Json::String(reason)), + ]) + }) + .collect(), + ), + ), ]; if full { pairs.push(("imports", Json::Array(internal))); @@ -424,7 +691,10 @@ fn edge_record(edge: &cache::InternalEdge, full: bool, in_cycle: bool) -> Json { let mut pairs = vec![ ("path".into(), Json::String(edge.path.clone())), ("line".into(), Json::Int(edge.line)), - ("names".into(), Json::Array(edge.names.iter().cloned().map(Json::String).collect())), + ( + "names".into(), + Json::Array(edge.names.iter().cloned().map(Json::String).collect()), + ), ("conditional".into(), Json::Bool(edge.conditional)), ("dynamic".into(), Json::Bool(edge.dynamic)), ("via_barrel".into(), Json::Bool(edge.via_barrel)), @@ -433,7 +703,10 @@ fn edge_record(edge: &cache::InternalEdge, full: bool, in_cycle: bool) -> Json { if full { pairs.push(("resolved_path".into(), Json::String(edge.path.clone()))); pairs.push(("confidence_score".into(), Json::Float(edge.confidence()))); - pairs.push(("confidence_reasons".into(), Json::Array(vec![Json::String(edge.acs_reason().into())]))); + pairs.push(( + "confidence_reasons".into(), + Json::Array(vec![Json::String(edge.acs_reason().into())]), + )); pairs.push(("acs".into(), Json::String(edge.acs_reason().into()))); pairs.push(("is_dynamic".into(), Json::Bool(edge.dynamic))); pairs.push(("is_conditional".into(), Json::Bool(edge.conditional))); @@ -447,16 +720,46 @@ pub fn load_dependents(root: &Path, rel: &str) -> Json { match cache::load_dependents(root, rel) { Ok(srcs) => ok(vec![ ("file", Json::String(rel.into())), - ("dependents", Json::Array(srcs.into_iter().map(Json::String).collect())), + ( + "dependents", + Json::Array(srcs.into_iter().map(Json::String).collect()), + ), ]), Err(e) => fail("CacheError", e), } } pub fn list_paths(root: &Path, prefix: &str) -> Json { + list_paths_at(root, prefix, 1, None) +} + +pub fn list_paths_at(root: &Path, prefix: &str, depth: i64, max_files: Option) -> Json { let _ = crate::legacy_cache::import_if_empty(root); - let paths = cache::list_prefix(root, prefix).unwrap_or_default(); + let mut paths = cache::list_prefix(root, prefix).unwrap_or_default(); let pref = prefix.trim_matches('/'); + if depth <= 0 { + let mut truncated = false; + if let Some(max) = max_files { + if paths.len() > max { + paths.truncate(max); + truncated = true; + } + } + let mut value = ok(vec![ + ("prefix", Json::String(pref.into())), + ("depth", Json::Int(0)), + ( + "files", + Json::Array(paths.iter().cloned().map(Json::String).collect()), + ), + ("file_count", Json::Int(paths.len() as i64)), + ("files_truncated", Json::Bool(truncated)), + ]); + if truncated { + value.insert("files_complete", Json::Bool(false)); + } + return value; + } // depth-1: a path directly under prefix is a file; a longer path contributes to its first segment let mut direct_files = Vec::new(); let mut dir_counts: BTreeMap = BTreeMap::new(); @@ -476,25 +779,65 @@ pub fn list_paths(root: &Path, prefix: &str) -> Json { direct_files.push(path.clone()); } } - ok(vec![ + let mut truncated = false; + if let Some(max) = max_files { + if direct_files.len() > max { + direct_files.truncate(max); + truncated = true; + } + } + let mut value = ok(vec![ ("prefix", Json::String(pref.into())), - ("files", Json::Array(direct_files.into_iter().map(Json::String).collect())), - ("dirs", Json::Array(dir_counts.into_iter().map(|(p, n)| Json::obj(vec![("p", Json::String(p)), ("n", Json::Int(n))])).collect())), + ("depth", Json::Int(1)), + ( + "files", + Json::Array(direct_files.into_iter().map(Json::String).collect()), + ), + ( + "dirs", + Json::Array( + dir_counts + .into_iter() + .map(|(p, n)| Json::obj(vec![("p", Json::String(p)), ("n", Json::Int(n))])) + .collect(), + ), + ), ("file_count", Json::Int(paths.len() as i64)), - ]) + ("files_truncated", Json::Bool(truncated)), + ]); + if truncated { + value.insert("files_complete", Json::Bool(false)); + } + value } pub fn session_bootstrap(root: &Path) -> Json { + session_bootstrap_scoped(root, None) +} + +pub fn session_bootstrap_scoped(root: &Path, directory: Option<&str>) -> Json { let scope = scope::detect(root); if scope.container || !scope.missing_dirs.is_empty() { let actions = Json::Array(vec![Json::obj(vec![ ("action", Json::String("fix_scope".into())), ("priority", Json::Int(1)), - ("reason", Json::String(scope.warnings.first().cloned().unwrap_or_else(|| "scope risk".into()))), + ( + "reason", + Json::String( + scope + .warnings + .first() + .cloned() + .unwrap_or_else(|| "scope risk".into()), + ), + ), ])]); return ok(vec![ ("version", Json::String(VERSION.into())), - ("readiness", Json::String(readiness_of(&scope, false, &[], false).into())), + ( + "readiness", + Json::String(readiness_of(&scope, false, &[], false).into()), + ), ("health_summary", health_summary(root)), ("actions", actions), ("files", Json::Array(vec![])), @@ -512,21 +855,40 @@ pub fn session_bootstrap(root: &Path) -> Json { if artifacts::folder_card(root, "").is_none() && !cache::all_paths(root).is_empty() { let _ = crate::maps_out::write(root); } - let files: Vec = cache::all_paths(root).into_iter().map(Json::String).collect(); - let rows = artifacts::read_health(root); - let actions = suggest_next_actions(root); - let action_list = actions.get("actions").cloned().unwrap_or(Json::Array(vec![])); + let mut file_names = cache::all_paths(root); + let mut rows = artifacts::read_health(root); + if let Some(dir) = directory + .map(|d| d.trim_matches('/')) + .filter(|d| !d.is_empty()) + { + file_names.retain(|rel| under_prefix(dir, rel)); + rows.retain(|row| under_prefix(dir, &row.file)); + } + let files: Vec = file_names.iter().cloned().map(Json::String).collect(); + let actions = suggest_next_scoped(root, directory); + let action_list = actions + .get("actions") + .cloned() + .unwrap_or(Json::Array(vec![])); let map_index = artifacts::folder_card(root, "") .and_then(|text| crate::jsonutil::parse(&text).ok()) .unwrap_or_else(|| Json::obj(vec![])); let mut value = ok(vec![ ("version", Json::String(VERSION.into())), - ("readiness", Json::String(readiness_of(&scope, !files.is_empty(), &rows, health_file_present(root)).into())), - ("health_summary", health_summary(root)), + ( + "readiness", + Json::String( + readiness_of(&scope, !files.is_empty(), &rows, health_file_present(root)).into(), + ), + ), + ("health_summary", health_summary_of(&rows)), ("actions", action_list), ("files", Json::Array(files)), ("map_index", map_index), - ("map_complete", Json::Bool(scope.missing_dirs.is_empty() && !scope.container)), + ( + "map_complete", + Json::Bool(scope.missing_dirs.is_empty() && !scope.container), + ), ("scope", scope_json(&scope)), ]); if let Some(warning) = legacy_warning { @@ -543,11 +905,14 @@ pub fn session_bootstrap(root: &Path) -> Json { } fn health_summary(root: &Path) -> Json { - let rows = artifacts::read_health(root); + health_summary_of(&artifacts::read_health(root)) +} + +fn health_summary_of(rows: &[HealthRow]) -> Json { let mut green = 0i64; let mut yellow = 0i64; let mut red = 0i64; - for row in &rows { + for row in rows { if row.status.contains("Green") || row.status.contains('🟢') { green += 1; } else if row.status.contains("Red") || row.status.contains('🔴') { @@ -562,7 +927,7 @@ fn health_summary(root: &Path) -> Json { let mut unchecked = 0i64; let mut legacy = 0i64; let mut other = 0i64; - for row in &rows { + for row in rows { if !is_green(&row.status) { continue; } @@ -580,18 +945,25 @@ fn health_summary(root: &Path) -> Json { ("green", Json::Int(green)), ("yellow", Json::Int(yellow)), ("red", Json::Int(red)), - ("green_breakdown", Json::obj(vec![ - ("verified", Json::Int(verified)), - ("forced", Json::Int(forced)), - ("no_wiki", Json::Int(no_wiki)), - ("unchecked", Json::Int(unchecked)), - ("legacy", Json::Int(legacy)), - ("other", Json::Int(other)), - ])), + ( + "green_breakdown", + Json::obj(vec![ + ("verified", Json::Int(verified)), + ("forced", Json::Int(forced)), + ("no_wiki", Json::Int(no_wiki)), + ("unchecked", Json::Int(unchecked)), + ("legacy", Json::Int(legacy)), + ("other", Json::Int(other)), + ]), + ), ]) } pub fn suggest_next_actions(root: &Path) -> Json { + suggest_next_scoped(root, None) +} + +pub fn suggest_next_scoped(root: &Path, directory: Option<&str>) -> Json { let scope = scope::detect(root); let mut actions = Vec::new(); if scope.container || !scope.missing_dirs.is_empty() { @@ -602,6 +974,14 @@ pub fn suggest_next_actions(root: &Path) -> Json { ])); } for row in artifacts::read_health(root) { + if let Some(dir) = directory + .map(|d| d.trim_matches('/')) + .filter(|d| !d.is_empty()) + { + if !under_prefix(dir, &row.file) { + continue; + } + } if row.status.contains("Red") || row.status.contains('🔴') { actions.push(Json::obj(vec![ ("action", Json::String("record_deletion".into())), @@ -609,7 +989,9 @@ pub fn suggest_next_actions(root: &Path) -> Json { ("priority", Json::Int(1)), ("reason", Json::String(row.reason)), ])); - } else if row.reason != "Initial stub" && (row.status.contains("Yellow") || row.status.contains('🟡')) { + } else if row.reason != "Initial stub" + && (row.status.contains("Yellow") || row.status.contains('🟡')) + { actions.push(Json::obj(vec![ ("action", Json::String("prepare_edit".into())), ("file", Json::String(row.file)), @@ -618,15 +1000,56 @@ pub fn suggest_next_actions(root: &Path) -> Json { ])); } } + let low_conf = crate::low_conf::actionable_total(root, directory); + if low_conf > 0 { + actions.push(Json::obj(vec![ + ("action", Json::String("review_low_confidence_edges".into())), + ("priority", Json::Int(4)), + ( + "reason", + Json::String(format!( + "{low_conf} actionable low-confidence project edges (see get_dependencies)" + )), + ), + ])); + } if actions.is_empty() { actions.push(Json::obj(vec![ ("action", Json::String("lookup_only".into())), ("priority", Json::Int(5)), - ("reason", Json::String("Health clean. Use prepare_edit for lookup.".into())), + ( + "reason", + Json::String("Health clean. Use prepare_edit for lookup.".into()), + ), ])); } + actions.sort_by(|a, b| { + let pa = match a.get("priority") { + Some(Json::Int(n)) => *n, + _ => 9, + }; + let pb = match b.get("priority") { + Some(Json::Int(n)) => *n, + _ => 9, + }; + pa.cmp(&pb) + }); let rows = artifacts::read_health(root); - ok(vec![("actions", Json::Array(actions)), ("readiness", Json::String(readiness_of(&scope, !cache::all_paths(root).is_empty(), &rows, health_file_present(root)).into()))]) + ok(vec![ + ("actions", Json::Array(actions)), + ( + "readiness", + Json::String( + readiness_of( + &scope, + !cache::all_paths(root).is_empty(), + &rows, + health_file_present(root), + ) + .into(), + ), + ), + ]) } pub fn check_changes(root: &Path) -> Json { @@ -653,7 +1076,10 @@ pub fn check_changes(root: &Path) -> Json { watched.insert(entry); } } - let cap = std::env::var("WIKIFIER_CHECK_CHANGES_MAX").ok().and_then(|s| s.parse::().ok()).unwrap_or(2000); + let cap = std::env::var("WIKIFIER_CHECK_CHANGES_MAX") + .ok() + .and_then(|s| s.parse::().ok()) + .unwrap_or(2000); let mut watched: Vec = watched.into_iter().collect(); watched.sort(); let truncated = watched.len() > cap; @@ -676,12 +1102,17 @@ pub fn check_changes(root: &Path) -> Json { local_stamp(), "missing on disk (check-changes)", idx.map(|i| rows[i].source_hash.clone()).unwrap_or_default(), - idx.map(|i| rows[i].verification.clone()).unwrap_or_default(), + idx.map(|i| rows[i].verification.clone()) + .unwrap_or_default(), ); if let Some(i) = idx { row.keep_baseline_from(&rows[i]); } - if let Some(i) = idx { rows[i] = row; } else { rows.push(row); } + if let Some(i) = idx { + rows[i] = row; + } else { + rows.push(row); + } let _ = artifacts::append_journal(root, "check-changes", &rel, "missing on disk"); changed_rows = true; } @@ -690,32 +1121,59 @@ pub fn check_changes(root: &Path) -> Json { let live = file_sha256_prefixed(&path).unwrap_or_default(); let baseline = idx.and_then(|i| { let h = rows[i].source_hash.clone(); - if h.starts_with("sha256:") { Some(h) } else { None } + if h.starts_with("sha256:") { + Some(h) + } else { + None + } }); let drifted = match &baseline { Some(base) => live != *base, - None => cache::indexed_rows(root).into_iter().find(|r| r.0 == rel).map(|r| r.3 != live).unwrap_or(false), + None => cache::indexed_rows(root) + .into_iter() + .find(|r| r.0 == rel) + .map(|r| r.3 != live) + .unwrap_or(false), }; if !drifted { continue; } dirty.push(rel.clone()); - let already = idx.map(|i| is_yellow(&rows[i].status) && rows[i].reason.contains("content changed since last mark-green")).unwrap_or(false); + let already = idx + .map(|i| { + is_yellow(&rows[i].status) + && rows[i] + .reason + .contains("content changed since last mark-green") + }) + .unwrap_or(false); if !already { let keep_hash = baseline.unwrap_or(live); - let verification = idx.map(|i| rows[i].verification.clone()).unwrap_or_default(); - let mut row = HealthRow::new(rel.clone(), "🟡 Yellow", local_stamp(), reason, keep_hash, verification); + let verification = idx + .map(|i| rows[i].verification.clone()) + .unwrap_or_default(); + let mut row = HealthRow::new( + rel.clone(), + "🟡 Yellow", + local_stamp(), + reason, + keep_hash, + verification, + ); if let Some(i) = idx { row.keep_baseline_from(&rows[i]); } - if let Some(i) = idx { rows[i] = row; } else { rows.push(row); } + if let Some(i) = idx { + rows[i] = row; + } else { + rows.push(row); + } let _ = artifacts::append_journal(root, "check-changes", &rel, reason); let pending = root.join("pending_updates.md"); - let mut text = fs::read_to_string(&pending).unwrap_or_else(|_| "# Pending Updates\n\n".into()); - if !text.contains(&rel) { - text.push_str(&format!("- {rel} — {reason}\n")); - let _ = fs::write(&pending, text); - } + let text = fs::read_to_string(&pending).unwrap_or_default(); + let text = + crate::pending_line::append_item(&text, &rel, crate::pending_line::CHECK_MESSAGE); + let _ = fs::write(&pending, text); changed_rows = true; } } @@ -725,12 +1183,20 @@ pub fn check_changes(root: &Path) -> Json { let mut value = ok(vec![ ("dirty_total", Json::Int(dirty.len() as i64)), ("changes_detected", Json::Int(dirty.len() as i64)), - ("files", Json::Array(dirty.into_iter().map(Json::String).collect())), + ( + "files", + Json::Array(dirty.into_iter().map(Json::String).collect()), + ), ("scope", scope_json(&scope)), ]); if truncated { value.insert("truncated", Json::Bool(true)); - value.insert("warnings", Json::Array(vec![Json::String(format!("check-changes stopped at {cap} files (WIKIFIER_CHECK_CHANGES_MAX)"))])); + value.insert( + "warnings", + Json::Array(vec![Json::String(format!( + "check-changes stopped at {cap} files (WIKIFIER_CHECK_CHANGES_MAX)" + ))]), + ); } value } @@ -740,16 +1206,36 @@ pub fn prepare_edit(root: &Path, file: &str) -> Json { let _ = crate::legacy_cache::import_if_empty(root); let deps = cache::load_dependencies(root, rel).unwrap_or_default(); let importers = cache::load_dependents(root, rel).unwrap_or_default(); - let row = artifacts::read_health(root).into_iter().find(|r| r.file == rel); + let row = artifacts::read_health(root) + .into_iter() + .find(|r| r.file == rel); ok(vec![ ("file", Json::String(rel.into())), - ("status", Json::String(row.as_ref().map(|r| r.status.clone()).unwrap_or_default())), - ("dependencies", Json::Array(deps.into_iter().map(|path| Json::obj(vec![ - ("path", Json::String(path.clone())), - ("resolved_path", Json::String(path)), - ])).collect())), - ("dependents", Json::Array(importers.into_iter().map(Json::String).collect())), - ("wiki", Json::String(fs::read_to_string(root.join(format!("{rel}.wiki.md"))).unwrap_or_default())), + ( + "status", + Json::String(row.as_ref().map(|r| r.status.clone()).unwrap_or_default()), + ), + ( + "dependencies", + Json::Array( + deps.into_iter() + .map(|path| { + Json::obj(vec![ + ("path", Json::String(path.clone())), + ("resolved_path", Json::String(path)), + ]) + }) + .collect(), + ), + ), + ( + "dependents", + Json::Array(importers.into_iter().map(Json::String).collect()), + ), + ( + "wiki", + Json::String(crate::wiki_files::read_wiki(root, rel).1), + ), ]) } @@ -769,19 +1255,21 @@ pub fn record_change(root: &Path, file: &str, reason: &str) -> Json { }; let hash = file_sha256_prefixed(&path).unwrap_or_default(); let mut row = HealthRow::new(rel, "🟡 Yellow", written.stamp, reason, hash, ""); - if let Some(old) = artifacts::read_health(root).into_iter().find(|r| r.file == rel) { + if let Some(old) = artifacts::read_health(root) + .into_iter() + .find(|r| r.file == rel) + { row.keep_baseline_from(&old); } if let Err(e) = artifacts::upsert_health_row(root, row) { return fail("HealthError", e); } let pending = root.join("pending_updates.md"); - let mut text = fs::read_to_string(&pending).unwrap_or_else(|_| "# Pending Updates\n\n".into()); - if !text.contains(rel) { - text.push_str(&format!("- {rel} — {reason}\n")); - if fs::write(&pending, text).is_err() { - return fail("PendingError", "could not write pending_updates.md"); - } + let text = fs::read_to_string(&pending).unwrap_or_default(); + let text = + crate::pending_line::append_item(&text, rel, &crate::pending_line::record_message(reason)); + if fs::write(&pending, text).is_err() { + return fail("PendingError", "could not write pending_updates.md"); } ok(vec![ ("file", Json::String(rel.into())), @@ -801,25 +1289,42 @@ pub fn mark_green(root: &Path, file: &str, reason: &str, force: bool) -> Json { return fail("FileNotFound", format!("file does not exist: {rel}")); } if force && reason.trim().is_empty() { - return fail("ValueError", "force=True needs a reason explaining why the wiki is still accurate"); + return fail( + "ValueError", + "force=True needs a reason explaining why the wiki is still accurate", + ); } let symbols = public_symbols(&path); - let wiki_path = root.join(format!("{rel}.wiki.md")); - let wiki = fs::read_to_string(&wiki_path).unwrap_or_default(); - let previous = artifacts::read_health(root).into_iter().find(|r| r.file == rel); + let (found_wiki, wiki) = crate::wiki_files::read_wiki(root, rel); + let wiki_path = found_wiki.unwrap_or_else(|| root.join(format!("{rel}.wiki.md"))); + let previous = artifacts::read_health(root) + .into_iter() + .find(|r| r.file == rel); let verdict = wiki_verdict(&symbols, &wiki, previous.as_ref(), &wiki_path); if wiki.is_empty() { let _ = write_green_row(root, rel, &path, reason, "no_wiki", &symbols.names); - return attach_warning(ok(vec![("file", Json::String(rel.into())), ("verification", Json::String("no_wiki".into()))]), symbols.warning); + return attach_warning( + ok(vec![ + ("file", Json::String(rel.into())), + ("verification", Json::String("no_wiki".into())), + ]), + symbols.warning, + ); } if !verdict.issues.is_empty() && !force { - return attach_warning(Json::obj(vec![ - ("success", Json::Bool(false)), - ("error", Json::String("wiki misses public symbols".into())), - ("error_type", Json::String("WikiVerificationError".into())), - ("file", Json::String(rel.into())), - ("issues", Json::Array(verdict.issues.into_iter().map(Json::String).collect())), - ]), symbols.warning.or(verdict.warning)); + return attach_warning( + Json::obj(vec![ + ("success", Json::Bool(false)), + ("error", Json::String("wiki misses public symbols".into())), + ("error_type", Json::String("WikiVerificationError".into())), + ("file", Json::String(rel.into())), + ( + "issues", + Json::Array(verdict.issues.into_iter().map(Json::String).collect()), + ), + ]), + symbols.warning.or(verdict.warning), + ); } let verification = if !symbols.extractor { "unchecked" @@ -831,7 +1336,13 @@ pub fn mark_green(root: &Path, file: &str, reason: &str, force: bool) -> Json { if let Err(e) = write_green_row(root, rel, &path, reason, verification, &symbols.names) { return attach_warning(fail("HealthError", e), symbols.warning); } - attach_warning(ok(vec![("file", Json::String(rel.into())), ("verification", Json::String(verification.into()))]), symbols.warning.or(verdict.warning)) + attach_warning( + ok(vec![ + ("file", Json::String(rel.into())), + ("verification", Json::String(verification.into())), + ]), + symbols.warning.or(verdict.warning), + ) } struct SymbolScan { @@ -845,7 +1356,12 @@ struct WikiVerdict { warning: Option, } -fn wiki_verdict(symbols: &SymbolScan, wiki: &str, previous: Option<&HealthRow>, wiki_path: &Path) -> WikiVerdict { +fn wiki_verdict( + symbols: &SymbolScan, + wiki: &str, + previous: Option<&HealthRow>, + wiki_path: &Path, +) -> WikiVerdict { let mut issues = Vec::new(); let mut warning = None; if wiki.is_empty() || !symbols.extractor { @@ -856,8 +1372,11 @@ fn wiki_verdict(symbols: &SymbolScan, wiki: &str, previous: Option<&HealthRow>, issues.push(format!("undocumented_new_symbols: {sym}")); } } - let baseline: Vec = previous.map(|row| split_symbols(&row.public_symbols)).unwrap_or_default(); - let had_baseline = previous.is_some_and(|row| !row.public_symbols.is_empty() || !row.wiki_hash.is_empty()); + let baseline: Vec = previous + .map(|row| split_symbols(&row.public_symbols)) + .unwrap_or_default(); + let had_baseline = + previous.is_some_and(|row| !row.public_symbols.is_empty() || !row.wiki_hash.is_empty()); if !had_baseline { warning = Some("no_symbol_baseline".into()); } else { @@ -868,7 +1387,9 @@ fn wiki_verdict(symbols: &SymbolScan, wiki: &str, previous: Option<&HealthRow>, } let surface_changed = !same_symbols(&baseline, &symbols.names); let wiki_hash = file_sha256_prefixed(wiki_path).unwrap_or_default(); - let old_hash = previous.map(|row| row.wiki_hash.clone()).unwrap_or_default(); + let old_hash = previous + .map(|row| row.wiki_hash.clone()) + .unwrap_or_default(); if surface_changed && !old_hash.is_empty() && wiki_hash == old_hash { issues.push("wiki_not_updated_for_api_change".into()); } @@ -877,7 +1398,10 @@ fn wiki_verdict(symbols: &SymbolScan, wiki: &str, previous: Option<&HealthRow>, } fn split_symbols(text: &str) -> Vec { - text.split(['\n', ',']).map(|s| s.trim().to_string()).filter(|s| !s.is_empty()).collect() + text.split(['\n', ',']) + .map(|s| s.trim().to_string()) + .filter(|s| !s.is_empty()) + .collect() } fn same_symbols(left: &[String], right: &[String]) -> bool { @@ -890,19 +1414,41 @@ fn same_symbols(left: &[String], right: &[String]) -> bool { a == b } -fn write_green_row(root: &Path, rel: &str, path: &Path, reason: &str, verification: &str, symbols: &[String]) -> Result<(), String> { - let wiki_path = root.join(format!("{rel}.wiki.md")); +fn write_green_row( + root: &Path, + rel: &str, + path: &Path, + reason: &str, + verification: &str, + symbols: &[String], +) -> Result<(), String> { + let wiki_path = crate::wiki_files::find_wiki(root, rel) + .unwrap_or_else(|| root.join(format!("{rel}.wiki.md"))); let mut row = HealthRow::new( rel, "🟢 Green", local_stamp(), - if reason.is_empty() { if verification == "no_wiki" { "no wiki" } else { "mark-green" } } else { reason }, + if reason.is_empty() { + if verification == "no_wiki" { + "no wiki" + } else { + "mark-green" + } + } else { + reason + }, file_sha256_prefixed(path).unwrap_or_default(), verification, ); row.public_symbols = symbols.join("\n"); - row.wiki_hash = if wiki_path.is_file() { file_sha256_prefixed(&wiki_path).unwrap_or_default() } else { String::new() }; - row.source_mtime_ns = cache::stat_ns(path).map(|(mtime, _)| mtime.to_string()).unwrap_or_default(); + row.wiki_hash = if wiki_path.is_file() { + file_sha256_prefixed(&wiki_path).unwrap_or_default() + } else { + String::new() + }; + row.source_mtime_ns = cache::stat_ns(path) + .map(|(mtime, _)| mtime.to_string()) + .unwrap_or_default(); artifacts::upsert_health_row(root, row)?; drop_pending_line(root, rel); Ok(()) @@ -910,109 +1456,88 @@ fn write_green_row(root: &Path, rel: &str, path: &Path, reason: &str, verificati fn public_symbols(path: &Path) -> SymbolScan { let (text, warning) = crate::parse::read_utf8(path); - let ext = path.extension().and_then(|s| s.to_str()).unwrap_or("").to_ascii_lowercase(); - if ext == "py" { - return SymbolScan { names: python_public_symbols(&text), warning, extractor: true }; + let ext = path + .extension() + .and_then(|s| s.to_str()) + .unwrap_or("") + .to_ascii_lowercase(); + if ext == "py" || ext == "pyi" { + return SymbolScan { + names: crate::symbols::python_public_symbols(&text), + warning, + extractor: true, + }; } - if matches!(ext.as_str(), "js" | "ts" | "jsx" | "tsx" | "mjs" | "cjs" | "mts" | "cts") { - return SymbolScan { names: js_public_symbols(&text), warning, extractor: true }; + if matches!( + ext.as_str(), + "js" | "ts" | "jsx" | "tsx" | "mjs" | "cjs" | "mts" | "cts" + ) { + return SymbolScan { + names: crate::symbols::js_public_symbols(&text), + warning, + extractor: true, + }; } if ext == "rs" { - return SymbolScan { names: crate::parse::rust_public_symbols(&text), warning, extractor: true }; + return SymbolScan { + names: crate::parse::rust_public_symbols(&text), + warning, + extractor: true, + }; + } + SymbolScan { + names: Vec::new(), + warning, + extractor: false, } - SymbolScan { names: Vec::new(), warning, extractor: false } } -fn python_public_symbols(text: &str) -> Vec { - let mut symbols = Vec::new(); - for line in text.lines() { - let t = line.trim(); - if t.starts_with("__all__") { - if let Some(start) = t.find('[') { - if let Some(end) = t.find(']') { - for name in t[start + 1..end].split(',') { - let name = name.trim().trim_matches(|c| c == '\'' || c == '"'); - if !name.is_empty() { - symbols.push(name.to_string()); - } - } - return symbols; - } - } - } - } - for line in text.lines() { - if line.starts_with("def ") || line.starts_with("class ") || line.starts_with("async def ") { - let rest = line.trim_start_matches("async ").trim_start_matches("def ").trim_start_matches("class "); - let name = rest.split(['(', ':']).next().unwrap_or("").trim(); - if !name.is_empty() && !name.starts_with('_') { - symbols.push(name.to_string()); - } - } - } - symbols +pub fn health(root: &Path) -> Json { + health_view(root, None, "json") } -fn js_public_symbols(text: &str) -> Vec { - let mut symbols = Vec::new(); - for line in text.lines() { - let t = line.trim().trim_end_matches(';').trim(); - let Some(rest) = t.strip_prefix("export ") else { continue }; - let rest = rest.trim(); - if rest.starts_with("default ") { - push_symbol(&mut symbols, "default"); - continue; - } - if rest.starts_with('{') { - let head = rest.split(" from ").next().unwrap_or(rest); - let inner = head.trim().trim_start_matches('{').trim_end_matches('}'); - for part in inner.split(',') { - let part = part.trim(); - if part.is_empty() { - continue; - } - let name = part.split(" as ").last().unwrap_or(part).trim(); - push_symbol(&mut symbols, name); - } - continue; - } - if rest.starts_with('*') { - continue; - } - let rest = rest.trim_start_matches("declare ").trim(); - let rest = rest.strip_prefix("async ").unwrap_or(rest).trim(); - let rest = ["function ", "class ", "interface ", "type ", "enum ", "const ", "let ", "var "] - .iter() - .find_map(|prefix| rest.strip_prefix(prefix)) - .unwrap_or(""); - let name = rest.split([' ', '(', ':', '=', '<', '{']).next().unwrap_or("").trim(); - if !name.is_empty() && name != "from" { - push_symbol(&mut symbols, name); +pub fn health_view(root: &Path, directory: Option<&str>, format: &str) -> Json { + if format == "healing-stats" { + return healing_stats(root); + } + let mut rows = artifacts::read_health(root); + if let Some(dir) = directory + .map(|d| d.trim_matches('/')) + .filter(|d| !d.is_empty()) + { + rows.retain(|row| under_prefix(dir, &row.file)); + } + let summary = health_summary_of(&rows); + if format == "summary" { + let mut value = ok(vec![("health_summary", summary)]); + if let Some(dir) = directory { + value.insert("directory", Json::String(dir.into())); } + return value; } - symbols + ok(vec![ + ("health_summary", summary), + ( + "entries", + Json::Array(rows.into_iter().map(health_entry).collect()), + ), + ]) } -fn push_symbol(symbols: &mut Vec, name: &str) { - let name = name.trim().trim_matches(|c| c == '\'' || c == '"' || c == '`'); - if !name.is_empty() && !name.starts_with('_') && !symbols.iter().any(|have| have == name) { - symbols.push(name.to_string()); - } +fn health_entry(row: HealthRow) -> Json { + Json::obj(vec![ + ("file", Json::String(row.file)), + ("status", Json::String(row.status)), + ("last_updated", Json::String(row.last_updated)), + ("reason", Json::String(row.reason)), + ("source_content_hash", Json::String(row.source_hash)), + ("verification", Json::String(row.verification)), + ]) } -pub fn health(root: &Path) -> Json { - let rows = artifacts::read_health(root); - ok(vec![ - ("health_summary", health_summary(root)), - ("entries", Json::Array(rows.into_iter().map(|r| Json::obj(vec![ - ("file", Json::String(r.file)), - ("status", Json::String(r.status)), - ("last_updated", Json::String(r.last_updated)), - ("reason", Json::String(r.reason)), - ("source_content_hash", Json::String(r.source_hash)), - ("verification", Json::String(r.verification)), - ])).collect())), - ]) +fn under_prefix(prefix: &str, rel: &str) -> bool { + let prefix = prefix.trim_matches('/'); + prefix.is_empty() || rel == prefix || rel.starts_with(&format!("{prefix}/")) } pub fn read_health(root: &Path) -> Vec { @@ -1026,7 +1551,10 @@ pub fn read_journal(root: &Path, date: Option<&str>) -> Vec { pub fn cache_status(root: &Path) -> Json { let paths = cache::all_paths(root); ok(vec![ - ("cache_backend", Json::String(if paths.is_empty() { "none" } else { "wk_cache" }.into())), + ( + "cache_backend", + Json::String(if paths.is_empty() { "none" } else { "wk_cache" }.into()), + ), ("files", Json::Int(paths.len() as i64)), ("sha256", Json::Bool(true)), ]) @@ -1035,8 +1563,15 @@ pub fn cache_status(root: &Path) -> Json { pub fn validate(root: &Path) -> Json { let scope = scope::detect(root); let files = scope::collect(root, &scope).unwrap_or_default(); - let known: BTreeSet = artifacts::read_health(root).into_iter().map(|r| r.file).collect(); - let missing: Vec = files.iter().map(|p| scope::rel_of(root, p)).filter(|rel| !known.contains(rel)).collect(); + let known: BTreeSet = artifacts::read_health(root) + .into_iter() + .map(|r| r.file) + .collect(); + let missing: Vec = files + .iter() + .map(|p| scope::rel_of(root, p)) + .filter(|rel| !known.contains(rel)) + .collect(); let check = cache::check_consistency(root); // Size and mtime can stay put while the bytes change. Hashing here is the // check that `update-maps` skips; repair with `update-maps --full`. @@ -1056,23 +1591,42 @@ pub fn validate(root: &Path) -> Json { } } let mut value = if !check.consistent { - fail("CacheInconsistent", "forward and reverse cache indexes disagree") + fail( + "CacheInconsistent", + "forward and reverse cache indexes disagree", + ) } else if !drift.is_empty() { - fail("StaleMap", "cached hashes do not match file bytes; run update-maps --full") + fail( + "StaleMap", + "cached hashes do not match file bytes; run update-maps --full", + ) } else { ok(vec![]) }; - value.insert("missing", Json::Array(missing.iter().cloned().map(Json::String).collect())); + value.insert( + "missing", + Json::Array(missing.iter().cloned().map(Json::String).collect()), + ); value.insert("missing_count", Json::Int(missing.len() as i64)); value.insert("cache_consistent", Json::Bool(check.consistent)); value.insert("cache_incomplete", Json::Bool(check.incomplete)); - value.insert("cache_mismatches", Json::Array(check.mismatches.into_iter().map(Json::String).collect())); - value.insert("content_drift", Json::Array(drift.iter().cloned().map(Json::String).collect())); + value.insert( + "cache_mismatches", + Json::Array(check.mismatches.into_iter().map(Json::String).collect()), + ); + value.insert( + "content_drift", + Json::Array(drift.iter().cloned().map(Json::String).collect()), + ); value.insert("content_drift_count", Json::Int(drift.len() as i64)); value } pub fn cycles(root: &Path) -> Json { + cycles_view(root, true, None) +} + +pub fn cycles_view(root: &Path, analysis: bool, max_items: Option) -> Json { let _ = crate::legacy_cache::import_if_empty(root); let edges = cache::edges(root); let meta = edge_meta(root); @@ -1083,39 +1637,77 @@ pub fn cycles(root: &Path) -> Json { } let sccs = tarjan(&graph); let cyclic: Vec> = sccs.into_iter().filter(|c| c.len() > 1).collect(); - ok(vec![ + let value = ok(vec![ ("count", Json::Int(cyclic.len() as i64)), - ("cycles", Json::Array(cyclic.into_iter().map(|files| { - let set: BTreeSet = files.iter().cloned().collect(); - let mut weakest: Option<(f64, String, String)> = None; - for src in &files { - for dst in graph.get(src).into_iter().flatten() { - if !set.contains(dst) { - continue; - } - let score = meta.get(&(src.clone(), dst.clone())).map(|edge| edge.confidence()).unwrap_or(0.9); - if weakest.as_ref().map(|(have, _, _)| score < *have).unwrap_or(true) { - weakest = Some((score, src.clone(), dst.clone())); - } - } - } - let outside = edges.iter().filter(|(src, dst)| set.contains(src) && !set.contains(dst)).count() as i64; - let mut obj = vec![ - ("size".into(), Json::Int(files.len() as i64)), - ("files".into(), Json::Array(files.into_iter().map(Json::String).collect())), - ("external_edges".into(), Json::Int(outside)), - ]; - if let Some((score, src, dst)) = weakest { - obj.push(("break_hint".into(), Json::obj(vec![ - ("src", Json::String(src)), - ("dst", Json::String(dst)), - ("confidence", Json::Float(score)), - ("reason", Json::String("lowest-confidence edge inside the component".into())), - ]))); - } - Json::from_pairs(obj) - }).collect())), - ]) + ( + "cycles", + Json::Array( + cyclic + .into_iter() + .map(|files| { + let set: BTreeSet = files.iter().cloned().collect(); + let mut weakest: Option<(f64, String, String)> = None; + for src in &files { + for dst in graph.get(src).into_iter().flatten() { + if !set.contains(dst) { + continue; + } + let score = meta + .get(&(src.clone(), dst.clone())) + .map(|edge| edge.confidence()) + .unwrap_or(0.9); + if weakest + .as_ref() + .map(|(have, _, _)| score < *have) + .unwrap_or(true) + { + weakest = Some((score, src.clone(), dst.clone())); + } + } + } + let outside = edges + .iter() + .filter(|(src, dst)| set.contains(src) && !set.contains(dst)) + .count() as i64; + let mut obj = vec![ + ("size".into(), Json::Int(files.len() as i64)), + ( + "files".into(), + Json::Array(files.into_iter().map(Json::String).collect()), + ), + ("external_edges".into(), Json::Int(outside)), + ]; + if let Some((score, src, dst)) = weakest { + obj.push(( + "break_hint".into(), + Json::obj(vec![ + ("src", Json::String(src)), + ("dst", Json::String(dst)), + ("confidence", Json::Float(score)), + ( + "reason", + Json::String( + "lowest-confidence edge inside the component".into(), + ), + ), + ]), + )); + } + Json::from_pairs(obj) + }) + .collect(), + ), + ), + ]); + let listed: Vec<(String, String)> = edges.iter().cloned().collect(); + let mut value = crate::cycle_report::enrich(value, analysis, max_items); + if analysis { + value.insert( + "reverse_dependencies", + Json::Array(crate::cycle_report::rank_dependents(&listed)), + ); + } + value } fn edge_meta(root: &Path) -> BTreeMap<(String, String), cache::InternalEdge> { @@ -1168,7 +1760,13 @@ impl Tarjan<'_> { self.index += 1; self.stack.push(node.to_string()); self.on.insert(node.to_string()); - let next: Vec = self.graph.get(node).into_iter().flatten().cloned().collect(); + let next: Vec = self + .graph + .get(node) + .into_iter() + .flatten() + .cloned() + .collect(); for nxt in next { if !self.idx.contains_key(&nxt) { self.strong(&nxt); @@ -1217,14 +1815,17 @@ fn tarjan(graph: &BTreeMap>) -> Vec> { } pub fn verify_wikis(root: &Path) -> Json { - verify_wikis_under(root, None) + verify_wikis_under(root, None, None) } -fn verify_wikis_under(root: &Path, prefix: Option<&str>) -> Json { +fn verify_wikis_under(root: &Path, prefix: Option<&str>, limit: Option) -> Json { let rows = artifacts::read_health(root); - let prefix = prefix.map(|p| p.trim_matches('/')).filter(|p| !p.is_empty()); + let prefix = prefix + .map(|p| p.trim_matches('/')) + .filter(|p| !p.is_empty()); let mut checked = 0i64; let mut failed = Vec::new(); + let mut failing_total = 0i64; let mut warnings = Vec::new(); for row in rows { if !is_green(&row.status) { @@ -1240,8 +1841,8 @@ fn verify_wikis_under(root: &Path, prefix: Option<&str>) -> Json { continue; } checked += 1; - let wiki_path = root.join(format!("{}.wiki.md", row.file)); - let wiki = fs::read_to_string(&wiki_path).unwrap_or_default(); + let (found_wiki, wiki) = crate::wiki_files::read_wiki(root, &row.file); + let wiki_path = found_wiki.unwrap_or_else(|| root.join(format!("{}.wiki.md", row.file))); if wiki.is_empty() { continue; } @@ -1254,22 +1855,37 @@ fn verify_wikis_under(root: &Path, prefix: Option<&str>) -> Json { warnings.push(format!("{}: {warning}", row.file)); } if !verdict.issues.is_empty() { - failed.push(Json::obj(vec![ - ("file", Json::String(row.file)), - ("issues", Json::Array(verdict.issues.into_iter().map(Json::String).collect())), - ])); + failing_total += 1; + if limit.map(|max| failed.len() < max).unwrap_or(true) { + failed.push(Json::obj(vec![ + ("file", Json::String(row.file)), + ( + "issues", + Json::Array(verdict.issues.into_iter().map(Json::String).collect()), + ), + ])); + } } } let warning_json = Json::Array(warnings.into_iter().map(Json::String).collect()); - if failed.is_empty() { - ok(vec![("checked", Json::Int(checked)), ("failed", Json::Array(vec![])), ("warnings", warning_json)]) + if failing_total == 0 { + ok(vec![ + ("checked", Json::Int(checked)), + ("failed", Json::Array(vec![])), + ("failing_count", Json::Int(0)), + ("warnings", warning_json), + ]) } else { Json::obj(vec![ ("success", Json::Bool(false)), - ("error", Json::String("one or more Green wikis miss public symbols".into())), + ( + "error", + Json::String("one or more Green wikis miss public symbols".into()), + ), ("error_type", Json::String("WikiVerificationError".into())), ("checked", Json::Int(checked)), ("failed", Json::Array(failed)), + ("failing_count", Json::Int(failing_total)), ("warnings", warning_json), ]) } @@ -1300,7 +1916,13 @@ pub fn record_changes_opts( Ok(changes) => changes, Err(err) => return fail("GitUnavailableError", err), }; - let reasons: Vec<(String, String)> = reason_lines.iter().filter_map(|line| line.split_once('=').map(|(p, w)| (p.to_string(), w.to_string()))).collect(); + let reasons: Vec<(String, String)> = reason_lines + .iter() + .filter_map(|line| { + line.split_once('=') + .map(|(p, w)| (p.to_string(), w.to_string())) + }) + .collect(); let mut selected = Vec::new(); for change in changes { if artifact_path(&change.path) { @@ -1314,21 +1936,48 @@ pub fn record_changes_opts( let mut missing = Vec::new(); let mut planned: Vec<(GitChange, String)> = Vec::new(); for change in &selected { - let why = reason_for(&change.path, &reasons).or_else(|| default_reason.map(|s| s.to_string())); + let why = + reason_for(&change.path, &reasons).or_else(|| default_reason.map(|s| s.to_string())); match why { Some(why) => planned.push((change.clone(), why)), None => missing.push(change.path.clone()), } } - let unused: Vec = reasons.iter().filter(|(key, _)| !selected.iter().any(|change| reason_key_matches(key, &change.path))).map(|(key, _)| key.clone()).collect(); + let unused: Vec = reasons + .iter() + .filter(|(key, _)| { + !selected + .iter() + .any(|change| reason_key_matches(key, &change.path)) + }) + .map(|(key, _)| key.clone()) + .collect(); if !missing.is_empty() { return Json::obj(vec![ ("success", Json::Bool(false)), - ("error", Json::String("every changed file needs a reason".into())), + ( + "error", + Json::String("every changed file needs a reason".into()), + ), ("error_type", Json::String("MissingReasons".into())), - ("missing_reasons", Json::Array(missing.iter().cloned().map(Json::String).collect())), - ("template", Json::String(missing.iter().map(|f| format!("{f}=WHY")).collect::>().join("\n"))), - ("unused_reasons", Json::Array(unused.into_iter().map(Json::String).collect())), + ( + "missing_reasons", + Json::Array(missing.iter().cloned().map(Json::String).collect()), + ), + ( + "template", + Json::String( + missing + .iter() + .map(|f| format!("{f}=WHY")) + .collect::>() + .join("\n"), + ), + ), + ( + "unused_reasons", + Json::Array(unused.into_iter().map(Json::String).collect()), + ), ("recorded", Json::Int(0)), ]); } @@ -1336,8 +1985,19 @@ pub fn record_changes_opts( return ok(vec![ ("dry_run", Json::Bool(true)), ("recorded", Json::Int(planned.len() as i64)), - ("files", Json::Array(planned.iter().map(|(c, _)| Json::String(c.path.clone())).collect())), - ("unused_reasons", Json::Array(unused.into_iter().map(Json::String).collect())), + ( + "files", + Json::Array( + planned + .iter() + .map(|(c, _)| Json::String(c.path.clone())) + .collect(), + ), + ), + ( + "unused_reasons", + Json::Array(unused.into_iter().map(Json::String).collect()), + ), ]); } let _lock = _lock; @@ -1358,8 +2018,20 @@ pub fn record_changes_opts( if marked.get("success").and_then(|s| s.as_bool()) != Some(true) { verification_failed.push(Json::obj(vec![ ("file", Json::String(change.path.clone())), - ("error", Json::String(marked.get("error").and_then(|s| s.as_str()).unwrap_or("wiki verification failed").into())), - ("issues", marked.get("issues").cloned().unwrap_or(Json::Array(vec![]))), + ( + "error", + Json::String( + marked + .get("error") + .and_then(|s| s.as_str()) + .unwrap_or("wiki verification failed") + .into(), + ), + ), + ( + "issues", + marked.get("issues").cloned().unwrap_or(Json::Array(vec![])), + ), ])); } } @@ -1367,9 +2039,15 @@ pub fn record_changes_opts( } ok(vec![ ("recorded", Json::Int(recorded)), - ("files", Json::Array(files.into_iter().map(Json::String).collect())), + ( + "files", + Json::Array(files.into_iter().map(Json::String).collect()), + ), ("verification_failed", Json::Array(verification_failed)), - ("unused_reasons", Json::Array(unused.into_iter().map(Json::String).collect())), + ( + "unused_reasons", + Json::Array(unused.into_iter().map(Json::String).collect()), + ), ("green", Json::Bool(green)), ]) } @@ -1383,23 +2061,35 @@ struct GitChange { fn git_changes(root: &Path, since: Option<&str>) -> Result, String> { let root_s = root.display().to_string(); if let Some(since) = since { - let diff = std::process::Command::new("git").args(["-C", &root_s, "diff", "--name-status", since]).output().map_err(|e| e.to_string())?; + let diff = std::process::Command::new("git") + .args(["-C", &root_s, "diff", "--name-status", since]) + .output() + .map_err(|e| e.to_string())?; if !diff.status.success() { return Err("git is not available or this directory is not a repository".into()); } let mut out = parse_name_status(&String::from_utf8_lossy(&diff.stdout)); - let untracked = std::process::Command::new("git").args(["-C", &root_s, "ls-files", "--others", "--exclude-standard"]).output().map_err(|e| e.to_string())?; + let untracked = std::process::Command::new("git") + .args(["-C", &root_s, "ls-files", "--others", "--exclude-standard"]) + .output() + .map_err(|e| e.to_string())?; if untracked.status.success() { for line in String::from_utf8_lossy(&untracked.stdout).lines() { let path = line.trim(); if !path.is_empty() { - out.push(GitChange { path: path.to_string(), deleted: false }); + out.push(GitChange { + path: path.to_string(), + deleted: false, + }); } } } return Ok(out); } - let status = std::process::Command::new("git").args(["-C", &root_s, "status", "--porcelain", "-uall"]).output().map_err(|e| e.to_string())?; + let status = std::process::Command::new("git") + .args(["-C", &root_s, "status", "--porcelain", "-uall"]) + .output() + .map_err(|e| e.to_string())?; if !status.status.success() { return Err("git is not available or this directory is not a repository".into()); } @@ -1415,12 +2105,21 @@ fn parse_porcelain(text: &str) -> Vec { let code = &line[..2]; let path = line[3..].trim().trim_matches('"'); if let Some((old, new)) = path.split_once(" -> ") { - out.push(GitChange { path: old.trim().trim_matches('"').to_string(), deleted: true }); - out.push(GitChange { path: new.trim().trim_matches('"').to_string(), deleted: false }); + out.push(GitChange { + path: old.trim().trim_matches('"').to_string(), + deleted: true, + }); + out.push(GitChange { + path: new.trim().trim_matches('"').to_string(), + deleted: false, + }); continue; } let deleted = code.contains('D'); - out.push(GitChange { path: path.to_string(), deleted }); + out.push(GitChange { + path: path.to_string(), + deleted, + }); } out } @@ -1436,13 +2135,22 @@ fn parse_name_status(text: &str) -> Vec { } if code.starts_with('R') || code.starts_with('C') { let new = parts.next().unwrap_or("").trim(); - out.push(GitChange { path: path.to_string(), deleted: true }); + out.push(GitChange { + path: path.to_string(), + deleted: true, + }); if !new.is_empty() { - out.push(GitChange { path: new.to_string(), deleted: false }); + out.push(GitChange { + path: new.to_string(), + deleted: false, + }); } continue; } - out.push(GitChange { path: path.to_string(), deleted: code.starts_with('D') }); + out.push(GitChange { + path: path.to_string(), + deleted: code.starts_with('D'), + }); } out } @@ -1470,7 +2178,10 @@ fn reason_for(path: &str, reasons: &[(String, String)]) -> Option { } let mut best: Option<(usize, &str)> = None; for (key, why) in reasons { - if key.ends_with('/') && (path.starts_with(key.as_str()) || path == key.trim_end_matches('/')) && best.map(|(len, _)| key.len() > len).unwrap_or(true) { + if key.ends_with('/') + && (path.starts_with(key.as_str()) || path == key.trim_end_matches('/')) + && best.map(|(len, _)| key.len() > len).unwrap_or(true) + { best = Some((key.len(), why)); } } @@ -1479,7 +2190,10 @@ fn reason_for(path: &str, reasons: &[(String, String)]) -> Option { } let mut glob_best: Option<(usize, &str)> = None; for (key, why) in reasons { - if (key.contains('*') || key.contains('?')) && glob_match(key, path) && glob_best.map(|(len, _)| key.len() > len).unwrap_or(true) { + if (key.contains('*') || key.contains('?')) + && glob_match(key, path) + && glob_best.map(|(len, _)| key.len() > len).unwrap_or(true) + { glob_best = Some((key.len(), why)); } } @@ -1522,8 +2236,22 @@ pub fn record_deletion(root: &Path, file: &str, reason: &str) -> Json { Ok(written) => written, Err(e) => return fail("JournalError", e), }; - let mut row = HealthRow::new(rel, "🔴 Red", written.stamp.clone(), if reason.is_empty() { "recorded deletion" } else { reason }, "", "deleted"); - if let Some(old) = artifacts::read_health(root).into_iter().find(|r| r.file == rel) { + let mut row = HealthRow::new( + rel, + "🔴 Red", + written.stamp.clone(), + if reason.is_empty() { + "recorded deletion" + } else { + reason + }, + "", + "deleted", + ); + if let Some(old) = artifacts::read_health(root) + .into_iter() + .find(|r| r.file == rel) + { row.source_hash = old.source_hash.clone(); row.keep_baseline_from(&old); } @@ -1551,10 +2279,7 @@ fn drop_pending_line(root: &Path, rel: &str) { } let mut out = String::new(); for line in text.lines() { - let hit = line.trim().strip_prefix("- ").is_some_and(|rest| { - let file = rest.split(" — ").next().unwrap_or(rest).split(" - ").next().unwrap_or(rest).trim(); - file == rel - }); + let hit = crate::pending_line::file_of(line) == Some(rel); if !hit { out.push_str(line); out.push('\n'); @@ -1574,7 +2299,7 @@ pub fn heal_stubs(root: &Path, min_len: usize, dry_run: bool) -> Json { if row.reason != "Initial stub" { continue; } - let wiki = fs::read_to_string(root.join(format!("{}.wiki.md", row.file))).unwrap_or_default(); + let wiki = crate::wiki_files::read_wiki(root, &row.file).1; if wiki.trim().len() >= min_len { if !dry_run { row.status = "🟢 Green".into(); @@ -1610,15 +2335,18 @@ pub fn prune_health_opts(root: &Path, deleted_missing: bool) -> Json { let watched = watched_files(root); let rows = artifacts::read_health(root); let before = rows.len(); - let kept: Vec<_> = rows.into_iter().filter(|row| { - if !kept_path(&scope, &watched, &row.file) { - return false; - } - if deleted_missing && !root.join(&row.file).is_file() { - return false; - } - true - }).collect(); + let kept: Vec<_> = rows + .into_iter() + .filter(|row| { + if !kept_path(&scope, &watched, &row.file) { + return false; + } + if deleted_missing && !root.join(&row.file).is_file() { + return false; + } + true + }) + .collect(); let changed = before - kept.len(); let _ = artifacts::write_health(root, &kept); if deleted_missing { @@ -1626,10 +2354,8 @@ pub fn prune_health_opts(root: &Path, deleted_missing: bool) -> Json { let text = fs::read_to_string(&pending).unwrap_or_default(); let mut out = String::new(); for line in text.lines() { - let drop_line = line.trim().strip_prefix("- ").is_some_and(|rest| { - let file = rest.split(" — ").next().unwrap_or(rest).trim(); - !root.join(file).is_file() - }); + let drop_line = + crate::pending_line::file_of(line).is_some_and(|file| !root.join(file).is_file()); if !drop_line { out.push_str(line); out.push('\n'); @@ -1637,7 +2363,10 @@ pub fn prune_health_opts(root: &Path, deleted_missing: bool) -> Json { } let _ = fs::write(pending, out); } - ok(vec![("changed", Json::Int(changed as i64)), ("deleted_missing", Json::Bool(deleted_missing))]) + ok(vec![ + ("changed", Json::Int(changed as i64)), + ("deleted_missing", Json::Bool(deleted_missing)), + ]) } pub fn prune_pending(root: &Path) -> Json { @@ -1652,10 +2381,16 @@ pub fn prune_pending(root: &Path) -> Json { let mut out = String::new(); let mut changed = 0i64; for line in text.lines() { - let keep = if let Some(rest) = line.trim().strip_prefix("- ") { - let file = rest.split(" — ").next().unwrap_or(rest).trim(); - if kept_path(&scope, &watched, file) { true } else { changed += 1; false } - } else { true }; + let keep = if let Some(file) = crate::pending_line::file_of(line) { + if kept_path(&scope, &watched, file) { + true + } else { + changed += 1; + false + } + } else { + true + }; if keep { out.push_str(line); out.push('\n'); @@ -1679,26 +2414,45 @@ fn kept_path(scope: &Scope, watched: &[String], rel: &str) -> bool { if scope.full_tree || scope.prefixes.is_empty() { return true; } - scope.prefixes.iter().any(|prefix| rel == prefix || rel.starts_with(&format!("{prefix}/"))) + scope + .prefixes + .iter() + .any(|prefix| rel == prefix || rel.starts_with(&format!("{prefix}/"))) } pub fn stale_wikis(root: &Path) -> Json { + stale_wikis_in(root, None, None) +} + +pub fn stale_wikis_in(root: &Path, directory: Option<&str>, limit: Option) -> Json { let mut stale = Vec::new(); for row in artifacts::read_health(root) { - let wiki = root.join(format!("{}.wiki.md", row.file)); - if !wiki.is_file() { + if let Some(dir) = directory + .map(|d| d.trim_matches('/')) + .filter(|d| !d.is_empty()) + { + if !under_prefix(dir, &row.file) { + continue; + } + } + let wiki = crate::wiki_files::find_wiki(root, &row.file); + if wiki.is_none() { continue; } let live = file_sha256_prefixed(&root.join(&row.file)).unwrap_or_default(); - let source_drift = row.source_hash.starts_with("sha256:") && !live.is_empty() && live != row.source_hash; - let content_flag = is_yellow(&row.status) && row.reason.contains("content changed since last mark-green"); + let source_drift = + row.source_hash.starts_with("sha256:") && !live.is_empty() && live != row.source_hash; + let content_flag = + is_yellow(&row.status) && row.reason.contains("content changed since last mark-green"); if source_drift || content_flag { let mut reasons = Vec::new(); if source_drift { reasons.push(Json::String("source_changed_since_mark_green".into())); } if content_flag { - reasons.push(Json::String("content changed since last mark-green (check-changes)".into())); + reasons.push(Json::String( + "content changed since last mark-green (check-changes)".into(), + )); } stale.push(Json::obj(vec![ ("file", Json::String(row.file)), @@ -1708,10 +2462,22 @@ pub fn stale_wikis(root: &Path) -> Json { ])); } } + let total = stale.len() as i64; + if let Some(max) = limit { + stale.truncate(max); + } ok(vec![ ("stale", Json::Array(stale.clone())), - ("files", Json::Array(stale.iter().filter_map(|v| v.get("file").cloned()).collect())), - ("count", Json::Int(stale.len() as i64)), + ( + "files", + Json::Array( + stale + .iter() + .filter_map(|v| v.get("file").cloned()) + .collect(), + ), + ), + ("count", Json::Int(total)), ]) } @@ -1756,7 +2522,10 @@ pub fn metrics_snapshot(root: &Path) -> Json { ("green", Json::Int(green)), ("yellow", Json::Int(yellow)), ("red", Json::Int(red)), - ("cache_files", Json::Int(cache::all_paths(root).len() as i64)), + ( + "cache_files", + Json::Int(cache::all_paths(root).len() as i64), + ), ]); let staging = root.join(".wikifier_staging"); if fs::create_dir_all(&staging).is_err() { @@ -1803,7 +2572,9 @@ pub fn resolution_diagnostics(root: &Path) -> Json { let mut counts: BTreeMap = BTreeMap::new(); let mut low = Vec::new(); for rel in cache::all_paths(root) { - let Ok(view) = cache::load_dependency_view(root, &rel) else { continue }; + let Ok(view) = cache::load_dependency_view(root, &rel) else { + continue; + }; for name in &view.external { *counts.entry("external_or_bare".into()).or_insert(0) += 1; notes.push(Json::obj(vec![ @@ -1845,13 +2616,19 @@ pub fn resolution_diagnostics(root: &Path) -> Json { } ok(vec![ ("notes", Json::Array(notes)), - ("reason_code_counts", Json::from_pairs(counts.into_iter().map(|(k, n)| (k, Json::Int(n))).collect())), + ( + "reason_code_counts", + Json::from_pairs(counts.into_iter().map(|(k, n)| (k, Json::Int(n))).collect()), + ), ("actionable_low_conf_edges", Json::Array(low)), ]) } pub fn barrel_reports(root: &Path) -> Json { - let dir = root.join(".wikifier_staging").join("wk_cache").join("barrels"); + let dir = root + .join(".wikifier_staging") + .join("wk_cache") + .join("barrels"); let mut barrels = Vec::new(); if let Ok(entries) = fs::read_dir(&dir) { for ent in entries.flatten() { @@ -1864,7 +2641,10 @@ pub fn barrel_reports(root: &Path) -> Json { } } barrels.sort_by(|a, b| { - a.get("src").and_then(|s| s.as_str()).unwrap_or("").cmp(b.get("src").and_then(|s| s.as_str()).unwrap_or("")) + a.get("src") + .and_then(|s| s.as_str()) + .unwrap_or("") + .cmp(b.get("src").and_then(|s| s.as_str()).unwrap_or("")) }); ok(vec![ ("count", Json::Int(barrels.len() as i64)), @@ -1875,7 +2655,10 @@ pub fn barrel_reports(root: &Path) -> Json { pub fn incremental_status(root: &Path) -> Json { let mut status = crate::maps_out::read_status(root); if status.get("map_complete").is_none() { - status = ok(vec![("map_complete", Json::Bool(!cache::all_paths(root).is_empty()))]); + status = ok(vec![( + "map_complete", + Json::Bool(!cache::all_paths(root).is_empty()), + )]); } else if status.get("success").is_none() { status.insert("success", Json::Bool(true)); } @@ -1967,10 +2750,19 @@ pub fn serve_loop(root: &Path, port: u16) -> Json { } } let _ = fs::remove_file(root.join(".wikifier_staging/serve.port")); - ok(vec![("bound", Json::Bool(true)), ("port", Json::Int(bound as i64)), ("served", Json::Bool(true))]) + ok(vec![ + ("bound", Json::Bool(true)), + ("port", Json::Int(bound as i64)), + ("served", Json::Bool(true)), + ]) } -fn write_http(sock: &mut std::net::TcpStream, status: &str, content_type: &str, body: &str) -> std::io::Result<()> { +fn write_http( + sock: &mut std::net::TcpStream, + status: &str, + content_type: &str, + body: &str, +) -> std::io::Result<()> { use std::io::Write; let resp = format!( "HTTP/1.1 {status}\r\nContent-Type: {content_type}\r\nContent-Length: {}\r\nConnection: close\r\n\r\n", @@ -1990,30 +2782,55 @@ fn request_line(req: &str) -> (&str, &str, &str) { } fn dashboard_run(root: &Path, req: &str) -> String { - let body = req.split_once("\r\n\r\n").map(|(_, rest)| rest).unwrap_or(""); + let body = req + .split_once("\r\n\r\n") + .map(|(_, rest)| rest) + .unwrap_or(""); let cmd = crate::jsonutil::parse(body) .ok() - .and_then(|value| value.get("cmd").and_then(|item| item.as_str()).map(|text| text.to_string())) + .and_then(|value| { + value + .get("cmd") + .and_then(|item| item.as_str()) + .map(|text| text.to_string()) + }) .unwrap_or_default(); let started = std::time::Instant::now(); let result = match cmd.as_str() { "update-maps" => update_maps(root), - "update-maps --full" => update_maps_run(root, &MapRun { full: true, quiet: true, directory: None, max_files: None, stream: false }), + "update-maps --full" => update_maps_run( + root, + &MapRun { + full: true, + quiet: true, + directory: None, + max_files: None, + stream: false, + }, + ), "check-changes" => check_changes(root), _ => fail("ValueError", "command is not available from the dashboard"), }; - let succeeded = result.get("success").and_then(|flag| flag.as_bool()).unwrap_or(false); + let succeeded = result + .get("success") + .and_then(|flag| flag.as_bool()) + .unwrap_or(false); let error = if succeeded { String::new() } else { - result.get("error").and_then(|item| item.as_str()).unwrap_or("failed").to_string() + result + .get("error") + .and_then(|item| item.as_str()) + .unwrap_or("failed") + .to_string() }; Json::obj(vec![ ("ok", Json::Bool(succeeded)), ("output", Json::String(result.to_string())), ("error", Json::String(error)), ("duration_s", Json::Float(started.elapsed().as_secs_f64())), - ]).to_string() + ]) + .to_string() } fn serve_response(root: &Path, req: &str) -> (&'static str, &'static str, String) { @@ -2022,36 +2839,79 @@ fn serve_response(root: &Path, req: &str) -> (&'static str, &'static str, String let method = parts.next().unwrap_or(""); let target = parts.next().unwrap_or("/"); if method != "GET" && method != "HEAD" { - return ("405 Method Not Allowed", "text/plain; charset=utf-8", "method not allowed\n".into()); + return ( + "405 Method Not Allowed", + "text/plain; charset=utf-8", + "method not allowed\n".into(), + ); } let (path, query) = target.split_once('?').unwrap_or((target, "")); if path.contains("..") || query.contains("..") { - return ("400 Bad Request", "application/json", fail("PathError", "path traversal refused").to_string()); - } - let file_arg = query_param(query, "path").or_else(|| query_param(query, "file")).unwrap_or_default(); + return ( + "400 Bad Request", + "application/json", + fail("PathError", "path traversal refused").to_string(), + ); + } + let file_arg = query_param(query, "path") + .or_else(|| query_param(query, "file")) + .unwrap_or_default(); if file_arg.contains("..") { - return ("400 Bad Request", "application/json", fail("PathError", "path traversal refused").to_string()); + return ( + "400 Bad Request", + "application/json", + fail("PathError", "path traversal refused").to_string(), + ); } match path { "/" | "/index.html" => ("200 OK", "text/html; charset=utf-8", dashboard_html(root)), - "/__wikifier/api/bootstrap" => ("200 OK", "application/json", session_bootstrap(root).to_string()), - "/__wikifier/api/dependencies" => ("200 OK", "application/json", load_dependencies(root, &file_arg).to_string()), - "/__wikifier/api/dependents" => ("200 OK", "application/json", load_dependents(root, &file_arg).to_string()), + "/__wikifier/api/bootstrap" => ( + "200 OK", + "application/json", + session_bootstrap(root).to_string(), + ), + "/__wikifier/api/dependencies" => ( + "200 OK", + "application/json", + load_dependencies(root, &file_arg).to_string(), + ), + "/__wikifier/api/dependents" => ( + "200 OK", + "application/json", + load_dependents(root, &file_arg).to_string(), + ), "/__wikifier/api/cycles" => ("200 OK", "application/json", cycles(root).to_string()), "/__wikifier/api/journal" => { let entries = read_journal(root, None); - let body = ok(vec![("entries", Json::Array(entries.into_iter().map(|e| Json::obj(vec![ - ("stamp", Json::String(e.stamp)), - ("action", Json::String(e.action)), - ("file", Json::String(e.file)), - ("reason", Json::String(e.reason)), - ])).collect()))]); + let body = ok(vec![( + "entries", + Json::Array( + entries + .into_iter() + .map(|e| { + Json::obj(vec![ + ("stamp", Json::String(e.stamp)), + ("action", Json::String(e.action)), + ("file", Json::String(e.file)), + ("reason", Json::String(e.reason)), + ]) + }) + .collect(), + ), + )]); ("200 OK", "application/json", body.to_string()) } - "/__wikifier/api/diagnostics" => ("200 OK", "application/json", resolution_diagnostics(root).to_string()), + "/__wikifier/api/diagnostics" => ( + "200 OK", + "application/json", + resolution_diagnostics(root).to_string(), + ), "/__wikifier/api/file" => serve_file(root, &file_arg), "/__wikifier/status" => { - let project = root.file_name().and_then(|name| name.to_str()).unwrap_or("project"); + let project = root + .file_name() + .and_then(|name| name.to_str()) + .unwrap_or("project"); let body = ok(vec![ ("wikifier_serve", Json::Bool(true)), ("version", Json::String(VERSION.into())), @@ -2067,7 +2927,11 @@ fn serve_response(root: &Path, req: &str) -> (&'static str, &'static str, String fn serve_project_file(root: &Path, path: &str) -> (&'static str, &'static str, String) { let rel = path.trim_start_matches('/'); if rel.is_empty() || rel.contains("..") || rel.split('/').any(|part| part == ".git") { - return ("404 Not Found", "application/json", fail("NotFound", "unknown path").to_string()); + return ( + "404 Not Found", + "application/json", + fail("NotFound", "unknown path").to_string(), + ); } let (status, _, body) = serve_file(root, rel); if !status.starts_with("200") { @@ -2087,21 +2951,41 @@ fn serve_project_file(root: &Path, path: &str) -> (&'static str, &'static str, S fn serve_file(root: &Path, rel: &str) -> (&'static str, &'static str, String) { if rel.is_empty() { - return ("400 Bad Request", "application/json", fail("PathError", "missing path").to_string()); + return ( + "400 Bad Request", + "application/json", + fail("PathError", "missing path").to_string(), + ); } let path = root.join(rel); let Ok(canon_root) = root.canonicalize() else { - return ("400 Bad Request", "application/json", fail("PathError", "root is not readable").to_string()); + return ( + "400 Bad Request", + "application/json", + fail("PathError", "root is not readable").to_string(), + ); }; let Ok(canon) = path.canonicalize() else { - return ("404 Not Found", "application/json", fail("FileNotFound", format!("file does not exist: {rel}")).to_string()); + return ( + "404 Not Found", + "application/json", + fail("FileNotFound", format!("file does not exist: {rel}")).to_string(), + ); }; if !canon.starts_with(&canon_root) { - return ("400 Bad Request", "application/json", fail("PathError", "path is outside the project").to_string()); + return ( + "400 Bad Request", + "application/json", + fail("PathError", "path is outside the project").to_string(), + ); } match fs::read_to_string(&canon) { Ok(text) => ("200 OK", "text/plain; charset=utf-8", text), - Err(err) => ("404 Not Found", "application/json", fail("FileNotFound", err.to_string()).to_string()), + Err(err) => ( + "404 Not Found", + "application/json", + fail("FileNotFound", err.to_string()).to_string(), + ), } } @@ -2148,7 +3032,13 @@ pub fn monitor_loop(root: &Path, interval_s: u64) -> Json { let once = std::env::var("WIKIFIER_MONITOR_ONCE").ok().as_deref() == Some("1"); loop { let report = check_changes(root); - let dirty = report.get("dirty_total").and_then(|v| match v { Json::Int(n) => Some(*n), _ => None }).unwrap_or(0); + let dirty = report + .get("dirty_total") + .and_then(|v| match v { + Json::Int(n) => Some(*n), + _ => None, + }) + .unwrap_or(0); eprintln!("monitor dirty={dirty}"); if once { return report; @@ -2159,64 +3049,214 @@ pub fn monitor_loop(root: &Path, interval_s: u64) -> Json { } pub fn daemon_ctl(root: &Path, action: &str, force: bool) -> Json { - let pidfile = root.join(".wikifier_staging").join("daemon.pid"); + daemon_command(root, action, force, 100, false) +} + +pub fn daemon_command( + root: &Path, + action: &str, + force: bool, + log_lines: usize, + follow: bool, +) -> Json { + let pidfile = crate::daemon_svc::pid_path(root); match action { - "run" => { - for _ in 0..50 { - if pidfile.is_file() { - break; - } - std::thread::sleep(std::time::Duration::from_millis(20)); - } - let maps = std::env::var("WIKIFIER_DAEMON_MAPS").ok().as_deref() != Some("0"); - while pidfile.is_file() { - let _ = check_changes(root); - if maps { - let _ = update_maps(root); - } - for _ in 0..10 { - if !pidfile.is_file() { - break; - } - std::thread::sleep(std::time::Duration::from_millis(200)); - } - } - ok(vec![("running", Json::Bool(false)), ("action", Json::String("run".into()))]) - } - "start" => { - let scope = scope::detect(root); - let ready = readiness_of(&scope, !cache::all_paths(root).is_empty(), &artifacts::read_health(root), health_file_present(root)); - if (ready == "blocked" || ready == "not_ready") && !force { - return fail("NotReady", format!("readiness is {ready}; pass --force to start the daemon anyway")); - } - let _ = fs::create_dir_all(pidfile.parent().unwrap()); + "run" => daemon_run(root, &pidfile), + "start" => daemon_start(root, &pidfile, force), + "restart" => { + let _ = daemon_stop(root, &pidfile); + daemon_start(root, &pidfile, force) + } + "status" => daemon_status(root, &pidfile), + "stop" => daemon_stop(root, &pidfile), + "logs" => daemon_logs(root, log_lines, follow), + "install-service" | "install_service" => { let exe = match std::env::current_exe() { - Ok(p) => p, - Err(e) => return fail("DaemonError", e.to_string()), + Ok(path) => path, + Err(err) => return fail("DaemonError", err.to_string()), }; - match spawn_daemon(&exe, root) { - Ok(child) => { - let _ = fs::write(&pidfile, child.id().to_string()); - ok(vec![("running", Json::Bool(true)), ("pid", Json::Int(child.id() as i64)), ("action", Json::String("start".into()))]) - } - Err(e) => fail("DaemonError", e.to_string()), + crate::daemon_svc::install(root, &exe) + } + "uninstall-service" | "uninstall_service" => crate::daemon_svc::uninstall(root), + other => fail("DaemonError", format!("unknown daemon action {other}")), + } +} + +fn daemon_run(root: &Path, pidfile: &Path) -> Json { + for _ in 0..50 { + if pidfile.is_file() || crate::daemon_svc::legacy_pid_path(root).is_file() { + break; + } + std::thread::sleep(std::time::Duration::from_millis(20)); + } + let poll = crate::daemon_svc::poll_seconds(); + let maps_every = crate::daemon_svc::maps_interval_seconds(); + let maps = crate::daemon_svc::maps_enabled(); + crate::daemon_svc::append_log( + root, + &format!("daemon started (check every {poll}s; maps every {maps_every}s)"), + ); + let checked = check_changes(root); + let ok_check = checked + .get("success") + .and_then(|v| v.as_bool()) + .unwrap_or(false); + crate::daemon_svc::write_heartbeat(root, "check-changes:start", ok_check, ""); + let mut last_maps = std::time::Instant::now(); + if maps { + let mapped = update_maps(root); + let ok_maps = mapped + .get("success") + .and_then(|v| v.as_bool()) + .unwrap_or(false); + crate::daemon_svc::write_heartbeat(root, "update-maps:start", ok_maps, ""); + crate::daemon_svc::append_log(root, "update-maps (start)"); + last_maps = std::time::Instant::now(); + } + while pidfile.is_file() { + let slices = (poll * 5).max(1); + for _ in 0..slices { + if !pidfile.is_file() { + break; } + std::thread::sleep(std::time::Duration::from_millis(200)); } - "status" => { - let running = pid_alive(&pidfile); - ok(vec![("running", Json::Bool(running)), ("action", Json::String("status".into()))]) + if !pidfile.is_file() { + break; } - "stop" => { - if let Ok(text) = fs::read_to_string(&pidfile) { - if let Ok(pid) = text.trim().parse::() { - let _ = crate::proc::signal_process(pid, 15); + let checked = check_changes(root); + let ok_check = checked + .get("success") + .and_then(|v| v.as_bool()) + .unwrap_or(false); + crate::daemon_svc::write_heartbeat(root, "check-changes:periodic", ok_check, ""); + if maps && maps_every > 0 && last_maps.elapsed().as_secs() >= maps_every { + let mapped = update_maps(root); + let ok_maps = mapped + .get("success") + .and_then(|v| v.as_bool()) + .unwrap_or(false); + crate::daemon_svc::write_heartbeat(root, "update-maps:interval", ok_maps, ""); + crate::daemon_svc::append_log(root, "update-maps (interval)"); + last_maps = std::time::Instant::now(); + } + } + crate::daemon_svc::append_log(root, "daemon stopped"); + ok(vec![ + ("running", Json::Bool(false)), + ("action", Json::String("run".into())), + ]) +} + +fn daemon_start(root: &Path, pidfile: &Path, force: bool) -> Json { + if let Some(pid) = running_pid(root) { + return ok(vec![ + ("running", Json::Bool(true)), + ("pid", Json::Int(pid)), + ("action", Json::String("start".into())), + ("message", Json::String("already running".into())), + ]); + } + let scope = scope::detect(root); + let ready = readiness_of( + &scope, + !cache::all_paths(root).is_empty(), + &artifacts::read_health(root), + health_file_present(root), + ); + if ready == "blocked" && !force { + return fail( + "NotReady", + format!("readiness is {ready}; pass --force to start the daemon anyway"), + ); + } + let _ = fs::create_dir_all(pidfile.parent().unwrap()); + let exe = match std::env::current_exe() { + Ok(path) => path, + Err(err) => return fail("DaemonError", err.to_string()), + }; + match spawn_daemon(&exe, root) { + Ok(child) => { + let _ = fs::write(pidfile, child.id().to_string()); + ok(vec![ + ("running", Json::Bool(true)), + ("pid", Json::Int(child.id() as i64)), + ("action", Json::String("start".into())), + ]) + } + Err(err) => fail("DaemonError", err.to_string()), + } +} + +fn daemon_status(root: &Path, pidfile: &Path) -> Json { + let pid = running_pid(root); + ok(vec![ + ("running", Json::Bool(pid.is_some())), + ("pid", pid.map(Json::Int).unwrap_or(Json::Null)), + ("action", Json::String("status".into())), + ("pid_file", Json::String(pidfile.display().to_string())), + ( + "log", + Json::String(crate::daemon_svc::log_path(root).display().to_string()), + ), + ]) +} + +fn daemon_stop(root: &Path, pidfile: &Path) -> Json { + for path in [ + pidfile.to_path_buf(), + crate::daemon_svc::legacy_pid_path(root), + ] { + if let Ok(text) = fs::read_to_string(&path) { + if let Ok(pid) = text.trim().parse::() { + let _ = crate::proc::signal_process(pid, 15); + } + } + let _ = fs::remove_file(path); + } + ok(vec![ + ("running", Json::Bool(false)), + ("action", Json::String("stop".into())), + ]) +} + +fn daemon_logs(root: &Path, lines: usize, follow: bool) -> Json { + if follow { + let log = crate::daemon_svc::log_path(root); + let mut seen = fs::read_to_string(&log).map(|text| text.len()).unwrap_or(0); + while running_pid(root).is_some() { + std::thread::sleep(std::time::Duration::from_millis(400)); + if let Ok(text) = fs::read_to_string(&log) { + if text.len() > seen { + eprint!("{}", &text[seen..]); + seen = text.len(); } } - let _ = fs::remove_file(&pidfile); - ok(vec![("running", Json::Bool(false)), ("action", Json::String("stop".into()))]) } - other => fail("DaemonError", format!("unknown daemon action {other}")), } + let text = crate::daemon_svc::read_log_tail(root, lines.max(1)); + ok(vec![ + ("action", Json::String("logs".into())), + ("log", Json::String(text)), + ( + "path", + Json::String(crate::daemon_svc::log_path(root).display().to_string()), + ), + ]) +} + +fn running_pid(root: &Path) -> Option { + for path in [ + crate::daemon_svc::pid_path(root), + crate::daemon_svc::legacy_pid_path(root), + ] { + if pid_alive(&path) { + return fs::read_to_string(path) + .ok() + .and_then(|text| text.trim().parse().ok()); + } + } + None } fn spawn_daemon(exe: &Path, root: &Path) -> std::io::Result { @@ -2261,94 +3301,298 @@ fn prevent_stdio_inheritance() { } fn pid_alive(pidfile: &Path) -> bool { - let Ok(text) = fs::read_to_string(pidfile) else { return false }; - let Ok(pid) = text.trim().parse::() else { return false }; + let Ok(text) = fs::read_to_string(pidfile) else { + return false; + }; + let Ok(pid) = text.trim().parse::() else { + return false; + }; crate::proc::pid_alive(pid) } -pub fn dispatch(name: &str, root: &Path, positional: &[String], flags: &BTreeMap) -> Json { +pub fn dispatch( + name: &str, + root: &Path, + positional: &[String], + flags: &BTreeMap, +) -> Json { let file = positional.first().map(|s| s.as_str()).unwrap_or(""); - let reason = if positional.len() > 1 { positional[1..].join(" ") } else { flags.get("reason").cloned().unwrap_or_default() }; + let reason = if positional.len() > 1 { + positional[1..].join(" ") + } else { + flags.get("reason").cloned().unwrap_or_default() + }; match name { "check-changes" | "check_changes" => check_changes(root), "record-change" | "record_change" => record_change(root, file, &reason), - "mark-green" | "mark_green" => mark_green(root, file, &reason, flags.get("force").map(|s| s == "true").unwrap_or(false)), + "mark-green" | "mark_green" => mark_green( + root, + file, + &reason, + flags.get("force").map(|s| s == "true").unwrap_or(false), + ), "record-deletion" | "record_deletion" => record_deletion(root, file, &reason), - "session-bootstrap" | "bootstrap" | "session-start" | "session_bootstrap" => session_bootstrap(root), - "suggest-next" | "suggest" | "suggest-next-actions" | "suggest_next_actions" => suggest_next_actions(root), + "session-bootstrap" | "bootstrap" | "session-start" | "session_bootstrap" => { + session_bootstrap_scoped(root, flag_text(flags, "directory")) + } + "suggest-next" | "suggest" | "suggest-next-actions" | "suggest_next_actions" => { + suggest_next_scoped(root, flag_text(flags, "directory")) + } "prepare-edit" | "prepare_edit" | "lookup" | "preflight" => prepare_edit(root, file), - "dependencies" | "deps" | "get_dependencies" => dependency_report(root, file, flag_on(flags, "full")), + "dependencies" | "deps" | "get_dependencies" => { + let low_only = + flag_on(flags, "low-confidence-only") || flag_on(flags, "low_confidence_only"); + let unresolved_only = + flag_on(flags, "unresolved-only") || flag_on(flags, "unresolved_only"); + let full = flag_on(flags, "full") + || flags.get("detail").map(|s| s == "full").unwrap_or(false) + || low_only + || unresolved_only; + let mut report = dependency_report(root, file, full); + filter_dependency_report(&mut report, low_only, unresolved_only); + report + } "dependents" | "importers" | "get_dependents" => load_dependents(root, file), - "list-paths" | "list_paths" => list_paths(root, flags.get("prefix").map(|s| s.as_str()).unwrap_or(file)), + "list-paths" | "list_paths" => { + let depth = if flag_on(flags, "recursive") { + 0 + } else { + flags.get("depth").and_then(|s| s.parse().ok()).unwrap_or(1) + }; + let max_files = flags + .get("max-files") + .or_else(|| flags.get("max_files")) + .and_then(|s| s.parse().ok()); + list_paths_at( + root, + flags.get("prefix").map(|s| s.as_str()).unwrap_or(file), + depth, + max_files, + ) + } "update-maps" | "update_maps" => { let full = flag_on(flags, "full"); let quiet = flag_on(flags, "quiet"); let directory = flags.get("directory").cloned().filter(|s| s != "true"); - let max_files = flags.get("max-files").or_else(|| flags.get("max_files")).and_then(|s| s.parse().ok()); + let max_files = flags + .get("max-files") + .or_else(|| flags.get("max_files")) + .and_then(|s| s.parse().ok()); let stream = flag_on(flags, "stream"); - update_maps_run(root, &MapRun { full, quiet, directory, max_files, stream }) + update_maps_run( + root, + &MapRun { + full, + quiet, + directory, + max_files, + stream, + }, + ) + } + "health" => { + let format = if flag_on(flags, "summary") { + "summary" + } else { + flags.get("format").map(|s| s.as_str()).unwrap_or("json") + }; + health_view(root, flag_text(flags, "directory"), format) } - "health" => health(root), "cache-status" | "cache" => cache_status(root), "list-core-tools" | "list_core_tools" => ok(vec![ - ("core", Json::Array(["session_bootstrap", "check_changes", "prepare_edit", "suggest_next_actions", "record_change", "mark_green", "update_maps", "list_paths"].into_iter().map(|s| Json::String(s.into())).collect())), - ("tools", Json::Array(MCP_TOOLS.iter().map(|s| Json::String((*s).into())).collect())), + ( + "core", + Json::Array( + [ + "session_bootstrap", + "check_changes", + "prepare_edit", + "suggest_next_actions", + "record_change", + "mark_green", + "update_maps", + "list_paths", + ] + .into_iter() + .map(|s| Json::String(s.into())) + .collect(), + ), + ), + ( + "tools", + Json::Array( + MCP_TOOLS + .iter() + .map(|s| Json::String((*s).into())) + .collect(), + ), + ), ]), "journal" => { - let entries = read_journal(root, flags.get("date").map(|s| s.as_str()).or(positional.first().map(|s| s.as_str()))); - ok(vec![("entries", Json::Array(entries.into_iter().map(|e| Json::obj(vec![ - ("stamp", Json::String(e.stamp)), - ("action", Json::String(e.action)), - ("file", Json::String(e.file)), - ("reason", Json::String(e.reason)), - ])).collect()))]) + let date = flags + .get("date") + .map(|s| s.as_str()) + .or(positional.first().map(|s| s.as_str())); + let date_text = date + .map(|s| s.to_string()) + .unwrap_or_else(|| crate::timeutil::local_stamp().chars().take(10).collect()); + let entries = read_journal(root, date); + let mut entries: Vec<_> = entries + .into_iter() + .map(|e| { + Json::obj(vec![ + ("stamp", Json::String(e.stamp)), + ("action", Json::String(e.action)), + ("file", Json::String(e.file)), + ("reason", Json::String(e.reason)), + ]) + }) + .collect(); + if let Some(limit) = flags.get("limit").and_then(|s| s.parse::().ok()) { + entries.truncate(limit); + } + ok(vec![ + ("date", Json::String(date_text)), + ("entries", Json::Array(entries)), + ]) } "why-file" | "why_file" | "why" => { - let entries = artifacts::read_journal(root, None).into_iter().filter(|e| e.file == file).collect::>(); - ok(vec![("file", Json::String(file.into())), ("entries", Json::Array(entries.into_iter().map(|e| Json::obj(vec![ - ("stamp", Json::String(e.stamp)), - ("reason", Json::String(e.reason)), - ])).collect()))]) + let entries = artifacts::read_journal(root, None) + .into_iter() + .filter(|e| e.file == file) + .collect::>(); + ok(vec![ + ("file", Json::String(file.into())), + ( + "entries", + Json::Array( + entries + .into_iter() + .map(|e| { + Json::obj(vec![ + ("stamp", Json::String(e.stamp)), + ("reason", Json::String(e.reason)), + ]) + }) + .collect(), + ), + ), + ]) } "search-journal" | "search_journal" => { - let q = flags.get("query").cloned().or_else(|| positional.first().cloned()).unwrap_or_default(); - let file_filter = flags.get("file").map(|s| s.as_str()).filter(|s| !s.is_empty() && *s != "true"); - let entries = artifacts::read_journal(root, None).into_iter().filter(|e| { - let text_ok = q.is_empty() || e.reason.contains(&q) || e.file.contains(&q); - let file_ok = file_filter.map(|f| e.file == f).unwrap_or(true); - text_ok && file_ok - }).collect::>(); - ok(vec![("entries", Json::Array(entries.into_iter().map(|e| Json::obj(vec![ - ("file", Json::String(e.file)), - ("reason", Json::String(e.reason)), - ])).collect()))]) + let q = flags + .get("query") + .cloned() + .or_else(|| positional.first().cloned()) + .unwrap_or_default(); + let file_filter = flags + .get("file") + .map(|s| s.as_str()) + .filter(|s| !s.is_empty() && *s != "true"); + let mut entries = artifacts::read_journal(root, None) + .into_iter() + .filter(|e| { + let text_ok = q.is_empty() || e.reason.contains(&q) || e.file.contains(&q); + let file_ok = file_filter.map(|f| e.file == f).unwrap_or(true); + text_ok && file_ok + }) + .collect::>(); + if let Some(limit) = flags.get("limit").and_then(|s| s.parse::().ok()) { + entries.truncate(limit); + } + ok(vec![( + "entries", + Json::Array( + entries + .into_iter() + .map(|e| { + Json::obj(vec![ + ("file", Json::String(e.file)), + ("reason", Json::String(e.reason)), + ]) + }) + .collect(), + ), + )]) } "validate" => validate(root), "rebuild-cache" | "rebuild_cache" => match cache::rebuild(root) { Ok(()) => ok(vec![("rebuilt", Json::Bool(true))]), Err(err) => fail("CacheError", err), }, - "issues" => suggest_next_actions(root), - "cycles" | "get_cycles" => cycles(root), + "issues" => suggest_next_scoped(root, flag_text(flags, "directory")), + "cycles" | "get_cycles" => { + let max_items = flags + .get("max-items") + .or_else(|| flags.get("max_items")) + .and_then(|s| s.parse().ok()); + cycles_view(root, !flag_on(flags, "no-analysis"), max_items) + } "verify-wikis" | "verify_wikis" => { - let prefix = positional.first().map(|s| s.as_str()).filter(|s| !s.is_empty()); - verify_wikis_under(root, prefix) + let prefix = flag_text(flags, "directory") + .map(|s| s.to_string()) + .or_else(|| positional.first().filter(|s| !s.is_empty()).cloned()); + let limit = flags.get("limit").and_then(|s| s.parse().ok()); + verify_wikis_under(root, prefix.as_deref(), limit) } "autonomous-status" | "readiness" => { let scope = scope::detect(root); - ok(vec![("readiness", Json::String(readiness_of(&scope, !cache::all_paths(root).is_empty(), &artifacts::read_health(root), health_file_present(root)).into())), ("scope", scope_json(&scope))]) + ok(vec![ + ( + "readiness", + Json::String( + readiness_of( + &scope, + !cache::all_paths(root).is_empty(), + &artifacts::read_health(root), + health_file_present(root), + ) + .into(), + ), + ), + ("scope", scope_json(&scope)), + ]) + } + "init" => init_project(root, flag_on(flags, "no-copy")), + "get_project_status" => session_bootstrap_scoped(root, flag_text(flags, "directory")), + "get_current_project_root" => ok(vec![( + "project_root", + Json::String(root.display().to_string()), + )]), + "get_file_wiki" => { + let (path, wiki) = crate::wiki_files::read_wiki(root, file); + ok(vec![ + ("file", Json::String(file.into())), + ("wiki", Json::String(wiki)), + ( + "wiki_file", + Json::String(path.map(|p| p.display().to_string()).unwrap_or_default()), + ), + ]) } - "init" => init_project(root), - "get_project_status" => session_bootstrap(root), - "get_current_project_root" => ok(vec![("project_root", Json::String(root.display().to_string()))]), - "get_file_wiki" => ok(vec![("file", Json::String(file.into())), ("wiki", Json::String(fs::read_to_string(root.join(format!("{file}.wiki.md"))).unwrap_or_default()))]), "get_files_needing_attention" => suggest_next_actions(root), "get_resolution_diagnostics" => resolution_diagnostics(root), "get_barrel_reports" => barrel_reports(root), "get_incremental_status" => incremental_status(root), "record-changes" | "record_changes" => { - let lines = flags.get("reasons").map(|s| s.split('\n').map(|x| x.to_string()).filter(|x| !x.is_empty()).collect::>()).unwrap_or_default(); - let only = flags.get("only").map(|s| s.split('\n').map(|x| x.to_string()).filter(|x| !x.is_empty()).collect::>()).unwrap_or_default(); + let lines = flags + .get("reasons") + .map(|s| { + s.split('\n') + .map(|x| x.to_string()) + .filter(|x| !x.is_empty()) + .collect::>() + }) + .unwrap_or_default(); + let only = flags + .get("only") + .map(|s| { + s.split('\n') + .map(|x| x.to_string()) + .filter(|x| !x.is_empty()) + .collect::>() + }) + .unwrap_or_default(); record_changes_opts( root, &lines, @@ -2358,106 +3602,307 @@ pub fn dispatch(name: &str, root: &Path, positional: &[String], flags: &BTreeMap flag_on(flags, "green"), flag_on(flags, "dry-run"), ) + } + "seed-health" => match seed_health_from_map(root) { + Ok(()) => ok(vec![("seeded", Json::Bool(true))]), + Err(err) => fail("HealthError", err), }, - "seed-health" | "seed-source-hashes" | "seed_source_content_hashes" => { - match seed_health_from_map(root) { - Ok(()) => ok(vec![("seeded", Json::Bool(true))]), - Err(err) => fail("HealthError", err), - } + "seed-source-hashes" | "seed_source_content_hashes" => { + seed_source_hashes(root, flag_on(flags, "force"), flag_text(flags, "directory")) } "prune-pending" => prune_pending(root), "prune-health" => prune_health_opts(root, flag_on(flags, "deleted-missing")), "heal-stubs" | "heal_stubs" => heal_stubs( root, - flags.get("min-wiki-length").and_then(|s| s.parse().ok()).unwrap_or(20), + flags + .get("min-wiki-length") + .or_else(|| flags.get("min_wiki_length")) + .and_then(|s| s.parse().ok()) + .unwrap_or(350), flag_on(flags, "dry-run"), ), "healable-stubs" | "list_healable_stubs" => { - let min = flags.get("min-wiki-length").and_then(|s| s.parse().ok()).unwrap_or(20usize); - let files: Vec = artifacts::read_health(root).into_iter().filter(|r| r.reason == "Initial stub" && fs::read_to_string(root.join(format!("{}.wiki.md", r.file))).unwrap_or_default().len() >= min).map(|r| Json::String(r.file)).collect(); + let min = flags + .get("min-wiki-length") + .or_else(|| flags.get("min_wiki_length")) + .and_then(|s| s.parse().ok()) + .unwrap_or(350usize); + let files: Vec = artifacts::read_health(root) + .into_iter() + .filter(|r| { + r.reason == "Initial stub" + && crate::wiki_files::read_wiki(root, &r.file).1.trim().len() >= min + }) + .map(|r| Json::String(r.file)) + .collect(); ok(vec![("files", Json::Array(files))]) } "healing-stats" | "stub-stats" => healing_stats(root), - "stale-wikis" | "detect-stale" => stale_wikis(root), + "stale-wikis" | "detect-stale" => stale_wikis_in( + root, + flag_text(flags, "directory").or(positional.first().map(|s| s.as_str())), + flags.get("limit").and_then(|s| s.parse().ok()), + ), "prune-barrels" | "prune-brc" | "gc-barrels" => { - let days = positional.first().and_then(|s| s.parse().ok()) - .or_else(|| flags.get("max-age-days").or_else(|| flags.get("max_age_days")).and_then(|s| s.parse().ok())) + let days = positional + .first() + .and_then(|s| s.parse().ok()) + .or_else(|| { + flags + .get("max-age-days") + .or_else(|| flags.get("max_age_days")) + .and_then(|s| s.parse().ok()) + }) .unwrap_or(90.0); let dry = flags.get("dry-run").map(|s| s == "true").unwrap_or(false); prune_barrels(root, days, dry) } "metrics-snapshot" | "metrics" => metrics_snapshot(root), "monitor" => { - let interval = flags.get("interval").and_then(|s| s.parse().ok()).unwrap_or(30); + let interval = flags + .get("interval") + .and_then(|s| s.parse().ok()) + .unwrap_or(30); monitor_loop(root, interval) } "serve" => { - let port = positional.first().and_then(|s| s.parse().ok()) + let port = positional + .first() + .and_then(|s| s.parse().ok()) .or_else(|| flags.get("port").and_then(|s| s.parse().ok())) .unwrap_or(8787); serve_loop(root, port) } - "daemon" => daemon_ctl(root, positional.first().map(|s| s.as_str()).unwrap_or("status"), flag_on(flags, "force")), - "help" => ok(vec![("commands", Json::Array(CLI_COMMANDS.iter().map(|s| Json::String((*s).into())).collect()))]), + "daemon" => { + let follow = + flag_on(flags, "follow") || positional.iter().any(|s| s == "-f" || s == "--follow"); + let lines = flags + .get("lines") + .and_then(|s| s.parse().ok()) + .unwrap_or(100); + daemon_command( + root, + positional.first().map(|s| s.as_str()).unwrap_or("status"), + flag_on(flags, "force"), + lines, + follow, + ) + } + "help" => ok(vec![( + "commands", + Json::Array( + CLI_COMMANDS + .iter() + .map(|s| Json::String((*s).into())) + .collect(), + ), + )]), other => fail("UnknownCommand", format!("unknown command {other}")), } } pub const CLI_COMMANDS: &[&str] = &[ - "check-changes", "record-change", "mark-green", "record-changes", "record-deletion", - "session-bootstrap", "suggest-next", "prepare-edit", "dependencies", "dependents", - "why-file", "search-journal", "list-paths", "cycles", "journal", "issues", "update-maps", - "health", "validate", "rebuild-cache", "seed-health", "seed-source-hashes", "prune-pending", "prune-health", - "heal-stubs", "healable-stubs", "healing-stats", "stale-wikis", "verify-wikis", "prune-barrels", - "cache-status", "autonomous-status", "metrics-snapshot", "list-core-tools", "init", "monitor", - "serve", "daemon", "help", "mcp", + "check-changes", + "record-change", + "mark-green", + "record-changes", + "record-deletion", + "session-bootstrap", + "suggest-next", + "prepare-edit", + "dependencies", + "dependents", + "why-file", + "search-journal", + "list-paths", + "cycles", + "journal", + "issues", + "update-maps", + "health", + "validate", + "rebuild-cache", + "seed-health", + "seed-source-hashes", + "prune-pending", + "prune-health", + "heal-stubs", + "healable-stubs", + "healing-stats", + "stale-wikis", + "verify-wikis", + "prune-barrels", + "cache-status", + "autonomous-status", + "metrics-snapshot", + "list-core-tools", + "init", + "monitor", + "serve", + "daemon", + "help", + "mcp", ]; fn flag_on(flags: &BTreeMap, key: &str) -> bool { - flags.get(key).map(|s| s == "true" || s == "1").unwrap_or(false) + flags + .get(key) + .map(|s| s == "true" || s == "1") + .unwrap_or(false) +} + +fn flag_text<'a>(flags: &'a BTreeMap, key: &str) -> Option<&'a str> { + flags + .get(key) + .map(|s| s.as_str()) + .filter(|s| !s.is_empty() && *s != "true") +} + +fn filter_dependency_report(value: &mut Json, low_only: bool, unresolved_only: bool) { + if !low_only && !unresolved_only { + return; + } + if unresolved_only && !low_only { + value.insert("internal", Json::Array(vec![])); + value.insert("external", Json::Array(vec![])); + return; + } + if let Some(Json::Array(items)) = value.get("internal").cloned() { + let kept: Vec = items + .into_iter() + .filter(|edge| { + let score = match edge.get("confidence") { + Some(Json::Float(n)) => *n, + Some(Json::Int(n)) => *n as f64, + _ => 1.0, + }; + let dynamic = edge + .get("dynamic") + .and_then(|v| v.as_bool()) + .unwrap_or(false); + !dynamic && score < 0.65 + }) + .collect(); + value.insert("internal", Json::Array(kept)); + } + if unresolved_only { + value.insert("external", Json::Array(vec![])); + } } fn path_list_has_entries(root: &Path, name: &str) -> bool { !scope::read_path_list(root, name).is_empty() } -fn init_project(root: &Path) -> Json { +fn init_project(root: &Path, no_copy: bool) -> Json { let _lock = match lock_or_fail(root) { Ok(lock) => lock, Err(err) => return err, }; - let src = root.join("src"); - let mut seeded = Vec::new(); - if !path_list_has_entries(root, "map_paths.txt") && src.is_dir() { - if fs::write(root.join("map_paths.txt"), "src\n").is_err() { - return fail("InitError", "could not write map_paths.txt"); + let mut created: Vec = Vec::new(); + let write_new = |rel: &str, text: &str, created: &mut Vec| -> bool { + let path = root.join(rel); + if path.exists() { + return true; + } + if let Some(parent) = path.parent() { + if fs::create_dir_all(parent).is_err() { + return false; + } } - seeded.push("map_paths.txt"); - } - if !path_list_has_entries(root, "monitored_paths.txt") && src.is_dir() { - if fs::write(root.join("monitored_paths.txt"), "src\n").is_err() { - return fail("InitError", "could not write monitored_paths.txt"); + if fs::write(&path, text).is_err() { + return false; } - seeded.push("monitored_paths.txt"); + created.push(rel.to_string()); + true + }; + let map_body = "\ +# map_paths.txt — import-map package roots for update-maps +# One directory per line. No bare \".\". +src/ +"; + let monitored_body = "\ +# monitored_paths.txt — wiki and health watch list +# Mapped sources are watched even when they are not listed here. +src/ +"; + let exclude_body = "\ +node_modules +.git +build +dist +__pycache__ +*.pyc +target +"; + let health_body = "\ +# Documentation Health Matrix + +| File | Status | Last Updated | Reason / Intent | +|------|--------|--------------|-----------------| +"; + let pending_body = + "# Pending Updates\n\n(no pending items — run check-changes after making edits)\n"; + let library_body = "# Library & Imports Map\n\nRun `wikifier update-maps` to populate.\n"; + if !path_list_has_entries(root, "map_paths.txt") + && !write_new("map_paths.txt", map_body, &mut created) + { + return fail("InitError", "could not write map_paths.txt"); + } + if !path_list_has_entries(root, "monitored_paths.txt") + && !write_new("monitored_paths.txt", monitored_body, &mut created) + { + return fail("InitError", "could not write monitored_paths.txt"); + } + if !write_new("exclude_patterns.txt", exclude_body, &mut created) { + return fail("InitError", "could not write exclude_patterns.txt"); + } + if !write_new("file_health.md", health_body, &mut created) { + return fail("InitError", "could not write file_health.md"); + } + if !write_new("pending_updates.md", pending_body, &mut created) { + return fail("InitError", "could not write pending_updates.md"); + } + if !write_new("library.md", library_body, &mut created) { + return fail("InitError", "could not write library.md"); + } + let config = format!("project_root={}\n", root.display()); + if !write_new(".wikifier/config", &config, &mut created) { + return fail("InitError", "could not write .wikifier/config"); } - if !path_list_has_entries(root, "map_paths.txt") && !src.is_dir() { - return fail("ScopeError", "no src/ directory and map_paths.txt has no roots; name map_paths before update-maps"); + for dir in ["journal", "Logged_issues", ".wikifier_staging"] { + let path = root.join(dir); + if !path.is_dir() { + if fs::create_dir_all(&path).is_err() { + return fail("InitError", format!("could not create {dir}")); + } + created.push(dir.to_string()); + } } let index = root.join("index.html"); let mut copied = false; - if !index.is_file() { + if !no_copy && !index.is_file() { if fs::write(&index, INDEX_HTML).is_err() { return fail("InitError", "could not write index.html"); } copied = true; + created.push("index.html".to_string()); } let marker = root.join(".wikifier"); let _ = fs::create_dir_all(&marker); let _ = fs::write(marker.join("marker"), format!("wikifier {VERSION}\n")); + let created_json = Json::Array(created.iter().cloned().map(Json::String).collect()); + let seeded_text = if created.is_empty() { + "kept".to_string() + } else { + created.join(",") + }; ok(vec![ - ("seeded", Json::String(if seeded.is_empty() { "kept".into() } else { seeded.join(",") })), + ("seeded", Json::String(seeded_text)), + ("created", created_json), ("index_html", Json::Bool(index.is_file())), ("index_copied", Json::Bool(copied)), + ("no_copy", Json::Bool(no_copy)), ("marker", Json::String(".wikifier".into())), ]) } diff --git a/src/artifacts.rs b/src/artifacts.rs index d4f447e..0356a6a 100644 --- a/src/artifacts.rs +++ b/src/artifacts.rs @@ -75,14 +75,28 @@ fn read_health_json(path: &Path) -> Option> { let mut row = HealthRow::new( file.clone(), ent.get("status").and_then(|s| s.as_str()).unwrap_or(""), - ent.get("last_updated").and_then(|s| s.as_str()).unwrap_or(""), + ent.get("last_updated") + .and_then(|s| s.as_str()) + .unwrap_or(""), ent.get("reason").and_then(|s| s.as_str()).unwrap_or(""), - ent.get("source_content_hash").and_then(|s| s.as_str()).unwrap_or(""), - ent.get("verification").and_then(|s| s.as_str()).unwrap_or(""), + ent.get("source_content_hash") + .and_then(|s| s.as_str()) + .unwrap_or(""), + ent.get("verification") + .and_then(|s| s.as_str()) + .unwrap_or(""), ); row.public_symbols = symbols_field(ent.get("public_symbols")); - row.wiki_hash = ent.get("wiki_hash").and_then(|s| s.as_str()).unwrap_or("").to_string(); - row.source_mtime_ns = ent.get("source_mtime_ns").and_then(|s| s.as_str()).unwrap_or("").to_string(); + row.wiki_hash = ent + .get("wiki_hash") + .and_then(|s| s.as_str()) + .unwrap_or("") + .to_string(); + row.source_mtime_ns = ent + .get("source_mtime_ns") + .and_then(|s| s.as_str()) + .unwrap_or("") + .to_string(); rows.push(row); } rows.sort_by(|a, b| a.file.cmp(&b.file)); @@ -97,11 +111,17 @@ fn read_health_md(path: &Path) -> Vec { if !line.starts_with('|') || line.contains("---") || line.contains("File |") { continue; } - let cells: Vec<&str> = line.trim_matches('|').split('|').map(|c| c.trim()).collect(); + let cells: Vec<&str> = line + .trim_matches('|') + .split('|') + .map(|c| c.trim()) + .collect(); if cells.len() < 4 || cells[0] == "File" { continue; } - rows.push(HealthRow::new(cells[0], cells[1], cells[2], cells[3], "", "")); + rows.push(HealthRow::new( + cells[0], cells[1], cells[2], cells[3], "", "", + )); } rows } @@ -111,13 +131,22 @@ pub fn write_health(root: &Path, rows: &[HealthRow]) -> Result<(), String> { for row in rows { let mut obj = Vec::new(); obj.push(("status".into(), Json::String(row.status.clone()))); - obj.push(("last_updated".into(), Json::String(row.last_updated.clone()))); + obj.push(( + "last_updated".into(), + Json::String(row.last_updated.clone()), + )); obj.push(("reason".into(), Json::String(row.reason.clone()))); if !row.source_hash.is_empty() { - obj.push(("source_content_hash".into(), Json::String(row.source_hash.clone()))); + obj.push(( + "source_content_hash".into(), + Json::String(row.source_hash.clone()), + )); } if !row.verification.is_empty() { - obj.push(("verification".into(), Json::String(row.verification.clone()))); + obj.push(( + "verification".into(), + Json::String(row.verification.clone()), + )); } entries.push((row.file.clone(), Json::from_pairs(obj))); } @@ -130,40 +159,64 @@ pub fn write_health(root: &Path, rows: &[HealthRow]) -> Result<(), String> { let _ = body; let stable = Json::obj(vec![ ("version", Json::Int(2)), - ("entries", Json::from_pairs( - rows.iter() - .map(|row| { - let mut obj = vec![ - ("status".into(), Json::String(row.status.clone())), - ("last_updated".into(), Json::String(row.last_updated.clone())), - ("reason".into(), Json::String(row.reason.clone())), - ]; - if !row.source_hash.is_empty() { - obj.push(("source_content_hash".into(), Json::String(row.source_hash.clone()))); - } - if !row.verification.is_empty() { - obj.push(("verification".into(), Json::String(row.verification.clone()))); - } - if !row.public_symbols.is_empty() { - obj.push(("public_symbols".into(), Json::String(row.public_symbols.clone()))); - } - if !row.wiki_hash.is_empty() { - obj.push(("wiki_hash".into(), Json::String(row.wiki_hash.clone()))); - } - if !row.source_mtime_ns.is_empty() { - obj.push(("source_mtime_ns".into(), Json::String(row.source_mtime_ns.clone()))); - } - (row.file.clone(), Json::from_pairs(obj)) - }) - .collect(), - )), + ( + "entries", + Json::from_pairs( + rows.iter() + .map(|row| { + let mut obj = vec![ + ("status".into(), Json::String(row.status.clone())), + ( + "last_updated".into(), + Json::String(row.last_updated.clone()), + ), + ("reason".into(), Json::String(row.reason.clone())), + ]; + if !row.source_hash.is_empty() { + obj.push(( + "source_content_hash".into(), + Json::String(row.source_hash.clone()), + )); + } + if !row.verification.is_empty() { + obj.push(( + "verification".into(), + Json::String(row.verification.clone()), + )); + } + if !row.public_symbols.is_empty() { + obj.push(( + "public_symbols".into(), + Json::String(row.public_symbols.clone()), + )); + } + if !row.wiki_hash.is_empty() { + obj.push(("wiki_hash".into(), Json::String(row.wiki_hash.clone()))); + } + if !row.source_mtime_ns.is_empty() { + obj.push(( + "source_mtime_ns".into(), + Json::String(row.source_mtime_ns.clone()), + )); + } + (row.file.clone(), Json::from_pairs(obj)) + }) + .collect(), + ), + ), ]); fs::write(root.join("file_health.json"), stable.to_string()).map_err(|e| e.to_string())?; let mut md = String::from("# Documentation Health Matrix\n\n| File | Status | Last Updated | Reason / Intent |\n|------|--------|--------------|-----------------|\n"); let mut ordered = rows.to_vec(); ordered.sort_by(|a, b| a.file.cmp(&b.file)); for row in ordered { - md.push_str(&format!("| {} | {} | {} | {} |\n", row.file, row.status, row.last_updated, row.reason.replace('|', "/"))); + md.push_str(&format!( + "| {} | {} | {} | {} |\n", + row.file, + row.status, + row.last_updated, + row.reason.replace('|', "/") + )); } fs::write(root.join("file_health.md"), md).map_err(|e| e.to_string())?; Ok(()) @@ -232,7 +285,12 @@ pub fn parse_journal(text: &str) -> Vec { if let Some(end) = rest.find(']') { let stamp = rest[..end].trim().to_string(); let action = rest[end + 1..].trim().to_string(); - cur = Some(JournalEntry { stamp, action, file: String::new(), reason: String::new() }); + cur = Some(JournalEntry { + stamp, + action, + file: String::new(), + reason: String::new(), + }); } } else if let Some(entry) = cur.as_mut() { if let Some(v) = line.strip_prefix("**File:**") { @@ -253,7 +311,12 @@ pub struct JournalWrite { pub journaled: bool, } -pub fn append_journal(root: &Path, action: &str, file: &str, reason: &str) -> Result { +pub fn append_journal( + root: &Path, + action: &str, + file: &str, + reason: &str, +) -> Result { let stamp = local_stamp(); let path = journal_path(root, &stamp); if let Some(parent) = path.parent() { @@ -261,19 +324,30 @@ pub fn append_journal(root: &Path, action: &str, file: &str, reason: &str) -> Re } let existing = fs::read_to_string(&path).unwrap_or_default(); if journal_duplicate(&existing, action, file, reason, &stamp) { - return Ok(JournalWrite { stamp, journaled: false }); + return Ok(JournalWrite { + stamp, + journaled: false, + }); } let block = format!("## [{stamp}] {action}\n**File:** {file}\n**Reason:** {reason}\n\n"); let mut body = existing; body.push_str(&block); fs::write(&path, body).map_err(|e| e.to_string())?; - Ok(JournalWrite { stamp, journaled: true }) + Ok(JournalWrite { + stamp, + journaled: true, + }) } fn journal_duplicate(existing: &str, action: &str, file: &str, reason: &str, stamp: &str) -> bool { - let Some(now) = stamp_key(stamp) else { return false }; + let Some(now) = stamp_key(stamp) else { + return false; + }; parse_journal(existing).iter().rev().any(|entry| { - entry.action == action && entry.file == file && entry.reason == reason && stamp_key(&entry.stamp).is_some_and(|prev| now.saturating_sub(prev) <= 120) + entry.action == action + && entry.file == file + && entry.reason == reason + && stamp_key(&entry.stamp).is_some_and(|prev| now.saturating_sub(prev) <= 120) }) } @@ -294,7 +368,12 @@ fn stamp_key(stamp: &str) -> Option { fn symbols_field(value: Option<&Json>) -> String { match value { Some(Json::String(text)) => text.clone(), - Some(Json::Array(items)) => items.iter().filter_map(|item| item.as_str()).filter(|s| !s.is_empty()).collect::>().join("\n"), + Some(Json::Array(items)) => items + .iter() + .filter_map(|item| item.as_str()) + .filter(|s| !s.is_empty()) + .collect::>() + .join("\n"), _ => String::new(), } } @@ -305,12 +384,20 @@ pub fn read_pending(root: &Path) -> String { pub fn folder_card(root: &Path, prefix: &str) -> Option { let rel = if prefix.is_empty() || prefix == "." { - root.join(".wikifier_staging").join("maps").join("_root.json") + root.join(".wikifier_staging") + .join("maps") + .join("_root.json") } else { - root.join(".wikifier_staging").join("maps").join(prefix).join("_index.json") + root.join(".wikifier_staging") + .join("maps") + .join(prefix) + .join("_index.json") }; fs::read_to_string(rel).ok().or_else(|| { - let alt = root.join(".wikifier_staging").join("maps").join("_root.json"); + let alt = root + .join(".wikifier_staging") + .join("maps") + .join("_root.json"); if prefix.is_empty() { fs::read_to_string(alt).ok() } else { diff --git a/src/cache.rs b/src/cache.rs index ac0e351..91e8876 100644 --- a/src/cache.rs +++ b/src/cache.rs @@ -5,9 +5,9 @@ //! The original path is stored in the payload JSON and as the `@` header of edge files. use std::collections::BTreeSet; -use std::fs::{self, OpenOptions}; #[cfg(not(windows))] use std::fs::File; +use std::fs::{self, OpenOptions}; use std::io::Write; use std::path::{Path, PathBuf}; use std::sync::atomic::{AtomicU64, Ordering}; @@ -49,11 +49,15 @@ pub(crate) fn legacy_enc(rel: &str) -> String { } pub(crate) fn store_file(root: &Path, sub: &str, rel: &str, ext: &str) -> PathBuf { - cache_root(root).join(sub).join(format!("{}.{}", enc(rel), ext)) + cache_root(root) + .join(sub) + .join(format!("{}.{}", enc(rel), ext)) } pub(crate) fn legacy_file(root: &Path, sub: &str, rel: &str, ext: &str) -> PathBuf { - cache_root(root).join(sub).join(format!("{}.{}", legacy_enc(rel), ext)) + cache_root(root) + .join(sub) + .join(format!("{}.{}", legacy_enc(rel), ext)) } fn read_file(root: &Path, sub: &str, rel: &str, ext: &str) -> PathBuf { @@ -103,7 +107,11 @@ pub(crate) fn atomic_write(path: &Path, bytes: &[u8]) -> Result<(), String> { let file_name = path.file_name().and_then(|s| s.to_str()).unwrap_or("cache"); let tmp = parent.join(format!(".{file_name}.{}.{seq}.tmp", std::process::id())); let write_result = (|| { - let mut file = OpenOptions::new().write(true).create_new(true).open(&tmp).map_err(|e| e.to_string())?; + let mut file = OpenOptions::new() + .write(true) + .create_new(true) + .open(&tmp) + .map_err(|e| e.to_string())?; file.write_all(bytes).map_err(|e| e.to_string())?; file.sync_all().map_err(|e| e.to_string())?; Ok(()) @@ -136,7 +144,8 @@ fn replace_file(from: &Path, to: &Path) -> Result<(), String> { } #[cfg(not(windows))] { - fs::rename(from, to).map_err(|e| format!("rename {} -> {}: {e}", from.display(), to.display())) + fs::rename(from, to) + .map_err(|e| format!("rename {} -> {}: {e}", from.display(), to.display())) } } @@ -156,9 +165,20 @@ fn replace_file_windows(from: &Path, to: &Path) -> Result<(), String> { } let src = wide(from); let dst = wide(to); - let ok = unsafe { MoveFileExW(src.as_ptr(), dst.as_ptr(), MOVEFILE_REPLACE_EXISTING | MOVEFILE_WRITE_THROUGH) }; + let ok = unsafe { + MoveFileExW( + src.as_ptr(), + dst.as_ptr(), + MOVEFILE_REPLACE_EXISTING | MOVEFILE_WRITE_THROUGH, + ) + }; if ok == 0 { - Err(format!("rename {} -> {}: {}", from.display(), to.display(), std::io::Error::last_os_error())) + Err(format!( + "rename {} -> {}: {}", + from.display(), + to.display(), + std::io::Error::last_os_error() + )) } else { Ok(()) } @@ -220,11 +240,19 @@ fn fsync_dir_windows(path: &Path) { } pub(crate) fn read_lines(path: &Path) -> Vec { - fs::read_to_string(path).unwrap_or_default().lines().map(|s| s.trim().to_string()).filter(|s| !s.is_empty()).collect() + fs::read_to_string(path) + .unwrap_or_default() + .lines() + .map(|s| s.trim().to_string()) + .filter(|s| !s.is_empty()) + .collect() } pub(crate) fn content_lines(path: &Path) -> Vec { - read_lines(path).into_iter().filter(|line| !line.starts_with("@\t")).collect() + read_lines(path) + .into_iter() + .filter(|line| !line.starts_with("@\t")) + .collect() } pub(crate) fn dst_of(line: &str) -> String { @@ -232,7 +260,11 @@ pub(crate) fn dst_of(line: &str) -> String { } pub fn load_dependencies(root: &Path, rel: &str) -> Result, String> { - let mut dsts: Vec = content_lines(&by_src_path(root, rel)).into_iter().map(|line| dst_of(&line)).filter(|s| !s.is_empty()).collect(); + let mut dsts: Vec = content_lines(&by_src_path(root, rel)) + .into_iter() + .map(|line| dst_of(&line)) + .filter(|s| !s.is_empty()) + .collect(); dsts.sort(); dsts.dedup(); Ok(dsts) @@ -278,7 +310,14 @@ pub fn confidence_of(conditional: bool, dynamic: bool, via_barrel: bool) -> f64 } } -pub fn format_edge_line(dst: &str, line: i64, names: &[String], conditional: bool, dynamic: bool, via_barrel: bool) -> String { +pub fn format_edge_line( + dst: &str, + line: i64, + names: &[String], + conditional: bool, + dynamic: bool, + via_barrel: bool, +) -> String { let mut flags = String::new(); if conditional { flags.push('c'); @@ -298,7 +337,11 @@ pub fn format_edge_line(dst: &str, line: i64, names: &[String], conditional: boo } fn parse_flags(field: &str) -> (bool, bool, bool) { - (field.contains('c'), field.contains('d'), field.contains('b')) + ( + field.contains('c'), + field.contains('d'), + field.contains('b'), + ) } #[derive(Clone, Debug)] @@ -321,9 +364,22 @@ pub fn load_dependency_view(root: &Path, rel: &str) -> Result Result Result Result, String> { @@ -423,7 +487,12 @@ pub fn payload_bytes(root: &Path, rel: &str) -> Option> { pub fn stat_ns(path: &Path) -> Option<(u128, u64)> { let meta = fs::metadata(path).ok()?; - let mtime = meta.modified().ok()?.duration_since(std::time::UNIX_EPOCH).ok()?.as_nanos(); + let mtime = meta + .modified() + .ok()? + .duration_since(std::time::UNIX_EPOCH) + .ok()? + .as_nanos(); Some((mtime, meta.len())) } @@ -449,9 +518,20 @@ pub fn write_barrel_record(root: &Path, rel: &str, dsts: &[String]) -> Result<() let _lock = crate::cache_commit::LockGuard::acquire(root)?; let body = crate::jsonutil::Json::obj(vec![ ("src", crate::jsonutil::Json::String(rel.to_string())), - ("dsts", crate::jsonutil::Json::Array(dsts.iter().cloned().map(crate::jsonutil::Json::String).collect())), + ( + "dsts", + crate::jsonutil::Json::Array( + dsts.iter() + .cloned() + .map(crate::jsonutil::Json::String) + .collect(), + ), + ), ]); - atomic_write(&store_file(root, "barrels", rel, "json"), body.to_string().as_bytes()) + atomic_write( + &store_file(root, "barrels", rel, "json"), + body.to_string().as_bytes(), + ) } pub fn prune_barrel_files(root: &Path, max_age_days: f64, dry_run: bool) -> Result { @@ -471,14 +551,25 @@ pub fn prune_barrel_files(root: &Path, max_age_days: f64, dry_run: bool) -> Resu } let text = fs::read_to_string(&path).unwrap_or_default(); let parsed = crate::jsonutil::parse(&text).ok(); - let src = parsed.as_ref().and_then(|v| v.get("src")).and_then(|s| s.as_str()).unwrap_or(""); + let src = parsed + .as_ref() + .and_then(|v| v.get("src")) + .and_then(|s| s.as_str()) + .unwrap_or(""); let dsts: Vec = parsed .as_ref() .and_then(|v| v.get("dsts")) .and_then(|v| v.as_array()) - .map(|items| items.iter().filter_map(|item| item.as_str().map(|s| s.to_string())).collect()) + .map(|items| { + items + .iter() + .filter_map(|item| item.as_str().map(|s| s.to_string())) + .collect() + }) .unwrap_or_default(); - let dangling = src.is_empty() || !root.join(src).is_file() || dsts.iter().any(|dst| !root.join(dst).is_file()); + let dangling = src.is_empty() + || !root.join(src).is_file() + || dsts.iter().any(|dst| !root.join(dst).is_file()); let aged = ent .metadata() .ok() diff --git a/src/cache_commit.rs b/src/cache_commit.rs index a7feebb..385346e 100644 --- a/src/cache_commit.rs +++ b/src/cache_commit.rs @@ -44,7 +44,10 @@ impl LockGuard { loop { match OpenOptions::new().write(true).create_new(true).open(&path) { Ok(mut file) => { - let stamp = SystemTime::now().duration_since(UNIX_EPOCH).map(|d| d.as_secs()).unwrap_or(0); + let stamp = SystemTime::now() + .duration_since(UNIX_EPOCH) + .map(|d| d.as_secs()) + .unwrap_or(0); writeln!(file, "{}\n{stamp}", std::process::id()).map_err(|e| e.to_string())?; file.sync_all().map_err(|e| e.to_string())?; return Ok(LockGuard { path }); @@ -74,7 +77,13 @@ fn try_steal(path: &Path) -> bool { Ok(text) => text, Err(_) => return false, }; - let pid: i32 = text.lines().next().unwrap_or("0").trim().parse().unwrap_or(0); + let pid: i32 = text + .lines() + .next() + .unwrap_or("0") + .trim() + .parse() + .unwrap_or(0); let stale = if pid > 0 { !crate::proc::pid_alive(pid) } else { @@ -138,16 +147,23 @@ impl Session { } pub fn known(&self, rel: &str) -> Option<(u128, u64, String)> { - self.rows.get(rel).map(|row| (row.mtime_ns, row.size, row.hash.clone())) + self.rows + .get(rel) + .map(|row| (row.mtime_ns, row.size, row.hash.clone())) } pub fn dsts_of(&self, rel: &str) -> BTreeSet { - self.rows.get(rel).map(|row| row.dsts.clone()).unwrap_or_default() + self.rows + .get(rel) + .map(|row| row.dsts.clone()) + .unwrap_or_default() } /// Record a new size or mtime when the bytes still match the stored hash. pub fn note_stat(&mut self, rel: &str, mtime_ns: u128, size: u64) { - let Some(row) = self.rows.get_mut(rel) else { return }; + let Some(row) = self.rows.get_mut(rel) else { + return; + }; if row.mtime_ns == mtime_ns && row.size == size { return; } @@ -162,7 +178,11 @@ impl Session { pub fn upsert(&mut self, upd: &FileUpdate) { self.removed.remove(&upd.rel); - let old = self.rows.get(&upd.rel).map(|row| row.dsts.clone()).unwrap_or_default(); + let old = self + .rows + .get(&upd.rel) + .map(|row| row.dsts.clone()) + .unwrap_or_default(); for gone in old.difference(&upd.dsts) { if let Some(set) = self.rev.get_mut(gone) { set.remove(&upd.rel); @@ -170,7 +190,10 @@ impl Session { self.dirty_dst.insert(gone.clone()); } for added in upd.dsts.difference(&old) { - self.rev.entry(added.clone()).or_default().insert(upd.rel.clone()); + self.rev + .entry(added.clone()) + .or_default() + .insert(upd.rel.clone()); self.dirty_dst.insert(added.clone()); } let mut src_lines = if upd.edge_lines.is_empty() { @@ -185,8 +208,20 @@ impl Session { "{{\"rel\":{},\"sha256\":{},\"imports\":{},\"resolved\":{}}}", crate::jsonutil::Json::String(upd.rel.clone()), crate::jsonutil::Json::String(upd.hash.clone()), - crate::jsonutil::Json::Array(upd.imports.iter().cloned().map(crate::jsonutil::Json::String).collect()), - crate::jsonutil::Json::Array(upd.dsts.iter().cloned().map(crate::jsonutil::Json::String).collect()), + crate::jsonutil::Json::Array( + upd.imports + .iter() + .cloned() + .map(crate::jsonutil::Json::String) + .collect() + ), + crate::jsonutil::Json::Array( + upd.dsts + .iter() + .cloned() + .map(crate::jsonutil::Json::String) + .collect() + ), ); self.payloads.insert(upd.rel.clone(), payload); let mut notes = String::new(); @@ -196,7 +231,10 @@ impl Session { for (raw, line, reason) in &upd.unresolved { notes.push_str(&format!("U\t{raw}\t{line}\t{reason}\n")); } - self.notes.insert(upd.rel.clone(), if notes.is_empty() { None } else { Some(notes) }); + self.notes.insert( + upd.rel.clone(), + if notes.is_empty() { None } else { Some(notes) }, + ); self.rows.insert( upd.rel.clone(), Row { @@ -211,7 +249,14 @@ impl Session { } pub fn remove(&mut self, rel: &str) { - let importers: Vec = self.rev.get(rel).cloned().unwrap_or_default().into_iter().filter(|src| src != rel && self.rows.contains_key(src)).collect(); + let importers: Vec = self + .rev + .get(rel) + .cloned() + .unwrap_or_default() + .into_iter() + .filter(|src| src != rel && self.rows.contains_key(src)) + .collect(); for src in importers { self.strip_dst(&src, rel); } @@ -238,7 +283,11 @@ impl Session { if let Some(set) = self.rev.get_mut(dst) { set.remove(src); } - let mut lines = self.dirty_src.get(src).cloned().unwrap_or_else(|| cache::content_lines(&cache::by_src_path(&self.root, src))); + let mut lines = self + .dirty_src + .get(src) + .cloned() + .unwrap_or_else(|| cache::content_lines(&cache::by_src_path(&self.root, src))); lines.retain(|line| cache::dst_of(line) != dst); self.dirty_src.insert(src.to_string(), lines); self.dirty_dst.insert(dst.to_string()); @@ -254,11 +303,19 @@ impl Session { return Ok(()); } let gen = read_generation(&self.root).saturating_add(1); - cache::atomic_write(&cache::commit_pending_path(&self.root), format!("{gen}\n").as_bytes())?; + cache::atomic_write( + &cache::commit_pending_path(&self.root), + format!("{gen}\n").as_bytes(), + )?; // by_dst is keyed by the importer's target. Dropping a file removes its // forward edges above; the reverse file for this path stays until importers change. for rel in &self.removed { - for (sub, ext) in [("payload", "json"), ("by_src", "edges"), ("notes", "txt"), ("barrels", "json")] { + for (sub, ext) in [ + ("payload", "json"), + ("by_src", "edges"), + ("notes", "txt"), + ("barrels", "json"), + ] { let _ = fs::remove_file(cache::store_file(&self.root, sub, rel, ext)); let _ = fs::remove_file(cache::legacy_file(&self.root, sub, rel, ext)); } @@ -267,11 +324,22 @@ impl Session { if self.removed.contains(rel) { continue; } - let body = if lines.is_empty() { String::new() } else { format!("{}\n", lines.join("\n")) }; - cache::write_labeled(&cache::store_file(&self.root, "by_src", rel, "edges"), rel, &body)?; + let body = if lines.is_empty() { + String::new() + } else { + format!("{}\n", lines.join("\n")) + }; + cache::write_labeled( + &cache::store_file(&self.root, "by_src", rel, "edges"), + rel, + &body, + )?; } for (rel, payload) in &self.payloads { - cache::atomic_write(&cache::store_file(&self.root, "payload", rel, "json"), payload.as_bytes())?; + cache::atomic_write( + &cache::store_file(&self.root, "payload", rel, "json"), + payload.as_bytes(), + )?; } for (rel, notes) in &self.notes { let path = cache::store_file(&self.root, "notes", rel, "txt"); @@ -285,9 +353,20 @@ impl Session { for (rel, dsts) in &self.barrels { let body = crate::jsonutil::Json::obj(vec![ ("src", crate::jsonutil::Json::String(rel.clone())), - ("dsts", crate::jsonutil::Json::Array(dsts.iter().cloned().map(crate::jsonutil::Json::String).collect())), + ( + "dsts", + crate::jsonutil::Json::Array( + dsts.iter() + .cloned() + .map(crate::jsonutil::Json::String) + .collect(), + ), + ), ]); - cache::atomic_write(&cache::store_file(&self.root, "barrels", rel, "json"), body.to_string().as_bytes())?; + cache::atomic_write( + &cache::store_file(&self.root, "barrels", rel, "json"), + body.to_string().as_bytes(), + )?; } // Reverse edges after forward edges. A crash here is repaired from by_src. for dst in &self.dirty_dst { @@ -302,7 +381,10 @@ impl Session { } } save_index(&self.root, &self.rows)?; - cache::atomic_write(&cache::generation_path(&self.root), format!("{gen}\n").as_bytes())?; + cache::atomic_write( + &cache::generation_path(&self.root), + format!("{gen}\n").as_bytes(), + )?; let _ = fs::remove_file(cache::commit_pending_path(&self.root)); cache::atomic_write(&cache::names_marker(&self.root), b"fnv1a64\n")?; Ok(()) @@ -310,7 +392,10 @@ impl Session { } fn read_generation(root: &Path) -> u64 { - fs::read_to_string(cache::generation_path(root)).ok().and_then(|text| text.trim().parse().ok()).unwrap_or(0) + fs::read_to_string(cache::generation_path(root)) + .ok() + .and_then(|text| text.trim().parse().ok()) + .unwrap_or(0) } fn join_dsts(dsts: &BTreeSet) -> String { @@ -325,7 +410,10 @@ fn join_dsts(dsts: &BTreeSet) -> String { } fn parse_dsts(raw: &str) -> BTreeSet { - raw.split('\u{1e}').filter(|s| !s.is_empty()).map(|s| s.to_string()).collect() + raw.split('\u{1e}') + .filter(|s| !s.is_empty()) + .map(|s| s.to_string()) + .collect() } fn index_record_lines(root: &Path) -> Vec { @@ -353,10 +441,22 @@ fn load_rows(root: &Path) -> (BTreeMap, bool) { Some(raw) => parse_dsts(raw), None => { upgrade = true; - cache::load_dependencies(root, &rel).unwrap_or_default().into_iter().collect() + cache::load_dependencies(root, &rel) + .unwrap_or_default() + .into_iter() + .collect() } }; - map.insert(rel.clone(), Row { rel, mtime_ns, size, hash, dsts }); + map.insert( + rel.clone(), + Row { + rel, + mtime_ns, + size, + hash, + dsts, + }, + ); } (map, upgrade) } @@ -365,7 +465,14 @@ fn save_index(root: &Path, rows: &BTreeMap) -> Result<(), String> { let mut body = String::new(); let mut paths = String::new(); for row in rows.values() { - body.push_str(&format!("{}\t{}\t{}\t{}\t{}\n", row.rel, row.mtime_ns, row.size, row.hash, join_dsts(&row.dsts))); + body.push_str(&format!( + "{}\t{}\t{}\t{}\t{}\n", + row.rel, + row.mtime_ns, + row.size, + row.hash, + join_dsts(&row.dsts) + )); paths.push_str(&row.rel); paths.push('\n'); } @@ -379,7 +486,11 @@ fn index_rels(root: &Path) -> Vec { .into_iter() .filter_map(|line| { let rel = line.split('\t').next().unwrap_or("").trim().to_string(); - if rel.is_empty() { None } else { Some(rel) } + if rel.is_empty() { + None + } else { + Some(rel) + } }) .collect() } @@ -391,7 +502,9 @@ fn migrate_legacy(root: &Path) -> Result<(), String> { } for sub in ["payload", "by_src", "by_dst", "notes", "barrels"] { let dir = root_dir.join(sub); - let Ok(entries) = fs::read_dir(&dir) else { continue }; + let Ok(entries) = fs::read_dir(&dir) else { + continue; + }; for ent in entries.flatten() { let name = ent.file_name().to_string_lossy().to_string(); if !name.contains('%') { @@ -452,24 +565,40 @@ fn hash_to_rel(root: &Path) -> HashMap { fn file_header(path: &Path) -> Option { let text = fs::read_to_string(path).ok()?; - text.lines().next().and_then(|line| line.trim().strip_prefix("@\t").map(|s| s.to_string())) + text.lines() + .next() + .and_then(|line| line.trim().strip_prefix("@\t").map(|s| s.to_string())) } -fn read_edge_dir(dir: &Path, names: &HashMap, orphans: &mut Vec) -> BTreeMap> { +fn read_edge_dir( + dir: &Path, + names: &HashMap, + orphans: &mut Vec, +) -> BTreeMap> { let mut out = BTreeMap::new(); - let Ok(entries) = fs::read_dir(dir) else { return out }; + let Ok(entries) = fs::read_dir(dir) else { + return out; + }; for ent in entries.flatten() { let path = ent.path(); if path.extension().and_then(|ext| ext.to_str()) != Some("edges") { continue; } - let stem = path.file_stem().and_then(|s| s.to_str()).unwrap_or("").to_string(); + let stem = path + .file_stem() + .and_then(|s| s.to_str()) + .unwrap_or("") + .to_string(); let rel = file_header(&path).or_else(|| names.get(&stem).cloned()); let Some(rel) = rel else { orphans.push(format!("edge file {} has no stored path", path.display())); continue; }; - let dsts = cache::content_lines(&path).into_iter().map(|line| cache::dst_of(&line)).filter(|s| !s.is_empty()).collect(); + let dsts = cache::content_lines(&path) + .into_iter() + .map(|line| cache::dst_of(&line)) + .filter(|s| !s.is_empty()) + .collect(); out.insert(rel, dsts); } out @@ -482,12 +611,24 @@ pub fn check_consistency(root: &Path) -> Consistency { mismatches.push("commit.pending is set; the last cache commit did not finish".into()); } if !cache::cache_root(root).is_dir() { - return Consistency { consistent: mismatches.is_empty(), incomplete, mismatches }; + return Consistency { + consistent: mismatches.is_empty(), + incomplete, + mismatches, + }; } let names = hash_to_rel(root); let mut orphans = Vec::new(); - let forward = read_edge_dir(&cache::cache_root(root).join("by_src"), &names, &mut orphans); - let reverse = read_edge_dir(&cache::cache_root(root).join("by_dst"), &names, &mut orphans); + let forward = read_edge_dir( + &cache::cache_root(root).join("by_src"), + &names, + &mut orphans, + ); + let reverse = read_edge_dir( + &cache::cache_root(root).join("by_dst"), + &names, + &mut orphans, + ); mismatches.append(&mut orphans); let mut expected: BTreeMap> = BTreeMap::new(); for (src, dsts) in &forward { @@ -519,7 +660,11 @@ pub fn check_consistency(root: &Path) -> Consistency { mismatches.push(format!("by_src {rel} is not in the index")); } } - Consistency { consistent: mismatches.is_empty(), incomplete, mismatches } + Consistency { + consistent: mismatches.is_empty(), + incomplete, + mismatches, + } } pub fn rebuild(root: &Path) -> Result<(), String> { @@ -532,7 +677,11 @@ fn rebuild_locked(root: &Path) -> Result<(), String> { let (old, _) = load_rows(root); let names = hash_to_rel(root); let mut orphans = Vec::new(); - let forward = read_edge_dir(&cache::cache_root(root).join("by_src"), &names, &mut orphans); + let forward = read_edge_dir( + &cache::cache_root(root).join("by_src"), + &names, + &mut orphans, + ); if !orphans.is_empty() { return Err(orphans.join("; ")); } diff --git a/src/cycle_report.rs b/src/cycle_report.rs new file mode 100644 index 0000000..0dc5f7e --- /dev/null +++ b/src/cycle_report.rs @@ -0,0 +1,124 @@ +//! Cycle severity and reverse-dependency ranking. +//! `library.md` stays the short tree. This rides on `cycles` JSON. + +use std::collections::BTreeMap; + +use crate::jsonutil::Json; + +pub fn enrich(value: Json, analysis: bool, max_items: Option) -> Json { + if !analysis { + return cap_cycles(value, max_items); + } + let mut value = value; + let ranked = reverse_rank(value.get("cycles")); + if let Some(Json::Array(cycles)) = value.get("cycles").cloned() { + let scored: Vec = cycles.into_iter().map(score_cycle).collect(); + value.insert("cycles", Json::Array(scored)); + } + value.insert("reverse_dependencies", Json::Array(ranked)); + value.insert("analysis", Json::Bool(true)); + cap_cycles(value, max_items) +} + +fn cap_cycles(mut value: Json, max_items: Option) -> Json { + let Some(limit) = max_items else { + return value; + }; + if let Some(Json::Array(cycles)) = value.get("cycles").cloned() { + let truncated = cycles.len() > limit; + let kept: Vec = cycles.into_iter().take(limit).collect(); + value.insert("cycles", Json::Array(kept)); + if truncated { + value.insert("truncated", Json::Bool(true)); + } + } + value +} + +fn score_cycle(cycle: Json) -> Json { + let size = cycle + .get("size") + .and_then(|v| match v { + Json::Int(n) => Some(*n), + _ => None, + }) + .unwrap_or(0); + let blast = cycle + .get("external_edges") + .and_then(|v| match v { + Json::Int(n) => Some(*n), + _ => None, + }) + .unwrap_or(0); + let score = (size as f64) * 2.0 + (blast as f64) * 0.8; + let severity = if size >= 8 || blast >= 20 { + "HIGH" + } else if size >= 3 || blast >= 5 { + "MEDIUM" + } else { + "LOW" + }; + let (strategy, rationale) = if size <= 2 { + ( + "audit_and_directional_split", + "Two-file cycle. Pick the owner of the shared idea and point the other file at it.", + ) + } else { + ("extract_shared_module", "More than two files depend on each other. Move the shared types into a module that none of them own.") + }; + let mut cycle = cycle; + cycle.insert("severity", Json::String(severity.into())); + cycle.insert("score", Json::Float(score)); + cycle.insert("external_blast_radius", Json::Int(blast)); + cycle.insert( + "recommendation", + Json::obj(vec![ + ("strategy", Json::String(strategy.into())), + ("rationale", Json::String(rationale.into())), + ]), + ); + cycle +} + +/// Rank files by how many cycle peers point at them is not the graph rank. +/// Callers pass the cycle list only when they have no edge index. +/// The API fills `reverse_dependencies` from the real edge list before enrich +/// when it can. This helper ranks destinations mentioned inside the cycles +/// so a cycles-only payload still has an order. +fn reverse_rank(cycles: Option<&Json>) -> Vec { + let mut counts: BTreeMap = BTreeMap::new(); + if let Some(Json::Array(items)) = cycles { + for cycle in items { + if let Some(Json::Array(files)) = cycle.get("files") { + for file in files { + if let Some(name) = file.as_str() { + *counts.entry(name.to_string()).or_insert(0) += 1; + } + } + } + } + } + ranked(counts) +} + +pub fn rank_dependents(edges: &[(String, String)]) -> Vec { + let mut counts: BTreeMap = BTreeMap::new(); + for (_src, dst) in edges { + *counts.entry(dst.clone()).or_insert(0) += 1; + } + ranked(counts) +} + +fn ranked(counts: BTreeMap) -> Vec { + let mut rows: Vec<(String, i64)> = counts.into_iter().collect(); + rows.sort_by(|a, b| b.1.cmp(&a.1).then_with(|| a.0.cmp(&b.0))); + rows.truncate(20); + rows.into_iter() + .map(|(file, count)| { + Json::obj(vec![ + ("file", Json::String(file)), + ("dependents", Json::Int(count)), + ]) + }) + .collect() +} diff --git a/src/daemon_svc.rs b/src/daemon_svc.rs new file mode 100644 index 0000000..015a668 --- /dev/null +++ b/src/daemon_svc.rs @@ -0,0 +1,310 @@ +//! Daemon log, heartbeat, and OS service install. +//! The poll loop lives in `api`. This module only writes the side files. + +use std::fs; +use std::io::Write; +use std::path::{Path, PathBuf}; + +use crate::jsonutil::Json; + +pub fn pid_path(root: &Path) -> PathBuf { + root.join(".wikifier_staging").join("wikifier.pid") +} + +pub fn legacy_pid_path(root: &Path) -> PathBuf { + root.join(".wikifier_staging").join("daemon.pid") +} + +pub fn log_path(root: &Path) -> PathBuf { + root.join(".wikifier_staging").join("daemon.log") +} + +pub fn heartbeat_path(root: &Path) -> PathBuf { + root.join(".wikifier_staging").join("daemon_heartbeat.json") +} + +pub fn poll_seconds() -> u64 { + std::env::var("WIKIFIER_POLL_INTERVAL") + .ok() + .and_then(|s| s.parse().ok()) + .unwrap_or(30) + .max(1) +} + +pub fn maps_interval_seconds() -> u64 { + std::env::var("WIKIFIER_DAEMON_MAPS_INTERVAL") + .ok() + .and_then(|s| s.parse().ok()) + .unwrap_or(600) +} + +pub fn maps_enabled() -> bool { + !matches!( + std::env::var("WIKIFIER_DAEMON_MAPS").ok().as_deref(), + Some("0") | Some("false") | Some("no") + ) +} + +pub fn append_log(root: &Path, message: &str) { + let _ = fs::create_dir_all(root.join(".wikifier_staging")); + let line = format!("[{}] {message}\n", stamp()); + if let Ok(mut file) = fs::OpenOptions::new() + .create(true) + .append(true) + .open(log_path(root)) + { + let _ = file.write_all(line.as_bytes()); + } +} + +pub fn write_heartbeat(root: &Path, event: &str, ok: bool, detail: &str) { + let path = heartbeat_path(root); + let previous = fs::read_to_string(&path) + .ok() + .and_then(|text| crate::jsonutil::parse(&text).ok()); + let prior = previous + .as_ref() + .and_then(|v| v.get("consecutive_failures")) + .and_then(|v| match v { + Json::Int(n) => Some(*n), + _ => None, + }) + .unwrap_or(0); + let streak = if ok { 0 } else { prior + 1 }; + let body = Json::obj(vec![ + ("ts", Json::String(stamp())), + ("event", Json::String(event.into())), + ("ok", Json::Bool(ok)), + ("detail", Json::String(detail.chars().take(400).collect())), + ("consecutive_failures", Json::Int(streak)), + ("project_root", Json::String(root.display().to_string())), + ]); + let _ = fs::create_dir_all(root.join(".wikifier_staging")); + let _ = fs::write(path, format!("{body}\n")); + if streak >= 5 { + append_log( + root, + &format!("WARNING: {streak} consecutive daemon step failures"), + ); + } +} + +pub fn read_log_tail(root: &Path, lines: usize) -> String { + let text = fs::read_to_string(log_path(root)).unwrap_or_default(); + let all: Vec<&str> = text.lines().collect(); + let start = all.len().saturating_sub(lines); + all[start..].join("\n") +} + +pub fn install(root: &Path, exe: &Path) -> Json { + match std::env::consts::OS { + "linux" => write_unit( + service_file("systemd"), + &systemd_unit(root, exe), + "systemd user unit", + ), + "macos" => write_unit( + service_file("launchd"), + &launchd_plist(root, exe), + "launchd agent", + ), + "windows" => install_windows(root, exe), + other => fail(&format!( + "no service manager for {other}; run `wikifier daemon run` in a terminal" + )), + } +} + +pub fn uninstall(_root: &Path) -> Json { + match std::env::consts::OS { + "linux" => remove_unit(service_file("systemd"), "systemd user unit"), + "macos" => remove_unit(service_file("launchd"), "launchd agent"), + "windows" => uninstall_windows(), + other => fail(&format!("no service manager for {other}")), + } +} + +pub fn systemd_unit(root: &Path, exe: &Path) -> String { + let log = log_path(root); + format!( + "[Unit]\n\ +Description=Wikifier daemon for {root}\n\ +After=network.target\n\ +\n\ +[Service]\n\ +Type=simple\n\ +WorkingDirectory={root}\n\ +Environment=WIKIFIER_PROJECT_ROOT={root}\n\ +ExecStart=\"{exe}\" --target \"{root}\" daemon run\n\ +Restart=always\n\ +RestartSec=10\n\ +KillSignal=SIGTERM\n\ +TimeoutStopSec=45\n\ +StandardOutput=append:{log}\n\ +StandardError=append:{log}\n\ +\n\ +[Install]\n\ +WantedBy=default.target\n", + root = root.display(), + exe = exe.display(), + log = log.display(), + ) +} + +pub fn launchd_plist(root: &Path, exe: &Path) -> String { + let log_path = log_path(root); + let log = log_path.display().to_string(); + format!( + "\n\ +\n\ +\n\ + Labelcom.wikifier.agent\n\ + WorkingDirectory{root}\n\ + EnvironmentVariables\n\ + WIKIFIER_PROJECT_ROOT{root}\n\ + \n\ + ProgramArguments\n\ + {exe}--target{root}daemonrun\n\ + \n\ + RunAtLoad\n\ + StandardOutPath{log}\n\ + StandardErrorPath{log}\n\ +\n", + root = xml_escape(&root.display().to_string()), + exe = xml_escape(&exe.display().to_string()), + log = xml_escape(&log.to_string()), + ) +} + +fn install_windows(root: &Path, exe: &Path) -> Json { + let task = format!( + "\"{}\" --target \"{}\" daemon run", + exe.display(), + root.display() + ); + let output = std::process::Command::new("schtasks") + .args([ + "/Create", "/F", "/TN", "Wikifier", "/SC", "ONLOGON", "/TR", &task, + ]) + .output(); + match output { + Ok(out) if out.status.success() => ok("windows", "Task Scheduler task Wikifier", &task), + Ok(out) => fail(&format!( + "Task Scheduler rejected the task ({}). Create it with: schtasks /Create /F /TN Wikifier /SC ONLOGON /TR {}", + String::from_utf8_lossy(&out.stderr).trim(), + task + )), + Err(err) => fail(&format!("Task Scheduler is not available ({err}). Run `wikifier daemon run` in a terminal, or create a task whose action is: {task}")), + } +} + +fn uninstall_windows() -> Json { + let output = std::process::Command::new("schtasks") + .args(["/Delete", "/F", "/TN", "Wikifier"]) + .output(); + match output { + Ok(out) if out.status.success() => { + ok("windows", "removed Task Scheduler task Wikifier", "") + } + Ok(out) => fail(&format!( + "could not delete the Wikifier task: {}", + String::from_utf8_lossy(&out.stderr).trim() + )), + Err(err) => fail(&format!("Task Scheduler is not available ({err})")), + } +} + +fn write_unit(path: PathBuf, body: &str, kind: &str) -> Json { + if let Some(parent) = path.parent() { + if fs::create_dir_all(parent).is_err() { + return fail(&format!("could not create {}", parent.display())); + } + } + if fs::write(&path, body).is_err() { + return fail(&format!("could not write {}", path.display())); + } + ok( + std::env::consts::OS, + &format!("wrote {kind}"), + &path.display().to_string(), + ) +} + +fn remove_unit(path: PathBuf, kind: &str) -> Json { + if path.is_file() { + if fs::remove_file(&path).is_err() { + return fail(&format!("could not remove {}", path.display())); + } + ok( + std::env::consts::OS, + &format!("removed {kind}"), + &path.display().to_string(), + ) + } else { + ok( + std::env::consts::OS, + &format!("no {kind} installed"), + &path.display().to_string(), + ) + } +} + +fn service_file(kind: &str) -> PathBuf { + let home = service_home(); + match kind { + "launchd" => home + .join("Library") + .join("LaunchAgents") + .join("com.wikifier.agent.plist"), + _ => { + let base = std::env::var_os("XDG_CONFIG_HOME") + .map(PathBuf::from) + .unwrap_or_else(|| home.join(".config")); + base.join("systemd").join("user").join("wikifier.service") + } + } +} + +fn service_home() -> PathBuf { + if let Some(path) = std::env::var_os("WIKIFIER_SERVICE_HOME") { + if !path.is_empty() { + return PathBuf::from(path); + } + } + if let Some(path) = std::env::var_os("HOME") { + if !path.is_empty() { + return PathBuf::from(path); + } + } + if let Some(path) = std::env::var_os("USERPROFILE") { + return PathBuf::from(path); + } + PathBuf::from(".") +} + +fn stamp() -> String { + crate::timeutil::local_stamp() +} + +fn xml_escape(text: &str) -> String { + text.replace('&', "&") + .replace('<', "<") + .replace('>', ">") +} + +fn ok(os: &str, message: &str, path: &str) -> Json { + Json::obj(vec![ + ("success", Json::Bool(true)), + ("os", Json::String(os.into())), + ("message", Json::String(message.into())), + ("path", Json::String(path.into())), + ]) +} + +fn fail(message: &str) -> Json { + Json::obj(vec![ + ("success", Json::Bool(false)), + ("error", Json::String(message.into())), + ("error_type", Json::String("DaemonError".into())), + ]) +} diff --git a/src/jsonutil.rs b/src/jsonutil.rs index 9ee69e1..d726060 100644 --- a/src/jsonutil.rs +++ b/src/jsonutil.rs @@ -129,7 +129,10 @@ fn escape(s: &str) -> String { } pub fn parse(input: &str) -> Result { - let mut p = Parser { s: input.as_bytes(), i: 0 }; + let mut p = Parser { + s: input.as_bytes(), + i: 0, + }; let v = p.value()?; p.skip(); if p.i != p.s.len() { @@ -246,7 +249,8 @@ impl Parser<'_> { if self.i + 4 > self.s.len() { return Err("bad unicode".into()); } - let hex = std::str::from_utf8(&self.s[self.i..self.i + 4]).map_err(|e| e.to_string())?; + let hex = std::str::from_utf8(&self.s[self.i..self.i + 4]) + .map_err(|e| e.to_string())?; let cp = u32::from_str_radix(hex, 16).map_err(|e| e.to_string())?; let ch = char::from_u32(cp).unwrap_or('\u{FFFD}'); let mut buf = [0u8; 4]; @@ -289,9 +293,13 @@ impl Parser<'_> { } let text = std::str::from_utf8(&self.s[start..self.i]).map_err(|e| e.to_string())?; if float { - text.parse::().map(Json::Float).map_err(|e| e.to_string()) + text.parse::() + .map(Json::Float) + .map_err(|e| e.to_string()) } else { - text.parse::().map(Json::Int).map_err(|e| e.to_string()) + text.parse::() + .map(Json::Int) + .map_err(|e| e.to_string()) } } fn array(&mut self) -> Result { diff --git a/src/legacy_cache.rs b/src/legacy_cache.rs index 14782f0..6d13c85 100644 --- a/src/legacy_cache.rs +++ b/src/legacy_cache.rs @@ -68,7 +68,10 @@ fn import_sqlite(root: &Path, db: &Path) -> Result { continue; } rels.insert(src.to_string()); - by_src.entry(src.to_string()).or_default().insert(dst.to_string()); + by_src + .entry(src.to_string()) + .or_default() + .insert(dst.to_string()); } if rels.is_empty() { return Ok(0); @@ -86,13 +89,19 @@ fn sqlite_lines(db: &Path, sql: &str) -> Result, String> { let err = String::from_utf8_lossy(&output.stderr); return Err(format!("sqlite3 exited {}: {err}", output.status)); } - Ok(String::from_utf8_lossy(&output.stdout).lines().map(|s| s.to_string()).filter(|s| !s.trim().is_empty()).collect()) + Ok(String::from_utf8_lossy(&output.stdout) + .lines() + .map(|s| s.to_string()) + .filter(|s| !s.trim().is_empty()) + .collect()) } fn import_json(root: &Path, path: &Path) -> Result { let text = fs::read_to_string(path).map_err(|e| e.to_string())?; let parsed = jsonutil::parse(&text).map_err(|e| e.to_string())?; - let obj = parsed.as_object().ok_or_else(|| "import_cache.json is not an object".to_string())?; + let obj = parsed + .as_object() + .ok_or_else(|| "import_cache.json is not an object".to_string())?; let mut rels: BTreeSet = BTreeSet::new(); let mut by_src: BTreeMap> = BTreeMap::new(); for (key, entry) in obj { @@ -106,7 +115,11 @@ fn import_json(root: &Path, path: &Path) -> Result { let mut dsts = BTreeSet::new(); if let Some(pairs) = entry.get("resolved_pairs").and_then(|v| v.as_array()) { for pair in pairs { - if let Some(dst) = pair.get("resolved").and_then(|v| v.as_str()).or_else(|| pair.as_str()) { + if let Some(dst) = pair + .get("resolved") + .and_then(|v| v.as_str()) + .or_else(|| pair.as_str()) + { if !dst.is_empty() && dst != key { dsts.insert(dst.to_string()); } @@ -133,12 +146,19 @@ fn import_json(root: &Path, path: &Path) -> Result { Ok(rels.len()) } -fn write_rows(root: &Path, rels: &BTreeSet, by_src: &BTreeMap>) -> Result<(), String> { +fn write_rows( + root: &Path, + rels: &BTreeSet, + by_src: &BTreeMap>, +) -> Result<(), String> { let _lock = crate::project_lock::ProjectLock::acquire(root)?; let mut session = Session::open(root)?; for rel in rels { let dsts = by_src.get(rel).cloned().unwrap_or_default(); - let edge_lines: Vec = dsts.iter().map(|dst| cache::format_edge_line(dst, 1, &[], false, false, false)).collect(); + let edge_lines: Vec = dsts + .iter() + .map(|dst| cache::format_edge_line(dst, 1, &[], false, false, false)) + .collect(); session.upsert(&FileUpdate { rel: rel.clone(), mtime_ns: 0, diff --git a/src/lib.rs b/src/lib.rs index 4341eaf..b7b04ed 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -5,16 +5,23 @@ pub mod api; pub mod artifacts; pub mod cache; mod cache_commit; +mod cycle_report; +mod daemon_svc; pub mod jsonutil; mod legacy_cache; +mod low_conf; pub mod maps_out; pub mod mcp; pub mod parse; +mod pending_line; mod proc; mod project_lock; pub mod scope; pub mod sha256; +pub mod symbols; +pub mod text_out; pub mod timeutil; +mod wiki_files; pub use api::{ cache_status, check_changes, health, list_paths, load_dependencies, load_dependents, diff --git a/src/low_conf.rs b/src/low_conf.rs new file mode 100644 index 0000000..862b946 --- /dev/null +++ b/src/low_conf.rs @@ -0,0 +1,85 @@ +//! Actionable low-confidence edge counts. +//! Updated for files `update-maps` actually parsed, so a warm skip does not rescan the tree. + +use std::collections::BTreeMap; +use std::fs; +use std::path::Path; + +use crate::cache; + +const NAME: &str = "low_conf.tsv"; + +fn path(root: &Path) -> std::path::PathBuf { + root.join(".wikifier_staging").join(NAME) +} + +fn confidence(conditional: bool, dynamic: bool, via_barrel: bool) -> f64 { + cache::confidence_of(conditional, dynamic, via_barrel) +} + +/// Resolved internal edges an agent should look at: below 0.65 and not dynamic. +pub fn count_edges(edges: &[(bool, bool, bool)]) -> i64 { + edges + .iter() + .filter(|(conditional, dynamic, via_barrel)| { + !dynamic && confidence(*conditional, *dynamic, *via_barrel) < 0.65 + }) + .count() as i64 +} + +/// Merge counts for files parsed in this run. Missing files are left as they were. +pub fn store_updates(root: &Path, updates: &[(String, i64)], removed: &[String]) { + if updates.is_empty() && removed.is_empty() && path(root).is_file() { + return; + } + let mut map = read_map(root); + for rel in removed { + map.remove(rel); + } + for (rel, count) in updates { + if *count <= 0 { + map.remove(rel); + } else { + map.insert(rel.clone(), *count); + } + } + let _ = fs::create_dir_all(root.join(".wikifier_staging")); + let mut body = String::new(); + for (rel, count) in &map { + body.push_str(rel); + body.push('\t'); + body.push_str(&count.to_string()); + body.push('\n'); + } + let _ = fs::write(path(root), body); +} + +pub fn actionable_total(root: &Path, directory: Option<&str>) -> i64 { + let prefix = directory + .map(|d| d.trim_matches('/')) + .filter(|d| !d.is_empty()); + read_map(root) + .into_iter() + .filter(|(rel, _)| { + prefix + .map(|p| rel == p || rel.starts_with(&format!("{p}/"))) + .unwrap_or(true) + }) + .map(|(_, n)| n) + .sum() +} + +fn read_map(root: &Path) -> BTreeMap { + let mut map = BTreeMap::new(); + let text = fs::read_to_string(path(root)).unwrap_or_default(); + for line in text.lines() { + if let Some((rel, count)) = line.split_once('\t') { + if let Ok(n) = count.trim().parse::() { + if n > 0 { + map.insert(rel.to_string(), n); + } + } + } + } + map +} diff --git a/src/main.rs b/src/main.rs index 01bf5cd..f936afe 100644 --- a/src/main.rs +++ b/src/main.rs @@ -44,22 +44,33 @@ fn main() -> ExitCode { continue; } if a == "--reasons" && i + 1 < args.len() { - let text = std::fs::read_to_string(&args[i + 1]).unwrap_or_default(); - let cur = flags.entry("reasons".into()).or_default(); - for line in text.lines() { - let line = line.trim(); - if line.is_empty() || line.starts_with('#') { - continue; - } - if !cur.is_empty() { - cur.push('\n'); - } - cur.push_str(line); + let spec = &args[i + 1]; + let text = if spec == "-" { + let mut body = String::new(); + let _ = std::io::Read::read_to_string(&mut std::io::stdin(), &mut body); + body + } else { + std::fs::read_to_string(spec).unwrap_or_default() + }; + let cur = flags.entry("reasons".into()).or_default(); + for line in reason_lines(&text) { + if !cur.is_empty() { + cur.push('\n'); } - i += 2; - continue; + cur.push_str(&line); + } + i += 2; + continue; } - if a == "--json" || a == "--full" || a == "--summary" || a == "--force" || a == "--stream" || a == "--quiet" || a == "--dry-run" || a == "--green" { + if a == "--json" + || a == "--full" + || a == "--summary" + || a == "--force" + || a == "--stream" + || a == "--quiet" + || a == "--dry-run" + || a == "--green" + { flags.insert(a.trim_start_matches('-').to_string(), "true".into()); i += 1; continue; @@ -104,17 +115,57 @@ fn main() -> ExitCode { // aliases already listed; unknown falls through to dispatch } let value = api::dispatch(&command, &root, &positional, &flags); - let success = value.get("success").and_then(|s| s.as_bool()).unwrap_or(false); - flush_stdout(&format!("{value}\n")); + let success = value + .get("success") + .and_then(|s| s.as_bool()) + .unwrap_or(false); + let textual = flags + .get("text") + .map(|s| s == "true" || s == "1") + .unwrap_or(false) + || (command == "health" && flags.get("format").map(|s| s == "text").unwrap_or(false)); + if textual { + if let Some(prose) = wikifier::text_out::render(&command, &value) { + flush_stdout(&prose); + } else { + flush_stdout(&format!("{value}\n")); + } + } else { + flush_stdout(&format!("{value}\n")); + } if success { ExitCode::SUCCESS - } else if value.get("error_type").and_then(|s| s.as_str()) == Some("ScopeError") && command == "autonomous-status" { + } else if value.get("error_type").and_then(|s| s.as_str()) == Some("ScopeError") + && command == "autonomous-status" + { ExitCode::from(2) } else { ExitCode::from(1) } } +fn reason_lines(text: &str) -> Vec { + let trimmed = text.trim(); + if trimmed.starts_with('{') { + if let Ok(wikifier::Json::Object(map)) = wikifier::jsonutil::parse(trimmed) { + return map + .into_iter() + .filter_map(|(key, value)| value.as_str().map(|reason| format!("{key}={reason}"))) + .collect(); + } + } + text.lines() + .filter_map(|line| { + let line = line.trim(); + if line.is_empty() || line.starts_with('#') { + None + } else { + Some(line.to_string()) + } + }) + .collect() +} + fn extract_target(args: &[String]) -> (Option, bool, Vec) { let mut root = env::var("WIKIFIER_PROJECT_ROOT").ok().map(PathBuf::from); let mut explicit = false; @@ -128,7 +179,10 @@ fn extract_target(args: &[String]) -> (Option, bool, Vec) { i += 2; continue; } - if let Some(v) = a.strip_prefix("--target=").or_else(|| a.strip_prefix("--project-root=")) { + if let Some(v) = a + .strip_prefix("--target=") + .or_else(|| a.strip_prefix("--project-root=")) + { root = Some(PathBuf::from(v)); explicit = true; i += 1; @@ -145,6 +199,10 @@ fn print_help() { names.sort(); flush_stdout(&format!( "wikifier {VERSION}\n\nUsage: wikifier [--target DIR] [args]\n\nCommands:\n{}\n", - names.iter().map(|n| format!(" {n}")).collect::>().join("\n") + names + .iter() + .map(|n| format!(" {n}")) + .collect::>() + .join("\n") )); } diff --git a/src/maps_out.rs b/src/maps_out.rs index fe818e2..b0c3302 100644 --- a/src/maps_out.rs +++ b/src/maps_out.rs @@ -29,9 +29,18 @@ pub fn write(root: &Path) -> Result<(), String> { Ok(()) } -fn write_library(root: &Path, paths: &[String], edges: &[(String, String)], flagged: i64) -> Result<(), String> { +fn write_library( + root: &Path, + paths: &[String], + edges: &[(String, String)], + flagged: i64, +) -> Result<(), String> { let mut md = String::from("# Library\n\nGenerated by `wikifier update-maps`. Agents orient with the root folder card and `list-paths`, not this mermaid.\n\n"); - md.push_str(&format!("Files: {}. Edges: {}.\n\n", paths.len(), edges.len())); + md.push_str(&format!( + "Files: {}. Edges: {}.\n\n", + paths.len(), + edges.len() + )); md.push_str("## File Tree\n\n```text\n"); for path in paths { md.push_str(path); @@ -44,11 +53,18 @@ fn write_library(root: &Path, paths: &[String], edges: &[(String, String)], flag for (src, dst) in edges.iter().take(shown) { let s = node_id(&mut ids, src); let d = node_id(&mut ids, dst); - md.push_str(&format!(" n{s}[\"{}\"] --> n{d}[\"{}\"]\n", esc(src), esc(dst))); + md.push_str(&format!( + " n{s}[\"{}\"] --> n{d}[\"{}\"]\n", + esc(src), + esc(dst) + )); } md.push_str("```\n\n"); if edges.len() > MERMAID_CAP { - md.push_str(&format!("Mermaid shows the first {MERMAID_CAP} edges of {}.\n\n", edges.len())); + md.push_str(&format!( + "Mermaid shows the first {MERMAID_CAP} edges of {}.\n\n", + edges.len() + )); } md.push_str("| Source | Depends on |\n|--------|------------|\n"); let mut by_src: BTreeMap<&str, Vec<&str>> = BTreeMap::new(); @@ -58,7 +74,9 @@ fn write_library(root: &Path, paths: &[String], edges: &[(String, String)], flag for (src, dsts) in &by_src { md.push_str(&format!("| {src} | {} |\n", dsts.join(", "))); } - md.push_str(&format!("\nEdges with a conditional, dynamic, barrel, or lower-confidence flag: {flagged}.\n")); + md.push_str(&format!( + "\nEdges with a conditional, dynamic, barrel, or lower-confidence flag: {flagged}.\n" + )); md.push_str("Run `wikifier cycles` for a break hint on each strongly connected component.\n"); fs::write(root.join("library.md"), md).map_err(|e| e.to_string()) } @@ -134,9 +152,31 @@ fn folder_json(prefix: &str, paths: &[String]) -> Json { direct.sort(); Json::obj(vec![ ("prefix", Json::String(pref.into())), - ("files", Json::Array(direct.iter().cloned().map(Json::String).collect())), - ("dirs", Json::Array(dirs.into_iter().map(|(p, n)| Json::obj(vec![("p", Json::String(p)), ("n", Json::Int(n))])).collect())), - ("file_count", Json::Int(paths.iter().filter(|p| pref.is_empty() || p.as_str() == pref || p.starts_with(&(pref.to_string() + "/"))).count() as i64)), + ( + "files", + Json::Array(direct.iter().cloned().map(Json::String).collect()), + ), + ( + "dirs", + Json::Array( + dirs.into_iter() + .map(|(p, n)| Json::obj(vec![("p", Json::String(p)), ("n", Json::Int(n))])) + .collect(), + ), + ), + ( + "file_count", + Json::Int( + paths + .iter() + .filter(|p| { + pref.is_empty() + || p.as_str() == pref + || p.starts_with(&(pref.to_string() + "/")) + }) + .count() as i64, + ), + ), ]) } @@ -147,6 +187,11 @@ pub fn write_status(root: &Path, body: &Json) -> Result<(), String> { } pub fn read_status(root: &Path) -> Json { - let path = root.join(".wikifier_staging").join("incremental_status.json"); - fs::read_to_string(path).ok().and_then(|text| crate::jsonutil::parse(&text).ok()).unwrap_or(Json::obj(vec![])) + let path = root + .join(".wikifier_staging") + .join("incremental_status.json"); + fs::read_to_string(path) + .ok() + .and_then(|text| crate::jsonutil::parse(&text).ok()) + .unwrap_or(Json::obj(vec![])) } diff --git a/src/mcp.rs b/src/mcp.rs index 524fd1d..0067809 100644 --- a/src/mcp.rs +++ b/src/mcp.rs @@ -9,6 +9,66 @@ use std::time::Duration; use crate::api::{self, MCP_TOOLS}; use crate::jsonutil::{self, Json}; +fn extra_props(name: &str) -> Vec<(&'static str, Json)> { + let boolean = |description: &str| { + Json::obj(vec![ + ("type", Json::String("boolean".into())), + ("description", Json::String(description.into())), + ]) + }; + let integer = |description: &str| { + Json::obj(vec![ + ("type", Json::String("integer".into())), + ("description", Json::String(description.into())), + ]) + }; + let mut props = vec![ + ( + "timeout_s", + integer( + "Stop this call after this many seconds. Falls back to WIKIFIER_MCP_DEADLINE_S.", + ), + ), + ( + "format", + string_prop( + "text or json. json is the default. text fills the content string with prose.", + ), + ), + ( + "directory", + string_prop("Limit the result to this project-relative prefix."), + ), + ]; + match name { + "get_dependencies" => props.extend([ + ("detail", string_prop("compact or full. full adds confidence reasons.")), + ("low_confidence_only", boolean("Keep resolved edges below 0.65 that are not dynamic.")), + ("unresolved_only", boolean("Keep only imports that did not resolve to a project file.")), + ]), + "list_paths" => props.extend([ + ("depth", integer("1 is the folder card. 0 lists every file under the prefix.")), + ("recursive", boolean("Same as depth 0.")), + ("max_files", integer("Cap the returned file list.")), + ]), + "get_cycles" => props.extend([ + ("max_items", integer("Cap the cycle list.")), + ("analysis", boolean("Include severity and reverse-dependency ranking. Default true.")), + ]), + "verify_wikis" | "health" | "suggest_next_actions" | "session_bootstrap" | "get_project_status" => { + props.push(("limit", integer("Cap listed failures or stale rows."))); + } + "record_changes" => props.push(("reasons", Json::obj(vec![ + ("type", Json::String("object".into())), + ("description", Json::String("Map of path, glob, or dir/ to a reason. A string array of PATH=REASON lines also works.".into())), + ]))), + "heal_stubs" | "list_healable_stubs" => props.push(("min_wiki_length", integer("Minimum wiki length. Default 350."))), + "mark_green" => props.push(("force", boolean("Override a failed wiki check. Requires a reason."))), + _ => {} + } + props +} + fn string_prop(description: &str) -> Json { Json::obj(vec![ ("type", Json::String("string".into())), @@ -16,47 +76,189 @@ fn string_prop(description: &str) -> Json { ]) } -fn tool_spec(name: &str) -> (&'static str, &'static [&'static str], &'static [&'static str]) { +fn tool_spec( + name: &str, +) -> ( + &'static str, + &'static [&'static str], + &'static [&'static str], +) { match name { - "check_changes" => ("Compare mapped files and Green baselines and report content drift.", &["project_root"], &[]), - "record_change" => ("Record a semantic edit against one file and mark that file actionable.", &["project_root", "file", "reason"], &[]), - "record_deletion" => ("Record that a mapped file was removed and should leave the health matrix.", &["project_root", "file", "reason"], &[]), - "mark_green" => ("Mark a file Green after its wiki names the file's public symbols.", &["project_root", "file"], &["reason"]), - "record_changes" => ("Record git changes in one batch. Pass per-file reasons when you have them.", &["project_root"], &[]), - "verify_wikis" => ("Check every Green wiki against the public symbols currently in the source.", &["project_root"], &[]), - "prepare_edit" => ("Look up health, wiki text, dependencies, and dependents before editing a file.", &["project_root", "file"], &[]), - "session_bootstrap" => ("Return readiness, health, suggested actions, and the mapped file list for this root.", &["project_root"], &[]), - "suggest_next_actions" => ("Return the next agent actions, highest priority first.", &["project_root"], &[]), - "list_core_tools" => ("List the daily core tools and the full MCP tool catalogue.", &["project_root"], &[]), - "list_paths" => ("List mapped files under a project-relative prefix.", &["project_root"], &["prefix"]), - "search_journal" => ("Search journal entries by file path or reason text.", &["project_root", "query"], &["text"]), - "why_file" => ("Return the journal reasons recorded for one file.", &["project_root", "file"], &[]), - "get_dependencies" => ("Return internal edges, external imports, and unresolved imports for one file.", &["project_root", "file"], &[]), - "get_dependents" => ("Return the mapped files that import a given file.", &["project_root", "file"], &[]), - "get_cycles" => ("Return import cycles in the mapped graph.", &["project_root"], &[]), - "get_resolution_diagnostics" => ("Return resolution and barrel diagnostics for the current map.", &["project_root"], &[]), - "get_file_wiki" => ("Return the wiki markdown stored beside a source file.", &["project_root", "file"], &[]), - "get_files_needing_attention" => ("Return files whose health is Red or an actionable Yellow.", &["project_root"], &[]), - "get_project_status" => ("Return the same readiness snapshot as session bootstrap.", &["project_root"], &[]), - "get_current_project_root" => ("Return the absolute project root this call is using.", &["project_root"], &[]), - "get_barrel_reports" => ("Return barrel and incremental-resolution status for the current map.", &["project_root"], &[]), - "get_incremental_status" => ("Return whether the map is incremental and which barrels are recorded.", &["project_root"], &[]), - "update_maps" => ("Parse the scoped tree and commit one cache generation for the whole run.", &["project_root"], &[]), - "health" => ("Return the health matrix entries and the green, yellow, and red counts.", &["project_root"], &[]), - "validate" => ("Check health coverage and that the forward and reverse cache indexes agree.", &["project_root"], &[]), - "list_healable_stubs" => ("List Initial-stub files whose wiki is long enough to heal.", &["project_root"], &[]), - "heal_stubs" => ("Promote Initial stubs that already have a long enough wiki.", &["project_root"], &[]), - "seed_source_content_hashes" => ("Seed health rows from the current map without rewriting wikis.", &["project_root"], &[]), - "journal" => ("Return journal entries, optionally for one date.", &["project_root"], &[]), - "issues" => ("Return the same prioritized actions as suggest_next_actions.", &["project_root"], &[]), - _ => ("Run one Wikifier agent operation against project_root.", &["project_root"], &[]), + "check_changes" => ( + "Compare mapped files and Green baselines and report content drift.", + &["project_root"], + &[], + ), + "record_change" => ( + "Record a semantic edit against one file and mark that file actionable.", + &["project_root", "file", "reason"], + &[], + ), + "record_deletion" => ( + "Record that a mapped file was removed and should leave the health matrix.", + &["project_root", "file", "reason"], + &[], + ), + "mark_green" => ( + "Mark a file Green after its wiki names the file's public symbols.", + &["project_root", "file"], + &["reason"], + ), + "record_changes" => ( + "Record git changes in one batch. Pass per-file reasons when you have them.", + &["project_root"], + &[], + ), + "verify_wikis" => ( + "Check every Green wiki against the public symbols currently in the source.", + &["project_root"], + &[], + ), + "prepare_edit" => ( + "Look up health, wiki text, dependencies, and dependents before editing a file.", + &["project_root", "file"], + &[], + ), + "session_bootstrap" => ( + "Return readiness, health, suggested actions, and the mapped file list for this root.", + &["project_root"], + &[], + ), + "suggest_next_actions" => ( + "Return the next agent actions, highest priority first.", + &["project_root"], + &[], + ), + "list_core_tools" => ( + "List the daily core tools and the full MCP tool catalogue.", + &["project_root"], + &[], + ), + "list_paths" => ( + "List mapped files under a project-relative prefix.", + &["project_root"], + &["prefix"], + ), + "search_journal" => ( + "Search journal entries by file path or reason text.", + &["project_root", "query"], + &["text"], + ), + "why_file" => ( + "Return the journal reasons recorded for one file.", + &["project_root", "file"], + &[], + ), + "get_dependencies" => ( + "Return internal edges, external imports, and unresolved imports for one file.", + &["project_root", "file"], + &[], + ), + "get_dependents" => ( + "Return the mapped files that import a given file.", + &["project_root", "file"], + &[], + ), + "get_cycles" => ( + "Return import cycles in the mapped graph.", + &["project_root"], + &[], + ), + "get_resolution_diagnostics" => ( + "Return resolution and barrel diagnostics for the current map.", + &["project_root"], + &[], + ), + "get_file_wiki" => ( + "Return the wiki markdown stored beside a source file.", + &["project_root", "file"], + &[], + ), + "get_files_needing_attention" => ( + "Return files whose health is Red or an actionable Yellow.", + &["project_root"], + &[], + ), + "get_project_status" => ( + "Return the same readiness snapshot as session bootstrap.", + &["project_root"], + &[], + ), + "get_current_project_root" => ( + "Return the absolute project root this call is using.", + &["project_root"], + &[], + ), + "get_barrel_reports" => ( + "Return barrel and incremental-resolution status for the current map.", + &["project_root"], + &[], + ), + "get_incremental_status" => ( + "Return whether the map is incremental and which barrels are recorded.", + &["project_root"], + &[], + ), + "update_maps" => ( + "Parse the scoped tree and commit one cache generation for the whole run.", + &["project_root"], + &[], + ), + "health" => ( + "Return the health matrix entries and the green, yellow, and red counts.", + &["project_root"], + &[], + ), + "validate" => ( + "Check health coverage and that the forward and reverse cache indexes agree.", + &["project_root"], + &[], + ), + "list_healable_stubs" => ( + "List Initial-stub files whose wiki is long enough to heal.", + &["project_root"], + &[], + ), + "heal_stubs" => ( + "Promote Initial stubs that already have a long enough wiki.", + &["project_root"], + &[], + ), + "seed_source_content_hashes" => ( + "Seed health rows from the current map without rewriting wikis.", + &["project_root"], + &[], + ), + "journal" => ( + "Return journal entries, optionally for one date.", + &["project_root"], + &[], + ), + "issues" => ( + "Return the same prioritized actions as suggest_next_actions.", + &["project_root"], + &[], + ), + _ => ( + "Run one Wikifier agent operation against project_root.", + &["project_root"], + &[], + ), } } pub fn tool_schema(name: &str) -> Json { let (description, required, optional) = tool_spec(name); - let mut props = vec![("project_root".into(), string_prop("Absolute project root. Pass it on every call."))]; - let mut fields: Vec<&str> = required.iter().copied().chain(optional.iter().copied()).filter(|field| *field != "project_root").collect(); + let mut props = vec![( + "project_root".into(), + string_prop("Absolute project root. Pass it on every call."), + )]; + let mut fields: Vec<&str> = required + .iter() + .copied() + .chain(optional.iter().copied()) + .filter(|field| *field != "project_root") + .collect(); fields.sort(); fields.dedup(); for field in fields { @@ -70,14 +272,28 @@ pub fn tool_schema(name: &str) -> Json { }; props.push((field.into(), string_prop(desc))); } + for (field, schema) in extra_props(name) { + props.push((field.into(), schema)); + } Json::obj(vec![ ("name", Json::String(name.into())), ("description", Json::String(description.into())), - ("inputSchema", Json::obj(vec![ - ("type", Json::String("object".into())), - ("properties", Json::from_pairs(props)), - ("required", Json::Array(required.iter().map(|field| Json::String((*field).into())).collect())), - ])), + ( + "inputSchema", + Json::obj(vec![ + ("type", Json::String("object".into())), + ("properties", Json::from_pairs(props)), + ( + "required", + Json::Array( + required + .iter() + .map(|field| Json::String((*field).into())) + .collect(), + ), + ), + ]), + ), ]) } @@ -88,31 +304,63 @@ pub fn handle_message(msg: &Json) -> Option { let result = match method { "initialize" => Json::obj(vec![ ("protocolVersion", Json::String("2024-11-05".into())), - ("capabilities", Json::obj(vec![ - ("tools", Json::obj(vec![])), - ("resources", Json::obj(vec![])), - ("prompts", Json::obj(vec![])), - ])), - ("serverInfo", Json::obj(vec![ - ("name", Json::String("wikifier".into())), - ("version", Json::String(api::VERSION.into())), - ])), - ]), - "tools/list" => Json::obj(vec![ - ("tools", Json::Array(MCP_TOOLS.iter().map(|n| tool_schema(n)).collect())), - ]), - "resources/list" => Json::obj(vec![ - ("resources", Json::Array(["wikifier://library", "wikifier://health", "wikifier://pending", "wikifier://journal"].into_iter().map(|u| { - Json::obj(vec![("uri", Json::String(u.into())), ("name", Json::String(u.into()))]) - }).collect())), - ]), - "prompts/list" => Json::obj(vec![ - ("prompts", Json::Array([ - "review_pending_changes", "audit_project_health", "plan_refactoring", - "find_architectural_smells", "understand_codebase_structure", "review_recent_changes", - "generate_project_health_report", "onboard_to_module", - ].into_iter().map(|n| Json::obj(vec![("name", Json::String(n.into()))])).collect())), + ( + "capabilities", + Json::obj(vec![ + ("tools", Json::obj(vec![])), + ("resources", Json::obj(vec![])), + ("prompts", Json::obj(vec![])), + ]), + ), + ( + "serverInfo", + Json::obj(vec![ + ("name", Json::String("wikifier".into())), + ("version", Json::String(api::VERSION.into())), + ]), + ), ]), + "tools/list" => Json::obj(vec![( + "tools", + Json::Array(MCP_TOOLS.iter().map(|n| tool_schema(n)).collect()), + )]), + "resources/list" => Json::obj(vec![( + "resources", + Json::Array( + [ + "wikifier://library", + "wikifier://health", + "wikifier://pending", + "wikifier://journal", + ] + .into_iter() + .map(|u| { + Json::obj(vec![ + ("uri", Json::String(u.into())), + ("name", Json::String(u.into())), + ]) + }) + .collect(), + ), + )]), + "prompts/list" => Json::obj(vec![( + "prompts", + Json::Array( + [ + "review_pending_changes", + "audit_project_health", + "plan_refactoring", + "find_architectural_smells", + "understand_codebase_structure", + "review_recent_changes", + "generate_project_health_report", + "onboard_to_module", + ] + .into_iter() + .map(|n| Json::obj(vec![("name", Json::String(n.into()))])) + .collect(), + ), + )]), "resources/read" => resource_read(msg), "prompts/get" => prompt_get(msg), "tools/call" => call_tool(msg), @@ -131,32 +379,104 @@ pub fn handle_message(msg: &Json) -> Option { fn call_tool(msg: &Json) -> Json { let params = msg.get("params"); - let name = params.and_then(|p| p.get("name")).and_then(|n| n.as_str()).unwrap_or("").to_string(); + let name = params + .and_then(|p| p.get("name")) + .and_then(|n| n.as_str()) + .unwrap_or("") + .to_string(); let args = params.and_then(|p| p.get("arguments")).cloned(); - let root = args.as_ref().and_then(|a| a.get("project_root")).and_then(|r| r.as_str()).map(PathBuf::from) - .or_else(|| std::env::var("WIKIFIER_PROJECT_ROOT").ok().map(PathBuf::from)) + let root = args + .as_ref() + .and_then(|a| a.get("project_root")) + .and_then(|r| r.as_str()) + .map(PathBuf::from) + .or_else(|| { + std::env::var("WIKIFIER_PROJECT_ROOT") + .ok() + .map(PathBuf::from) + }) .unwrap_or_else(|| std::env::current_dir().unwrap_or_else(|_| PathBuf::from("."))); let mut flags = BTreeMap::new(); let mut positional = Vec::new(); - if let Some(file) = args.as_ref().and_then(|a| a.get("file")).and_then(|f| f.as_str()) { + if let Some(file) = args + .as_ref() + .and_then(|a| a.get("file")) + .and_then(|f| f.as_str()) + { positional.push(file.to_string()); } - if let Some(reason) = args.as_ref().and_then(|a| a.get("reason")).and_then(|f| f.as_str()) { + if let Some(reason) = args + .as_ref() + .and_then(|a| a.get("reason")) + .and_then(|f| f.as_str()) + { positional.push(reason.to_string()); } copy_str(&args, &mut flags, "prefix"); - if let Some(q) = args.as_ref().and_then(|a| a.get("query").or_else(|| a.get("text"))).and_then(|f| f.as_str()) { + if let Some(q) = args + .as_ref() + .and_then(|a| a.get("query").or_else(|| a.get("text"))) + .and_then(|f| f.as_str()) + { flags.insert("query".into(), q.into()); } copy_str(&args, &mut flags, "directory"); copy_str(&args, &mut flags, "since"); copy_str(&args, &mut flags, "default"); - if let Some(max) = args.as_ref().and_then(|a| a.get("max_files").or_else(|| a.get("max-files"))) { + copy_str(&args, &mut flags, "detail"); + copy_str(&args, &mut flags, "format"); + copy_str(&args, &mut flags, "prefix"); + if let Some(depth) = args.as_ref().and_then(|a| a.get("depth")) { + if let Some(text) = json_text(depth) { + flags.insert("depth".into(), text); + } + } + if let Some(limit) = args + .as_ref() + .and_then(|a| a.get("limit").or_else(|| a.get("max_items"))) + { + if let Some(text) = json_text(limit) { + flags.insert("limit".into(), text.clone()); + flags.insert("max-items".into(), text); + } + } + if let Some(min) = args.as_ref().and_then(|a| { + a.get("min_wiki_length") + .or_else(|| a.get("min-wiki-length")) + }) { + if let Some(text) = json_text(min) { + flags.insert("min-wiki-length".into(), text); + } + } + if arg_on(args.as_ref(), "recursive") { + flags.insert("recursive".into(), "true".into()); + } + if arg_on(args.as_ref(), "low_confidence_only") || arg_on(args.as_ref(), "low-confidence-only") + { + flags.insert("low-confidence-only".into(), "true".into()); + } + if arg_on(args.as_ref(), "unresolved_only") || arg_on(args.as_ref(), "unresolved-only") { + flags.insert("unresolved-only".into(), "true".into()); + } + if args + .as_ref() + .and_then(|a| a.get("analysis")) + .and_then(|v| v.as_bool()) + == Some(false) + { + flags.insert("no-analysis".into(), "true".into()); + } + if let Some(max) = args + .as_ref() + .and_then(|a| a.get("max_files").or_else(|| a.get("max-files"))) + { if let Some(text) = json_text(max) { flags.insert("max-files".into(), text); } } - for key in ["full", "quiet", "stream", "green", "force", "dry_run", "dry-run"] { + for key in [ + "full", "quiet", "stream", "green", "force", "dry_run", "dry-run", + ] { if arg_on(args.as_ref(), key) { let stored = if key == "dry_run" { "dry-run" } else { key }; flags.insert(stored.into(), "true".into()); @@ -168,7 +488,11 @@ fn call_tool(msg: &Json) -> Json { flags.insert("only".into(), text.clone()); } Json::Array(items) => { - let joined = items.iter().filter_map(|v| v.as_str()).collect::>().join("\n"); + let joined = items + .iter() + .filter_map(|v| v.as_str()) + .collect::>() + .join("\n"); flags.insert("only".into(), joined); } _ => {} @@ -180,13 +504,45 @@ fn call_tool(msg: &Json) -> Json { flags.insert("reasons".into(), text.clone()); } Json::Array(items) => { - let joined = items.iter().filter_map(|v| v.as_str()).collect::>().join("\n"); + let joined = items + .iter() + .filter_map(|v| v.as_str()) + .collect::>() + .join("\n"); + flags.insert("reasons".into(), joined); + } + Json::Object(map) => { + let joined = map + .iter() + .filter_map(|(key, value)| { + value.as_str().map(|reason| format!("{key}={reason}")) + }) + .collect::>() + .join("\n"); flags.insert("reasons".into(), joined); } _ => {} } } - let deadline = std::env::var("WIKIFIER_MCP_DEADLINE_S").ok().and_then(|s| s.parse::().ok()).unwrap_or(60).max(1); + let textual = flags.get("format").map(|s| s == "text").unwrap_or(false); + let tool_name = name.clone(); + let from_arg = args + .as_ref() + .and_then(|a| a.get("timeout_s")) + .and_then(|value| match value { + Json::Int(n) if *n > 0 => Some(*n as u64), + Json::Float(n) if *n > 0.0 => Some(*n as u64), + Json::String(text) => text.parse::().ok().filter(|n| *n > 0), + _ => None, + }); + let deadline = from_arg + .or_else(|| { + std::env::var("WIKIFIER_MCP_DEADLINE_S") + .ok() + .and_then(|s| s.parse().ok()) + }) + .unwrap_or(60) + .max(1); let (tx, rx) = mpsc::channel(); std::thread::spawn(move || { let value = if !root.is_dir() { @@ -204,29 +560,44 @@ fn call_tool(msg: &Json) -> Json { Ok(value) => value, Err(_) => Json::obj(vec![ ("success", Json::Bool(false)), - ("error", Json::String(format!("mcp call exceeded {deadline}s deadline"))), + ( + "error", + Json::String(format!("mcp call exceeded {deadline}s deadline")), + ), ("error_type", Json::String("DeadlineExceeded".into())), ("timed_out", Json::Bool(true)), ("may_still_complete", Json::Bool(true)), ]), }; - tool_result(value) + tool_result(value, textual, &tool_name) } -fn tool_result(value: Json) -> Json { +fn tool_result(value: Json, textual: bool, tool_name: &str) -> Json { let is_error = value.get("success").and_then(|s| s.as_bool()) == Some(false); + let text = if textual { + crate::text_out::render(tool_name, &value).unwrap_or_else(|| value.to_string()) + } else { + value.to_string() + }; Json::obj(vec![ - ("content", Json::Array(vec![Json::obj(vec![ - ("type", Json::String("text".into())), - ("text", Json::String(value.to_string())), - ])])), + ( + "content", + Json::Array(vec![Json::obj(vec![ + ("type", Json::String("text".into())), + ("text", Json::String(text)), + ])]), + ), ("structuredContent", value), ("isError", Json::Bool(is_error)), ]) } fn copy_str(args: &Option, flags: &mut BTreeMap, key: &str) { - if let Some(text) = args.as_ref().and_then(|a| a.get(key)).and_then(|v| v.as_str()) { + if let Some(text) = args + .as_ref() + .and_then(|a| a.get(key)) + .and_then(|v| v.as_str()) + { flags.insert(key.into(), text.into()); } } @@ -249,42 +620,68 @@ fn arg_on(args: Option<&Json>, key: &str) -> bool { fn resource_read(msg: &Json) -> Json { let params = msg.get("params"); - let uri = params.and_then(|p| p.get("uri")).and_then(|u| u.as_str()).unwrap_or(""); - let root = params.and_then(|p| p.get("project_root")).and_then(|r| r.as_str()).map(PathBuf::from) - .or_else(|| std::env::var("WIKIFIER_PROJECT_ROOT").ok().map(PathBuf::from)) + let uri = params + .and_then(|p| p.get("uri")) + .and_then(|u| u.as_str()) + .unwrap_or(""); + let root = params + .and_then(|p| p.get("project_root")) + .and_then(|r| r.as_str()) + .map(PathBuf::from) + .or_else(|| { + std::env::var("WIKIFIER_PROJECT_ROOT") + .ok() + .map(PathBuf::from) + }) .unwrap_or_else(|| std::env::current_dir().unwrap_or_else(|_| PathBuf::from("."))); let text = match uri { - "wikifier://library" => std::fs::read_to_string(root.join("library.md")).unwrap_or_default(), - "wikifier://health" => std::fs::read_to_string(root.join("file_health.md")).unwrap_or_default(), - "wikifier://pending" => std::fs::read_to_string(root.join("pending_updates.md")).unwrap_or_default(), + "wikifier://library" => { + std::fs::read_to_string(root.join("library.md")).unwrap_or_default() + } + "wikifier://health" => { + std::fs::read_to_string(root.join("file_health.md")).unwrap_or_default() + } + "wikifier://pending" => { + std::fs::read_to_string(root.join("pending_updates.md")).unwrap_or_default() + } "wikifier://journal" => { let entries = api::read_journal(&root, None); let mut text = String::new(); for entry in entries { let _ = std::fmt::Write::write_fmt( &mut text, - format_args!("## [{}] {}\n**File:** {}\n**Reason:** {}\n", entry.stamp, entry.action, entry.file, entry.reason), + format_args!( + "## [{}] {}\n**File:** {}\n**Reason:** {}\n", + entry.stamp, entry.action, entry.file, entry.reason + ), ); } text } - _ => return Json::obj(vec![ - ("success", Json::Bool(false)), - ("error", Json::String(format!("unknown resource {uri}"))), - ("error_type", Json::String("ResourceNotFound".into())), - ]), + _ => { + return Json::obj(vec![ + ("success", Json::Bool(false)), + ("error", Json::String(format!("unknown resource {uri}"))), + ("error_type", Json::String("ResourceNotFound".into())), + ]) + } }; - Json::obj(vec![ - ("contents", Json::Array(vec![Json::obj(vec![ + Json::obj(vec![( + "contents", + Json::Array(vec![Json::obj(vec![ ("uri", Json::String(uri.into())), ("mimeType", Json::String("text/markdown".into())), ("text", Json::String(text)), - ])])), - ]) + ])]), + )]) } fn prompt_get(msg: &Json) -> Json { - let name = msg.get("params").and_then(|p| p.get("name")).and_then(|n| n.as_str()).unwrap_or(""); + let name = msg + .get("params") + .and_then(|p| p.get("name")) + .and_then(|n| n.as_str()) + .unwrap_or(""); if name.is_empty() { return Json::obj(vec![ ("success", Json::Bool(false)), @@ -292,19 +689,39 @@ fn prompt_get(msg: &Json) -> Json { ("error_type", Json::String("PromptNotFound".into())), ]); } - let text = format!("Use Wikifier. Prompt {name}. Call session_bootstrap, then check_changes, then act only on Red and actionable Yellow files."); + let text = prompt_text(name).to_string(); Json::obj(vec![ ("description", Json::String(name.into())), - ("messages", Json::Array(vec![Json::obj(vec![ - ("role", Json::String("user".into())), - ("content", Json::obj(vec![ - ("type", Json::String("text".into())), - ("text", Json::String(text)), - ])), - ])])), + ( + "messages", + Json::Array(vec![Json::obj(vec![ + ("role", Json::String("user".into())), + ( + "content", + Json::obj(vec![ + ("type", Json::String("text".into())), + ("text", Json::String(text)), + ]), + ), + ])]), + ), ]) } +fn prompt_text(name: &str) -> &'static str { + match name { + "review_pending_changes" => "Review pending documentation work.\n1. session_bootstrap() for actions[].\n2. For each actionable file: prepare_edit(file), get_file_wiki(file), get_dependents(file).\n3. Refresh the wiki, then mark_green(file). Never skip record_change after code edits.", + "audit_project_health" => "Audit documentation health.\n1. get_project_status.\n2. get_files_needing_attention.\n3. search_journal for recent activity.\n4. suggest_next_actions and report priorities from actions[].", + "plan_refactoring" => "Plan a refactor of the target file.\n1. get_dependents for the blast radius.\n2. get_dependencies with detail=full and read confidence.\n3. get_cycles and use severity plus the break hint when the file is in a cycle.", + "find_architectural_smells" => "Find architectural smells.\nCycles: get_cycles and rank by severity and external_blast_radius.\nFragile edges: get_dependencies with low_confidence_only=true.\nHubs: files with many get_dependents results.", + "understand_codebase_structure" => "Onboard to this codebase.\n1. session_bootstrap, then list_paths for the folders you need.\n2. prepare_edit, get_dependencies, and get_dependents for the key files.\n3. get_cycles for structural risks.", + "review_recent_changes" => "Review recent activity.\n1. search_journal and journal for record-change entries.\n2. why_file on each changed file: is it Green again?\n3. Report files that were recorded but not marked green.", + "generate_project_health_report" => "Write a project health report.\nInclude green, yellow, and red counts, the files needing attention, the most depended-on modules, and cycle severity.", + "onboard_to_module" => "Understand one module.\n1. get_file_wiki.\n2. get_dependencies with detail=full.\n3. get_dependents.\n4. get_cycles when the file is in a cycle.\nSummarize its role, fragile edges, and blast radius.", + _ => "Use Wikifier. Call session_bootstrap, then check_changes, then act only on Red and actionable Yellow files.", + } +} + pub fn serve_stdio() -> Result<(), String> { let stdin = std::io::stdin(); let mut reader = stdin.lock(); diff --git a/src/parse.rs b/src/parse.rs index 6ab4eba..e36c2f0 100644 --- a/src/parse.rs +++ b/src/parse.rs @@ -81,7 +81,13 @@ pub fn read_utf8(path: &Path) -> (String, Option) { Ok(text) => (text, None), Err(err) => { let text = String::from_utf8_lossy(err.as_bytes()).into_owned(); - (text, Some("not valid UTF-8; parsed with lossy replacement so ASCII imports are kept".into())) + ( + text, + Some( + "not valid UTF-8; parsed with lossy replacement so ASCII imports are kept" + .into(), + ), + ) } } } @@ -107,7 +113,8 @@ pub fn scan_file_warned(path: &Path) -> (Scan, String, Option) { match parsed { Ok(scan) => (scan, text, warning), Err(_) => { - let fail = "parser failed; file skipped so the rest of the map can continue".to_string(); + let fail = + "parser failed; file skipped so the rest of the map can continue".to_string(); let warning = Some(match warning { Some(prev) => format!("{prev}; {fail}"), None => fail, @@ -140,7 +147,10 @@ pub fn resolve_project(root: &Path, files: &[PathBuf]) -> BTreeMap BTreeMap, file: &str, name: &str, depth: usize) -> Option { +fn defining_file( + resolver: &Resolver, + scans: &BTreeMap, + file: &str, + name: &str, + depth: usize, +) -> Option { if depth > 8 { return Some(file.to_string()); } @@ -208,10 +224,21 @@ impl Resolver { let ts_paths = load_ts_paths(root); let packages = load_packages(root); let rust_libs = load_rust_libs(root); - Resolver { files, ts_paths, packages, rust_libs } + Resolver { + files, + ts_paths, + packages, + rust_libs, + } } - fn resolve(&self, from_rel: &str, spec: &str, lang: &str, inline_depth: usize) -> Option { + fn resolve( + &self, + from_rel: &str, + spec: &str, + lang: &str, + inline_depth: usize, + ) -> Option { match lang { "python" => None, // python resolves inside scan_python via statements; handled below "javascript" => self.resolve_js(from_rel, spec), @@ -245,7 +272,9 @@ impl Resolver { if self.files.contains(&as_is) { return Some(as_is); } - const EXTS: &[&str] = &[".ts", ".tsx", ".js", ".jsx", ".mjs", ".cjs", ".mts", ".cts", ".d.ts"]; + const EXTS: &[&str] = &[ + ".ts", ".tsx", ".js", ".jsx", ".mjs", ".cjs", ".mts", ".cts", ".d.ts", + ]; for ext in EXTS { let cand = format!("{as_is}{ext}"); if self.files.contains(&cand) { @@ -267,7 +296,11 @@ impl Resolver { if let Some(rest) = match_star(pattern, spec) { let mut target = replacement.replace('*', &rest); if !target.starts_with('.') && !target.starts_with('/') { - target = format!("{}/{}", base.trim_end_matches('/'), target.trim_start_matches('/')); + target = format!( + "{}/{}", + base.trim_end_matches('/'), + target.trim_start_matches('/') + ); } target = normalize(Path::new(&target)); if let Some(hit) = self.try_extensions(&target) { @@ -286,9 +319,26 @@ impl Resolver { if spec != pkg.name && !spec.starts_with(&(pkg.name.clone() + "/")) { continue; } - let sub = if spec == pkg.name { ".".to_string() } else { format!("./{}", &spec[pkg.name.len() + 1..]) }; - let mapped = pkg.exports.iter().find(|(k, _)| k == &sub).map(|(_, v)| v.clone()) - .or_else(|| if sub == "." { pkg.exports.iter().find(|(k, _)| k == ".").map(|(_, v)| v.clone()) } else { None }); + let sub = if spec == pkg.name { + ".".to_string() + } else { + format!("./{}", &spec[pkg.name.len() + 1..]) + }; + let mapped = pkg + .exports + .iter() + .find(|(k, _)| k == &sub) + .map(|(_, v)| v.clone()) + .or_else(|| { + if sub == "." { + pkg.exports + .iter() + .find(|(k, _)| k == ".") + .map(|(_, v)| v.clone()) + } else { + None + } + }); let rel_target = if let Some(m) = mapped { normalize(&Path::new(&pkg.dir).join(m.trim_start_matches("./"))) } else { @@ -325,14 +375,21 @@ impl Resolver { } fn resolve_rust_path(&self, from_rel: &str, spec: &str, inline_depth: usize) -> Option { - let segments: Vec<&str> = spec.split("::").map(str::trim).filter(|part| !part.is_empty() && *part != "*").collect(); + let segments: Vec<&str> = spec + .split("::") + .map(str::trim) + .filter(|part| !part.is_empty() && *part != "*") + .collect(); if segments.is_empty() { return None; } let head = segments[0]; if head == "crate" { let root = self.file_crate_root(from_rel); - let dir = root.as_deref().map(rust_submodule_dir).unwrap_or_else(|| PathBuf::from("src")); + let dir = root + .as_deref() + .map(rust_submodule_dir) + .unwrap_or_else(|| PathBuf::from("src")); return self.rust_walk(&dir, &segments[1..], root.as_deref()); } if let Some(lib) = self.lib_by_name(head) { @@ -459,17 +516,25 @@ impl Resolver { } fn lib_by_name(&self, name: &str) -> Option<&RustLib> { - self.rust_libs.iter().find(|lib| lib.exported && lib.names.iter().any(|have| have == name)) + self.rust_libs + .iter() + .find(|lib| lib.exported && lib.names.iter().any(|have| have == name)) } fn package_for(&self, from_rel: &str) -> Option<&RustLib> { let mut best: Option<&RustLib> = None; for lib in &self.rust_libs { - let matches = lib.dir.is_empty() || from_rel == lib.dir || from_rel.starts_with(&(lib.dir.clone() + "/")); + let matches = lib.dir.is_empty() + || from_rel == lib.dir + || from_rel.starts_with(&(lib.dir.clone() + "/")); if !matches { continue; } - if best.as_ref().map(|have| lib.dir.len() > have.dir.len()).unwrap_or(true) { + if best + .as_ref() + .map(|have| lib.dir.len() > have.dir.len()) + .unwrap_or(true) + { best = Some(lib); } } @@ -477,13 +542,20 @@ impl Resolver { } fn file_crate_root(&self, from_rel: &str) -> Option { - let package_dir = self.package_for(from_rel).map(|lib| lib.dir.clone()).unwrap_or_default(); + let package_dir = self + .package_for(from_rel) + .map(|lib| lib.dir.clone()) + .unwrap_or_default(); if let Some(root) = separate_in_package(from_rel, &package_dir) { if self.files.contains(&root) { return Some(root); } } - let main_rs = if package_dir.is_empty() { "src/main.rs".to_string() } else { format!("{package_dir}/src/main.rs") }; + let main_rs = if package_dir.is_empty() { + "src/main.rs".to_string() + } else { + format!("{package_dir}/src/main.rs") + }; if from_rel == main_rs { return Some(main_rs); } @@ -504,12 +576,19 @@ impl Resolver { return Some(exact); } let prefix = format!("{joined}/"); - return self.files.iter().find(|f| f.starts_with(&prefix) && f.ends_with(".go")).cloned(); + return self + .files + .iter() + .find(|f| f.starts_with(&prefix) && f.ends_with(".go")) + .cloned(); } let tail = spec.rsplit('/').next().unwrap_or(spec); let exact = format!("{tail}.go"); let suffix = format!("/{tail}.go"); - self.files.iter().find(|f| f.ends_with(&suffix) || f.as_str() == exact).cloned() + self.files + .iter() + .find(|f| f.ends_with(&suffix) || f.as_str() == exact) + .cloned() } fn resolve_c(&self, from_rel: &str, spec: &str) -> Option { @@ -525,7 +604,10 @@ impl Resolver { } } let name = Path::new(spec).file_name()?.to_string_lossy().to_string(); - self.files.iter().find(|f| f.ends_with(&format!("/{name}")) || f.as_str() == name).cloned() + self.files + .iter() + .find(|f| f.ends_with(&format!("/{name}")) || f.as_str() == name) + .cloned() } fn resolve_java(&self, spec: &str) -> Option { @@ -535,14 +617,20 @@ impl Resolver { } let suffix = format!("/{class}.java"); let exact = format!("{class}.java"); - self.files.iter().find(|f| f.ends_with(&suffix) || f.as_str() == exact).cloned() + self.files + .iter() + .find(|f| f.ends_with(&suffix) || f.as_str() == exact) + .cloned() } fn resolve_csharp(&self, spec: &str) -> Option { let name = spec.split('.').next_back()?.trim(); let suffix = format!("/{name}.cs"); let exact = format!("{name}.cs"); - self.files.iter().find(|f| f.ends_with(&suffix) || f.as_str() == exact).cloned() + self.files + .iter() + .find(|f| f.ends_with(&suffix) || f.as_str() == exact) + .cloned() } } @@ -580,10 +668,22 @@ fn load_ts_paths(root: &Path) -> Vec<(String, String, String)> { if text.is_empty() { return Vec::new(); } - let Ok(json) = jsonutil::parse_jsonc(&text) else { return Vec::new() }; - let base = json.get("compilerOptions").and_then(|c| c.get("baseUrl")).and_then(|b| b.as_str()).unwrap_or(".").trim_end_matches('/').to_string(); + let Ok(json) = jsonutil::parse_jsonc(&text) else { + return Vec::new(); + }; + let base = json + .get("compilerOptions") + .and_then(|c| c.get("baseUrl")) + .and_then(|b| b.as_str()) + .unwrap_or(".") + .trim_end_matches('/') + .to_string(); let mut out = Vec::new(); - let Some(paths) = json.get("compilerOptions").and_then(|c| c.get("paths")).and_then(|p| p.as_object()) else { + let Some(paths) = json + .get("compilerOptions") + .and_then(|c| c.get("paths")) + .and_then(|p| p.as_object()) + else { return out; }; for (pat, val) in paths { @@ -625,15 +725,27 @@ fn load_packages(root: &Path) -> Vec { let mut pkgs = Vec::new(); for dir in dirs { let text = fs::read_to_string(dir.join("package.json")).unwrap_or_default(); - let Ok(json) = jsonutil::parse(&text) else { continue }; - let Some(name) = json.get("name").and_then(|n| n.as_str()) else { continue }; + let Ok(json) = jsonutil::parse(&text) else { + continue; + }; + let Some(name) = json.get("name").and_then(|n| n.as_str()) else { + continue; + }; let mut exports = Vec::new(); if let Some(exp) = json.get("exports") { collect_exports(exp, &mut exports); } let rel_dir = rel_of(root, &dir); - let rel_dir = if rel_dir.is_empty() { ".".to_string() } else { rel_dir }; - pkgs.push(Package { dir: rel_dir, name: name.to_string(), exports }); + let rel_dir = if rel_dir.is_empty() { + ".".to_string() + } else { + rel_dir + }; + pkgs.push(Package { + dir: rel_dir, + name: name.to_string(), + exports, + }); } pkgs } @@ -651,7 +763,10 @@ fn collect_exports(exp: &Json, out: &mut Vec<(String, String)>) { .find_map(|key| inner.get(*key).and_then(|x| x.as_str())); if let Some(s) = pref { out.push((k.clone(), s.to_string())); - } else if let Some((_, s)) = inner.iter().find_map(|(_, val)| val.as_str().map(|s| ((), s.to_string()))) { + } else if let Some((_, s)) = inner + .iter() + .find_map(|(_, val)| val.as_str().map(|s| ((), s.to_string()))) + { out.push((k.clone(), s)); } } @@ -667,12 +782,21 @@ fn scan_python(src: &str) -> Scan { let (masked, dynamic) = mask_python(src); let mut imports = Vec::new(); for spec in dynamic { - imports.push(RawImport { spec, names: Vec::new(), kind: Kind::Dynamic, conditional: false, inline_depth: 0 }); + imports.push(RawImport { + spec, + names: Vec::new(), + kind: Kind::Dynamic, + conditional: false, + inline_depth: 0, + }); } for stmt in python_statements(&masked) { imports.push(stmt); } - Scan { imports, local_exports: Vec::new() } + Scan { + imports, + local_exports: Vec::new(), + } } fn mask_python(src: &str) -> (String, Vec) { @@ -772,7 +896,9 @@ fn recent_call(tail: &str) -> &str { } fn is_py_module_lit(s: &str) -> bool { - !s.is_empty() && s.chars().all(|c| c.is_ascii_alphanumeric() || c == '_' || c == '.') + !s.is_empty() + && s.chars() + .all(|c| c.is_ascii_alphanumeric() || c == '_' || c == '.') } fn python_quote_at(b: &[u8], i: usize) -> Option { @@ -819,7 +945,10 @@ fn python_statements(masked: &str) -> Vec { let mut i = 0; while i < lines.len() { let raw_line = lines[i]; - let indent = raw_line.chars().take_while(|c| *c == ' ' || *c == '\t').count(); + let indent = raw_line + .chars() + .take_while(|c| *c == ' ' || *c == '\t') + .count(); let line = raw_line.trim(); if line.starts_with("import ") || line.starts_with("from ") { let mut buf = line.to_string(); @@ -843,22 +972,41 @@ fn python_statements(masked: &str) -> Vec { } fn paren_delta(s: &str) -> i32 { - s.chars().fold(0, |n, c| n + match c { '(' => 1, ')' => -1, _ => 0 }) + s.chars().fold(0, |n, c| { + n + match c { + '(' => 1, + ')' => -1, + _ => 0, + } + }) } fn parse_py_stmt(stmt: &str) -> Option { let stmt = stmt.trim().trim_end_matches('\\').trim(); if let Some(rest) = stmt.strip_prefix("import ") { - let names = rest.split(',').map(|s| s.split_whitespace().next().unwrap_or("")).filter(|s| !s.is_empty() && *s != "__future__").collect::>().join(","); + let names = rest + .split(',') + .map(|s| s.split_whitespace().next().unwrap_or("")) + .filter(|s| !s.is_empty() && *s != "__future__") + .collect::>() + .join(","); if names.is_empty() { return None; } - return Some(RawImport { spec: format!("py:import:0::{names}"), names: Vec::new(), kind: Kind::Import, conditional: false, inline_depth: 0 }); + return Some(RawImport { + spec: format!("py:import:0::{names}"), + names: Vec::new(), + kind: Kind::Import, + conditional: false, + inline_depth: 0, + }); } if let Some(rest) = stmt.strip_prefix("from ") { let import_at = rest.find(" import ")?; let left = rest[..import_at].trim(); - let right = rest[import_at + " import ".len()..].trim().trim_matches(|c| c == '(' || c == ')'); + let right = rest[import_at + " import ".len()..] + .trim() + .trim_matches(|c| c == '(' || c == ')'); let mut level = 0; for c in left.chars() { if c == '.' { @@ -880,7 +1028,8 @@ fn parse_py_stmt(stmt: &str) -> Option { spec: format!("py:from:{level}:{module}:{}", names.join(",")), names, kind: Kind::Import, - conditional: false, inline_depth: 0, + conditional: false, + inline_depth: 0, }); } None @@ -935,7 +1084,13 @@ fn scan_js(src: &str) -> Scan { let after = i + "import".len(); if after < b.len() && b[after] == b'(' { if let Some((spec, next)) = read_call_spec(src, after) { - imports.push(RawImport { spec, names: Vec::new(), kind: Kind::Dynamic, conditional: false, inline_depth: 0 }); + imports.push(RawImport { + spec, + names: Vec::new(), + kind: Kind::Dynamic, + conditional: false, + inline_depth: 0, + }); i = next; tail.clear(); continue; @@ -952,7 +1107,8 @@ fn scan_js(src: &str) -> Scan { let rest = &src[i + "export".len()..]; let trimmed = rest.trim_start(); let lead = rest.len() - trimmed.len(); - if trimmed.starts_with('*') || (trimmed.starts_with('{') && trimmed.contains(" from ")) { + if trimmed.starts_with('*') || (trimmed.starts_with('{') && trimmed.contains(" from ")) + { if let Some((imp, n)) = read_export_from(trimmed) { imports.push(imp); i = i + "export".len() + lead + n; @@ -969,7 +1125,13 @@ fn scan_js(src: &str) -> Scan { let after = i + "require".len(); if after < b.len() && b[after] == b'(' { if let Some((spec, next)) = read_call_spec(src, after) { - imports.push(RawImport { spec, names: Vec::new(), kind: Kind::Dynamic, conditional: false, inline_depth: 0 }); + imports.push(RawImport { + spec, + names: Vec::new(), + kind: Kind::Dynamic, + conditional: false, + inline_depth: 0, + }); i = next; tail.clear(); continue; @@ -987,10 +1149,18 @@ fn scan_js(src: &str) -> Scan { } i += ch.len_utf8(); } - Scan { imports, local_exports } + Scan { + imports, + local_exports, + } } -fn scan_template(b: &[u8], mut i: usize, imports: &mut Vec, local_exports: &mut Vec) -> usize { +fn scan_template( + b: &[u8], + mut i: usize, + imports: &mut Vec, + local_exports: &mut Vec, +) -> usize { i += 1; while i < b.len() && b[i] != b'`' { if b[i] == b'$' && i + 1 < b.len() && b[i + 1] == b'{' { @@ -1030,9 +1200,22 @@ fn scan_template(b: &[u8], mut i: usize, imports: &mut Vec, local_exp fn export_local_name(trimmed: &str) -> Option { let rest = trimmed.trim_start_matches("default ").trim_start(); - for kw in ["const ", "let ", "var ", "function ", "async function ", "class ", "interface ", "type ", "enum "] { + for kw in [ + "const ", + "let ", + "var ", + "function ", + "async function ", + "class ", + "interface ", + "type ", + "enum ", + ] { if let Some(after) = rest.strip_prefix(kw) { - let name = after.split(['=', '(', '<', '{', ':', ' ', '\n', '\t']).next().unwrap_or(""); + let name = after + .split(['=', '(', '<', '{', ':', ' ', '\n', '\t']) + .next() + .unwrap_or(""); if is_ident(name) { return Some(name.to_string()); } @@ -1046,7 +1229,10 @@ fn export_local_name(trimmed: &str) -> Option { fn js_regex_start(code: &str) -> bool { let t = code.trim_end(); - t.is_empty() || t.ends_with(['=', '(', '[', ',', ';', '!', '&', '|', '?', ':', '{', '}', '+', '-']) + t.is_empty() + || t.ends_with([ + '=', '(', '[', ',', ';', '!', '&', '|', '?', ':', '{', '}', '+', '-', + ]) } fn skip_js_string(b: &[u8], i: usize) -> usize { @@ -1068,7 +1254,9 @@ fn skip_js_string(b: &[u8], i: usize) -> usize { fn is_ident(s: &str) -> bool { let mut cs = s.chars(); match cs.next() { - Some(c) if c.is_ascii_alphabetic() || c == '_' || c == '$' => cs.all(|c| c.is_ascii_alphanumeric() || c == '_' || c == '$'), + Some(c) if c.is_ascii_alphabetic() || c == '_' || c == '$' => { + cs.all(|c| c.is_ascii_alphanumeric() || c == '_' || c == '$') + } _ => false, } } @@ -1115,7 +1303,16 @@ fn read_import_statement(rest: &str) -> Option<(RawImport, usize)> { let lead = rest.len() - t.len(); if t.starts_with('\'') || t.starts_with('"') { let (spec, n) = read_quoted(t)?; - return Some((RawImport { spec, names: Vec::new(), kind: Kind::Import, conditional: false, inline_depth: 0 }, lead + n)); + return Some(( + RawImport { + spec, + names: Vec::new(), + kind: Kind::Import, + conditional: false, + inline_depth: 0, + }, + lead + n, + )); } let from = t.find(" from ")?; let head = &t[..from]; @@ -1123,7 +1320,16 @@ fn read_import_statement(rest: &str) -> Option<(RawImport, usize)> { let ws = after.len() - after.trim_start().len(); let (spec, n) = read_quoted(after.trim_start())?; let names = brace_names(head); - Some((RawImport { spec, names, kind: Kind::Import, conditional: false, inline_depth: 0 }, lead + from + " from ".len() + ws + n)) + Some(( + RawImport { + spec, + names, + kind: Kind::Import, + conditional: false, + inline_depth: 0, + }, + lead + from + " from ".len() + ws + n, + )) } fn read_export_from(trimmed: &str) -> Option<(RawImport, usize)> { @@ -1131,23 +1337,47 @@ fn read_export_from(trimmed: &str) -> Option<(RawImport, usize)> { let from = trimmed.find(" from ")?; let after = trimmed[from + " from ".len()..].trim_start(); let (spec, n) = read_quoted(after)?; - let consumed = from + " from ".len() + (trimmed[from + " from ".len()..].len() - after.len()) + n; - return Some((RawImport { spec, names: Vec::new(), kind: Kind::ExportStar, conditional: false, inline_depth: 0 }, consumed)); + let consumed = + from + " from ".len() + (trimmed[from + " from ".len()..].len() - after.len()) + n; + return Some(( + RawImport { + spec, + names: Vec::new(), + kind: Kind::ExportStar, + conditional: false, + inline_depth: 0, + }, + consumed, + )); } if trimmed.starts_with('{') { let from = trimmed.find(" from ")?; let names = brace_names(&trimmed[..from]); let after = trimmed[from + " from ".len()..].trim_start(); let (spec, n) = read_quoted(after)?; - let consumed = from + " from ".len() + (trimmed[from + " from ".len()..].len() - after.len()) + n; - return Some((RawImport { spec, names, kind: Kind::ExportFrom, conditional: false, inline_depth: 0 }, consumed)); + let consumed = + from + " from ".len() + (trimmed[from + " from ".len()..].len() - after.len()) + n; + return Some(( + RawImport { + spec, + names, + kind: Kind::ExportFrom, + conditional: false, + inline_depth: 0, + }, + consumed, + )); } None } fn brace_names(head: &str) -> Vec { - let Some(a) = head.find('{') else { return Vec::new() }; - let Some(b) = head.find('}') else { return Vec::new() }; + let Some(a) = head.find('{') else { + return Vec::new(); + }; + let Some(b) = head.find('}') else { + return Vec::new(); + }; head[a + 1..b] .split(',') .filter_map(|part| { @@ -1162,7 +1392,11 @@ fn brace_names(head: &str) -> Vec { }; // `type Foo` inside braces let name = name.trim_start_matches("type ").trim(); - if is_ident(name) { Some(name.to_string()) } else { None } + if is_ident(name) { + Some(name.to_string()) + } else { + None + } }) .collect() } @@ -1269,12 +1503,28 @@ fn scan_rust(src: &str) -> Scan { i += 1; } let mut seen = BTreeSet::new(); - imports.retain(|imp| seen.insert(format!("{}:{}:{}", imp.inline_depth, kind_tag(imp.kind), imp.spec))); - Scan { imports, local_exports: Vec::new() } + imports.retain(|imp| { + seen.insert(format!( + "{}:{}:{}", + imp.inline_depth, + kind_tag(imp.kind), + imp.spec + )) + }); + Scan { + imports, + local_exports: Vec::new(), + } } fn rust_import(spec: String, kind: Kind, inline_depth: usize) -> RawImport { - RawImport { spec, names: Vec::new(), kind, conditional: false, inline_depth } + RawImport { + spec, + names: Vec::new(), + kind, + conditional: false, + inline_depth, + } } fn kind_tag(kind: Kind) -> u8 { @@ -1308,7 +1558,11 @@ fn read_rust_use(b: &[u8], start: usize) -> Option<(usize, Vec)> { let i = skip_noise(b, start); let (end, paths) = parse_use_tree(b, i, &[])?; let end = skip_noise(b, end); - let end = if end < b.len() && b[end] == b';' { end + 1 } else { end }; + let end = if end < b.len() && b[end] == b';' { + end + 1 + } else { + end + }; Some((end, paths)) } @@ -1343,7 +1597,11 @@ fn parse_use_tree(b: &[u8], start: usize, prefix: &[String]) -> Option<(usize, V segs.push(ident); } let path = segs.join("::"); - let paths = if path.is_empty() { Vec::new() } else { vec![path] }; + let paths = if path.is_empty() { + Vec::new() + } else { + vec![path] + }; Some((j, paths)) } @@ -1557,7 +1815,11 @@ fn separate_in_package(from_rel: &str, dir: &str) -> Option { } fn join_rel(dir: &str, rest: &str) -> String { - if dir.is_empty() { rest.to_string() } else { format!("{dir}/{rest}") } + if dir.is_empty() { + rest.to_string() + } else { + format!("{dir}/{rest}") + } } /// Public Rust names: unrestricted `pub` items, `pub` methods and associated items on a @@ -1770,7 +2032,8 @@ fn read_impl_header(b: &[u8], start: usize) -> Option<(usize, bool, String)> { continue; } if let Some((ident, end)) = read_ident(b, i) { - if angle == 0 && !in_where && ident != "const" && ident != "unsafe" && ident != "async" { + if angle == 0 && !in_where && ident != "const" && ident != "unsafe" && ident != "async" + { last_ident = ident; } i = end; @@ -1799,7 +2062,11 @@ fn read_assoc_item(b: &[u8], start: usize) -> Option<(String, usize)> { } } if starts_word(b, i, "fn") || starts_word(b, i, "type") { - let word = if starts_word(b, i, "fn") { "fn" } else { "type" }; + let word = if starts_word(b, i, "fn") { + "fn" + } else { + "type" + }; let (name, end) = read_ident(b, skip_noise(b, i + word.len()))?; return Some((name, end)); } @@ -1828,7 +2095,15 @@ fn rust_pub_names(b: &[u8], start: usize) -> Option<(Vec, usize)> { } break; } - for (word, len) in [("fn", 2usize), ("struct", 6), ("enum", 4), ("trait", 5), ("mod", 3), ("type", 4), ("static", 6)] { + for (word, len) in [ + ("fn", 2usize), + ("struct", 6), + ("enum", 4), + ("trait", 5), + ("mod", 3), + ("type", 4), + ("static", 6), + ] { if starts_word(b, i, word) { let mut next = skip_noise(b, i + len); if word == "static" && starts_word(b, next, "mut") { @@ -1895,7 +2170,11 @@ fn use_export_names(b: &[u8], start: usize) -> Option<(Vec, usize)> { } let (end, names) = walk(b, skip_noise(b, start))?; let end = skip_noise(b, end); - let end = if end < b.len() && b[end] == b';' { end + 1 } else { end }; + let end = if end < b.len() && b[end] == b';' { + end + 1 + } else { + end + }; Some((names, end)) } @@ -1920,7 +2199,9 @@ fn load_rust_libs(root: &Path) -> Vec { } let mut libs = Vec::new(); for dir in dirs { - let exported = dir.is_empty() || expand_member_dirs(root, &members, &exclude).contains(&dir) || (members.is_empty() && dir.is_empty()); + let exported = dir.is_empty() + || expand_member_dirs(root, &members, &exclude).contains(&dir) + || (members.is_empty() && dir.is_empty()); if let Some(lib) = rust_lib_at(root, &dir, exported && !exclude_matches(&exclude, &dir)) { libs.push(lib); } @@ -1960,12 +2241,21 @@ fn rust_lib_at(root: &Path, dir: &str, exported: bool) -> Option { names.push(normalized); } } - Some(RustLib { names, root_file, dir: dir.to_string(), exported: exported && has_lib }) + Some(RustLib { + names, + root_file, + dir: dir.to_string(), + exported: exported && has_lib, + }) } fn cargo_package_name(text: &str) -> Option { let (package, _, _) = cargo_identity(text); - if package.is_empty() { None } else { Some(package) } + if package.is_empty() { + None + } else { + Some(package) + } } fn cargo_identity(text: &str) -> (String, String, Option) { @@ -1985,7 +2275,9 @@ fn cargo_identity(text: &str) -> (String, String, Option) { if section != "package" && section != "lib" { continue; } - let Some((key, value)) = line.split_once('=') else { continue }; + let Some((key, value)) = line.split_once('=') else { + continue; + }; let key = key.trim(); let value = unquote_toml(value.trim()); if value.is_empty() { @@ -2003,7 +2295,10 @@ fn cargo_identity(text: &str) -> (String, String, Option) { } fn unquote_toml(value: &str) -> String { - value.trim().trim_matches(|c| c == '"' || c == '\'').to_string() + value + .trim() + .trim_matches(|c| c == '"' || c == '\'') + .to_string() } fn workspace_arrays(text: &str) -> (Vec, Vec) { @@ -2031,13 +2326,18 @@ fn workspace_arrays(text: &str) -> (Vec, Vec) { buf.push_str(line); if line.contains(']') { let items = quoted_strings(&buf); - if key == "members" { members = items; } - else if key == "exclude" { exclude = items; } + if key == "members" { + members = items; + } else if key == "exclude" { + exclude = items; + } collecting = false; } continue; } - let Some((raw_key, value)) = line.split_once('=') else { continue }; + let Some((raw_key, value)) = line.split_once('=') else { + continue; + }; let raw_key = raw_key.trim(); if raw_key != "members" && raw_key != "exclude" { continue; @@ -2048,8 +2348,11 @@ fn workspace_arrays(text: &str) -> (Vec, Vec) { collecting = true; } else { let items = quoted_strings(value); - if raw_key == "members" { members = items; } - else if raw_key == "exclude" { exclude = items; } + if raw_key == "members" { + members = items; + } else if raw_key == "exclude" { + exclude = items; + } } } (members, exclude) @@ -2090,11 +2393,17 @@ fn expand_member_dirs(root: &Path, patterns: &[String], exclude: &[String]) -> V } if pattern.contains('*') || pattern.contains('?') { for dir in cargo_package_dirs(root) { - if cargo_glob_match(&pattern, &dir) && !exclude_matches(exclude, &dir) && !found.contains(&dir) { + if cargo_glob_match(&pattern, &dir) + && !exclude_matches(exclude, &dir) + && !found.contains(&dir) + { found.push(dir); } } - } else if root.join(&pattern).join("Cargo.toml").is_file() && !exclude_matches(exclude, &pattern) && !found.contains(&pattern) { + } else if root.join(&pattern).join("Cargo.toml").is_file() + && !exclude_matches(exclude, &pattern) + && !found.contains(&pattern) + { found.push(pattern); } } @@ -2104,7 +2413,9 @@ fn expand_member_dirs(root: &Path, patterns: &[String], exclude: &[String]) -> V fn cargo_package_dirs(root: &Path) -> Vec { let mut out = Vec::new(); fn walk(root: &Path, dir: &Path, out: &mut Vec) { - let Ok(entries) = fs::read_dir(dir) else { return }; + let Ok(entries) = fs::read_dir(dir) else { + return; + }; for entry in entries.flatten() { let path = entry.path(); if !path.is_dir() { @@ -2116,7 +2427,11 @@ fn cargo_package_dirs(root: &Path) -> Vec { } if path.join("Cargo.toml").is_file() { if let Ok(rel) = path.strip_prefix(root) { - let rel = rel.components().map(|c| c.as_os_str().to_string_lossy()).collect::>().join("/"); + let rel = rel + .components() + .map(|c| c.as_os_str().to_string_lossy()) + .collect::>() + .join("/"); if !rel.is_empty() { out.push(rel); } @@ -2181,7 +2496,13 @@ fn scan_go(src: &str) -> Scan { while j < bytes.len() && bytes[j] != b')' { if bytes[j] == b'"' { if let Some((spec, n)) = read_quoted(&text[j..]) { - imports.push(RawImport { spec, names: Vec::new(), kind: Kind::Import, conditional: false, inline_depth: 0 }); + imports.push(RawImport { + spec, + names: Vec::new(), + kind: Kind::Import, + conditional: false, + inline_depth: 0, + }); j += n; continue; } @@ -2193,7 +2514,13 @@ fn scan_go(src: &str) -> Scan { } if j < bytes.len() && bytes[j] == b'"' { if let Some((spec, n)) = read_quoted(&text[j..]) { - imports.push(RawImport { spec, names: Vec::new(), kind: Kind::Import, conditional: false, inline_depth: 0 }); + imports.push(RawImport { + spec, + names: Vec::new(), + kind: Kind::Import, + conditional: false, + inline_depth: 0, + }); i = j + n; continue; } @@ -2201,7 +2528,10 @@ fn scan_go(src: &str) -> Scan { } i += 1; } - Scan { imports, local_exports: Vec::new() } + Scan { + imports, + local_exports: Vec::new(), + } } fn scan_c(src: &str) -> Scan { @@ -2211,13 +2541,28 @@ fn scan_c(src: &str) -> Scan { if let Some(rest) = t.strip_prefix("#include") { let rest = rest.trim(); if let Some(inner) = rest.strip_prefix('"').and_then(|r| r.strip_suffix('"')) { - imports.push(RawImport { spec: inner.to_string(), names: Vec::new(), kind: Kind::Import, conditional: false, inline_depth: 0 }); + imports.push(RawImport { + spec: inner.to_string(), + names: Vec::new(), + kind: Kind::Import, + conditional: false, + inline_depth: 0, + }); } else if let Some(inner) = rest.strip_prefix('<').and_then(|r| r.strip_suffix('>')) { - imports.push(RawImport { spec: inner.to_string(), names: Vec::new(), kind: Kind::Import, conditional: false, inline_depth: 0 }); + imports.push(RawImport { + spec: inner.to_string(), + names: Vec::new(), + kind: Kind::Import, + conditional: false, + inline_depth: 0, + }); } } } - Scan { imports, local_exports: Vec::new() } + Scan { + imports, + local_exports: Vec::new(), + } } fn scan_java(src: &str) -> Scan { @@ -2227,11 +2572,20 @@ fn scan_java(src: &str) -> Scan { if let Some(rest) = t.strip_prefix("import ") { let rest = rest.trim_start_matches("static ").trim(); if !rest.is_empty() { - imports.push(RawImport { spec: rest.to_string(), names: Vec::new(), kind: Kind::Import, conditional: false, inline_depth: 0 }); + imports.push(RawImport { + spec: rest.to_string(), + names: Vec::new(), + kind: Kind::Import, + conditional: false, + inline_depth: 0, + }); } } } - Scan { imports, local_exports: Vec::new() } + Scan { + imports, + local_exports: Vec::new(), + } } fn scan_csharp(src: &str) -> Scan { @@ -2244,11 +2598,20 @@ fn scan_csharp(src: &str) -> Scan { continue; } if !rest.is_empty() && !rest.starts_with('(') { - imports.push(RawImport { spec: rest.to_string(), names: Vec::new(), kind: Kind::Import, conditional: false, inline_depth: 0 }); + imports.push(RawImport { + spec: rest.to_string(), + names: Vec::new(), + kind: Kind::Import, + conditional: false, + inline_depth: 0, + }); } } } - Scan { imports, local_exports: Vec::new() } + Scan { + imports, + local_exports: Vec::new(), + } } fn strip_c_comments(src: &str) -> String { @@ -2329,7 +2692,11 @@ pub fn resolve_python_spec(root: &Path, from_rel: &str, spec: &str) -> Vec Vec Option { @@ -2429,18 +2798,24 @@ fn module_file(base: &Path, dotted: &str) -> Option { t.set_file_name(name); t }; - [py, init, pyi].into_iter().find(|cand| exists_exact(base, cand)) + [py, init, pyi] + .into_iter() + .find(|cand| exists_exact(base, cand)) } fn exists_exact(base: &Path, candidate: &Path) -> bool { if !candidate.is_file() { return false; } - let Ok(rel) = candidate.strip_prefix(base) else { return candidate.is_file() }; + let Ok(rel) = candidate.strip_prefix(base) else { + return candidate.is_file(); + }; let mut current = base.to_path_buf(); for name in rel.components() { let name = name.as_os_str().to_string_lossy().to_string(); - let Ok(rd) = fs::read_dir(¤t) else { return false }; + let Ok(rd) = fs::read_dir(¤t) else { + return false; + }; let mut found = false; for ent in rd.flatten() { if ent.file_name().to_string_lossy() == name { @@ -2465,7 +2840,10 @@ mod dbg_scan { assert!( scan.imports.iter().any(|i| i.spec == "./lazy"), "specs {:?}", - scan.imports.iter().map(|i| format!("{} {:?}", i.spec, i.kind)).collect::>() + scan.imports + .iter() + .map(|i| format!("{} {:?}", i.spec, i.kind)) + .collect::>() ); } @@ -2477,8 +2855,18 @@ mod dbg_scan { } src.push_str("from . import b\nimport_module(\"pkg.mod\")\n"); let scan = super::scan_python(&src); - assert!(scan.imports.iter().any(|imp| imp.spec.contains("py:from") && imp.spec.contains('b')), "{:?}", scan.imports); - assert!(scan.imports.iter().any(|imp| imp.spec == "pkg.mod"), "{:?}", scan.imports); + assert!( + scan.imports + .iter() + .any(|imp| imp.spec.contains("py:from") && imp.spec.contains('b')), + "{:?}", + scan.imports + ); + assert!( + scan.imports.iter().any(|imp| imp.spec == "pkg.mod"), + "{:?}", + scan.imports + ); } #[test] @@ -2490,10 +2878,19 @@ mod dbg_scan { } src.push_str("import { add } from \"./math\";\nexport const 値 = 1;\n"); let scan = super::scan_js(&src); - assert!(scan.imports.iter().any(|imp| imp.spec == "./math"), "{:?}", scan.imports); - let ts = "import type { 設定 } from \"./types\";\nconst emoji = \"🐍\";\nexport { 設定 };\n"; + assert!( + scan.imports.iter().any(|imp| imp.spec == "./math"), + "{:?}", + scan.imports + ); + let ts = + "import type { 設定 } from \"./types\";\nconst emoji = \"🐍\";\nexport { 設定 };\n"; let scan = super::scan_js(ts); - assert!(scan.imports.iter().any(|imp| imp.spec == "./types"), "{:?}", scan.imports); + assert!( + scan.imports.iter().any(|imp| imp.spec == "./types"), + "{:?}", + scan.imports + ); } } @@ -2515,7 +2912,15 @@ pub struct FileFacts { pub unresolved: Vec<(String, i64, String)>, } -fn add_edge(facts: &mut FileFacts, dst: String, line: i64, names: Vec, conditional: bool, dynamic: bool, via_barrel: bool) { +fn add_edge( + facts: &mut FileFacts, + dst: String, + line: i64, + names: Vec, + conditional: bool, + dynamic: bool, + via_barrel: bool, +) { if let Some(existing) = facts.internal.iter_mut().find(|edge| edge.dst == dst) { if !via_barrel { existing.via_barrel = false; @@ -2529,7 +2934,14 @@ fn add_edge(facts: &mut FileFacts, dst: String, line: i64, names: Vec, c return; } facts.dsts.insert(dst.clone()); - facts.internal.push(FactEdge { dst, line, names, conditional, dynamic, via_barrel }); + facts.internal.push(FactEdge { + dst, + line, + names, + conditional, + dynamic, + via_barrel, + }); } fn note_unresolved(facts: &mut FileFacts, spec: String, line: i64, kind: Kind) { @@ -2551,7 +2963,10 @@ pub struct Analysis { fn line_of(text: &str, needles: &[String]) -> i64 { for (i, line) in text.lines().enumerate() { - if needles.iter().any(|n| !n.is_empty() && line.contains(n.as_str())) { + if needles + .iter() + .any(|n| !n.is_empty() && line.contains(n.as_str())) + { return (i + 1) as i64; } } @@ -2560,14 +2975,27 @@ fn line_of(text: &str, needles: &[String]) -> i64 { fn py_parts(spec: &str) -> (String, Vec) { if let Some(rest) = spec.strip_prefix("py:import:") { - let names = rest.split("::").nth(1).unwrap_or("").split(',').filter(|s| !s.is_empty()).map(|s| s.to_string()).collect::>(); + let names = rest + .split("::") + .nth(1) + .unwrap_or("") + .split(',') + .filter(|s| !s.is_empty()) + .map(|s| s.to_string()) + .collect::>(); return (names.first().cloned().unwrap_or_default(), names); } if let Some(rest) = spec.strip_prefix("py:from:") { let mut parts = rest.splitn(3, ':'); let _level = parts.next(); let module = parts.next().unwrap_or("").to_string(); - let names = parts.next().unwrap_or("").split(',').filter(|s| !s.is_empty()).map(|s| s.to_string()).collect::>(); + let names = parts + .next() + .unwrap_or("") + .split(',') + .filter(|s| !s.is_empty()) + .map(|s| s.to_string()) + .collect::>(); return (module, names); } (spec.to_string(), Vec::new()) @@ -2584,7 +3012,11 @@ pub fn analyze_changed(root: &Path, universe: &[PathBuf], changed: &[PathBuf]) - analyze_files_filtered(root, universe, Some(only)) } -fn analyze_files_filtered(root: &Path, files: &[PathBuf], only: Option>) -> Analysis { +fn analyze_files_filtered( + root: &Path, + files: &[PathBuf], + only: Option>, +) -> Analysis { let mut scans: HashMap = HashMap::new(); let mut texts: HashMap = HashMap::new(); let mut warnings = Vec::new(); @@ -2611,7 +3043,11 @@ fn analyze_files_filtered(root: &Path, files: &[PathBuf], only: Option { - add_edge(&mut facts, dst.clone(), line, imp.names.clone(), imp.conditional, dynamic, false); - if lang == "javascript" && (imp.kind == Kind::Import || imp.kind == Kind::ExportFrom) { + add_edge( + &mut facts, + dst.clone(), + line, + imp.names.clone(), + imp.conditional, + dynamic, + false, + ); + if lang == "javascript" + && (imp.kind == Kind::Import || imp.kind == Kind::ExportFrom) + { for name in &imp.names { - if let Some(def) = defining_file(&resolver, &{ - let mut m = BTreeMap::new(); - for (k, v) in &scans { - m.insert(k.clone(), v.clone()); - } - m - }, &dst, name, 0) { + if let Some(def) = defining_file( + &resolver, + &{ + let mut m = BTreeMap::new(); + for (k, v) in &scans { + m.insert(k.clone(), v.clone()); + } + m + }, + &dst, + name, + 0, + ) { if def != *rel && def != dst && !facts.dsts.contains(&def) { - add_edge(&mut facts, def, line, vec![name.clone()], imp.conditional, dynamic, true); + add_edge( + &mut facts, + def, + line, + vec![name.clone()], + imp.conditional, + dynamic, + true, + ); } } } @@ -2675,14 +3147,24 @@ fn analyze_files_filtered(root: &Path, files: &[PathBuf], only: Option { if lang == "rust" { - if target.is_none() && imp.kind != Kind::PathExpr && resolver.rust_spec_is_internal(rel, &imp.spec) { + if target.is_none() + && imp.kind != Kind::PathExpr + && resolver.rust_spec_is_internal(rel, &imp.spec) + { note_unresolved(&mut facts, imp.spec.clone(), line, imp.kind); - } else if target.is_none() && imp.kind != Kind::PathExpr && !imp.spec.is_empty() && !resolver.rust_spec_is_internal(rel, &imp.spec) { + } else if target.is_none() + && imp.kind != Kind::PathExpr + && !imp.spec.is_empty() + && !resolver.rust_spec_is_internal(rel, &imp.spec) + { facts.external.push(imp.spec.clone()); } } else if imp.spec.starts_with('.') { note_unresolved(&mut facts, imp.spec.clone(), line, imp.kind); - } else if !imp.spec.is_empty() && !imp.spec.starts_with("mod:") && !imp.spec.starts_with("crate::") { + } else if !imp.spec.is_empty() + && !imp.spec.starts_with("mod:") + && !imp.spec.starts_with("crate::") + { facts.external.push(imp.spec.clone()); } else if target.is_none() && imp.spec.starts_with("mod:") { note_unresolved(&mut facts, imp.spec.clone(), line, imp.kind); @@ -2708,9 +3190,17 @@ fn analyze_files_filtered(root: &Path, files: &[PathBuf], only: Option BTreeMap> { - analyze_files(root, files).facts.into_iter().map(|(k, v)| (k, v.dsts)).collect() + analyze_files(root, files) + .facts + .into_iter() + .map(|(k, v)| (k, v.dsts)) + .collect() } diff --git a/src/pending_line.rs b/src/pending_line.rs new file mode 100644 index 0000000..fe3a6b5 --- /dev/null +++ b/src/pending_line.rs @@ -0,0 +1,48 @@ +//! Pending-update bullets. +//! New lines use the Python shape `- path: message`. +//! Readers still accept the older `- path — message` lines. + +/// File path of a pending bullet, if `line` is an item. +pub fn file_of(line: &str) -> Option<&str> { + let body = line.trim().strip_prefix("- ")?; + if let Some((path, _)) = body.split_once(": ") { + let path = path.trim(); + if !path.is_empty() { + return Some(path); + } + } + let legacy = body.split(" — ").next().unwrap_or(body); + let legacy = legacy.split(" - ").next().unwrap_or(legacy).trim(); + if legacy.is_empty() { + None + } else { + Some(legacy) + } +} + +pub fn format_item(rel: &str, message: &str) -> String { + format!("- {rel}: {message}\n") +} + +/// Append `message` for `rel` when that file is not already listed. +pub fn append_item(text: &str, rel: &str, message: &str) -> String { + if text.lines().any(|line| file_of(line) == Some(rel)) { + return text.to_string(); + } + let mut out = if text.trim().is_empty() { + "# Pending Updates\n\n".to_string() + } else { + text.to_string() + }; + if !out.is_empty() && !out.ends_with('\n') { + out.push('\n'); + } + out.push_str(&format_item(rel, message)); + out +} + +pub fn record_message(reason: &str) -> String { + format!("LLM/agent edit — {reason}") +} + +pub const CHECK_MESSAGE: &str = "Content change detected — review, refresh wiki, then mark-green"; diff --git a/src/proc.rs b/src/proc.rs index 4143ff3..434bd26 100644 --- a/src/proc.rs +++ b/src/proc.rs @@ -58,7 +58,11 @@ fn windows_signal(pid: i32, sig: i32) -> bool { fn CloseHandle(handle: *mut core::ffi::c_void) -> i32; } unsafe { - let access = if sig == 0 { PROCESS_QUERY_LIMITED_INFORMATION } else { PROCESS_TERMINATE }; + let access = if sig == 0 { + PROCESS_QUERY_LIMITED_INFORMATION + } else { + PROCESS_TERMINATE + }; let handle = OpenProcess(access, 0, pid as u32); if handle.is_null() || handle == (-1isize as *mut core::ffi::c_void) { return false; diff --git a/src/project_lock.rs b/src/project_lock.rs index 3b328f2..9b8f148 100644 --- a/src/project_lock.rs +++ b/src/project_lock.rs @@ -50,7 +50,10 @@ impl ProjectLock { loop { match OpenOptions::new().write(true).create_new(true).open(&path) { Ok(mut file) => { - let stamp = SystemTime::now().duration_since(UNIX_EPOCH).map(|d| d.as_secs()).unwrap_or(0); + let stamp = SystemTime::now() + .duration_since(UNIX_EPOCH) + .map(|d| d.as_secs()) + .unwrap_or(0); writeln!(file, "{}\n{stamp}", std::process::id()).map_err(|e| e.to_string())?; file.sync_all().map_err(|e| e.to_string())?; DEPTH.with(|depth| depth.set(1)); @@ -81,7 +84,13 @@ fn try_steal(path: &Path) -> bool { Ok(text) => text, Err(_) => return false, }; - let pid: i32 = text.lines().next().unwrap_or("0").trim().parse().unwrap_or(0); + let pid: i32 = text + .lines() + .next() + .unwrap_or("0") + .trim() + .parse() + .unwrap_or(0); let stale = if pid > 0 { !crate::proc::pid_alive(pid) } else { diff --git a/src/scope.rs b/src/scope.rs index 8c771dd..4bcbfcc 100644 --- a/src/scope.rs +++ b/src/scope.rs @@ -10,8 +10,22 @@ pub const SOURCE_EXTS: &[&str] = &[ ]; const DEFAULT_EXCLUDES: &[&str] = &[ - "node_modules", ".git", "dist", "build", ".next", "coverage", "__pycache__", "tmp", "temp", - ".turbo", ".cache", "target", "out", ".wikifier_staging", ".venv", "venv", + "node_modules", + ".git", + "dist", + "build", + ".next", + "coverage", + "__pycache__", + "tmp", + "temp", + ".turbo", + ".cache", + "target", + "out", + ".wikifier_staging", + ".venv", + "venv", ]; #[derive(Clone, Debug)] @@ -29,7 +43,9 @@ pub struct Scope { impl Scope { pub fn allows_walk(&self) -> bool { - !self.container && self.missing_dirs.is_empty() && (self.full_tree || !self.prefixes.is_empty()) + !self.container + && self.missing_dirs.is_empty() + && (self.full_tree || !self.prefixes.is_empty()) } } @@ -46,9 +62,8 @@ pub fn detect(root: &Path) -> Scope { let map_paths = read_path_list(root, "map_paths.txt"); let bare_dot = monitored.is_empty() || monitored.iter().any(|p| p == "." || p == "./"); if bare_dot && !container { - warnings.push( - "monitored_paths is bare '.' or missing. Prefer lean package roots.".to_string(), - ); + warnings + .push("monitored_paths is bare '.' or missing. Prefer lean package roots.".to_string()); } let (prefixes, missing, full_tree) = resolve_prefixes(root, &map_paths, &monitored); if !missing.is_empty() { @@ -88,7 +103,9 @@ fn child_projects(root: &Path) -> (usize, Vec) { if !path.is_dir() { continue; } - if path.join(".git").exists() || path.join("file_health.json").exists() || path.join(".wikifier_staging").is_dir() + if path.join(".git").exists() + || path.join("file_health.json").exists() + || path.join(".wikifier_staging").is_dir() { n += 1; if sample.len() < 8 { @@ -99,7 +116,11 @@ fn child_projects(root: &Path) -> (usize, Vec) { (n, sample) } -fn resolve_prefixes(root: &Path, map_paths: &[String], monitored: &[String]) -> (Vec, Vec, bool) { +fn resolve_prefixes( + root: &Path, + map_paths: &[String], + monitored: &[String], +) -> (Vec, Vec, bool) { if !map_paths.is_empty() { return split_existing(root, map_paths); } @@ -119,7 +140,10 @@ fn resolve_prefixes(root: &Path, map_paths: &[String], monitored: &[String]) -> } fn looks_like_dir(p: &str) -> bool { - let suf = Path::new(p.trim_end_matches('/')).extension().and_then(|s| s.to_str()).unwrap_or(""); + let suf = Path::new(p.trim_end_matches('/')) + .extension() + .and_then(|s| s.to_str()) + .unwrap_or(""); suf.is_empty() } @@ -182,7 +206,11 @@ pub fn read_path_list(root: &Path, name: &str) -> Vec { if p.is_empty() || p.starts_with('#') { continue; } - let p = p.split_whitespace().next().unwrap_or("").trim_end_matches('/'); + let p = p + .split_whitespace() + .next() + .unwrap_or("") + .trim_end_matches('/'); if p.is_empty() || p == "." || p == "./" { continue; } diff --git a/src/sha256.rs b/src/sha256.rs index cee4354..2540de1 100644 --- a/src/sha256.rs +++ b/src/sha256.rs @@ -13,7 +13,8 @@ const K: [u32; 64] = [ pub fn sha256(data: &[u8]) -> [u8; 32] { let mut h: [u32; 8] = [ - 0x6a09e667, 0xbb67ae85, 0x3c6ef372, 0xa54ff53a, 0x510e527f, 0x9b05688c, 0x1f83d9ab, 0x5be0cd19, + 0x6a09e667, 0xbb67ae85, 0x3c6ef372, 0xa54ff53a, 0x510e527f, 0x9b05688c, 0x1f83d9ab, + 0x5be0cd19, ]; let bit_len = (data.len() as u64) * 8; let mut msg = data.to_vec(); @@ -30,7 +31,10 @@ pub fn sha256(data: &[u8]) -> [u8; 32] { for i in 16..64 { let s0 = w[i - 15].rotate_right(7) ^ w[i - 15].rotate_right(18) ^ (w[i - 15] >> 3); let s1 = w[i - 2].rotate_right(17) ^ w[i - 2].rotate_right(19) ^ (w[i - 2] >> 10); - w[i] = w[i - 16].wrapping_add(s0).wrapping_add(w[i - 7]).wrapping_add(s1); + w[i] = w[i - 16] + .wrapping_add(s0) + .wrapping_add(w[i - 7]) + .wrapping_add(s1); } let mut a = h[0]; let mut b = h[1]; @@ -43,7 +47,11 @@ pub fn sha256(data: &[u8]) -> [u8; 32] { for i in 0..64 { let s1 = e.rotate_right(6) ^ e.rotate_right(11) ^ e.rotate_right(25); let ch = (e & f) ^ (!e & g); - let t1 = hh.wrapping_add(s1).wrapping_add(ch).wrapping_add(K[i]).wrapping_add(w[i]); + let t1 = hh + .wrapping_add(s1) + .wrapping_add(ch) + .wrapping_add(K[i]) + .wrapping_add(w[i]); let s0 = a.rotate_right(2) ^ a.rotate_right(13) ^ a.rotate_right(22); let maj = (a & b) ^ (a & c) ^ (b & c); let t2 = s0.wrapping_add(maj); diff --git a/src/symbols.rs b/src/symbols.rs new file mode 100644 index 0000000..02a3a41 --- /dev/null +++ b/src/symbols.rs @@ -0,0 +1,555 @@ +//! Public names for wiki checks. +//! Python is a small tokenizer (strings, comments, and brackets), not a line scan. +//! JS/TS exports are read after comments and strings are masked. + +use std::collections::BTreeSet; + +/// `__all__` when it is a literal list or tuple (including `+=`), otherwise +/// top-level `def` / `async def` / `class` names that do not start with `_`. +pub fn python_public_symbols(text: &str) -> Vec { + let b = text.as_bytes(); + let mut i = 0; + let mut depth = 0i32; + let mut defs = BTreeSet::new(); + let mut exported: Option> = None; + while i < b.len() { + if depth == 0 && at_line_start(b, i) { + let (col, j) = indent_at(b, i); + if j >= b.len() { + break; + } + if b[j] == b'#' { + i = skip_line(b, j); + continue; + } + if b[j] == b'\n' || b[j] == b'\r' { + i = j + 1; + continue; + } + if col == 0 { + if starts_keyword(b, j, "__all__") { + let (names, next, replaced) = read_all_assignment(b, j); + if let Some(names) = names { + let set = exported.get_or_insert_with(BTreeSet::new); + if replaced { + set.clear(); + } + set.extend(names); + } + i = next; + continue; + } + if let Some((name, next)) = read_top_def(b, j) { + if !name.starts_with('_') { + defs.insert(name); + } + i = next; + continue; + } + } + i = j; + } + i = consume_one(b, i, &mut depth); + } + let chosen = exported.unwrap_or(defs); + chosen.into_iter().collect() +} + +/// Exported JS/TS names. Comments, strings, and template text do not count. +pub fn js_public_symbols(text: &str) -> Vec { + let masked = mask_js(text); + let b = masked.as_bytes(); + let mut names = BTreeSet::new(); + let mut i = 0; + while i < b.len() { + if word_at(b, i, "export") { + i += "export".len(); + i = skip_ws(b, i); + if word_at(b, i, "default") { + i += "default".len(); + i = skip_ws(b, i); + } + if word_at(b, i, "type") { + let after = i + "type".len(); + let next = skip_ws(b, after); + if b.get(next) == Some(&b'{') { + i = next; + } + } + for word in ["declare", "abstract", "async"] { + if word_at(b, i, word) { + i += word.len(); + i = skip_ws(b, i); + } + } + if b.get(i) == Some(&b'{') { + i += 1; + let end = b[i..] + .iter() + .position(|c| *c == b'}') + .map(|n| i + n) + .unwrap_or(b.len()); + for part in masked[i..end].split(',') { + let part = part.trim().trim_start_matches("type ").trim(); + if part.is_empty() { + continue; + } + let name = part.split(" as ").last().unwrap_or(part).trim(); + push_js(&mut names, name); + } + i = (end + 1).min(b.len()); + continue; + } + if b.get(i) == Some(&b'*') { + i += 1; + continue; + } + for kind in [ + "function", + "class", + "const", + "let", + "var", + "interface", + "enum", + "namespace", + "type", + ] { + if word_at(b, i, kind) { + i += kind.len(); + if b.get(i) == Some(&b'*') { + i += 1; + } + i = skip_ws(b, i); + let (name, next) = read_js_ident(b, i); + push_js(&mut names, &name); + i = next; + break; + } + } + continue; + } + i += 1; + } + names.into_iter().collect() +} + +fn push_js(names: &mut BTreeSet, name: &str) { + let name = name + .trim() + .trim_matches(|c| c == '\'' || c == '"' || c == '`'); + if name.is_empty() || name == "default" || name == "from" || name.starts_with('_') { + return; + } + if name + .chars() + .all(|c| c.is_ascii_alphanumeric() || c == '_' || c == '$') + { + names.insert(name.to_string()); + } +} + +fn at_line_start(b: &[u8], i: usize) -> bool { + i == 0 || b[i - 1] == b'\n' +} + +fn indent_at(b: &[u8], mut i: usize) -> (usize, usize) { + let mut col = 0; + while i < b.len() && (b[i] == b' ' || b[i] == b'\t') { + col += 1; + i += 1; + } + (col, i) +} + +fn skip_line(b: &[u8], mut i: usize) -> usize { + while i < b.len() && b[i] != b'\n' { + i += 1; + } + i +} + +fn skip_ws(b: &[u8], mut i: usize) -> usize { + while i < b.len() && b[i].is_ascii_whitespace() { + i += 1; + } + i +} + +fn starts_keyword(b: &[u8], i: usize, word: &str) -> bool { + let w = word.as_bytes(); + i + w.len() <= b.len() + && &b[i..i + w.len()] == w + && !b.get(i + w.len()).is_some_and(|c| is_ident(*c)) +} + +fn word_at(b: &[u8], i: usize, word: &str) -> bool { + if i > 0 && is_ident(b[i - 1]) { + return false; + } + starts_keyword(b, i, word) +} + +fn is_ident(c: u8) -> bool { + c.is_ascii_alphanumeric() || c == b'_' +} + +fn read_ident(b: &[u8], mut i: usize) -> (String, usize) { + let start = i; + while i < b.len() && is_ident(b[i]) { + i += 1; + } + (String::from_utf8_lossy(&b[start..i]).into_owned(), i) +} + +fn read_top_def(b: &[u8], i: usize) -> Option<(String, usize)> { + let mut j = i; + if starts_keyword(b, j, "async") { + j = skip_ws(b, j + "async".len()); + } + if starts_keyword(b, j, "def") || starts_keyword(b, j, "class") { + let key = if starts_keyword(b, j, "def") { + "def" + } else { + "class" + }; + j = skip_ws(b, j + key.len()); + let (name, next) = read_ident(b, j); + if name.is_empty() { + return None; + } + return Some((name, next)); + } + None +} + +/// Returns names, the index after the statement's opening literal, and whether `=` replaced. +fn read_all_assignment(b: &[u8], i: usize) -> (Option>, usize, bool) { + let mut j = i + "__all__".len(); + j = skip_ws(b, j); + let replaced = if b.get(j) == Some(&b'+') && b.get(j + 1) == Some(&b'=') { + j += 2; + false + } else if b.get(j) == Some(&b'=') { + j += 1; + true + } else { + return (None, i + 1, false); + }; + j = skip_ws(b, j); + if b.get(j) != Some(&b'[') && b.get(j) != Some(&b'(') { + return (None, j, replaced); + } + let (names, next) = collect_literal_strings(b, j); + (Some(names), next, replaced) +} + +fn collect_literal_strings(b: &[u8], mut i: usize) -> (Vec, usize) { + let mut depth = 0i32; + let mut names = Vec::new(); + while i < b.len() { + if let Some(next) = try_skip_string(b, i) { + if depth == 1 { + if let Some(text) = string_contents(b, i) { + names.push(text); + } + } + i = next; + continue; + } + if b[i] == b'#' { + i = skip_line(b, i); + continue; + } + match b[i] { + b'(' | b'[' | b'{' => { + depth += 1; + i += 1; + } + b')' | b']' | b'}' => { + depth -= 1; + i += 1; + if depth == 0 { + break; + } + } + _ => i += 1, + } + } + (names, i) +} + +fn string_contents(b: &[u8], i: usize) -> Option { + let mut j = i; + let mut pref = 0; + while pref < 3 && j < b.len() && is_prefix_char(b[j]) { + j += 1; + pref += 1; + } + if j >= b.len() || (b[j] != b'\'' && b[j] != b'"') { + return None; + } + let quote = b[j]; + let triple = b.get(j + 1) == Some("e) && b.get(j + 2) == Some("e); + let start = if triple { j + 3 } else { j + 1 }; + let mut k = start; + let mut out = Vec::new(); + if triple { + while k + 2 < b.len() && !(b[k] == quote && b[k + 1] == quote && b[k + 2] == quote) { + out.push(b[k]); + k += 1; + } + } else { + while k < b.len() && b[k] != quote && b[k] != b'\n' { + if b[k] == b'\\' && k + 1 < b.len() { + out.push(b[k + 1]); + k += 2; + continue; + } + out.push(b[k]); + k += 1; + } + } + Some(String::from_utf8_lossy(&out).into_owned()) +} + +fn consume_one(b: &[u8], i: usize, depth: &mut i32) -> usize { + if i >= b.len() { + return i; + } + if b[i] == b'#' { + return skip_line(b, i); + } + if let Some(next) = try_skip_string(b, i) { + return next; + } + match b[i] { + b'(' | b'[' | b'{' => *depth += 1, + b')' | b']' | b'}' => *depth -= 1, + _ => {} + } + i + 1 +} + +fn is_prefix_char(c: u8) -> bool { + matches!(c, b'r' | b'R' | b'b' | b'B' | b'f' | b'F' | b'u' | b'U') +} + +fn try_skip_string(b: &[u8], i: usize) -> Option { + if i > 0 && is_ident(b[i - 1]) && !is_prefix_char(b[i - 1]) { + return None; + } + let mut j = i; + let mut pref = 0; + if i == 0 || !is_ident(b[i - 1]) { + while pref < 3 && j < b.len() && is_prefix_char(b[j]) { + j += 1; + pref += 1; + } + } + if j >= b.len() || (b[j] != b'\'' && b[j] != b'"') { + return None; + } + if pref > 0 && i > 0 && is_ident(b[i - 1]) { + return None; + } + Some(skip_quoted(b, j)) +} + +fn skip_quoted(b: &[u8], i: usize) -> usize { + let quote = b[i]; + let triple = b.get(i + 1) == Some("e) && b.get(i + 2) == Some("e); + let mut k = if triple { i + 3 } else { i + 1 }; + if triple { + while k + 2 < b.len() && !(b[k] == quote && b[k + 1] == quote && b[k + 2] == quote) { + k += 1; + } + (k + 3).min(b.len()) + } else { + while k < b.len() && b[k] != quote && b[k] != b'\n' { + if b[k] == b'\\' { + k += 2; + continue; + } + k += 1; + } + if k < b.len() && b[k] == quote { + k + 1 + } else { + k + } + } +} + +fn read_js_ident(b: &[u8], mut i: usize) -> (String, usize) { + let start = i; + while i < b.len() { + let c = b[i]; + if c.is_ascii_alphanumeric() || c == b'_' || c == b'$' { + i += 1; + } else { + break; + } + } + (String::from_utf8_lossy(&b[start..i]).into_owned(), i) +} + +fn mask_js(text: &str) -> String { + let b = text.as_bytes(); + let mut out = vec![b' '; b.len()]; + mask_range(b, &mut out, 0, false); + String::from_utf8(out).unwrap_or_else(|_| text.to_string()) +} + +/// Copy code into `out`. When `stop_on_brace` is set, return at the `}` that +/// closes a `${ ... }` expression. +fn mask_range(b: &[u8], out: &mut [u8], mut i: usize, stop_on_brace: bool) -> usize { + let mut brace = 0i32; + while i < b.len() { + if b[i] == b'\n' { + out[i] = b'\n'; + i += 1; + continue; + } + if b[i] == b'/' && i + 1 < b.len() && b[i + 1] == b'/' { + while i < b.len() && b[i] != b'\n' { + i += 1; + } + continue; + } + if b[i] == b'/' && i + 1 < b.len() && b[i + 1] == b'*' { + i += 2; + while i + 1 < b.len() && !(b[i] == b'*' && b[i + 1] == b'/') { + i += 1; + } + i = (i + 2).min(b.len()); + continue; + } + if b[i] == b'\'' || b[i] == b'"' { + let quote = b[i]; + i += 1; + while i < b.len() && b[i] != quote && b[i] != b'\n' { + if b[i] == b'\\' { + i = (i + 2).min(b.len()); + continue; + } + i += 1; + } + if i < b.len() && b[i] == quote { + i += 1; + } + continue; + } + if b[i] == b'`' { + i += 1; + while i < b.len() && b[i] != b'`' { + if b[i] == b'$' && i + 1 < b.len() && b[i + 1] == b'{' { + out[i] = b[i]; + out[i + 1] = b[i + 1]; + i = mask_range(b, out, i + 2, true); + continue; + } + if b[i] == b'\\' { + i = (i + 2).min(b.len()); + continue; + } + i += 1; + } + if i < b.len() && b[i] == b'`' { + i += 1; + } + continue; + } + if b[i] == b'{' { + brace += 1; + out[i] = b[i]; + i += 1; + continue; + } + if b[i] == b'}' { + if stop_on_brace && brace == 0 { + return i + 1; + } + if brace > 0 { + brace -= 1; + } + out[i] = b[i]; + i += 1; + continue; + } + out[i] = b[i]; + i += 1; + } + i +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn multiline_all_tuple_plus_and_hidden_defs() { + let text = "\ +__all__ = [ + \"visible\", + # \"nope\" + 'also', +] +__all__ += (\"extra\",) +def hidden(): + return 1 +def _private(): + return 2 +"; + assert_eq!( + python_public_symbols(text), + vec![ + "also".to_string(), + "extra".to_string(), + "visible".to_string() + ] + ); + } + + #[test] + fn decorators_async_and_strings_do_not_invent_names() { + let text = "\ +# def ghost(): +s = \"def fake():\" +''' +def doc(): + pass +''' +@decorator +async def real(): + def inner(): + return 1 +class Shown: + def method(self): + return 1 +"; + assert_eq!( + python_public_symbols(text), + vec!["Shown".to_string(), "real".to_string()] + ); + } + + #[test] + fn js_ignores_comments_and_strings() { + let text = "\ +// export function ghost() {} +const s = \"export function fake() {}\"; +/* export function block() {} */ +export function real() {} +export { kept as publicName, type Kind } +"; + assert_eq!( + js_public_symbols(text), + vec![ + "Kind".to_string(), + "publicName".to_string(), + "real".to_string() + ] + ); + } +} diff --git a/src/text_out.rs b/src/text_out.rs new file mode 100644 index 0000000..fa8d780 --- /dev/null +++ b/src/text_out.rs @@ -0,0 +1,278 @@ +//! Prose for `wikifier --text`. The default CLI print stays JSON. + +use crate::jsonutil::Json; + +pub fn render(command: &str, value: &Json) -> Option { + if value.get("success").and_then(|s| s.as_bool()) == Some(false) { + let error = value + .get("error") + .and_then(|v| v.as_str()) + .unwrap_or("failed"); + let mut text = format!("error: {error}\n"); + if let Some(Json::Array(issues)) = value.get("issues") { + for issue in issues { + if let Some(line) = issue.as_str() { + text.push_str(&format!(" - {line}\n")); + } + } + } + return Some(text); + } + match command { + "check-changes" | "check_changes" => { + let n = int_of(value, "dirty_total"); + Some(format!( + "check_changes: {n} file(s) newly flagged Yellow.\n" + )) + } + "record-change" | "record_change" => { + let file = text_of(value, "file"); + if value.get("duplicate_skipped").and_then(|v| v.as_bool()) == Some(true) { + Some(format!( + "Recorded change to {file} was already journaled in the last 120 seconds.\n" + )) + } else { + Some(format!("Recorded change to {file}: Yellow, pending + journal updated. Refresh its wiki, then mark_green.\n")) + } + } + "mark-green" | "mark_green" => { + let file = text_of(value, "file"); + let verification = text_of(value, "verification"); + Some(format!("Marked {file} Green ({verification}).\n")) + } + "record-deletion" | "record_deletion" => Some(format!( + "Recorded deletion of {}.\n", + text_of(value, "file") + )), + "health" => Some(health_text(value)), + "suggest-next" | "suggest" | "suggest-next-actions" | "suggest_next_actions" => { + Some(actions_text(value)) + } + "cycles" | "get_cycles" => Some(cycles_text(value)), + "journal" => Some(journal_text(value)), + "init" => Some(init_text(value)), + "heal-stubs" | "heal_stubs" => { + let n = int_of(value, "healed_count"); + let verb = if value.get("dry_run").and_then(|v| v.as_bool()) == Some(true) { + "Would heal" + } else { + "Healed" + }; + let noun = if n == 1 { "entry" } else { "entries" }; + Some(format!("{verb} {n} stub {noun}.\n")) + } + "daemon" => Some(daemon_text(value)), + "issues" => Some(actions_text(value)), + _ => None, + } +} + +fn health_text(value: &Json) -> String { + if value.get("entries").is_none() { + if let Some(summary) = value.get("health_summary") { + return format!("# Health summary\n\n{summary}\n"); + } + return format!("{value}\n"); + } + let mut lines = vec!["# Documentation Health Matrix".to_string(), String::new()]; + let entries = value + .get("entries") + .and_then(|v| v.as_array()) + .map(|items| items.to_vec()) + .unwrap_or_default(); + for entry in entries.iter().take(60) { + let file = entry.get("file").and_then(|v| v.as_str()).unwrap_or(""); + let status = entry.get("status").and_then(|v| v.as_str()).unwrap_or(""); + let updated = entry + .get("last_updated") + .and_then(|v| v.as_str()) + .unwrap_or(""); + let reason = entry.get("reason").and_then(|v| v.as_str()).unwrap_or(""); + let reason = if reason.chars().count() > 80 { + reason.chars().take(80).collect() + } else { + reason.to_string() + }; + lines.push(format!("- {file}: {status} | {updated} | {reason}")); + } + if entries.len() > 60 { + lines.push(format!( + "... ({} more; use --json or --summary)", + entries.len() - 60 + )); + } + lines.push(String::new()); + lines.join("\n") +} + +fn actions_text(value: &Json) -> String { + let actions = value + .get("actions") + .and_then(|v| v.as_array()) + .map(|items| items.to_vec()) + .unwrap_or_default(); + if actions.is_empty() { + return "No suggested actions.\n".into(); + } + let mut lines = Vec::new(); + for (i, action) in actions.iter().enumerate() { + let reason = action.get("reason").and_then(|v| v.as_str()).unwrap_or(""); + let file = action.get("file").and_then(|v| v.as_str()).unwrap_or(""); + if file.is_empty() { + lines.push(format!("{}. {reason}", i + 1)); + } else { + lines.push(format!("{}. {file}: {reason}", i + 1)); + } + } + lines.push(String::new()); + lines.join("\n") +} + +fn cycles_text(value: &Json) -> String { + let count = int_of(value, "count"); + let mut lines = vec![ + "=== Circular Dependencies ===".to_string(), + format!("Clusters: {count}"), + ]; + if let Some(Json::Array(cycles)) = value.get("cycles") { + if cycles.is_empty() { + lines.push("No circular dependencies detected.".into()); + } + for (i, cycle) in cycles.iter().enumerate() { + let files = cycle + .get("files") + .and_then(|v| v.as_array()) + .map(|items| { + items + .iter() + .filter_map(|v| v.as_str()) + .collect::>() + .join(" → ") + }) + .unwrap_or_default(); + lines.push(format!("{}. {files}", i + 1)); + if let Some(severity) = cycle.get("severity").and_then(|v| v.as_str()) { + let score = cycle + .get("score") + .map(|v| v.to_string()) + .unwrap_or_default(); + let blast = cycle + .get("external_blast_radius") + .map(|v| v.to_string()) + .unwrap_or_default(); + let strategy = cycle + .get("recommendation") + .and_then(|v| v.get("strategy")) + .and_then(|v| v.as_str()) + .unwrap_or(""); + lines.push(format!( + " severity={severity} score={score} blast={blast} fix={strategy}" + )); + } + } + } + if let Some(Json::Array(ranked)) = value.get("reverse_dependencies") { + if !ranked.is_empty() { + lines.push(String::new()); + lines.push("Most depended-on files:".into()); + for row in ranked.iter().take(10) { + let file = row.get("file").and_then(|v| v.as_str()).unwrap_or(""); + let n = row + .get("dependents") + .map(|v| v.to_string()) + .unwrap_or_default(); + lines.push(format!("- {file} ({n})")); + } + } + } + lines.push(String::new()); + lines.join("\n") +} + +fn journal_text(value: &Json) -> String { + let entries = value + .get("entries") + .and_then(|v| v.as_array()) + .map(|items| items.to_vec()) + .unwrap_or_default(); + if entries.is_empty() { + let date = value + .get("date") + .and_then(|v| v.as_str()) + .unwrap_or("today"); + return format!("No journal entries for {date} yet.\n"); + } + let mut text = String::new(); + for entry in entries { + let stamp = entry.get("stamp").and_then(|v| v.as_str()).unwrap_or(""); + let action = entry.get("action").and_then(|v| v.as_str()).unwrap_or(""); + let file = entry.get("file").and_then(|v| v.as_str()).unwrap_or(""); + let reason = entry.get("reason").and_then(|v| v.as_str()).unwrap_or(""); + text.push_str(&format!( + "## [{stamp}] {action}\n**File:** {file}\n**Reason:** {reason}\n\n" + )); + } + text +} + +fn init_text(value: &Json) -> String { + let mut lines = vec!["[wikifier] Initialised the project".to_string()]; + if let Some(Json::Array(created)) = value.get("created") { + for item in created { + if let Some(name) = item.as_str() { + lines.push(format!(" created {name}")); + } + } + } + if value.get("index_copied").and_then(|v| v.as_bool()) == Some(false) { + lines.push(" left index.html uncopied (--no-copy)".into()); + } + lines.push("Next steps:".into()); + lines.push(" 1. Edit monitored_paths.txt and map_paths.txt".into()); + lines.push(" 2. wikifier update-maps".into()); + lines.push(" 3. wikifier session-bootstrap".into()); + lines.push(String::new()); + lines.join("\n") +} + +fn daemon_text(value: &Json) -> String { + let action = text_of(value, "action"); + if action == "logs" { + let log = value.get("log").and_then(|v| v.as_str()).unwrap_or(""); + if log.is_empty() { + return "No daemon log file found yet.\n".into(); + } + return format!("{log}\n"); + } + if action == "status" || action == "start" || action == "stop" || action == "restart" { + let running = value + .get("running") + .and_then(|v| v.as_bool()) + .unwrap_or(false); + let pid = value.get("pid").map(|v| v.to_string()).unwrap_or_default(); + if running { + return format!("Wikifier daemon is running (PID {pid}).\n"); + } + return "Wikifier daemon is not running.\n".into(); + } + value + .get("message") + .and_then(|v| v.as_str()) + .map(|s| format!("{s}\n")) + .unwrap_or_else(|| format!("{value}\n")) +} + +fn text_of(value: &Json, key: &str) -> String { + value + .get(key) + .and_then(|v| v.as_str()) + .unwrap_or("") + .to_string() +} + +fn int_of(value: &Json, key: &str) -> i64 { + match value.get(key) { + Some(Json::Int(n)) => *n, + _ => 0, + } +} diff --git a/src/timeutil.rs b/src/timeutil.rs index 0959185..c1e02b7 100644 --- a/src/timeutil.rs +++ b/src/timeutil.rs @@ -45,7 +45,10 @@ pub fn local_stamp() -> String { pub fn local_stamp() -> String { // Windows is out of the ship promise. UTC is a visible fallback, not a TZif reader. use std::time::{SystemTime, UNIX_EPOCH}; - let secs = SystemTime::now().duration_since(UNIX_EPOCH).map(|d| d.as_secs()).unwrap_or(0); + let secs = SystemTime::now() + .duration_since(UNIX_EPOCH) + .map(|d| d.as_secs()) + .unwrap_or(0); format_unix(secs) } @@ -63,13 +66,34 @@ fn format_unix(secs: u64) -> String { rem -= diy; y += 1; } - let mdays = [31, if is_leap(y) { 29 } else { 28 }, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31]; + let mdays = [ + 31, + if is_leap(y) { 29 } else { 28 }, + 31, + 30, + 31, + 30, + 31, + 31, + 30, + 31, + 30, + 31, + ]; let mut m = 0; while m < 12 && rem >= mdays[m] { rem -= mdays[m]; m += 1; } - format!("{:04}-{:02}-{:02} {:02}:{:02}:{:02}", y, m + 1, rem + 1, tod / 3600, (tod % 3600) / 60, tod % 60) + format!( + "{:04}-{:02}-{:02} {:02}:{:02}:{:02}", + y, + m + 1, + rem + 1, + tod / 3600, + (tod % 3600) / 60, + tod % 60 + ) } #[cfg(windows)] @@ -80,7 +104,11 @@ fn is_leap(y: i32) -> bool { pub fn date_parts(stamp: &str) -> Option<(String, String, String)> { let date = stamp.split(' ').next()?; let mut p = date.split('-'); - Some((p.next()?.to_string(), p.next()?.to_string(), p.next()?.to_string())) + Some(( + p.next()?.to_string(), + p.next()?.to_string(), + p.next()?.to_string(), + )) } pub fn flush_stdout(text: &str) { diff --git a/src/wiki_files.rs b/src/wiki_files.rs new file mode 100644 index 0000000..c629ebd --- /dev/null +++ b/src/wiki_files.rs @@ -0,0 +1,47 @@ +//! Where a source file's wiki lives. +//! Sibling `.wiki.md` wins, then the same fallbacks the Python package used. + +use std::path::{Path, PathBuf}; + +const WIKI_DIRS: &[&str] = &["docs/wiki", "docs", "wiki", "documentation", ".wiki"]; + +/// First existing wiki for `rel`, or None when the source has no companion summary. +pub fn find_wiki(root: &Path, rel: &str) -> Option { + let rel_path = Path::new(rel); + let source = root.join(rel); + let mut candidates = Vec::new(); + candidates.push(root.join(format!("{rel}.wiki.md"))); + candidates.push(root.join(format!("{rel}.md"))); + if let (Some(parent), Some(stem)) = (rel_path.parent(), rel_path.file_stem()) { + let stem = stem.to_string_lossy(); + candidates.push(root.join(parent).join(format!("{stem}.wiki.md"))); + candidates.push(root.join(parent).join(format!("{stem}.md"))); + } + let name = rel_path + .file_name() + .map(|s| s.to_string_lossy().to_string()) + .unwrap_or_default(); + let stem = rel_path + .file_stem() + .map(|s| s.to_string_lossy().to_string()) + .unwrap_or_default(); + for dir in WIKI_DIRS { + candidates.push(root.join(dir).join(format!("{name}.wiki.md"))); + candidates.push(root.join(dir).join(format!("{name}.md"))); + candidates.push(root.join(dir).join(format!("{stem}.wiki.md"))); + candidates.push(root.join(dir).join(format!("{stem}.md"))); + } + candidates + .into_iter() + .find(|path| path.is_file() && path != &source) +} + +pub fn read_wiki(root: &Path, rel: &str) -> (Option, String) { + match find_wiki(root, rel) { + Some(path) => { + let text = std::fs::read_to_string(&path).unwrap_or_default(); + (Some(path), text) + } + None => (None, String::new()), + } +} diff --git a/tests/behaviour_gaps.rs b/tests/behaviour_gaps.rs index 63454d6..d659294 100644 --- a/tests/behaviour_gaps.rs +++ b/tests/behaviour_gaps.rs @@ -9,10 +9,15 @@ use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH}; use wikifier::cache::indexed_rows; use wikifier::jsonutil::{parse, Json}; use wikifier::mcp; -use wikifier::{load_dependencies, mapped_edges, mark_green, read_health, session_bootstrap, update_maps}; +use wikifier::{ + load_dependencies, mapped_edges, mark_green, read_health, session_bootstrap, update_maps, +}; fn scratch(name: &str) -> PathBuf { - let n = SystemTime::now().duration_since(UNIX_EPOCH).unwrap().as_nanos(); + let n = SystemTime::now() + .duration_since(UNIX_EPOCH) + .unwrap() + .as_nanos(); let dir = std::env::temp_dir().join(format!("wk-gap-{name}-{n}")); let _ = std::fs::remove_dir_all(&dir); std::fs::create_dir_all(dir.join("src")).unwrap(); @@ -27,7 +32,10 @@ fn json_text(value: &Json) -> String { fn mcp_call(bin: &str, deadline: Option<&str>, body: &str) -> (Json, Duration) { let mut cmd = Command::new(bin); - cmd.arg("mcp").stdin(Stdio::piped()).stdout(Stdio::piped()).stderr(Stdio::null()); + cmd.arg("mcp") + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .stderr(Stdio::null()); match deadline { Some(value) => { cmd.env("WIKIFIER_MCP_DEADLINE_S", value); @@ -51,14 +59,21 @@ fn mcp_call(bin: &str, deadline: Option<&str>, body: &str) -> (Json, Duration) { } fn tool_body(root: &std::path::Path, name: &str) -> String { - let root_text = root.to_str().unwrap().replace('\\', "\\\\").replace('"', "\\\""); + let root_text = root + .to_str() + .unwrap() + .replace('\\', "\\\\") + .replace('"', "\\\""); format!( r#"{{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{{"name":"{name}","arguments":{{"project_root":"{root_text}"}}}}}}"# ) } fn structured(message: &Json) -> &Json { - message.get("result").and_then(|result| result.get("structuredContent")).unwrap_or_else(|| panic!("no structured content in {message}")) + message + .get("result") + .and_then(|result| result.get("structuredContent")) + .unwrap_or_else(|| panic!("no structured content in {message}")) } #[test] @@ -69,25 +84,83 @@ fn mcp_deadline_stops_a_blocking_call_and_lets_a_fast_call_finish() { let (fast, fast_elapsed) = mcp_call(bin, Some("1"), &tool_body(&root, "health")); let fast_body = structured(&fast); - assert_eq!(fast_body.get("success").and_then(|flag| flag.as_bool()), Some(true), "{fast}"); - assert!(fast_elapsed < Duration::from_secs(1), "health took {fast_elapsed:?}"); + assert_eq!( + fast_body.get("success").and_then(|flag| flag.as_bool()), + Some(true), + "{fast}" + ); + assert!( + fast_elapsed < Duration::from_secs(1), + "health took {fast_elapsed:?}" + ); let (slow, slow_elapsed) = mcp_call(bin, Some("1"), &tool_body(&root, "monitor")); let slow_body = structured(&slow); - assert_eq!(slow_body.get("success").and_then(|flag| flag.as_bool()), Some(false), "{slow}"); - assert_eq!(slow_body.get("error_type").and_then(|flag| flag.as_str()), Some("DeadlineExceeded"), "{slow}"); - assert_eq!(slow_body.get("timed_out").and_then(|flag| flag.as_bool()), Some(true), "{slow}"); - assert_eq!(slow_body.get("may_still_complete").and_then(|flag| flag.as_bool()), Some(true), "{slow}"); - assert!(slow_body.get("error").and_then(|flag| flag.as_str()).unwrap_or("").contains("WIKIFIER_MCP_DEADLINE_S") || slow_body.to_string().contains("deadline"), "{slow}"); - assert!(slow_elapsed >= Duration::from_millis(700), "deadline 1 returned in {slow_elapsed:?}"); - assert!(slow_elapsed < Duration::from_secs(8), "deadline 1 hung for {slow_elapsed:?}"); - assert_eq!(fast.get("result").and_then(|result| result.get("isError")).and_then(|flag| flag.as_bool()), Some(false), "{fast}"); - assert_eq!(slow.get("result").and_then(|result| result.get("isError")).and_then(|flag| flag.as_bool()), Some(true), "{slow}"); + assert_eq!( + slow_body.get("success").and_then(|flag| flag.as_bool()), + Some(false), + "{slow}" + ); + assert_eq!( + slow_body.get("error_type").and_then(|flag| flag.as_str()), + Some("DeadlineExceeded"), + "{slow}" + ); + assert_eq!( + slow_body.get("timed_out").and_then(|flag| flag.as_bool()), + Some(true), + "{slow}" + ); + assert_eq!( + slow_body + .get("may_still_complete") + .and_then(|flag| flag.as_bool()), + Some(true), + "{slow}" + ); + assert!( + slow_body + .get("error") + .and_then(|flag| flag.as_str()) + .unwrap_or("") + .contains("WIKIFIER_MCP_DEADLINE_S") + || slow_body.to_string().contains("deadline"), + "{slow}" + ); + assert!( + slow_elapsed >= Duration::from_millis(700), + "deadline 1 returned in {slow_elapsed:?}" + ); + assert!( + slow_elapsed < Duration::from_secs(8), + "deadline 1 hung for {slow_elapsed:?}" + ); + assert_eq!( + fast.get("result") + .and_then(|result| result.get("isError")) + .and_then(|flag| flag.as_bool()), + Some(false), + "{fast}" + ); + assert_eq!( + slow.get("result") + .and_then(|result| result.get("isError")) + .and_then(|flag| flag.as_bool()), + Some(true), + "{slow}" + ); let (zero, zero_elapsed) = mcp_call(bin, Some("0"), &tool_body(&root, "monitor")); let zero_body = structured(&zero); - assert_eq!(zero_body.get("timed_out").and_then(|flag| flag.as_bool()), Some(true), "{zero}"); - assert!(zero_elapsed >= Duration::from_millis(700), "deadline 0 was not raised to 1s ({zero_elapsed:?})"); + assert_eq!( + zero_body.get("timed_out").and_then(|flag| flag.as_bool()), + Some(true), + "{zero}" + ); + assert!( + zero_elapsed >= Duration::from_millis(700), + "deadline 0 was not raised to 1s ({zero_elapsed:?})" + ); let _ = std::fs::remove_dir_all(&root); } @@ -98,7 +171,11 @@ fn check_changes_cap_stops_the_pass_and_names_the_env_var() { std::fs::write(root.join("src").join(name), format!("{name} = 1\n")).unwrap(); } let mapped = update_maps(&root); - assert_eq!(mapped.get("success").and_then(|flag| flag.as_bool()), Some(true), "{mapped}"); + assert_eq!( + mapped.get("success").and_then(|flag| flag.as_bool()), + Some(true), + "{mapped}" + ); let bin = env!("CARGO_BIN_EXE_wikifier"); let capped = Command::new(bin) @@ -106,11 +183,26 @@ fn check_changes_cap_stops_the_pass_and_names_the_env_var() { .env("WIKIFIER_CHECK_CHANGES_MAX", "1") .output() .unwrap(); - let capped_json = parse(&String::from_utf8(capped.stdout.clone()).unwrap()).unwrap_or_else(|err| panic!("{err}: {}", String::from_utf8_lossy(&capped.stdout))); - assert_eq!(capped_json.get("success").and_then(|flag| flag.as_bool()), Some(true), "{capped_json}"); - assert_eq!(capped_json.get("truncated").and_then(|flag| flag.as_bool()), Some(true), "{capped_json}"); - assert!(json_text(&capped_json).contains("WIKIFIER_CHECK_CHANGES_MAX"), "{capped_json}"); - assert!(json_text(&capped_json).contains("stopped at 1 files"), "{capped_json}"); + let capped_json = parse(&String::from_utf8(capped.stdout.clone()).unwrap()) + .unwrap_or_else(|err| panic!("{err}: {}", String::from_utf8_lossy(&capped.stdout))); + assert_eq!( + capped_json.get("success").and_then(|flag| flag.as_bool()), + Some(true), + "{capped_json}" + ); + assert_eq!( + capped_json.get("truncated").and_then(|flag| flag.as_bool()), + Some(true), + "{capped_json}" + ); + assert!( + json_text(&capped_json).contains("WIKIFIER_CHECK_CHANGES_MAX"), + "{capped_json}" + ); + assert!( + json_text(&capped_json).contains("stopped at 1 files"), + "{capped_json}" + ); let open = Command::new(bin) .args(["--target", root.to_str().unwrap(), "check-changes"]) @@ -126,27 +218,45 @@ fn check_changes_cap_stops_the_pass_and_names_the_env_var() { .output() .unwrap(); let ignored_json = parse(&String::from_utf8(ignored.stdout).unwrap()).unwrap(); - assert_eq!(ignored_json.get("truncated"), None, "a non-numeric cap must stay at the default: {ignored_json}"); + assert_eq!( + ignored_json.get("truncated"), + None, + "a non-numeric cap must stay at the default: {ignored_json}" + ); let _ = std::fs::remove_dir_all(&root); } fn sqlite3_available() -> bool { - Command::new("sqlite3").arg("-version").output().map(|out| out.status.success()).unwrap_or(false) + Command::new("sqlite3") + .arg("-version") + .output() + .map(|out| out.status.success()) + .unwrap_or(false) } #[test] fn sqlite_import_fills_an_empty_cache_and_reports_an_unreadable_file() { let broken = scratch("sqlite-bad"); std::fs::create_dir_all(broken.join(".wikifier_staging")).unwrap(); - std::fs::write(broken.join(".wikifier_staging/import_cache.sqlite"), b"not a sqlite database").unwrap(); + std::fs::write( + broken.join(".wikifier_staging/import_cache.sqlite"), + b"not a sqlite database", + ) + .unwrap(); std::fs::write(broken.join("src/a.py"), "x = 1\n").unwrap(); let boot = session_bootstrap(&broken); - assert_eq!(boot.get("success").and_then(|flag| flag.as_bool()), Some(true), "{boot}"); + assert_eq!( + boot.get("success").and_then(|flag| flag.as_bool()), + Some(true), + "{boot}" + ); let warnings = json_text(boot.get("warnings").unwrap_or(&Json::Null)); assert!(warnings.contains("sqlite_present_unread"), "{boot}"); let _ = std::fs::remove_dir_all(&broken); if !sqlite3_available() { - eprintln!("sqlite3 CLI is not on PATH; the import assertions run where the CLI is installed"); + eprintln!( + "sqlite3 CLI is not on PATH; the import assertions run where the CLI is installed" + ); return; } @@ -163,11 +273,27 @@ fn sqlite_import_fills_an_empty_cache_and_reports_an_unreadable_file() { assert!(created.success(), "sqlite3 could not create the cache"); let deps = load_dependencies(&root, "src/a.py"); - assert_eq!(deps.get("success").and_then(|flag| flag.as_bool()), Some(true), "{deps}"); - let internal = deps.get("internal").and_then(|item| item.as_array()).unwrap_or(&[]); - assert!(internal.iter().any(|edge| edge.get("path").and_then(|path| path.as_str()) == Some("src/b.py")), "{deps}"); + assert_eq!( + deps.get("success").and_then(|flag| flag.as_bool()), + Some(true), + "{deps}" + ); + let internal = deps + .get("internal") + .and_then(|item| item.as_array()) + .unwrap_or(&[]); + assert!( + internal + .iter() + .any(|edge| edge.get("path").and_then(|path| path.as_str()) == Some("src/b.py")), + "{deps}" + ); let rows = indexed_rows(&root); - assert!(rows.iter().any(|row| row.0 == "src/a.py" && row.3 == "sha256:legacy" && row.1 == 0), "{rows:?}"); + assert!( + rows.iter() + .any(|row| row.0 == "src/a.py" && row.3 == "sha256:legacy" && row.1 == 0), + "{rows:?}" + ); let extra = Command::new("sqlite3") .arg(&db) @@ -176,8 +302,18 @@ fn sqlite_import_fills_an_empty_cache_and_reports_an_unreadable_file() { .unwrap(); assert!(extra.success()); let again = load_dependencies(&root, "src/a.py"); - let paths: Vec<_> = again.get("internal").and_then(|item| item.as_array()).unwrap_or(&[]).iter().filter_map(|edge| edge.get("path").and_then(|path| path.as_str())).collect(); - assert_eq!(paths, ["src/b.py"], "a filled cache must ignore a later sqlite file: {again}"); + let paths: Vec<_> = again + .get("internal") + .and_then(|item| item.as_array()) + .unwrap_or(&[]) + .iter() + .filter_map(|edge| edge.get("path").and_then(|path| path.as_str())) + .collect(); + assert_eq!( + paths, + ["src/b.py"], + "a filled cache must ignore a later sqlite file: {again}" + ); let _ = std::fs::remove_dir_all(&root); } @@ -195,24 +331,48 @@ fn actionable_low_conf_edges_keep_conditional_and_drop_dynamic() { "from . import b\n\ndef load():\n from . import c\n import importlib\n return importlib.import_module('pkg.d')\n", ).unwrap(); let mapped = update_maps(&root); - assert_eq!(mapped.get("success").and_then(|flag| flag.as_bool()), Some(true), "{mapped}"); + assert_eq!( + mapped.get("success").and_then(|flag| flag.as_bool()), + Some(true), + "{mapped}" + ); - let root_text = root.to_str().unwrap().replace('\\', "\\\\").replace('"', "\\\""); + let root_text = root + .to_str() + .unwrap() + .replace('\\', "\\\\") + .replace('"', "\\\""); let raw = format!( r#"{{"jsonrpc":"2.0","id":7,"method":"tools/call","params":{{"name":"get_resolution_diagnostics","arguments":{{"project_root":"{root_text}"}}}}}}"# ); let message = mcp::handle_message(&parse(&raw).unwrap()).unwrap(); let body = structured(&message); - assert_eq!(body.get("success").and_then(|flag| flag.as_bool()), Some(true), "{body}"); - let low = body.get("actionable_low_conf_edges").and_then(|item| item.as_array()).unwrap_or(&[]); + assert_eq!( + body.get("success").and_then(|flag| flag.as_bool()), + Some(true), + "{body}" + ); + let low = body + .get("actionable_low_conf_edges") + .and_then(|item| item.as_array()) + .unwrap_or(&[]); assert!(low.iter().any(|edge| { edge.get("file").and_then(|value| value.as_str()) == Some("src/pkg/a.py") && edge.get("path").and_then(|value| value.as_str()) == Some("src/pkg/c.py") && edge.get("reason").and_then(|value| value.as_str()) == Some("conditional_guard") && matches!(edge.get("confidence"), Some(Json::Float(n)) if (*n - 0.55).abs() < 0.001) }), "{body}"); - assert!(low.iter().all(|edge| edge.get("path").and_then(|value| value.as_str()) != Some("src/pkg/b.py")), "static edge was treated as low confidence: {body}"); - assert!(low.iter().all(|edge| edge.get("reason").and_then(|value| value.as_str()) != Some("dynamic_literal")), "dynamic edges are noise, not actionable: {body}"); + assert!( + low.iter() + .all(|edge| edge.get("path").and_then(|value| value.as_str()) != Some("src/pkg/b.py")), + "static edge was treated as low confidence: {body}" + ); + assert!( + low.iter() + .all(|edge| edge.get("reason").and_then(|value| value.as_str()) + != Some("dynamic_literal")), + "dynamic edges are noise, not actionable: {body}" + ); let counts = body.get("reason_code_counts").unwrap_or(&Json::Null); assert!(counts.get("conditional_guard").is_some(), "{counts}"); assert!(counts.get("dynamic_literal").is_some(), "{counts}"); @@ -250,29 +410,93 @@ impl Hidden { } "; std::fs::write(root.join("src/lib.rs"), source).unwrap(); - std::fs::write(root.join("src/lib.rs.wiki.md"), "Widget has documented, FLAG, and fmt.\n").unwrap(); + std::fs::write( + root.join("src/lib.rs.wiki.md"), + "Widget has documented, FLAG, and fmt.\n", + ) + .unwrap(); let marked = mark_green(&root, "src/lib.rs", "wiki names the methods", false); - assert_eq!(marked.get("verification").and_then(|flag| flag.as_str()), Some("verified"), "{marked}"); - let row = read_health(&root).into_iter().find(|row| row.file == "src/lib.rs").unwrap(); + assert_eq!( + marked.get("verification").and_then(|flag| flag.as_str()), + Some("verified"), + "{marked}" + ); + let row = read_health(&root) + .into_iter() + .find(|row| row.file == "src/lib.rs") + .unwrap(); for name in ["Widget", "documented", "FLAG", "fmt"] { - assert!(row.public_symbols.contains(name), "missing {name} in {}", row.public_symbols); + assert!( + row.public_symbols.contains(name), + "missing {name} in {}", + row.public_symbols + ); } - for name in ["crate_only", "super_only", "private", "drop", "not_exported", "Hidden"] { - assert!(!row.public_symbols.split(['\n', ',']).any(|have| have.trim() == name), "{name} should stay out: {}", row.public_symbols); + for name in [ + "crate_only", + "super_only", + "private", + "drop", + "not_exported", + "Hidden", + ] { + assert!( + !row.public_symbols + .split(['\n', ',']) + .any(|have| have.trim() == name), + "{name} should stay out: {}", + row.public_symbols + ); } - std::fs::write(root.join("src/lib.rs.wiki.md"), "Widget has FLAG and fmt.\n").unwrap(); + std::fs::write( + root.join("src/lib.rs.wiki.md"), + "Widget has FLAG and fmt.\n", + ) + .unwrap(); let missing = mark_green(&root, "src/lib.rs", "wiki dropped documented", false); - assert_eq!(missing.get("error_type").and_then(|flag| flag.as_str()), Some("WikiVerificationError"), "{missing}"); - assert!(missing.to_string().contains("undocumented_new_symbols: documented"), "{missing}"); + assert_eq!( + missing.get("error_type").and_then(|flag| flag.as_str()), + Some("WikiVerificationError"), + "{missing}" + ); + assert!( + missing + .to_string() + .contains("undocumented_new_symbols: documented"), + "{missing}" + ); - std::fs::write(root.join("src/lib.rs"), "pub struct Widget;\nimpl Widget {\n pub const FLAG: i32 = 1;\n}\n").unwrap(); - std::fs::write(root.join("src/lib.rs.wiki.md"), "Widget keeps FLAG. `documented` and `fmt` are gone.\n").unwrap(); - let removed = mark_green(&root, "src/lib.rs", "wiki still names removed methods", false); - assert_eq!(removed.get("error_type").and_then(|flag| flag.as_str()), Some("WikiVerificationError"), "{removed}"); + std::fs::write( + root.join("src/lib.rs"), + "pub struct Widget;\nimpl Widget {\n pub const FLAG: i32 = 1;\n}\n", + ) + .unwrap(); + std::fs::write( + root.join("src/lib.rs.wiki.md"), + "Widget keeps FLAG. `documented` and `fmt` are gone.\n", + ) + .unwrap(); + let removed = mark_green( + &root, + "src/lib.rs", + "wiki still names removed methods", + false, + ); + assert_eq!( + removed.get("error_type").and_then(|flag| flag.as_str()), + Some("WikiVerificationError"), + "{removed}" + ); let text = removed.to_string(); - assert!(text.contains("wiki_mentions_removed_symbols: documented"), "{removed}"); - assert!(text.contains("wiki_mentions_removed_symbols: fmt"), "{removed}"); + assert!( + text.contains("wiki_mentions_removed_symbols: documented"), + "{removed}" + ); + assert!( + text.contains("wiki_mentions_removed_symbols: fmt"), + "{removed}" + ); let _ = std::fs::remove_dir_all(&root); } @@ -302,7 +526,11 @@ fn workspace_members_resolve_across_crates_and_keep_crate_paths_local() { std::fs::write(path, text).unwrap(); } let mapped = update_maps(&root); - assert_eq!(mapped.get("success").and_then(|flag| flag.as_bool()), Some(true), "{mapped}"); + assert_eq!( + mapped.get("success").and_then(|flag| flag.as_bool()), + Some(true), + "{mapped}" + ); let edges = mapped_edges(&root); let expect = [ ("crates/alpha/src/lib.rs", "crates/alpha/src/child.rs"), @@ -311,14 +539,31 @@ fn workspace_members_resolve_across_crates_and_keep_crate_paths_local() { ("crates/beta/src/lib.rs", "crates/alpha/src/lib.rs"), ("crates/beta/src/main.rs", "crates/alpha/src/lib.rs"), ("crates/beta/tests/use_alpha.rs", "crates/alpha/src/lib.rs"), - ("crates/beta/tests/use_alpha.rs", "crates/beta/tests/use_alpha/local.rs"), + ( + "crates/beta/tests/use_alpha.rs", + "crates/beta/tests/use_alpha/local.rs", + ), ("tools/skip/src/lib.rs", "tools/skip/src/local.rs"), ("tools/skip/src/lib.rs", "crates/alpha/src/lib.rs"), ]; for (src, dst) in expect { - assert!(edges.contains(&(src.into(), dst.into())), "missing {src} -> {dst} in {edges:?}"); + assert!( + edges.contains(&(src.into(), dst.into())), + "missing {src} -> {dst} in {edges:?}" + ); } - assert!(!edges.iter().any(|(src, dst)| src.contains("skip") && dst.contains("helper")), "excluded crate resolved as a member: {edges:?}"); - assert!(!edges.iter().any(|(src, dst)| src.starts_with("crates/beta/tests/") && dst.starts_with("crates/beta/src/")), "crate:: in a test escaped into the library: {edges:?}"); + assert!( + !edges + .iter() + .any(|(src, dst)| src.contains("skip") && dst.contains("helper")), + "excluded crate resolved as a member: {edges:?}" + ); + assert!( + !edges + .iter() + .any(|(src, dst)| src.starts_with("crates/beta/tests/") + && dst.starts_with("crates/beta/src/")), + "crate:: in a test escaped into the library: {edges:?}" + ); let _ = std::fs::remove_dir_all(&root); } diff --git a/tests/cache_and_artifacts.rs b/tests/cache_and_artifacts.rs index bd14916..e151eb7 100644 --- a/tests/cache_and_artifacts.rs +++ b/tests/cache_and_artifacts.rs @@ -6,10 +6,16 @@ use std::path::PathBuf; use std::time::{SystemTime, UNIX_EPOCH}; use wikifier::cache::payload_bytes; -use wikifier::{load_dependencies, load_dependents, list_paths, read_health, read_journal, record_change, update_maps}; +use wikifier::{ + list_paths, load_dependencies, load_dependents, read_health, read_journal, record_change, + update_maps, +}; fn scratch(name: &str) -> PathBuf { - let nanos = SystemTime::now().duration_since(UNIX_EPOCH).unwrap().as_nanos(); + let nanos = SystemTime::now() + .duration_since(UNIX_EPOCH) + .unwrap() + .as_nanos(); let dir = std::env::temp_dir().join(format!("wk-{name}-{nanos}")); let _ = fs::remove_dir_all(&dir); fs::create_dir_all(dir.join("src")).unwrap(); @@ -27,39 +33,93 @@ fn one_file_update_does_not_rewrite_other_payloads_and_lookups_skip_them() { fs::write(root.join("map_paths.txt"), "src\n").unwrap(); let first = update_maps(&root); - assert_eq!(first.get("success").and_then(|s| s.as_bool()), Some(true), "{first}"); + assert_eq!( + first.get("success").and_then(|s| s.as_bool()), + Some(true), + "{first}" + ); let b_before = payload_bytes(&root, "src/pkg/b.py").expect("b payload"); let a_before = payload_bytes(&root, "src/pkg/a.py").expect("a payload"); - fs::write(root.join("src/pkg/a.py"), "from . import b\n\ndef extra():\n return 2\n").unwrap(); + fs::write( + root.join("src/pkg/a.py"), + "from . import b\n\ndef extra():\n return 2\n", + ) + .unwrap(); // mtime may not move on some filesystems within the same nanosecond; bump by rewriting after a stat let _ = fs::File::open(root.join("src/pkg/a.py")).unwrap(); let second = update_maps(&root); - assert_eq!(second.get("success").and_then(|s| s.as_bool()), Some(true), "{second}"); + assert_eq!( + second.get("success").and_then(|s| s.as_bool()), + Some(true), + "{second}" + ); let b_after = payload_bytes(&root, "src/pkg/b.py").unwrap(); let a_after = payload_bytes(&root, "src/pkg/a.py").unwrap(); assert_eq!(b_before, b_after, "untouched file payload was rewritten"); assert_ne!(a_before, a_after, "changed file payload should change"); // Corrupt every other payload. Src lookup and dst lookup read indexes only. - fs::write(wikifier::cache::payload_path(&root, "src/pkg/b.py"), b"NOT JSON").unwrap(); + fs::write( + wikifier::cache::payload_path(&root, "src/pkg/b.py"), + b"NOT JSON", + ) + .unwrap(); let deps = load_dependencies(&root, "src/pkg/a.py"); - assert_eq!(deps.get("success").and_then(|s| s.as_bool()), Some(true), "{deps}"); + assert_eq!( + deps.get("success").and_then(|s| s.as_bool()), + Some(true), + "{deps}" + ); let internal = deps.get("internal").and_then(|v| v.as_array()).unwrap(); - assert!(internal.iter().any(|v| v.get("path").and_then(|p| p.as_str()) == Some("src/pkg/b.py")), "{deps}"); - assert!(deps.get("external").and_then(|v| v.as_array()).is_some(), "{deps}"); - assert!(deps.get("needs_review").and_then(|v| v.as_array()).is_some(), "{deps}"); + assert!( + internal + .iter() + .any(|v| v.get("path").and_then(|p| p.as_str()) == Some("src/pkg/b.py")), + "{deps}" + ); + assert!( + deps.get("external").and_then(|v| v.as_array()).is_some(), + "{deps}" + ); + assert!( + deps.get("needs_review") + .and_then(|v| v.as_array()) + .is_some(), + "{deps}" + ); - fs::write(wikifier::cache::payload_path(&root, "src/pkg/a.py"), b"NOT JSON").unwrap(); + fs::write( + wikifier::cache::payload_path(&root, "src/pkg/a.py"), + b"NOT JSON", + ) + .unwrap(); let importers = load_dependents(&root, "src/pkg/b.py"); - assert_eq!(importers.get("success").and_then(|s| s.as_bool()), Some(true), "{importers}"); - let dep_list = importers.get("dependents").and_then(|v| v.as_array()).unwrap(); - assert!(dep_list.iter().any(|v| v.as_str() == Some("src/pkg/a.py")), "{importers}"); + assert_eq!( + importers.get("success").and_then(|s| s.as_bool()), + Some(true), + "{importers}" + ); + let dep_list = importers + .get("dependents") + .and_then(|v| v.as_array()) + .unwrap(); + assert!( + dep_list.iter().any(|v| v.as_str() == Some("src/pkg/a.py")), + "{importers}" + ); let listed = list_paths(&root, "src/pkg"); - assert_eq!(listed.get("success").and_then(|s| s.as_bool()), Some(true), "{listed}"); - let count = listed.get("file_count").and_then(|n| match n { wikifier::Json::Int(i) => Some(*i), _ => None }); + assert_eq!( + listed.get("success").and_then(|s| s.as_bool()), + Some(true), + "{listed}" + ); + let count = listed.get("file_count").and_then(|n| match n { + wikifier::Json::Int(i) => Some(*i), + _ => None, + }); assert!(count.unwrap_or(0) >= 2, "{listed}"); let _ = fs::remove_dir_all(&root); } @@ -85,7 +145,11 @@ fn reads_python_47_health_and_journal_without_rewriting_them() { fs::create_dir_all(root.join("journal/2026/10")).unwrap(); let journal = "## [2026-10-01 18:02:17] record-change\n**File:** src/app.py\n**Reason:** content changed since last mark-green\n\n"; fs::write(root.join("journal/2026/10/01.md"), journal).unwrap(); - fs::write(root.join("pending_updates.md"), "# Pending Updates\n\n(no pending items — run check-changes after making edits)\n").unwrap(); + fs::write( + root.join("pending_updates.md"), + "# Pending Updates\n\n(no pending items — run check-changes after making edits)\n", + ) + .unwrap(); let health_before = fs::read(root.join("file_health.json")).unwrap(); let journal_before = fs::read(root.join("journal/2026/10/01.md")).unwrap(); let md_before = fs::read(root.join("file_health.md")).unwrap(); @@ -93,7 +157,11 @@ fn reads_python_47_health_and_journal_without_rewriting_them() { let rows = read_health(&root); assert_eq!(rows.len(), 1); assert_eq!(rows[0].file, "src/app.py"); - assert!(rows[0].source_hash.starts_with("sha256:"), "{}", rows[0].source_hash); + assert!( + rows[0].source_hash.starts_with("sha256:"), + "{}", + rows[0].source_hash + ); assert!(rows[0].status.contains("Green")); let entries = read_journal(&root, Some("2026-10-01")); @@ -102,14 +170,24 @@ fn reads_python_47_health_and_journal_without_rewriting_them() { assert_eq!(entries[0].action, "record-change"); assert!(entries[0].reason.contains("mark-green")); - assert_eq!(fs::read(root.join("file_health.json")).unwrap(), health_before); - assert_eq!(fs::read(root.join("journal/2026/10/01.md")).unwrap(), journal_before); + assert_eq!( + fs::read(root.join("file_health.json")).unwrap(), + health_before + ); + assert_eq!( + fs::read(root.join("journal/2026/10/01.md")).unwrap(), + journal_before + ); assert_eq!(fs::read(root.join("file_health.md")).unwrap(), md_before); // A later record_change appends a journal entry and must not be required to migrate the old row's hash. fs::write(root.join("src/app.py"), "print(1)\n").unwrap(); let rec = record_change(&root, "src/app.py", "touch the copy only"); - assert_eq!(rec.get("success").and_then(|s| s.as_bool()), Some(true), "{rec}"); + assert_eq!( + rec.get("success").and_then(|s| s.as_bool()), + Some(true), + "{rec}" + ); let entries = read_journal(&root, None); assert!(entries.iter().any(|e| e.reason == "touch the copy only")); let rows = read_health(&root); diff --git a/tests/cli_surface.rs b/tests/cli_surface.rs index 9e79fc3..110cb0b 100644 --- a/tests/cli_surface.rs +++ b/tests/cli_surface.rs @@ -12,7 +12,10 @@ fn bin() -> PathBuf { fn project() -> PathBuf { use std::sync::atomic::{AtomicU64, Ordering}; static SEQ: AtomicU64 = AtomicU64::new(0); - let n = SystemTime::now().duration_since(UNIX_EPOCH).unwrap().as_nanos(); + let n = SystemTime::now() + .duration_since(UNIX_EPOCH) + .unwrap() + .as_nanos(); let seq = SEQ.fetch_add(1, Ordering::Relaxed); let root = std::env::temp_dir().join(format!("wk-cli-{}-{n}-{seq}", std::process::id())); std::fs::create_dir_all(root.join("src")).unwrap(); @@ -28,7 +31,10 @@ fn project() -> PathBuf { fn version_matches_package() { let out = Command::new(bin()).arg("--version").output().unwrap(); assert!(out.status.success()); - assert_eq!(String::from_utf8_lossy(&out.stdout).trim(), format!("wikifier {}", env!("CARGO_PKG_VERSION"))); + assert_eq!( + String::from_utf8_lossy(&out.stdout).trim(), + format!("wikifier {}", env!("CARGO_PKG_VERSION")) + ); } #[test] @@ -79,37 +85,80 @@ fn mcp_stdio_is_newline_delimited_json() { let mut recv = || { let mut line = String::new(); lines.read_line(&mut line).unwrap(); - assert!(!line.to_ascii_lowercase().contains("content-length"), "{line}"); - assert!(line.ends_with('\n'), "reply was not one JSON line: {line:?}"); + assert!( + !line.to_ascii_lowercase().contains("content-length"), + "{line}" + ); + assert!( + line.ends_with('\n'), + "reply was not one JSON line: {line:?}" + ); parse_line(&line) }; send(r#"{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}"#); let init = recv(); - assert_eq!(init.get("id").and_then(|v| match v { wikifier::Json::Int(n) => Some(*n), _ => None }), Some(1), "{init}"); - assert_eq!(init.get("result").and_then(|v| v.get("protocolVersion")).and_then(|v| v.as_str()), Some("2024-11-05"), "{init}"); + assert_eq!( + init.get("id").and_then(|v| match v { + wikifier::Json::Int(n) => Some(*n), + _ => None, + }), + Some(1), + "{init}" + ); + assert_eq!( + init.get("result") + .and_then(|v| v.get("protocolVersion")) + .and_then(|v| v.as_str()), + Some("2024-11-05"), + "{init}" + ); send(r#"{"jsonrpc":"2.0","id":2,"method":"tools/list"}"#); let listed = recv(); - let tools = listed.get("result").and_then(|v| v.get("tools")).and_then(|v| v.as_array()).unwrap_or_else(|| panic!("{listed}")); + let tools = listed + .get("result") + .and_then(|v| v.get("tools")) + .and_then(|v| v.as_array()) + .unwrap_or_else(|| panic!("{listed}")); assert_eq!(tools.len(), wikifier::api::MCP_TOOLS.len()); for tool in tools { let name = tool.get("name").and_then(|v| v.as_str()).unwrap_or(""); - let description = tool.get("description").and_then(|v| v.as_str()).unwrap_or(""); + let description = tool + .get("description") + .and_then(|v| v.as_str()) + .unwrap_or(""); assert!(description.len() > 40, "{name} description {description}"); - assert!(!description.starts_with("Wikifier "), "{name} still has a placeholder description"); - let required = tool.get("inputSchema").and_then(|v| v.get("required")).and_then(|v| v.as_array()).unwrap_or_else(|| panic!("{name} schema {tool}")); - assert!(required.iter().any(|v| v.as_str() == Some("project_root")), "{name} required {required:?}"); + assert!( + !description.starts_with("Wikifier "), + "{name} still has a placeholder description" + ); + let required = tool + .get("inputSchema") + .and_then(|v| v.get("required")) + .and_then(|v| v.as_array()) + .unwrap_or_else(|| panic!("{name} schema {tool}")); + assert!( + required.iter().any(|v| v.as_str() == Some("project_root")), + "{name} required {required:?}" + ); } - let root_text = root.to_str().unwrap().replace('\\', "\\\\").replace('"', "\\\""); + let root_text = root + .to_str() + .unwrap() + .replace('\\', "\\\\") + .replace('"', "\\\""); send(&format!( r#"{{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{{"name":"get_current_project_root","arguments":{{"project_root":"{root_text}"}}}}}}"# )); let called = recv(); let text = called.to_string(); assert!(text.contains("\"success\":true"), "{called}"); - assert!(text.contains(&root.file_name().unwrap().to_string_lossy().to_string()), "{called}"); + assert!( + text.contains(&root.file_name().unwrap().to_string_lossy().to_string()), + "{called}" + ); drop(stdin); let status = child.wait().unwrap(); assert!(status.success(), "mcp exit {status}"); diff --git a/tests/contract_gaps.rs b/tests/contract_gaps.rs index 3fcfb7b..c61477a 100644 --- a/tests/contract_gaps.rs +++ b/tests/contract_gaps.rs @@ -5,11 +5,17 @@ use std::path::PathBuf; use std::process::Command; use std::time::{SystemTime, UNIX_EPOCH}; -use wikifier::api::{check_changes, cycles, load_dependencies, mark_green, read_health, read_journal, record_changes, serve_once, session_bootstrap, update_maps, validate, verify_wikis}; +use wikifier::api::{ + check_changes, cycles, load_dependencies, mark_green, read_health, read_journal, + record_changes, serve_once, session_bootstrap, update_maps, validate, verify_wikis, +}; use wikifier::jsonutil::Json; fn scratch(name: &str) -> PathBuf { - let n = SystemTime::now().duration_since(UNIX_EPOCH).unwrap().as_nanos(); + let n = SystemTime::now() + .duration_since(UNIX_EPOCH) + .unwrap() + .as_nanos(); let dir = std::env::temp_dir().join(format!("wk-gap-{name}-{n}")); let _ = fs::remove_dir_all(&dir); fs::create_dir_all(dir.join("src")).unwrap(); @@ -29,23 +35,56 @@ fn int_field(v: &Json, key: &str) -> i64 { fn check_changes_yellows_green_from_health_baseline_and_update_maps_keeps_it() { let root = scratch("dirty"); fs::write(root.join("src/a.py"), "def hello():\n return 1\n").unwrap(); - assert_eq!(update_maps(&root).get("success").and_then(|s| s.as_bool()), Some(true)); + assert_eq!( + update_maps(&root).get("success").and_then(|s| s.as_bool()), + Some(true) + ); let marked = mark_green(&root, "src/a.py", "wiki matches", false); - assert_eq!(marked.get("success").and_then(|s| s.as_bool()), Some(true), "{marked}"); + assert_eq!( + marked.get("success").and_then(|s| s.as_bool()), + Some(true), + "{marked}" + ); fs::write(root.join("src/a.py"), "def hello():\n return 2\n").unwrap(); let dirty = check_changes(&root); - assert_eq!(dirty.get("success").and_then(|s| s.as_bool()), Some(true), "{dirty}"); + assert_eq!( + dirty.get("success").and_then(|s| s.as_bool()), + Some(true), + "{dirty}" + ); assert!(int_field(&dirty, "dirty_total") >= 1, "{dirty}"); - let row = read_health(&root).into_iter().find(|r| r.file == "src/a.py").unwrap(); - assert!(row.status.contains("Yellow"), "health stayed {:?}", row.status); + let row = read_health(&root) + .into_iter() + .find(|r| r.file == "src/a.py") + .unwrap(); + assert!( + row.status.contains("Yellow"), + "health stayed {:?}", + row.status + ); assert!(row.reason.contains("content changed")); let journal = read_journal(&root, None); - assert!(journal.iter().any(|e| e.file == "src/a.py" && e.reason.contains("content changed")), "{journal:?}"); + assert!( + journal + .iter() + .any(|e| e.file == "src/a.py" && e.reason.contains("content changed")), + "{journal:?}" + ); let pending = fs::read_to_string(root.join("pending_updates.md")).unwrap_or_default(); assert!(pending.contains("src/a.py"), "{pending}"); - assert_eq!(update_maps(&root).get("success").and_then(|s| s.as_bool()), Some(true)); - let row = read_health(&root).into_iter().find(|r| r.file == "src/a.py").unwrap(); - assert!(row.status.contains("Yellow"), "update_maps cleared yellow: {}", row.status); + assert_eq!( + update_maps(&root).get("success").and_then(|s| s.as_bool()), + Some(true) + ); + let row = read_health(&root) + .into_iter() + .find(|r| r.file == "src/a.py") + .unwrap(); + assert!( + row.status.contains("Yellow"), + "update_maps cleared yellow: {}", + row.status + ); let _ = fs::remove_dir_all(&root); } @@ -55,44 +94,96 @@ fn readiness_blocks_without_health_and_refuses_reds() { fs::write(root.join("src/a.py"), "x = 1\n").unwrap(); let blocked = session_bootstrap(&root); // bootstrap builds the map and seeds health, so a fresh tree becomes ready or wiki-work, not a lie. - let ready = blocked.get("readiness").and_then(|s| s.as_str()).unwrap_or(""); + let ready = blocked + .get("readiness") + .and_then(|s| s.as_str()) + .unwrap_or(""); assert_ne!(ready, "", "{blocked}"); // Strip the health file and ask again without a map rebuild. fs::remove_file(root.join("file_health.json")).ok(); fs::remove_file(root.join("file_health.md")).ok(); let again = wikifier::health(&root); assert_eq!(int_field(again.get("health_summary").unwrap(), "total"), 0); - let status = wikifier::api::dispatch("autonomous-status", &root, &[], &std::collections::BTreeMap::new()); - assert_eq!(status.get("readiness").and_then(|s| s.as_str()), Some("blocked"), "{status}"); + let status = wikifier::api::dispatch( + "autonomous-status", + &root, + &[], + &std::collections::BTreeMap::new(), + ); + assert_eq!( + status.get("readiness").and_then(|s| s.as_str()), + Some("blocked"), + "{status}" + ); fs::write(root.join("file_health.json"), r#"{"version":2,"entries":{"src/a.py":{"status":"🔴 Red","last_updated":"2026-10-01 00:00:00","reason":"missing","source_content_hash":"sha256:abc"}}}"#).unwrap(); let red = session_bootstrap(&root); - assert_eq!(red.get("readiness").and_then(|s| s.as_str()), Some("not_ready"), "{red}"); + assert_eq!( + red.get("readiness").and_then(|s| s.as_str()), + Some("not_ready"), + "{red}" + ); let _ = fs::remove_dir_all(&root); } #[test] fn dependencies_are_compact_and_force_overrides_wiki_check() { let root = scratch("deps"); - fs::write(root.join("src/a.py"), "import os\nfrom . import b\n\ndef hello():\n return 1\n").unwrap(); + fs::write( + root.join("src/a.py"), + "import os\nfrom . import b\n\ndef hello():\n return 1\n", + ) + .unwrap(); fs::write(root.join("src/__init__.py"), "").unwrap(); fs::write(root.join("src/b.py"), "VALUE = 1\n").unwrap(); - assert_eq!(update_maps(&root).get("success").and_then(|s| s.as_bool()), Some(true)); + assert_eq!( + update_maps(&root).get("success").and_then(|s| s.as_bool()), + Some(true) + ); let deps = load_dependencies(&root, "src/a.py"); - assert_eq!(deps.get("success").and_then(|s| s.as_bool()), Some(true), "{deps}"); + assert_eq!( + deps.get("success").and_then(|s| s.as_bool()), + Some(true), + "{deps}" + ); let internal = deps.get("internal").and_then(|v| v.as_array()).unwrap(); - assert!(internal.iter().any(|e| e.get("path").and_then(|p| p.as_str()) == Some("src/b.py")), "{deps}"); - assert!(internal.iter().all(|e| e.get("line").is_some() && e.get("names").is_some()), "{deps}"); + assert!( + internal + .iter() + .any(|e| e.get("path").and_then(|p| p.as_str()) == Some("src/b.py")), + "{deps}" + ); + assert!( + internal + .iter() + .all(|e| e.get("line").is_some() && e.get("names").is_some()), + "{deps}" + ); let external = deps.get("external").and_then(|v| v.as_array()).unwrap(); assert!(external.iter().any(|e| e.as_str() == Some("os")), "{deps}"); - assert!(deps.get("needs_review").and_then(|v| v.as_array()).is_some()); + assert!(deps + .get("needs_review") + .and_then(|v| v.as_array()) + .is_some()); fs::write(root.join("src/a.py.wiki.md"), "mentions nothing useful\n").unwrap(); let refused = mark_green(&root, "src/a.py", "", false); - assert_eq!(refused.get("error_type").and_then(|s| s.as_str()), Some("WikiVerificationError"), "{refused}"); + assert_eq!( + refused.get("error_type").and_then(|s| s.as_str()), + Some("WikiVerificationError"), + "{refused}" + ); let forced = mark_green(&root, "src/a.py", "wiki is still accurate", true); - assert_eq!(forced.get("success").and_then(|s| s.as_bool()), Some(true), "{forced}"); - assert_eq!(forced.get("verification").and_then(|s| s.as_str()), Some("forced"), "{forced}"); + assert_eq!( + forced.get("success").and_then(|s| s.as_bool()), + Some(true), + "{forced}" + ); + assert_eq!( + forced.get("verification").and_then(|s| s.as_str()), + Some("forced"), + "{forced}" + ); let _ = fs::remove_dir_all(&root); } @@ -101,33 +192,83 @@ fn cycles_record_changes_validate_and_verify_are_not_canned() { let root = scratch("cmds"); fs::write(root.join("src/a.py"), "import b\n").unwrap(); fs::write(root.join("src/b.py"), "import a\n").unwrap(); - assert_eq!(update_maps(&root).get("success").and_then(|s| s.as_bool()), Some(true)); + assert_eq!( + update_maps(&root).get("success").and_then(|s| s.as_bool()), + Some(true) + ); let report = cycles(&root); - assert_eq!(report.get("success").and_then(|s| s.as_bool()), Some(true), "{report}"); + assert_eq!( + report.get("success").and_then(|s| s.as_bool()), + Some(true), + "{report}" + ); assert!(int_field(&report, "count") >= 1, "{report}"); fs::write(root.join("src/c.py"), "import missing_local_thing\n").unwrap(); let missing = validate(&root); let list = missing.get("missing").and_then(|v| v.as_array()).unwrap(); - assert!(list.iter().any(|v| v.as_str() == Some("src/c.py")), "{missing}"); + assert!( + list.iter().any(|v| v.as_str() == Some("src/c.py")), + "{missing}" + ); fs::write(root.join("src/a.py.wiki.md"), "no symbols here\n").unwrap(); let _ = mark_green(&root, "src/a.py", "force", true); // a.py public surface may be empty so verify can pass; give it a function the wiki misses. - fs::write(root.join("src/a.py"), "import b\n\ndef added():\n return 1\n").unwrap(); + fs::write( + root.join("src/a.py"), + "import b\n\ndef added():\n return 1\n", + ) + .unwrap(); let _ = mark_green(&root, "src/a.py", "force the new symbol", true); let checked = verify_wikis(&root); assert!(int_field(&checked, "checked") >= 1, "{checked}"); - assert_eq!(checked.get("success").and_then(|s| s.as_bool()), Some(false), "{checked}"); + assert_eq!( + checked.get("success").and_then(|s| s.as_bool()), + Some(false), + "{checked}" + ); let no_git = record_changes(&root, &[], None); - assert_eq!(no_git.get("error_type").and_then(|s| s.as_str()), Some("GitUnavailableError"), "{no_git}"); - Command::new("git").args(["init"]).current_dir(&root).output().unwrap(); - Command::new("git").args(["add", "-A"]).current_dir(&root).output().unwrap(); - Command::new("git").args(["-c", "user.email=t@example.com", "-c", "user.name=t", "commit", "-m", "init"]).current_dir(&root).output().unwrap(); - fs::write(root.join("src/a.py"), "import b\n\ndef added():\n return 2\n").unwrap(); + assert_eq!( + no_git.get("error_type").and_then(|s| s.as_str()), + Some("GitUnavailableError"), + "{no_git}" + ); + Command::new("git") + .args(["init"]) + .current_dir(&root) + .output() + .unwrap(); + Command::new("git") + .args(["add", "-A"]) + .current_dir(&root) + .output() + .unwrap(); + Command::new("git") + .args([ + "-c", + "user.email=t@example.com", + "-c", + "user.name=t", + "commit", + "-m", + "init", + ]) + .current_dir(&root) + .output() + .unwrap(); + fs::write( + root.join("src/a.py"), + "import b\n\ndef added():\n return 2\n", + ) + .unwrap(); let recorded = record_changes(&root, &["src/a.py=edited the cycle".into()], None); - assert_eq!(recorded.get("success").and_then(|s| s.as_bool()), Some(true), "{recorded}"); + assert_eq!( + recorded.get("success").and_then(|s| s.as_bool()), + Some(true), + "{recorded}" + ); assert!(int_field(&recorded, "recorded") >= 1, "{recorded}"); let _ = fs::remove_dir_all(&root); } @@ -136,44 +277,93 @@ fn cycles_record_changes_validate_and_verify_are_not_canned() { fn stale_wikis_prune_barrels_and_metrics_do_real_work() { let root = scratch("hygiene"); fs::write(root.join("src/index.ts"), "export * from './math';\n").unwrap(); - fs::write(root.join("src/math.ts"), "export const add = (n: number) => n;\n").unwrap(); + fs::write( + root.join("src/math.ts"), + "export const add = (n: number) => n;\n", + ) + .unwrap(); fs::write(root.join("src/a.py"), "def hello():\n return 1\n").unwrap(); let mapped = update_maps(&root); - assert_eq!(mapped.get("success").and_then(|s| s.as_bool()), Some(true), "{mapped}"); + assert_eq!( + mapped.get("success").and_then(|s| s.as_bool()), + Some(true), + "{mapped}" + ); let barrel_dir = root.join(".wikifier_staging/wk_cache/barrels"); - let barrels: Vec<_> = fs::read_dir(&barrel_dir).unwrap().flatten().map(|e| e.path()).collect(); - assert!(!barrels.is_empty(), "update_maps did not write a barrel record"); + let barrels: Vec<_> = fs::read_dir(&barrel_dir) + .unwrap() + .flatten() + .map(|e| e.path()) + .collect(); + assert!( + !barrels.is_empty(), + "update_maps did not write a barrel record" + ); let flags = std::collections::BTreeMap::new(); let snap = wikifier::api::dispatch("metrics-snapshot", &root, &[], &flags); - assert_eq!(snap.get("success").and_then(|s| s.as_bool()), Some(true), "{snap}"); - assert_eq!(snap.get("appended").and_then(|s| s.as_bool()), Some(true), "{snap}"); + assert_eq!( + snap.get("success").and_then(|s| s.as_bool()), + Some(true), + "{snap}" + ); + assert_eq!( + snap.get("appended").and_then(|s| s.as_bool()), + Some(true), + "{snap}" + ); let latest = fs::read_to_string(root.join(".wikifier_staging/metrics_latest.json")).unwrap(); assert!(latest.contains("\"total\":"), "{latest}"); let hist1 = fs::read_to_string(root.join(".wikifier_staging/metrics_history.jsonl")).unwrap(); - assert_eq!(hist1.lines().filter(|l| !l.is_empty()).count(), 1, "{hist1}"); + assert_eq!( + hist1.lines().filter(|l| !l.is_empty()).count(), + 1, + "{hist1}" + ); let snap2 = wikifier::api::dispatch("metrics-snapshot", &root, &[], &flags); - assert_eq!(snap2.get("success").and_then(|s| s.as_bool()), Some(true), "{snap2}"); + assert_eq!( + snap2.get("success").and_then(|s| s.as_bool()), + Some(true), + "{snap2}" + ); let hist2 = fs::read_to_string(root.join(".wikifier_staging/metrics_history.jsonl")).unwrap(); - assert_eq!(hist2.lines().filter(|l| !l.is_empty()).count(), 2, "{hist2}"); + assert_eq!( + hist2.lines().filter(|l| !l.is_empty()).count(), + 2, + "{hist2}" + ); fs::write(root.join("src/a.py.wiki.md"), "hello returns 1\n").unwrap(); let marked = mark_green(&root, "src/a.py", "wiki names hello", false); - assert_eq!(marked.get("success").and_then(|s| s.as_bool()), Some(true), "{marked}"); + assert_eq!( + marked.get("success").and_then(|s| s.as_bool()), + Some(true), + "{marked}" + ); let before = wikifier::api::dispatch("stale-wikis", &root, &[], &flags); let before_files = before.get("files").and_then(|v| v.as_array()).unwrap(); - assert!(before_files.iter().all(|f| f.as_str() != Some("src/a.py")), "{before}"); + assert!( + before_files.iter().all(|f| f.as_str() != Some("src/a.py")), + "{before}" + ); fs::write(root.join("src/a.py"), "def hello():\n return 2\n").unwrap(); let dirty = check_changes(&root); assert!(int_field(&dirty, "dirty_total") >= 1, "{dirty}"); let after = wikifier::api::dispatch("stale-wikis", &root, &[], &flags); let after_files = after.get("files").and_then(|v| v.as_array()).unwrap(); - assert!(after_files.iter().any(|f| f.as_str() == Some("src/a.py")), "{after}"); + assert!( + after_files.iter().any(|f| f.as_str() == Some("src/a.py")), + "{after}" + ); assert!(int_field(&after, "count") >= 1, "{after}"); fs::remove_file(root.join("src/math.ts")).unwrap(); let pruned = wikifier::api::dispatch("prune-barrels", &root, &[], &flags); - assert_eq!(pruned.get("success").and_then(|s| s.as_bool()), Some(true), "{pruned}"); + assert_eq!( + pruned.get("success").and_then(|s| s.as_bool()), + Some(true), + "{pruned}" + ); assert!(int_field(&pruned, "pruned") >= 1, "{pruned}"); let left: Vec<_> = fs::read_dir(&barrel_dir).unwrap().flatten().collect(); assert!(left.is_empty(), "barrel cache files were not removed"); @@ -186,20 +376,59 @@ fn serve_binds_and_daemon_starts() { fs::write(root.join("src/a.py"), "x = 1\n").unwrap(); let _ = update_maps(&root); let served = serve_once(&root); - assert_eq!(served.get("success").and_then(|s| s.as_bool()), Some(true), "{served}"); - assert_eq!(served.get("bound").and_then(|s| s.as_bool()), Some(true), "{served}"); - assert_eq!(served.get("served").and_then(|s| s.as_bool()), Some(true), "{served}"); + assert_eq!( + served.get("success").and_then(|s| s.as_bool()), + Some(true), + "{served}" + ); + assert_eq!( + served.get("bound").and_then(|s| s.as_bool()), + Some(true), + "{served}" + ); + assert_eq!( + served.get("served").and_then(|s| s.as_bool()), + Some(true), + "{served}" + ); assert!(int_field(&served, "port") > 0, "{served}"); let bin = env!("CARGO_BIN_EXE_wikifier"); - let start = Command::new(bin).args(["--target", root.to_str().unwrap(), "daemon", "start"]).output().unwrap(); - assert!(start.status.success(), "{}", String::from_utf8_lossy(&start.stderr)); - let start_json = wikifier::jsonutil::parse(&String::from_utf8(start.stdout.clone()).unwrap()).unwrap(); - assert_eq!(start_json.get("running").and_then(|s| s.as_bool()), Some(true), "{start_json}"); - let status = Command::new(bin).args(["--target", root.to_str().unwrap(), "daemon", "status"]).output().unwrap(); - let status_json = wikifier::jsonutil::parse(&String::from_utf8(status.stdout).unwrap()).unwrap(); - assert_eq!(status_json.get("running").and_then(|s| s.as_bool()), Some(true), "{status_json}"); - let stop = Command::new(bin).args(["--target", root.to_str().unwrap(), "daemon", "stop"]).output().unwrap(); - assert!(stop.status.success(), "{}", String::from_utf8_lossy(&stop.stderr)); + let start = Command::new(bin) + .args(["--target", root.to_str().unwrap(), "daemon", "start"]) + .output() + .unwrap(); + assert!( + start.status.success(), + "{}", + String::from_utf8_lossy(&start.stderr) + ); + let start_json = + wikifier::jsonutil::parse(&String::from_utf8(start.stdout.clone()).unwrap()).unwrap(); + assert_eq!( + start_json.get("running").and_then(|s| s.as_bool()), + Some(true), + "{start_json}" + ); + let status = Command::new(bin) + .args(["--target", root.to_str().unwrap(), "daemon", "status"]) + .output() + .unwrap(); + let status_json = + wikifier::jsonutil::parse(&String::from_utf8(status.stdout).unwrap()).unwrap(); + assert_eq!( + status_json.get("running").and_then(|s| s.as_bool()), + Some(true), + "{status_json}" + ); + let stop = Command::new(bin) + .args(["--target", root.to_str().unwrap(), "daemon", "stop"]) + .output() + .unwrap(); + assert!( + stop.status.success(), + "{}", + String::from_utf8_lossy(&stop.stderr) + ); let _ = fs::remove_dir_all(&root); } diff --git a/tests/dogfood.rs b/tests/dogfood.rs index d149a6f..72d83af 100644 --- a/tests/dogfood.rs +++ b/tests/dogfood.rs @@ -20,14 +20,42 @@ fn write_child(dir: &Path) { } fn assert_scope_risk(label: &str, value: &Json) { - assert_eq!(value.get("success").and_then(|s| s.as_bool()), Some(true), "{label} {value}"); - assert_eq!(value.get("map_complete").and_then(|s| s.as_bool()), Some(false), "{label} claimed a complete map: {value}"); - assert_eq!(value.get("readiness").and_then(|s| s.as_str()), Some("map_ok_scope_risk"), "{label} readiness: {value}"); + assert_eq!( + value.get("success").and_then(|s| s.as_bool()), + Some(true), + "{label} {value}" + ); + assert_eq!( + value.get("map_complete").and_then(|s| s.as_bool()), + Some(false), + "{label} claimed a complete map: {value}" + ); + assert_eq!( + value.get("readiness").and_then(|s| s.as_str()), + Some("map_ok_scope_risk"), + "{label} readiness: {value}" + ); let scope = value.get("scope").unwrap_or(value); - let count = scope.get("child_project_count").and_then(|n| match n { Json::Int(i) => Some(*i), _ => None }).unwrap_or(0); - let container = scope.get("container").and_then(|s| s.as_bool()).unwrap_or(false); - assert!(container || count >= 3, "{label} did not report a container scope risk: {value}"); - assert_ne!(scope.get("full_tree").and_then(|s| s.as_bool()), Some(true), "{label} full_tree walk: {value}"); + let count = scope + .get("child_project_count") + .and_then(|n| match n { + Json::Int(i) => Some(*i), + _ => None, + }) + .unwrap_or(0); + let container = scope + .get("container") + .and_then(|s| s.as_bool()) + .unwrap_or(false); + assert!( + container || count >= 3, + "{label} did not report a container scope risk: {value}" + ); + assert_ne!( + scope.get("full_tree").and_then(|s| s.as_bool()), + Some(true), + "{label} full_tree walk: {value}" + ); } #[test] @@ -40,29 +68,64 @@ fn containers_are_refused_and_scoped_children_map() { fs::write(container.join("alpha/src/pkg/__init__.py"), "").unwrap(); fs::write(container.join("alpha/src/pkg/a.py"), "from . import b\n").unwrap(); fs::write(container.join("alpha/src/pkg/b.py"), "VALUE = 1\n").unwrap(); - fs::write(container.join("alpha/src/pkg/skip.cbl"), "IDENTIFICATION DIVISION.\n").unwrap(); + fs::write( + container.join("alpha/src/pkg/skip.cbl"), + "IDENTIFICATION DIVISION.\n", + ) + .unwrap(); fs::write(container.join("beta/lib.py"), "VALUE = 1\n").unwrap(); fs::write(container.join("gamma/main.py"), "VALUE = 1\n").unwrap(); write_child(&container.join("cobol")); fs::create_dir_all(container.join("cobol/examples")).unwrap(); - fs::write(container.join("cobol/examples/hello.cbl"), "IDENTIFICATION DIVISION.\n").unwrap(); + fs::write( + container.join("cobol/examples/hello.cbl"), + "IDENTIFICATION DIVISION.\n", + ) + .unwrap(); let bin = env!("CARGO_BIN_EXE_wikifier"); - let boot = Command::new(bin).args(["--target", container.to_str().unwrap(), "session-bootstrap"]).output().unwrap(); - assert!(boot.status.success(), "bootstrap status {:?} {}", boot.status, String::from_utf8_lossy(&boot.stderr)); - let boot_json = parse(&String::from_utf8(boot.stdout.clone()).unwrap()).unwrap_or_else(|err| panic!("bootstrap json {err}: {}", String::from_utf8_lossy(&boot.stdout))); + let boot = Command::new(bin) + .args(["--target", container.to_str().unwrap(), "session-bootstrap"]) + .output() + .unwrap(); + assert!( + boot.status.success(), + "bootstrap status {:?} {}", + boot.status, + String::from_utf8_lossy(&boot.stderr) + ); + let boot_json = parse(&String::from_utf8(boot.stdout.clone()).unwrap()).unwrap_or_else(|err| { + panic!( + "bootstrap json {err}: {}", + String::from_utf8_lossy(&boot.stdout) + ) + }); assert_scope_risk("container", &boot_json); - let maps = Command::new(bin).args(["--target", container.to_str().unwrap(), "update-maps"]).output().unwrap(); + let maps = Command::new(bin) + .args(["--target", container.to_str().unwrap(), "update-maps"]) + .output() + .unwrap(); assert!(maps.status.success(), "update-maps {:?}", maps.status); let maps_json = parse(&String::from_utf8(maps.stdout.clone()).unwrap()).unwrap(); assert_scope_risk("container update-maps", &maps_json); - assert_eq!(maps_json.get("parseable_files").and_then(|n| match n { Json::Int(i) => Some(*i), _ => None }), Some(0), "walked files: {maps_json}"); + assert_eq!( + maps_json.get("parseable_files").and_then(|n| match n { + Json::Int(i) => Some(*i), + _ => None, + }), + Some(0), + "walked files: {maps_json}" + ); let copy = temp_root("scoped"); fs::create_dir_all(copy.join("pkg")).unwrap(); for name in ["__init__.py", "a.py", "b.py", "skip.cbl"] { - fs::copy(container.join("alpha/src/pkg").join(name), copy.join("pkg").join(name)).unwrap(); + fs::copy( + container.join("alpha/src/pkg").join(name), + copy.join("pkg").join(name), + ) + .unwrap(); } fs::write(copy.join("map_paths.txt"), "pkg\n").unwrap(); fs::write(copy.join("monitored_paths.txt"), "pkg\n").unwrap(); @@ -71,36 +134,92 @@ fn containers_are_refused_and_scoped_children_map() { fs::write(copy.join("decoy/skip.cbl"), "IDENTIFICATION DIVISION.\n").unwrap(); let child_boot = session_bootstrap(©); - assert_eq!(child_boot.get("success").and_then(|s| s.as_bool()), Some(true), "child bootstrap {child_boot}"); - assert_ne!(child_boot.get("readiness").and_then(|s| s.as_str()), Some("map_ok_scope_risk"), "child treated as container"); + assert_eq!( + child_boot.get("success").and_then(|s| s.as_bool()), + Some(true), + "child bootstrap {child_boot}" + ); + assert_ne!( + child_boot.get("readiness").and_then(|s| s.as_str()), + Some("map_ok_scope_risk"), + "child treated as container" + ); let child_maps = update_maps(©); - assert_eq!(child_maps.get("success").and_then(|s| s.as_bool()), Some(true), "child maps {child_maps}"); - let n = child_maps.get("parseable_files").and_then(|v| match v { Json::Int(i) => Some(*i), _ => None }).unwrap(); + assert_eq!( + child_maps.get("success").and_then(|s| s.as_bool()), + Some(true), + "child maps {child_maps}" + ); + let n = child_maps + .get("parseable_files") + .and_then(|v| match v { + Json::Int(i) => Some(*i), + _ => None, + }) + .unwrap(); assert!(n > 0, "parsed nothing: {child_maps}"); - let edges = child_maps.get("edges").and_then(|e| e.as_array()).unwrap_or(&[]); - assert!(edges.iter().any(|edge| edge.get("dst").and_then(|d| d.as_str()) == Some("pkg/b.py")), "{child_maps}"); + let edges = child_maps + .get("edges") + .and_then(|e| e.as_array()) + .unwrap_or(&[]); + assert!( + edges + .iter() + .any(|edge| edge.get("dst").and_then(|d| d.as_str()) == Some("pkg/b.py")), + "{child_maps}" + ); for edge in edges { let dst = edge.get("dst").and_then(|d| d.as_str()).unwrap_or(""); let src = edge.get("src").and_then(|d| d.as_str()).unwrap_or(""); for bad in [".cbl", ".cob", ".cpy", ".swift", ".s"] { - assert!(!src.ends_with(bad) && !dst.ends_with(bad), "unsupported edge {src} -> {dst}"); + assert!( + !src.ends_with(bad) && !dst.ends_with(bad), + "unsupported edge {src} -> {dst}" + ); } - assert!(!src.starts_with("decoy/") && !dst.starts_with("decoy/"), "escaped scope {src} -> {dst}"); + assert!( + !src.starts_with("decoy/") && !dst.starts_with("decoy/"), + "escaped scope {src} -> {dst}" + ); } let again = session_bootstrap(©); let files = again.get("files").and_then(|f| f.as_array()).unwrap(); - assert!(files.iter().all(|f| f.as_str().unwrap_or("").starts_with("pkg/")), "files left scope: {again}"); + assert!( + files + .iter() + .all(|f| f.as_str().unwrap_or("").starts_with("pkg/")), + "files left scope: {again}" + ); let cobol = temp_root("cobol"); fs::create_dir_all(cobol.join("examples")).unwrap(); - fs::copy(container.join("cobol/examples/hello.cbl"), cobol.join("examples/hello.cbl")).unwrap(); + fs::copy( + container.join("cobol/examples/hello.cbl"), + cobol.join("examples/hello.cbl"), + ) + .unwrap(); fs::write(cobol.join("map_paths.txt"), "examples\n").unwrap(); fs::write(cobol.join("monitored_paths.txt"), "examples\n").unwrap(); let cobol_maps = update_maps(&cobol); - assert_eq!(cobol_maps.get("success").and_then(|s| s.as_bool()), Some(true), "{cobol_maps}"); - assert_eq!(cobol_maps.get("parseable_files").and_then(|n| match n { Json::Int(i) => Some(*i), _ => None }), Some(0), "{cobol_maps}"); - assert!(cobol_maps.get("edges").and_then(|e| e.as_array()).unwrap_or(&[]).is_empty()); + assert_eq!( + cobol_maps.get("success").and_then(|s| s.as_bool()), + Some(true), + "{cobol_maps}" + ); + assert_eq!( + cobol_maps.get("parseable_files").and_then(|n| match n { + Json::Int(i) => Some(*i), + _ => None, + }), + Some(0), + "{cobol_maps}" + ); + assert!(cobol_maps + .get("edges") + .and_then(|e| e.as_array()) + .unwrap_or(&[]) + .is_empty()); let _ = fs::remove_dir_all(&container); let _ = fs::remove_dir_all(©); @@ -140,11 +259,23 @@ fn prepare_edit_on_this_repo_lists_api_dependents() { copy_tree(&live.join("tests"), &root.join("tests")); let built = update_maps(&root); - assert_eq!(built.get("success").and_then(|s| s.as_bool()), Some(true), "{built}"); - assert_eq!(built.get("map_complete").and_then(|s| s.as_bool()), Some(true), "{built}"); + assert_eq!( + built.get("success").and_then(|s| s.as_bool()), + Some(true), + "{built}" + ); + assert_eq!( + built.get("map_complete").and_then(|s| s.as_bool()), + Some(true), + "{built}" + ); let prep = prepare_edit(&root, "src/api.rs"); - assert_eq!(prep.get("success").and_then(|s| s.as_bool()), Some(true), "{prep}"); + assert_eq!( + prep.get("success").and_then(|s| s.as_bool()), + Some(true), + "{prep}" + ); let mut dependents = string_list(&prep, "dependents"); dependents.sort(); let expected = vec![ @@ -153,6 +284,7 @@ fn prepare_edit_on_this_repo_lists_api_dependents() { "src/mcp.rs", "tests/cli_surface.rs", "tests/contract_gaps.rs", + "tests/parity_fixes.rs", "tests/protocol_v07.rs", "tests/robustness.rs", "tests/skip_maps.rs", @@ -160,15 +292,30 @@ fn prepare_edit_on_this_repo_lists_api_dependents() { assert_eq!(dependents, expected, "prepare-edit src/api.rs dependents"); let main = prepare_edit(&root, "src/main.rs"); - assert!(main.get("dependencies").and_then(|item| item.as_array()).is_some_and(|rows| !rows.is_empty()), "{main}"); + assert!( + main.get("dependencies") + .and_then(|item| item.as_array()) + .is_some_and(|rows| !rows.is_empty()), + "{main}" + ); let main_deps: Vec = main .get("dependencies") .and_then(|item| item.as_array()) .unwrap_or(&[]) .iter() - .filter_map(|item| item.get("path").and_then(|path| path.as_str()).map(|text| text.to_string())) + .filter_map(|item| { + item.get("path") + .and_then(|path| path.as_str()) + .map(|text| text.to_string()) + }) .collect(); - assert!(main_deps.iter().any(|path| path == "src/api.rs"), "main.rs dependencies: {main_deps:?}"); - assert!(main_deps.iter().any(|path| path == "src/timeutil.rs"), "main.rs dependencies: {main_deps:?}"); + assert!( + main_deps.iter().any(|path| path == "src/api.rs"), + "main.rs dependencies: {main_deps:?}" + ); + assert!( + main_deps.iter().any(|path| path == "src/timeutil.rs"), + "main.rs dependencies: {main_deps:?}" + ); let _ = fs::remove_dir_all(&root); } diff --git a/tests/parity.rs b/tests/parity.rs index 3c5f067..6007224 100644 --- a/tests/parity.rs +++ b/tests/parity.rs @@ -15,7 +15,8 @@ struct Fixture { #[test] fn accuracy_fixtures_match_frozen_edges() { for fixture in FIXTURES { - let root = std::env::temp_dir().join(format!("wk-acc-{}-{}", fixture.name, std::process::id())); + let root = + std::env::temp_dir().join(format!("wk-acc-{}-{}", fixture.name, std::process::id())); let _ = fs::remove_dir_all(&root); for (rel, text) in fixture.files { let path = root.join(rel); @@ -23,13 +24,25 @@ fn accuracy_fixtures_match_frozen_edges() { fs::write(&path, text).unwrap(); } let built = update_maps(&root); - assert_eq!(built.get("success").and_then(|s| s.as_bool()), Some(true), "{}: {built}", fixture.name); + assert_eq!( + built.get("success").and_then(|s| s.as_bool()), + Some(true), + "{}: {built}", + fixture.name + ); let found = mapped_edges(&root); - let expected: BTreeSet<(String, String)> = fixture.edges.iter().map(|(a, b)| ((*a).to_string(), (*b).to_string())).collect(); + let expected: BTreeSet<(String, String)> = fixture + .edges + .iter() + .map(|(a, b)| ((*a).to_string(), (*b).to_string())) + .collect(); if found != expected { let missed: Vec<_> = expected.difference(&found).cloned().collect(); let extra: Vec<_> = found.difference(&expected).cloned().collect(); - panic!("{}: edges diverged\nmissed {missed:?}\nextra {extra:?}", fixture.name); + panic!( + "{}: edges diverged\nmissed {missed:?}\nextra {extra:?}", + fixture.name + ); } let _ = fs::remove_dir_all(&root); } @@ -166,11 +179,19 @@ fn each_parsed_language_emits_a_local_edge() { fs::write(root.join("src/a.c"), "#include \"b.h\"\n").unwrap(); fs::write(root.join("src/b.h"), "#pragma once\n").unwrap(); fs::write(root.join("src/A.java"), "import demo.B;\nclass A {}\n").unwrap(); - fs::write(root.join("src/B.java"), "package demo;\npublic class B {}\n").unwrap(); + fs::write( + root.join("src/B.java"), + "package demo;\npublic class B {}\n", + ) + .unwrap(); fs::write(root.join("src/A.cs"), "using Demo;\nclass A {}\n").unwrap(); fs::write(root.join("src/Demo.cs"), "class Demo {}\n").unwrap(); let built = update_maps(&root); - assert_eq!(built.get("success").and_then(|s| s.as_bool()), Some(true), "{built}"); + assert_eq!( + built.get("success").and_then(|s| s.as_bool()), + Some(true), + "{built}" + ); let edges = mapped_edges(&root); for (src, dst) in [ ("src/a.py", "src/b.py"), @@ -181,7 +202,10 @@ fn each_parsed_language_emits_a_local_edge() { ("src/A.java", "src/B.java"), ("src/A.cs", "src/Demo.cs"), ] { - assert!(edges.contains(&(src.into(), dst.into())), "missing {src} -> {dst} in {edges:?}"); + assert!( + edges.contains(&(src.into(), dst.into())), + "missing {src} -> {dst} in {edges:?}" + ); } let _ = fs::remove_dir_all(&root); } @@ -193,17 +217,28 @@ fn unknown_suffixes_emit_no_edges() { fs::create_dir_all(root.join("src")).unwrap(); fs::write(root.join("map_paths.txt"), "src\n").unwrap(); fs::write(root.join("monitored_paths.txt"), "src\n").unwrap(); - fs::write(root.join("src/hello.cbl"), " IDENTIFICATION DIVISION.\n PROGRAM-ID. HELLO.\n").unwrap(); + fs::write( + root.join("src/hello.cbl"), + " IDENTIFICATION DIVISION.\n PROGRAM-ID. HELLO.\n", + ) + .unwrap(); fs::write(root.join("src/hello.cob"), " PROCEDURE DIVISION.\n").unwrap(); fs::write(root.join("src/hello.cpy"), " COPY HELLO.\n").unwrap(); fs::write(root.join("src/hello.swift"), "import Foundation\n").unwrap(); fs::write(root.join("src/hello.s"), "mov x0, #0\n").unwrap(); fs::write(root.join("src/real.py"), "def hi():\n return 1\n").unwrap(); let built = update_maps(&root); - assert_eq!(built.get("success").and_then(|s| s.as_bool()), Some(true), "{built}"); + assert_eq!( + built.get("success").and_then(|s| s.as_bool()), + Some(true), + "{built}" + ); let edges = mapped_edges(&root); assert!(edges.is_empty(), "{edges:?}"); - let files = built.get("parseable_files").and_then(|n| match n { Json::Int(i) => Some(*i), _ => None }); + let files = built.get("parseable_files").and_then(|n| match n { + Json::Int(i) => Some(*i), + _ => None, + }); assert_eq!(files, Some(1), "only real.py is parseable, got {built}"); let _ = fs::remove_dir_all(&root); } diff --git a/tests/parity_fixes.rs b/tests/parity_fixes.rs new file mode 100644 index 0000000..674187a --- /dev/null +++ b/tests/parity_fixes.rs @@ -0,0 +1,467 @@ +//! Parity with the Python 4.7 package: wiki lookup, symbols, daemon, init, flags. + +use std::fs; +use std::path::PathBuf; +use std::process::Command; +use std::time::{SystemTime, UNIX_EPOCH}; + +use wikifier::api::{self, mark_green, update_maps}; +use wikifier::jsonutil::Json; + +fn scratch(name: &str) -> PathBuf { + let n = SystemTime::now() + .duration_since(UNIX_EPOCH) + .unwrap() + .as_nanos(); + let dir = std::env::temp_dir().join(format!("wk-par-{name}-{n}")); + let _ = fs::remove_dir_all(&dir); + fs::create_dir_all(dir.join("src")).unwrap(); + fs::write(dir.join("map_paths.txt"), "src\n").unwrap(); + dir +} + +fn bin() -> PathBuf { + PathBuf::from(env!("CARGO_BIN_EXE_wikifier")) +} + +#[test] +fn wiki_lookup_falls_back_to_docs_wiki() { + let root = scratch("wiki"); + fs::write(root.join("src/a.py"), "def hello():\n return 1\n").unwrap(); + fs::create_dir_all(root.join("docs/wiki")).unwrap(); + fs::write( + root.join("docs/wiki/a.py.md"), + "`hello` lives in the docs wiki.\n", + ) + .unwrap(); + assert_eq!( + update_maps(&root).get("success").and_then(|v| v.as_bool()), + Some(true) + ); + let edit = api::prepare_edit(&root, "src/a.py"); + assert!( + edit.get("wiki") + .and_then(|v| v.as_str()) + .unwrap_or("") + .contains("hello"), + "{edit}" + ); + let marked = mark_green(&root, "src/a.py", "docs wiki names hello", false); + assert_eq!( + marked.get("verification").and_then(|v| v.as_str()), + Some("verified"), + "{marked}" + ); + let _ = fs::remove_dir_all(&root); +} + +#[test] +fn multiline_all_and_pyi_are_public_symbols() { + let root = scratch("syms"); + fs::write( + root.join("src/api.py"), + "__all__ = [\n \"visible\",\n]\ndef hidden():\n return 1\n", + ) + .unwrap(); + fs::write(root.join("src/api.py.wiki.md"), "`visible` is public.\n").unwrap(); + let marked = mark_green(&root, "src/api.py", "only all", false); + assert_eq!( + marked.get("success").and_then(|v| v.as_bool()), + Some(true), + "{marked}" + ); + fs::write(root.join("src/types.pyi"), "def stubbed():\n ...\n").unwrap(); + fs::write( + root.join("src/types.pyi.wiki.md"), + "`stubbed` is the stub.\n", + ) + .unwrap(); + let pyi = mark_green(&root, "src/types.pyi", "stub", false); + assert_eq!( + pyi.get("verification").and_then(|v| v.as_str()), + Some("verified"), + "{pyi}" + ); + let _ = fs::remove_dir_all(&root); +} + +#[test] +fn js_comment_exports_are_not_public() { + let root = scratch("js"); + fs::write( + root.join("src/mod.ts"), + "// export function ghost() {}\nexport function real() {}\n", + ) + .unwrap(); + fs::write(root.join("src/mod.ts.wiki.md"), "`real` is exported.\n").unwrap(); + let marked = mark_green(&root, "src/mod.ts", "real only", false); + assert_eq!( + marked.get("success").and_then(|v| v.as_bool()), + Some(true), + "{marked}" + ); + let _ = fs::remove_dir_all(&root); +} + +#[test] +fn pending_line_uses_the_python_shape() { + let root = scratch("pend"); + fs::write(root.join("src/a.py"), "x = 1\n").unwrap(); + assert_eq!( + api::record_change(&root, "src/a.py", "first reason") + .get("success") + .and_then(|v| v.as_bool()), + Some(true) + ); + let text = fs::read_to_string(root.join("pending_updates.md")).unwrap(); + assert!( + text.contains("- src/a.py: LLM/agent edit — first reason"), + "{text}" + ); + let _ = fs::remove_dir_all(&root); +} + +#[test] +fn init_writes_the_python_files_and_honors_no_copy() { + let root = scratch("init"); + let dest = root.join("proj"); + fs::create_dir_all(dest.join("src")).unwrap(); + let out = Command::new(bin()) + .arg("init") + .arg(&dest) + .arg("--no-copy") + .output() + .unwrap(); + assert!( + out.status.success(), + "{}", + String::from_utf8_lossy(&out.stderr) + ); + assert!(!dest.join("index.html").exists()); + assert!(dest.join("exclude_patterns.txt").is_file()); + assert!(dest.join("file_health.md").is_file()); + assert!(dest.join("pending_updates.md").is_file()); + assert!(dest.join("library.md").is_file()); + assert!(dest.join(".wikifier/config").is_file()); + assert!(dest.join("journal").is_dir()); + assert!(dest.join("Logged_issues").is_dir()); + let _ = fs::remove_dir_all(&root); +} + +#[test] +fn record_changes_accepts_a_json_reason_object() { + let root = scratch("reasons"); + fs::write(root.join("src/a.py"), "x = 1\n").unwrap(); + Command::new("git") + .args(["init"]) + .current_dir(&root) + .output() + .unwrap(); + Command::new("git") + .args(["add", "src/a.py", "map_paths.txt"]) + .current_dir(&root) + .output() + .unwrap(); + Command::new("git") + .args([ + "-c", + "user.email=a@b.c", + "-c", + "user.name=t", + "commit", + "-qm", + "init", + ]) + .current_dir(&root) + .output() + .unwrap(); + fs::write(root.join("src/a.py"), "x = 2\n").unwrap(); + fs::write(root.join("reasons.json"), "{\"src/a.py\":\"bumped x\"}\n").unwrap(); + let out = Command::new(bin()) + .args([ + "--target", + root.to_str().unwrap(), + "record-changes", + "--reasons", + root.join("reasons.json").to_str().unwrap(), + "--dry-run", + "--only", + "src/a.py", + ]) + .output() + .unwrap(); + let text = String::from_utf8_lossy(&out.stdout); + assert!( + out.status.success(), + "{text} {}", + String::from_utf8_lossy(&out.stderr) + ); + assert!( + text.contains("bumped x") || text.contains("src/a.py"), + "{text}" + ); + let _ = fs::remove_dir_all(&root); +} + +#[test] +fn health_summary_list_paths_and_cycles_flags() { + let root = scratch("flags"); + fs::create_dir_all(root.join("src/pkg")).unwrap(); + fs::write(root.join("src/pkg/a.py"), "import b\n").unwrap(); + fs::write(root.join("src/pkg/b.py"), "import a\n").unwrap(); + fs::write(root.join("monitored_paths.txt"), "src\n").unwrap(); + assert_eq!( + update_maps(&root).get("success").and_then(|v| v.as_bool()), + Some(true) + ); + let summary = api::dispatch( + "health", + &root, + &[], + &flag(&[("summary", "true"), ("directory", "src/pkg")]), + ); + assert!(summary.get("entries").is_none(), "{summary}"); + assert!(summary.get("health_summary").is_some(), "{summary}"); + let listed = api::dispatch( + "list-paths", + &root, + &[], + &flag(&[("prefix", "src"), ("recursive", "true"), ("max-files", "1")]), + ); + let files = listed.get("files").and_then(|v| v.as_array()).unwrap(); + assert_eq!(files.len(), 1, "{listed}"); + assert_eq!( + listed.get("files_truncated").and_then(|v| v.as_bool()), + Some(true), + "{listed}" + ); + let cycles = api::dispatch("cycles", &root, &[], &flag(&[("max-items", "1")])); + assert!( + cycles + .get("cycles") + .and_then(|v| v.as_array()) + .unwrap() + .len() + <= 1, + "{cycles}" + ); + assert!(cycles.get("reverse_dependencies").is_some(), "{cycles}"); + let bare = api::dispatch("cycles", &root, &[], &flag(&[("no-analysis", "true")])); + assert!(bare.get("reverse_dependencies").is_none(), "{bare}"); + let _ = fs::remove_dir_all(&root); +} + +#[test] +fn suggest_next_names_low_confidence_edges() { + let root = scratch("low"); + fs::create_dir_all(root.join("src/pkg")).unwrap(); + fs::write(root.join("src/pkg/__init__.py"), "").unwrap(); + fs::write(root.join("src/pkg/b.py"), "VALUE = 1\n").unwrap(); + fs::write( + root.join("src/pkg/a.py"), + "def load():\n from . import b\n", + ) + .unwrap(); + fs::write(root.join("monitored_paths.txt"), "src\n").unwrap(); + assert_eq!( + update_maps(&root).get("success").and_then(|v| v.as_bool()), + Some(true) + ); + let next = api::suggest_next_actions(&root); + let actions = next.get("actions").and_then(|v| v.as_array()).unwrap(); + assert!( + actions.iter().any( + |a| a.get("action").and_then(|v| v.as_str()) == Some("review_low_confidence_edges") + ), + "{next}" + ); + let _ = fs::remove_dir_all(&root); +} + +#[test] +fn seed_source_hashes_does_not_change_status() { + let root = scratch("hash"); + fs::write(root.join("src/a.py"), "def hello():\n return 1\n").unwrap(); + fs::write(root.join("monitored_paths.txt"), "src\n").unwrap(); + assert_eq!( + update_maps(&root).get("success").and_then(|v| v.as_bool()), + Some(true) + ); + fs::write(root.join("src/a.py.wiki.md"), "`hello` says hello.\n").unwrap(); + assert_eq!( + mark_green(&root, "src/a.py", "named", false) + .get("success") + .and_then(|v| v.as_bool()), + Some(true) + ); + fs::write(root.join("src/a.py"), "def hello():\n return 2\n").unwrap(); + let before = api::read_health(&root) + .into_iter() + .find(|r| r.file == "src/a.py") + .unwrap(); + let seeded = api::dispatch( + "seed-source-hashes", + &root, + &[], + &flag(&[("force", "true")]), + ); + assert_eq!( + seeded.get("success").and_then(|v| v.as_bool()), + Some(true), + "{seeded}" + ); + let after = api::read_health(&root) + .into_iter() + .find(|r| r.file == "src/a.py") + .unwrap(); + assert_eq!(after.status, before.status, "{after:?}"); + assert_ne!(after.source_hash, before.source_hash); + let _ = fs::remove_dir_all(&root); +} + +#[test] +fn monitored_paths_scope_health_seeding() { + let root = scratch("seed"); + fs::write(root.join("src/a.py"), "x = 1\n").unwrap(); + fs::write(root.join("README.md"), "# hi\n").unwrap(); + fs::write(root.join("monitored_paths.txt"), "README.md\n").unwrap(); + assert_eq!( + update_maps(&root).get("success").and_then(|v| v.as_bool()), + Some(true) + ); + let files: Vec = api::read_health(&root) + .into_iter() + .map(|r| r.file) + .collect(); + assert!( + files.iter().all(|f| f != "src/a.py"), + "mapped file outside monitored_paths was seeded: {files:?}" + ); + fs::remove_file(root.join("monitored_paths.txt")).unwrap(); + fs::remove_file(root.join("file_health.json")).ok(); + fs::remove_file(root.join("file_health.md")).ok(); + assert_eq!( + update_maps(&root).get("success").and_then(|v| v.as_bool()), + Some(true) + ); + let files: Vec = api::read_health(&root) + .into_iter() + .map(|r| r.file) + .collect(); + assert!( + files.iter().any(|f| f == "src/a.py"), + "absent monitored_paths should seed the map: {files:?}" + ); + let _ = fs::remove_dir_all(&root); +} + +#[test] +fn daemon_install_writes_a_unit_and_logs_exist() { + let root = scratch("daemon"); + fs::write(root.join("src/a.py"), "x = 1\n").unwrap(); + fs::write(root.join("monitored_paths.txt"), "src\n").unwrap(); + assert_eq!( + update_maps(&root).get("success").and_then(|v| v.as_bool()), + Some(true) + ); + let home = scratch("home"); + let install = Command::new(bin()) + .args([ + "--target", + root.to_str().unwrap(), + "daemon", + "install-service", + ]) + .env("WIKIFIER_SERVICE_HOME", &home) + .env("XDG_CONFIG_HOME", home.join(".config")) + .output() + .unwrap(); + let text = String::from_utf8_lossy(&install.stdout); + let err = String::from_utf8_lossy(&install.stderr); + let unit = home.join(".config/systemd/user/wikifier.service"); + if cfg!(target_os = "linux") { + assert!(install.status.success(), "{text} {err}"); + let body = fs::read_to_string(&unit).unwrap_or_else(|_| panic!("{text}")); + assert!(body.contains("daemon run"), "{body}"); + } else if cfg!(target_os = "macos") { + assert!(install.status.success(), "{text} {err}"); + assert!( + home.join("Library/LaunchAgents/com.wikifier.agent.plist") + .is_file(), + "{text}" + ); + } else { + assert!( + text.contains("Task Scheduler") + || text.contains("daemon run") + || text.contains("\"success\":true"), + "{text} {err}" + ); + } + let _ = Command::new(bin()) + .args([ + "--target", + root.to_str().unwrap(), + "daemon", + "uninstall-service", + ]) + .env("WIKIFIER_SERVICE_HOME", &home) + .env("XDG_CONFIG_HOME", home.join(".config")) + .output(); + let logs = Command::new(bin()) + .args(["--target", root.to_str().unwrap(), "daemon", "logs"]) + .output() + .unwrap(); + assert!( + logs.status.success(), + "{}", + String::from_utf8_lossy(&logs.stdout) + ); + let _ = fs::remove_dir_all(&root); + let _ = fs::remove_dir_all(&home); +} + +#[test] +fn text_flag_prints_prose_and_default_stays_json() { + let root = scratch("text"); + fs::write(root.join("src/a.py"), "x = 1\n").unwrap(); + let plain = Command::new(bin()) + .args(["--target", root.to_str().unwrap(), "check-changes"]) + .output() + .unwrap(); + let plain_text = String::from_utf8_lossy(&plain.stdout); + assert!(plain_text.contains("\"success\""), "{plain_text}"); + let prose = Command::new(bin()) + .args([ + "--target", + root.to_str().unwrap(), + "check-changes", + "--text", + ]) + .output() + .unwrap(); + let prose_text = String::from_utf8_lossy(&prose.stdout); + assert!(prose_text.contains("check_changes:"), "{prose_text}"); + assert!(!prose_text.contains("\"success\""), "{prose_text}"); + let _ = fs::remove_dir_all(&root); +} + +fn flag(pairs: &[(&str, &str)]) -> std::collections::BTreeMap { + pairs + .iter() + .map(|(k, v)| ((*k).to_string(), (*v).to_string())) + .collect() +} + +#[test] +fn mcp_prompt_is_specific_and_reasons_object_is_accepted() { + let prompt = wikifier::jsonutil::parse(r#"{"jsonrpc":"2.0","id":1,"method":"prompts/get","params":{"name":"review_pending_changes"}}"#).unwrap(); + let got = wikifier::mcp::handle_message(&prompt).unwrap(); + let text = got.to_string(); + assert!(text.contains("session_bootstrap"), "{text}"); + assert!( + !text.contains("Use Wikifier. Prompt review_pending_changes"), + "{text}" + ); + let _ = Json::Null; +} diff --git a/tests/protocol_v07.rs b/tests/protocol_v07.rs index d58186e..3fac434 100644 --- a/tests/protocol_v07.rs +++ b/tests/protocol_v07.rs @@ -5,12 +5,18 @@ use std::path::PathBuf; use std::process::Command; use std::time::{SystemTime, UNIX_EPOCH}; -use wikifier::api::{self, dependency_report, load_dependencies, mark_green, read_health, read_journal, record_deletion, update_maps, update_maps_run, MapRun}; +use wikifier::api::{ + self, dependency_report, load_dependencies, mark_green, read_health, read_journal, + record_deletion, update_maps, update_maps_run, MapRun, +}; use wikifier::jsonutil::{self, Json}; use wikifier::mcp; fn scratch(name: &str) -> PathBuf { - let n = SystemTime::now().duration_since(UNIX_EPOCH).unwrap().as_nanos(); + let n = SystemTime::now() + .duration_since(UNIX_EPOCH) + .unwrap() + .as_nanos(); let dir = std::env::temp_dir().join(format!("wk-v07-{name}-{n}")); let _ = fs::remove_dir_all(&dir); fs::create_dir_all(dir.join("src")).unwrap(); @@ -33,8 +39,16 @@ fn update_maps_writes_library_and_folder_cards() { fs::write(root.join("src/b.py"), "VALUE = 1\n").unwrap(); fs::write(root.join("src/__init__.py"), "").unwrap(); let mapped = update_maps(&root); - assert_eq!(mapped.get("success").and_then(|s| s.as_bool()), Some(true), "{mapped}"); - assert_eq!(mapped.get("map_complete").and_then(|s| s.as_bool()), Some(true), "{mapped}"); + assert_eq!( + mapped.get("success").and_then(|s| s.as_bool()), + Some(true), + "{mapped}" + ); + assert_eq!( + mapped.get("map_complete").and_then(|s| s.as_bool()), + Some(true), + "{mapped}" + ); let library = fs::read_to_string(root.join("library.md")).unwrap(); assert!(library.contains("src/a.py"), "{library}"); assert!(library.contains("```mermaid"), "{library}"); @@ -44,7 +58,8 @@ fn update_maps_writes_library_and_folder_cards() { assert!(card.contains("src/"), "{card}"); let src_card = fs::read_to_string(root.join(".wikifier_staging/maps/src/_index.json")).unwrap(); assert!(src_card.contains("src/a.py"), "{src_card}"); - let status = fs::read_to_string(root.join(".wikifier_staging/incremental_status.json")).unwrap(); + let status = + fs::read_to_string(root.join(".wikifier_staging/incremental_status.json")).unwrap(); assert!(status.contains("map_complete"), "{status}"); let _ = fs::remove_dir_all(&root); } @@ -56,37 +71,65 @@ fn directory_max_files_and_stream_are_honored() { fs::write(root.join("src/a.py"), "x = 1\n").unwrap(); fs::write(root.join("src/b.py"), "y = 2\n").unwrap(); fs::write(root.join("other/c.py"), "z = 3\n").unwrap(); - let missing = update_maps_run(&root, &MapRun { - full: false, - quiet: true, - directory: Some("missing".into()), - max_files: None, - stream: false, - }); - assert_eq!(missing.get("error_type").and_then(|s| s.as_str()), Some("ScopeError"), "{missing}"); + let missing = update_maps_run( + &root, + &MapRun { + full: false, + quiet: true, + directory: Some("missing".into()), + max_files: None, + stream: false, + }, + ); + assert_eq!( + missing.get("error_type").and_then(|s| s.as_str()), + Some("ScopeError"), + "{missing}" + ); - let budget = update_maps_run(&root, &MapRun { - full: false, - quiet: false, - directory: Some("src".into()), - max_files: Some(1), - stream: true, - }); - assert_eq!(budget.get("success").and_then(|s| s.as_bool()), Some(true), "{budget}"); - assert_eq!(budget.get("map_complete").and_then(|s| s.as_bool()), Some(false), "{budget}"); + let budget = update_maps_run( + &root, + &MapRun { + full: false, + quiet: false, + directory: Some("src".into()), + max_files: Some(1), + stream: true, + }, + ); + assert_eq!( + budget.get("success").and_then(|s| s.as_bool()), + Some(true), + "{budget}" + ); + assert_eq!( + budget.get("map_complete").and_then(|s| s.as_bool()), + Some(false), + "{budget}" + ); assert!(int_field(&budget, "files_remaining_dirty") >= 1, "{budget}"); assert!(int_field(&budget, "files_parsed") <= 1, "{budget}"); let paths = wikifier::cache::all_paths(&root); - assert!(paths.iter().all(|p| p.starts_with("src/")), "directory leaked: {paths:?}"); + assert!( + paths.iter().all(|p| p.starts_with("src/")), + "directory leaked: {paths:?}" + ); - let rest = update_maps_run(&root, &MapRun { - full: false, - quiet: true, - directory: Some("src".into()), - max_files: Some(10), - stream: false, - }); - assert_eq!(rest.get("map_complete").and_then(|s| s.as_bool()), Some(true), "{rest}"); + let rest = update_maps_run( + &root, + &MapRun { + full: false, + quiet: true, + directory: Some("src".into()), + max_files: Some(10), + stream: false, + }, + ); + assert_eq!( + rest.get("map_complete").and_then(|s| s.as_bool()), + Some(true), + "{rest}" + ); assert_eq!(int_field(&rest, "files_remaining_dirty"), 0, "{rest}"); let _ = fs::remove_dir_all(&root); } @@ -97,23 +140,56 @@ fn record_deletion_marks_red_and_drops_the_graph() { fs::write(root.join("src/a.py"), "from . import b\n").unwrap(); fs::write(root.join("src/b.py"), "VALUE = 1\n").unwrap(); fs::write(root.join("src/__init__.py"), "").unwrap(); - assert_eq!(update_maps(&root).get("success").and_then(|s| s.as_bool()), Some(true)); - fs::write(root.join("pending_updates.md"), "# Pending Updates\n\n- src/b.py — old\n").unwrap(); + assert_eq!( + update_maps(&root).get("success").and_then(|s| s.as_bool()), + Some(true) + ); + fs::write( + root.join("pending_updates.md"), + "# Pending Updates\n\n- src/b.py — old\n", + ) + .unwrap(); fs::remove_file(root.join("src/b.py")).unwrap(); let deleted = record_deletion(&root, "src/b.py", "removed the helper"); - assert_eq!(deleted.get("success").and_then(|s| s.as_bool()), Some(true), "{deleted}"); - assert_eq!(deleted.get("removed_from_graph").and_then(|s| s.as_bool()), Some(true), "{deleted}"); - let row = read_health(&root).into_iter().find(|r| r.file == "src/b.py").unwrap(); + assert_eq!( + deleted.get("success").and_then(|s| s.as_bool()), + Some(true), + "{deleted}" + ); + assert_eq!( + deleted.get("removed_from_graph").and_then(|s| s.as_bool()), + Some(true), + "{deleted}" + ); + let row = read_health(&root) + .into_iter() + .find(|r| r.file == "src/b.py") + .unwrap(); assert!(row.status.contains("Red"), "{:?}", row.status); let deps = load_dependencies(&root, "src/a.py"); let internal = deps.get("internal").and_then(|v| v.as_array()).unwrap(); - assert!(internal.iter().all(|e| e.get("path").and_then(|p| p.as_str()) != Some("src/b.py")), "{deps}"); + assert!( + internal + .iter() + .all(|e| e.get("path").and_then(|p| p.as_str()) != Some("src/b.py")), + "{deps}" + ); let pending = fs::read_to_string(root.join("pending_updates.md")).unwrap(); assert!(!pending.contains("src/b.py"), "{pending}"); let again = record_deletion(&root, "src/b.py", "removed the helper"); - assert_eq!(again.get("duplicate_skipped").and_then(|s| s.as_bool()), Some(true), "{again}"); + assert_eq!( + again.get("duplicate_skipped").and_then(|s| s.as_bool()), + Some(true), + "{again}" + ); let journal = read_journal(&root, None); - assert_eq!(journal.iter().filter(|e| e.action == "record-deletion" && e.file == "src/b.py").count(), 1); + assert_eq!( + journal + .iter() + .filter(|e| e.action == "record-deletion" && e.file == "src/b.py") + .count(), + 1 + ); let _ = fs::remove_dir_all(&root); } @@ -123,24 +199,83 @@ fn record_changes_matches_globs_deletions_and_green() { fs::write(root.join("src/a.py"), "def hello():\n return 1\n").unwrap(); fs::write(root.join("src/b.py"), "VALUE = 1\n").unwrap(); fs::write(root.join("src/a.py.wiki.md"), "hello is the greeting\n").unwrap(); - Command::new("git").args(["init"]).current_dir(&root).output().unwrap(); - Command::new("git").args(["add", "-A"]).current_dir(&root).output().unwrap(); - Command::new("git").args(["-c", "user.email=t@example.com", "-c", "user.name=t", "commit", "-m", "init"]).current_dir(&root).output().unwrap(); + Command::new("git") + .args(["init"]) + .current_dir(&root) + .output() + .unwrap(); + Command::new("git") + .args(["add", "-A"]) + .current_dir(&root) + .output() + .unwrap(); + Command::new("git") + .args([ + "-c", + "user.email=t@example.com", + "-c", + "user.name=t", + "commit", + "-m", + "init", + ]) + .current_dir(&root) + .output() + .unwrap(); fs::write(root.join("src/a.py"), "def hello():\n return 2\n").unwrap(); fs::remove_file(root.join("src/b.py")).unwrap(); - let dry = api::record_changes_opts(&root, &["src/*.py=touched the package".into()], None, &[], None, false, true); - assert_eq!(dry.get("success").and_then(|s| s.as_bool()), Some(true), "{dry}"); + let dry = api::record_changes_opts( + &root, + &["src/*.py=touched the package".into()], + None, + &[], + None, + false, + true, + ); + assert_eq!( + dry.get("success").and_then(|s| s.as_bool()), + Some(true), + "{dry}" + ); assert!(int_field(&dry, "recorded") >= 2, "{dry}"); - assert!(read_journal(&root, None).is_empty(), "dry-run wrote a journal"); + assert!( + read_journal(&root, None).is_empty(), + "dry-run wrote a journal" + ); - let recorded = api::record_changes_opts(&root, &["src/=touched the package".into()], None, &["src/a.py".into()], None, true, false); - assert_eq!(recorded.get("success").and_then(|s| s.as_bool()), Some(true), "{recorded}"); + let recorded = api::record_changes_opts( + &root, + &["src/=touched the package".into()], + None, + &["src/a.py".into()], + None, + true, + false, + ); + assert_eq!( + recorded.get("success").and_then(|s| s.as_bool()), + Some(true), + "{recorded}" + ); let files = recorded.get("files").and_then(|v| v.as_array()).unwrap(); - assert!(files.iter().any(|v| v.as_str() == Some("src/a.py")), "{recorded}"); - assert!(files.iter().all(|v| v.as_str() != Some("src/b.py")), "only filter leaked: {recorded}"); - let failed = recorded.get("verification_failed").and_then(|v| v.as_array()).unwrap(); + assert!( + files.iter().any(|v| v.as_str() == Some("src/a.py")), + "{recorded}" + ); + assert!( + files.iter().all(|v| v.as_str() != Some("src/b.py")), + "only filter leaked: {recorded}" + ); + let failed = recorded + .get("verification_failed") + .and_then(|v| v.as_array()) + .unwrap(); assert!(failed.is_empty(), "{recorded}"); - let row = read_health(&root).into_iter().find(|r| r.file == "src/a.py").unwrap(); + let row = read_health(&root) + .into_iter() + .find(|r| r.file == "src/a.py") + .unwrap(); assert!(row.status.contains("Green"), "{:?}", row.status); let _ = fs::remove_dir_all(&root); } @@ -150,41 +285,90 @@ fn full_dependencies_carry_confidence_and_cycles_have_a_break_hint() { let root = scratch("intel"); fs::write(root.join("src/a.py"), "import b\n").unwrap(); fs::write(root.join("src/b.py"), "import a\n").unwrap(); - assert_eq!(update_maps(&root).get("success").and_then(|s| s.as_bool()), Some(true)); + assert_eq!( + update_maps(&root).get("success").and_then(|s| s.as_bool()), + Some(true) + ); let full = dependency_report(&root, "src/a.py", true); - assert_eq!(full.get("detail").and_then(|s| s.as_str()), Some("full"), "{full}"); - let edge = full.get("imports").and_then(|v| v.as_array()).unwrap().iter().find(|e| e.get("path").and_then(|p| p.as_str()) == Some("src/b.py")).unwrap(); + assert_eq!( + full.get("detail").and_then(|s| s.as_str()), + Some("full"), + "{full}" + ); + let edge = full + .get("imports") + .and_then(|v| v.as_array()) + .unwrap() + .iter() + .find(|e| e.get("path").and_then(|p| p.as_str()) == Some("src/b.py")) + .unwrap(); assert!(edge.get("confidence").is_some(), "{edge}"); - assert!(edge.get("conditional").is_some() && edge.get("via_barrel").is_some(), "{edge}"); + assert!( + edge.get("conditional").is_some() && edge.get("via_barrel").is_some(), + "{edge}" + ); let report = api::cycles(&root); let cycle = report.get("cycles").and_then(|v| v.as_array()).unwrap(); assert!(!cycle.is_empty(), "{report}"); assert!(cycle[0].get("break_hint").is_some(), "{report}"); let barrels = api::barrel_reports(&root); - assert_eq!(barrels.get("success").and_then(|s| s.as_bool()), Some(true), "{barrels}"); - assert!(barrels.get("barrels").and_then(|v| v.as_array()).is_some(), "{barrels}"); + assert_eq!( + barrels.get("success").and_then(|s| s.as_bool()), + Some(true), + "{barrels}" + ); + assert!( + barrels.get("barrels").and_then(|v| v.as_array()).is_some(), + "{barrels}" + ); let _ = fs::remove_dir_all(&root); } #[test] fn wiki_check_covers_js_exports_removed_symbols_and_an_unchanged_wiki() { let root = scratch("wiki"); - fs::write(root.join("src/mod.ts"), "export function hello() { return 1 }\n").unwrap(); + fs::write( + root.join("src/mod.ts"), + "export function hello() { return 1 }\n", + ) + .unwrap(); fs::write(root.join("src/mod.ts.wiki.md"), "no exports named here\n").unwrap(); let refused = mark_green(&root, "src/mod.ts", "", false); - assert_eq!(refused.get("error_type").and_then(|s| s.as_str()), Some("WikiVerificationError"), "{refused}"); + assert_eq!( + refused.get("error_type").and_then(|s| s.as_str()), + Some("WikiVerificationError"), + "{refused}" + ); let issues = refused.get("issues").and_then(|v| v.as_array()).unwrap(); - assert!(issues.iter().any(|v| v.as_str().unwrap_or("").contains("undocumented_new_symbols")), "{refused}"); + assert!( + issues.iter().any(|v| v + .as_str() + .unwrap_or("") + .contains("undocumented_new_symbols")), + "{refused}" + ); fs::write(root.join("src/mod.ts.wiki.md"), "`hello` is the greeting\n").unwrap(); let marked = mark_green(&root, "src/mod.ts", "wiki names hello", false); - assert_eq!(marked.get("success").and_then(|s| s.as_bool()), Some(true), "{marked}"); + assert_eq!( + marked.get("success").and_then(|s| s.as_bool()), + Some(true), + "{marked}" + ); let wiki = fs::read_to_string(root.join("src/mod.ts.wiki.md")).unwrap(); - fs::write(root.join("src/mod.ts"), "export function goodbye() { return 1 }\n").unwrap(); + fs::write( + root.join("src/mod.ts"), + "export function goodbye() { return 1 }\n", + ) + .unwrap(); fs::write(root.join("src/mod.ts.wiki.md"), &wiki).unwrap(); let stale = mark_green(&root, "src/mod.ts", "same wiki", false); - assert_eq!(stale.get("error_type").and_then(|s| s.as_str()), Some("WikiVerificationError"), "{stale}"); + assert_eq!( + stale.get("error_type").and_then(|s| s.as_str()), + Some("WikiVerificationError"), + "{stale}" + ); let text = stale.to_string(); assert!(text.contains("wiki_mentions_removed_symbols"), "{stale}"); assert!(text.contains("wiki_not_updated_for_api_change"), "{stale}"); @@ -203,9 +387,18 @@ fn legacy_json_cache_is_readable_before_a_remap() { r#"{"src/a.py":{"resolved_pairs":[{"resolved":"src/b.py"}]},"src/b.py":{"resolved_pairs":[]}}"#, ).unwrap(); let deps = load_dependencies(&root, "src/a.py"); - assert_eq!(deps.get("success").and_then(|s| s.as_bool()), Some(true), "{deps}"); + assert_eq!( + deps.get("success").and_then(|s| s.as_bool()), + Some(true), + "{deps}" + ); let internal = deps.get("internal").and_then(|v| v.as_array()).unwrap(); - assert!(internal.iter().any(|e| e.get("path").and_then(|p| p.as_str()) == Some("src/b.py")), "{deps}"); + assert!( + internal + .iter() + .any(|e| e.get("path").and_then(|p| p.as_str()) == Some("src/b.py")), + "{deps}" + ); let _ = fs::remove_dir_all(&root); } @@ -215,10 +408,27 @@ fn init_keeps_existing_path_lists_and_copies_the_dashboard() { fs::write(root.join("map_paths.txt"), "pkg\n").unwrap(); fs::write(root.join("monitored_paths.txt"), "README.md\n").unwrap(); let value = api::dispatch("init", &root, &[], &std::collections::BTreeMap::new()); - assert_eq!(value.get("success").and_then(|s| s.as_bool()), Some(true), "{value}"); - assert_eq!(fs::read_to_string(root.join("map_paths.txt")).unwrap().trim(), "pkg"); - assert_eq!(fs::read_to_string(root.join("monitored_paths.txt")).unwrap().trim(), "README.md"); - assert!(root.join("index.html").is_file(), "init did not copy index.html"); + assert_eq!( + value.get("success").and_then(|s| s.as_bool()), + Some(true), + "{value}" + ); + assert_eq!( + fs::read_to_string(root.join("map_paths.txt")) + .unwrap() + .trim(), + "pkg" + ); + assert_eq!( + fs::read_to_string(root.join("monitored_paths.txt")) + .unwrap() + .trim(), + "README.md" + ); + assert!( + root.join("index.html").is_file(), + "init did not copy index.html" + ); assert!(root.join(".wikifier").is_dir()); let _ = fs::remove_dir_all(&root); } @@ -226,22 +436,39 @@ fn init_keeps_existing_path_lists_and_copies_the_dashboard() { #[test] fn mcp_reports_tool_errors_and_reads_resources() { let root = scratch("mcp"); - let root_text = root.to_str().unwrap().replace('\\', "\\\\").replace('"', "\\\""); + let root_text = root + .to_str() + .unwrap() + .replace('\\', "\\\\") + .replace('"', "\\\""); let call = format!( r#"{{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{{"name":"record_change","arguments":{{"project_root":"{root_text}","file":"src/missing.py","reason":"gone"}}}}}}"# ); let msg = jsonutil::parse(&call).unwrap(); let resp = mcp::handle_message(&msg).unwrap(); let result = resp.get("result").unwrap(); - assert_eq!(result.get("isError").and_then(|s| s.as_bool()), Some(true), "{resp}"); + assert_eq!( + result.get("isError").and_then(|s| s.as_bool()), + Some(true), + "{resp}" + ); let read = jsonutil::parse(&format!( r#"{{"jsonrpc":"2.0","id":2,"method":"resources/read","params":{{"uri":"wikifier://health","project_root":"{root_text}"}}}}"# )).unwrap(); let resource = mcp::handle_message(&read).unwrap(); - assert!(resource.get("result").and_then(|v| v.get("contents")).is_some(), "{resource}"); + assert!( + resource + .get("result") + .and_then(|v| v.get("contents")) + .is_some(), + "{resource}" + ); let prompt = jsonutil::parse(r#"{"jsonrpc":"2.0","id":3,"method":"prompts/get","params":{"name":"review_pending_changes"}}"#).unwrap(); let got = mcp::handle_message(&prompt).unwrap(); - assert!(got.get("result").and_then(|v| v.get("messages")).is_some(), "{got}"); + assert!( + got.get("result").and_then(|v| v.get("messages")).is_some(), + "{got}" + ); let _ = fs::remove_dir_all(&root); } @@ -249,25 +476,48 @@ fn mcp_reports_tool_errors_and_reads_resources() { fn prune_keeps_monitored_files_outside_the_map_roots() { let root = scratch("prune"); fs::write(root.join("README.md"), "# hi\n").unwrap(); - fs::write(root.join("monitored_paths.txt"), "README.md\n").unwrap(); + fs::write(root.join("monitored_paths.txt"), "README.md\nsrc\n").unwrap(); fs::write(root.join("src/a.py"), "x = 1\n").unwrap(); fs::create_dir_all(root.join("other")).unwrap(); fs::write(root.join("other/gone.py"), "z = 1\n").unwrap(); let mapped = update_maps(&root); - assert_eq!(mapped.get("success").and_then(|s| s.as_bool()), Some(true), "{mapped}"); + assert_eq!( + mapped.get("success").and_then(|s| s.as_bool()), + Some(true), + "{mapped}" + ); let readme = api::record_change(&root, "README.md", "watch the readme"); - assert_eq!(readme.get("success").and_then(|s| s.as_bool()), Some(true), "{readme}"); + assert_eq!( + readme.get("success").and_then(|s| s.as_bool()), + Some(true), + "{readme}" + ); let other = api::record_change(&root, "other/gone.py", "outside the map"); - assert_eq!(other.get("success").and_then(|s| s.as_bool()), Some(true), "{other}"); + assert_eq!( + other.get("success").and_then(|s| s.as_bool()), + Some(true), + "{other}" + ); let pruned = api::prune_health(&root); - assert_eq!(pruned.get("success").and_then(|s| s.as_bool()), Some(true), "{pruned}"); + assert_eq!( + pruned.get("success").and_then(|s| s.as_bool()), + Some(true), + "{pruned}" + ); assert!(int_field(&pruned, "changed") >= 1, "{pruned}"); let files: Vec = read_health(&root).into_iter().map(|row| row.file).collect(); assert!(files.iter().any(|file| file == "README.md"), "{files:?}"); assert!(files.iter().any(|file| file == "src/a.py"), "{files:?}"); - assert!(files.iter().all(|file| file != "other/gone.py"), "{files:?}"); + assert!( + files.iter().all(|file| file != "other/gone.py"), + "{files:?}" + ); let pending = api::prune_pending(&root); - assert_eq!(pending.get("success").and_then(|s| s.as_bool()), Some(true), "{pending}"); + assert_eq!( + pending.get("success").and_then(|s| s.as_bool()), + Some(true), + "{pending}" + ); let pending_text = fs::read_to_string(root.join("pending_updates.md")).unwrap_or_default(); assert!(pending_text.contains("README.md"), "{pending_text}"); assert!(!pending_text.contains("other/gone.py"), "{pending_text}"); @@ -277,26 +527,47 @@ fn prune_keeps_monitored_files_outside_the_map_roots() { #[test] fn serve_loop_serves_the_dashboard_files_and_shuts_down() { let root = scratch("serve-dash"); - fs::write(root.join("file_health.json"), r#"{"entries":{"src/a.py":{"status":"green"}}}"#).unwrap(); + fs::write( + root.join("file_health.json"), + r#"{"entries":{"src/a.py":{"status":"green"}}}"#, + ) + .unwrap(); fs::write(root.join("library.md"), "# map\n").unwrap(); let server_root = root.clone(); let handle = std::thread::spawn(move || api::serve_loop(&server_root, 0)); let port_file = root.join(".wikifier_staging/serve.port"); - let port = (0..50).find_map(|_| { - std::thread::sleep(std::time::Duration::from_millis(20)); - fs::read_to_string(&port_file).ok().and_then(|text| text.trim().parse::().ok()) - }).expect("serve did not publish a port"); - let health = http_exchange(port, "GET /file_health.json HTTP/1.1\r\nHost: 127.0.0.1\r\nConnection: close\r\n\r\n"); + let port = (0..50) + .find_map(|_| { + std::thread::sleep(std::time::Duration::from_millis(20)); + fs::read_to_string(&port_file) + .ok() + .and_then(|text| text.trim().parse::().ok()) + }) + .expect("serve did not publish a port"); + let health = http_exchange( + port, + "GET /file_health.json HTTP/1.1\r\nHost: 127.0.0.1\r\nConnection: close\r\n\r\n", + ); assert!(health.contains("200"), "{health}"); assert!(health.contains("src/a.py"), "{health}"); - let status = http_exchange(port, "GET /__wikifier/status HTTP/1.1\r\nHost: 127.0.0.1\r\nConnection: close\r\n\r\n"); + let status = http_exchange( + port, + "GET /__wikifier/status HTTP/1.1\r\nHost: 127.0.0.1\r\nConnection: close\r\n\r\n", + ); assert!(status.contains("wikifier_serve"), "{status}"); - let library = http_exchange(port, "GET /library.md HTTP/1.1\r\nHost: 127.0.0.1\r\nConnection: close\r\n\r\n"); + let library = http_exchange( + port, + "GET /library.md HTTP/1.1\r\nHost: 127.0.0.1\r\nConnection: close\r\n\r\n", + ); assert!(library.contains("# map"), "{library}"); let stopped = http_exchange(port, "POST /__wikifier/shutdown HTTP/1.1\r\nHost: 127.0.0.1\r\nConnection: close\r\nContent-Length: 0\r\n\r\n"); assert!(stopped.contains("ok"), "{stopped}"); let finished = handle.join().expect("serve thread"); - assert_eq!(finished.get("success").and_then(|flag| flag.as_bool()), Some(true), "{finished}"); + assert_eq!( + finished.get("success").and_then(|flag| flag.as_bool()), + Some(true), + "{finished}" + ); let _ = fs::remove_dir_all(&root); } @@ -306,21 +577,50 @@ fn heal_stubs_dry_run_leaves_initial_stubs() { let root = scratch("heal-dry"); fs::write(root.join("src/a.py"), "def alpha():\n return 1\n").unwrap(); let mapped = update_maps(&root); - assert_eq!(mapped.get("success").and_then(|flag| flag.as_bool()), Some(true), "{mapped}"); - fs::write(root.join("src/a.py.wiki.md"), "This wiki is long enough to heal the alpha stub.\n").unwrap(); + assert_eq!( + mapped.get("success").and_then(|flag| flag.as_bool()), + Some(true), + "{mapped}" + ); + fs::write( + root.join("src/a.py.wiki.md"), + format!( + "This wiki is long enough to heal the alpha stub.\n{}", + "word ".repeat(80) + ), + ) + .unwrap(); let mut flags = BTreeMap::new(); flags.insert("dry-run".into(), "true".into()); let dry = api::dispatch("heal-stubs", &root, &[], &flags); - assert_eq!(dry.get("success").and_then(|flag| flag.as_bool()), Some(true), "{dry}"); - assert_eq!(dry.get("dry_run").and_then(|flag| flag.as_bool()), Some(true), "{dry}"); + assert_eq!( + dry.get("success").and_then(|flag| flag.as_bool()), + Some(true), + "{dry}" + ); + assert_eq!( + dry.get("dry_run").and_then(|flag| flag.as_bool()), + Some(true), + "{dry}" + ); assert!(int_field(&dry, "healed_count") >= 1, "{dry}"); assert_eq!(int_field(&dry, "changed"), 0, "{dry}"); let during = read_health(&root); - assert!(during.iter().any(|row| row.file == "src/a.py" && row.reason == "Initial stub"), "{during:?}"); + assert!( + during + .iter() + .any(|row| row.file == "src/a.py" && row.reason == "Initial stub"), + "{during:?}" + ); let wet = api::dispatch("heal-stubs", &root, &[], &BTreeMap::new()); assert!(int_field(&wet, "changed") >= 1, "{wet}"); let after = read_health(&root); - assert!(after.iter().any(|row| row.file == "src/a.py" && row.reason == "healed stub"), "{after:?}"); + assert!( + after + .iter() + .any(|row| row.file == "src/a.py" && row.reason == "healed stub"), + "{after:?}" + ); let _ = fs::remove_dir_all(&root); } @@ -329,13 +629,24 @@ fn init_positional_directory_is_the_project_root() { let root = scratch("init-pos"); let dest = root.join("proj"); fs::create_dir_all(dest.join("src")).unwrap(); - let out = Command::new(env!("CARGO_BIN_EXE_wikifier")).arg("init").arg(&dest).output().unwrap(); - assert!(out.status.success(), "{}", String::from_utf8_lossy(&out.stderr)); + let out = Command::new(env!("CARGO_BIN_EXE_wikifier")) + .arg("init") + .arg(&dest) + .output() + .unwrap(); + assert!( + out.status.success(), + "{}", + String::from_utf8_lossy(&out.stderr) + ); let seeded = fs::read_to_string(dest.join("map_paths.txt")).unwrap_or_default(); assert!(seeded.contains("src"), "{seeded}"); assert!(dest.join(".wikifier/marker").is_file()); assert!(dest.join("index.html").is_file()); - assert_eq!(fs::read_to_string(root.join("map_paths.txt")).unwrap(), "src\n"); + assert_eq!( + fs::read_to_string(root.join("map_paths.txt")).unwrap(), + "src\n" + ); let _ = fs::remove_dir_all(&root); } @@ -345,12 +656,24 @@ fn search_journal_file_flag_keeps_one_path() { let root = scratch("search"); fs::write(root.join("src/a.py"), "x = 1\n").unwrap(); fs::write(root.join("src/b.py"), "y = 2\n").unwrap(); - assert_eq!(api::record_change(&root, "src/a.py", "alpha reason").get("success").and_then(|flag| flag.as_bool()), Some(true)); - assert_eq!(api::record_change(&root, "src/b.py", "beta reason").get("success").and_then(|flag| flag.as_bool()), Some(true)); + assert_eq!( + api::record_change(&root, "src/a.py", "alpha reason") + .get("success") + .and_then(|flag| flag.as_bool()), + Some(true) + ); + assert_eq!( + api::record_change(&root, "src/b.py", "beta reason") + .get("success") + .and_then(|flag| flag.as_bool()), + Some(true) + ); let mut flags = BTreeMap::new(); flags.insert("file".into(), "src/a.py".into()); let found = api::dispatch("search-journal", &root, &[], &flags); - let Json::Array(entries) = found.get("entries").unwrap() else { panic!("{found}") }; + let Json::Array(entries) = found.get("entries").unwrap() else { + panic!("{found}") + }; assert_eq!(entries.len(), 1, "{found}"); let _ = fs::remove_dir_all(&root); } @@ -362,15 +685,44 @@ fn verify_wikis_prefix_checks_one_file() { fs::write(root.join("src/a.py"), "def alpha():\n return 1\n").unwrap(); fs::write(root.join("src/b.py"), "def beta():\n return 2\n").unwrap(); let mapped = update_maps(&root); - assert_eq!(mapped.get("success").and_then(|flag| flag.as_bool()), Some(true), "{mapped}"); - fs::write(root.join("src/a.py.wiki.md"), "alpha is the public function.\n").unwrap(); - fs::write(root.join("src/b.py.wiki.md"), "beta is the public function.\n").unwrap(); + assert_eq!( + mapped.get("success").and_then(|flag| flag.as_bool()), + Some(true), + "{mapped}" + ); + fs::write( + root.join("src/a.py.wiki.md"), + "alpha is the public function.\n", + ) + .unwrap(); + fs::write( + root.join("src/b.py.wiki.md"), + "beta is the public function.\n", + ) + .unwrap(); let marked_a = mark_green(&root, "src/a.py", "wiki names alpha", false); let marked_b = mark_green(&root, "src/b.py", "wiki names beta", false); - assert_eq!(marked_a.get("success").and_then(|flag| flag.as_bool()), Some(true), "{marked_a}"); - assert_eq!(marked_b.get("success").and_then(|flag| flag.as_bool()), Some(true), "{marked_b}"); - let checked = api::dispatch("verify-wikis", &root, &["src/a.py".into()], &BTreeMap::new()); - assert_eq!(checked.get("success").and_then(|flag| flag.as_bool()), Some(true), "{checked}"); + assert_eq!( + marked_a.get("success").and_then(|flag| flag.as_bool()), + Some(true), + "{marked_a}" + ); + assert_eq!( + marked_b.get("success").and_then(|flag| flag.as_bool()), + Some(true), + "{marked_b}" + ); + let checked = api::dispatch( + "verify-wikis", + &root, + &["src/a.py".into()], + &BTreeMap::new(), + ); + assert_eq!( + checked.get("success").and_then(|flag| flag.as_bool()), + Some(true), + "{checked}" + ); assert_eq!(int_field(&checked, "checked"), 1, "{checked}"); let _ = fs::remove_dir_all(&root); } @@ -397,32 +749,101 @@ mod tests { pub fn inside() {} } "; fs::write(root.join("src/lib.rs"), source).unwrap(); fs::write(root.join("src/child.rs"), "pub fn exported() {}\n").unwrap(); - fs::write(root.join("src/lib.rs.wiki.md"), "Names kept, Widget, Phase, Run, child, Alias, FLAG, exported, method.\n").unwrap(); + fs::write( + root.join("src/lib.rs.wiki.md"), + "Names kept, Widget, Phase, Run, child, Alias, FLAG, exported, method.\n", + ) + .unwrap(); let marked = mark_green(&root, "src/lib.rs", "wiki names the public items", false); - assert_eq!(marked.get("success").and_then(|flag| flag.as_bool()), Some(true), "{marked}"); - assert_eq!(marked.get("verification").and_then(|flag| flag.as_str()), Some("verified"), "{marked}"); - let row = read_health(&root).into_iter().find(|row| row.file == "src/lib.rs").unwrap(); - assert!(row.public_symbols.contains("kept"), "{}", row.public_symbols); - assert!(row.public_symbols.contains("method"), "inherent pub method missing: {}", row.public_symbols); - assert!(!row.public_symbols.contains("hidden"), "pub(crate) counted: {}", row.public_symbols); - assert!(!row.public_symbols.contains("crate_only"), "restricted method counted: {}", row.public_symbols); - assert!(!row.public_symbols.contains("inside"), "nested pub counted: {}", row.public_symbols); + assert_eq!( + marked.get("success").and_then(|flag| flag.as_bool()), + Some(true), + "{marked}" + ); + assert_eq!( + marked.get("verification").and_then(|flag| flag.as_str()), + Some("verified"), + "{marked}" + ); + let row = read_health(&root) + .into_iter() + .find(|row| row.file == "src/lib.rs") + .unwrap(); + assert!( + row.public_symbols.contains("kept"), + "{}", + row.public_symbols + ); + assert!( + row.public_symbols.contains("method"), + "inherent pub method missing: {}", + row.public_symbols + ); + assert!( + !row.public_symbols.contains("hidden"), + "pub(crate) counted: {}", + row.public_symbols + ); + assert!( + !row.public_symbols.contains("crate_only"), + "restricted method counted: {}", + row.public_symbols + ); + assert!( + !row.public_symbols.contains("inside"), + "nested pub counted: {}", + row.public_symbols + ); let wiki = fs::read_to_string(root.join("src/lib.rs.wiki.md")).unwrap(); - fs::write(root.join("src/lib.rs"), "pub fn kept() -> i32 { 1 }\npub fn added() -> i32 { 2 }\n").unwrap(); + fs::write( + root.join("src/lib.rs"), + "pub fn kept() -> i32 { 1 }\npub fn added() -> i32 { 2 }\n", + ) + .unwrap(); fs::write(root.join("src/lib.rs.wiki.md"), &wiki).unwrap(); let refused = mark_green(&root, "src/lib.rs", "same wiki", false); - assert_eq!(refused.get("error_type").and_then(|flag| flag.as_str()), Some("WikiVerificationError"), "{refused}"); - assert!(refused.to_string().contains("undocumented_new_symbols: added"), "{refused}"); + assert_eq!( + refused.get("error_type").and_then(|flag| flag.as_str()), + Some("WikiVerificationError"), + "{refused}" + ); + assert!( + refused + .to_string() + .contains("undocumented_new_symbols: added"), + "{refused}" + ); - fs::write(root.join("src/lib.rs.wiki.md"), "kept stays. `Widget` was removed. added is the new function.\n").unwrap(); + fs::write( + root.join("src/lib.rs.wiki.md"), + "kept stays. `Widget` was removed. added is the new function.\n", + ) + .unwrap(); let removed = mark_green(&root, "src/lib.rs", "wiki still names Widget", false); - assert_eq!(removed.get("error_type").and_then(|flag| flag.as_str()), Some("WikiVerificationError"), "{removed}"); - assert!(removed.to_string().contains("wiki_mentions_removed_symbols: Widget"), "{removed}"); + assert_eq!( + removed.get("error_type").and_then(|flag| flag.as_str()), + Some("WikiVerificationError"), + "{removed}" + ); + assert!( + removed + .to_string() + .contains("wiki_mentions_removed_symbols: Widget"), + "{removed}" + ); - fs::write(root.join("src/lib.rs.wiki.md"), "kept and added are the public functions.\n").unwrap(); + fs::write( + root.join("src/lib.rs.wiki.md"), + "kept and added are the public functions.\n", + ) + .unwrap(); let again = mark_green(&root, "src/lib.rs", "wiki names kept and added", false); - assert_eq!(again.get("verification").and_then(|flag| flag.as_str()), Some("verified"), "{again}"); + assert_eq!( + again.get("verification").and_then(|flag| flag.as_str()), + Some("verified"), + "{again}" + ); let _ = fs::remove_dir_all(&root); } diff --git a/tests/robustness.rs b/tests/robustness.rs index 21be480..0d68595 100644 --- a/tests/robustness.rs +++ b/tests/robustness.rs @@ -6,12 +6,18 @@ use std::path::PathBuf; use std::thread; use std::time::{SystemTime, UNIX_EPOCH}; -use wikifier::cache::{self, by_src_path, check_consistency, commit_pending_path, payload_bytes, payload_path, rebuild, FileUpdate, Session}; +use wikifier::cache::{ + self, by_src_path, check_consistency, commit_pending_path, payload_bytes, payload_path, + rebuild, FileUpdate, Session, +}; use wikifier::jsonutil::Json; use wikifier::{load_dependencies, load_dependents, mark_green, update_maps}; fn scratch(name: &str) -> PathBuf { - let n = SystemTime::now().duration_since(UNIX_EPOCH).unwrap().as_nanos(); + let n = SystemTime::now() + .duration_since(UNIX_EPOCH) + .unwrap() + .as_nanos(); let dir = std::env::temp_dir().join(format!("wk-robust-{name}-{n}-{}", std::process::id())); let _ = fs::remove_dir_all(&dir); fs::create_dir_all(dir.join("src/pkg")).unwrap(); @@ -20,13 +26,25 @@ fn scratch(name: &str) -> PathBuf { } fn strings(value: &Json, key: &str) -> Vec { - value.get(key).and_then(|v| v.as_array()).unwrap_or(&[]).iter().filter_map(|v| v.as_str().map(|s| s.to_string())).collect() + value + .get(key) + .and_then(|v| v.as_array()) + .unwrap_or(&[]) + .iter() + .filter_map(|v| v.as_str().map(|s| s.to_string())) + .collect() } fn has_dep(value: &Json, key: &str, needle: &str) -> bool { - value.get(key).and_then(|v| v.as_array()).unwrap_or(&[]).iter().any(|item| { - item.as_str() == Some(needle) || item.get("path").and_then(|p| p.as_str()) == Some(needle) - }) + value + .get(key) + .and_then(|v| v.as_array()) + .unwrap_or(&[]) + .iter() + .any(|item| { + item.as_str() == Some(needle) + || item.get("path").and_then(|p| p.as_str()) == Some(needle) + }) } #[test] @@ -41,25 +59,51 @@ fn non_ascii_and_non_utf8_files_do_not_abort_the_map() { fs::write(root.join("src/pkg/a.py"), python).unwrap(); fs::write(root.join("src/pkg/b.py"), "VALUE = 1\n").unwrap(); fs::write(root.join("src/pkg/bad.py"), b"from . import b\n\xff\n").unwrap(); - fs::write(root.join("src/app.ts"), "import { add } from \"./math\";\nconst café = \"🐍漢字\";\n").unwrap(); - fs::write(root.join("src/math.ts"), "export const add = (n: number) => n;\n").unwrap(); + fs::write( + root.join("src/app.ts"), + "import { add } from \"./math\";\nconst café = \"🐍漢字\";\n", + ) + .unwrap(); + fs::write( + root.join("src/math.ts"), + "export const add = (n: number) => n;\n", + ) + .unwrap(); let mapped = update_maps(&root); - assert_eq!(mapped.get("success").and_then(|s| s.as_bool()), Some(true), "{mapped}"); + assert_eq!( + mapped.get("success").and_then(|s| s.as_bool()), + Some(true), + "{mapped}" + ); let warnings = strings(&mapped, "warnings"); - assert!(warnings.iter().any(|w| w.contains("src/pkg/bad.py") && w.contains("UTF-8")), "{warnings:?}"); - assert!(warnings.iter().all(|w| !w.contains("src/pkg/a.py")), "valid non-ASCII file warned: {warnings:?}"); + assert!( + warnings + .iter() + .any(|w| w.contains("src/pkg/bad.py") && w.contains("UTF-8")), + "{warnings:?}" + ); + assert!( + warnings.iter().all(|w| !w.contains("src/pkg/a.py")), + "valid non-ASCII file warned: {warnings:?}" + ); let deps = load_dependencies(&root, "src/pkg/a.py"); assert!(has_dep(&deps, "internal", "src/pkg/b.py"), "{deps}"); let bad = load_dependencies(&root, "src/pkg/bad.py"); - assert!(has_dep(&bad, "internal", "src/pkg/b.py"), "lossy UTF-8 dropped the import: {bad}"); + assert!( + has_dep(&bad, "internal", "src/pkg/b.py"), + "lossy UTF-8 dropped the import: {bad}" + ); let ts = load_dependencies(&root, "src/app.ts"); assert!(has_dep(&ts, "internal", "src/math.ts"), "{ts}"); let marked = mark_green(&root, "src/pkg/bad.py", "", false); let marked_warnings = strings(&marked, "warnings"); - assert!(marked_warnings.iter().any(|w| w.contains("UTF-8")), "{marked}"); + assert!( + marked_warnings.iter().any(|w| w.contains("UTF-8")), + "{marked}" + ); let _ = fs::remove_dir_all(&root); } @@ -91,7 +135,10 @@ fn hashed_cache_names_round_trip_long_paths() { let stored = payload_path(&root, &rel); let name = stored.file_name().unwrap().to_string_lossy(); - assert!(name.len() < 32, "cache file name is still the raw path: {name}"); + assert!( + name.len() < 32, + "cache file name is still the raw path: {name}" + ); let payload = String::from_utf8(payload_bytes(&root, &rel).expect("payload")).unwrap(); assert!(payload.contains(&rel), "{payload}"); let edges = fs::read_to_string(by_src_path(&root, &rel)).unwrap(); @@ -107,34 +154,73 @@ fn crash_window_mismatch_is_detected_and_rebuilt() { fs::write(root.join("src/pkg/b.py"), "VALUE = 1\n").unwrap(); fs::write(root.join("src/pkg/d.py"), "VALUE = 2\n").unwrap(); let mapped = update_maps(&root); - assert_eq!(mapped.get("success").and_then(|s| s.as_bool()), Some(true), "{mapped}"); + assert_eq!( + mapped.get("success").and_then(|s| s.as_bool()), + Some(true), + "{mapped}" + ); let before_b = load_dependents(&root, "src/pkg/b.py"); - assert!(has_dep(&before_b, "dependents", "src/pkg/a.py"), "{before_b}"); + assert!( + has_dep(&before_b, "dependents", "src/pkg/a.py"), + "{before_b}" + ); - fs::write(by_src_path(&root, "src/pkg/a.py"), "@\tsrc/pkg/a.py\nsrc/pkg/d.py\t1\t\n").unwrap(); + fs::write( + by_src_path(&root, "src/pkg/a.py"), + "@\tsrc/pkg/a.py\nsrc/pkg/d.py\t1\t\n", + ) + .unwrap(); fs::write(commit_pending_path(&root), "99\n").unwrap(); let stale_d = load_dependents(&root, "src/pkg/d.py"); - assert!(!has_dep(&stale_d, "dependents", "src/pkg/a.py"), "reverse index updated despite the crash: {stale_d}"); + assert!( + !has_dep(&stale_d, "dependents", "src/pkg/a.py"), + "reverse index updated despite the crash: {stale_d}" + ); let stale_b = load_dependents(&root, "src/pkg/b.py"); - assert!(has_dep(&stale_b, "dependents", "src/pkg/a.py"), "forward crash should leave the old reverse edge: {stale_b}"); + assert!( + has_dep(&stale_b, "dependents", "src/pkg/a.py"), + "forward crash should leave the old reverse edge: {stale_b}" + ); let report = wikifier::api::validate(&root); - assert_eq!(report.get("success").and_then(|s| s.as_bool()), Some(false), "{report}"); - assert_eq!(report.get("cache_consistent").and_then(|s| s.as_bool()), Some(false), "{report}"); + assert_eq!( + report.get("success").and_then(|s| s.as_bool()), + Some(false), + "{report}" + ); + assert_eq!( + report.get("cache_consistent").and_then(|s| s.as_bool()), + Some(false), + "{report}" + ); assert!(!strings(&report, "cache_mismatches").is_empty(), "{report}"); rebuild(&root).unwrap(); let fixed_d = load_dependents(&root, "src/pkg/d.py"); assert!(has_dep(&fixed_d, "dependents", "src/pkg/a.py"), "{fixed_d}"); let fixed_b = load_dependents(&root, "src/pkg/b.py"); - assert!(!has_dep(&fixed_b, "dependents", "src/pkg/a.py"), "{fixed_b}"); + assert!( + !has_dep(&fixed_b, "dependents", "src/pkg/a.py"), + "{fixed_b}" + ); let deps = load_dependencies(&root, "src/pkg/a.py"); assert!(has_dep(&deps, "internal", "src/pkg/d.py"), "{deps}"); let again = wikifier::api::validate(&root); - assert_eq!(again.get("cache_consistent").and_then(|s| s.as_bool()), Some(true), "{again}"); - assert_eq!(again.get("success").and_then(|s| s.as_bool()), Some(true), "{again}"); - assert!(!commit_pending_path(&root).is_file(), "pending marker survived rebuild"); + assert_eq!( + again.get("cache_consistent").and_then(|s| s.as_bool()), + Some(true), + "{again}" + ); + assert_eq!( + again.get("success").and_then(|s| s.as_bool()), + Some(true), + "{again}" + ); + assert!( + !commit_pending_path(&root).is_file(), + "pending marker survived rebuild" + ); let _ = check_consistency(&root); let _ = fs::remove_dir_all(&root); } @@ -150,9 +236,16 @@ fn parallel_updates_do_not_corrupt_the_cache() { let root = root.clone(); handles.push(thread::spawn(move || update_maps(&root))); } - let results: Vec<_> = handles.into_iter().map(|handle| handle.join().expect("thread")).collect(); + let results: Vec<_> = handles + .into_iter() + .map(|handle| handle.join().expect("thread")) + .collect(); for result in &results { - assert_eq!(result.get("success").and_then(|s| s.as_bool()), Some(true), "{result}"); + assert_eq!( + result.get("success").and_then(|s| s.as_bool()), + Some(true), + "{result}" + ); } let paths = cache::all_paths(&root); assert_eq!(paths.len(), 12, "{paths:?}"); diff --git a/tests/skip_maps.rs b/tests/skip_maps.rs index 3bfbd8c..fbd4e5f 100644 --- a/tests/skip_maps.rs +++ b/tests/skip_maps.rs @@ -11,7 +11,10 @@ use wikifier::jsonutil::Json; use wikifier::{load_dependencies, update_maps}; fn scratch(name: &str) -> PathBuf { - let n = SystemTime::now().duration_since(UNIX_EPOCH).unwrap().as_nanos(); + let n = SystemTime::now() + .duration_since(UNIX_EPOCH) + .unwrap() + .as_nanos(); let dir = std::env::temp_dir().join(format!("wk-skip-{name}-{n}-{}", std::process::id())); let _ = fs::remove_dir_all(&dir); fs::create_dir_all(dir.join("src/pkg")).unwrap(); @@ -27,13 +30,25 @@ fn int_field(value: &Json, key: &str) -> i64 { } fn strings(value: &Json, key: &str) -> Vec { - value.get(key).and_then(|v| v.as_array()).unwrap_or(&[]).iter().filter_map(|v| v.as_str().map(|s| s.to_string())).collect() + value + .get(key) + .and_then(|v| v.as_array()) + .unwrap_or(&[]) + .iter() + .filter_map(|v| v.as_str().map(|s| s.to_string())) + .collect() } fn has_dep(value: &Json, key: &str, needle: &str) -> bool { - value.get(key).and_then(|v| v.as_array()).unwrap_or(&[]).iter().any(|item| { - item.as_str() == Some(needle) || item.get("path").and_then(|p| p.as_str()) == Some(needle) - }) + value + .get(key) + .and_then(|v| v.as_array()) + .unwrap_or(&[]) + .iter() + .any(|item| { + item.as_str() == Some(needle) + || item.get("path").and_then(|p| p.as_str()) == Some(needle) + }) } #[test] @@ -44,48 +59,121 @@ fn stat_skip_covers_unchanged_content_mtime_delete_and_new() { fs::write(root.join("src/pkg/b.py"), "VALUE = 2\n").unwrap(); let cold = update_maps(&root); - assert_eq!(cold.get("success").and_then(|s| s.as_bool()), Some(true), "{cold}"); + assert_eq!( + cold.get("success").and_then(|s| s.as_bool()), + Some(true), + "{cold}" + ); let n = int_field(&cold, "parseable_files"); - assert_eq!(int_field(&cold, "files_parsed"), n, "cold run should parse every file: {cold}"); + assert_eq!( + int_field(&cold, "files_parsed"), + n, + "cold run should parse every file: {cold}" + ); assert_eq!(int_field(&cold, "files_skipped"), 0, "{cold}"); - assert!(!cold.get("edges").and_then(|v| v.as_array()).unwrap().is_empty(), "default output still lists edges: {cold}"); + assert!( + !cold + .get("edges") + .and_then(|v| v.as_array()) + .unwrap() + .is_empty(), + "default output still lists edges: {cold}" + ); let deps = load_dependencies(&root, "src/pkg/a.py"); assert!(has_dep(&deps, "internal", "src/pkg/b.py"), "{deps}"); let warm = update_maps(&root); assert_eq!(int_field(&warm, "files_parsed"), 0, "{warm}"); - assert_eq!(int_field(&warm, "files_hashed"), 0, "unchanged files were read: {warm}"); + assert_eq!( + int_field(&warm, "files_hashed"), + 0, + "unchanged files were read: {warm}" + ); assert_eq!(int_field(&warm, "files_skipped"), n, "{warm}"); - assert!(has_dep(&load_dependencies(&root, "src/pkg/a.py"), "internal", "src/pkg/b.py")); + assert!(has_dep( + &load_dependencies(&root, "src/pkg/a.py"), + "internal", + "src/pkg/b.py" + )); - fs::write(root.join("src/pkg/a.py"), "from . import b\nVALUE = 1\n\ndef extra():\n return 3\n").unwrap(); + fs::write( + root.join("src/pkg/a.py"), + "from . import b\nVALUE = 1\n\ndef extra():\n return 3\n", + ) + .unwrap(); let edited = update_maps(&root); assert!(int_field(&edited, "files_parsed") >= 1, "{edited}"); - assert!(int_field(&edited, "files_skipped") >= 1, "untouched files were parsed: {edited}"); - assert!(has_dep(&load_dependencies(&root, "src/pkg/a.py"), "internal", "src/pkg/b.py"), "{edited}"); + assert!( + int_field(&edited, "files_skipped") >= 1, + "untouched files were parsed: {edited}" + ); + assert!( + has_dep( + &load_dependencies(&root, "src/pkg/a.py"), + "internal", + "src/pkg/b.py" + ), + "{edited}" + ); let b = root.join("src/pkg/b.py"); let later = SystemTime::now() + Duration::from_secs(30); - File::options().write(true).open(&b).unwrap().set_modified(later).unwrap(); + File::options() + .write(true) + .open(&b) + .unwrap() + .set_modified(later) + .unwrap(); let touched = update_maps(&root); - assert_eq!(int_field(&touched, "files_parsed"), 0, "mtime-only change was reparsed: {touched}"); - assert!(int_field(&touched, "files_hashed") >= 1, "mtime-only change skipped the hash: {touched}"); - assert!(has_dep(&load_dependencies(&root, "src/pkg/a.py"), "internal", "src/pkg/b.py")); + assert_eq!( + int_field(&touched, "files_parsed"), + 0, + "mtime-only change was reparsed: {touched}" + ); + assert!( + int_field(&touched, "files_hashed") >= 1, + "mtime-only change skipped the hash: {touched}" + ); + assert!(has_dep( + &load_dependencies(&root, "src/pkg/a.py"), + "internal", + "src/pkg/b.py" + )); let settled = update_maps(&root); - assert_eq!(int_field(&settled, "files_hashed"), 0, "stored mtime was not refreshed: {settled}"); + assert_eq!( + int_field(&settled, "files_hashed"), + 0, + "stored mtime was not refreshed: {settled}" + ); assert_eq!(int_field(&settled, "files_skipped"), n, "{settled}"); fs::remove_file(&b).unwrap(); let deleted = update_maps(&root); - assert_eq!(deleted.get("success").and_then(|s| s.as_bool()), Some(true), "{deleted}"); + assert_eq!( + deleted.get("success").and_then(|s| s.as_bool()), + Some(true), + "{deleted}" + ); let paths = wikifier::cache::all_paths(&root); assert!(!paths.iter().any(|p| p == "src/pkg/b.py"), "{paths:?}"); - assert!(!has_dep(&load_dependencies(&root, "src/pkg/a.py"), "internal", "src/pkg/b.py"), "dangling edge kept"); + assert!( + !has_dep( + &load_dependencies(&root, "src/pkg/a.py"), + "internal", + "src/pkg/b.py" + ), + "dangling edge kept" + ); fs::write(root.join("src/pkg/c.py"), "VALUE = 4\n").unwrap(); let added = update_maps(&root); assert!(int_field(&added, "files_parsed") >= 1, "{added}"); - assert!(wikifier::cache::all_paths(&root).iter().any(|p| p == "src/pkg/c.py"), "{added}"); + assert!( + wikifier::cache::all_paths(&root) + .iter() + .any(|p| p == "src/pkg/c.py"), + "{added}" + ); let _ = fs::remove_dir_all(&root); } @@ -93,29 +181,70 @@ fn stat_skip_covers_unchanged_content_mtime_delete_and_new() { fn same_size_and_mtime_is_caught_by_validate_and_full_rehash() { let root = scratch("stealth"); fs::write(root.join("src/a.py"), "VALUE = 1\n").unwrap(); - assert_eq!(update_maps(&root).get("success").and_then(|s| s.as_bool()), Some(true)); + assert_eq!( + update_maps(&root).get("success").and_then(|s| s.as_bool()), + Some(true) + ); let path = root.join("src/a.py"); let mtime = fs::metadata(&path).unwrap().modified().unwrap(); fs::write(&path, "VALUE = 9\n").unwrap(); - File::options().write(true).open(&path).unwrap().set_modified(mtime).unwrap(); - assert_eq!(fs::metadata(&path).unwrap().modified().unwrap(), mtime, "this platform did not preserve mtime"); + File::options() + .write(true) + .open(&path) + .unwrap() + .set_modified(mtime) + .unwrap(); + assert_eq!( + fs::metadata(&path).unwrap().modified().unwrap(), + mtime, + "this platform did not preserve mtime" + ); let skipped = update_maps(&root); - assert_eq!(int_field(&skipped, "files_parsed"), 0, "stat hit reparsed a same-size same-mtime edit: {skipped}"); + assert_eq!( + int_field(&skipped, "files_parsed"), + 0, + "stat hit reparsed a same-size same-mtime edit: {skipped}" + ); assert_eq!(int_field(&skipped, "files_hashed"), 0, "{skipped}"); let report = validate(&root); - assert_eq!(report.get("success").and_then(|s| s.as_bool()), Some(false), "{report}"); - assert_eq!(report.get("error_type").and_then(|s| s.as_str()), Some("StaleMap"), "{report}"); - assert!(strings(&report, "content_drift").iter().any(|p| p == "src/a.py"), "{report}"); + assert_eq!( + report.get("success").and_then(|s| s.as_bool()), + Some(false), + "{report}" + ); + assert_eq!( + report.get("error_type").and_then(|s| s.as_str()), + Some("StaleMap"), + "{report}" + ); + assert!( + strings(&report, "content_drift") + .iter() + .any(|p| p == "src/a.py"), + "{report}" + ); let mut flags = BTreeMap::new(); flags.insert("full".into(), "true".into()); let repaired = dispatch("update-maps", &root, &[], &flags); - assert_eq!(repaired.get("success").and_then(|s| s.as_bool()), Some(true), "{repaired}"); - assert_eq!(repaired.get("rehash").and_then(|s| s.as_bool()), Some(true), "{repaired}"); + assert_eq!( + repaired.get("success").and_then(|s| s.as_bool()), + Some(true), + "{repaired}" + ); + assert_eq!( + repaired.get("rehash").and_then(|s| s.as_bool()), + Some(true), + "{repaired}" + ); assert!(int_field(&repaired, "files_parsed") >= 1, "{repaired}"); let clean = validate(&root); - assert_eq!(clean.get("success").and_then(|s| s.as_bool()), Some(true), "{clean}"); + assert_eq!( + clean.get("success").and_then(|s| s.as_bool()), + Some(true), + "{clean}" + ); assert!(strings(&clean, "content_drift").is_empty(), "{clean}"); let _ = fs::remove_dir_all(&root); } @@ -129,16 +258,38 @@ fn quiet_omits_the_edge_list_without_dropping_the_count() { let loud = update_maps(&root); let loud_edges = strings(&loud, "edges"); assert!(!loud_edges.is_empty() || int_field(&loud, "edges_persisted") >= 0); - assert!(!loud.get("edges").and_then(|v| v.as_array()).unwrap().is_empty(), "{loud}"); + assert!( + !loud + .get("edges") + .and_then(|v| v.as_array()) + .unwrap() + .is_empty(), + "{loud}" + ); let persisted = int_field(&loud, "edges_persisted"); assert!(persisted >= 1, "{loud}"); let mut flags = BTreeMap::new(); flags.insert("quiet".into(), "true".into()); let quiet = dispatch("update-maps", &root, &[], &flags); - assert_eq!(quiet.get("success").and_then(|s| s.as_bool()), Some(true), "{quiet}"); - assert_eq!(quiet.get("edges_omitted").and_then(|s| s.as_bool()), Some(true), "{quiet}"); - assert!(quiet.get("edges").and_then(|v| v.as_array()).unwrap().is_empty(), "{quiet}"); + assert_eq!( + quiet.get("success").and_then(|s| s.as_bool()), + Some(true), + "{quiet}" + ); + assert_eq!( + quiet.get("edges_omitted").and_then(|s| s.as_bool()), + Some(true), + "{quiet}" + ); + assert!( + quiet + .get("edges") + .and_then(|v| v.as_array()) + .unwrap() + .is_empty(), + "{quiet}" + ); assert_eq!(int_field(&quiet, "edges_persisted"), persisted, "{quiet}"); assert_eq!(int_field(&quiet, "files_parsed"), 0, "{quiet}"); let _ = fs::remove_dir_all(&root);