Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
211 changes: 195 additions & 16 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ on:
workflow_dispatch:

permissions:
contents: write
contents: read

jobs:
test:
Expand All @@ -19,28 +19,207 @@ jobs:
- name: Test
run: cargo test --locked

release:
name: GitHub release
runs-on: ubuntu-latest
build:
name: Build ${{ matrix.target }}
needs: test
runs-on: ${{ matrix.os }}
permissions:
contents: read
actions: write
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-latest
target: x86_64-unknown-linux-gnu
archive: tar.gz
- os: macos-latest
target: aarch64-apple-darwin
archive: tar.gz
- os: macos-latest
target: x86_64-apple-darwin
archive: tar.gz
- os: windows-latest
target: x86_64-pc-windows-msvc
archive: zip
steps:
- uses: actions/checkout@v6

- uses: dtolnay/rust-toolchain@1.83.0
with:
targets: ${{ matrix.target }}

- name: Resolve version
shell: bash
run: |
set -euo pipefail
cargo_version="$(awk -F '"' '/^version = / { print $2; exit }' Cargo.toml)"
if [[ -z "$cargo_version" ]]; then
echo "Cargo.toml has no version" >&2
exit 1
fi
version="$cargo_version"
if [[ "$GITHUB_REF" == refs/tags/v*.*.* ]]; then
version="${GITHUB_REF_NAME#v}"
if [[ "$version" != "$cargo_version" ]]; then
echo "tag v${version} does not match Cargo.toml ${cargo_version}" >&2
exit 1
fi
fi
mkdir -p dist
printf '%s\n' "$version" > dist/VERSION

- name: Build release binary
run: cargo build --release --locked
- name: Create GitHub Release
uses: softprops/action-gh-release@v3
shell: bash
env:
TARGET: ${{ matrix.target }}
run: cargo build --release --locked --target "$TARGET"

- name: Package tar.gz
if: matrix.archive == 'tar.gz'
shell: bash
env:
TARGET: ${{ matrix.target }}
run: |
set -euo pipefail
version="$(tr -d '\r\n' < dist/VERSION)"
name="wikifier-${version}-${TARGET}"
stage="$(mktemp -d)"
cp "target/${TARGET}/release/wikifier" "${stage}/wikifier"
tar -C "$stage" -czf "${name}.tar.gz" wikifier
rm -rf "$stage" dist/VERSION
mv "${name}.tar.gz" dist/
listing="$(tar -tzf "dist/${name}.tar.gz")"
if [[ "$listing" != "wikifier" ]]; then
echo "archive listing: ${listing}" >&2
exit 1
fi

- name: Package zip
if: matrix.archive == 'zip'
shell: pwsh
env:
TARGET: ${{ matrix.target }}
run: |
$ErrorActionPreference = 'Stop'
$version = (Get-Content -Raw 'dist/VERSION').Trim()
$name = "wikifier-$version-$env:TARGET"
Copy-Item "target/$env:TARGET/release/wikifier.exe" 'dist/wikifier.exe'
Remove-Item 'dist/VERSION'
Push-Location dist
try {
Compress-Archive -LiteralPath 'wikifier.exe' -DestinationPath "$name.zip" -Force
} finally {
Pop-Location
}
Remove-Item 'dist/wikifier.exe'
if (-not (Test-Path "dist/$name.zip")) { throw "missing dist/$name.zip" }

- name: Upload archive
uses: actions/upload-artifact@v7
with:
name: release-${{ matrix.target }}
path: dist/wikifier-*
if-no-files-found: error

release:
name: GitHub release
needs: build
runs-on: ubuntu-latest
permissions:
contents: write
actions: write
steps:
- uses: actions/checkout@v6

- name: Download archives
uses: actions/download-artifact@v8
with:
generate_release_notes: true
files: target/release/wikifier
body: |
## Installation
path: dist
pattern: release-*
merge-multiple: true

- name: SHA256SUMS
shell: bash
run: |
set -euo pipefail
cd dist
shopt -s nullglob
archives=(wikifier-*.tar.gz wikifier-*.zip)
if [[ ${#archives[@]} -ne 4 ]]; then
echo "expected 4 release archives, found ${#archives[@]}" >&2
ls -la >&2
exit 1
fi
for archive in "${archives[@]}"; do
case "$archive" in
*.tar.gz)
listing="$(tar -tzf "$archive")"
if [[ "$listing" != "wikifier" ]]; then
echo "$archive listing: ${listing}" >&2
exit 1
fi
;;
*.zip)
sig="$(od -An -tx1 -N4 "$archive" | tr -d ' \n')"
if [[ "$sig" != "504b0304" ]]; then
echo "$archive is not a zip (magic $sig)" >&2
exit 1
fi
listing="$(unzip -l "$archive")"
if [[ "$listing" != *wikifier.exe* ]]; then
echo "$archive listing: ${listing}" >&2
exit 1
fi
;;
esac
done
sha256sum -- "${archives[@]}" | LC_ALL=C sort -k2 > SHA256SUMS
sha256sum --check SHA256SUMS
cat SHA256SUMS

- name: Write release body
shell: bash
run: |
set -euo pipefail
cargo_version="$(awk -F '"' '/^version = / { print $2; exit }' Cargo.toml)"
version="$cargo_version"
if [[ "$GITHUB_REF" == refs/tags/v*.*.* ]]; then
version="${GITHUB_REF_NAME#v}"
fi
cat > release_body.md <<'EOF'
## Install

Download the archive for your platform and check it against `SHA256SUMS` before you unpack it.

Download the `wikifier` binary from this release, or build from source:
| Platform | File |
|---|---|
| Linux x86_64 | `wikifier-@@VERSION@@-x86_64-unknown-linux-gnu.tar.gz` |
| macOS Apple silicon (arm64) | `wikifier-@@VERSION@@-aarch64-apple-darwin.tar.gz` |
| macOS Intel (x86_64) | `wikifier-@@VERSION@@-x86_64-apple-darwin.tar.gz` |
| Windows x86_64 | `wikifier-@@VERSION@@-x86_64-pc-windows-msvc.zip` |

```bash
cargo build --release
./target/release/wikifier --version
```
On Linux and macOS, `tar -xzf wikifier-@@VERSION@@-<target>.tar.gz` and run `./wikifier`. On Windows, unzip the archive and run `wikifier.exe`. Put that binary on your `PATH`.

Or build from source:

```bash
cargo build --release
./target/release/wikifier --version
```

`pip install wikifier` still installs the old Python 4.7 package.

The change list is in CHANGELOG.md. Measured startup, `update-maps`, and session commands are in the README Benchmarks section.
EOF
sed -i "s/@@VERSION@@/${version}/g" release_body.md

- name: Create GitHub Release
uses: softprops/action-gh-release@v3
with:
body_path: release_body.md
files: |
dist/wikifier-*
dist/SHA256SUMS
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
4 changes: 2 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# AGENTS.md

Entry point for any coding agent working in this repository, or using Wikifier on another tree. The day-to-day contract is `skills/run.md` (Agent Protocol v0.7, package 4.7.x). Read that file before substantive work. This file says what is live, what is history, and which rules are easy to miss.
Entry point for any coding agent working in this repository, or using Wikifier on another tree. The day-to-day contract is `skills/run.md` (Agent Protocol v0.7, package 5.0.0). Read that file before substantive work. This file says what is live, what is history, and which rules are easy to miss.

## What this program is

Wikifier is a zero-dependency, agent-to-agent codebase wiki. Agents look up a health matrix, a dependency map, and short per-file wikis instead of re-reading full sources. The human dashboard (`index.html`) is a read-only view of the same artifacts.

The program is the std-only Rust binary under `src/`. Version is `Cargo.toml` (`wikifier --version`). `wikifier.sh`, `wikifier.ps1`, and `wikifier.bat` only exec that binary. MCP is `wikifier mcp` in the same binary. `.mcp.json` launches `wikifier` with args `["mcp"]`. There is no Python package. `pip install wikifier` and `python -m wikifier` are the archived product.
The program is the std-only Rust binary under `src/`. Version is `Cargo.toml` (`wikifier --version`). `wikifier.sh`, `wikifier.ps1`, and `wikifier.bat` only exec that binary. MCP is `wikifier mcp` in the same binary. `.mcp.json` launches `wikifier` with args `["mcp"]`. `pip install wikifier` and `python -m wikifier` are the archived Python 4.7 package.

`src/main.rs` and `src/mcp.rs` both call `src/api.rs`. Every call returns JSON with `success`. An operational failure is `{"success": false, "error", "error_type"}`.

Expand Down
2 changes: 1 addition & 1 deletion AGENTS.md.wiki.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@

**Role:** Entry point for any coding agent in this repository.

It points at `skills/run.md` (protocol v0.7) and names the live Rust binary, the daily loop, and the Python-era records kept under `Findings/archive/`. Map roots stay `src/` and `tests/`. The import scan stays comment-masked regex plus named re-exports. Rust also resolves `crate::`, `self::`, `super::`, root crate names, and `[workspace] members`, so `src/main.rs` and `tests/*.rs` are dependents of the library modules they import. `mark-green` checks Rust `pub` names, impl methods, and trait-impl items.
It points at `skills/run.md` (protocol v0.7, package 5.0.0) and names the live Rust binary, the daily loop, and the Python-era records kept under `Findings/archive/`. `pip install wikifier` is the archived Python 4.7 package. Map roots stay `src/` and `tests/`. The import scan stays comment-masked regex plus named re-exports. Rust also resolves `crate::`, `self::`, `super::`, root crate names, and `[workspace] members`, so `src/main.rs` and `tests/*.rs` are dependents of the library modules they import. `mark-green` checks Rust `pub` names, impl methods, and trait-impl items.
9 changes: 8 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [5.0.0] - 2026-11-03

First release of the std-only Rust rewrite. `[dependencies]` is empty: no crates.io crates and no Python runtime. The binary implements the Python 4.7 agent contract. Protocol v0.7 stays. `pip install wikifier` still installs that Python 4.7 package.

Python 4.7 at `24812ef` versus the Rust binary, 5 runs, median milliseconds (min–max). That Rust binary reported 4.7.0 when it was timed (`scripts/benchmark_results.md`). Figures: [Benchmarks](README.md#benchmarks). On the large tree, cold `update-maps` was 12675 (12540–13094) for Python and 3142 (3114–3305) for Rust; warm was 422 (417–433) and 246 (243–253). Resolved file edges on that tree were 9074 and 9074. Medium (this repo) was 44 Python and 98 Rust. Startup (`--version`) was 63 (61–66) and 2 (1–2).

### Breaking
- The Python package is gone. Wikifier is the std-only Rust binary (`cargo build --release`). `pip install wikifier` and `python -m wikifier` are no longer the product. `wikifier.sh` / `.ps1` / `.bat` launch the Rust binary. MCP is `wikifier mcp`.
- This repository no longer ships the Python package. Wikifier 5.0.0 is the std-only Rust binary (`cargo build --release`, or a release archive). `pip install wikifier` still installs the archived Python 4.7 package. `wikifier.sh` / `.ps1` / `.bat` launch the Rust binary. MCP is `wikifier mcp`.

### Added
- A `v*.*.*` tag, after the test job, builds release binaries for `x86_64-unknown-linux-gnu`, `aarch64-apple-darwin`, `x86_64-apple-darwin`, and `x86_64-pc-windows-msvc` with the same Rust 1.83.0 pin as CI. Unix archives are `.tar.gz`, Windows is `.zip`, named `wikifier-<version>-<target>`. `SHA256SUMS` is uploaded with them.
- `update-maps` writes `library.md` (file tree, Mermaid, dependency table) and sharded folder cards under `.wikifier_staging/maps/`. `--directory`, `--max-files`, and `--stream` are honored. A `--max-files` budget leaves `map_complete=false`; the next run resumes because parsed files are stat-skipped.
- `prune-health --deleted-missing` drops health rows whose path is no longer a file, including rows that still sit under a map root.
- `dependencies --full` returns per-edge confidence, conditional / dynamic / via-barrel flags, and ACS reason codes. `cycles` adds `break_hint` (lowest-confidence edge in the component) and `external_edges`.
Expand Down
4 changes: 2 additions & 2 deletions CHANGELOG.md.wiki.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# CHANGELOG.md

**Role:** Keep a Changelog history for package 4.7.x.
**Role:** Keep a Changelog history. 5.0.0 is the first Rust release (std-only, Python 4.7 contract, release archives). The 4.7.0 section is the Python package that shipped then.

The 4.7.0 section is the record of that release, including the Python package that shipped then. Unreleased notes describe the Rust binary: map artifacts, dependency flags, wiki checks, deletion, background commands, the dashboard serve routes, MCP error reporting, `heal-stubs --dry-run`, and `init DIR`. The import scan is recorded as `regex+reexport`. Unreleased notes that Rust crate-name imports resolve to the library and to `[workspace] members`, that the unreproducible lookup benchmark was removed, that Rust `pub` names including impl methods can be verified Green, and that the MCP deadline, check-changes cap, sqlite import, and low-confidence edges have tests. CI pins Rust 1.83.0 and requires Ubuntu, macOS, and Windows. CI checks the empty dependency list without Python.
The 5.0.0 notes cover map artifacts, dependency flags, wiki checks, deletion, background commands, the dashboard serve routes, MCP error reporting, Rust crate and workspace resolution, and the tag workflow that uploads `wikifier-<version>-<target>` archives plus `SHA256SUMS`. The import scan is recorded as `regex+reexport`. Benchmarks link to the README Benchmarks section and `scripts/benchmark_results.md`. CI pins Rust 1.83.0 and requires Ubuntu, macOS, and Windows.
4 changes: 2 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co

Wikifier is a **zero-dependency, agent-to-agent codebase wiki**: it gives AI agents a token-efficient, queryable map of a codebase (health matrix, dependency graph, per-file wiki summaries) so they can look things up instead of re-reading full sources. The human layer (`index.html` dashboard) is strictly secondary — a read-only viewer of the same artifacts. It is deliberately NOT a general human documentation tool.

**This repo dogfoods itself.** Wikifier runs on its own source, and you are expected to follow its agent protocol when working here (see below). `AGENTS.md` is the entry point for any coding agent. The authoritative contract is `skills/run.md` (Agent Protocol v0.7, package 4.7.x). Read it before substantive work.
**This repo dogfoods itself.** Wikifier runs on its own source, and you are expected to follow its agent protocol when working here (see below). `AGENTS.md` is the entry point for any coding agent. The authoritative contract is `skills/run.md` (Agent Protocol v0.7, package 5.0.0). Read it before substantive work.

**USE WIKIFIER FIRST (non-negotiable in this repo, preferred on any managed target):** At session start call `session_bootstrap` (MCP or `wikifier session-bootstrap`). Prefer Core daily tools (`list_core_tools`): bootstrap, `check_changes`, `prepare_edit`, `suggest_next_actions` (json `actions[]`), `record_change`, `mark_green`. Look up via map/wiki/deps before re-reading full sources. Selective work only (🔴 / actionable 🟡). Never skip `record_change` after edits.

Expand Down Expand Up @@ -56,7 +56,7 @@ wikifier mcp # MCP server on stdio

## Architecture

Pipeline: **scan → parse → resolve → cache → health → artifacts**, implemented in Rust under `src/` (version in `Cargo.toml`, currently **4.7.x**). The former Python package has been removed. CLI (`src/main.rs`) and MCP (`src/mcp.rs`) both call `src/api.rs`. Every function returns a JSON object with `success`. Operational failures return `{"success": false, "error", "error_type"}`.
Pipeline: **scan → parse → resolve → cache → health → artifacts**, implemented in Rust under `src/` (version in `Cargo.toml`, currently **5.0.0**). The Python package is no longer in this repository. `pip install wikifier` still installs Python 4.7. CLI (`src/main.rs`) and MCP (`src/mcp.rs`) both call `src/api.rs`. Every function returns a JSON object with `success`. Operational failures return `{"success": false, "error", "error_type"}`.

- `src/api.rs` — the library surface: `session_bootstrap`, `check_changes`, `prepare_edit`, `suggest_next_actions`, `record_change`, `record_changes`, `record_deletion`, `mark_green`, `verify_wikis`, `update_maps` / `update_maps_run`, `dependency_report`, `load_dependents`, `cycles`, `init_project`, `serve_loop`, `monitor_loop`, `daemon_ctl`. `check_changes` watches mapped files, Green files, and `monitored_paths.txt`. `record_changes` reads `git status` (git is optional).
- `src/scope.rs` — project root, `map_paths.txt`, `monitored_paths.txt`, excludes, and the file walk. A missing configured map directory is a scope error. Three or more child projects make the root a container and the walk stops.
Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md.wiki.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@

**Role:** Repo guidance for an agent working in this tree.

Hard constraints: empty Cargo dependencies, no hand-edits of `library.md` or the health and pending files, and the human dashboard stays secondary. The Python-era plan logs live under `Findings/archive/`. It points agents at `AGENTS.md` and `skills/run.md`. The architecture section names the Rust modules under `src/`, the library file tree, and this repo's map roots `src/` and `tests/`. The parse note includes Rust `crate::`, `self::`, `super::`, Cargo.toml crate names, `[workspace] members`, and `rust_public_symbols` for wiki checks. JSON failures use `success: false`.
Hard constraints: empty Cargo dependencies, no hand-edits of `library.md` or the health and pending files, and the human dashboard stays secondary. The Python-era plan logs live under `Findings/archive/`. It points agents at `AGENTS.md` and `skills/run.md` (protocol v0.7, package 5.0.0). The architecture section names the Rust modules under `src/`, the library file tree, and this repo's map roots `src/` and `tests/`. The parse note includes Rust `crate::`, `self::`, `super::`, Cargo.toml crate names, `[workspace] members`, and `rust_public_symbols` for wiki checks. JSON failures use `success: false`.
2 changes: 1 addition & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "wikifier"
version = "4.7.0"
version = "5.0.0"
edition = "2021"
description = "Wikifier — zero-dependency agent-to-agent codebase wiki."
license = "MIT"
Expand Down
Loading
Loading