diff --git a/.github/workflows/install.yml b/.github/workflows/install.yml new file mode 100644 index 00000000..061eb67d --- /dev/null +++ b/.github/workflows/install.yml @@ -0,0 +1,19 @@ +name: Installer controls +on: + pull_request: + push: + branches: [main] +permissions: + contents: read +jobs: + installer: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: + python-version: '3.11' + - name: Validate standalone entrypoint and refusal controls + run: | + sh -n install.sh + python3 scripts/test-install-release.py diff --git a/README.md b/README.md index f2b00e81..dd1310ea 100644 --- a/README.md +++ b/README.md @@ -1,78 +1,28 @@

Bobbin

-

Thread bobbin spool

-

bobbin

-

🧶 Find the code your next change needs.

-

- Apache-2.0 license - CI - Quipu stack -

-[Watch the 60-second context and native hook recording](docs/demos/injection-v0.27.3/README.md). +# bobbin **Bobbin gives developers and coding agents search and context over their repositories. Use its command line, Model Context Protocol (MCP) server, or agent hooks to find code by meaning, search exact words, and discover files that change together. Local indexing and search need no API key.** -## Why you would want it - -- Find relevant functions even when you do not know their names. -- Give your agent a context bundle that fits a line or token budget. -- See related files that imports alone do not reveal, using Git history. - -Read [how Bobbin combines these signals](https://scbrown.github.io/bobbin/). - ## Install -**Debian and Ubuntu (amd64 and arm64):** install from the signed apt repository, which -republishes each checksum-verified release. The package puts `bobbin` in -`/usr/bin`. Check the signing key's fingerprint before trusting it; the -commands stop if it does not match: - -```bash -curl -fsSL https://scbrown.github.io/apt/key.gpg -o scbrown-stack.gpg -gpg --show-keys --with-colons scbrown-stack.gpg \ - | grep -qx 'fpr:::::::::AB5625C5612F643BF76464DF0DC99924994EC959:' \ - && sudo install -D -m 0644 scbrown-stack.gpg /etc/apt/keyrings/scbrown-stack.gpg \ - && echo "deb [signed-by=/etc/apt/keyrings/scbrown-stack.gpg] https://scbrown.github.io/apt stable main" \ - | sudo tee /etc/apt/sources.list.d/scbrown-stack.list \ - && sudo apt update && sudo apt install bobbin +```sh +curl -fsSL https://raw.githubusercontent.com/scbrown/bobbin/main/install.sh | sh ``` -`apt upgrade` then picks up new releases. unattended-upgrades only installs -from origins it allows, so to have it upgrade bobbin too, add -`"origin=scbrown,codename=stable";` to `Unattended-Upgrade::Origins-Pattern`. -Channels, pinning and rollback: -[scbrown/apt](https://github.com/scbrown/apt#channels-and-pinning). +Python 3.9+, curl, and a Linux or macOS shell. Downloads the latest published archive, verifies SHA-256 and the executable +version before publishing command links in `~/.local/bin`. Add it to `PATH`. +The complete versioned bundle stays in `~/.local/lib/bobbin/TAG` for rollback. +Published targets: Linux and macOS, x86_64 and arm64. -Checksummed releases support Linux and macOS on x86-64 and ARM64. For Linux -x86-64, with `curl`, `tar` and `sha256sum` installed: +Set `STACK_INSTALL_ROOT` on the receiving shell for another prefix. +Pinned releases, alternatives, configuration and removal are in the +[installation and usage reference](docs/installation-reference.md). -```bash -mkdir -p bobbin-download && cd bobbin-download -base=https://github.com/scbrown/bobbin/releases/download/v0.26.0 -asset=bobbin-v0.26.0-x86_64-unknown-linux-gnu.tar.gz -curl -fLO "$base/$asset" && curl -fLO "$base/SHA256SUMS.txt" -sha256sum --check --ignore-missing SHA256SUMS.txt && tar xzf "$asset" -export PATH="$PWD/bobbin-v0.26.0-x86_64-unknown-linux-gnu:$PATH" -bobbin --version -``` - -Keep the extracted directory intact: its `lib/` contains ONNX Runtime. -The pinned release reports: - -```text -bobbin 0.26.0 (58f4726fe4973240966704bbc9fe171c5ddb3e9a) -``` - -For [other platforms and runtime prerequisites](https://scbrown.github.io/bobbin/getting-started/installation.html), -use the matching release asset. With the Rust build prerequisites installed, -`cargo install bobbin-ai --locked` builds from source and installs `bobbin`. -Run `bobbin --version`; if it reports an older version, check `command -v bobbin` -for another installation earlier in `PATH`. - -## First success in three commands +## Try it Run in a shell with Python 3 and Git installed. This creates a disposable project; initial indexing downloads the embedding model and runs locally on the CPU. @@ -92,66 +42,15 @@ greeting:1-2 Bobbin found a parsed function, with its source range, in the index you just built. See the [quick start](https://scbrown.github.io/bobbin/getting-started/quick-start.html) for semantic search. -## On your own code +## Next steps -Run `bobbin init` and `bobbin index` from your repository first. - -| Question | Command | -|---|---| -| Where is error handling implemented? | `bobbin search "error handling"` | -| What context would help fix login? | `bobbin context "fix the login bug"` | -| Where does an exact name appear? | `bobbin grep greeting` | -| What changes alongside this file? | `bobbin related greeting.py` | -| What is indexed? | `bobbin status` | +- [Tambo](https://github.com/intigraph/tambo): guided stack installation and verification. +- [Camayoc](https://github.com/scbrown/camayoc): governed memory and vocabulary. +- [Detailed examples and agent setup](docs/installation-reference.md). -See the [CLI reference](https://scbrown.github.io/bobbin/cli/overview.html) for flags and additional commands. +**Load this repository’s graph:** public repository-pendant assets have not yet been verified. When a release supplies the archive, checksum and provenance, follow the [verified import workflow](https://scbrown.github.io/quipu/sharing/index.html): extract the share directory and use `quipu import SHARE_DIR --db repository.db`. Import stages knowledge; ROOT stays unchanged. -## Wire it into your agent - -From the indexed repository, `bobbin serve` starts an MCP server on standard input/output. -Register it by **absolute path**: MCP clients launch servers without your shell's `PATH`, -so a bare `bobbin` fails with `ENOENT`. For Claude Code, from the indexed repository: - -```bash -claude mcp add bobbin -- "$(command -v bobbin)" serve "$PWD" -``` - -Other clients take the same command as JSON (use the output of `command -v bobbin`): - -```json -{"mcpServers":{"bobbin":{"command":"/absolute/path/to/bobbin","args":["serve","/path/to/repo"]}}} -``` - -For automatic Claude Code context injection, run `bobbin hook install` in the -repository; it updates that project's `.claude/settings.json`. -See [agent setup](https://scbrown.github.io/bobbin/getting-started/agent-setup.html) for client configuration -and [hooks](https://scbrown.github.io/bobbin/guides/hooks.html) for gating, deduplication and removal. -Prefer MCP when available, then the CLI; the [HTTP API](https://scbrown.github.io/bobbin/mcp/http-mode.html) -is the transport fallback. - -## Before you start - -| Requirement | What to expect | -|---|---| -| Platform | Release binaries for Linux and macOS, x86-64 and ARM64 | -| Network | Needed to download releases and the model; local search then uses the cached index | -| Parsing | Rust, TypeScript, Python, Go, Java and C++ use tree-sitter; Markdown has section-aware parsing; other text uses line chunks | -| Git | Commit history enables temporal coupling; a new repository has none yet | -| GPU | Optional; CPU indexing works without CUDA | -| Knowledge graph | Optional Quipu integration requires a build with the `knowledge` feature | - -See [installation](https://scbrown.github.io/bobbin/getting-started/installation.html) for system libraries -and source-build requirements, and [language support](https://scbrown.github.io/bobbin/architecture/languages.html). - -## What's next - -- [Read the Bobbin book](https://scbrown.github.io/bobbin/). -- [Find every document in the docs map](https://scbrown.github.io/bobbin/docs-map.html). -- [Build useful context for a task](https://scbrown.github.io/bobbin/guides/context-assembly.html). - -## 🧺 The stack - -Caboodle installs these together and proves each one works; every tool also stands alone. +## The stack @@ -177,16 +76,4 @@ when. -## Contributing - -```bash -just build -just test -just check -``` - -See [CONTRIBUTING.md](CONTRIBUTING.md) for prerequisites and documentation checks. - -## 📜 License - -[Apache License, Version 2.0](LICENSE). Releases before 2026-09-24 were MIT-licensed. +[License](LICENSE). diff --git a/docs/installation-reference.md b/docs/installation-reference.md new file mode 100644 index 00000000..8c8a174a --- /dev/null +++ b/docs/installation-reference.md @@ -0,0 +1,168 @@ +

Bobbin

+

Thread bobbin spool

+

bobbin

+

🧶 Find the code your next change needs.

+

+ Apache-2.0 license + CI + Quipu stack +

+ +[Watch the 60-second context and native hook recording](https://github.com/scbrown/bobbin/blob/main/docs/demos/injection-v0.27.3/README.md). + +**Bobbin gives developers and coding agents search and context over their +repositories. Use its command line, Model Context Protocol (MCP) server, or +agent hooks to find code by meaning, search exact words, and discover files +that change together. Local indexing and search need no API key.** + +## Why you would want it + +- Find relevant functions even when you do not know their names. +- Give your agent a context bundle that fits a line or token budget. +- See related files that imports alone do not reveal, using Git history. + +Read [how Bobbin combines these signals](https://scbrown.github.io/bobbin/). + +## Install + +**Debian and Ubuntu (amd64 and arm64):** install from the signed apt repository, which +republishes each checksum-verified release. The package puts `bobbin` in +`/usr/bin`. Check the signing key's fingerprint before trusting it; the +commands stop if it does not match: + +```bash +curl -fsSL https://scbrown.github.io/apt/key.gpg -o scbrown-stack.gpg +gpg --show-keys --with-colons scbrown-stack.gpg \ + | grep -qx 'fpr:::::::::AB5625C5612F643BF76464DF0DC99924994EC959:' \ + && sudo install -D -m 0644 scbrown-stack.gpg /etc/apt/keyrings/scbrown-stack.gpg \ + && echo "deb [signed-by=/etc/apt/keyrings/scbrown-stack.gpg] https://scbrown.github.io/apt stable main" \ + | sudo tee /etc/apt/sources.list.d/scbrown-stack.list \ + && sudo apt update && sudo apt install bobbin +``` + +`apt upgrade` then picks up new releases. unattended-upgrades only installs +from origins it allows, so to have it upgrade bobbin too, add +`"origin=scbrown,codename=stable";` to `Unattended-Upgrade::Origins-Pattern`. +Channels, pinning and rollback: +[scbrown/apt](https://github.com/scbrown/apt#channels-and-pinning). + +Checksummed releases support Linux and macOS on x86-64 and ARM64. For Linux +x86-64, with `curl`, `tar` and `sha256sum` installed: + +```bash +mkdir -p bobbin-download && cd bobbin-download +base=https://github.com/scbrown/bobbin/releases/download/v0.26.0 +asset=bobbin-v0.26.0-x86_64-unknown-linux-gnu.tar.gz +curl -fLO "$base/$asset" && curl -fLO "$base/SHA256SUMS.txt" +sha256sum --check --ignore-missing SHA256SUMS.txt && tar xzf "$asset" +export PATH="$PWD/bobbin-v0.26.0-x86_64-unknown-linux-gnu:$PATH" +bobbin --version +``` + +Keep the extracted directory intact: its `lib/` contains ONNX Runtime. +The pinned release reports: + +```text +bobbin 0.26.0 (58f4726fe4973240966704bbc9fe171c5ddb3e9a) +``` + +For [other platforms and runtime prerequisites](https://scbrown.github.io/bobbin/getting-started/installation.html), +use the matching release asset. With the Rust build prerequisites installed, +`cargo install bobbin-ai --locked` builds from source and installs `bobbin`. +Run `bobbin --version`; if it reports an older version, check `command -v bobbin` +for another installation earlier in `PATH`. + +## First success in three commands + +Run in a shell with Python 3 and Git installed. This creates a disposable project; +initial indexing downloads the embedding model and runs locally on the CPU. + +```bash +mkdir -p bobbin-demo && cd bobbin-demo && git init -q && printf 'def greeting(name):\n return "Hello, " + name\n' > greeting.py +bobbin init --quiet && BOBBIN_GPU=0 bobbin index --quiet --skip-calibrate +bobbin grep greeting --json | python3 -c 'import json,sys; r=json.load(sys.stdin)["results"][0]; print("{}:{}-{}".format(r["name"], r["start_line"], r["end_line"]))' +``` + +The final command prints: + +```text +greeting:1-2 +``` + +Bobbin found a parsed function, with its source range, in the index you just built. +See the [quick start](https://scbrown.github.io/bobbin/getting-started/quick-start.html) for semantic search. + +## On your own code + +Run `bobbin init` and `bobbin index` from your repository first. + +| Question | Command | +|---|---| +| Where is error handling implemented? | `bobbin search "error handling"` | +| What context would help fix login? | `bobbin context "fix the login bug"` | +| Where does an exact name appear? | `bobbin grep greeting` | +| What changes alongside this file? | `bobbin related greeting.py` | +| What is indexed? | `bobbin status` | + +See the [CLI reference](https://scbrown.github.io/bobbin/cli/overview.html) for flags and additional commands. + +## Wire it into your agent + +From the indexed repository, `bobbin serve` starts an MCP server on standard input/output. +Register it by **absolute path**: MCP clients launch servers without your shell's `PATH`, +so a bare `bobbin` fails with `ENOENT`. For Claude Code, from the indexed repository: + +```bash +claude mcp add bobbin -- "$(command -v bobbin)" serve "$PWD" +``` + +Other clients take the same command as JSON (use the output of `command -v bobbin`): + +```json +{"mcpServers":{"bobbin":{"command":"/absolute/path/to/bobbin","args":["serve","/path/to/repo"]}}} +``` + +For automatic Claude Code context injection, run `bobbin hook install` in the +repository; it updates that project's `.claude/settings.json`. +See [agent setup](https://scbrown.github.io/bobbin/getting-started/agent-setup.html) for client configuration +and [hooks](https://scbrown.github.io/bobbin/guides/hooks.html) for gating, deduplication and removal. +Prefer MCP when available, then the CLI; the [HTTP API](https://scbrown.github.io/bobbin/mcp/http-mode.html) +is the transport fallback. + +## Before you start + +| Requirement | What to expect | +|---|---| +| Platform | Release binaries for Linux and macOS, x86-64 and ARM64 | +| Network | Needed to download releases and the model; local search then uses the cached index | +| Parsing | Rust, TypeScript, Python, Go, Java and C++ use tree-sitter; Markdown has section-aware parsing; other text uses line chunks | +| Git | Commit history enables temporal coupling; a new repository has none yet | +| GPU | Optional; CPU indexing works without CUDA | +| Knowledge graph | Optional Quipu integration requires a build with the `knowledge` feature | + +See [installation](https://scbrown.github.io/bobbin/getting-started/installation.html) for system libraries +and source-build requirements, and [language support](https://scbrown.github.io/bobbin/architecture/languages.html). + +## What's next + +- [Read the Bobbin book](https://scbrown.github.io/bobbin/). +- [Find every document in the docs map](https://scbrown.github.io/bobbin/docs-map.html). +- [Build useful context for a task](https://scbrown.github.io/bobbin/guides/context-assembly.html). + +## 🧺 The stack + +Caboodle installs these together and proves each one works; every tool also stands alone. + +## Contributing + +```bash +just build +just test +just check +``` + +See [CONTRIBUTING.md](https://github.com/scbrown/bobbin/blob/main/CONTRIBUTING.md) for prerequisites and documentation checks. + +## 📜 License + +[Apache License, Version 2.0](https://github.com/scbrown/bobbin/blob/main/LICENSE). Releases before 2026-09-24 were MIT-licensed. diff --git a/docs/knowledge/repository-knowledge.json b/docs/knowledge/repository-knowledge.json index db5d509a..60d80108 100644 --- a/docs/knowledge/repository-knowledge.json +++ b/docs/knowledge/repository-knowledge.json @@ -1,6 +1,6 @@ { - "source_revision": "b036433f55b75ddb8520d5bf4ad0b681464187eb", - "reviewed_at": "2026-10-20T22:30:00Z", + "source_revision": "a233e28419a1e83b6cfa38e24026853f890cc2ba", + "reviewed_at": "2026-10-24T03:35:26.186803Z", "max_age_seconds": 2592000, "records": [ { @@ -109,7 +109,7 @@ "source": "README.md", "subject": "http://aegis.gastown.local/ontology/doc/bobbin/README.md", "summary": "Start with installation and the first successful search, then read architecture and contributor guidance.", - "source_sha256": "3203391e3716daec383401674f28d2310a7482d3ecf89abfe1d65f8acbdaef0e", + "source_sha256": "c6d3f8698efcd328dbd26d7b15a032bde737db79fba8093934faa73285f59317", "properties": { "startHereOrder": "1" } diff --git a/install.sh b/install.sh new file mode 100755 index 00000000..2480faa5 --- /dev/null +++ b/install.sh @@ -0,0 +1,187 @@ +#!/bin/sh +# Install verified published bytes; never build the invoking checkout. +set -eu +exec python3 - "$@" <<'PYINSTALL' +import hashlib +import json +import os +from pathlib import Path, PurePosixPath +import platform +import re +import shutil +import subprocess +import sys +import tarfile +import tempfile +import time +from urllib.error import HTTPError, URLError +from urllib.request import Request, urlopen + +CONFIG = {'archive': 'bobbin-{tag}-{target}.tar.gz', 'checksum': 'SHA256SUMS.txt', 'directory': '{archive}', 'commands': ['bobbin'], 'targets': {'Linux:x86_64': 'x86_64-unknown-linux-gnu', 'Linux:aarch64': 'aarch64-unknown-linux-gnu', 'Darwin:x86_64': 'x86_64-apple-darwin', 'Darwin:arm64': 'aarch64-apple-darwin'}, 'repo': 'bobbin'} + + +def fetch(url, destination): + for attempt in range(3): + try: + with urlopen(Request(url, headers={'User-Agent': CONFIG['repo'] + '-installer'}), timeout=60) as response: + with destination.open('wb') as output: + shutil.copyfileobj(response, output) + return + except (HTTPError, URLError, TimeoutError) as error: + transient = not isinstance(error, HTTPError) or error.code in (403, 429, 500, 502, 503, 504) + if attempt == 2 or not transient: + raise + time.sleep(attempt + 1) + + +def extract(archive, destination): + with tarfile.open(archive, 'r:gz') as bundle: + seen, links = set(), [] + total = 0 + for member in bundle: + name = PurePosixPath(member.name) + if name.is_absolute() or '..' in name.parts or str(name) in seen: + raise ValueError('unsafe or duplicate archive path') + seen.add(str(name)) + target = destination.joinpath(*name.parts) + target.parent.mkdir(parents=True, exist_ok=True) + if member.isdir(): + target.mkdir(exist_ok=True) + elif member.isfile(): + total += member.size + if total > 1024 * 1024 * 1024: + raise ValueError('release exceeds 1 GiB unpacked limit') + source = bundle.extractfile(member) + if source is None: + raise ValueError('missing archive bytes') + with source, target.open('xb') as output: + shutil.copyfileobj(source, output) + target.chmod(0o755 if member.mode & 0o111 else 0o644) + elif member.issym(): + link = PurePosixPath(member.linkname) + if link.is_absolute() or '..' in link.parts: + raise ValueError('unsafe archive link') + links.append((target, member.linkname)) + else: + raise ValueError('unsupported archive member') + for target, link in links: + target.symlink_to(link) + target.resolve(strict=True).relative_to(destination.resolve()) + + +def bundle_identity(root): + result = {} + for path in root.rglob('*'): + relative = str(path.relative_to(root)) + if relative == '.release.json': + continue + if path.is_symlink(): + result[relative] = ('link', os.readlink(path)) + elif path.is_file(): + result[relative] = ('file', path.stat().st_mode & 0o777, + hashlib.sha256(path.read_bytes()).hexdigest()) + elif path.is_dir(): + result[relative] = ('directory',) + else: + raise ValueError('unexpected installed bundle entry') + return result + + +def install(): + if len(sys.argv) > 1: + raise ValueError('configure STACK_INSTALL_ROOT and STACK_VERSION through the environment; no positional arguments') + system, machine = platform.system(), platform.machine() + target = CONFIG['targets'].get(system + ':' + machine) + if target is None: + raise ValueError('no published archive for ' + system + ' ' + machine) + with tempfile.TemporaryDirectory(prefix=CONFIG['repo'] + '-release-') as temporary: + work = Path(temporary) + tag = os.environ.get('STACK_VERSION') + if not tag: + metadata = work / 'latest.json' + fetch('https://api.github.com/repos/scbrown/' + CONFIG['repo'] + '/releases/latest', metadata) + release = json.loads(metadata.read_text()) + if release.get('draft') or release.get('prerelease'): + raise ValueError('latest release is not stable') + tag = release['tag_name'] + if not re.fullmatch(r'[A-Za-z0-9][A-Za-z0-9._-]{0,100}', tag): + raise ValueError('invalid release tag') + match = re.search(r'(?:^|v)([0-9]+\.[0-9]+\.[0-9]+)$', tag) + if not match: + raise ValueError('unsupported release version') + version = match.group(1) + name = CONFIG['archive'].format(tag=tag, version=version, target=target) + checksum = CONFIG['checksum'].format(archive=name) + base = os.environ.get('STACK_RELEASE_BASE_URL', 'https://github.com/scbrown/' + CONFIG['repo'] + '/releases/download/' + tag) + archive, manifest = work / name, work / 'checksums' + fetch(base + '/' + name, archive) + fetch(base + '/' + checksum, manifest) + matches = [] + for line in manifest.read_text().splitlines(): + fields = line.split() + if len(fields) == 2 and fields[1].lstrip('*') == name: + matches.append(fields[0].lower()) + if len(matches) != 1 or not re.fullmatch('[0-9a-f]{64}', matches[0]): + raise ValueError('missing or ambiguous checksum entry for ' + name) + digest = hashlib.sha256(archive.read_bytes()).hexdigest() + if digest != matches[0]: + raise ValueError('release checksum mismatch') + unpacked = work / 'unpacked' + unpacked.mkdir() + extract(archive, unpacked) + bundle = unpacked / CONFIG['directory'].format(archive=name.removesuffix('.tar.gz')) + for command in CONFIG['commands']: + executable = bundle / command + if not executable.is_file() or not os.access(executable, os.X_OK): + raise ValueError('release lacks executable ' + command) + output = subprocess.check_output([str(bundle / CONFIG['commands'][0])] + CONFIG.get('version_args', ['--version']), text=True, timeout=20) + if not re.search(r'(?