diff --git a/.github/workflows/install.yml b/.github/workflows/install.yml new file mode 100644 index 0000000..061eb67 --- /dev/null +++ b/.github/workflows/install.yml @@ -0,0 +1,19 @@ +name: Installer controls +on: + pull_request: + push: + branches: [main] +permissions: + contents: read +jobs: + installer: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: + python-version: '3.11' + - name: Validate standalone entrypoint and refusal controls + run: | + sh -n install.sh + python3 scripts/test-install-release.py diff --git a/README.md b/README.md index 9467638..bd6e859 100644 --- a/README.md +++ b/README.md @@ -18,57 +18,22 @@ fewer repeated tool-call failures. It stores calls locally, ranks recurring failures, and records mappings from mistaken tool names to real ones.** The name comes from the trails people wear through grass: evidence of where a path is needed. -## Why you would want it - -- Find repeated failures across sessions instead of reading each transcript. -- Compare mistaken tool names with known tools and record the intended mapping. -- Use the failure history to improve instructions, tool names, or integrations. - -Read [how the workflow fits together](https://scbrown.github.io/desire-path/introduction.html). - ## Install -**Debian and Ubuntu (amd64 and arm64):** install from the signed apt repository, which -republishes each checksum-verified release. The package puts `dp` in -`/usr/bin`. Check the signing key's fingerprint before trusting it; the -commands stop if it does not match: - -```bash -curl -fsSL https://scbrown.github.io/apt/key.gpg -o scbrown-stack.gpg \ - && gpg --show-keys --with-colons scbrown-stack.gpg \ - | grep -qx 'fpr:::::::::AB5625C5612F643BF76464DF0DC99924994EC959:' \ - && sudo install -D -m 0644 scbrown-stack.gpg /etc/apt/keyrings/scbrown-stack.gpg \ - && echo "deb [signed-by=/etc/apt/keyrings/scbrown-stack.gpg] https://scbrown.github.io/apt stable main" \ - | sudo tee /etc/apt/sources.list.d/scbrown-stack.list \ - && sudo apt update && sudo apt install desire-path +```sh +curl -fsSL https://raw.githubusercontent.com/scbrown/desire-path/main/install.sh | sh ``` -`apt upgrade` then picks up new releases. unattended-upgrades only installs -from origins it allows, so to have it upgrade desire-path too, add -`"origin=scbrown,codename=stable";` to `Unattended-Upgrade::Origins-Pattern`. -Channels, pinning and rollback: -[scbrown/apt](https://github.com/scbrown/apt#channels-and-pinning). +Python 3.9+, curl, and a Linux or macOS shell. Downloads the latest published archive, verifies SHA-256 and the executable +version before publishing command links in `~/.local/bin`. Add it to `PATH`. +The complete versioned bundle stays in `~/.local/lib/desire-path/TAG` for rollback. +Published targets: Linux and macOS, x86_64 and arm64. -Linux x86-64, pinned release with checksum verification: +Set `STACK_INSTALL_ROOT` on the receiving shell for another prefix. +Pinned releases, alternatives, configuration and removal are in the +[installation and usage reference](docs/installation-reference.md). -```bash -mkdir -p /tmp/dp-install && cd /tmp/dp-install -curl -fLO https://github.com/scbrown/desire-path/releases/download/v0.7.0/desire-path_0.7.0_linux_amd64.tar.gz -curl -fLO https://github.com/scbrown/desire-path/releases/download/v0.7.0/checksums.txt -sha256sum --check --ignore-missing checksums.txt -tar -xzf desire-path_0.7.0_linux_amd64.tar.gz dp -mkdir -p "$HOME/.local/bin" && install -m 755 dp "$HOME/.local/bin/dp" -export PATH="$HOME/.local/bin:$PATH" -dp version -``` - -Expected: `dp 0.7.0 (5ca33ba)`. If you see something else, check `command -v dp`. - -From source (Go 1.24+): `go install github.com/scbrown/desire-path/cmd/dp@v0.7.0`, -then add `$(go env GOPATH)/bin` to PATH and run `dp version`. -See [installation](https://scbrown.github.io/desire-path/getting-started.html) for macOS, Windows, and source-checkout builds. - -## First success in three commands +## Try it Use a temporary database and one synthetic failed call; Python 3 formats the result: @@ -87,56 +52,15 @@ read_file 1 The failure became a queryable pattern without configuring an agent or touching your usual database. An optional-metrics diagnostic may also appear on stderr; see [troubleshooting](https://scbrown.github.io/desire-path/getting-started.html#troubleshooting). -## On your own data - -After your agent has recorded calls: - -| question | command | -|---|---| -| Which failures repeat? | `dp paths --top 5` | -| What happened for this tool? | `dp inspect read_file` | -| What tool might have been intended? | `dp similar read_file` | -| How do I record the intended mapping? | `dp alias read_file Read` | -| How do I export the failures? | `dp export --format json` | - -An alias records a mapping. Active interception is a separate [pave setup](https://scbrown.github.io/desire-path/commands/pave.html). -See the [CLI reference](https://scbrown.github.io/desire-path/commands/) for flags and all commands. - -## Wire it into your agent - -For Claude Code, with `dp` on the agent's PATH: - -```bash -dp init --source claude-code -dp sources -``` - -The installer merges hooks into `~/.claude/settings.json`. It installs ingestion -for successes and failures, plus signposting and correction hooks. Review that -file after installation. [Using agents](https://scbrown.github.io/desire-path/integrations/) -explains the installed hooks and the Codex, Cursor, and Kiro source plugins. - -## Before you start - -| requirement | support | -|---|---| -| Release archives | Linux, macOS, Windows; amd64 and arm64 | -| Source build | Go 1.24+; no CGo required | -| README demo | Bash-compatible shell and Python 3 | -| Local storage | SQLite at `~/.dp/desires.db`; override with `--db` | -| Hook capture | A supported agent and `dp` on its PATH | - -Keep tool inputs and error messages in mind when sharing [exports](https://scbrown.github.io/desire-path/commands/export.html). - -## What's next +## Next steps -- [Read the book](https://scbrown.github.io/desire-path/introduction.html) -- [Find every design and reference document](https://scbrown.github.io/desire-path/docs-map.html) -- [Configure capture for your agent](https://scbrown.github.io/desire-path/integrations/) +- [Tambo](https://github.com/scbrown/tambo): guided stack installation and verification. +- [Camayoc](https://github.com/scbrown/camayoc): governed memory and vocabulary. +- [Detailed examples and agent setup](docs/installation-reference.md). -## 🧺 The stack +**Load this repository’s graph:** public repository-pendant assets have not yet been verified. When a release supplies the archive, checksum and provenance, follow the [verified import workflow](https://scbrown.github.io/quipu/sharing/index.html): extract the share directory and use `quipu import SHARE_DIR --db repository.db`. Import stages knowledge; ROOT stays unchanged. -Caboodle installs these together and proves each one works; every tool also stands alone. +## The stack @@ -152,8 +76,8 @@ when. | [camayoc](https://github.com/scbrown/camayoc) | prepares knowledge and vocabulary for Quipu, with checks before it is admitted | | [quechua](https://github.com/scbrown/quechua) | shared vocabulary and shapes for interoperable knowledge | | [chaski](https://github.com/scbrown/chaski) | reacts to graph changes and delivers governed notifications | -| [bobbin](https://github.com/scbrown/bobbin) | indexes your codebase and injects context from that index and Quipu knowledge | -| [yupana](https://github.com/scbrown/yupana) | shows code impact and can read policies from Quipu to govern agent actions | +| [bobbin](https://github.com/scbrown/bobbin) | searches your code and can include related knowledge from Quipu | +| [yupana](https://github.com/scbrown/yupana) | shows code impact and can read policies from Quipu to check changes | | [desire-path](https://github.com/scbrown/desire-path) | turns agent mistakes into feature requests; works on its own without Quipu | | [seeds](https://github.com/scbrown/seeds) | a work item tracker backed by the Quipu knowledge base | | [shantytown](https://github.com/scbrown/shantytown) | runs crews of AI agents on work items, triggered by hand, on a schedule or by events; can share work and memory through Quipu | diff --git a/docs/installation-reference.md b/docs/installation-reference.md new file mode 100644 index 0000000..19843b4 --- /dev/null +++ b/docs/installation-reference.md @@ -0,0 +1,174 @@ +

+ + + + Desire Path + +

+

Desire Path

+

👣 Find the tool calls your agents get wrong, and turn repeats into fixes.

+

+ License: MIT + CI + Quipu stack +

+ +**Desire Path (`dp`) is a CLI and a set of agent hooks for developers who want +fewer repeated tool-call failures. It stores calls locally, ranks recurring +failures, and records mappings from mistaken tool names to real ones.** +The name comes from the trails people wear through grass: evidence of where a path is needed. + +## Why you would want it + +- Find repeated failures across sessions instead of reading each transcript. +- Compare mistaken tool names with known tools and record the intended mapping. +- Use the failure history to improve instructions, tool names, or integrations. + +Read [how the workflow fits together](https://scbrown.github.io/desire-path/introduction.html). + +## Install + +**Debian and Ubuntu (amd64 and arm64):** install from the signed apt repository, which +republishes each checksum-verified release. The package puts `dp` in +`/usr/bin`. Check the signing key's fingerprint before trusting it; the +commands stop if it does not match: + +```bash +curl -fsSL https://scbrown.github.io/apt/key.gpg -o scbrown-stack.gpg \ + && gpg --show-keys --with-colons scbrown-stack.gpg \ + | grep -qx 'fpr:::::::::AB5625C5612F643BF76464DF0DC99924994EC959:' \ + && sudo install -D -m 0644 scbrown-stack.gpg /etc/apt/keyrings/scbrown-stack.gpg \ + && echo "deb [signed-by=/etc/apt/keyrings/scbrown-stack.gpg] https://scbrown.github.io/apt stable main" \ + | sudo tee /etc/apt/sources.list.d/scbrown-stack.list \ + && sudo apt update && sudo apt install desire-path +``` + +`apt upgrade` then picks up new releases. unattended-upgrades only installs +from origins it allows, so to have it upgrade desire-path too, add +`"origin=scbrown,codename=stable";` to `Unattended-Upgrade::Origins-Pattern`. +Channels, pinning and rollback: +[scbrown/apt](https://github.com/scbrown/apt#channels-and-pinning). + +Linux x86-64, pinned release with checksum verification: + +```bash +mkdir -p /tmp/dp-install && cd /tmp/dp-install +curl -fLO https://github.com/scbrown/desire-path/releases/download/v0.7.0/desire-path_0.7.0_linux_amd64.tar.gz +curl -fLO https://github.com/scbrown/desire-path/releases/download/v0.7.0/checksums.txt +sha256sum --check --ignore-missing checksums.txt +tar -xzf desire-path_0.7.0_linux_amd64.tar.gz dp +mkdir -p "$HOME/.local/bin" && install -m 755 dp "$HOME/.local/bin/dp" +export PATH="$HOME/.local/bin:$PATH" +dp version +``` + +Expected: `dp 0.7.0 (5ca33ba)`. If you see something else, check `command -v dp`. + +From source (Go 1.24+): `go install github.com/scbrown/desire-path/cmd/dp@v0.7.0`, +then add `$(go env GOPATH)/bin` to PATH and run `dp version`. +See [installation](https://scbrown.github.io/desire-path/getting-started.html) for macOS, Windows, and source-checkout builds. + +## First success in three commands + +Use a temporary database and one synthetic failed call; Python 3 formats the result: + +```bash +DP_DEMO=$(mktemp -d) +printf '%s\n' '{"tool_name":"read_file","error":"unknown tool","session_id":"demo"}' | dp --db "$DP_DEMO/desires.db" ingest --source claude-code --json > "$DP_DEMO/record.json" +dp --db "$DP_DEMO/desires.db" paths --json | python3 -c 'import json,sys; p=json.load(sys.stdin)[0]; print(p["pattern"], p["count"])' +``` + +Expected stdout: + +```text +read_file 1 +``` + +The failure became a queryable pattern without configuring an agent or touching your usual database. +An optional-metrics diagnostic may also appear on stderr; see [troubleshooting](https://scbrown.github.io/desire-path/getting-started.html#troubleshooting). + +## On your own data + +After your agent has recorded calls: + +| question | command | +|---|---| +| Which failures repeat? | `dp paths --top 5` | +| What happened for this tool? | `dp inspect read_file` | +| What tool might have been intended? | `dp similar read_file` | +| How do I record the intended mapping? | `dp alias read_file Read` | +| How do I export the failures? | `dp export --format json` | + +An alias records a mapping. Active interception is a separate [pave setup](https://scbrown.github.io/desire-path/commands/pave.html). +See the [CLI reference](https://scbrown.github.io/desire-path/commands/) for flags and all commands. + +## Wire it into your agent + +For Claude Code, with `dp` on the agent's PATH: + +```bash +dp init --source claude-code +dp sources +``` + +The installer merges hooks into `~/.claude/settings.json`. It installs ingestion +for successes and failures, plus signposting and correction hooks. Review that +file after installation. [Using agents](https://scbrown.github.io/desire-path/integrations/) +explains the installed hooks and the Codex, Cursor, and Kiro source plugins. + +## Before you start + +| requirement | support | +|---|---| +| Release archives | Linux, macOS, Windows; amd64 and arm64 | +| Source build | Go 1.24+; no CGo required | +| README demo | Bash-compatible shell and Python 3 | +| Local storage | SQLite at `~/.dp/desires.db`; override with `--db` | +| Hook capture | A supported agent and `dp` on its PATH | + +Keep tool inputs and error messages in mind when sharing [exports](https://scbrown.github.io/desire-path/commands/export.html). + +## What's next + +- [Read the book](https://scbrown.github.io/desire-path/introduction.html) +- [Find every design and reference document](https://scbrown.github.io/desire-path/docs-map.html) +- [Configure capture for your agent](https://scbrown.github.io/desire-path/integrations/) + +## 🧺 The stack + +Caboodle installs these together and proves each one works; every tool also stands alone. + +## Contributing + +From a checkout, with [just](https://github.com/casey/just), Go, Python 3, +pre-commit, and mdBook installed: + +```bash +just build +just test +just check +``` + +See the [development guide](https://github.com/scbrown/desire-path/blob/main/book/src/development/contributing.md). + +## 📜 License + +[MIT](https://github.com/scbrown/desire-path/blob/main/LICENSE). + +Repeated failures can be measured with `dp failure-families --since --until --window 1h --max-records 10000 --json`. It groups by tool, parsed shell command-program shape and a bounded error class; compound commands are grouped as programs, without claiming which subcommand failed. Argument values, paths and error messages are absent from output. Unsupported wrappers and dynamic command identities are counted separately as UNKNOWN. Synthetic turn-pattern observations are excluded. `peak_window` measures the historical rolling maximum; `current_window` measures the window ending at `--until`. These are observations, not an alerting policy or a claim of complete capture. + +`--max-records` defaults to 10000 and accepts 1 through 100000. The store query +loads at most the cap plus one sentinel; overflow returns an error without partial +counts. Future records after `--until` conservatively consume this candidate cap +because the existing store selector bounds only `--since`. JSON also identifies +the exact `since`/`until`, `max_records`, `input_records` and `complete: true`. +Complete means the candidate inventory fit the cap, not that normalization or +failure capture was complete: consumers must still inspect `unknown` and +`excluded`. A record cap bounds materialized rows, not total input bytes or query +latency. Prefer one current window for scheduled threshold evaluation; a historical +report requires its own bounded read-cost admission. + +Local `failure-families` reads require an existing database at the current schema +version. The command opens it read-only: missing databases and schema mismatches +return an error without creating a store or migrating it. Configured remote +stores continue to use their existing read API. diff --git a/docs/knowledge/repository-knowledge.json b/docs/knowledge/repository-knowledge.json index 8567872..17dda11 100644 --- a/docs/knowledge/repository-knowledge.json +++ b/docs/knowledge/repository-knowledge.json @@ -8,7 +8,7 @@ "source": "README.md", "subject": "http://aegis.gastown.local/ontology/doc/desire-path/README.md", "summary": "Desire Path records tool-call failures, groups repeated desires and records mappings to real tool names. An alias records a mapping; active interception has separate setup.", - "source_sha256": "eb133a1842d47e1e64c5e6340479c0f39ec2bf58ef2f0ba3e5784e7082370d1e", + "source_sha256": "02ecab7b213451577521843dabc159c85a7e6bcd6e88f5d1fe0cbdaf82a4c51e", "type": "Concept" }, { @@ -83,7 +83,7 @@ "source": "README.md", "subject": "http://aegis.gastown.local/ontology/doc/desire-path/README.md", "summary": "Start with the temporary-database failed-call example before installing agent hooks or sharing exports.", - "source_sha256": "eb133a1842d47e1e64c5e6340479c0f39ec2bf58ef2f0ba3e5784e7082370d1e", + "source_sha256": "02ecab7b213451577521843dabc159c85a7e6bcd6e88f5d1fe0cbdaf82a4c51e", "properties": { "startHereOrder": 1, "glossaryTerm": "https://scbrown.github.io/quechua/ns#Tool" diff --git a/install.sh b/install.sh new file mode 100755 index 0000000..d9e56df --- /dev/null +++ b/install.sh @@ -0,0 +1,187 @@ +#!/bin/sh +# Install verified published bytes; never build the invoking checkout. +set -eu +exec python3 - "$@" <<'PYINSTALL' +import hashlib +import json +import os +from pathlib import Path, PurePosixPath +import platform +import re +import shutil +import subprocess +import sys +import tarfile +import tempfile +import time +from urllib.error import HTTPError, URLError +from urllib.request import Request, urlopen + +CONFIG = {'archive': 'desire-path_{version}_{target}.tar.gz', 'checksum': 'checksums.txt', 'directory': '.', 'commands': ['dp'], 'targets': {'Linux:x86_64': 'linux_amd64', 'Linux:aarch64': 'linux_arm64', 'Darwin:x86_64': 'darwin_amd64', 'Darwin:arm64': 'darwin_arm64'}, 'repo': 'desire-path', 'version_args': ['version']} + + +def fetch(url, destination): + for attempt in range(3): + try: + with urlopen(Request(url, headers={'User-Agent': CONFIG['repo'] + '-installer'}), timeout=60) as response: + with destination.open('wb') as output: + shutil.copyfileobj(response, output) + return + except (HTTPError, URLError, TimeoutError) as error: + transient = not isinstance(error, HTTPError) or error.code in (403, 429, 500, 502, 503, 504) + if attempt == 2 or not transient: + raise + time.sleep(attempt + 1) + + +def extract(archive, destination): + with tarfile.open(archive, 'r:gz') as bundle: + seen, links = set(), [] + total = 0 + for member in bundle: + name = PurePosixPath(member.name) + if name.is_absolute() or '..' in name.parts or str(name) in seen: + raise ValueError('unsafe or duplicate archive path') + seen.add(str(name)) + target = destination.joinpath(*name.parts) + target.parent.mkdir(parents=True, exist_ok=True) + if member.isdir(): + target.mkdir(exist_ok=True) + elif member.isfile(): + total += member.size + if total > 1024 * 1024 * 1024: + raise ValueError('release exceeds 1 GiB unpacked limit') + source = bundle.extractfile(member) + if source is None: + raise ValueError('missing archive bytes') + with source, target.open('xb') as output: + shutil.copyfileobj(source, output) + target.chmod(0o755 if member.mode & 0o111 else 0o644) + elif member.issym(): + link = PurePosixPath(member.linkname) + if link.is_absolute() or '..' in link.parts: + raise ValueError('unsafe archive link') + links.append((target, member.linkname)) + else: + raise ValueError('unsupported archive member') + for target, link in links: + target.symlink_to(link) + target.resolve(strict=True).relative_to(destination.resolve()) + + +def bundle_identity(root): + result = {} + for path in root.rglob('*'): + relative = str(path.relative_to(root)) + if relative == '.release.json': + continue + if path.is_symlink(): + result[relative] = ('link', os.readlink(path)) + elif path.is_file(): + result[relative] = ('file', path.stat().st_mode & 0o777, + hashlib.sha256(path.read_bytes()).hexdigest()) + elif path.is_dir(): + result[relative] = ('directory',) + else: + raise ValueError('unexpected installed bundle entry') + return result + + +def install(): + if len(sys.argv) > 1: + raise ValueError('configure STACK_INSTALL_ROOT and STACK_VERSION through the environment; no positional arguments') + system, machine = platform.system(), platform.machine() + target = CONFIG['targets'].get(system + ':' + machine) + if target is None: + raise ValueError('no published archive for ' + system + ' ' + machine) + with tempfile.TemporaryDirectory(prefix=CONFIG['repo'] + '-release-') as temporary: + work = Path(temporary) + tag = os.environ.get('STACK_VERSION') + if not tag: + metadata = work / 'latest.json' + fetch('https://api.github.com/repos/scbrown/' + CONFIG['repo'] + '/releases/latest', metadata) + release = json.loads(metadata.read_text()) + if release.get('draft') or release.get('prerelease'): + raise ValueError('latest release is not stable') + tag = release['tag_name'] + if not re.fullmatch(r'[A-Za-z0-9][A-Za-z0-9._-]{0,100}', tag): + raise ValueError('invalid release tag') + match = re.search(r'(?:^|v)([0-9]+\.[0-9]+\.[0-9]+)$', tag) + if not match: + raise ValueError('unsupported release version') + version = match.group(1) + name = CONFIG['archive'].format(tag=tag, version=version, target=target) + checksum = CONFIG['checksum'].format(archive=name) + base = os.environ.get('STACK_RELEASE_BASE_URL', 'https://github.com/scbrown/' + CONFIG['repo'] + '/releases/download/' + tag) + archive, manifest = work / name, work / 'checksums' + fetch(base + '/' + name, archive) + fetch(base + '/' + checksum, manifest) + matches = [] + for line in manifest.read_text().splitlines(): + fields = line.split() + if len(fields) == 2 and fields[1].lstrip('*') == name: + matches.append(fields[0].lower()) + if len(matches) != 1 or not re.fullmatch('[0-9a-f]{64}', matches[0]): + raise ValueError('missing or ambiguous checksum entry for ' + name) + digest = hashlib.sha256(archive.read_bytes()).hexdigest() + if digest != matches[0]: + raise ValueError('release checksum mismatch') + unpacked = work / 'unpacked' + unpacked.mkdir() + extract(archive, unpacked) + bundle = unpacked / CONFIG['directory'].format(archive=name.removesuffix('.tar.gz')) + for command in CONFIG['commands']: + executable = bundle / command + if not executable.is_file() or not os.access(executable, os.X_OK): + raise ValueError('release lacks executable ' + command) + output = subprocess.check_output([str(bundle / CONFIG['commands'][0])] + CONFIG.get('version_args', ['--version']), text=True, timeout=20) + if not re.search(r'(?