Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions docs/case-studies/issue-160/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# Case Study — Issue #160: a rate-limited integration probe blocked release

- Issue: https://github.com/link-foundation/start/issues/160
- Pull request: https://github.com/link-foundation/start/pull/161
- Failing JavaScript run: https://github.com/link-foundation/start/actions/runs/31380353470
- Same-revision Rust run: https://github.com/link-foundation/start/actions/runs/31380353804
- Fixed JavaScript run: https://github.com/link-foundation/start/actions/runs/31388354106
- Fixed Rust run: https://github.com/link-foundation/start/actions/runs/31388355823

## Result

The product's multi-network behavior was correct. Its integration test combined
two different claims in one shell command: reach a private Docker alias and get
a successful HTTP response from the unauthenticated GitHub API. The first claim
passed. The second intermittently returned HTTP 403 after the runner's source IP
exhausted GitHub's 60-request hourly unauthenticated quota. BusyBox `wget`
returned a nonzero status, so the container and test failed even though DNS,
routing, TCP, and TLS had all succeeded.

PR #159 removed the public request from both language suites before this issue's
prepared branch was created. It replaced the endpoint with two local sidecars,
added container-log diagnostics, and added opt-in Docker lifecycle tracing. Its
successful main workflow published the previously blocked JavaScript 0.32.0
release.

This PR preserves that hermetic design and closes one remaining coverage gap:
the probe now starts on Docker's default `bridge`, reaches sidecars on two
additional internal networks, and checks that a default route remains present.
It therefore retains all three assertions without depending on any public
service. A fast, daemon-independent invariant in each language rejects any
future HTTP URL in the probe.

## Documents

| File | Purpose |
| -------------------- | -------------------------------------------------------------------------------------- |
| `requirements.md` | Complete requirement inventory and disposition. |
| `timeline.md` | Event and release sequence reconstructed from GitHub metadata. |
| `root-cause.md` | Technical and process root causes, with raw-log line references. |
| `solutions.md` | Alternatives considered, selected design, and verification plan. |
| `online-research.md` | Official documentation and existing components reviewed. |
| `ci-logs/` | Complete pre-fix, fixed-main, and PR-policy JavaScript and Rust workflow logs. |
| `data/` | Issue, PR, review, workflow, release, registry, source, diff, and local-test evidence. |

## Evidence highlights

- The failing log invokes the public probe at
`ci-logs/js-31380353470.log:4654`, reports only `'1' !== '0'` at lines
4663–4677, and names the failed integration test at line 4680.
- That job finishes with 714 passing tests and exactly one failure at lines
5676–5683.
- The fixed main run uses only local aliases at
`ci-logs/js-31388354106.log:4835`, passes the integration test at line 4840,
and publishes `start-command@0.32.0` at line 8195.
- `data/npm-package.json` independently records `latest: 0.32.0` and its
publication timestamp.
- `data/js-focused-test.log` and `data/rust-focused-test.log` record successful
real-Docker verification of the final route-preserving probe.
- The first PR run exposed mandatory release metadata: JavaScript reports zero
changesets at `ci-logs/javascript-pr-31405026454.log:1837`, while Rust reports
one changed source file and no fragment at
`ci-logs/rust-pr-31405026291.log:1616`–1618. Both validated patch fragments
are included in the final branch.

No upstream defect was filed. GitHub's rate limit, GitHub-hosted runner address
model, IANA's best-effort example service, and BusyBox's nonzero response to an
HTTP error all behave as documented or expected. The defect was the test's
choice of an uncontrolled dependency.
2,088 changes: 2,088 additions & 0 deletions docs/case-studies/issue-160/ci-logs/javascript-pr-31405026454.log

Large diffs are not rendered by default.

7,187 changes: 7,187 additions & 0 deletions docs/case-studies/issue-160/ci-logs/js-31380353470.log

Large diffs are not rendered by default.

8,480 changes: 8,480 additions & 0 deletions docs/case-studies/issue-160/ci-logs/js-31388354106.log

Large diffs are not rendered by default.

8,817 changes: 8,817 additions & 0 deletions docs/case-studies/issue-160/ci-logs/rust-31380353804.log

Large diffs are not rendered by default.

9,383 changes: 9,383 additions & 0 deletions docs/case-studies/issue-160/ci-logs/rust-31388355823.log

Large diffs are not rendered by default.

2,628 changes: 2,628 additions & 0 deletions docs/case-studies/issue-160/ci-logs/rust-pr-31405026291.log

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions docs/case-studies/issue-160/data/issue-156-comments.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[]
1 change: 1 addition & 0 deletions docs/case-studies/issue-160/data/issue-156.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"body":"# Docker isolation: allow attaching *additional* networks (`--network` should be repeatable)\n\nFollow-up to [#154](https://github.com/link-foundation/start/issues/154) / [PR #155](https://github.com/link-foundation/start/pull/155), which shipped in [js-0.31.0](https://github.com/link-foundation/start/releases/tag/js-v0.31.0) and [rust-0.18.0](https://github.com/link-foundation/start/releases/tag/rust-v0.18.0). Thank you — the single-network case works exactly as specified.\n\nThe one acceptance item from #154 that did not ship is the multi-network case:\n\n> If multiple networks are supported, create the container on the first network and connect the remaining networks before starting it so the user command cannot race the network setup.\n\nWithout it, `--network` cannot express \"keep normal connectivity **and** join this private network\", which is the shape almost every sidecar use case needs.\n\n## Why the single-network form is not sufficient\n\n`docker run --network <name>` **replaces** the default bridge rather than adding to it. In `js/src/lib/docker-network-options.js` the option is single-valued:\n\n```js\nif (arg === '--network') {\n options.network = args[index + 1]; // last one wins; not accumulated\n}\n```\n\nand `buildDockerRuntimeArgs` (`js/src/lib/isolation.js:522`) emits exactly one `--network`.\n\nSo for a task that must reach both the public internet and a private service:\n\n```bash\ndocker network create --internal my-sidecar-net\n\n# Joins the internal network — and loses its route to github.com / registry.npmjs.org\n$ --isolated docker --image node:22 --network my-sidecar-net -- \\\n sh -c 'curl -sS -o /dev/null -w \"%{http_code}\\n\" https://api.github.com'\n```\n\nThe container resolves the sidecar but cannot reach anything outside, because `--internal` networks have no egress and the bridge is gone. Dropping `--network` gives the opposite failure: egress works, the sidecar is unreachable. There is no single-network answer.\n\n## Current downstream workaround\n\n[link-assistant/hive-mind#2146](https://github.com/link-assistant/hive-mind/issues/2146) needs exactly this: an agent task container that talks to a Formal AI sidecar over a private `--internal` network while still pushing to GitHub. It launches the detached session with no `--network`, holds the task behind its own per-session startup gate, and runs\n\n```bash\ndocker network connect my-sidecar-net \"$SESSION_UUID\"\n```\n\ninside the closed gate before releasing it. That is correct for Hive Mind because the gate already exists for an unrelated reason, but it is downstream lifecycle code that every other caller would have to reinvent, and callers without a gate have a real race: the child command can start before the second network is attached.\n\n## Suggested implementation\n\nMake `--network` repeatable and accumulate into a list, keeping the current behavior for a single value:\n\n- `js/src/lib/docker-network-options.js` — `options.networks.push(value)` instead of `options.network = value`; keep `options.network` as a compatibility accessor for the first entry.\n- `js/src/lib/isolation.js` — emit `--network <first>` in `buildDockerRuntimeArgs` as today. For entries 2..n, use `docker create` + `docker network connect` per extra network + `docker start`, so every attachment completes **before** the child command runs. Docker Engine 25+ also accepts repeated `--network` on `docker run`; if a version probe is acceptable, that path is simpler, but the create/connect/start path works on every supported engine.\n- `--network-alias` should apply to the network it follows on the command line (`--network a --network-alias x --network b --network-alias y`), which is what `docker network connect --alias` supports natively; if that is too subtle, applying all aliases to the first network and documenting it is acceptable.\n- Human-readable isolation status and the JSON execution record should list all networks, not just the first (`isolation.js:552`, `:576`).\n\n## Acceptance tests\n\n- Parser accepts repeated `--network a --network b` and `--network=a --network=b`, and does not consume child-command arguments after `--`.\n- `buildDockerRuntimeArgs` emits the first network inline in a stable position; extra networks appear as connect steps, not as extra `--network` flags on the `run` (unless the engine-version path is chosen).\n- Integration: create one bridge network and one `--internal` network, launch a container attached to both, and assert **in the same run** that it resolves the sidecar alias *and* reaches an external host. This is the test that fails today for every possible single-network invocation.\n- Detached mode does not release the child command until every requested attachment succeeds; a failure to attach network 2 leaves no orphaned running container and exits non-zero.\n- Status output and JSON metadata list all networks and aliases.\n\n## Impact\n\nNot a blocker for Hive Mind — the gate workaround is in production and is described in its case study — but it is the difference between \"Docker isolation can express a private sidecar\" and \"each caller must write container-lifecycle code to express a private sidecar\".\n","closedAt":"2026-08-10T10:43:15Z","createdAt":"2026-08-10T09:45:06Z","number":156,"state":"CLOSED","title":"Docker isolation: allow attaching additional networks (`--network` should be repeatable)","updatedAt":"2026-08-10T10:43:15Z","url":"https://github.com/link-foundation/start/issues/156"}
1 change: 1 addition & 0 deletions docs/case-studies/issue-160/data/issue-158-comments.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[]
1 change: 1 addition & 0 deletions docs/case-studies/issue-160/data/issue-158.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"body":"### Recent CI/CD runs on `main`\n\n| Workflow | Status | Conclusion | Commit | Run |\n| --- | --- | --- | --- | --- |\n| Rust CI/CD | completed | success | `d959632` | [run](https://github.com/link-foundation/start/actions/runs/31380353804) |\n| JavaScript CI/CD | completed | failure | `d959632` | [run](https://github.com/link-foundation/start/actions/runs/31380353470) |\n\nUse all the best practices from CI/CD templates (check full file tree to compare for all GitHub workflow and CI/CD scripts file), if the same issue is found in template report issue also in templates:\n\n- https://github.com/link-foundation/js-ai-driven-development-pipeline-template\n- https://github.com/link-foundation/rust-ai-driven-development-pipeline-template\n\nWe should compare all files, so we don't have more CI/CD errors in the future and reuse all the best practices from these templates.\n\nFollow the CI/CD best practices collected in [https://github.com/link-assistant/hive-mind/blob/main/docs/CI-CD-BEST-PRACTICES.md](https://github.com/link-assistant/hive-mind/blob/main/docs/CI-CD-BEST-PRACTICES.md).\n\nPlease plan and execute everything in this single pull request, you have unlimited time and context, as context auto-compacts and you can continue indefinitely, until it is each and every requirement fully addressed, and everything is totally done.\n\n---\n\n<details>\n<summary>Context collected by <code>/fix --ci-cd</code></summary>\n\n- **Repository:** [link-foundation/start](https://github.com/link-foundation/start)\n- **Default branch:** `main`\n- **Latest commit:** `81a49f8` ([commit](https://github.com/link-foundation/start/commit/81a49f8ca41f171710c3e08d16235b77bfc02a6b)) — rust-v0.19.0\n- **CI/CD runs found:** 2 (1 not passing)\n\n**Detected languages**\n\n- **JavaScript** — 58.2%\n- **Rust** — 40.5%\n- **Shell** — 1.3%\n\n**Recommended CI/CD templates**\n\nApply the best practices from these templates, in priority order (most-used language first):\n\n1. **JavaScript / TypeScript** — [link-foundation/js-ai-driven-development-pipeline-template](https://github.com/link-foundation/js-ai-driven-development-pipeline-template) _(detected: JavaScript)_\n2. **Rust** — [link-foundation/rust-ai-driven-development-pipeline-template](https://github.com/link-foundation/rust-ai-driven-development-pipeline-template) _(detected: Rust)_\n\nOther detected languages without a dedicated template: Shell.\n\n</details>","closedAt":"2026-08-10T12:30:33Z","createdAt":"2026-08-10T11:22:32Z","number":158,"state":"CLOSED","title":"Check for all false positives, false negatives, warnings and errors in CI/CD and fix them all","updatedAt":"2026-08-10T12:30:33Z","url":"https://github.com/link-foundation/start/issues/158"}
26 changes: 26 additions & 0 deletions docs/case-studies/issue-160/data/js-focused-test.log
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
bun test v1.3.14 (0d9b296a)

test/docker-network-integration.js:
(pass) Docker named network integration > keeps the multi-network probe hermetic [1.03ms]
$ ping -c 1 formal-ai && ping -c 1 formal-db && ip route | grep -q '^default '

$ ping -c 1 formal-ai

ping: bad address 'formal-ai'
(pass) Docker named network integration > reaches a private sidecar on each of two networks [4034.64ms]
$ echo should-not-run

$ echo should-not-run

$ echo should-not-run

docker: Error response from daemon: failed to set up container networking: network start-network-914dc7c2-absent not found

Run 'docker run --help' for more information
$ echo should-not-run

(pass) Docker named network integration > fails for a missing network without orphaning a container [4382.70ms]

3 pass
0 fail
Ran 3 tests across 1 file. [10.57s]
1 change: 1 addition & 0 deletions docs/case-studies/issue-160/data/js-run-31380353470.json

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions docs/case-studies/issue-160/data/js-run-31388354106.json

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions docs/case-studies/issue-160/data/js-v0.32.0-release.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"body":"### Minor Changes\n\n- Allow Docker isolation to join multiple networks before starting the command.\n\n Make the Docker multi-network integration test hermetic and add opt-in\n `START_DEBUG` tracing to the Docker network lifecycle helpers.\n\n---\n\n[![npm version](https://img.shields.io/badge/npm-0.32.0-blue.svg)](https://www.npmjs.com/package/start-command/v/0.32.0)","name":"[JavaScript] 0.32.0","publishedAt":"2026-08-10T12:32:47Z","tagName":"js-v0.32.0","targetCommitish":"main","url":"https://github.com/link-foundation/start/releases/tag/js-v0.32.0"}
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Found 1 changeset file(s)
Validating changeset: js/.changeset/issue-160.md
✅ Changeset validation passed
Type: patch
Description: Keep Docker multi-network regression coverage hermetic while verifying the default bridge route.
1 change: 1 addition & 0 deletions docs/case-studies/issue-160/data/local-doc-js.log
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Checked 4 documented example(s) for js.
1 change: 1 addition & 0 deletions docs/case-studies/issue-160/data/local-doc-rust.log
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Checked 4 documented example(s) for rust.
5 changes: 5 additions & 0 deletions docs/case-studies/issue-160/data/local-js-filesize.log
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@

Checking JavaScript files for maximum 1000 lines...

✓ All files are within the line limit

3 changes: 3 additions & 0 deletions docs/case-studies/issue-160/data/local-js-format.log
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
$ prettier --check .
Checking formatting...
All matched files use Prettier code style!
Loading
Loading