fix(ci): eliminate CI/CD false positives, false negatives, warnings and errors by konard · Pull Request #159 · link-foundation/start

konard · 2026-08-10T11:23:27Z

Fixes #158.

What was wrong

Run 31380353470 (JS CI/CD on d959632) was red, and several gates that looked green could never have gone red.

Class	Problem	Root cause
Error	Docker network integration test failed	The test asserted reachability with `wget -q --spider https://api.github.com`. GitHub allows 60 unauthenticated API requests/hour per IP and hosted runners share egress IPs, so the probe returned `403 rate limit exceeded`. The `ping -c 1 formal-ai` half always passed — the product feature was never broken. The same probe existed in `rust/tests/docker_network.rs:80`.
False negative	Coverage threshold never enforced	`grep -oP '\d+\.\d+(?=%)'` never matched, because Bun's totals row has no `%` (`All files \| 92.60 \| 88.09 \|`). `parseFloat('')` → `NaN` → "Could not determine coverage, skipping check" → exit 0.
False negative	Failing tests reported as success	The coverage step ended in `\| tee coverage.txt \|\| true`.
False negative	A timed-out job hid a red run	GitHub reports a `timeout-minutes` kill as `cancelled`, and there was no aggregate status job.
Warning	15× `hint: Using 'master' as the name for the initial branch`	`actions/checkout` with no `init.defaultBranch` configured.
Risk	Releases could be cancelled or could race	Workflow-level `concurrency` with `cancel-in-progress` also cancels a release that already started pushing; JS and Rust release jobs used unrelated groups.

What changed

Tests made hermetic (both languages). js/test/docker-network-integration.js and rust/tests/docker_network.rs now create a second local --internal network with a formal-db-aliased sidecar and assert ping -c 1 formal-ai && ping -c 1 formal-db. This still proves two-network attachment, with no external dependency. docker logs output is folded into the assertion messages.

Coverage gate made real. New scripts/check-js-coverage.mjs parses the All files row by column and treats an unparsable report as a hard failure, not a skip. The test step runs with set -o pipefail and no || true.

Workflow hardening (all four workflows): workflow-level concurrency removed in favour of per-job cancellable check-* groups; every job that writes to main shares one non-cancellable main-writer-${{ github.repository }}-main group across both workflow files; permissions: contents: read by default; timeout-minutes on every job; always() → !cancelled() outside the status job; GIT_CONFIG_* to silence the git-init hints; a pipeline-status aggregate gate that fails on failure and on cancelled when on main.

New workflows from the templates: security.yml (CodeQL for javascript-typescript/actions/rust, dependency-review-action@v5, secretlint) and links.yml (lychee with a Wayback Machine fallback and .lycheeignore).

Verbose mode, default OFF. START_DEBUG=1 now traces the whole docker create → docker network connect → docker start sequence — command, exit status, stdout, stderr — in both js/src/lib/docker-network-lifecycle.js and rust/src/lib/docker_network_lifecycle.rs, matching the existing START_DEBUG convention.

Tests

Test	Guards
`js/test/check-js-coverage.mjs` (7)	the coverage parser, incl. "unparsable ⇒ fail, not skip"
`js/test/ci-workflow-invariants.js` (12)	timeouts, per-job concurrency, writer serialisation, `!cancelled()`, `pipeline-status` completeness, git hints, `\|\| true`, script existence
`rust/tests/ci_workflow_invariants.rs` (17)	the same, plus `set -o pipefail`, the parity gate, and the security/link workflows
`js/test/docker-network-lifecycle.js` (5) + `rust/src/lib/docker_network_lifecycle.rs` (5)	network-list resolution, verbose-off-by-default

All pass locally, along with bun run lint, bun run format:check, cargo fmt --check, cargo clippy -D warnings, check-file-size.mjs and check-test-parity.mjs (Rust 698 / JS 754 = 92.1 %).

Upstream report

.jscpd.json in the JS pipeline template sets "format": "console", but in jscpd format is the list of languages to analyze (output is reporters), so jscpd analyzes zero files and the duplication gate can never fail. Reported with a minimal reproduction, workaround and code fix: link-foundation/js-ai-driven-development-pipeline-template#126. The Rust template has no .jscpd.json, and its only workflow-level concurrency block cancels on pull_request only, so no report was warranted there.

Analysis

Evidence, timeline, per-requirement root causes and the library survey are in dev/log/issues/158/pulls/159/ANALYSIS.md, alongside the downloaded run logs.

Adding .gitkeep for PR creation (default mode). This file will be removed when the task is complete. Issue: #158

…us gate

…ck workflows

…t hermetic - rust/tests/docker_network.rs no longer probes https://api.github.com; it pings a second local --internal network sidecar instead, so the shared runner IP hitting the 60 req/hour unauthenticated GitHub API limit can no longer fail the suite (same root cause as the JS test, issue #158) - docker logs are folded into the exit-code assertion message - new rust/tests/ci_workflow_invariants.rs (17 tests) and js/test/ci-workflow-invariants.js (12 tests) guard timeouts, per-job concurrency, writer serialisation, !cancelled(), pipeline-status coverage, git-init hints, pipefail and the coverage helper

Adds verbose logging (default OFF) around the docker create -> network connect -> start sequence in both the JS and Rust lifecycle helpers, so the next failure carries the command, exit status, stdout and stderr instead of only an exit code (issue #158). Covered by 5 JS and 5 Rust unit tests, including one asserting the default-off behaviour.

github-advanced-security · 2026-08-10T12:09:18Z

You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool.

What Enabling Code Scanning Means:

The 'Security' tab will display more code scanning analysis results (e.g., for the default branch).
Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results.
You will be able to see the analysis results for the pull request's branch on this overview once the scans have completed and the checks have passed.

For more information about GitHub Code Scanning, check out the documentation.

…cy graph is off The dependency graph is disabled for this repository (the compare endpoint returns 403), so actions/dependency-review-action@v5 failed with "Dependency review is not supported on this repository" — a configuration gap reported as a red gate. The job now probes the endpoint and emits an actionable ::warning:: instead, rather than hiding the gate behind continue-on-error. Once the setting is enabled the gate becomes enforcing again with no workflow change.

konard · 2026-08-10T12:27:05Z

All four workflows are green on a2c9373: JavaScript CI/CD, Rust CI/CD, Security, Broken Link Checker.

Two things worth flagging for a maintainer:

The dependency graph is disabled for this repository. actions/dependency-review-action@v5 failed with Dependency review is not supported on this repository, and GET /repos/link-foundation/start/dependency-graph/compare/... returns 403 Forbidden — this is an owner-level setting that cannot be changed from a workflow or via the REST API. The job now probes that endpoint and emits an actionable ::warning:: instead of a misleading red gate; enabling Settings → Code security → Dependency graph makes it enforcing again with no workflow change. I did not flip the setting myself since it is outside this PR.
The new pipeline-status gate already proved itself. The first Rust run on this commit came back cancelled with every real job green — the Pipeline Status runner was lost mid-step (no log blob was ever uploaded, and the job ran 10 minutes against a 5-minute timeout). Before this PR that shape of failure produced no red signal at all. Re-running the job succeeded in 7 seconds, so it was infrastructure, not the script — but it is exactly the class of silent failure the issue asked to eliminate.

konard · 2026-08-10T12:27:58Z

🤖 Solution Draft Log

This log file contains the complete execution trace of the AI solution draft process.

💰 Cost: $12.743766

📊 Context and tokens usage:

Claude Opus 5: (3 sub-sessions)

116.5K / 1M (12%) input tokens, 19.5K / 128K (15%) output tokens
116.1K / 1M (12%) input tokens, 36K / 128K (28%) output tokens
96.3K / 1M (10%) input tokens, 32.4K / 128K (25%) output tokens

Total: (8.1K new + 286.9K cache writes + 14.0M cache reads) input tokens, 112.4K output tokens, $12.741757 cost

Claude Haiku 4.5:

1.5K / 200K (1%) input tokens, 105 / 64K (0%) output tokens

Total: 1.5K input tokens, 105 output tokens, $0.002009 cost

🤖 Models used:

Tool: Anthropic Claude Code
Requested: opus (claude-opus-5)
Thinking level: off (disabled)
Main model: Claude Opus 5 (claude-opus-5)
Additional models:
- Claude Haiku 4.5 (claude-haiku-4-5-20251001)

📎 Log file uploaded as Gist (4890KB)

View complete solution draft log

Now working session is ended, feel free to review and add any feedback on the solution draft.

konard · 2026-08-10T12:30:33Z

🎉 Auto-merged

This pull request has been automatically merged by hive-mind.

All CI checks have passed

Auto-merged by hive-mind with --auto-merge flag

Initial commit with task details

41d4295

Adding .gitkeep for PR creation (default mode). This file will be removed when the task is complete. Issue: #158

konard self-assigned this Aug 10, 2026

konard and others added 7 commits August 10, 2026 11:46

fix(ci): make docker network test hermetic and enforce JS coverage gate

c9efac0

ci: job-level concurrency, least-privilege permissions, pipeline-stat…

32388f1

…us gate

ci: add security (CodeQL, dependency review, secretlint) and link-che…

aaf5dc0

…ck workflows

chore: add changeset and changelog fragment for issue #158

633bf2d

docs(dev): deep analysis of issue #158 (timeline, root causes, fixes)

5daf94c

konard changed the title ~~[WIP] Check for all false positives, false negatives, warnings and errors in CI/CD and fix them all~~ fix(ci): eliminate CI/CD false positives, false negatives, warnings and errors Aug 10, 2026

konard marked this pull request as ready for review August 10, 2026 12:27

konard merged commit e8ec5e5 into main Aug 10, 2026
49 of 50 checks passed

konard mentioned this pull request Aug 10, 2026

test(docker): keep network probes hermetic #161

Merged

Navigation Menu

Uh oh!

fix(ci): eliminate CI/CD false positives, false negatives, warnings and errors - #159

fix(ci): eliminate CI/CD false positives, false negatives, warnings and errors#159
konard merged 9 commits into
mainfrom
issue-158-ecf4c4dc2e55

konard commented Aug 10, 2026 •

edited

Loading

Uh oh!

github-advanced-security AI commented Aug 10, 2026

Uh oh!

konard commented Aug 10, 2026

Uh oh!

konard commented Aug 10, 2026

Uh oh!

Uh oh!

konard commented Aug 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

konard commented Aug 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What was wrong

What changed

Tests

Upstream report

Analysis

Uh oh!

github-advanced-security AI commented Aug 10, 2026

What Enabling Code Scanning Means:

Uh oh!

konard commented Aug 10, 2026

Uh oh!

konard commented Aug 10, 2026

🤖 Solution Draft Log

💰 Cost: $12.743766

📊 Context and tokens usage:

🤖 Models used:

📎 Log file uploaded as Gist (4890KB)

Uh oh!

Uh oh!

konard commented Aug 10, 2026

🎉 Auto-merged

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

konard commented Aug 10, 2026 •

edited

Loading