From 494720fe9d61d4608dc25167d393c51c9eb407a8 Mon Sep 17 00:00:00 2001 From: Claude Opus 5 Date: Mon, 3 Aug 2026 11:53:27 +0200 Subject: [PATCH 1/2] fix(upstream-docs): the watcher cannot push to a protected main The weekly run ended red on its first outing. It fetches the watched vendor documents, rewrites upstream-docs-state.json, and pushed the result straight to main -- which is protected, so the push came back "GH006: Protected branch update failed... 5 of 5 required status checks are expected". The baseline could never advance, and it would have failed the same way on the week a document actually moved, which is the week the tool exists for. The baseline now moves the way everything else here moves: as a pull request on one branch, force-pushed so the newest proposal replaces the previous one rather than stacking on it -- the baseline is a whole snapshot, not a delta. [skip ci] is gone from the commit message, because the required checks are now the point. A pull request opened with GITHUB_TOKEN starts no workflow run, so the step dispatches validate against the branch head; workflow_dispatch is the documented exception to that rule, and validate already accepts it. That run also had nothing to say: zero alerts, and 31 insertions and 31 deletions in a 31-line file. upstream-docs-state.json was the only file in the repository committed with CRLF, written by a Windows checkout through write_text's text mode. Every Linux run rewrote all 31 lines, so the watcher would have proposed a baseline every single week whether or not a document changed. The writer now pins LF, the file is normalized, and .gitattributes keeps the whole class shut -- the same translation would land in index.yaml, devices.yaml and support-status.json, which CI compares with git diff --exit-code, and in the Lua that gets hashed into the signed artifacts. Two tests hold the line: what the tool writes, and what is committed. The second is the one that catches this in CI, since on Linux the first passes even against the old code. Signed-off-by: HuggeK <48095810+HuggeK@users.noreply.github.com> Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com> --- .gitattributes | 11 ++++ .github/workflows/watch-upstream-docs.yml | 52 +++++++++++++++++-- tests/test_upstream_docs_watch.py | 18 +++++++ tools/check_upstream_docs.py | 7 ++- upstream-docs-state.json | 62 +++++++++++------------ 5 files changed, 113 insertions(+), 37 deletions(-) create mode 100644 .gitattributes diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..be1aefa --- /dev/null +++ b/.gitattributes @@ -0,0 +1,11 @@ +# The catalog files here are written by a tool and compared back byte for byte +# -- `git diff --exit-code -- index.yaml`, and the watcher's committed baseline. +# A checkout that hands a generator CRLF makes those comparisons fail on line +# endings alone, on a tree where nothing actually changed. Keep them LF in the +# working tree on every platform, so regenerating anywhere reads the same. +*.json text eol=lf +*.yaml text eol=lf +*.md text eol=lf +*.lua text eol=lf +*.py text eol=lf +*.sh text eol=lf diff --git a/.github/workflows/watch-upstream-docs.yml b/.github/workflows/watch-upstream-docs.yml index 1c183ea..a62291a 100644 --- a/.github/workflows/watch-upstream-docs.yml +++ b/.github/workflows/watch-upstream-docs.yml @@ -10,8 +10,10 @@ on: workflow_dispatch: permissions: - contents: write # commit the updated baseline + contents: write # push the baseline branch issues: write # open tracking issues + pull-requests: write # propose the updated baseline + actions: write # start validate on the proposal (see that step) concurrency: group: watch-upstream-docs @@ -66,15 +68,55 @@ jobs: gh issue create --title "$title" --body-file "$RUNNER_TEMP/body.md" --label "$label" done - - name: Commit the updated baseline + # main is protected: a direct push is refused for the required checks it + # has not run, so the baseline could never advance and every week's run + # ended red. The baseline moves the same way anything else moves here -- + # as a pull request. One branch, force-pushed: the baseline is a whole + # snapshot, so the newest proposal replaces the previous one rather than + # stacking on it. + - name: Propose the updated baseline + env: + GH_TOKEN: ${{ github.token }} + BRANCH: upstream-docs-baseline + BOT_EMAIL: 41898282+github-actions[bot]@users.noreply.github.com run: | set -euo pipefail if git diff --quiet -- upstream-docs-state.json; then echo "baseline unchanged" exit 0 fi + git config user.name "github-actions[bot]" - git config user.email "41898282+github-actions[bot]@users.noreply.github.com" + git config user.email "$BOT_EMAIL" git add upstream-docs-state.json - git commit -m "chore(upstream-docs): update watched-document baseline [skip ci]" - git push + git commit -m "chore(upstream-docs): update watched-document baseline" \ + -m "Signed-off-by: github-actions[bot] <$BOT_EMAIL>" + git push --force origin "HEAD:refs/heads/$BRANCH" + + existing="$(gh pr list --head "$BRANCH" --state open --json number \ + --jq '.[0].number // empty')" + if [ -n "$existing" ]; then + echo "refreshed the open proposal: #$existing" + else + cat > "$RUNNER_TEMP/pr-body.md" <<'BODY' + A watched upstream document moved, or a fetch failure changed its + count, so `upstream-docs-state.json` no longer describes what the + watcher sees. Merging this makes the new observation the baseline the + next run compares against. + + Read the `upstream-doc` issues this run opened before merging: this + accepts the new bytes as normal, and on its own says nothing about + whether the registers a driver decodes by have moved. + + Opened by the `watch-upstream-docs` workflow. + BODY + gh pr create --base main --head "$BRANCH" \ + --title "chore(upstream-docs): update watched-document baseline" \ + --body-file "$RUNNER_TEMP/pr-body.md" + fi + + # A pull request opened with GITHUB_TOKEN starts no workflow run, so + # the required checks would sit "Expected" forever and the proposal + # could never merge. workflow_dispatch is the documented exception: + # ask validate for a verdict on the head this proposal carries. + gh workflow run validate.yml --ref "$BRANCH" diff --git a/tests/test_upstream_docs_watch.py b/tests/test_upstream_docs_watch.py index bf533d7..58b7f9b 100644 --- a/tests/test_upstream_docs_watch.py +++ b/tests/test_upstream_docs_watch.py @@ -143,3 +143,21 @@ def test_state_file_round_trips(tmp_path): write_state(path, docs) assert load_state(path) == docs assert load_state(tmp_path / "missing.json") == {} + + +# The runner diffs the baseline it rewrites against the committed one and +# proposes the difference. Written in the platform's line ending, a baseline +# authored on Windows is 31 changed lines to Linux on a week when no watched +# document moved -- a pull request that says nothing. Both halves are pinned: +# what the tool writes, and what is committed. + + +def test_state_file_is_written_with_lf_on_every_platform(tmp_path): + docs, _ = compute_updates({}, CURRENT, ok(SHA_A), "t0") + path = tmp_path / "upstream-docs-state.json" + write_state(path, docs) + assert b"\r" not in path.read_bytes() + + +def test_committed_baseline_is_lf(): + assert b"\r" not in (ROOT / "upstream-docs-state.json").read_bytes() diff --git a/tools/check_upstream_docs.py b/tools/check_upstream_docs.py index 499df1f..37c90b6 100644 --- a/tools/check_upstream_docs.py +++ b/tools/check_upstream_docs.py @@ -293,7 +293,12 @@ def write_state(path: Path, docs: dict[str, dict]) -> None: "schema_version": SCHEMA_VERSION, "docs": {key: docs[key] for key in sorted(docs)}, } - path.write_text(json.dumps(payload, indent=2) + "\n", encoding="utf-8") + # Without newline="\n", text mode writes the platform's line ending. A + # baseline written on Windows then reads as a whole-file change to the + # Linux runner that rewrites it, and the watcher proposes 31 changed lines + # on a week when no watched document moved. + path.write_text(json.dumps(payload, indent=2) + "\n", + encoding="utf-8", newline="\n") # ---- CLI ---------------------------------------------------------------- diff --git a/upstream-docs-state.json b/upstream-docs-state.json index e716d43..19cfa43 100644 --- a/upstream-docs-state.json +++ b/upstream-docs-state.json @@ -1,31 +1,31 @@ -{ - "schema_version": "sourceful.upstream-docs-state/v1", - "docs": { - "myuplink::https://www.nibe.eu/webdav/files/myuplink_changelog/nibe-n.pdf": { - "driver": "myuplink", - "url": "https://www.nibe.eu/webdav/files/myuplink_changelog/nibe-n.pdf", - "title": "NIBE myUplink register/parameter changelog (nibe-n.pdf)", - "kind": "changelog", - "url_stability": "stable", - "sha256": "fa1186981a020f6c262cf35abd50fa447343cfaa16a4eedb018cb2525c00f221", - "content_length": 850716, - "first_seen": "2026-07-29T15:08:58+00:00", - "last_changed": "2026-07-29T15:08:58+00:00", - "consecutive_failures": 0, - "last_error": null - }, - "nibe_local::https://www.nibe.eu/webdav/files/myuplink_changelog/nibe-n.pdf": { - "driver": "nibe_local", - "url": "https://www.nibe.eu/webdav/files/myuplink_changelog/nibe-n.pdf", - "title": "NIBE S-series myUplink register/parameter changelog (nibe-n.pdf)", - "kind": "changelog", - "url_stability": "stable", - "sha256": "fa1186981a020f6c262cf35abd50fa447343cfaa16a4eedb018cb2525c00f221", - "content_length": 850716, - "first_seen": "2026-07-29T15:08:58+00:00", - "last_changed": "2026-07-29T15:08:58+00:00", - "consecutive_failures": 0, - "last_error": null - } - } -} +{ + "schema_version": "sourceful.upstream-docs-state/v1", + "docs": { + "myuplink::https://www.nibe.eu/webdav/files/myuplink_changelog/nibe-n.pdf": { + "driver": "myuplink", + "url": "https://www.nibe.eu/webdav/files/myuplink_changelog/nibe-n.pdf", + "title": "NIBE myUplink register/parameter changelog (nibe-n.pdf)", + "kind": "changelog", + "url_stability": "stable", + "sha256": "fa1186981a020f6c262cf35abd50fa447343cfaa16a4eedb018cb2525c00f221", + "content_length": 850716, + "first_seen": "2026-07-29T15:08:58+00:00", + "last_changed": "2026-07-29T15:08:58+00:00", + "consecutive_failures": 0, + "last_error": null + }, + "nibe_local::https://www.nibe.eu/webdav/files/myuplink_changelog/nibe-n.pdf": { + "driver": "nibe_local", + "url": "https://www.nibe.eu/webdav/files/myuplink_changelog/nibe-n.pdf", + "title": "NIBE S-series myUplink register/parameter changelog (nibe-n.pdf)", + "kind": "changelog", + "url_stability": "stable", + "sha256": "fa1186981a020f6c262cf35abd50fa447343cfaa16a4eedb018cb2525c00f221", + "content_length": 850716, + "first_seen": "2026-07-29T15:08:58+00:00", + "last_changed": "2026-07-29T15:08:58+00:00", + "consecutive_failures": 0, + "last_error": null + } + } +} From c17131a1fdcab22470927bfe7134fc33992f9e6b Mon Sep 17 00:00:00 2001 From: Claude Opus 5 Date: Mon, 3 Aug 2026 12:21:46 +0200 Subject: [PATCH 2/2] fix(upstream-docs): say what dispatching validate actually buys The step claimed the dispatched run made the required checks green. It does not, and this branch is the measurement: run 30804965042 put six successful check runs on the head of #67 -- drivers, packages, public-boundary, history-secret-scan, the channel preflight, and dco skipped -- and the pull request still reads BLOCKED with an empty status rollup, while #66, which got its run from a pull_request event, reads CLEAN with five. A commit whose only check suite came from a dispatch has no rollup for branch protection to read. The dispatch is still worth keeping: without it nothing runs against a bot's proposal at all, and a verdict a maintainer can read before spending attention on the diff is the whole ask. It just is not the thing that unblocks the merge, so neither the comment nor the pull request body says it is. Reopening the proposal is what starts the checks that count, and the body now says so where the maintainer will be standing when they need it. Signed-off-by: HuggeK <48095810+HuggeK@users.noreply.github.com> Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com> --- .github/workflows/watch-upstream-docs.yml | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/.github/workflows/watch-upstream-docs.yml b/.github/workflows/watch-upstream-docs.yml index a62291a..252212f 100644 --- a/.github/workflows/watch-upstream-docs.yml +++ b/.github/workflows/watch-upstream-docs.yml @@ -108,6 +108,12 @@ jobs: accepts the new bytes as normal, and on its own says nothing about whether the registers a driver decodes by have moved. + **The required checks do not start on their own here.** A pull request + opened with `GITHUB_TOKEN` starts no workflow run, so this one arrives + with an empty status rollup. Close and reopen it to run them. The + `validate` run dispatched against the branch head is a verdict you can + read in the meantime -- GitHub does not count it toward the rollup. + Opened by the `watch-upstream-docs` workflow. BODY gh pr create --base main --head "$BRANCH" \ @@ -116,7 +122,13 @@ jobs: fi # A pull request opened with GITHUB_TOKEN starts no workflow run, so - # the required checks would sit "Expected" forever and the proposal - # could never merge. workflow_dispatch is the documented exception: - # ask validate for a verdict on the head this proposal carries. + # nothing would run against this proposal at all. workflow_dispatch is + # the documented exception to that rule, so this asks validate for a + # verdict on the head the proposal carries -- which is worth having + # before a maintainer decides to spend attention on it. + # + # It does NOT turn the required checks green: a commit whose only + # check suite came from a dispatch has no status rollup, so branch + # protection still reads the proposal as blocked. Reopening it is what + # starts the checks that count. Measured on #67, not assumed. gh workflow run validate.yml --ref "$BRANCH"