Skip to content

build(worker): raise the pypdf floor and pin to 6.18.0 - #188

Merged
PeterGuy326 merged 4 commits into
mainfrom
chore/worker-pypdf-6.18.0
Sep 10, 2026
Merged

PeterGuy326 merged 4 commits into
mainfrom
chore/worker-pypdf-6.18.0

Conversation

@waterbro-8

@waterbro-8 waterbro-8 commented Sep 8, 2026

Copy link
Copy Markdown
Collaborator

Not a review vote, and not an acceptance

I am the author of this branch, so under this org's require_code_owner +
require_last_push_approval configuration my own ticket cannot be the one that
clears it. Nothing here asks for a merge, a close, a label, or a tag. The
commitment is only: here is a change, here is what I measured.

Supersedes nothing by itself: #156 is a separate, still-valid Dependabot PR.
See Sequencing below.

What changes

Two files, +39/−39:

  • worker/pyproject.toml:31"pypdf>=4.0""pypdf>=6.18.0"
  • worker/uv.lockpypdf 6.15.0 → 6.18.0, nothing else moves

That is the whole diff. No test, no docs, no CI file, no CHANGELOG.md entry:
grep -i 'pypdf\|numpy\|torch\|dependabot' CHANGELOG.md on main returns
zero dependency-bump entries across its 381 lines, and #156 is 2 files too,
so an entry here would be inventing a convention.

Why the floor is written >=6.18.0 and not >=6.16.1

Three open Dependabot alerts name pypdf in this lock — #71
(GHSA-jp53-mhqp-8xcg, < 6.16.0), #72 (GHSA-763m-79hh-57f2, < 6.16.1),
#73 (GHSA-23w6-3w8w-8484, < 6.16.1). This satisfies all three, the same as
#156, once it is on the default branch.

It additionally takes two upstream releases that have no advisory, so no
alert will ever schedule them:

  • 6.17.0 (2026-09-04) — Security (SEC): Limit value for Roman numerals (#4047)
  • 6.18.0 (2026-09-07) — Security (SEC): Limit allowed length of indirect object tokens (#4055)

Re-measured for this PR: GET /advisories?affects=pypdf43 entries, all
type: reviewed, and the newest first_patched_version anywhere in that set is
6.16.1. Both new entries are the same hardening class as the three above
(bound an unbounded parser input), and the surface is the untrusted one:
worker/mem_worker/processors/pdf.py:79 runs PdfReader(BytesIO(file.data)) on
user-uploaded files.

Writing the floor at 6.18.0 rather than letting the lock alone carry it is the
part that survives a from-scratch resolve. Measured: uv resolves to the newest
release satisfying a floor, not to the floor
>=6.16.1 re-resolved today
lands on 6.18.0, and after the next release it would land past it, in a manifest
whose stated requirement was never 6.18.0.

Evidence

All of it executed on a tarball of main @ 2986fe38175f54d99f15dd38a498708c6ecd88cd
whose tree was proved equal to the remote (.commit.tree.sha
f447ca554377e85ba26090d59882b8d4f2b78731 == git init && git add -A && git write-tree), with the tool CI pins — uv 0.9.27 (ci.yml:185-187) — in a
sanitized env (no mirror/proxy variables; uv lock through a mirror-configured
uv rewrites thousands of URL lines and its "lock is stale" verdict is then
worthless).

# check result
1 uv lock on the untouched tree 0 diff lines vs committed worker/uv.lock — the pin is reproducible, so everything below is about the resolve, not a stale tool
2 uv lock after only the constraint edit Updated pypdf v6.15.0 -> v6.18.0, 76 lock lines
3 name/version pairs, both locks 85 packages each, exactly one difference: pypdf 6.15.0 → 6.18.0
4 uv lock --locked rc=0 — so the Worker leg's uv sync --frozen (ci.yml:193, UV_FROZEN: "1" at :170) accepts it
5 dist hashes vs the registry sdist ae58b7d93c22c169ffb02c3b06321c45c4f223b4916536568adb57d789d95d01, wheel 05b762b77bcb9dcb4a7c91fcf5dded585b25bee7269ab3d3001d7c55fa1b324b — byte-identical to pypi.org/pypi/pypdf/json
6 the repo's own fixture (test_processor_logic.py:419) extracted at 6.15.0 / 6.16.1 / 6.18.0 on Python 3.11 (ci.yml:182) identical page_count, identical sha256(extracted_text) (46e7c072b2684841…, 214 chars), "1800 RMB" in text true on all three, malformed input raises the same PdfStreamError on all three
7 6.18.0's only behavioural change (DEP: Rework configuration value handling (#4044)) grep over the whole tree for overwrite_configuration / apply_configuration / disable_legacy_handling / pypdf.constants0 hits; pypdf appears in only 3 files (worker/pyproject.toml, worker/mem_worker/processors/pdf.py, scripts/seed_demo_data.sh)

Checks 1–4 and 7 are the ones CI cannot shortcut; 6 is the one that says "the
worker's PDF path behaves the same", and 4 is the one that says "this lock is
self-consistent". The Worker test leg on the exact head remains the real proof
of the full pipeline (test_processor_logic.py:457) — that runs in CI, I did
not run it here
; the local venv has none of the Worker's own dependencies
installed.

About the diff size: of the 76 changed lock lines, 8 are pypdf (specifier,
version, sdist, wheel — each on both sides). The other 68 are uv's marker
renormalisation: the python_full_version >= '3.15' and sys_platform == 'darwin' fork marker hopping between torch 2.13.0 and 2.13.0+cpu (likewise
torchvision 0.28.0 / +cpu), python_full_version narrowing dropping off
numpy / scipy / tifffile, and the resolution-markers list reordering.
That is not avoidable churn I chose: a plain uv lock on either side of a pin
costs 0 lines, but any resolve that moves pypdf pays them
#156's own
+53/−53 is the same shape. No other package's version changes.

Process: this PR is ahead of its issue's readiness gate

AGENTS.md rule 2 says not to implement a material change until its issue has
acceptance criteria and status:ready. Refs #187, which I filed with the AC
list and reproduction steps for exactly this; it is status:needs-triage, and
status:ready is a maintainer's label, not mine to set. So this is a draft,
and undrafting it is the step that should follow that label rather than precede
it. If triage would rather the change go in behind #156, #187 can simply wait.

Sequencing with #156

Both branches touch the same two files, so they will conflict with each other,
not with main.

What I did not do

No review submitted, no vote, no merge, no close, no label change on anything
except the new issue's own labels, no ref moved on #156, no Update branch
clicked anywhere, no CHANGELOG.md edit, no tag, no publish.

Refs #187

The advisory stream first-patches everything it knows about at 6.16.1, so an
advisory-driven floor never reaches 6.17.0 ("Limit value for Roman numerals",
2026-09-04) or 6.18.0 ("Limit allowed length of indirect object tokens",
2026-09-07). Both are the same hardening class as the three advisories #156
exists for, and pypdf here parses user-uploaded files
(worker/mem_worker/processors/pdf.py:79).

Naming the floor 6.18.0 rather than leaning on the lock is what makes the
requirement survive a from-scratch resolve: uv resolves to the newest release
satisfying a floor, not to the floor.

Locked with the CI-pinned uv 0.9.27. Of the 76 changed lock lines, 8 are pypdf;
no other package's version moves (85 name/version pairs differ only for pypdf).
@waterbro-8

Copy link
Copy Markdown
Collaborator Author

Head unchanged; the body needed a tracking-record keyword

The first PR Policy run on this head (34192719765) failed at 05:58:55Z
with Link a tracking record in the PR body. That was my mistake, and the
mechanism is worth writing down for the next person: the gate
(.github/workflows/pr-policy.yml) reads only pullRequest.body — my
commit message says Refs #187, and that does not count. The body referred to
the issue as bold text with no keyword, so the regex had nothing to match.

Fixed by one clause in the Process section: Refs #187, which I filed with the AC list…. No commit was added or rewritten — the head is still
6e7536fe975a
, and the only thing that changed is this PR's body.

Re-run 34193405908success at 06:08:47Z. Conventional title and linked issue is now green, and so are the other 19 check names on this head, each read
at its own highest check_suite.id. The older failed PR Policy record stays
in the history of this commit, which is expected: commits/<sha>/check-runs
returns every round, not just the current one.

The CI result the body said it had not run

The Evidence table had one honest gap — I wrote that the Worker test leg runs in
CI and that I had not run it here. It has now run here, so here is the readback
from Worker (job 101953897734, success, attempt 1):

uv sync --frozen --extra test --extra dev
 + pypdf==6.18.0
266 passed, 1 skipped in 5.19s

--frozen with UV_FROZEN=1 is the part that matters for this change: CI
resolved nothing, it installed exactly the lock in this diff and the suite
passed on it.

Everything else in this comment is a status record, not new evidence for the
acceptance criteria in #187. Still a draft, still not a review vote, still not
asking for a merge — and #156 is untouched.

@PeterGuy326 PeterGuy326 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review result

APPROVE for the current head 6e7536fe975ac60d38f47f25441e1e2f1dd0bfa7.

The diff is limited to the pypdf floor and the locked dependency resolution; git diff --check is clean. The latest head-matched policy run, Go, Worker, Web, PostgreSQL integration, recall, wrapper-compatibility, CodeQL, and dependency-review checks are green. The earlier linked-issue check failure is superseded by the later successful run. This is an approval of the current draft only; it does not mark the draft ready or authorize a release.

@PeterGuy326
PeterGuy326 enabled auto-merge (squash) September 10, 2026 16:11
@PeterGuy326
PeterGuy326 merged commit 55d09b2 into main Sep 10, 2026
20 checks passed
@PeterGuy326
PeterGuy326 deleted the chore/worker-pypdf-6.18.0 branch September 10, 2026 16:23
waterbro-8 added a commit that referenced this pull request Sep 17, 2026
## Canonical requirement

Refs bytefolk/.github#32

- Canonical Issue URL: bytefolk/.github#32
- Consumed revision: R1
- No automatic close keywords: acknowledged

Decision reference: the initial R1 Issue body. It explicitly records
that local candidates preceded this prospective publication record; no
retrospective approval is claimed.

## Requirement trace

| REQ/AC IDs | Changed files / domain | Tests or review evidence |
|---|---|---|
| REQ-001 / AC-001 | 4 exact-pinned version annotations | Exact
expected-byte replacement PASS |
| REQ-002 / AC-002 | 2 files in bytefolk/mem | Repository inventory
PASS; aggregate 7 repositories, 13 files, 21 lines |
| REQ-003 / AC-003 | Existing workflow content and modes | Parsed YAML
and comment-stripped bytes identical |
| REQ-004 / AC-004 | Current-head CI and independent review | Local
independent replay recorded in the canonical R1 Issue linked above;
hosted CI collected on head `f464f686` (19 of 20 checks succeed, see
Validation); independent human review requested and still pending |

## File domains

`.github/workflows/bytefolk-scorecard.yml` (47);
`.github/workflows/bytefolk-security.yml` (58, 65, 68).

Prepared parent / merge base: `2986fe38175f54d99f15dd38a498708c6ecd88cd`
PR base at publication: `87db0dfe0507be2190fe2fdcce0e267be8224f4d`.
Since that baseline `main` advanced by six commits through `3c13f04e`
(#162, #160, #165, #188, #158, #204) — not only `web/package-lock.json`
as previously stated here. None of them touched `.github/workflows/`, so
the F9 workflow blobs and the PR diff are unchanged. The reviewed commit
and original parent are preserved.
Head: `f464f68636adc6bb5295c3818aa6654c46a3caad` — `d2a9ec5` plus one
non-forced `Merge branch 'main'` commit (`f464f686`) that brought the
branch up to `3c13f04e` so it is no longer `BEHIND`. Verified: `git
rev-parse d2a9ec5:.github/workflows/bytefolk-scorecard.yml` and
`...:bytefolk-security.yml` return the same blobs (`2058126c`,
`48a507e0`) as at `f464f686`, and `git diff main...f464f68` is still
exactly these 2 files, `+4/-4`. The comment-only payload is therefore
byte-identical to the reviewed commit and the equality proof above holds
on the current head.

## Scope and non-goals

Correct only `# v4.37.4` to `# v4.37.9` on CodeQL uses-lines pinned to
`cdf488f595d80d6e07e03d4674febd5ab45fa938`. The [official tag
object](https://api.github.com/repos/github/codeql-action/git/tags/a35ac6e6798d72df5475948b28efb89edc2e19ca)
resolves to that existing pin. Action SHAs, permissions, triggers,
steps, matrices, other pins, and runtime code are unchanged.

## Validation

- Exact commands: `ruby evidence/verify.rb --baseline` and `ruby
evidence/verify.rb --committed` from the retained review packet; `git
diff --check 2986fe3
d2a9ec5` from this repository.
- Observed counts/results: PASS 2/2 files and 4/4 replacements here;
aggregate PASS 13/13 files and 21/21 replacements. Baseline
intentionally exits 1 after detecting all 21 stale annotations;
committed verification exits 0.
- Check URLs: collected on head `f464f686` — 19 of 20 checks succeed.
The single failure is [`HTTP, CLI and MCP
lifecycle`](https://github.com/bytefolk/mem/actions/runs/34807101354/job/103861020551),
whose log is `pull access denied for minio/minio` at ~13s: a
container-image pull failure in an unrelated job. The same workflow was
green on `main` at `3c13f04e`, and the identical failure is present on
#198 and #199, so it is not caused by this comment-only change.
Root-cause tracking is separate and open.

The strict verifier checks the changed-file allowlist; exact old blobs
and line inventory; complete expected-byte replacement; absence of stale
target annotations; parsed YAML equality; comment-stripped byte equality
and SHA-256 digests; whitespace and unchanged modes; one commit with the
exact parent; and clean worktrees with no untracked files. All passed.
The independent replay is recorded in canonical R1. The verifier and
inventory are retained outside repository commits.

| ID | REQ/AC | Observable acceptance criterion | Command or manual
steps | Environment | Expected | Observed | Status |
|---|---|---|---|---|---|---|---|
| V1 | AC-001, AC-002, AC-003 | Exact annotations with executable YAML
unchanged | `ruby evidence/verify.rb --committed` | Ruby 2.6.10, Psych
3.1.0, isolated review packet | Exact scoped replacements and equality |
2/2 files; 4/4 lines; all invariants pass | PASS |
| V2 | AC-004 | Hosted checks on this exact head | Inspect this PR's
checks at `f464f686` | GitHub Actions | Applicable checks succeed | 19
of 20 succeed; `HTTP, CLI and MCP lifecycle` fails on `pull access
denied for minio/minio` (infra, unrelated job, also failing on
#198/#199, green on `main`) | PARTIAL |

## Security and compatibility

Documentation annotation only. No dependencies, permissions,
credentials, data flows, or runtime behavior change. The diff and commit
identity were inspected for public-safe content. No CHANGELOG entry or
behavior-documentation update is needed because only explanatory
comments change.

## Known limitations

Runtime suites, build, coverage, and dependency audits were not rerun
for this comment-only change; no runtime test result is claimed. Hosted
CI is separate from local equality proof. Two limits now apply: (1) the
strict verifier's `one commit with the exact parent` invariant describes
the reviewed payload commit `d2a9ec5`, not the current branch shape,
which carries two additional `Merge branch 'main'` commits; (2) this PR
is **not merge-ready yet** — repository `AGENTS.md` step 6 requires
passing CI *and* an approval from someone other than the author, and
`HTTP, CLI and MCP lifecycle` is red on the unrelated `minio` pull, so
the green-CI half is unmet until that infrastructure failure is fixed.

## Risk and rollback

Low-risk annotation correction. Roll back through an ordinary revert of
this single commit. There is no migration or release action.

## Product review handoff

- Implementation/publication owner: @PeterGuy326
- Automated pre-review result: independent local replay recorded in R1;
no human approval implied.
- Human final review: PENDING; no human review requested by this
publication.
- Merge ledger owner: @PeterGuy326
- Product reviewer: @PeterGuy326
- Milestone or release packet: N/A: bounded documentation annotation
maintenance
- Merge, CI, release, and model judgment do not accept or close the
Issue: acknowledged


## Maintenance update (2026-09-14, @waterbro-8)

Records written by the maintainer account, not by the implementation
owner:

- `f464f686 Merge branch 'main'` was pushed to this head branch
(non-forced, `main` at `3c13f04e` is an ancestor of the head) to clear
the `BEHIND` state this PR's own body said blocked merging. No workflow
file content changed: both blobs are identical to `d2a9ec5`.
- The stale facts above were corrected in place: the recorded head SHA,
the "Main advanced only `web/package-lock.json` in PR #192" claim, the
`NOT VERIFIED` hosted-CI rows, and the "this is a draft, not
merge-ready" note.
- This PR was marked ready for review and an independent review was
requested. The maintainer account that pushed the merge commit did
**not** approve it: `AGENTS.md` step 6 requires an approval from someone
other than the author, and a commit author on the head cannot supply
that approval for their own push. `@PeterGuy326` remains implementation
and merge-ledger owner.

Co-authored-by: 勒布朗-詹姆斯 <2986253039@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants