[tkittel]

Another small potential race condition spotted in our new CI. Fix looks simple enough.

Claude says:

GitHubRegistry wrote its cached registry file in place. When several processes create the same registry at the same time (e.g. two notebooks run in parallel with an empty cache), one of them can read the file while another is writing it, and gets an empty or truncated registry, or fails with:

ValueError: not enough values to unpack (expected 2, got 1)

The file is now written to a temporary file which is then renamed, as the in-memory registries already do. If the rename fails (on Windows, while another process is reading the file), the temporary file is dropped, since the other process wrote the same content.

The new test creates the same registry (20,000 entries) with 8 processes at once, 5 times, and takes about a second. Without the fix it fails in every run.

🤖 Generated with Claude Code

[tkittel]

@g5t Claude had a look at that spurious test failure, the proposed solution is added to this PR in a second commit. Claude says:

As discussed, the second commit fixes the failure of test_parallel_fetch_into_new_directories on Windows, which is a second race in pooch, not a flaky test:

pooch downloads to a temporary file and moves it into place with shutil.move. On Windows, the rename fails when another process already put the same file there, so shutil.move falls back to copying onto it, which fails with PermissionError: [WinError 32] The process cannot access the file because it is being used by another process if another process has the file open at that moment. This is not covered by the pooch PR (fatiando/pooch#554), which only fixes the directory creation.

pooch_fetch() now retries the fetch a few times on PermissionError (waiting at most about a second in total). The retry then finds the finished file and only checks its hash. A new test checks the retry with a fake pooch; the race itself only happens on Windows, where test_parallel_fetch_into_new_directories exercises it.

[tkittel]

Claude also privately admitted to me:

Since the race is intermittent, one green Windows run doesn't prove much. If the test still fails occasionally after this, I'll need to look again.

[tkittel]

well, that worry was premature claude :-)

[tkittel]

Claude fixed the new failure as well, probably. Claude:

The Windows 3.14 job then failed in the new test_parallel_registry_file, with PermissionError(13, 'Permission denied') in one of the 8 processes: on Windows, opening the cached registry file can fail for a moment while another process replaces it with its own (identical) copy. The third commit retries that read a few times, as for the pooch fetches.