Repository navigation
wikidata_stats: send maxlag on the edit only, then edit anyway - #119
Merged
Merged
Conversation
wikidata-stats kept failing with "Wikidata is still lagging after 600 s" (run 37676802595, during login). Wikidata's maxlag includes its query service's lag (WDQS lag / 60), which has stayed at 6-13 s for hours, so with maxlag=5 on every request the job waited out the read, the tokens and the login, and gave up before ever trying the edit. - Only wbeditentity carries maxlag; reads, tokens and login don't. - The edit waits out the lag for up to MAXLAG_WAIT (300 s), then is sent without maxlag: it is one edit per DB import, not mass editing. - 429 and 5xx are retried after Retry-After for up to RETRY_WAIT (300 s); Wikimedia rate-limits shared IPs such as GitHub's runners. - Run with python -u so the CI log shows progress as it happens. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GyVN76yBTdzjfo4EzqM6uv
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
wikidata-stats kept failing with
Wikidata is still lagging after 600 s, giving up: Waiting for wdqs1013: 6.47 seconds lagged.(run 37676802595, during login; same cause in #114's runs).Wikidata's maxlag includes its query service's lag (WDQS lag / 60), which has been at 6–13 s for hours. With
maxlag=5on every request, the job waited out the read, the tokens and the login, and gave up before ever trying the edit. Pywikibot would not help: it sends the samemaxlag=5with a bounded number of retries.wbeditentitycarriesmaxlag; reads, tokens and login don't.MAXLAG_WAIT(300 s), then is sent withoutmaxlag: it is one edit per DB import (a few a week), not mass editing.Retry-Afterfor up toRETRY_WAIT(300 s); Wikimedia rate-limits shared IPs such as GitHub's runners. They used to fail immediately.make wikidata_statsrunspython -u, so the CI log shows progress as it happens (the failed run printed everything at the end).Tests cover edit-only maxlag, editing without maxlag after the wait, and 429 retry / give-up.
🤖 Generated with Claude Code
https://claude.ai/code/session_01GyVN76yBTdzjfo4EzqM6uv
Generated by Claude Code