Skip to content

tolerate the failure of a small number of images #66

Description

@yzqzss
Hard retry... (3), due to: HTTPSConnectionPool(host='static.wikia.nocookie.net', port=443): Max retries exceeded with url: /szbus/images/b/b6/%E5%85%B4%E4%B8%9A%E8%B7%AF%E5%85%AC%E4%BA%A4%E6%80%BB%E7%AB%99_%E5%85%A8%E6%99%AF_%E6%89%8B%E5%8A%A8%E6%8B%BC%E6%8E%A5_211109.jpg/revision/latest?cb=20250921160807 (Caused by ResponseError('too many 500 error responses'))
Hard retry... (2), due to: HTTPSConnectionPool(host='static.wikia.nocookie.net', port=443): Max retries exceeded with url: /szbus/images/b/b6/%E5%85%B4%E4%B8%9A%E8%B7%AF%E5%85%AC%E4%BA%A4%E6%80%BB%E7%AB%99_%E5%85%A8%E6%99%AF_%E6%89%8B%E5%8A%A8%E6%8B%BC%E6%8E%A5_211109.jpg/revision/latest?cb=20250921160807 (Caused by ResponseError('too many 500 error responses'))
Hard retry... (1), due to: HTTPSConnectionPool(host='static.wikia.nocookie.net', port=443): Max retries exceeded with url: /szbus/images/b/b6/%E5%85%B4%E4%B8%9A%E8%B7%AF%E5%85%AC%E4%BA%A4%E6%80%BB%E7%AB%99_%E5%85%A8%E6%99%AF_%E6%89%8B%E5%8A%A8%E6%8B%BC%E6%8E%A5_211109.jpg/revision/latest?cb=20250921160807 (Caused by ResponseError('too many 500 error responses'))
urllib3.exceptions.ResponseError: too many 500 error responses

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/root/.local/share/uv/tools/wikiteam3/lib/python3.13/site-packages/requests/adapters.py", line 696, in send
    resp = conn.urlopen(
        method=request.method,
    ...<9 lines>...
        chunked=chunked,
    )
  File "/root/.local/share/uv/tools/wikiteam3/lib/python3.13/site-packages/urllib3/connectionpool.py", line 955, in urlopen
    return self.urlopen(
           ~~~~~~~~~~~~^
        method,
        ^^^^^^^
    ...<13 lines>...
        **response_kw,
        ^^^^^^^^^^^^^^
    )
    ^
  File "/root/.local/share/uv/tools/wikiteam3/lib/python3.13/site-packages/urllib3/connectionpool.py", line 955, in urlopen
    return self.urlopen(
           ~~~~~~~~~~~~^
        method,
        ^^^^^^^
    ...<13 lines>...
        **response_kw,
        ^^^^^^^^^^^^^^
    )
    ^
  File "/root/.local/share/uv/tools/wikiteam3/lib/python3.13/site-packages/urllib3/connectionpool.py", line 955, in urlopen
    return self.urlopen(
           ~~~~~~~~~~~~^
        method,
        ^^^^^^^
    ...<13 lines>...
        **response_kw,
        ^^^^^^^^^^^^^^
    )
    ^
  [Previous line repeated 2 more times]
  File "/root/.local/share/uv/tools/wikiteam3/lib/python3.13/site-packages/urllib3/connectionpool.py", line 945, in urlopen
    retries = retries.increment(method, url, response=response, _pool=self)
  File "/root/.local/share/uv/tools/wikiteam3/lib/python3.13/site-packages/wikiteam3/dumpgenerator/cli/cli.py", line 343, in increment
    return super(CustomRetry, self).increment(method=method, url=url, *args, **kwargs)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/share/uv/tools/wikiteam3/lib/python3.13/site-packages/urllib3/util/retry.py", line 543, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='static.wikia.nocookie.net', port=443): Max retries exceeded with url: /szbus/images/b/b6/%E5%85%B4%E4%B8%9A%E8%B7%AF%E5%85%AC%E4%BA%A4%E6%80%BB%E7%AB%99_%E5%85%A8%E6%99%AF_%E6%89%8B%E5%8A%A8%E6%8B%BC%E6%8E%A5_211109.jpg/revision/latest?cb=20250921160807 (Caused by ResponseError('too many 500 error responses'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/.local/bin/wikiteam3dumpgenerator", line 10, in <module>
    sys.exit(main())
             ~~~~^^
  File "/root/.local/share/uv/tools/wikiteam3/lib/python3.13/site-packages/wikiteam3/dumpgenerator/__init__.py", line 4, in main
    DumpGenerator()
    ~~~~~~~~~~~~~^^
  File "/root/.local/share/uv/tools/wikiteam3/lib/python3.13/site-packages/wikiteam3/dumpgenerator/dump/generator.py", line 86, in __init__
    DumpGenerator.createNewDump(config=config, other=other)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/share/uv/tools/wikiteam3/lib/python3.13/site-packages/wikiteam3/dumpgenerator/dump/generator.py", line 119, in createNewDump
    Image.generate_image_dump(
    ~~~~~~~~~~~~~~~~~~~~~~~~~^
        config=config, other=other, images=images, session=other.session
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/root/.local/share/uv/tools/wikiteam3/lib/python3.13/site-packages/wikiteam3/dumpgenerator/dump/image/image.py", line 251, in generate_image_dump
    try:

  File "/root/.local/share/uv/tools/wikiteam3/lib/python3.13/site-packages/requests/sessions.py", line 671, in get
    return self.request("GET", url, params=params, **kwargs)
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/share/uv/tools/wikiteam3/lib/python3.13/site-packages/requests/sessions.py", line 651, in request
    resp = self.send(prep, **send_kwargs)
  File "/root/.local/share/uv/tools/wikiteam3/lib/python3.13/site-packages/wikiteam3/utils/monkey_patch.py", line 150, in new_send
    return self.old_send_method(request, **kwargs)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/root/.local/share/uv/tools/wikiteam3/lib/python3.13/site-packages/requests/sessions.py", line 784, in send
    r = adapter.send(request, **kwargs)
  File "/root/.local/share/uv/tools/wikiteam3/lib/python3.13/site-packages/requests/adapters.py", line 720, in send
    raise RetryError(e, request=request)
requests.exceptions.RetryError: HTTPSConnectionPool(host='static.wikia.nocookie.net', port=443): Max retries exceeded with url: /szbus/images/b/b6/%E5%85%B4%E4%B8%9A%E8%B7%AF%E5%85%AC%E4%BA%A4%E6%80%BB%E7%AB%99_%E5%85%A8%E6%99%AF_%E6%89%8B%E5%8A%A8%E6%8B%BC%E6%8E%A5_211109.jpg/revision/latest?cb=20250921160807 (Caused by ResponseError('too many 500 error responses'))
Undo monkey patch...

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions