A Scrapy middleware for scraping time series data from Archive.org's Wayback Machine.
-
Updated
Feb 18, 2024 - Python
#
scrapy-extension
Here are 11 public repositories matching this topic...
使 scrapy 开发不用在意 item，pipeline，middleware 等通用场景下模块的编写，解放开发者的双手。
-
Updated
Aug 31, 2026 - Python
🕶 Awesome list of Scrapy tools and libraries
-
Updated
Jul 6, 2020
scrapy 常用爬网必备工具包
-
Updated
Feb 8, 2023 - Python
A Scrapy extension to log items coverage when the spider shuts down
-
Updated
Apr 11, 2020 - Python
A Scrapy extension for sending notification to Slack channels
-
Updated
Nov 29, 2022 - Python
[MIRROR OF https://codeberg.org/sp1thas/scrapy-folder-tree] A scrapy pipeline which stores files using folder trees.
-
Updated
May 4, 2025 - Python
Flexible and modern User-Agent rotator middleware for Scrapy, supporting Faker, fake-useragent, and custom providers.
python middleware crawler user-agent scraping faker python3 scrapy scrapy-extension scrapy-framework fake-useragent user-agent-generator user-agent-rotation scrapy-middleware
-
Updated
Jul 13, 2025 - Python
将 Spider Stats 存储到 MongoDB 的扩展，可以用于爬虫监控和统计
-
Updated
Nov 17, 2020 - Python
A Python library that acts as a middleware for Scrapy, a popular web scraping framework. This middleware integrates with the Ujeebu API to provide additional functionality to your Scrapy spiders
-
Updated
Dec 30, 2025 - Python
Scrapy downloader middleware for FlareSolverr with multiple backends, concurrency control, and optional sessions and proxies.
python middleware crawler chrome proxy scraping chromium python3 cloudflare web-scraping scrapy scrapy-extension scrapy-framework flaresolverr scrapy-middleware
-
Updated
Jul 26, 2026 - Python
Add this topic to your repo
To associate your repository with the scrapy-extension topic, visit your repo's landing page and select "manage topics."