Skip to content

Latest commit

 

History

7 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 

Repository files navigation

WEB CRAWLER

overview

backend

backend: django, celery, rabbitmq, Elasticsearch

# functionality
Fixed URL Crawling
Continious URL Crawling
WEB Indexing (using Elasticsearch)

frontend

frontend: react

# functionality
Setting seed URL
Set Crawling Type (Fixed | Continious)
Searching by (Keyword | long-string)
Show Search Result

About

a web crawler in python

Topics

Resources

Stars

0 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages