pferd/PFERD
Joscha 296a169dd3 Make limiter logic more complex
The limiter can now distinguish between crawl and download actions and has a
fancy slot system and delay logic.
2021-05-15 15:25:05 +02:00
..
authenticators Add simple authenticator 2021-05-11 00:28:03 +02:00
crawlers Add options to slow down local crawler 2021-05-15 15:25:01 +02:00
__init__.py Single-source version number 2021-05-14 21:27:44 +02:00
__main__.py Single-source version number 2021-05-14 21:27:44 +02:00
authenticator.py Add simple authenticator 2021-05-11 00:28:03 +02:00
conductor.py Add options to slow down local crawler 2021-05-15 15:25:01 +02:00
config.py Make limiter logic more complex 2021-05-15 15:25:05 +02:00
crawler.py Make limiter logic more complex 2021-05-15 15:25:05 +02:00
limiter.py Make limiter logic more complex 2021-05-15 15:25:05 +02:00
output_dir.py Fix output dir 2021-05-14 21:28:38 +02:00
pferd.py Let crawlers obtain authenticators 2021-05-13 18:57:20 +02:00
report.py Add HttpCrawler 2021-05-13 22:28:14 +02:00
transformer.py Elaborate on transforms and implement changes 2021-04-29 20:24:18 +02:00
utils.py Fix mypy errors 2021-04-29 15:47:52 +02:00
version.py Single-source version number 2021-05-14 21:27:44 +02:00