pferd/PFERD
Joscha 1591cb9197 Add options to slow down local crawler
These options are meant to make the local crawler behave more like a
network-based crawler for purposes of testing and debugging other parts of the
code base.
2021-05-15 15:25:01 +02:00
..
authenticators Add simple authenticator 2021-05-11 00:28:03 +02:00
crawlers Add options to slow down local crawler 2021-05-15 15:25:01 +02:00
__init__.py Single-source version number 2021-05-14 21:27:44 +02:00
__main__.py Single-source version number 2021-05-14 21:27:44 +02:00
authenticator.py Add simple authenticator 2021-05-11 00:28:03 +02:00
conductor.py Add options to slow down local crawler 2021-05-15 15:25:01 +02:00
config.py Add simple authenticator 2021-05-11 00:28:03 +02:00
crawler.py Set user agent to "pferd/<version>" 2021-05-14 21:27:44 +02:00
limiter.py Simplify Limiter implementation 2021-04-29 20:20:25 +02:00
output_dir.py Fix output dir 2021-05-14 21:28:38 +02:00
pferd.py Let crawlers obtain authenticators 2021-05-13 18:57:20 +02:00
report.py Add HttpCrawler 2021-05-13 22:28:14 +02:00
transformer.py Elaborate on transforms and implement changes 2021-04-29 20:24:18 +02:00
utils.py Fix mypy errors 2021-04-29 15:47:52 +02:00
version.py Single-source version number 2021-05-14 21:27:44 +02:00