Joscha 1591cb9197 Add options to slow down local crawler ...

These options are meant to make the local crawler behave more like a
network-based crawler for purposes of testing and debugging other parts of the
code base.

2021-05-15 15:25:01 +02:00

..

authenticators

Add simple authenticator

2021-05-11 00:28:03 +02:00

crawlers

Add options to slow down local crawler

2021-05-15 15:25:01 +02:00

__init__.py

Single-source version number

2021-05-14 21:27:44 +02:00

__main__.py

Single-source version number

2021-05-14 21:27:44 +02:00

authenticator.py

Add simple authenticator

2021-05-11 00:28:03 +02:00

conductor.py

Add options to slow down local crawler

2021-05-15 15:25:01 +02:00

config.py

Add simple authenticator

2021-05-11 00:28:03 +02:00

crawler.py

Set user agent to "pferd/<version>"

2021-05-14 21:27:44 +02:00

limiter.py

Simplify Limiter implementation

2021-04-29 20:20:25 +02:00

output_dir.py

Fix output dir

2021-05-14 21:28:38 +02:00

pferd.py

Let crawlers obtain authenticators

2021-05-13 18:57:20 +02:00

report.py

Add HttpCrawler

2021-05-13 22:28:14 +02:00

transformer.py

Elaborate on transforms and implement changes

2021-04-29 20:24:18 +02:00

utils.py

Fix mypy errors

2021-04-29 15:47:52 +02:00

version.py

Single-source version number

2021-05-14 21:27:44 +02:00