Joscha a9af56a5e9 Improve specifying crawlers via CLI ...

Instead of removing the sections of unselected crawlers from the config file,
crawler selection now happens in the Pferd after loading the crawlers and is
more sophisticated. It also has better error messages.

2021-05-23 18:18:50 +02:00

..

authenticators

Fix simple authenticator output

2021-05-23 10:45:37 +02:00

cli

Implement CLI Command for ilias crawler

2021-05-23 13:30:42 +02:00

crawlers

Explain ILIAS HTML parsing and add some warnings

2021-05-23 18:14:54 +02:00

__init__.py

Single-source version number

2021-05-14 21:27:44 +02:00

__main__.py

Improve specifying crawlers via CLI

2021-05-23 18:18:50 +02:00

authenticator.py

Move logging logic to singleton

2021-05-18 22:45:19 +02:00

config.py

Clean up logging

2021-05-23 11:37:19 +02:00

crawler.py

Replace asyncio.gather with custom Crawler function

2021-05-23 17:25:16 +02:00

http_crawler.py

Document authentication in HTTP crawler and rename prepare_request

2021-05-23 11:55:34 +02:00

limiter.py

Change limiter logic

2021-05-15 15:25:05 +02:00

logging.py

Add warn_contd

2021-05-23 18:14:54 +02:00

output_dir.py

Log crawl and download actions more extensively

2021-05-23 16:25:44 +02:00

pferd.py

Improve specifying crawlers via CLI

2021-05-23 18:18:50 +02:00

report.py

Improve output dir and report error handling

2021-05-22 20:54:42 +02:00

transformer.py

Improve transformer explain wording

2021-05-23 11:45:14 +02:00

utils.py

Clean up logging

2021-05-23 11:37:19 +02:00

version.py

Implement --version flag

2021-05-19 17:33:36 +02:00