57 Commits

Author	SHA1	Message	Date
Joscha	f9b2fd60e2	Document local crawler and auth	2021-05-09 01:33:47 +02:00
Joscha	60cd9873bc	Add local file crawler	2021-05-06 01:02:40 +02:00
Joscha	273d56c39a	Properly load crawler config	2021-05-05 23:45:10 +02:00
Joscha	f776186480	Use PurePath instead of Path ... Path should only be used when we need to access the file system. For all other purposes (mainly crawling), we use PurePath instead since the paths don't correspond to paths in the local file system.	2021-04-29 20:20:25 +02:00
Joscha	d96a361325	Test and fix exclusive output	2021-04-29 15:27:16 +02:00
Joscha	ac3bfd7388	Make progress bars easier to use ... The crawler now supports two types of progress bars	2021-04-29 13:53:16 +02:00
Joscha	bbc792f9fb	Implement Crawler and DummyCrawler	2021-04-29 13:44:29 +02:00