44 Commits

Author	SHA1	Message	Date
I-Al-Istannen	9d6ce331a5	Use IliasCrawlerEntry entries in the ilias scraper	2020-05-30 15:20:51 +02:00
I-Al-Istannen	821c7ade26	Move video url extraction logic to crawler	2020-05-30 00:22:31 +02:00
I-Al-Istannen	62535b4452	Unpack videos in ILIAS downloader	2020-05-21 22:12:52 +02:00
I-Al-Istannen	c0056e5669	Correctly crawl video pages with multiple pages	2020-05-21 21:38:07 +02:00
I-Al-Istannen	4f56c8f192	Pass element type to ilias directory filter	2020-05-12 14:41:13 +02:00
I-Al-Istannen	a0f9d31d94	Use PrettyLogger warning everywhere	2020-05-10 21:56:12 +02:00
I-Al-Istannen	9850ab1d73	Allow crawling the ILIAS Personal Desktop	2020-05-10 12:16:42 +02:00
I-Al-Istannen	f6faacabb0	Move FatalException to errors.py	2020-05-09 00:11:21 +02:00
I-Al-Istannen	19c1e3ac6f	Fail on invalid ILIAS course ids	2020-05-09 00:11:20 +02:00
I-Al-Istannen	a4c518bf4c	Update date find regex	2020-05-08 22:17:58 +02:00
I-Al-Istannen	057135022f	Try to accept that life sometimes is in English	2020-05-08 22:10:43 +02:00
I-Al-Istannen	755e9aa0d3	Try to add support for Shibboleth TFA token	2020-05-08 21:52:51 +02:00
I-Al-Istannen	56f2394001	Add a download progress bar	2020-05-08 17:09:56 +02:00
I-Al-Istannen	42345ecc61	Demangle "Morgen" too	2020-04-30 12:05:25 +02:00
I-Al-Istannen	920d521d68	Change PrettyLogger.warn to PrettyLogger.warning	2020-04-25 20:11:51 +02:00
I-Al-Istannen	e0b46a306a	Use warn method in IliasCrawler	2020-04-25 20:07:40 +02:00
I-Al-Istannen	8a42a2a396	Move logging into its own file	2020-04-25 20:02:01 +02:00
I-Al-Istannen	80247400a4	Debug log when starting an ilias download	2020-04-25 13:02:07 +02:00
Joscha	292e516297	Change crawler and downloader output	2020-04-24 18:24:44 +00:00
Joscha	5b929f09a2	Move download strategies to downloader ... Also fixes an issue where the downloader didn't mark files that were not downloaded due to the strategy used.	2020-04-24 14:27:40 +00:00
I-Al-Istannen	f3f4be2690	More free functions	2020-04-23 19:21:49 +02:00
I-Al-Istannen	076b8c5a1f	Add download strategies to save bandwith ... Only download files that are newer than the local version.	2020-04-23 18:29:20 +02:00
I-Al-Istannen	13bc78c889	Display reason for ignoring an element in ilias crawler	2020-04-23 13:54:58 +02:00
I-Al-Istannen	dc964a9d98	Remove finished TODOs	2020-04-23 13:30:34 +02:00
I-Al-Istannen	c2b14f3db9	ilias crawler: Use direct download link if possible	2020-04-23 13:08:12 +02:00
Joscha	4b59a7c375	Move around TODOs	2020-04-23 10:49:01 +00:00
I-Al-Istannen	bef210ae77	Rename and implement IliasDirectoryFilter	2020-04-23 12:35:18 +02:00
Joscha	2de4255a78	Add Pferd class	2020-04-23 09:50:32 +00:00
Joscha	3c808879c9	Add Transforms and Transformables	2020-04-22 18:25:09 +00:00
I-Al-Istannen	a051e3bcca	ilias crawler: Add some unhelpful documentation	2020-04-22 17:58:19 +02:00
I-Al-Istannen	eb7df036df	WIP: ilias crawler: Also crawl assignments	2020-04-22 14:32:20 +02:00
I-Al-Istannen	23db59e733	WIP: ilias-crawler: Demangle dates	2020-04-22 12:58:44 +02:00
I-Al-Istannen	ac65b06a8e	Satisfy pylint a bit	2020-04-22 01:37:34 +02:00
I-Al-Istannen	8891041069	WIP: crawler: Add opencast video crawler	2020-04-21 23:01:19 +02:00
I-Al-Istannen	70d63e3e90	WIP: Start small ILIAS crawler	2020-04-21 13:32:03 +02:00
I-Al-Istannen	b2a7af2e3e	Store modification_date in IliasDownloadInfo, remove parameters	2020-04-21 13:31:50 +02:00
I-Al-Istannen	23bed48c8c	Satisfy autopep8	2020-04-21 13:30:42 +02:00
Joscha	0926d33798	Use downloader-specific data classes	2020-04-20 18:07:45 +00:00
Joscha	4ef0ffe3bf	Listen to pylint and mypy	2020-04-20 17:44:58 +00:00
I-Al-Istannen	ed9245c14d	Remove old organizer	2020-04-20 18:50:23 +02:00
I-Al-Istannen	01e6972c96	Add ilias downloader	2020-04-20 18:49:01 +02:00
Joscha	6407190ae0	Soupify requests responses properly	2020-04-20 16:38:30 +00:00
Joscha	154d6b29dd	Listen to pylint	2020-04-20 15:16:22 +00:00
Joscha	879a2c7c80	Rewrite ILIAS authenticator	2020-04-20 14:26:30 +00:00