Commit Graph

40 Commits

Author SHA1 Message Date
I-Al-Istannen
4f56c8f192 Pass element type to ilias directory filter 2020-05-12 14:41:13 +02:00
I-Al-Istannen
a0f9d31d94 Use PrettyLogger warning everywhere 2020-05-10 21:56:12 +02:00
I-Al-Istannen
9850ab1d73 Allow crawling the ILIAS Personal Desktop 2020-05-10 12:16:42 +02:00
I-Al-Istannen
f6faacabb0 Move FatalException to errors.py 2020-05-09 00:11:21 +02:00
I-Al-Istannen
19c1e3ac6f Fail on invalid ILIAS course ids 2020-05-09 00:11:20 +02:00
I-Al-Istannen
a4c518bf4c Update date find regex 2020-05-08 22:17:58 +02:00
I-Al-Istannen
057135022f Try to accept that life sometimes is in English 2020-05-08 22:10:43 +02:00
I-Al-Istannen
755e9aa0d3 Try to add support for Shibboleth TFA token 2020-05-08 21:52:51 +02:00
I-Al-Istannen
56f2394001 Add a download progress bar 2020-05-08 17:09:56 +02:00
I-Al-Istannen
42345ecc61 Demangle "Morgen" too 2020-04-30 12:05:25 +02:00
I-Al-Istannen
920d521d68 Change PrettyLogger.warn to PrettyLogger.warning 2020-04-25 20:11:51 +02:00
I-Al-Istannen
e0b46a306a Use warn method in IliasCrawler 2020-04-25 20:07:40 +02:00
I-Al-Istannen
8a42a2a396 Move logging into its own file 2020-04-25 20:02:01 +02:00
I-Al-Istannen
80247400a4 Debug log when starting an ilias download 2020-04-25 13:02:07 +02:00
Joscha
292e516297 Change crawler and downloader output 2020-04-24 18:24:44 +00:00
Joscha
5b929f09a2 Move download strategies to downloader
Also fixes an issue where the downloader didn't mark files that were not
downloaded due to the strategy used.
2020-04-24 14:27:40 +00:00
I-Al-Istannen
f3f4be2690 More free functions 2020-04-23 19:21:49 +02:00
I-Al-Istannen
076b8c5a1f Add download strategies to save bandwith
Only download files that are newer than the local version.
2020-04-23 18:29:20 +02:00
I-Al-Istannen
13bc78c889 Display reason for ignoring an element in ilias crawler 2020-04-23 13:54:58 +02:00
I-Al-Istannen
dc964a9d98 Remove finished TODOs 2020-04-23 13:30:34 +02:00
I-Al-Istannen
c2b14f3db9 ilias crawler: Use direct download link if possible 2020-04-23 13:08:12 +02:00
Joscha
4b59a7c375 Move around TODOs 2020-04-23 10:49:01 +00:00
I-Al-Istannen
bef210ae77 Rename and implement IliasDirectoryFilter 2020-04-23 12:35:18 +02:00
Joscha
2de4255a78 Add Pferd class 2020-04-23 09:50:32 +00:00
Joscha
3c808879c9 Add Transforms and Transformables 2020-04-22 18:25:09 +00:00
I-Al-Istannen
a051e3bcca ilias crawler: Add some unhelpful documentation 2020-04-22 17:58:19 +02:00
I-Al-Istannen
eb7df036df WIP: ilias crawler: Also crawl assignments 2020-04-22 14:32:20 +02:00
I-Al-Istannen
23db59e733 WIP: ilias-crawler: Demangle dates 2020-04-22 12:58:44 +02:00
I-Al-Istannen
ac65b06a8e Satisfy pylint a bit 2020-04-22 01:37:34 +02:00
I-Al-Istannen
8891041069 WIP: crawler: Add opencast video crawler 2020-04-21 23:01:19 +02:00
I-Al-Istannen
70d63e3e90 WIP: Start small ILIAS crawler 2020-04-21 13:32:03 +02:00
I-Al-Istannen
b2a7af2e3e Store modification_date in IliasDownloadInfo, remove parameters 2020-04-21 13:31:50 +02:00
I-Al-Istannen
23bed48c8c Satisfy autopep8 2020-04-21 13:30:42 +02:00
Joscha
0926d33798 Use downloader-specific data classes 2020-04-20 18:07:45 +00:00
Joscha
4ef0ffe3bf Listen to pylint and mypy 2020-04-20 17:44:58 +00:00
I-Al-Istannen
ed9245c14d Remove old organizer 2020-04-20 18:50:23 +02:00
I-Al-Istannen
01e6972c96 Add ilias downloader 2020-04-20 18:49:01 +02:00
Joscha
6407190ae0 Soupify requests responses properly 2020-04-20 16:38:30 +00:00
Joscha
154d6b29dd Listen to pylint 2020-04-20 15:16:22 +00:00
Joscha
879a2c7c80 Rewrite ILIAS authenticator 2020-04-20 14:26:30 +00:00