5990098ef8
Add UserPassAuthenticator
2020-04-20 13:26:45 +00:00
f3d3d6bb65
Add some docs to cookie_jar
2020-04-20 14:38:03 +02:00
b2fe7cc064
Add preliminary logging to organizer and tmp_dir
2020-04-20 14:37:44 +02:00
930d821dd7
Add a simple organizer
2020-04-20 14:29:48 +02:00
5c2ff14839
Add "prompt_yes_no" to utils
2020-04-20 14:29:48 +02:00
a3d6dc7873
Clean up temp_folder
2020-04-20 14:29:48 +02:00
53ad1c924b
Add cookie jar
2020-04-20 11:35:26 +00:00
8c431c7d81
Add a simple temporary folder
2020-04-20 12:08:52 +02:00
d5dd5aac06
Fix some mypy errors
2020-04-20 01:54:47 +00:00
25043a4aaa
Remove unnecessary files
...
Also document some plans for the new program structure in REWRITE.md
2020-04-19 19:49:43 +00:00
cf3553175f
Add OS_Exams synchronizer
2020-02-27 14:51:29 +01:00
bf8b3cf9f7
Hack in support for TI exams
...
This just adds an additional crawl check for AlteKlausuren. This is not
present on the root site but at the suffix `/Klausuren`.
Example config:
```py
# The "Klausur" needs to be copied verbatim!
ti.synchronize("Klausur", "sync dir name",
transform=ro_19_klausur_transform, filter=ro_19_klausur_filter)
```
2020-02-24 20:58:27 +01:00
f5bc49160f
Lose 50 minutes of my life (and fix the TGI tut)
2019-12-12 12:50:16 +01:00
4433696509
[TGI] Add TGi tut
2019-11-18 09:58:16 +01:00
1407c6d264
Download all TGI files and not just lectures
2019-10-17 22:14:32 +02:00
1973c931bd
Add support for other years in TGI downloader
2019-10-15 15:37:52 +02:00
458cc1c6d6
Add support for TGI website
2019-10-15 15:34:59 +02:00
f94629a7fa
Fix exceptions with weird content types
...
(hopefully)
2019-09-22 11:55:47 +00:00
2752e98621
Fix relative url joining in ti downloader
2019-07-26 10:06:01 +02:00
ea01dc7cb2
Allow even more types of files
2019-07-05 08:48:43 +00:00
77056e6f8d
Allow more types of files
2019-07-04 12:16:42 +00:00
d468a45662
Allow wolfram files
2019-06-11 12:42:55 +00:00
67da4e69fa
Add colorful log output
...
Highlight the important operations (new, modified) in different colours.
2019-06-07 13:28:55 +02:00
2016f61bf8
Crawl more of the TI page
2019-05-09 11:04:24 +00:00
c72e92db18
Make Ti downloader authentication more robust
2019-05-06 12:04:01 +00:00
44b4204517
Add basic Ti downloader
2019-05-06 11:54:36 +00:00
d730d0064c
Conform to other files' __all__
2019-04-26 09:45:24 +00:00
ae6cc40fb5
Rename ILIAS crawler to ilias
...
To be consistent with the other classes' capitalisation of acronyms
2019-04-26 04:29:12 +00:00
0891e7f1bc
Fix logging messages not appearing
2019-04-26 03:58:11 +00:00
9693e1d968
Make logging easier
2019-04-25 19:53:13 +00:00
f1ba618378
Remove unnecessary files
2019-04-25 19:18:19 +00:00
dfddc93039
Move norbert from aiohttp to requests
...
Also fix streaming (when downloading) in the other classes.
2019-04-25 19:15:36 +00:00
f0c42ce8ec
Clean up
...
Use shorter name for responses, like in the requests doc.
Change Organizer's __all__ to be more in line with the other __all__s.
2019-04-25 19:02:48 +00:00
82adeb324f
Move ffm stuff from aiohttp to requests
2019-04-25 19:01:53 +00:00
9bae030186
Move ilias stuff from aiohttp to requests
2019-04-25 18:52:48 +00:00
c7a9a42b3d
Allow files of type application/msword
2019-04-24 12:34:50 +00:00
5a1bf2188b
Switch from tabs to spaces
2019-04-24 12:34:20 +00:00
3019e4255b
Replace "/" in file names with "."
2018-12-14 09:27:12 +00:00
616a8d96a2
Sort norbert files while downloading
2018-12-05 11:44:35 +00:00
2d9223b8e6
Add norbert synchronizer
2018-11-29 10:26:58 +00:00
bdc0e8ad03
Remember files correctly for cleanin up
2018-11-28 08:59:07 +00:00
dad33b8c7f
Save identically named files under different names
2018-11-27 17:23:32 +00:00
98a2b5db34
Fix tut crawling
2018-11-27 10:28:39 +00:00
c824ae4f6d
Add more allowed file types
2018-11-27 10:27:20 +00:00
8b1a34233a
Add and use utility functions for changing paths
...
This fixes a small bug in the example config, where some files were
put in the wrong locations.
2018-11-27 08:52:27 +00:00
a084b05433
Change log message
...
for better readability
2018-11-26 17:33:27 +00:00
068fe77dcf
Clean up minor things
...
- improve logging messages
- allow more download file formats
- strip file names
2018-11-26 17:00:17 +00:00
34da5d4d19
Sync files from ILIAS
2018-11-26 13:39:06 +00:00
529c4a7dda
Don't overwrite files if the contents match
2018-11-26 13:37:01 +00:00
2034c9d426
Add FfM (Fachschaft für Mathematik) synchronizer
...
This commit moves exceptions and some other things into utils.py and
renames files according to python's file naming guides (kinda).
It also adds a new example config using the new FfM downloader.
2018-11-24 08:27:33 +00:00
