I-Al-Istannen
135a8dce4b
Fix resolve_path allowing paths outside its folder
...
This happened if the directory name was a prefix of the offending file name.
2020-04-20 16:07:14 +02:00
I-Al-Istannen
63bbcad918
Add resolve method to tmp_dir
2020-04-20 15:40:07 +02:00
I-Al-Istannen
6584d6a905
Elaborate accept_file in new_organizer
2020-04-20 15:40:07 +02:00
Joscha
5990098ef8
Add UserPassAuthenticator
2020-04-20 13:26:45 +00:00
I-Al-Istannen
f3d3d6bb65
Add some docs to cookie_jar
2020-04-20 14:38:03 +02:00
I-Al-Istannen
b2fe7cc064
Add preliminary logging to organizer and tmp_dir
2020-04-20 14:37:44 +02:00
I-Al-Istannen
930d821dd7
Add a simple organizer
2020-04-20 14:29:48 +02:00
I-Al-Istannen
5c2ff14839
Add "prompt_yes_no" to utils
2020-04-20 14:29:48 +02:00
I-Al-Istannen
a3d6dc7873
Clean up temp_folder
2020-04-20 14:29:48 +02:00
Joscha
53ad1c924b
Add cookie jar
2020-04-20 11:35:26 +00:00
I-Al-Istannen
8c431c7d81
Add a simple temporary folder
2020-04-20 12:08:52 +02:00
Joscha
d5dd5aac06
Fix some mypy errors
2020-04-20 01:54:47 +00:00
Joscha
25043a4aaa
Remove unnecessary files
...
Also document some plans for the new program structure in REWRITE.md
2020-04-19 19:49:43 +00:00
I-Al-Istannen
cf3553175f
Add OS_Exams synchronizer
2020-02-27 14:51:29 +01:00
I-Al-Istannen
bf8b3cf9f7
Hack in support for TI exams
...
This just adds an additional crawl check for AlteKlausuren. This is not
present on the root site but at the suffix `/Klausuren`.
Example config:
```py
# The "Klausur" needs to be copied verbatim!
ti.synchronize("Klausur", "sync dir name",
transform=ro_19_klausur_transform, filter=ro_19_klausur_filter)
```
2020-02-24 20:58:27 +01:00
I-Al-Istannen
f5bc49160f
Lose 50 minutes of my life (and fix the TGI tut)
2019-12-12 12:50:16 +01:00
I-Al-Istannen
4433696509
[TGI] Add TGi tut
2019-11-18 09:58:16 +01:00
I-Al-Istannen
1407c6d264
Download all TGI files and not just lectures
2019-10-17 22:14:32 +02:00
I-Al-Istannen
1973c931bd
Add support for other years in TGI downloader
2019-10-15 15:37:52 +02:00
I-Al-Istannen
458cc1c6d6
Add support for TGI website
2019-10-15 15:34:59 +02:00
Joscha
f94629a7fa
Fix exceptions with weird content types
...
(hopefully)
2019-09-22 11:55:47 +00:00
I-Al-Istannen
2752e98621
Fix relative url joining in ti downloader
2019-07-26 10:06:01 +02:00
Joscha
ea01dc7cb2
Allow even more types of files
2019-07-05 08:48:43 +00:00
Joscha
77056e6f8d
Allow more types of files
2019-07-04 12:16:42 +00:00
Joscha
d468a45662
Allow wolfram files
2019-06-11 12:42:55 +00:00
I-Al-Istannen
67da4e69fa
Add colorful log output
...
Highlight the important operations (new, modified) in different colours.
2019-06-07 13:28:55 +02:00
Joscha
2016f61bf8
Crawl more of the TI page
2019-05-09 11:04:24 +00:00
Joscha
c72e92db18
Make Ti downloader authentication more robust
2019-05-06 12:04:01 +00:00
Joscha
44b4204517
Add basic Ti downloader
2019-05-06 11:54:36 +00:00
Joscha
d730d0064c
Conform to other files' __all__
2019-04-26 09:45:24 +00:00
Joscha
ae6cc40fb5
Rename ILIAS crawler to ilias
...
To be consistent with the other classes' capitalisation of acronyms
2019-04-26 04:29:12 +00:00
Joscha
0891e7f1bc
Fix logging messages not appearing
2019-04-26 03:58:11 +00:00
Joscha
9693e1d968
Make logging easier
2019-04-25 19:53:13 +00:00
Joscha
f1ba618378
Remove unnecessary files
2019-04-25 19:18:19 +00:00
Joscha
dfddc93039
Move norbert from aiohttp to requests
...
Also fix streaming (when downloading) in the other classes.
2019-04-25 19:15:36 +00:00
Joscha
f0c42ce8ec
Clean up
...
Use shorter name for responses, like in the requests doc.
Change Organizer's __all__ to be more in line with the other __all__s.
2019-04-25 19:02:48 +00:00
Joscha
82adeb324f
Move ffm stuff from aiohttp to requests
2019-04-25 19:01:53 +00:00
Joscha
9bae030186
Move ilias stuff from aiohttp to requests
2019-04-25 18:52:48 +00:00
Joscha
c7a9a42b3d
Allow files of type application/msword
2019-04-24 12:34:50 +00:00
Joscha
5a1bf2188b
Switch from tabs to spaces
2019-04-24 12:34:20 +00:00
Joscha
3019e4255b
Replace "/" in file names with "."
2018-12-14 09:27:12 +00:00
Joscha
616a8d96a2
Sort norbert files while downloading
2018-12-05 11:44:35 +00:00
Joscha
2d9223b8e6
Add norbert synchronizer
2018-11-29 10:26:58 +00:00
Joscha
bdc0e8ad03
Remember files correctly for cleanin up
2018-11-28 08:59:07 +00:00
Joscha
dad33b8c7f
Save identically named files under different names
2018-11-27 17:23:32 +00:00
Joscha
98a2b5db34
Fix tut crawling
2018-11-27 10:28:39 +00:00
Joscha
c824ae4f6d
Add more allowed file types
2018-11-27 10:27:20 +00:00
Joscha
8b1a34233a
Add and use utility functions for changing paths
...
This fixes a small bug in the example config, where some files were
put in the wrong locations.
2018-11-27 08:52:27 +00:00
Joscha
a084b05433
Change log message
...
for better readability
2018-11-26 17:33:27 +00:00
Joscha
068fe77dcf
Clean up minor things
...
- improve logging messages
- allow more download file formats
- strip file names
2018-11-26 17:00:17 +00:00