61 Commits

Author	SHA1	Message	Date
Joscha	ff06c5215e	Fix authenticator	2020-04-20 14:26:29 +00:00
I-Al-Istannen	135a8dce4b	Fix resolve_path allowing paths outside its folder ... This happened if the directory name was a prefix of the offending file name.	2020-04-20 16:07:14 +02:00
I-Al-Istannen	63bbcad918	Add resolve method to tmp_dir	2020-04-20 15:40:07 +02:00
I-Al-Istannen	6584d6a905	Elaborate accept_file in new_organizer	2020-04-20 15:40:07 +02:00
Joscha	5990098ef8	Add UserPassAuthenticator	2020-04-20 13:26:45 +00:00
I-Al-Istannen	f3d3d6bb65	Add some docs to cookie_jar	2020-04-20 14:38:03 +02:00
I-Al-Istannen	b2fe7cc064	Add preliminary logging to organizer and tmp_dir	2020-04-20 14:37:44 +02:00
I-Al-Istannen	930d821dd7	Add a simple organizer	2020-04-20 14:29:48 +02:00
I-Al-Istannen	5c2ff14839	Add "prompt_yes_no" to utils	2020-04-20 14:29:48 +02:00
I-Al-Istannen	a3d6dc7873	Clean up temp_folder	2020-04-20 14:29:48 +02:00
Joscha	53ad1c924b	Add cookie jar	2020-04-20 11:35:26 +00:00
I-Al-Istannen	8c431c7d81	Add a simple temporary folder	2020-04-20 12:08:52 +02:00
Joscha	d5dd5aac06	Fix some mypy errors	2020-04-20 01:54:47 +00:00
Joscha	25043a4aaa	Remove unnecessary files ... Also document some plans for the new program structure in REWRITE.md	2020-04-19 19:49:43 +00:00
I-Al-Istannen	cf3553175f	Add OS_Exams synchronizer	2020-02-27 14:51:29 +01:00
I-Al-Istannen	bf8b3cf9f7	Hack in support for TI exams ... This just adds an additional crawl check for AlteKlausuren. This is not present on the root site but at the suffix `/Klausuren`. Example config: ```py # The "Klausur" needs to be copied verbatim! ti.synchronize("Klausur", "sync dir name", transform=ro_19_klausur_transform, filter=ro_19_klausur_filter) ```	2020-02-24 20:58:27 +01:00
I-Al-Istannen	f5bc49160f	Lose 50 minutes of my life (and fix the TGI tut)	2019-12-12 12:50:16 +01:00
I-Al-Istannen	4433696509	[TGI] Add TGi tut	2019-11-18 09:58:16 +01:00
I-Al-Istannen	1407c6d264	Download all TGI files and not just lectures	2019-10-17 22:14:32 +02:00
I-Al-Istannen	1973c931bd	Add support for other years in TGI downloader	2019-10-15 15:37:52 +02:00
I-Al-Istannen	458cc1c6d6	Add support for TGI website	2019-10-15 15:34:59 +02:00
Joscha	f94629a7fa	Fix exceptions with weird content types ... (hopefully)	2019-09-22 11:55:47 +00:00
I-Al-Istannen	2752e98621	Fix relative url joining in ti downloader	2019-07-26 10:06:01 +02:00
Joscha	ea01dc7cb2	Allow even more types of files	2019-07-05 08:48:43 +00:00
Joscha	77056e6f8d	Allow more types of files	2019-07-04 12:16:42 +00:00
Joscha	d468a45662	Allow wolfram files	2019-06-11 12:42:55 +00:00
I-Al-Istannen	67da4e69fa	Add colorful log output ... Highlight the important operations (new, modified) in different colours.	2019-06-07 13:28:55 +02:00
Joscha	2016f61bf8	Crawl more of the TI page	2019-05-09 11:04:24 +00:00
Joscha	c72e92db18	Make Ti downloader authentication more robust	2019-05-06 12:04:01 +00:00
Joscha	44b4204517	Add basic Ti downloader	2019-05-06 11:54:36 +00:00
Joscha	d730d0064c	Conform to other files' __all__	2019-04-26 09:45:24 +00:00
Joscha	ae6cc40fb5	Rename ILIAS crawler to ilias ... To be consistent with the other classes' capitalisation of acronyms	2019-04-26 04:29:12 +00:00
Joscha	0891e7f1bc	Fix logging messages not appearing	2019-04-26 03:58:11 +00:00
Joscha	9693e1d968	Make logging easier	2019-04-25 19:53:13 +00:00
Joscha	f1ba618378	Remove unnecessary files	2019-04-25 19:18:19 +00:00
Joscha	dfddc93039	Move norbert from aiohttp to requests ... Also fix streaming (when downloading) in the other classes.	2019-04-25 19:15:36 +00:00
Joscha	f0c42ce8ec	Clean up ... Use shorter name for responses, like in the requests doc. Change Organizer's __all__ to be more in line with the other __all__s.	2019-04-25 19:02:48 +00:00
Joscha	82adeb324f	Move ffm stuff from aiohttp to requests	2019-04-25 19:01:53 +00:00
Joscha	9bae030186	Move ilias stuff from aiohttp to requests	2019-04-25 18:52:48 +00:00
Joscha	c7a9a42b3d	Allow files of type application/msword	2019-04-24 12:34:50 +00:00
Joscha	5a1bf2188b	Switch from tabs to spaces	2019-04-24 12:34:20 +00:00
Joscha	3019e4255b	Replace "/" in file names with "."	2018-12-14 09:27:12 +00:00
Joscha	616a8d96a2	Sort norbert files while downloading	2018-12-05 11:44:35 +00:00
Joscha	2d9223b8e6	Add norbert synchronizer	2018-11-29 10:26:58 +00:00
Joscha	bdc0e8ad03	Remember files correctly for cleanin up	2018-11-28 08:59:07 +00:00
Joscha	dad33b8c7f	Save identically named files under different names	2018-11-27 17:23:32 +00:00
Joscha	98a2b5db34	Fix tut crawling	2018-11-27 10:28:39 +00:00
Joscha	c824ae4f6d	Add more allowed file types	2018-11-27 10:27:20 +00:00
Joscha	8b1a34233a	Add and use utility functions for changing paths ... This fixes a small bug in the example config, where some files were put in the wrong locations.	2018-11-27 08:52:27 +00:00
Joscha	a084b05433	Change log message ... for better readability	2018-11-26 17:33:27 +00:00