66f0e398a1
Await result in tfa authenticate path
2021-05-25 19:19:51 +02:00
30be4e29fa
Add workaround for RuntimeError after program finishes on Windows
2021-05-25 16:34:22 +00:00
263780e6a3
Use certifi to ensure CA certificates are bundled in pyinstaller
2021-05-25 18:24:06 +02:00
07a75a37c3
Fix FileNotFoundError on Windows
2021-05-25 15:57:03 +00:00
f85b75df8c
Switch from exit() to sys.exit()
...
Pyinstaller doesn't recognize exit().
2021-05-25 17:33:38 +02:00
519a7ef435
Split --dump-config into two options
...
--dump-config with its optional argument tended to consume the command name, so
it had to be split up.
2021-05-25 17:17:35 +02:00
a848194601
Rename plaintext link option to "plaintext"
2021-05-25 17:15:13 +02:00
aabce764ac
Clean up TODOs
2021-05-25 15:54:01 +02:00
5a331663e4
Rename functions for consistency
2021-05-25 15:49:06 +02:00
40144f8bd8
Fix rule error messages
2021-05-25 15:47:09 +02:00
f68849c65f
Fix rules not being parsed entirely
2021-05-25 15:42:46 +02:00
edb52a989e
Print report even if exiting due to Ctrl+C
2021-05-25 15:35:36 +02:00
980578d05a
Avoid downloading in some cases
...
Depending on how on_conflict is set, we can determine a few situations where
downloading is never necessary.
2021-05-25 15:20:30 +02:00
486699cef3
Create anonymous TFA authenticator in ilias crawler
...
This ensures that *some* TFA authenticator is always present when
authenticating, even if none is specified in the config.
The TfaAuthenticator does not depend on any configured values, so it can
be created on-demand.
2021-05-25 15:11:52 +02:00
0096a0c077
Remove section and config parameter from Authenticator
2021-05-25 15:11:33 +02:00
d905e95dbb
Allow invalidation of keyring authenticator
2021-05-25 15:02:35 +02:00
61430c8739
Overhaul config and CLI option names
2021-05-25 14:23:38 +02:00
eb8b915813
Fix path prefix on windows
...
Previously, the path prefix was only set if "windows_paths" was true, regardless
of OS. Now the path prefix is always set on windows and never set on other OSes.
2021-05-25 14:23:38 +02:00
22c2259adb
Clean up authenticator exceptions
...
- Renamed to *Error for consistency
- Treating AuthError like CrawlError
2021-05-25 14:23:38 +02:00
c15a1aecdf
Rename keyring authenticator file for consistency
2021-05-25 14:20:26 +02:00
651b087932
Use cl/dl deduplication mechanism for ILIAS crawler
2021-05-25 12:15:38 +02:00
bce3dc384d
Deduplicate path names in crawler
...
Also rename files so they follow the restrictions for windows file names if
we're on windows.
2021-05-25 12:11:15 +02:00
c21ddf225b
Add a CLI option to configure ILIAS links behaviour
2021-05-25 11:58:41 +02:00
4fefb98d71
Add a wrapper to pretty-print ValueErrors in argparse parsers
2021-05-25 11:57:59 +02:00
ffda4e43df
Add extension to link files
2021-05-25 11:41:57 +02:00
69cb2a7734
Add Links option to ilias crawler
...
This allows you to configure what type the link files should have and
whether to create them at all.
2021-05-25 11:41:57 +02:00
85f89a7ff3
Interpret accordions and expandable headers as virtual folders
...
This allows us to find a file named "Test" in an accordion "Acc" as "Acc/Test".
2021-05-24 18:54:26 +02:00
9ce20216b5
Do not set a timeout for whole HTTP request
...
Downloads might take longer!
2021-05-24 18:54:26 +02:00
86ba47541b
Fix cookie loading and saving
2021-05-24 16:55:11 +02:00
492ec6a932
Detect and skip ILIAS tests
2021-05-24 16:36:15 +02:00
342076ee0e
Handle exercise detail containers in ILIAS html parser
2021-05-24 16:22:51 +02:00
d44f6966c2
Log authentication attempts in HTTP crawler
2021-05-24 16:22:11 +02:00
1c1f781be4
Reword some log messages
2021-05-24 13:17:28 +02:00
c687d4a51a
Implement cookie sharing
2021-05-24 13:10:44 +02:00
fca62541ca
De-duplicate element names in ILIAS crawler
...
This prevents any conflicts caused by multiple files with the same name.
Conflicts may still arise due to transforms, but that is out of our
control and a user error.
2021-05-24 00:24:31 +02:00
3ab3581f84
Add timeout for HTTP connection
2021-05-23 23:41:05 +02:00
8dd0689420
Add keyring authentication to ILIAS CLI
2021-05-23 23:04:18 +02:00
79be6e1dc5
Switch some other options to BooleanOptionalAction
2021-05-23 22:49:09 +02:00
edbd92dbbf
Add --status and --report flags
2021-05-23 22:41:59 +02:00
27b5a8e490
Rename log.action to log.status
2021-05-23 22:40:33 +02:00
1f400d5964
Implement BooleanOptionalAction
2021-05-23 22:26:59 +02:00
0ca0680165
Simplify --version
2021-05-23 21:40:48 +02:00
ce1dbda5b4
Overhaul colours
...
"Crawled" and "Downloaded" are now printed less bright than "Crawling" and
"Downloading" as they're not as important. Explain topics are printed in yellow
to stand out a bit more from the cyan action messages.
2021-05-23 21:33:04 +02:00
9cce78669f
Print report after all crawlers have finished
2021-05-23 21:17:13 +02:00
6ca0ecdf05
Load and store reports
2021-05-23 20:46:29 +02:00
6e9f8fd391
Add a keyring authenticator
2021-05-23 19:44:12 +02:00
2fdf24495b
Restructure crawling and auth related modules
2021-05-23 19:16:42 +02:00
bbf9f8f130
Add -C as alias for --crawler
2021-05-23 19:06:09 +02:00
37f8d84a9c
Output total amount of http requests in HTTP Crawler
2021-05-23 19:00:01 +02:00
5edd868d5b
Fix always-smart redownloading the wrong files
2021-05-23 18:49:34 +02:00
