mirrors
/
pferd
mirror of https://github.com/Garmelon/PFERD.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
673 Commits 6 Branches 43 Tags 1.6 MiB
Commit Graph

673 Commits

Author SHA1 Message Date
Joscha 474aa7e1cc Use sorted path order when debugging transforms 2021-05-27 15:41:00 +00:00
I-Al-Istannen 5beb4d9a2d Fix renaming conflict with multi-stage video elements 2021-05-27 15:41:00 +02:00
I-Al-Istannen 19eed5bdff Fix authentication logic conflicts with videos 2021-05-27 15:41:00 +02:00
Joscha 6fa9cfd4c3 Fix error when capturing group is None 2021-05-27 15:41:00 +02:00
Joscha 80acc4b50d Implement new name arrows 2021-05-27 13:43:02 +02:00
Joscha 2c72a9112c Reword `-name->` and `-name-re->` docs and remove `-name-exact->` 2021-05-27 13:20:37 +02:00
Joscha 17207546e9 Document --debug-transforms 2021-05-26 11:47:51 +02:00
Joscha 533f75ea71 Add --debug-transforms flag 2021-05-26 11:37:32 +02:00
Joscha adb5d4ade3 Print files that are *not* deleted by cleanup 
These are files that are not present on the remote source any more, but still
present locally. They also show up in the report.
 2021-05-26 10:58:19 +02:00
Joscha a879c6ab6e Fix function being printed 2021-05-26 10:54:01 +02:00
Joscha 915e42fd07 Fix report not being printed if pferd exits normally 2021-05-26 10:53:54 +02:00
I-Al-Istannen 2d8dcc87ff Send CSRF token in TFA request 2021-05-25 22:50:40 +02:00
I-Al-Istannen 66f0e398a1 Await result in tfa authenticate path 2021-05-25 19:19:51 +02:00
Joscha 30be4e29fa Add workaround for RuntimeError after program finishes on Windows 2021-05-25 16:34:22 +00:00
I-Al-Istannen 263780e6a3 Use certifi to ensure CA certificates are bundled in pyinstaller 2021-05-25 18:24:06 +02:00
Joscha 07a75a37c3 Fix FileNotFoundError on Windows 2021-05-25 15:57:03 +00:00
Joscha f85b75df8c Switch from exit() to sys.exit() 
Pyinstaller doesn't recognize exit().
 2021-05-25 17:33:38 +02:00
Joscha 6644126b5d Fix package discovery 2021-05-25 17:29:39 +02:00
Joscha c665c36d88 Update README, CHANGELOG 2021-05-25 17:18:31 +02:00
Joscha 519a7ef435 Split --dump-config into two options 
--dump-config with its optional argument tended to consume the command name, so
it had to be split up.
 2021-05-25 17:17:35 +02:00
I-Al-Istannen a848194601 Rename plaintext link option to "plaintext" 2021-05-25 17:15:13 +02:00
Joscha aabce764ac Clean up TODOs 2021-05-25 15:54:01 +02:00
Joscha 5a331663e4 Rename functions for consistency 2021-05-25 15:49:06 +02:00
Joscha 40144f8bd8 Fix rule error messages 2021-05-25 15:47:09 +02:00
Joscha f68849c65f Fix rules not being parsed entirely 2021-05-25 15:42:46 +02:00
Joscha edb52a989e Print report even if exiting due to Ctrl+C 2021-05-25 15:35:36 +02:00
Joscha 980578d05a Avoid downloading in some cases 
Depending on how on_conflict is set, we can determine a few situations where
downloading is never necessary.
 2021-05-25 15:20:30 +02:00
I-Al-Istannen 486699cef3 Create anonymous TFA authenticator in ilias crawler 
This ensures that *some* TFA authenticator is always present when
authenticating, even if none is specified in the config.

The TfaAuthenticator does not depend on any configured values, so it can
be created on-demand.
 2021-05-25 15:11:52 +02:00
I-Al-Istannen 0096a0c077 Remove section and config parameter from Authenticator 2021-05-25 15:11:33 +02:00
I-Al-Istannen d905e95dbb Allow invalidation of keyring authenticator 2021-05-25 15:02:35 +02:00
Joscha 61430c8739 Overhaul config and CLI option names 2021-05-25 14:23:38 +02:00
Joscha eb8b915813 Fix path prefix on windows 
Previously, the path prefix was only set if "windows_paths" was true, regardless
of OS. Now the path prefix is always set on windows and never set on other OSes.
 2021-05-25 14:23:38 +02:00
Joscha 22c2259adb Clean up authenticator exceptions 
- Renamed to *Error for consistency
- Treating AuthError like CrawlError
 2021-05-25 14:23:38 +02:00
Joscha c15a1aecdf Rename keyring authenticator file for consistency 2021-05-25 14:20:26 +02:00
Joscha 16d50b6626 Document why /pferd.py exists 2021-05-25 13:31:29 +02:00
I-Al-Istannen 651b087932 Use cl/dl deduplication mechanism for ILIAS crawler 2021-05-25 12:15:38 +02:00
Joscha bce3dc384d Deduplicate path names in crawler 
Also rename files so they follow the restrictions for windows file names if
we're on windows.
 2021-05-25 12:11:15 +02:00
I-Al-Istannen c21ddf225b Add a CLI option to configure ILIAS links behaviour 2021-05-25 11:58:41 +02:00
I-Al-Istannen 4fefb98d71 Add a wrapper to pretty-print ValueErrors in argparse parsers 2021-05-25 11:57:59 +02:00
I-Al-Istannen ffda4e43df Add extension to link files 2021-05-25 11:41:57 +02:00
I-Al-Istannen 69cb2a7734 Add Links option to ilias crawler 
This allows you to configure what type the link files should have and
whether to create them at all.
 2021-05-25 11:41:57 +02:00
Joscha c33de233dc Add script for releasing new versions 2021-05-24 20:23:14 +02:00
I-Al-Istannen 85f89a7ff3 Interpret accordions and expandable headers as virtual folders 
This allows us to find a file named "Test" in an accordion "Acc" as "Acc/Test".
 2021-05-24 18:54:26 +02:00
I-Al-Istannen 9ce20216b5 Do not set a timeout for whole HTTP request 
Downloads might take longer!
 2021-05-24 18:54:26 +02:00
Joscha 1739c54091 Add checklist for releasing new versions 2021-05-24 17:50:17 +02:00
Joscha d8bd1f518a Set up build and release workflow 2021-05-24 17:27:39 +02:00
Joscha 86ba47541b Fix cookie loading and saving 2021-05-24 16:55:11 +02:00
I-Al-Istannen 492ec6a932 Detect and skip ILIAS tests 2021-05-24 16:36:15 +02:00
I-Al-Istannen 342076ee0e Handle exercise detail containers in ILIAS html parser 2021-05-24 16:22:51 +02:00
I-Al-Istannen d44f6966c2 Log authentication attempts in HTTP crawler 2021-05-24 16:22:11 +02:00