mirrors/pferd
1
0
Fork 0
mirror of https://github.com/Garmelon/PFERD.git synced 2023-12-21 10:23:01 +01:00
Code Issues Packages Projects Releases Wiki Activity

Commit Graph

  • ced8b9a2d0 Fix some accordions I-Al-Istannen 2022-01-08 16:58:30 +01:00
  • 6f3cfd4396 Fix personal desktop crawling I-Al-Istannen 2022-01-08 16:58:15 +01:00
  • 462d993fbc Fix local video path cache (hopefully) I-Al-Istannen 2022-01-08 00:27:48 +01:00
  • a99356f2a2 Fix video stream extraction I-Al-Istannen 2022-01-08 00:27:34 +01:00
  • eac2e34161 Fix is_logged_in for ILIAS 7 I-Al-Istannen 2022-01-07 23:32:31 +01:00
  • a82a0b19c2 Collect crawler warnings/errors and include them in the report I-Al-Istannen 2021-11-07 21:40:22 +01:00
  • 90cb6e989b Do not download single videos if cache does not exist I-Al-Istannen 2021-11-06 23:20:24 +01:00
  • 6289938d7c Do not stop crawling files when encountering a CrawlWarning I-Al-Istannen 2021-11-06 12:09:51 +01:00
  • 13b8c3d9c6 Add regex option to config and CLI parser I-Al-Istannen 2021-11-02 09:30:46 +01:00
  • 88afe64a92 Refactor IPD crawler a bit I-Al-Istannen 2021-11-01 10:43:13 +01:00
  • 6b2a657573 Fix IPD crawler for different subpages (#42) Julius Rüberg 2021-11-01 10:09:50 +01:00
  • d6f38a61e1 Fixed minor spelling mistakes Toorero 2021-10-25 21:34:51 +02:00
  • ad3f4955f7 Update changelog I-Al-Istannen 2021-10-30 18:14:39 +02:00
  • e42ab83d32 Add support for ILIAS cards I-Al-Istannen 2021-10-25 11:07:25 +02:00
  • f9a3f9b9f2 Handle multi-stream videos I-Al-Istannen 2021-10-30 18:12:29 +02:00
  • ef7d5ea2d3 Allow storing crawler-specific data in reports I-Al-Istannen 2021-10-30 18:09:05 +02:00
  • 55ea304ff3 Disable interpolation of ConfigParser lukasprobst 2021-10-25 22:32:54 +02:00
  • fee12b3d9e Fix changelog Joscha 2021-10-25 17:44:12 +00:00
  • 6673077397 Add kit-ipd crawler I-Al-Istannen 2021-10-21 12:01:41 +02:00
  • 742632ed8d Bump version to 3.2.0 v3.2.0 Joscha 2021-08-04 18:27:26 +00:00
  • 544d45cbc5 Catch non-critical exceptions at crawler top level Joscha 2021-07-13 15:42:11 +02:00
  • 86f79ff1f1 Update changelog Joscha 2021-07-07 14:26:20 +02:00
  • ee67f9f472 Sort elements by ILIAS id to ensure deterministic ordering I-Al-Istannen 2021-07-06 17:45:12 +02:00
  • 8ec3f41251 Crawl ilias booking objects as links I-Al-Istannen 2021-07-06 16:13:23 +02:00
  • 89be07d4d3 Use final crawl path in HTML parsing message I-Al-Istannen 2021-07-03 17:05:48 +02:00
  • 91200f3684 Fix nondeterministic name deduplication I-Al-Istannen 2021-07-03 12:07:18 +02:00
  • 9ffd603357 Error when using multiple segments with -name-> Joscha 2021-07-01 11:14:50 +02:00
  • 80eeb8fe97 Add --skip option Joscha 2021-07-01 11:01:55 +02:00
  • 75fde870c2 Bump version to 3.1.0 v3.1.0 Joscha 2021-06-13 17:23:18 +02:00
  • 6e4d423c81 Crawl all video stages in one crawl bar I-Al-Istannen 2021-06-13 16:50:29 +02:00
  • 57aef26217 Fix name arrows Joscha 2021-06-13 16:32:22 +02:00
  • 70ec64a48b Fix wrong base URL for multi-stage pages I-Al-Istannen 2021-06-13 15:39:22 +02:00
  • 70b33ecfd9 Add migration notes to changelog Joscha 2021-06-13 15:06:50 +02:00
  • 601e4b936b Use new arrow logic in README example config Joscha 2021-06-12 15:00:52 +02:00
  • a292c4c437 Add example for ">>" arrow heads Joscha 2021-06-12 14:57:29 +02:00
  • bc65ea7ab6 Fix mypy complaining about missing type hints Joscha 2021-06-09 22:35:55 +02:00
  • f28bbe6b0c Update transform rule documentation Joscha 2021-06-09 22:22:40 +02:00
  • 61d902d715 Overhaul transform logic Joscha 2021-06-09 17:42:38 +02:00
  • 8ab462fb87 Use the exercise label instead of the button name as path I-Al-Istannen 2021-06-04 19:23:33 +02:00
  • df3ad3d890 Add 'skip' option to crawlers Joscha 2021-06-04 18:33:02 +02:00
  • fc31100a0f Always use '/' as path separator for regex rules Joscha 2021-06-04 18:02:45 +02:00
  • 31b6311e99 Remove incorrect tmp file explain message Joscha 2021-06-01 19:02:55 +02:00
  • 1fc8e9eb7a Document credential file authenticator config options Joscha 2021-06-01 10:00:59 +00:00
  • 85b9f45085 Bump version to 3.0.1 v3.0.1 Joscha 2021-06-01 09:49:30 +00:00
  • f656e3ff34 Fix credential parsing Joscha 2021-06-01 09:18:17 +00:00
  • e1bda94329 Load credential file from correct path Joscha 2021-06-01 09:18:08 +00:00
  • f6b26f4ead Fix unexpected exception when credential file not found Joscha 2021-06-01 09:10:58 +00:00
  • 722970a255 Store cookies in text-based format Joscha 2021-05-31 20:04:56 +00:00
  • f40820c41f Warn if using concurrent tasks with kit-ilias-web Joscha 2021-05-31 21:07:13 +02:00
  • 49ad1b6e46 Clean up authenticator code formatting Joscha 2021-05-31 18:21:18 +02:00
  • 1ce32d2f18 Add CLI option for credential file auth to kit-ilias-web Joscha 2021-05-31 18:19:05 +02:00
  • 9d5ec84b91 Add credential file authenticator Joscha 2021-05-31 17:55:56 +02:00
  • 1fba96abcb Fix exercise date parsing for non-group submissions I-Al-Istannen 2021-05-31 18:00:42 +02:00
  • 921cec7ddc Bump version to 3.0.0 v3.0.0 Joscha 2021-05-31 12:49:04 +02:00
  • 7b062883f6 Use raw paths for --debug-transforms Joscha 2021-05-31 12:28:11 +02:00
  • 64a2960751 Align paths in status messages and progress bars Joscha 2021-05-31 12:21:05 +02:00
  • 17879a7f69 Print box around message for unexpected exceptions Joscha 2021-05-31 11:50:20 +02:00
  • 1dd24551a5 Add link to repo in --version output Joscha 2021-05-31 11:44:17 +02:00
  • 84f775013f Use event loop workaround only on windows Joscha 2021-05-31 11:41:20 +02:00
  • b78eb64f3d Document versioning scheme Joscha 2021-05-29 21:38:36 +02:00
  • d65efed561 Slightly adjust phrasing Joscha 2021-05-28 21:21:04 +00:00
  • 1ca6740e05 Improve log messages when parsing ILIAS HTML I-Al-Istannen 2021-05-27 17:59:22 +02:00
  • 474aa7e1cc Use sorted path order when debugging transforms Joscha 2021-05-27 15:41:00 +00:00
  • 5beb4d9a2d Fix renaming conflict with multi-stage video elements I-Al-Istannen 2021-05-27 00:55:46 +02:00
  • 19eed5bdff Fix authentication logic conflicts with videos I-Al-Istannen 2021-05-27 00:31:36 +02:00
  • 6fa9cfd4c3 Fix error when capturing group is None Joscha 2021-05-27 13:56:01 +02:00
  • 80acc4b50d Implement new name arrows Joscha 2021-05-27 13:42:49 +02:00
  • 2c72a9112c Reword -name-> and -name-re-> docs and remove -name-exact-> Joscha 2021-05-27 13:20:37 +02:00
  • 17207546e9 Document --debug-transforms Joscha 2021-05-26 11:47:51 +02:00
  • 533f75ea71 Add --debug-transforms flag Joscha 2021-05-26 11:37:32 +02:00
  • adb5d4ade3 Print files that are *not* deleted by cleanup Joscha 2021-05-26 10:58:19 +02:00
  • a879c6ab6e Fix function being printed Joscha 2021-05-26 10:52:04 +02:00
  • 915e42fd07 Fix report not being printed if pferd exits normally Joscha 2021-05-26 10:51:41 +02:00
  • 2d8dcc87ff Send CSRF token in TFA request I-Al-Istannen 2021-05-25 19:23:06 +02:00
  • 66f0e398a1 Await result in tfa authenticate path I-Al-Istannen 2021-05-25 19:19:51 +02:00
  • 30be4e29fa Add workaround for RuntimeError after program finishes on Windows Joscha 2021-05-25 16:34:18 +00:00
  • 263780e6a3 Use certifi to ensure CA certificates are bundled in pyinstaller I-Al-Istannen 2021-05-25 18:09:51 +02:00
  • 07a75a37c3 Fix FileNotFoundError on Windows Joscha 2021-05-25 15:57:03 +00:00
  • f85b75df8c Switch from exit() to sys.exit() Joscha 2021-05-25 17:33:05 +02:00
  • 6644126b5d Fix package discovery Joscha 2021-05-25 17:29:39 +02:00
  • c665c36d88 Update README, CHANGELOG Joscha 2021-05-25 17:16:57 +02:00
  • 519a7ef435 Split --dump-config into two options Joscha 2021-05-25 16:57:14 +02:00
  • a848194601 Rename plaintext link option to "plaintext" I-Al-Istannen 2021-05-25 17:15:13 +02:00
  • aabce764ac Clean up TODOs Joscha 2021-05-25 15:54:01 +02:00
  • 5a331663e4 Rename functions for consistency Joscha 2021-05-25 15:49:06 +02:00
  • 40144f8bd8 Fix rule error messages Joscha 2021-05-25 15:47:09 +02:00
  • f68849c65f Fix rules not being parsed entirely Joscha 2021-05-25 15:42:46 +02:00
  • edb52a989e Print report even if exiting due to Ctrl+C Joscha 2021-05-25 15:35:36 +02:00
  • 980578d05a Avoid downloading in some cases Joscha 2021-05-25 15:12:07 +02:00
  • 486699cef3 Create anonymous TFA authenticator in ilias crawler I-Al-Istannen 2021-05-25 15:11:52 +02:00
  • 0096a0c077 Remove section and config parameter from Authenticator I-Al-Istannen 2021-05-25 15:11:33 +02:00
  • d905e95dbb Allow invalidation of keyring authenticator I-Al-Istannen 2021-05-25 15:02:35 +02:00
  • 61430c8739 Overhaul config and CLI option names Joscha 2021-05-25 14:12:19 +02:00
  • eb8b915813 Fix path prefix on windows Joscha 2021-05-25 14:21:52 +02:00
  • 22c2259adb Clean up authenticator exceptions Joscha 2021-05-25 14:21:12 +02:00
  • c15a1aecdf Rename keyring authenticator file for consistency Joscha 2021-05-25 13:32:00 +02:00
  • 16d50b6626 Document why /pferd.py exists Joscha 2021-05-25 13:31:29 +02:00
  • 651b087932 Use cl/dl deduplication mechanism for ILIAS crawler I-Al-Istannen 2021-05-25 12:15:38 +02:00
  • bce3dc384d Deduplicate path names in crawler Joscha 2021-05-25 11:58:01 +02:00
  • c21ddf225b Add a CLI option to configure ILIAS links behaviour I-Al-Istannen 2021-05-25 11:58:41 +02:00