52fdeae752
Crawl custom item groups as folders
2024-10-21 23:43:48 +02:00
f9bb2e41cf
Sanitize slashes in exercise container names
2024-10-21 22:30:16 +02:00
c897d9e2f5
Support finding entries for course overview page
...
Related to issue #93
2024-06-26 16:54:07 +02:00
b01f093474
fix: Element detection for other universities
...
Other universities might use other URL schemes
for different element types
2024-05-04 17:52:06 +02:00
3e831c7e23
Fix normalization of meeting names in cards
2024-04-24 22:32:26 +02:00
3db186a978
Fix personal desktop crawling HTML warnings
2024-04-10 11:15:25 +02:00
4a5959fd58
Fix personal desktop crawling without favorites
2024-04-10 11:15:25 +02:00
eb0c956d32
Add compatibility with ILIAS 8
2024-04-05 19:08:05 +02:00
266812f90e
Move is_logged_in helper to kit_ilias_html
2023-11-16 11:19:20 +01:00
50b50513c6
Ignore SCORM learning modules
2023-08-29 13:51:19 +02:00
df3514cd03
Crawl paginated past meetings
2023-08-29 12:41:21 +02:00
87b67e9271
Crawl files in the info tab
2023-08-29 12:41:15 +02:00
b54b3b979c
Remove size suffix for content pages
2023-08-27 11:43:05 +02:00
2184ac8040
Add support for ILIAS mediacast listings
2023-08-27 11:43:05 +02:00
68c398f1fe
Add support for ILIAS learning modules
2023-08-02 13:34:54 +02:00
d204dac8ce
Detect unexpected root page redirects and abort operation
2023-07-29 18:36:33 +02:00
6f30c6583d
Fix crawling of cards without descriptions
2023-03-21 23:52:33 +01:00
467fc526e8
Fix crawling of file/video cards
2023-03-21 23:52:24 +01:00
722d2eb393
Fix crawling of courses with preselected timeline tab
2023-03-21 23:36:47 +01:00
55a2de6b88
Fix crawling English opencast
2022-11-29 18:13:56 +01:00
c020cccc64
Include found paths in "second path found" warning
2022-10-29 14:08:29 +02:00
1b6be6bd79
Handle content pages in cards
2022-10-24 18:37:26 +02:00
e1430e6298
Handle (and ignore) surveys
2022-10-24 18:37:26 +02:00
5fdd40204b
Unwrap future meetings when ILIAS hides them behind a pagination
2022-10-24 14:33:58 +02:00
d72fc2760b
Handle empty forums
2022-10-24 13:12:17 +02:00
4a51aaa4f5
Fix forum crawling crashing for empty threads
2022-10-19 22:59:33 +02:00
46fb782798
Add forum crawling
...
This downloads all forum posts when needed and saves each thread in its
own html file, named after the thread title.
2022-05-24 23:43:53 +02:00
846c29aee1
Download page descriptions
2022-05-11 21:16:56 +02:00
a5015fe9b1
Correctly parse day-only meeting dates
...
I failed to recognize the correct format in the previous adjustment, so
this (hopefully) fixes it for good.
Meetings apparently don't always have a time portion.
2022-05-08 23:22:26 +02:00
bcc537468c
Fix crawling of expanded meetings
...
The last meeting on every page is expanded by default.
Its content is then shown inline *and* in the meeting page itself.
We should skip the inline content.
2022-05-05 22:53:37 +02:00
694ffb4d77
Fix meeting date parsing
...
Apparently the new pattern "<relative time qualifier>: <date>," was
added. This patch adds support for it.
2022-05-05 22:28:30 +02:00
7872fe5221
Fix tables with more columns than expected
2022-01-18 22:38:48 +01:00
4ee919625d
Add rudimentary support for content pages
2022-01-08 20:47:35 +01:00
43c5453e10
Correctly crawl files on desktop
...
The files on the desktop do not include a download link, so we need to
rewrite it.
2022-01-08 20:00:53 +01:00
5f527bc697
Remove Python 3.9 Pattern typehints
2022-01-08 17:14:40 +01:00
ced8b9a2d0
Fix some accordions
2022-01-08 16:58:30 +01:00
6f3cfd4396
Fix personal desktop crawling
2022-01-08 16:58:15 +01:00
a99356f2a2
Fix video stream extraction
2022-01-08 00:27:34 +01:00
e42ab83d32
Add support for ILIAS cards
2021-10-30 18:13:44 +02:00
f9a3f9b9f2
Handle multi-stream videos
2021-10-30 18:12:29 +02:00
ee67f9f472
Sort elements by ILIAS id to ensure deterministic ordering
2021-07-06 17:45:48 +02:00
8ec3f41251
Crawl ilias booking objects as links
2021-07-06 16:15:25 +02:00
6e4d423c81
Crawl all video stages in one crawl bar
...
This ensures folders are not renamed, as they are crawled twice
2021-06-13 17:18:45 +02:00
70ec64a48b
Fix wrong base URL for multi-stage pages
2021-06-13 15:44:47 +02:00
8ab462fb87
Use the exercise label instead of the button name as path
2021-06-04 19:24:23 +02:00
1fba96abcb
Fix exercise date parsing for non-group submissions
...
ILIAS apparently changes the order of the fields as it sees fit, so we
now try to parse *every* column, starting at from the right, as a date.
The first column that parses successfully is then used.
2021-05-31 18:15:12 +02:00
1ca6740e05
Improve log messages when parsing ILIAS HTML
...
Previously some logs were split around an "await", which isn't a great
idea.
2021-05-27 17:59:22 +02:00
5beb4d9a2d
Fix renaming conflict with multi-stage video elements
2021-05-27 15:41:00 +02:00
aabce764ac
Clean up TODOs
2021-05-25 15:54:01 +02:00
651b087932
Use cl/dl deduplication mechanism for ILIAS crawler
2021-05-25 12:15:38 +02:00
