mirrors/pferd
1
0
Fork 0
mirror of https://github.com/Garmelon/PFERD.git synced 2023-12-21 10:23:01 +01:00
Code Issues Packages Projects Releases Wiki Activity

Fix forum crawling crashing for empty threads

Browse Source
This commit is contained in:
I-Al-Istannen 2022-10-19 22:59:33 +02:00
parent 66a5b1ba02
commit 4a51aaa4f5
2 changed files with 10 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
CHANGELOG.md
View File

@ -22,6 +22,9 @@ ambiguous situations.
## Unreleased ## Unreleased
### Fixed
- Forum crawling crashing when parsing empty (= 0 messages) threads
## 3.4.1 - 2022-08-17 ## 3.4.1 - 2022-08-17
### Added ### Added

7
PFERD/crawl/ilias/kit_ilias_html.py
View File

@ -937,6 +937,13 @@ def parse_ilias_forum_export(forum_export: BeautifulSoup) -> List[IliasForumThre
for p in forum_export.select("body > p"): for p in forum_export.select("body > p"):
title_tag = p title_tag = p
content_tag = p.find_next_sibling("ul") content_tag = p.find_next_sibling("ul")
if not content_tag:
# ILIAS allows users to delete the initial post while keeping the thread open
# This produces empty threads without *any* content.
# I am not sure why you would want this, but ILIAS makes it easy to do.
continue
title = p.find("b").text title = p.find("b").text
if ":" in title: if ":" in title:
title = title[title.find(":") + 1:] title = title[title.find(":") + 1:]