mirrors/pferd
1
0
Fork 0
mirror of https://github.com/Garmelon/PFERD.git synced 2023-12-21 10:23:01 +01:00
Code Issues Packages Projects Releases Wiki Activity

Append trailing slash to kit-ipd links to ensure urljoin works as expected

Browse Source
This commit is contained in:
I-Al-Istannen 2022-10-25 20:28:06 +02:00
parent 1b6be6bd79
commit f47d2f11d8
2 changed files with 4 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
CHANGELOG.md
View File

@ -25,6 +25,7 @@ ambiguous situations.
### Fixed ### Fixed
- Forum crawling crashing when parsing empty (= 0 messages) threads - Forum crawling crashing when parsing empty (= 0 messages) threads
- Forum crawling crashing when a forum has no threads at all - Forum crawling crashing when a forum has no threads at all
- kit-ipd crawler if URL did not end with a trailing slash
## 3.4.1 - 2022-08-17 ## 3.4.1 - 2022-08-17

3
PFERD/crawl/kit_ipd_crawler.py
View File

@ -24,6 +24,9 @@ class KitIpdCrawlerSection(HttpCrawlerSection):
if not target.startswith("https://"): if not target.startswith("https://"):
self.invalid_value("target", target, "Should be a URL") self.invalid_value("target", target, "Should be a URL")
if not target.endswith("/"):
target = target + "/"
return target return target
def link_regex(self) -> Pattern[str]: def link_regex(self) -> Pattern[str]: