mirrors/pferd
1
0
Fork 0
mirror of https://github.com/Garmelon/PFERD.git synced 2023-12-21 10:23:01 +01:00
Code Issues Packages Projects Releases Wiki Activity

Fix wrong base URL for multi-stage pages

Browse Source
This commit is contained in:
I-Al-Istannen
2021-06-13 15:39:22 +02:00
parent 70b33ecfd9
commit 70ec64a48b
3 changed files with 5 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
PFERD/crawl/ilias/kit_ilias_web_crawler.py
View File

@ -253,7 +253,7 @@ instance's greatest bottleneck.
soup = await self._get_page(next_stage_url)
log.explain_topic(f"Parsing HTML page for {fmt_path(path)}")
log.explain(f"URL: {next_stage_url}")
page = IliasPage(soup, url, parent)
page = IliasPage(soup, next_stage_url, parent)
next_stage_url = page.get_next_stage_url()
elements.extend(page.get_child_elements())