Add forum crawling

This downloads all forum posts when needed and saves each thread in its
own html file, named after the thread title.
This commit is contained in:
I-Al-Istannen
2022-05-24 23:28:09 +02:00
parent 846c29aee1
commit 46fb782798
5 changed files with 208 additions and 16 deletions

View File

@ -24,6 +24,7 @@ ambiguous situations.
### Added
- Download of page descriptions
- Forum download support
### Changed
- Add `cpp` extension to default `link_regex` of IPD crawler