mirrors/pferd
1
0
Fork 0
mirror of https://github.com/Garmelon/PFERD.git synced 2023-12-21 10:23:01 +01:00
Code Issues Packages Projects Releases Wiki Activity

Add 'skip' option to crawlers

Browse Source
This commit is contained in:
Joscha
2021-06-04 18:33:02 +02:00
parent fc31100a0f
commit df3ad3d890
6 changed files with 48 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
CONFIG.md
View File

@ -49,6 +49,9 @@ see the type's [documentation](#crawler-types) below. The following options are
common to all crawlers:
- `type`: The available types are specified in [this section](#crawler-types).
- `skip`: Whether the crawler should be skipped during normal execution. The
crawler can still be executed manually using the `--crawler` or `-C` flags.
(Default: `no`)
- `output_dir`: The directory the crawler synchronizes files to. A crawler will
never place any files outside of this directory. (Default: the crawler's name)
- `redownload`: When to download a file that is already present locally.