Commit Graph

11 Commits

Author SHA1 Message Date
9fd356d290 Ensure tmp files are deleted
This doesn't seem to fix the case where an exception bubbles up to the top of
the event loop. It also doesn't seem to fix the case when a KeyboardInterrupt is
thrown, since that never makes its way into the event loop in the first place.

Both of these cases lead to the event loop stopping, which means that the tmp
file cleanup doesn't get executed even though it's inside a "with" or "finally".
2021-05-15 23:00:40 +02:00
989032fe0c Fix cookies getting deleted 2021-05-15 22:25:48 +02:00
05573ccc53 Add fancy CLI options 2021-05-15 22:22:01 +02:00
0c9167512c Fix output dir
I missed these while renaming the resolve function. Shame on me for not running
mypy earlier.
2021-05-14 21:28:38 +02:00
d565df27b3 Add HttpCrawler 2021-05-13 22:28:14 +02:00
38bb66a776 Update file metadata in more cases
PFERD now not only updates file metadata when a file is successfully added or
changed, but also when a file is downloaded and then detected to be unchanged.

This could occur for example if a remote file's modification time was bumped,
possibly because somebody touched the file without changing it.
2021-05-13 19:40:10 +02:00
68781a88ab Fix asynchronous methods being not awaited 2021-05-13 19:39:49 +02:00
910462bb72 Log stuff happening to files 2021-05-13 19:37:27 +02:00
6bd6adb977 Fix tmp file names 2021-05-13 19:36:46 +02:00
60cd9873bc Add local file crawler 2021-05-06 01:02:40 +02:00
bbfdadc463 Implement output directory 2021-05-05 18:08:34 +02:00