1
0
mirror of https://github.com/Garmelon/PFERD.git synced 2023-12-21 10:23:01 +01:00
Commit Graph

6 Commits

Author SHA1 Message Date
Joscha
acd674f0a0 Change limiter logic
Now download tasks are a subset of all tasks.
2021-05-15 15:25:05 +02:00
Joscha
296a169dd3 Make limiter logic more complex
The limiter can now distinguish between crawl and download actions and has a
fancy slot system and delay logic.
2021-05-15 15:25:05 +02:00
Joscha
0096d83387 Simplify Limiter implementation 2021-04-29 20:20:25 +02:00
Joscha
7e127cd5cc Clean up and fix conductor and limiter
Turns out you have to await an async lock, who knew...
2021-04-29 13:44:04 +02:00
Joscha
c4fb92c658 Make type hints compatible with Python 3.8 2021-04-29 13:11:58 +02:00
Joscha
b915e393dd Implement limiter 2021-04-29 10:24:28 +02:00