[319117] Unroll loop to speed performance on large folders
From: "Nikita V. Youshchenko" I trued to use offlineimap and found that while being quite fast on small folders, it takes up to several minutes (of 100% busy CPU and almost no network traffic) to sync a folder with 2000+ messages. While looking into the code, I found why this happens. In folder/Base.py, in method BaseFolder.syncmessagesto_copy(), dest.getmessagelist() is called inside a loop, while being a loop invariant. Similar thing happens in BaseFolder.syncmessagesto_delete() for self.getmessagelist(). This causes quadratic complexity over folder size. Moving these calls out of loops make large folder sync fast (several seconds instead of several minutes for folder with 2000 messages on 700MHz P3).
This commit is contained in:
parent
7a903f3bb1
commit
0841e03a4c
@ -288,10 +288,11 @@ class BaseFolder:
|
||||
them to dest."""
|
||||
threads = []
|
||||
|
||||
dest_messagelist = dest.getmessagelist()
|
||||
for uid in self.getmessagelist().keys():
|
||||
if uid < 0: # Ignore messages that pass 1 missed.
|
||||
continue
|
||||
if not uid in dest.getmessagelist():
|
||||
if not uid in dest_messagelist:
|
||||
if self.suggeststhreads():
|
||||
self.waitforthread()
|
||||
thread = InstanceLimitedThread(\
|
||||
@ -314,10 +315,11 @@ class BaseFolder:
|
||||
Look for message present in dest but not in self.
|
||||
If any, delete them."""
|
||||
deletelist = []
|
||||
self_messagelist = self.getmessagelist()
|
||||
for uid in dest.getmessagelist().keys():
|
||||
if uid < 0:
|
||||
continue
|
||||
if not uid in self.getmessagelist():
|
||||
if not uid in self_messagelist:
|
||||
deletelist.append(uid)
|
||||
if len(deletelist):
|
||||
UIBase.getglobalui().deletingmessages(deletelist, applyto)
|
||||
|
Loading…
Reference in New Issue
Block a user