[319117] Unroll loop to speed performance on large folders

From: "Nikita V. Youshchenko"
I trued to use offlineimap and found that while being quite fast on
small folders, it takes up to several minutes (of 100% busy CPU and
almost no network traffic) to sync a folder with 2000+ messages.

While looking into the code, I found why this happens.
In folder/Base.py, in method BaseFolder.syncmessagesto_copy(),
dest.getmessagelist() is called inside a loop, while being a loop
invariant. Similar thing happens in BaseFolder.syncmessagesto_delete()
for self.getmessagelist().
This causes quadratic complexity over folder size.

Moving these calls out of loops make large folder sync fast (several
seconds instead of several minutes for folder with 2000 messages on
700MHz P3).
This commit is contained in:
John Goerzen 2005-08-23 08:15:09 +01:00
parent 7a903f3bb1
commit 0841e03a4c

View File

@ -288,10 +288,11 @@ class BaseFolder:
them to dest."""
threads = []
dest_messagelist = dest.getmessagelist()
for uid in self.getmessagelist().keys():
if uid < 0: # Ignore messages that pass 1 missed.
continue
if not uid in dest.getmessagelist():
if not uid in dest_messagelist:
if self.suggeststhreads():
self.waitforthread()
thread = InstanceLimitedThread(\
@ -314,10 +315,11 @@ class BaseFolder:
Look for message present in dest but not in self.
If any, delete them."""
deletelist = []
self_messagelist = self.getmessagelist()
for uid in dest.getmessagelist().keys():
if uid < 0:
continue
if not uid in self.getmessagelist():
if not uid in self_messagelist:
deletelist.append(uid)
if len(deletelist):
UIBase.getglobalui().deletingmessages(deletelist, applyto)