tobias/docker-offlineimap
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[319117] Unroll loop to speed performance on large folders

Browse Source 
From: "Nikita V. Youshchenko"
I trued to use offlineimap and found that while being quite fast on
small folders, it takes up to several minutes (of 100% busy CPU and
almost no network traffic) to sync a folder with 2000+ messages.

While looking into the code, I found why this happens.
In folder/Base.py, in method BaseFolder.syncmessagesto_copy(),
dest.getmessagelist() is called inside a loop, while being a loop
invariant. Similar thing happens in BaseFolder.syncmessagesto_delete()
for self.getmessagelist().
This causes quadratic complexity over folder size.

Moving these calls out of loops make large folder sync fast (several
seconds instead of several minutes for folder with 2000 messages on
700MHz P3).
This commit is contained in:
John Goerzen 2005-08-23 08:15:09 +01:00
parent 7a903f3bb1
commit 0841e03a4c
1 changed files with 4 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
offlineimap/folder/Base.py
View File

@ -288,10 +288,11 @@ class BaseFolder:
them to dest.""" them to dest."""
threads = [] threads = []
dest_messagelist = dest.getmessagelist()
for uid in self.getmessagelist().keys(): for uid in self.getmessagelist().keys():
if uid < 0: # Ignore messages that pass 1 missed. if uid < 0: # Ignore messages that pass 1 missed.
continue continue
if not uid in dest.getmessagelist(): if not uid in dest_messagelist:
if self.suggeststhreads(): if self.suggeststhreads():
self.waitforthread() self.waitforthread()
thread = InstanceLimitedThread(\ thread = InstanceLimitedThread(\
@ -314,10 +315,11 @@ class BaseFolder:
Look for message present in dest but not in self. Look for message present in dest but not in self.
If any, delete them.""" If any, delete them."""
deletelist = [] deletelist = []
self_messagelist = self.getmessagelist()
for uid in dest.getmessagelist().keys(): for uid in dest.getmessagelist().keys():
if uid < 0: if uid < 0:
continue continue
if not uid in self.getmessagelist(): if not uid in self_messagelist:
deletelist.append(uid) deletelist.append(uid)
if len(deletelist): if len(deletelist):
UIBase.getglobalui().deletingmessages(deletelist, applyto) UIBase.getglobalui().deletingmessages(deletelist, applyto)