tobias/docker-offlineimap
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

sqlite: properly serialize operations on the databases

Browse Source 
1. There is one database per folder and sqlite requires to serialize the
writings. Instead of locking at LocalStatusSQLiteFolder instance level,
introduce a new DatabaseFileLock object which is shared across threads. This
fixes the concurrent writes issues that some users might experience by
duplications or flags restored to the previous state.

2. Close the database only when we are sure no other threads will use the
connection on a *per-file* basis. Previous fix 677afb8d8f is wrong
because the same lock is shared for all the database files.

Github-fix: https://github.com/OfflineIMAP/offlineimap/issues/350
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
This commit is contained in:
Nicolas Sebrecht
2016-07-25 14:39:11 +02:00
parent 5aa2a883f0
commit 3c42913120
3 changed files with 66 additions and 40 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
offlineimap/accounts.py
View File

@ -425,7 +425,8 @@ class SyncableAccount(Account):
def syncfolder(account, remotefolder, quick):
"""Synchronizes given remote folder for the specified account.
Filtered folders on the remote side will not invoke this function."""
Filtered folders on the remote side will not invoke this function. However,
this might be called in a concurrently."""
def check_uid_validity(localfolder, remotefolder, statusfolder):
# If either the local or the status folder has messages and
@ -454,7 +455,7 @@ def syncfolder(account, remotefolder, quick):
fd.close()
def cachemessagelists_upto_date(localfolder, remotefolder, date):
""" Returns messages with uid > min(uids of messages newer than date)."""
"""Returns messages with uid > min(uids of messages newer than date)."""
localfolder.cachemessagelist(min_date=date)
check_uid_validity(localfolder, remotefolder, statusfolder)
@ -474,7 +475,7 @@ def syncfolder(account, remotefolder, quick):
min_date=time.gmtime(time.mktime(date) + 24*60*60))
def cachemessagelists_startdate(new, partial, date):
""" Retrieve messagelists when startdate has been set for
"""Retrieve messagelists when startdate has been set for
the folder 'partial'.
Idea: suppose you want to clone the messages after date in one
@ -489,8 +490,7 @@ def syncfolder(account, remotefolder, quick):
might not correspond. But, if we're cloning a folder into a new one,
[min_uid, ...] does correspond to [1, ...].
This is just for IMAP-IMAP. For Maildir-IMAP, use maxage instead.
"""
This is just for IMAP-IMAP. For Maildir-IMAP, use maxage instead."""
new.cachemessagelist()
min_uid = partial.retrieve_min_uid()
@ -602,8 +602,8 @@ def syncfolder(account, remotefolder, quick):
if e.severity > OfflineImapError.ERROR.FOLDER:
raise
else:
ui.error(e, exc_info()[2], msg = "Aborting sync, folder '%s' "
"[acc: '%s']" % (localfolder, account))
ui.error(e, exc_info()[2], msg="Aborting sync, folder '%s' "
"[acc: '%s']"% (localfolder, account))
except Exception as e:
ui.error(e, msg = "ERROR in syncfolder for %s folder %s: %s"%
(account, remotefolder.getvisiblename(), traceback.format_exc()))