1. There is one database per folder and sqlite requires to serialize the
writings. Instead of locking at LocalStatusSQLiteFolder instance level,
introduce a new DatabaseFileLock object which is shared across threads. This
fixes the concurrent writes issues that some users might experience by
duplications or flags restored to the previous state.
2. Close the database only when we are sure no other threads will use the
connection on a *per-file* basis. Previous fix 677afb8d8f is wrong
because the same lock is shared for all the database files.
Github-fix: https://github.com/OfflineIMAP/offlineimap/issues/350
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
This did not work and is not usefull since the purge() method was introduced in
1410a391bc. Actually, the purge() does what deletemessagelist() was supposed to
achieve.
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
It's required to have more than one connection to the database for the
maxconnections configuration option to work with threads. However,
connection.close() is closing all the connections. Only close the connection
when no more thread need it.
Github-fix: https://github.com/OfflineIMAP/offlineimap/issues/350
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Output a warning so that we can dig into this. For now, the patch doesn't fix
the root cause. If the server returns UID 0 as valid UID number, this must be
ignored as soon as possible.
Github-ref: https://github.com/OfflineIMAP/offlineimap/issues/336
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
When a maildir is removed it must be considered new for the sync. However, the
local cache of the folder remains. This means the sync of the folder removes all
the missing emails.
Avoid loosing of data for users not aware of the local cache by removing any
pre-existing status cache of a folder when we actually want to create the
database.
Improve style.
Github-fix: https://github.com/OfflineIMAP/offlineimap/issues/333
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
The virtual imaplib2 first try to import imaplib2 when provided by the system.
If not provided or if the version is not supported, fallback on the bundled
imaplib2 version.
Distribution maintainers can now easily remove the bundled imaplib2 version if
they want to get it packaged outside of offlineimap.
We still want to provide imaplib2 by default because:
- this library is neither in Python core nor packaged by a lot of distributions;
- users expect to be able to run offlineimap by just downloading the tarball or
after a git clone.
In order to avoid unexpected (too old) versions of imaplib2, we restrict the
supported versions of this librabry.
Reviewed-by: Łukasz Żarnowiecki <dolohow@outlook.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Some servers are broken: they return zero as valid UID and respond "BAD invalid
parameter: 0" on the FETCH command.
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
There is no other way to make Python2 and Python3 happy, because syntax
raise E, V, T is incompatible with the latter.
Signed-off-by: Łukasz Żarnowiecki <dolohow@outlook.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Python3 accepts binary input for md5 function.
This patch is known to break setups using folder names not strictly conforming
the IMAP UTF-7 encoding. We always made it clear that such setup is unsupported
and might be broken at some point in time. See documentation about
'decodefoldernames' in the provided configuration file. This is why this patch
is considered introducing no regression for this use case.
Patches to support both Python 3 and Python 2 by re-encoding the MD5 in the
filenames are welcome. This likely requires a new CLI option to allow
backporting the feature for users downgrading or changing of Python environment.
Signed-off-by: Łukasz Żarnowiecki <dolohow@outlook.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
We currently close the database as soon as possible while we handle the status
backend but this is still too early because autorefresh induces looping on this
code block.
Instead of delaying the closing outside of the loop, it's easier to delay the
opening as late as possible (inside the loop). The downside is that the database
is opened/closed more than once when autorefresh is enabled. The good news is
that this make the code much easier.
Fixes regression introduces by 6fb5700.
Reported-by: Łukasz Żarnowiecki <dolohow@outlook.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
In Python3, zip returns iterator instead of list.
Signed-off-by: Łukasz Żarnowiecki <dolohow@outlook.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
xrange was removed in Python3 while range exists in boths versions.
Signed-off-by: Łukasz Żarnowiecki <dolohow@outlook.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
There is no such method in Python3 any more.
Signed-off-by: Łukasz Żarnowiecki <dolohow@outlook.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
It is more readable and returns a list therefore it is compatible both
with Python 2 and 3.
Signed-off-by: Łukasz Żarnowiecki <dolohow@outlook.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
"imapobj.search" returns a list with one string element of numbers
separated by one whitespace character for regular box (GMail, AOL...).
['1 2 3 4 5 6 7 8 9 10 11 12']
But if we would like to sync from Davmail it would return a list of
numbers.
['1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11', '12'].
The code "return res_data[0].split()" in the first case will return what
we already have when using Davmail, hence only one email will be
fetched. But if only the first sync would be with maxage the emails
will be removed, because offlineimap will think that they were removed
by us.
The patch distinguishes between syncing with Davmail and regular box and
applies split on the first element only when it finds whitespace
character. It also handles the case when the first element is empty on
first sync.
Closes#327
Signed-off-by: Łukasz Żarnowiecki <dolohow@outlook.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Sometimes it might happen that you put wrong date and you except emails
to be fetched, but they are not and you do not have an idea why.
By raising exception the user will see a proper message telling that he
used the wrong date for maxage/startdate property.
If someone wants to set a future date intentionally might as well sync
in the future.
Signed-off-by: Łukasz Żarnowiecki <dolohow@outlook.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
This enables the "append" mode feature. Configuration option is sync_deletes in
both local and remote repositories. Marked EXPERIMENTAL and UNTESTED.
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Introduce the '--migrate-fmd5-using-nametrans' option which migrates the
FMD5 hashes from versions prior to 6.3.5.
It seems that commit 'Apply nametrans to all Foldertypes' (6b2ec956cf)
introduced a regression because it changed the FMD5 part of the filename
calculated by OfflineIMAP. Thus, OfflineIMAP believes that the messages
has been removed and adds them back.
For more information, see:
http://www.offlineimap.org/configuration/2016/02/12/debian-upgrade-from-jessie-to-stretch.html
Bug-Debian: https://bugs.debian.org/812108
Reported-by: François <francois@avalenn.eu>
Signed-off-by: Ilias Tsitsimpis <i.tsitsimpis@gmail.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
