Since skipping a folder means no new data is downloaded, the UID validity
problem is a backup failure. Make it possible to alert or work around
it in scripts by signaling with the exit code.
Signed-off-by: Alex Kapranoff <alex@kapranoff.ru>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Now it is possible to handle failed syncs in scripts.
Signed-off-by: Alex Kapranoff <alex@kapranoff.ru>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
1. When using maxage, local and remote messagelists are supposed to only
contain messages from at most maxage days ago. But local and remote used
different timezones to calculate what "maxage days ago" means, resulting
in removals on one side. Now, we ask the local folder for maxage days'
worth of mail, find the lowest UID, and then ask the remote folder for
all UID's starting with that lowest one.
2. maxage was fundamentally wrong in the IMAP-IMAP case: it assumed that
remote messages have UIDs in the same order as their local counterparts,
which could be false, e.g. when messages are copied in quick succession.
So, remove support for maxage in the IMAP-IMAP case.
3. Add startdate option for IMAP-IMAP syncs: use messages from the given
repository starting at startdate, and all messages from the other
repository. In the first sync, the other repository must be empty.
4. Allow maxage to be specified either as number of days to sync (as
previously) or as a fixed date.
Signed-off-by: Janna Martl <janna.martl109@gmail.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Previously, syncing labels on a message always resulted in updating the
file modification time, even with utime_from_headers=true
This patch restores the file mtime to the previous value when
utime_from_headers=true, preventing a label synchronization from
breaking the promise that the file mtimes coincide with the header date.
Signed-off-by: Abdo Roig-Maranges <abdo.roig@gmail.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
We were using rtime for two different purposes:
- to store remote internal date
- to use in the utime_from_header option
Let's decouple the utime_from_header logic from rtime, now rtime means
remote internal date.
Signed-off-by: Abdo Roig-Maranges <abdo.roig@gmail.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Upstream bug has been fixed, since imaplib2 v2.42 the untagged responses are
flushed (as stated by the documentation).
See https://sourceforge.net/p/imaplib2/bugs/7/
Signed-off-by: Pierre-Louis Bonicoli <pierre-louis.bonicoli@gmx.fr>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
This partially reverts commit 25513e9038.
Only changes about dates and times are reverted. The changes about the style are
kept.
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
The basic problem is in the context of syncing multiple accounts where
one is fast and the others are slower (due to the number of folders).
When the fast account completes, the other accounts are partially written
through the list and if the file is read during this time, the list can
be useless. However, in the general case, the file is probably left
around from a previous run of offlineimap and is more correct, so add an
option to leave it alone until all syncing is done.
Incremental is still the default since this running offlineimap using
its own timer setup is likely the most common setup. Turning it off
works best with one-shot mode triggered by cron or systemd timers.
Signed-off-by: Ben Boeckel <mathstuf@gmail.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Fix regresssion introduced in 428349e3.
Prevent messages with UID's already in the destination folder from getting
excluded from the copy list.
Signed-off-by: Janna Martl <janna.martl109@gmail.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Some messages were excluded from the copy/delete list after the UI message said
they were copied/deleted.
Also fix Internaldate2epoch(), which was wrong.
Signed-off-by: Janna Martl <janna.martl109@gmail.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Suppose messages A and B were delivered to the remote folder at
"maxage + 1" days ago.
A was downloaded to the local folder "maxage + 1" days ago, but B was only
downloaded "maxage - 1" days ago (contrived scenario to illustrate the two
things that could happen). The behavior was that B gets deleted from the local
folder, but A did not. The expected behavior is that neither is deleted.
Starting where Base.py: __syncmessagesto_delete(self, dstfolder, statusfolder)
is called where:
- self is the remote folder
and
- dstfolder is the local folder.
It defines deletelist to be the list of messages in the status folder
messagelist that aren't in the remote folder messagelist with
not self.uidexists(uid)
A and B are both in the status folder. They're also both *NOT* in the remote
folder messagelist: this list is formed in IMAP.py: cachemessagelist(), which
calls _msgs_to_fetch(), which only asks the IMAP server for messages that are
"< maxage" days old.
Back to Base.py __syncmessagesto_delete(), look at the call
folder.deletemessages(deletelist), where folder is the local folder. This ends
up calling Maildir.py deletemessage() for each message on the deletelist. But we
see that this methods returns (instead of deleting anything) if the message is
in the local folder's messagelist. This messagelist was created by Maildir.py's
cachemessagelist(), which calls _scanfolder(), which tries to exclude messages
based on maxage. So at this point, we *WANT* A and B to be excluded -- then they
will be spared from deletion. This maxage check calls _iswithinmaxage(), and
actually does the date comparison based on the time found at the beginning of
the message's filename. These filenames were originally created in Maildir.py's
new_message_filename(), which calls _gettimeseq() to get the current time (i.e.
the time of retrieval).
Upshot: A's filename has an older timestamp than B's filename. A is excluded
from the local folder messagelist in _scanfolder(), hence spared from deletion
in deletemessage(); B is not excluded, and is deleted.
This patch does not address the timezone issue. As for the IMAP/timezone issue,
a similar issue is discussed in the thunderbird bug tracker here:
https://bugzilla.mozilla.org/show_bug.cgi?id=886534
In the end, they're solving a different problem, but they agree that
there is really no reliable way of guessing the IMAP server's internal
timezone.
Signed-off-by: Janna Martl <janna.martl109@gmail.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Read proxy option in imapserver, instantiate a class in imaplibutil
using a self-defined keyword and a socket instance, and use this socket
instance to substitute the default socket instance used in imaplib2.
Signed-off-by: 夏恺(Xia Kai) <xiaket@gmail.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Get real tracebacks (at the point that calls Request.abort()
and not from the point that handles collected abort requests)
and pass them to our calling functions to ease debugging
of user problems.
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
This simplifies logics for the user, especially if he uses both
fingerprint and certificate validation: it is hard to maintain
the compatibility with the prior behaviour and to avoid getting
default CA bundle to be disabled when fingerprint verification
is requested.
See
http://thread.gmane.org/gmane.mail.imap.offlineimap.general/6695
for discussion about this change.
Default CA bundle is requested via 'sslcertfile = OS-DEFAULT'.
I had also enforced all cases where explicitely-requested CA bundles
are non-existent to be hard errors: when users asks us to use CA
bundle (and, thus, certificate validation), but we can't find one,
we must error out rather than happily continue and downgrade to
no validation.
Reported-By: Edd Barrett <edd@theunixzoo.co.uk>
Reviewed-By: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
