Commit Graph

130 Commits

Author SHA1 Message Date
James E. Blair
a5ea1827fb folder: Base: sort message UID list
Return a sorted list of UIDs in getmessageuidlist.

Some MUAs (and their users) like to display messages in UID order
(or "order received").  If offlineimap is used in IMAP<->IMAP mode,
then the order messages are received by the second IMAP server will
be different because offlineimap iterates over a UID list produced
from the keys of a dictionary, which is unsorted.

This change sorts that list of UIDs so that both IMAP servers will
have their messages in the same order (except those times where
messages are appended to folders on both repositories between
syncs).

Signed-off-by: James E. Blair <corvus@gnu.org>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-08-14 01:56:13 +02:00
Nicolas Sebrecht
473be4c66f folder: Base: minor fix style
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-08-09 17:36:26 +02:00
Nicolas Sebrecht
6b313aabd3 don't delete messages in local cache in dry-run mode
This makes dry-run consitent when called more than once.

Github-ref: https://github.com/OfflineIMAP/offlineimap/issues/370
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-08-09 15:02:03 +02:00
Łukasz Żarnowiecki
f634546980 explicitly set __hash__ of Base class to None
This suppressing the warning generated when running offlineimap on
python2 with -3 switch:

  offlineimap/folder/Base.py:29: DeprecationWarning: Overriding __eq__
  blocks inheritance of __hash__ in 3.x

Since this object is mutable it should not be hashable.

From Python documentation[1]:

  A class that overrides __eq__() and does not define __hash__() will
  have its __hash__() implicitly set to None.

Therefore old behaviour is preserved.

[1] https://docs.python.org/3/reference/datamodel.html

Signed-off-by: Łukasz Żarnowiecki <dolohow@outlook.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-08-06 01:32:26 +02:00
Nicolas Sebrecht
cbd1a8929a folder: IMAP: change raw assert to OfflineImapError
There is not reason to block remainings actions on first error on UID STORE.

Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-08-04 02:21:22 +02:00
Nicolas Sebrecht
59b712ecbc remove dead code: the description of the passes is never used
Some minor style fixes.

Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-07-18 18:41:41 +02:00
Nicolas Sebrecht
29e06a60f9 learn to not download UIDs defined by the user
Allow users to workaround offending emails that offlineimap can't download.

Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-07-03 20:02:45 +02:00
Nicolas Sebrecht
1da5bd938c folder: Base: fix line continuation
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-06-27 15:27:11 +02:00
Nicolas Sebrecht
65d4c94f07 don't try to copy messages with UID == 0
Output a warning so that we can dig into this. For now, the patch doesn't fix
the root cause. If the server returns UID 0 as valid UID number, this must be
ignored as soon as possible.

Github-ref: https://github.com/OfflineIMAP/offlineimap/issues/336
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-06-27 15:13:03 +02:00
Nicolas Sebrecht
9a7603af37 minor: folder: Base: improve style and comments
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-06-27 15:03:27 +02:00
Nicolas Sebrecht
7fcd160911 GmailMaildir: quick mode is not compatible with utime_from_header
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-06-08 19:02:04 +02:00
Nicolas Sebrecht
922bed05a7 threading: improve comments
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-06-04 02:25:44 +02:00
Nicolas Sebrecht
46a6bcbf1c threading: suggeststhreads must honor CLI and conf options
There's no need to honor them twice.

Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-05-18 04:23:02 +02:00
Nicolas Sebrecht
e0fdcb3852 threading: improve variable names and factorize code
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-05-18 04:23:02 +02:00
Nicolas Sebrecht
83539f8601 threading: minor improvements
Factorize string, enhance comments and minor code improvements.

Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-05-18 01:32:06 +02:00
Nicolas Sebrecht
14ef992444 conf: newmail_hook is a remote option
Improve documentation.

Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-05-12 18:04:41 +02:00
Nicolas Sebrecht
bf8bfbc872 folder: utime_from_header is for Maildir only
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-05-11 04:10:13 +02:00
Łukasz Żarnowiecki
19c43308b9 Change filter with lambda to list comprehension
It is more readable and returns a list therefore it is compatible both
with Python 2 and 3.

Signed-off-by: Łukasz Żarnowiecki <dolohow@outlook.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-05-10 02:53:09 +02:00
Łukasz Żarnowiecki
068ac7c410 Replace calls to long with int calls
long was removed from Python3

Signed-off-by: Łukasz Żarnowiecki <dolohow@outlook.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-05-10 02:53:09 +02:00
Łukasz Żarnowiecki
4fbb5640ac Convert basestring to str
Signed-off-by: Łukasz Żarnowiecki <dolohow@outlook.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-05-10 02:53:09 +02:00
Łukasz Żarnowiecki
92e5455eb4 Inform when maxage/startdate is in the future
Sometimes it might happen that you put wrong date and you except emails
to be fetched, but they are not and you do not have an idea why.

By raising exception the user will see a proper message telling that he
used the wrong date for maxage/startdate property.

If someone wants to set a future date intentionally might as well sync
in the future.

Signed-off-by: Łukasz Żarnowiecki <dolohow@outlook.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-05-05 23:50:24 +02:00
Nicolas Sebrecht
d5853b5d65 learn to not delete messages
This enables the "append" mode feature. Configuration option is sync_deletes in
both local and remote repositories. Marked EXPERIMENTAL and UNTESTED.

Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-04-09 22:08:28 +02:00
Nicolas Sebrecht
657f258196 folder: properly factorize initialization and dropping of self.message
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-04-09 18:12:18 +02:00
Nicolas Sebrecht
d6077a09cf Keywords: avoid warning at each message when no keywords are used
This fix does not apply when any keyword in configured which is already
harmless.

Written-by: Igor Almeida <igor.contato@gmail.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2015-11-22 20:05:34 +01:00
Igor Almeida
58a6f8b401 __syncmessagesto_flags: refactor for readability
Extract the flag/keyword translation and combination logic to a
function.

Signed-off-by: Igor Almeida <igor.contato@gmail.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2015-11-22 19:51:33 +01:00
Igor Almeida
61ee6e783e __syncmessagesto_flags: store keywords
This uses the destination folder's keyword mapping to translate the
message's keywords into some appropriate format.

Tested only with local Maildir.

Signed-off-by: Igor Almeida <igor.contato@gmail.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2015-11-22 19:51:20 +01:00
Igor Almeida
73a3767d11 IMAP folder: expose the message keywords
The keywords are in the flag string, so imaputil can just strip the
usual \Flags.

Signed-off-by: Igor Almeida <igor.contato@gmail.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2015-11-22 19:51:02 +01:00
Sébastien Gross
3b30c4aa93 add new config option filename_use_mail_timestamp
If this value is true, use (if possible) a timestamp based on message
Date or Delivery-date headers. The current system time is used
otherwise.

filename_use_mail_timestamp and utime_from_header are now completely
separated option that do not interfere one with other.

To handle this feature in a multithread context we use a hash to count
the number of mail with the same timestamp. This method is more accurate
than using the old lasttime and timeseq variables.

Signed-off-by: Sébastien Gross <seb•ɑƬ•chezwam•ɖɵʈ•org>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2015-11-04 00:56:46 +01:00
Matthew Krafczyk
48bb2f4113 Added the newmail_hook
When new mail arrives, this hook is triggered, allowing the user to
play a sound, or launch a popup.

Signed-off-by: Matthew Krafczyk <krafczyk.matthew@gmail.com>
2015-09-11 20:34:38 +02:00
Janna Martl
8096f6cd5b maxage: fix timezone issues, remove IMAP-IMAP support, add startdate option
1. When using maxage, local and remote messagelists are supposed to only
contain messages from at most maxage days ago. But local and remote used
different timezones to calculate what "maxage days ago" means, resulting
in removals on one side. Now, we ask the local folder for maxage days'
worth of mail, find the lowest UID, and then ask the remote folder for
all UID's starting with that lowest one.

2. maxage was fundamentally wrong in the IMAP-IMAP case: it assumed that
remote messages have UIDs in the same order as their local counterparts,
which could be false, e.g. when messages are copied in quick succession.
So, remove support for maxage in the IMAP-IMAP case.

3. Add startdate option for IMAP-IMAP syncs: use messages from the given
repository starting at startdate, and all messages from the other
repository. In the first sync, the other repository must be empty.

4. Allow maxage to be specified either as number of days to sync (as
previously) or as a fixed date.

Signed-off-by: Janna Martl <janna.martl109@gmail.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2015-04-07 11:54:35 +02:00
Nicolas Sebrecht
71693b7d8c Maildir: fix imports of emailutil
Commit 0c17350e4f revisited utime_for_header without taking care of the
imports.

Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2015-04-06 23:50:56 +02:00
Abdo Roig-Maranges
0c17350e4f decouple utime_from_header from rtime
We were using rtime for two different purposes:
- to store remote internal date
- to use in the utime_from_header option

Let's decouple the utime_from_header logic from rtime, now rtime means
remote internal date.

Signed-off-by: Abdo Roig-Maranges <abdo.roig@gmail.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2015-04-03 11:30:30 +02:00
Nicolas Sebrecht
5a4e4fd0c2 rename utime_from_message to utime_from_header
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2015-03-27 16:50:17 +01:00
Janna Martl
731129396c fix: don't copy messages if not necessary
Fix regresssion introduced in 428349e3.

Prevent messages with UID's already in the destination folder from getting
excluded from the copy list.

Signed-off-by: Janna Martl <janna.martl109@gmail.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2015-03-22 10:42:29 +01:00
Janna Martl
428349e3f4 fix inaccurate UI messages when some messages are excluded from the cached lists
Some messages were excluded from the copy/delete list after the UI message said
they were copied/deleted.
Also fix Internaldate2epoch(), which was wrong.

Signed-off-by: Janna Martl <janna.martl109@gmail.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2015-03-21 14:10:57 +01:00
Nicolas Sebrecht
9e63fa3784 fix: folder/*: never set self.messagelist to None
Empty the list by setting an empty dict.
Introduce BaseFolder().ismessagelistempty().

Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2015-02-13 17:07:18 +01:00
Nicolas Sebrecht
baee2b6fd9 fix: folder/Base: missing """ for doc string
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2015-02-13 17:07:18 +01:00
Nicolas Sebrecht
461554b7b1 more style consistency
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2015-02-10 17:25:00 +01:00
Wieland Hoffmann
d08f6d15c2 addmessageheader: Add a note about the incorrect rendering of the docstring
The note tells people to look at the source of the method, which
spinx.ext.viewcode conveniently links right next to the methods
signature.

Signed-off-by: Wieland Hoffmann <themineo@gmail.com>
2015-01-12 17:17:42 +01:00
Giovanni Mascellani
29e9b7ab39 Drop caches after having processed folders.
This enhances a lot memory consumption when you have many folders
to process.

Signed-off-by: Giovanni Mascellani <mascellani@poisson.phc.unipi.it>
2015-01-12 17:17:42 +01:00
Wieland Hoffmann
2f9d8d22ee folder/Base.py: fix comment: para -> param
Signed-off-by: Wieland Hoffmann <themineo@gmail.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2015-01-09 01:10:49 +01:00
Nicolas Sebrecht
0f40ca4799 more style consistency
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2015-01-08 23:36:19 +01:00
Nicolas Sebrecht
61021260cb more consistent style
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2015-01-07 21:31:43 +01:00
Abdo Roig-Maranges
e70607e3e3 Warn about a tricky piece of code in addmessageheader
As requested by Nicholas Sebrecht.

Signed-off-by: Abdo Roig-Maranges <abdo.roig@gmail.com>
2014-11-20 14:18:35 +01:00
Abdo Roig-Maranges
fc4c7549d6 Do not ignore gmail labels if header appears multiple times
There should be just one header storing gmail labels, but due to a bug,
multiple X-Keywords (or equivalent) headers may be found on the local
messages.

Now we, when extracting the labels from a message, we read all label
headers, instead of just the first one.

This has the consequence that some old labels stored locally in a second
X-Keywords (or third...) header, which effectively was rendered
invisible to offlineimap until now, may pop back up again and be pushed
to gmail. No labels will be removed by the changes in this commit,
though.

Signed-off-by: Abdo Roig-Maranges <abdo.roig@gmail.com>
2014-11-20 14:15:59 +01:00
Eygene Ryabinkin
7df765cfdb Properly manipulate contents of messagelist for folder
Create initializer function that puts default values to all fields
of message list item.  Fix all code that directly assigns some hash
to the elements of messagelist: for direct assignments only initializer
is now permitted, all other modification are done in-place.

Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
2014-08-03 16:47:26 +04:00
Eygene Ryabinkin
863113efa3 Match header names case-insensitively
http://tools.ietf.org/html/rfc5234#section-2.3 says that ABNF strings
are case-insensitive.

Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
2014-06-30 16:41:17 +04:00
Adam Spiers
807f3da880 addmessageheader(): add debug for header insertion
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
2014-06-24 23:36:48 +04:00
Adam Spiers
37f74d859a addmessageheader: fix case #2 and flesh out docstring
The example illustrations were slightly cryptic;
modify them to be more obvious.

Fix case #2 (message starts with two line breaks)
where additional line break was shown and coded.

Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
2014-06-24 23:25:45 +04:00
Adam Spiers
afead6c48e Rename addmessageheader()'s crlf parameter to linebreak
The parameter's value is a string representing the linebreak,
and can sometimes contain just '\n', in which case naming it
crlf is slightly misleading.

Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
2014-06-24 23:12:06 +04:00