Commit Graph

1483 Commits

Author SHA1 Message Date
Nicolas Sebrecht
90244b7fa6 provide more details in error message when SSL fails on non-standard port
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2017-06-30 05:10:14 +02:00
Nicolas Sebrecht
ba47138616 folder/IMAP: introduce dedicated parsing for davmail (not supporting UIDPLUS)
Some returned responses end with ')' rather than 'UID XXX)' as expected.

BTW, a better policy could be to request for the 'X-OfflineIMAP' header only
rather than fetching all the headers and looking for it manually.

Also:
- Strip the output when error occurs: we don't need the full response unless
  'imap' debug mode is enabled.
- Improve the comments.

Github-ref: https://github.com/OfflineIMAP/offlineimap/issues/479
Tested-by: https://github.com/secomi
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2017-06-27 20:32:58 +02:00
Nicolas Sebrecht
ce83efc3c7 folder/IMAP: improve the warning when we can't parse the returned UID
Github-ref: https://github.com/OfflineIMAP/offlineimap/issues/479
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2017-06-14 09:47:51 +02:00
Nicolas Sebrecht
05ff68c7e1 IMAP/IMAP: continue to sync if the local side does not return a valid UID on upload
There's no reason we should abort the full sync.

Github-ref: https://github.com/OfflineIMAP/offlineimap/issues/479
Tested-by: https://github.com/secomi
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2017-06-14 09:12:15 +02:00
Nicolas Sebrecht
6b51d4ed49 v7.1.1
Tested-by: Remi Locherer <remi.locherer@relo.ch>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2017-05-28 18:08:54 +02:00
Nicolas Sebrecht
f37b97c61b maxage: use the remote folder first to compute min_uid
On each folder scan, we must compute the min_uid. Locally, this is done by
relying on the prefix timestamp in the filenames. If for whatever reason they do
not reflect the Date header of the email, changing maxage leads to undefined
behaviour.

To prevent from this, we rather rely on the remote folder. We assume that the
remotes are more reliable to provide correct sets of UIDs based on dates than we
are.

Github-ref: https://github.com/OfflineIMAP/offlineimap/issues/384
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2017-05-21 00:50:04 +02:00
Nicolas Sebrecht
5d5ad62fa7 mbnames: don't duplicate entries in autorefresh mode
mbnames is initialized and written once in the run from OfflineImap.__sync().

However, the in-memory instance is fed with data at sync time for each folder
and the intermediate files are written as soon as all the folders are synced for
the account. All of this is done inside the SyncableAccount.__sync() method
while the syncrunner is looping on this.

This means that we duplicate entries for mbnames in each loop (most likely when
autorefresh is enabled).

It is wrong to have duplicates in mbnames for each account. Ignore duplicates
when adding data in the mbnames intermediate files.

Github-ref: https://github.com/OfflineIMAP/offlineimap/issues/467
Reported-and-tested-by: Shin Kojima <shin@kojima.org>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2017-05-19 08:32:22 +02:00
Nicolas Sebrecht
ffeefd9459 env info (used by -V and banner): add openssl version
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2017-05-02 17:09:03 +02:00
Nicolas Sebrecht
22a163a768 display the imaplib and python versions for each normal run
Some users might use more than one python/imaplib version. This tends to make
debugging and support harder.

Displaying this information by default for each run should help.

Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2017-04-30 13:25:40 +02:00
Nicolas Sebrecht
8c04684f3b imapserver: provide some SSL info while in imap debug mode
Github-ref: https://github.com/OfflineIMAP/offlineimap/issues/457
Tested-by: Maximilian Kaul <https://github.com/maximiliankaul>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2017-04-30 12:15:08 +02:00
Nicolas Sebrecht
39b657194a XOAUTH2: don't try this authentication method when not configured
Checking if we have to run this authentication method once it's already passed
to imaplib2 is too late. Make the checks at correct time, before we try the
authentication method. IOW, before we call

  imapobj.authenticate('XOAUTH2', self.__xoauth2handler)

Github-ref: https://github.com/OfflineIMAP/offlineimap/issues/442
Tested-by: Klemens Schölhorn <klemens@schoelhorn.eu>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2017-04-27 21:17:25 +02:00
Nicolas Sebrecht
e8f0e82f6c IMAP: UIDPLUS: correctly warn about weird responses from some servers
Github-ref: https://github.com/OfflineIMAP/offlineimap/issues/455
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2017-04-26 19:39:04 +02:00
Nicolas Sebrecht
e9d8e87a71 IMAP: UIDPLUS: improve error message on response error for new UID
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2017-04-26 18:40:51 +02:00
Ilias Tsitsimpis
7bc54d241c Acquire lock before updating the CursesLogHandler window
Make sure that we refresh the screen atomically, since the emit()
function may be called by more that one threads at a time.

Also, modify the draw_bannerwin() method which used to fail in case the
window would become too small. Make sure that the provided offsets to
the window.addstr() method are properly bounded.

Closes #160: blinkenlights display is broken
Tested-by: Cyril Brulebois
Tested-by: Gaudenz Steinlin
Signed-off-by: Ilias Tsitsimpis <i.tsitsimpis@gmail.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2017-04-23 12:48:12 +02:00
Nicolas Sebrecht
f669b3ea2f v7.1.0
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2017-04-16 00:40:54 +02:00
Nicolas Sebrecht
2c6fac6449 folder/IMAP: improve handling of "matchinguids" error while searching headers
Raise OfflineImapError instead of ValueError.

Github-ref: https://github.com/OfflineIMAP/offlineimap/issues/452
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2017-04-07 20:27:39 +02:00
Nicolas Sebrecht
c8206f24e1 remove support for the status_backend configuration option
Stop the run when this option is found. Migrating from the historical plain
text status cache is still supported.

Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2017-03-25 13:47:30 +01:00
Nicolas Sebrecht
ebf5fd527e folder: UIDMaps: ignore KeyError failure while removing keys
This would require more digging to understand how this is possible. I suspect
that a previous run has been interrupted.

Github-ref: https://github.com/OfflineIMAP/offlineimap/issues/445
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2017-03-22 14:33:31 +01:00
Nicolas Sebrecht
c883160814 v7.0.14
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2017-03-11 13:22:18 +01:00
Nicolas Sebrecht
f9360ba2d7 bump from imaplib2 v2.55 to v2.57
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2017-03-05 13:28:47 +01:00
Nicolas Sebrecht
fc079499e9 prevent synchronization of identical folders from multiple threads
Use a mutex to lock the sync transaction, uniquely identified by the remote
folder name.

Serializing the syncs prevents any IDLE thread from syncing the same folder of
a concurrent autorefresh thread. Othewise, we could download new messages
twice, or read "one message past the last one" which in some cases creates a
duplicated messages.  Also prevents IDLE threads from reading new messages
before the autorefresh thread finishes synchronization.

Github-ref: #421
Originally-submitted-by: Michael Hohmuth <hohmuth@sax.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2017-02-28 19:16:18 +01:00
Stéphane Graber
e3bbf75feb Fix ipv6 configuration handling
According to documentation and the code, the following behavior is expected:

ipv6 = no
    AF_INET is used

ipv6 = yes
    AF_INET6 is used

ipv6 undefined
    AF_UNDEF is used

Unfortunately the code parsing the "ipv6" configuration option was
returning "False" rather than "None" when failing to locate the option,
making it impossible to detect whether "ipv6" isn't set or if it was set
to "false" and leading offlineimap to use AF_INET for the connection.

This fixes the use of offlineimap on hosts that occasionaly are
connected to IPv6-only networks.

Signed-off-by: Stéphane Graber <stgraber@ubuntu.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2017-02-03 18:43:02 +01:00
Nicolas Sebrecht
4df06d57c3 SQLite: avoid concurrent writes on backend migration
The saveall() method must acquire the lock to make writes.

Reported-and-tested-by: Julien Cubizolles <j.cubizolles@free.fr>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2017-01-31 17:59:25 +01:00
Nicolas Sebrecht
c1df95b7d3 v7.0.13
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2017-01-27 19:49:42 +01:00
Nicolas Sebrecht
ba52030923 folder: IMAP: improve error message when Dovecot returns any data for UID FETCH
Github-ref: https://github.com/OfflineIMAP/offlineimap/issues/429
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2017-01-24 19:11:16 +01:00
Nicolas Sebrecht
6c0828b77c folder: IMAP: add missing whitespace in error message
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2017-01-24 16:08:41 +01:00
Chris Smart
eb2bd80851 init: register SIGABRT and handle as per SIGUSR2
systemd supports a watchdog (via the WatchdogSec service file option)
which will send the program a SIGABRT when the timer expires, however
currently this causes offlineimap to be killed immediately.

This patch registers SIGABRT and handles it in the same manner as
SIGUSR2, so that the current synchronisation is completed before the
program exits safely.

This makes offlineimap more flexible and robust for persistent setups
that make use of holdconnectionopen and autorefresh options.

For example, it may be useful in assisting with the occasional
situation where offlineimap may not return successfully after a suspend
and resume.

To make use of this, users could add the following to the [Service]
section of their systemd offlineimap service file (restart every 5
minutes):

Restart=on-watchdog
WatchdogSec=300

Signed-off-by: Chris Smart <mail@csmart.io>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2017-01-23 14:20:07 +01:00
Nicolas Sebrecht
11655cb613 repository: IMAP: correctly check the response while listing remote folders
Github-ref: https://github.com/OfflineIMAP/offlineimap/issues/423
Reported-by: Periklis Tsirakidis <periklis.tsirakidis@mayflower.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2017-01-22 19:20:32 +01:00
lkcl
dab5737265 learn repository retrycount configuration option
Allow retrying the download of messages more than twice.

Signed-off-by: Luke Kenneth Casson Leighton <lkcl@lkcl.net>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-12-19 12:55:55 +01:00
lkcl
dda27e0962 learn authproxy configuration option
Allow to use a another proxy for authentication than for IMAP.

Signed-off-by: Luke Kenneth Casson Leighton <lkcl@lkcl.net>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-12-19 12:20:19 +01:00
Nicolas Sebrecht
d5edf2dc11 v7.0.12
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-11-30 22:40:59 +01:00
Abdo Roig-Maranges
91486482cc repository: Base: fix typo in folder variable name
Probably fixes #417.

Signed-off-by: Abdo Roig-Maranges <abdo.roig@gmail.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-11-30 22:33:05 +01:00
Darshit Shah
b5a4925efc Enable environment variable expansion on Repository.localfolders
Commit e51ed80e claims to add tilde and environment variable expansion
to multiple locations including Repository.localfolders. However, this
particular options seems to have been missed in that commit, and
apparently no one noticed till date

Signed-off-by: Darshit Shah <darnir@gmail.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-11-30 19:22:57 +01:00
Nicolas Sebrecht
ed79e56f8e v7.0.11
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-11-30 14:33:44 +01:00
Nicolas Sebrecht
2e2684d445 repository: Base: fix folder structure comparison
481efa95f6 is wrong to assume the local nametrans is set up and working.

Now, we correctly check that the local folder name is the same once both
nametrans are applied to itself. This check is only done when we have to create
the folder on remote.

Github-ref: https://github.com/OfflineIMAP/offlineimap/issues/413
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-11-30 14:27:33 +01:00
Nicolas Sebrecht
683ae2b77b v7.0.10
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-11-28 19:10:28 +01:00
Abdo Roig-Maranges
5c59d3ecbf repository: Base: fix name of the status folder
When creating a folder on the remote, the status folder was created
using the remote folder name, but then Account assumed the statusfolder
name matched the local folder. Correct this.

Signed-off-by: Abdo Roig-Maranges <abdo.roig@gmail.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-11-28 14:11:01 +01:00
Nicolas Sebrecht
481efa95f6 repository: Base: rework the structure folders comparison
Ensure we work on the correct names when coparing the structures.

This might revert changes made in 22641331c1 and would require mode
checks. However, having correct folder structure comparison is more important
than having the not official UTF-8 support.

Github-ref: https://github.com/OfflineIMAP/offlineimap/issues/405
Tested-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-11-22 22:29:29 +01:00
Nicolas Sebrecht
04ae3c8dad folder: IMAP: display error message before starting next try
Fetching messages is tried more than once. Display the error message at correct
time.

Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-11-22 00:58:14 +01:00
Nicolas Sebrecht
54bbe79930 accounts: minor code refactoring
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-11-20 20:22:37 +01:00
Nicolas Sebrecht
e3ad46a27b Merge branch 'avar/pr404' into next
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-11-17 17:58:13 +01:00
Ævar Arnfjörð Bjarmason
7b59d6b3ae GMail: Add ability to set a custom host/port/ssl etc.
The GMail repository added in 81b86fb has the restriction that you
can't override the host/port/ssl etc. There's no good reason for this,
and I'd like to use some of the defaults while connecting to GMail via
an stunnel, i.e.:

    $ grep -v -e ^# -e ^$ .stunnel.work-gmail-imap
    foreground = yes
    debug = info
    pid =
    [work-imap-gmail-ssl-tunnel]
    client = yes
    accept = 127.0.0.1:1431
    connect = imap.gmail.com:993
    delay = yes

And then in my .offlineimaprc:

    type = Gmail
    remotehost = localhost
    remoteport = 1431
    ssl = no

While I was at it I also allowed overriding all the other previously
hardcoded options, e.g. the "oauth2_request_url" can now be overridden
to e.g. a local non-https proxy to sniff & debug the traffic going
over the wire.

This doesn't break any existing configuration, since we still use the
provided defaults. It just allows odd uses like my own to use this
repository while e.g. not having to supply "trashfolder", "spamfolder"
& "oauth2_request_url". While I'm at it document in the
offlineimap.conf that we're setting those options to defaults in this
repository.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-11-17 17:53:38 +01:00
Abdo Roig-Maranges
c4fcd9f7f2 fix remaining instance of check_uid_validity refactoring
Signed-off-by: Abdo Roig-Maranges <abdo.roig@gmail.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-11-10 11:24:33 +01:00
Nicolas Sebrecht
673edb0b9f fix the profile mode
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-11-08 07:11:33 +01:00
Nicolas Sebrecht
5eb131ef2b accounts: minor refactoring
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-11-08 07:11:30 +01:00
Nicolas Sebrecht
01a744a0f8 folder: Maildir: actually try to use Delivery-Date if Date is broken
This value is used to set the timestamp prefix in the filename.

Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-11-07 23:14:32 +01:00
Nicolas Sebrecht
e452c344d9 minor code refactoring
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-11-07 23:03:40 +01:00
Nicolas Sebrecht
9606cfcfc1 don't allow negative values for autorefresh
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-11-07 23:01:15 +01:00
Nicolas Sebrecht
15eac5b3cb repository: Gmail: fix copyright line
Minor code rewriting, too.

Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-11-05 19:27:21 +01:00
Nicolas Sebrecht
0fe4562269 manual: add known issues entry about XOAUTH2 "invalid_grant"
- Split the manual asciidoc source file.
- Improve the Makefile to build the manuals.

Github-ref: https://github.com/OfflineIMAP/offlineimap/issues/374
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-11-05 19:20:39 +01:00