Commit Graph

67 Commits

Author SHA1 Message Date
Olivier Mehani
0d5496ba0a implement Happy Eyeballs
This allows OfflineIMAP to not stall on malfunctional IPv6 connections,
and fall-back to a functional IPv4 connection, if faster, as described
in RFC6555.

Signed-off-by: Olivier Mehani <shtrom@ssji.net>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2019-06-10 14:36:53 +02:00
velleto
5180b964d8 Removed uneccessary call of list() on zip() object.
Signed-off-by: velleto <rr@velleto.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2018-05-15 00:38:06 +02:00
velleto
ac9ed47262 Changed the 'exception raised' message, to be more understandable.
Signed-off-by: velleto <rr@velleto.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2018-05-15 00:38:06 +02:00
velleto
b150daaa49 Added support for sha512, sha384, sha256, sha224 hashing algorithms to calculate server certificate fingerprints.
Signed-off-by: velleto <rr@velleto.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2018-05-15 00:38:06 +02:00
Nicolas Sebrecht
03859e89c3 improve error message when ssl_version must be set due to the tls_level
Improve the ssl documentation.

Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-07-25 03:29:25 +02:00
Nicolas Sebrecht
7945e10a76 correctly reraise errors with six
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-07-03 19:56:14 +02:00
Nicolas Sebrecht
acc9c3cf49 introduce a virtual imaplib2
The virtual imaplib2 first try to import imaplib2 when provided by the system.
If not provided or if the version is not supported, fallback on the bundled
imaplib2 version.

Distribution maintainers can now easily remove the bundled imaplib2 version if
they want to get it packaged outside of offlineimap.

We still want to provide imaplib2 by default because:
- this library is neither in Python core nor packaged by a lot of distributions;
- users expect to be able to run offlineimap by just downloading the tarball or
  after a git clone.

In order to avoid unexpected (too old) versions of imaplib2, we restrict the
supported versions of this librabry.

Reviewed-by: Łukasz Żarnowiecki <dolohow@outlook.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-06-04 12:17:53 +02:00
Łukasz Żarnowiecki
0addcbabf0 py3: raise exceptions using six module
There is no other way to make Python2 and Python3 happy, because syntax
raise E, V, T is incompatible with the latter.

Signed-off-by: Łukasz Żarnowiecki <dolohow@outlook.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-05-18 01:49:09 +02:00
Łukasz Żarnowiecki
c327bec6c8 Convert except X,T to except X as T
Signed-off-by: Łukasz Żarnowiecki <dolohow@outlook.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-05-10 02:53:08 +02:00
Ebben Aries
9bfd610230 change hard coding of AF_UNSPEC to user-defined address-families per repository
Some environments that return AAAA records for their IMAP servers can pose
problems for clients that do not have end-to-end IPv6 connectivity for a number
of reasons (e.g. policy, lack of full routing, security, etc..)

Even with a fallback mechanism in place, you can still arrive at IMAP
implementations that could prevent authentication from unknown IPv6 space. This
in itself is not enough to fallback to IPv4 since there is an actual connection
on that socket.

This change is for introducing a user-defined value:

  [Repository imap-remote]
  ipv6 = no

to create a preference per repository on which AF to connect to the remote
server on

ipv6 = yes (AF_INET6)
ipv6 = no (AF_INET)
unspecified = default

Signed-off-by: Ebben Aries <e@dscp.org>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2016-02-23 06:57:21 +01:00
Nicolas Sebrecht
3a056519f9 Merge branch 'pilou--tests1' into next 2015-04-02 19:32:37 +02:00
Pierre-Louis Bonicoli
1cdf34e100 remove unnecessary imaplib2 workaround
Upstream bug has been fixed, since imaplib2 v2.42 the untagged responses are
flushed (as stated by the documentation).
See https://sourceforge.net/p/imaplib2/bugs/7/

Signed-off-by: Pierre-Louis Bonicoli <pierre-louis.bonicoli@gmx.fr>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2015-04-02 19:32:15 +02:00
Janna Martl
428349e3f4 fix inaccurate UI messages when some messages are excluded from the cached lists
Some messages were excluded from the copy/delete list after the UI message said
they were copied/deleted.
Also fix Internaldate2epoch(), which was wrong.

Signed-off-by: Janna Martl <janna.martl109@gmail.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2015-03-21 14:10:57 +01:00
Nicolas Sebrecht
ca06819e70 imaplibutil: add missing errno import
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2015-02-17 10:34:20 +01:00
夏恺(Xia Kai)
d36209a305 add proxy support powered by PySocks
Read proxy option in imapserver, instantiate a class in imaplibutil
using a self-defined keyword and a socket instance, and use this socket
instance to substitute the default socket instance used in imaplib2.

Signed-off-by: 夏恺(Xia Kai) <xiaket@gmail.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2015-02-17 10:34:20 +01:00
Nicolas Sebrecht
461554b7b1 more style consistency
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2015-02-10 17:25:00 +01:00
Eygene Ryabinkin
e7fabf9e57 Properly re-raise exception to save original tracebacks
We usually mutate some exceptions to OfflineImapError() and it is
a whole lot better if such exception will show up with the original
traceback, so all valid occurrences of such mutations were transformed
to the 3-tuple form of "raise".  Had also added coding guidelines
document where this re-raise strategy is documented.

Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
2015-01-12 17:51:21 +03:00
Nicolas Sebrecht
61021260cb more consistent style
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2015-01-07 21:31:43 +01:00
Nicolas Sebrecht
54cff7f786 imaplibutil.py: remove unused imports
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2015-01-01 22:23:35 +01:00
Eygene Ryabinkin
d2ec2a4e9e Extend handling of cert_fingerprint
Add ability to specify multiple fingerprints.

Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
2014-05-11 22:45:34 +04:00
Eygene Ryabinkin
6cbd2498ae Refactoring: make functions to be private if we can
Make external API of class/module to be smaller, explicitely mark
all internal functions.  Also annotate methods that are implemented
as the part of the parent class interface.

Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
2014-04-30 19:35:56 +04:00
Eygene Ryabinkin
131ac5c827 IMAPlib mixin class: pass 'readonly' exception to our callers
This will allow our callers who are capable of dealing with
readonly folders to properly detect this condition and act
accordingly.

One example is Gmail's "Chats" folder that is read-only,
but contains logs of the quick chats.

Minor Changelog improvements.

Tested-by: Abdó Roig-Maranges <abdo.roig@gmail.com>
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2012-08-11 11:59:26 +02:00
Sebastian Spaeth
895e709bf2 Fix SSL fingerprint was not always checked
As reported by James Cook, we would not check the fingerprint of the SSL
server, as we were looking for the 'ssl' module in locals() rather than
globals(). Ooops!

Rather than using globals() though, I simply remove the by-now
superfluous check. We now rely on python2.6 and we unconditionally
import the SSL module in any case, so it needs to be there.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2012-04-19 18:26:29 +02:00
Sebastian Spaeth
5c598d7e74 dict.has_key(a) --> a in dict
has_key() is gone in python3, so use a more modern syntax here.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2012-02-06 17:41:43 +01:00
Sebastian Spaeth
81fc20c7ca Remove python<2.6 import workarounds (set & ssl)
'set' is builtin since python2.6, so remove the imports. Also 'ssl' exists
since 2.6 and has everything we need, so no need for conditional import
tests here anymore.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2012-02-06 17:41:42 +01:00
Sebastian Spaeth
0844d27f9f except Ex, e: --> except Ex as e:
Nudge us towards python3 compatability by converting deprecated python2 syntax.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2012-02-06 17:41:42 +01:00
Sebastian Spaeth
50de2174bf Allow to pass 'force' arg to selectro() to enforce a new select
Pass through the 'force' argument from selectro() to select() so that it
can also enforce a new SELECT even if we already are on that folder.

Also change the default parameter from '0' to 'False' to make clear that
this is a Bool.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2012-01-08 11:29:54 +01:00
Sebastian Spaeth
33b4a16dac Fix mbox.select(foldername) readonly parameter comparison
The default parameter value was "None", and we were comparing that
directly to the imaplib2 value of is_readonly which is False or True, so
the comparison always returned "False".

Fix this by setting the default parameter to "False" and not
"None". Also convert all users of that function to use False/True.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-11-02 08:57:01 +01:00
Sebastian Spaeth
bc10e05600 Check for SSL module existence and only do fingerprint check then
Python 2.5 has no ssl module, and we can therefor not get the server
certificate for fingerprint verification. Add a check that disables
fingerprint verification for python 2.5.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-12 19:45:16 +02:00
Sebastian Spaeth
eafea0c880 IMAP4.sslobj -> sock
imaplib2 has changed internally to use self.sock for its ssl socket when
it used to be sslobj. Reflect that change.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-12 19:45:16 +02:00
Sebastian Spaeth
8800fa37a3 Implement Server SSL fingerprint check
If we connect to a SSL server (not STARTTLS) and no CA cert has been
specified for verification, we check the configured SSL fingerprint and
bail out in case it has not been set yet, or it does not match.

This means one more mandatory option for SSL configuration, but it
improves security a lot.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-12 19:45:15 +02:00
Sebastian Spaeth
61fffd5c5f imaplibutil: Have SELECT throw an OfflineImapError when the connection died
Imapserver.acquireconnection will succeed even whent the server connection
has been terminated and the first IMAP operation will throw an exception.

Often this is the folder SELECT operation (e.g. after an idle timeout), as
has been reported by John Wiegley. Catch this case and throw an
OfflineImapError with severity FOLDER_RETRY to notify consumers that they
are supposed to retry.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-06 19:00:21 +02:00
Sebastian Spaeth
3a91e296f0 Adapt the code to work with the new imaplib2
imaplib renamed self.sslobj to self.sock and our overriden open()
functions were failing for that reason when updating imaplib2 to
v2.28. It turns out that all of our custom initializations are being
done by stock imaplib2 now anyway, so there is no need to override them
anymore. This lets us simplify the code we have to worry about.

Move the verifycert() function to the imapserver.py file, it is now a
callback function that is being handed to imaplib from there, so it
makes sense to also define it in our imapserver function...
(this also lets us easily make use of the verifycert function in the
starttls case in the future)

TODO: we need to examine if and why we still need to override the
select() function, it is the only reason why we still wrap the IMAP4
classes.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-15 12:21:38 +02:00
Sebastian Spaeth
3c89a72be9 remove unused 'mustquote' regex
We set an imapobj.mustquote which apparently was used in previous
incarnations of imaplib or imaplib2, however, nothing in our codebase
makes use of that. So let us remove it.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-14 17:15:38 +02:00
Sebastian Spaeth
c293e64119 Fix caching inconsistencies in getselectedfolder()
getselectedfolder was using a cached variable that we were setting in
select(), but sometimes the IMAP4 instance got into the SELECTED state
without explicitely select()ing, it seems, and our variable was unset.

Let us just use the self.mailbox variable that imaplib2 is setting when
select()ing rather than doing our own caching. Also remove the part
where we were setting the cache.

Just access self.state rather than looking up self.state via
self.getstate() every time, it is just an unnecessary layer of
redirection.

Original-patch-by: Arnaud Fontaine <arnau@debian.org>
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-07-07 18:33:45 +02:00
Sebastian Spaeth
d122dde93f Check SSL certificate for expiration
We currently don't care about expiration dates of the servers SSL
certificate. This patch adds a check that fails Cert verification when
it is past its due date. There is no way or option to override this
check.

Unfortunately we only seem to be able to get SSL certificate data when
we passed in a CA cert file? How do we get that date when we don't have
a ca cert file?

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-06-13 16:47:03 +02:00
Nicolas Sebrecht
fc6c12d96c improve message "error 111" if connection failed
Raise OfflineImapError with severity REPO explaining that the connection failed.
Before, no valuable information was given to the user.

Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-05-25 20:32:41 +02:00
Nicolas Sebrecht
a39128516b explicitly define symbols to import instead of 'import *'
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-05-09 22:43:03 +02:00
Sebastian Spaeth
b3a383d151 accounts: handle OfflineImapError severity FOLDER
Throw an OfflineImapError when SELECTing a folder is unsuccessful and
bail out with a FOLDER serverity. In accounts.py catch all
OfflineImapErrors and either just log the error and skip the folder or
bubble it up if it's severe.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-05-08 13:56:04 +02:00
Ethan Glasser-Camp
9e734006f6 Fix IMAP4_Tunnel to work with imaplib2
* IMAP4_Tunnel constructor should support base-class arguments, in
  order to support the timeout argument.

* IMAP4_Tunnel needs to store the member IMAP4.host, which is normally
  done in IMAP4.open().

* Update IMAP4_Tunnel.read() and IMAP4_Tunnel.send().  We turn on
  nonblocking mode for these sockets, so we can return immediately
  with whatever data is available.

Signed-off-by: Ethan Glasser-Camp <ethan@betacantrips.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-04-13 18:41:53 +02:00
Vincent Beffara
bb48b6deaf Get rid of the UsefulIMAP4 classes
The three classes with names starting with UsefulIMAP4 were used for two
purposes, to include the UsefulIMAPMixIn class and to implement various
system-specific kludges. None of these kludges remain, so it is cleaner
to include UsefulIMAPMixIn directly in imaplibutil and forget about them
for good.

Signed-off-by: Vincent Beffara <vbeffara@ens-lyon.fr>
Reviewed-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-03-23 20:00:35 +01:00
Vincent Beffara
9f03f41b70 Move UsefulIMAPMixIn to imaplibutil.py
Now that we do not need any system-specific hack, the three UsefulIMAP4
classes have become empty and should be removed. This implies importing
UsefulIMAPMixIn directly from the classes defined in imaplibutil.

Prepare this change by moving the code into imaplibutil.py. Functionally
this is a no-op.

Signed-off-by: Vincent Beffara <vbeffara@ens-lyon.fr>
Reviewed-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-03-23 20:00:30 +01:00
Nicolas Sebrecht
44eefae043 cleanup import statements and conform to PEP-8
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-03-14 21:35:33 +01:00
Ethan Glasser-Camp
197030da1a Remove obsolete read(), readline(), _read_upto() methods
For read(), the imaplib2 version seems to work perfectly well. The
others aren't used any more, either by imaplib2, nor by us, so we may
as well get rid of them.

Signed-off-by: Ethan Glasser-Camp <ethan@betacantrips.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-03-14 19:51:56 +01:00
Ethan Glasser-Camp
1bf4bee5e6 Update to match semantics of new imaplib2
The biggest change here is that imapobj.untagged_responses is no
longer a dictionary, but a list. To access it, I use the semi-private
_get_untagged_response method.

* offlineimap/folder/IMAP.py (IMAPFolder.quickchanged,
  IMAPFolder.cachemessagelist): imaplib2 now explicitly removes its
  EXISTS response on select(), so instead we use the return values from
  select() to get the number of messages.

* offlineimap/imapserver.py (UsefulIMAPMixIn.select): imaplib2 now
  stores untagged_responses for different mailboxes, which confuses us
  because it seems like our mailboxes are "still" in read-only mode when
  we just re-opened them.  Additionally, we have to return the value
  from imaplib2's select() so that the above thing works.

* offlineimap/imapserver.py (UsefulIMAPMixIn._mesg): imaplib2 now
  calls _mesg with the name of a thread, so we display this
  information in debug output. This requires a corresponding change to
  imaplibutil.new_mesg.

* offlineimap/imaplibutil.py: We override IMAP4_SSL.open, whose
  default arguments have changed, so update the default arguments. We
  also subclass imaplib.IMAP4 in a few different places, which now
  relies on having a read_fd file descriptor to poll on.

Signed-off-by: Ethan Glasser-Camp <ethan@betacantrips.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-03-14 19:51:56 +01:00
Ethan Glasser-Camp
f9413226b8 Import imaplib2 instead of imaplib
imaplib2 has slightly different semantics than standard imaplib, so
this patch will break the build, but I thought it was helpful to have it as
a separate commit.

Signed-off-by: Ethan Glasser-Camp <ethan@betacantrips.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-03-14 19:51:56 +01:00
Nicolas Sebrecht
7b8d7501d1 Merge branch 'master' into next
Conflicts:
	Changelog.draft.rst
	offlineimap/imapserver.py
2011-03-07 21:55:43 +01:00
Haojun Bao
b94bf79258 fix hang because of infinite loop reading EOF
Read() should return empty string when EOF happen, instead of looping
forever. This is the right semantics of read(), and a wrapped version
should not change it.

If you read the read(2) system call manpage, it tells you that when EOF
is seen, return value is 0; it does not say

	``loop forever when EOF happen''.

After the EOF detection is patched you can see the
following exception:

  WARNING: ERROR attempting to copy message 344 for account Gmail:Traceback (most recent call last):
    File "/usr/lib/pymodules/python2.6/offlineimap/folder/Base.py", line 282, in copymessageto
      message = self.getmessage(uid)
    File "/usr/lib/pymodules/python2.6/offlineimap/folder/IMAP.py", line 216, in getmessage
      initialresult = imapobj.uid('fetch', '%d' % uid, '(BODY.PEEK[])')
    File "/usr/lib/python2.6/imaplib.py", line 753, in uid
      typ, dat = self._simple_command(name, command, *args)
    File "/usr/lib/python2.6/imaplib.py", line 1060, in _simple_command
      return self._command_complete(name, self._command(name, *args))
    File "/usr/lib/python2.6/imaplib.py", line 890, in _command_complete
      raise self.abort('command: %s => %s' % (name, val))
  abort: command: UID => socket error: EOF

Signed-off-by: Bao Haojun <baohaojun@gmail.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-03-07 21:54:28 +01:00
Gábor Melis
966841c8d6 Allow SSL connections to send keep-alive messages
This seems to prevent mysterious hangs with SSL imap servers (especially
gmail?) and does not harm in any case. So let us enable keep-alive
messages for ssl connections.
Our thread pool should be made more robust against closed SSL
connections (which do not always seem to raise Exceptions), and not
deadlock while waiting for resources or data that will never arrive.

Reviewed-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Gábor Melis <mega@retes.hu>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-03-01 18:55:29 +01:00
Thomas Jost
838a67bc40 Support subjectAltName in SSL certificates
Signed-off-by: Thomas Jost <schnouki@schnouki.net>
Reviewed-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-02-23 18:27:44 +01:00