Added the configuration setting usecompression for the IMAP repositories.
When enabled, the data from and to the IMAP server is compressed.
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
Make external API of class/module to be smaller, explicitely mark
all internal functions. Also annotate methods that are implemented
as the part of the parent class interface.
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
Added configuration option "auth_mechanisms" to the config file:
it is a list of mechanisms that will be tried in the specified order.
Author: Andreas Mack <andreas.mack@konsec.com>
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
It's nice to set up an ssh tunnel command which forwards an IMAP tcp
port inside an encrypted session, e.g. with ssh's "-W" flag. In this
case the tunnelled connection still requires authentication inside
IMAP session, because this is transport-only tunnel that substitutes
normal TCP/SSL connection.
New directive, 'transporttunnel' was added: it specifies the command
that will create the tunnel. Only one type of tunnel must be
specified for a single repository: we can't have both preauthenticated
and transport-type tunnels, they won't chain together.
From: Steve Purcell <steve@sanityinc.com>
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
- this method isn't as deprecated as IMAP LOGIN;
- it allows to keep hashed passwords on the server side;
- it has the ability to specify that the remote identity
is different from authenticating username, so it even
can be useful in some cases (e.g., migrated mailboxes);
configuration variable "remote_identity" was introduced
to leverage this functionality.
From: Andreas Mack <andreas.mack@konsec.com>
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
- created helper routine that will do authentication;
- routine tries each method in turn, first successful
one terminates it: makes things easier to read
and handle;
- renamed plainauth() inside offlineimap/imapserver.py
to loginauth(): the function does IMAP LOGIN authentication
and there is PLAIN SASL method, so previous name was
a bit misleading;
- slightly improved error reporting: all exceptions during
authentication will be reported at the end of the run;
- now loginauth() is never called if LOGINDISABLED is advertized
by the server; it used to be invoked unconditionally when
CRAM-MD5 fails, but we should respect server's opinion on
how to handle its users.
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
Make IDLE syncs be equal to the regular synchronisations
in respect to pre-sync and post-sync hooks.
From: mxgr7 <maxgerer@gmail.com>
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
Run the locked code under 'with': this guarantees that lock
will be released in any case.
This modification also avoids the case when our thread wasn't running
locked when exception was caught, another thread got the lock, our
code checked it via self.connectionlock.locked() and errorneously
released the lock thinking that is was running locked.
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
They are redundant in all pruned cases and sometimes even create some
problems, e.g., when one tries to jump through paragraphs in vi.
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
We now allow setting the SSL version used when connecting to IMAPS servers, and
do so via the `ssl_version` configuration option. We default to the current
practice (letting python's "ssl" library automatically detect the correct
version). There are however rare cases where one must specify the version to
use.
Signed-off-by: Ryan Kavanagh <rak@debian.org>
Replace low-level thread.get_ident() with threading.currentThread().ident.
This works both in python2.6 and python3. (thread is renamed _thread and its
direct use is not recommended)
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
'set' is builtin since python2.6, so remove the imports. Also 'ssl' exists
since 2.6 and has everything we need, so no need for conditional import
tests here anymore.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
These were needed for python <2.6 compatability, but since we depend on
python 2.6 now, these can go.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
imapserver.getdelim() was not used at all, so remove this function. The
folder delimiter is available via the repository.getsep() call.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
The remote|local|statusrepo is an anttribute of each SyncableAccount()
anyway, so we don't need to pass it in, we can simply get it from the
Account().
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
This function can IMHO lead to possible deadlocks when waiting for the
connectionlock. Do add a comment to that regard, this will need to audit.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
To make sure, the lock gets released even if we raise an exception between
acquire() and release()
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
During cleanup we often call releaseconnection in a finally: block. But
in cases of error, we might have dropped the connection earlier already
and set it to "None". In this case don't fail releaseconnection() but
make it a NOOP.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Drop a connection, if the NOOP to keep a connection open fails due to
broken connections.
Note that I believe this function is not working as intended. We grab
one random connection and send a NOOP. This is not enough to keep all
connections open, and if we invoke this function multiple times, we
might well always get the same connection to send a NOOP through.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
When checking for the IMAP4.abort() exception, we need of course to
perform:
except imapobj.abort:
and not
except imapobj.abort():
Thanks to Johannes Stezenbach <js@sig21.net> for pointing to the glitch.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Some Webservers (I am looking at you Gmail) send different capabilities
before and after login, so they can tailor their server capabilities to
the user. While legal, this is uncommon and we were not updating our
server capabilities. Doing so allows us to detect that Gmail actually
supports the UIDPLUS extension, and we will stop mangling headers when
uploading to Gmail. This could lead to some performance gains when we
upload many messages to Gmail.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Warn the user and abort when we attempt a plaintext login, but the
server has explicitly disabled plaintext logins.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
repos.getuesr() asks for a username if none is specified, but in the
case of a tunnel connection, we don't need one, so we need to skip the
repos.getuser() call here.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Finally, actually discard dropped connections when we detect them as an
imapobj.abort() has been thrown. In this case, invoke releaseconnection
with drop_conn=True.
We don't need the self.aborted attribute to get signified of dropped
connections. An Execption during the noop will do.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
while True: if a: return
is equivalent to
while not a:
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Variable name 'event' is as bad as it gets. Rename it to something that
actually describes what it is about.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Add code documentation throughout the idle() function.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Don't redefine the idle callback function on every run in the while
loop, define it once when we enter the function.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
If we connect to a SSL server (not STARTTLS) and no CA cert has been
specified for verification, we check the configured SSL fingerprint and
bail out in case it has not been set yet, or it does not match.
This means one more mandatory option for SSL configuration, but it
improves security a lot.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
If a connection is broken, we want to have it really dropped and not be
reused. So far, we are checking the .Terminate attribute for this, but
according to the imaplib2 author, it is only set on normal shutdown and
it is an undocumented attribute whose meaning could change any time.
This patch introduces the parameter drop_conn which allows to tell
releaseconnection() that we really want to connection being dropped from
the pool of available connections and properly destroy it.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Handle the case gracefully where a server has closed an IMAP connection
that we want to use for IDLEing. Simply have it dropped and get a new one
in this case. THis should get rid of the errors reported by John Wiegley
in mail id:"m2sjohd16t.fsf@gmail.com".
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
443 is of course the https and not the IMAPS standard port. Fix.
Thanks to Daniel Shahaf <d.s@daniel.shahaf.name> for the heads up.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
imaplib2 always attempts to verify a certificate if a verification
callback function is passed in, even the certificate is None
specified. Disable the verification excplictly by setting the
verification function to None in that case.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
we need errno.CONNREFUSED, but through some merging mishaps(?) the part
that actually imported errno was missing. Import the errno module.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
imaplib renamed self.sslobj to self.sock and our overriden open()
functions were failing for that reason when updating imaplib2 to
v2.28. It turns out that all of our custom initializations are being
done by stock imaplib2 now anyway, so there is no need to override them
anymore. This lets us simplify the code we have to worry about.
Move the verifycert() function to the imapserver.py file, it is now a
callback function that is being handed to imaplib from there, so it
makes sense to also define it in our imapserver function...
(this also lets us easily make use of the verifycert function in the
starttls case in the future)
TODO: we need to examine if and why we still need to override the
select() function, it is the only reason why we still wrap the IMAP4
classes.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
We set an imapobj.mustquote which apparently was used in previous
incarnations of imaplib or imaplib2, however, nothing in our codebase
makes use of that. So let us remove it.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Remove a level of wrapper abstraction that is not needed. Just use
IMAPserver and be done with it.
We do this by passing in the IMAPRepository() instance rather than a
long list of single paramters to the IMAPServer instanciation. This way
we can retrieve all repository parameters ourselves, rather than passing
a dozen paramters into IMAPServer. Also, this enables us to pass the
repository() object into our WrappedIMAP4() instance, so that it can
query, e.g. the SSL fingerprint configuration.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
In commit 89cbdc9, usage of SSLError was dropped but later reintroduced
without importing SSLError exception.
Signed-off-by: Arnaud Fontaine <arnau@debian.org>
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
We we using the variable 'severity' in a few places to throw
OfflineImapErrorrs of severity REPO. Somehow, that variable is now not
accessible in all places that refer to it, so we move where it is
defined to before all the 'if' checks which might make use of it.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
If we do not use a SSL connection anyway and if the server supports it,
authenticate automatically with STARTTLS.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
During a sync run, someone might remove or move IMAP messages. As we
only cache the list of UIDs in the beginning, we might be requesting
UIDs that don't exist anymore. Protect folder.IMAP.getmessage() against
the response that we get when we ask for unknown UIDs.
Also, if the server responds with anything else than "OK", (eg. Gmail
seems to be saying frequently ['NO', 'Dave I can't let you do that now']
:-) so we should also be throwing OfflineImapErrors here rather than
AssertionErrors.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
