Make external API of class/module to be smaller, explicitely mark
all internal functions. Also annotate methods that are implemented
as the part of the parent class interface.
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
Allow people who want folder filtering to depend on the external
conditions or to make it dynamic for other reasons, to do what
they want.
New repository configuration knob 'dynamic_folderfilter' was
introduced; it defaults to 'False' that matches historical behaviour.
GitHub: #73
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
This commit fixes the case when we're invoking releaseconnection()
for a given imapobj twice.
This bug manifests itself as
{{{
ValueError: list.remove(x): x not in list
File "[...]/offlineimap/folder/IMAP.py", line 615, in savemessage
self.imapserver.releaseconnection(imapobj)
File "[...]/offlineimap/imapserver.py", line 130, in releaseconnection
self.assignedconnections.remove(connection)
}}}
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
When imaputil was parsing quoted strings, it treated "abcd\\"
as incomplete quoted string having escaped quote, rather than
properly-quoted string having escaped backslash.
GitHub issue: https://github.com/OfflineIMAP/offlineimap/issues/53
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
2.36 it includes support for SSL version override that was integrated
into our code before, no other changes.
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
- Fix for Gmail sending a BYE response after reading >100 messages
in a session.
- Includes fix for GitHub#15: patch was accepted upstream.
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
Second argument is exception traceback, not the message; without this
tracebacks like mentioned in
http://permalink.gmane.org/gmane.mail.imap.offlineimap.general/5712
were happening when this exception handling block was hit.
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
This greatly simplifies developer's life and will, possibly, allow
users familiar with Python to debug and fix the problems by
themselves.
We, possibly, should not give tracebacks for the problems like
"can't open connection", but this is up to the caller of this
routine not to provide traceback in this case.
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
self._expecting_data was used both as the expected data length and the
flag that we expect some data. This obviously fails when advertized
data length is zero, so self._expecting_data_len was introduced to
hold the length of the expected data and self._expecting_data was left
as the flag that we expect the data to come.
GitHub issue: https://github.com/OfflineIMAP/offlineimap/issues/15
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
It is called localfolders and holds expanded name for the same
variable for the local repository of the account that is being
processed.
GitHub issue: https://github.com/OfflineIMAP/offlineimap/issues/21
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
Bring the description in the template offlineimap.conf in sync to the
actual implementation: pass folder names to the sorting function, not
the offlineimap.folder.IMAP.IMAPFolder objects themselves.
GitHub issue: https://github.com/OfflineIMAP/offlineimap/issues/27
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
Global or per-repository option utime_from_message tells OfflineIMAP
to set file modification time of messages pushed from one repository
to another basing on the message's "Date" header.
This is useful if you are doing some processing/finding on your
Maildir (for example, finding messages older than 3 months),
without parsing each file/message content.
From: Cyril RUSSO <boite.pour.spam@gmail.com>
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
Added configuration option "auth_mechanisms" to the config file:
it is a list of mechanisms that will be tried in the specified order.
Author: Andreas Mack <andreas.mack@konsec.com>
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
It's nice to set up an ssh tunnel command which forwards an IMAP tcp
port inside an encrypted session, e.g. with ssh's "-W" flag. In this
case the tunnelled connection still requires authentication inside
IMAP session, because this is transport-only tunnel that substitutes
normal TCP/SSL connection.
New directive, 'transporttunnel' was added: it specifies the command
that will create the tunnel. Only one type of tunnel must be
specified for a single repository: we can't have both preauthenticated
and transport-type tunnels, they won't chain together.
From: Steve Purcell <steve@sanityinc.com>
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
- this method isn't as deprecated as IMAP LOGIN;
- it allows to keep hashed passwords on the server side;
- it has the ability to specify that the remote identity
is different from authenticating username, so it even
can be useful in some cases (e.g., migrated mailboxes);
configuration variable "remote_identity" was introduced
to leverage this functionality.
From: Andreas Mack <andreas.mack@konsec.com>
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
- created helper routine that will do authentication;
- routine tries each method in turn, first successful
one terminates it: makes things easier to read
and handle;
- renamed plainauth() inside offlineimap/imapserver.py
to loginauth(): the function does IMAP LOGIN authentication
and there is PLAIN SASL method, so previous name was
a bit misleading;
- slightly improved error reporting: all exceptions during
authentication will be reported at the end of the run;
- now loginauth() is never called if LOGINDISABLED is advertized
by the server; it used to be invoked unconditionally when
CRAM-MD5 fails, but we should respect server's opinion on
how to handle its users.
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
IMAP servers can return `NO` responses to the `APPEND` command,
e.g. here's an example response from Groupwise's IMAP server:
NO APPEND The 1500 MB storage limit has been exceeded.
In this case, savemessage() should abort the repository sync rather
than returning UID 0 which would cause the local copy of the message
being saved to get irreversibly deleted.
Signed-off-by: Adam Spiers <offlineimap@adamspiers.org>
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
mutt-sidebar and, probably, other MUA show mailboxes in the order
they are listed in the file written by mbnames. Therefore, to allow
customization of the order with which mailboxes are listed, introduce
the new 'sort_keyfunc' directive in the [mbnames] section.
'sort_keyfunc' must be a function that will be called once for each
mailbox. It must accept the only argument -- a dict with 2 items,
'accountname' and 'foldername', and should return an object that
will be used as the sorting key for each mailbox.
Default key function returns (d['accountname'], d['foldername']),
thus sorting by account name and then by the folder name.
Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
Make IDLE syncs be equal to the regular synchronisations
in respect to pre-sync and post-sync hooks.
From: mxgr7 <maxgerer@gmail.com>
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
Run the locked code under 'with': this guarantees that lock
will be released in any case.
This modification also avoids the case when our thread wasn't running
locked when exception was caught, another thread got the lock, our
code checked it via self.connectionlock.locked() and errorneously
released the lock thinking that is was running locked.
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
They are redundant in all pruned cases and sometimes even create some
problems, e.g., when one tries to jump through paragraphs in vi.
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
We now allow setting the SSL version used when connecting to IMAPS servers, and
do so via the `ssl_version` configuration option. We default to the current
practice (letting python's "ssl" library automatically detect the correct
version). There are however rare cases where one must specify the version to
use.
Signed-off-by: Ryan Kavanagh <rak@debian.org>
This eases testing of option values inside the code. This instance
is implemented as the read-only copy of the obtained 'options' object,
so callers won't be able to modify its contents.
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
The problem lies in the fact that offlineimap.folder.Base's method
syncmessagesto_copy() uses threaded code everytime it is suggested by
the derived class's suggeststhreads() (currently, only IMAP does this
suggestion), but offlineimap/init.py will not spawn the
exitnotifymonitorloop() from offlineimap.threadutil.
The root cause is that ExitNotifyThread-derived threads need
offlineimap.threadutil's exitnotifymonitorloop() to be running the
cleaner for the exitthreads Queue(), because it fills the queue via
the run() method from this class: it wants to put() itself to the
Queue on exit, so when no exitnotifymonitorloop() is running, the
queue will fill up. And if this thread is an instance of
InstanceLimitedThread that hits the limit on the number of threads,
then it will hold the instancelimitedsems[] semaphore will prevent
other InstanceLimitedThread()s of the same name to pass its start()
method.
The fix is to avoid using threaded code if we're running
single-threaded.
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
Obtained-from: X-Ryl669 <boite.pour.spam@gmail.com>
This is handy when we're debugging the thread locks: we can try to
understand which thread does what and how it was called.
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
offlineimap has several frontends that encourage running it from a
terminal under an X session. When X session closes for a system
shutdown, the terminals exit, after sending SIGHUP to their children.
Previously SIGHUP was treated to be equivalent to SIGUSR1, i.e. wake
up and sync all accounts. This causes delays during shutdown.
According to Wikipedia [0], SIGHUP has been repurposed from a
historical meaning to one of:
* re-read configuration files, or reinitialize (e.g. Apache, sendmail)
* controlling pseudo or virtual terminal has been closed
I believe second meaning is more appropriate for offlineimap, and
hence this patch makes SIGHUP to be handled in the same way SIGTERM
and SIGINT are handled.
[0] http://en.wikipedia.org/wiki/SIGHUP
Debian-Bug: http://bugs.debian.org/670120
Reported-By: Steve Langasek <steve.langasek@canonical.com>
Signed-off-by: Dmitrijs Ledkovs <xnox@debian.org>
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
Fix imapfolder.getmessageinternaldate misparsing the Date:
header from emails due to a bug or surprising behaviour by
email.utils.parsedate. This is because email.utils.parsedate's
return value contains the unadjusted hour value from the string
parsed but does not include information about the time zone in
which it is specified. For example (Python 2.7.3):
$ python -c "import email.utils;
print email.utils.parsedate('Mon, 20 Nov 1995 19:12:08 -0500')"
(1995, 11, 20, 19, 12, 8, 0, 1, -1)
(the -1 is the isdst field); the -0500 time zone is completely
ignored, so e.g. the same input with time "19:12:08 +0300" has
the same result. When passed to time.struct_time as allowed per
the parsedate documentation, this time is interpreted in GMT and
thus deviates from the correct value by the timezone offset
(in this example, -5 hours).
I consider this a bug in email.utils.parsedate: In my opinion,
since the return value of the parsetime doesn't include a timezone,
it should be expressed in terms of UTC rather than in terms of the
time zone from the Date header; the existence of
email.utils.parsedate_tz, to which I've switched, indicates that
maybe the authors were aware of this problem.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Previous commit e7ca5b25cb combined
checks for filtered folders in one place. However, it turns out there
was a reason to have them separate. getfolder() on a non-existent Maildir
fails and there might not be an equivalent local Maildir folder for a
filtered out IMAP folder.
Fix this by first checking if the remote folder should be filtered, and
only then retrieving the local folder (which should exist then).
This bug was found by our test suite!
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
the LocalStatus._folders cache was changed to be a dict that can be
searched for names. One instance were _folders was set to "None" was
accidentally left over. Fix this.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
commit e94642bb4d centralized folder filtering by using the
repository.should_sync_folder() function. Therefore there is no need
to check for folderfilter in the Maildir backend separately.
Origina patch by Dave, split into 3 by Sebastian
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Rather than later when retrieving them by IMAP, we determine the
"sync_this" value of a folder on Folder() initialization.
This also fixes a bug where folder structure was not propagated when
a folder was filtered out but included in the folderincludes list.
Patch by Dave, split and modified slightly by Sebastian
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
- Factor out the code to find a local folder given a remote folder
Patch by Dave, split and modified by Sebastian.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
If we ask twice for a LocalStatusFolder via getfolder(), we would
get a newly created instance each time. This can lead to problems,
as e.g. write locks protecting files only work within the same Folder
instance. Make it so, that we cache all Folder instances that we have
asked for and hand back the existing one if we ask again for it,
rather than recreate a new instance.
Also, make getfolders() a noop for LocalStatus. We attempted to
derive the foldername from the name of the LocalStatusfile. However,
this is not really possible, as we do file name mangling
(".$" -> "dot", "/" -> ".") and there is no way to get the original folder
name from the LocalStatus file name anyway.
This commit could potentially solve the "file not found" errors, that people
have been seeing with their LocalStatusCache files. If we have 2
instances of a LocalStatusFolder pointing to the same file, our locking
system would not work.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
When logging to a file using the -l switch, we would still write an initial
banner to the file. This was never intended. Quiet should be really quiet
unless it experiences an error. Simplify the logging statement, to do nothing
if logevel is set to "WARNING" aka quiet.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
While looking at the code to investigate if an why we sometimes don't
seem to honor the write lock, I made it use the more modern "with lock:"
pattern.
Still have not found out how we could ever be using 2 instances of the
LocalStatusFolder for the same folder.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
If we throw an OfflineImapError in case of the Repository()
initialization, we display the nice error message and exit rather
than bomb out with a traceback. Misconfiguring a repository name in
the configuration file is now nicely pointed out to the user.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
When misconfiguring OLI, e.g. by specifying a repository name that was
not configured anywhere, we would bomb out with cryptic "NoSectionError".
Throw OfflineImapError that explains what has happened. We still need to
avoid throwing exceptions with Tracebacks here though.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
