If when we request a LocalStatus folder, the folder has to be created,
we look whether the other backend has data, and if it does we migrate
it to the new backend.
The old backend data is left untouched, so that if you change back say
from sqlite to plaintext, the older data is still there. That should
not lead to data loss, only a slower sync while the status folder gets
updated.
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
When synclabels config flag is set to "yes" for the GMail repo,
offlineimap fetches the message labels along with the messages, and
embeds them into the body under the header X-Keywords (or whatever
'labelsheader' was set to), as a comma separated list.
It also adds an extra pass to savemessageto, that performs label
synchronization on existing messages from GMail to local, the same way
it is done with flags.
We also introduce GmailMaildir repository that adds functionality to
change message labels. It keeps track of messages modification time,
so one can quickly detect when the labels may have changed.
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
When filterheaders is set to a comma-separated list of headers,
OfflineIMAP removes those headers from messages before uploading them
to the server.
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
Added the configuration setting usecompression for the IMAP repositories.
When enabled, the data from and to the IMAP server is compressed.
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
Allow people who want folder filtering to depend on the external
conditions or to make it dynamic for other reasons, to do what
they want.
New repository configuration knob 'dynamic_folderfilter' was
introduced; it defaults to 'False' that matches historical behaviour.
GitHub: #73
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
When imaputil was parsing quoted strings, it treated "abcd\\"
as incomplete quoted string having escaped quote, rather than
properly-quoted string having escaped backslash.
GitHub issue: https://github.com/OfflineIMAP/offlineimap/issues/53
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
2.36 it includes support for SSL version override that was integrated
into our code before, no other changes.
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
- Fix for Gmail sending a BYE response after reading >100 messages
in a session.
- Includes fix for GitHub#15: patch was accepted upstream.
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
self._expecting_data was used both as the expected data length and the
flag that we expect some data. This obviously fails when advertized
data length is zero, so self._expecting_data_len was introduced to
hold the length of the expected data and self._expecting_data was left
as the flag that we expect the data to come.
GitHub issue: https://github.com/OfflineIMAP/offlineimap/issues/15
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
It is called localfolders and holds expanded name for the same
variable for the local repository of the account that is being
processed.
GitHub issue: https://github.com/OfflineIMAP/offlineimap/issues/21
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
Bring the description in the template offlineimap.conf in sync to the
actual implementation: pass folder names to the sorting function, not
the offlineimap.folder.IMAP.IMAPFolder objects themselves.
GitHub issue: https://github.com/OfflineIMAP/offlineimap/issues/27
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
Global or per-repository option utime_from_message tells OfflineIMAP
to set file modification time of messages pushed from one repository
to another basing on the message's "Date" header.
This is useful if you are doing some processing/finding on your
Maildir (for example, finding messages older than 3 months),
without parsing each file/message content.
From: Cyril RUSSO <boite.pour.spam@gmail.com>
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
Added configuration option "auth_mechanisms" to the config file:
it is a list of mechanisms that will be tried in the specified order.
Author: Andreas Mack <andreas.mack@konsec.com>
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
It's nice to set up an ssh tunnel command which forwards an IMAP tcp
port inside an encrypted session, e.g. with ssh's "-W" flag. In this
case the tunnelled connection still requires authentication inside
IMAP session, because this is transport-only tunnel that substitutes
normal TCP/SSL connection.
New directive, 'transporttunnel' was added: it specifies the command
that will create the tunnel. Only one type of tunnel must be
specified for a single repository: we can't have both preauthenticated
and transport-type tunnels, they won't chain together.
From: Steve Purcell <steve@sanityinc.com>
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
- this method isn't as deprecated as IMAP LOGIN;
- it allows to keep hashed passwords on the server side;
- it has the ability to specify that the remote identity
is different from authenticating username, so it even
can be useful in some cases (e.g., migrated mailboxes);
configuration variable "remote_identity" was introduced
to leverage this functionality.
From: Andreas Mack <andreas.mack@konsec.com>
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
IMAP servers can return `NO` responses to the `APPEND` command,
e.g. here's an example response from Groupwise's IMAP server:
NO APPEND The 1500 MB storage limit has been exceeded.
In this case, savemessage() should abort the repository sync rather
than returning UID 0 which would cause the local copy of the message
being saved to get irreversibly deleted.
Signed-off-by: Adam Spiers <offlineimap@adamspiers.org>
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
mutt-sidebar and, probably, other MUA show mailboxes in the order
they are listed in the file written by mbnames. Therefore, to allow
customization of the order with which mailboxes are listed, introduce
the new 'sort_keyfunc' directive in the [mbnames] section.
'sort_keyfunc' must be a function that will be called once for each
mailbox. It must accept the only argument -- a dict with 2 items,
'accountname' and 'foldername', and should return an object that
will be used as the sorting key for each mailbox.
Default key function returns (d['accountname'], d['foldername']),
thus sorting by account name and then by the folder name.
Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
Make IDLE syncs be equal to the regular synchronisations
in respect to pre-sync and post-sync hooks.
From: mxgr7 <maxgerer@gmail.com>
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
They are redundant in all pruned cases and sometimes even create some
problems, e.g., when one tries to jump through paragraphs in vi.
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
The problem lies in the fact that offlineimap.folder.Base's method
syncmessagesto_copy() uses threaded code everytime it is suggested by
the derived class's suggeststhreads() (currently, only IMAP does this
suggestion), but offlineimap/init.py will not spawn the
exitnotifymonitorloop() from offlineimap.threadutil.
The root cause is that ExitNotifyThread-derived threads need
offlineimap.threadutil's exitnotifymonitorloop() to be running the
cleaner for the exitthreads Queue(), because it fills the queue via
the run() method from this class: it wants to put() itself to the
Queue on exit, so when no exitnotifymonitorloop() is running, the
queue will fill up. And if this thread is an instance of
InstanceLimitedThread that hits the limit on the number of threads,
then it will hold the instancelimitedsems[] semaphore will prevent
other InstanceLimitedThread()s of the same name to pass its start()
method.
The fix is to avoid using threaded code if we're running
single-threaded.
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
Obtained-from: X-Ryl669 <boite.pour.spam@gmail.com>
This is handy when we're debugging the thread locks: we can try to
understand which thread does what and how it was called.
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
offlineimap has several frontends that encourage running it from a
terminal under an X session. When X session closes for a system
shutdown, the terminals exit, after sending SIGHUP to their children.
Previously SIGHUP was treated to be equivalent to SIGUSR1, i.e. wake
up and sync all accounts. This causes delays during shutdown.
According to Wikipedia [0], SIGHUP has been repurposed from a
historical meaning to one of:
* re-read configuration files, or reinitialize (e.g. Apache, sendmail)
* controlling pseudo or virtual terminal has been closed
I believe second meaning is more appropriate for offlineimap, and
hence this patch makes SIGHUP to be handled in the same way SIGTERM
and SIGINT are handled.
[0] http://en.wikipedia.org/wiki/SIGHUP
Debian-Bug: http://bugs.debian.org/670120
Reported-By: Steve Langasek <steve.langasek@canonical.com>
Signed-off-by: Dmitrijs Ledkovs <xnox@debian.org>
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
Fix imapfolder.getmessageinternaldate misparsing the Date:
header from emails due to a bug or surprising behaviour by
email.utils.parsedate. This is because email.utils.parsedate's
return value contains the unadjusted hour value from the string
parsed but does not include information about the time zone in
which it is specified. For example (Python 2.7.3):
$ python -c "import email.utils;
print email.utils.parsedate('Mon, 20 Nov 1995 19:12:08 -0500')"
(1995, 11, 20, 19, 12, 8, 0, 1, -1)
(the -1 is the isdst field); the -0500 time zone is completely
ignored, so e.g. the same input with time "19:12:08 +0300" has
the same result. When passed to time.struct_time as allowed per
the parsedate documentation, this time is interpreted in GMT and
thus deviates from the correct value by the timezone offset
(in this example, -5 hours).
I consider this a bug in email.utils.parsedate: In my opinion,
since the return value of the parsetime doesn't include a timezone,
it should be expressed in terms of UTC rather than in terms of the
time zone from the Date header; the existence of
email.utils.parsedate_tz, to which I've switched, indicates that
maybe the authors were aware of this problem.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
When misconfiguring OLI, e.g. by specifying a repository name that was
not configured anywhere, we would bomb out with cryptic "NoSectionError".
Throw OfflineImapError that explains what has happened. We still need to
avoid throwing exceptions with Tracebacks here though.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
If 'readonly' is True, folders shouldn't be created (regardless of
'createfolders' option). With old behavior, instead folders were always created
when 'readonly' is True (even if 'createfolders' was also False), which is a
serious bug (offlineimap was creating folders in all read-only repositories).
'createfolders' should only play a role if 'readonly' is False, in which case
folders should only be created if 'createfolders' is True.
Submitted-by: Vladimir Nesov <robotact@gmail.com>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
This will allow our callers who are capable of dealing with
readonly folders to properly detect this condition and act
accordingly.
One example is Gmail's "Chats" folder that is read-only,
but contains logs of the quick chats.
Minor Changelog improvements.
Tested-by: Abdó Roig-Maranges <abdo.roig@gmail.com>
Signed-off-by: Eygene Ryabinkin <rea@codelabs.ru>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
As Gmail was only announcing the presence of the UIDPLUS extension
after we logged in, and we were then only getting server
capabilities before, a hack was introduced that checked the
existence of an APPENDUID reply, even if the server did not claim
to support it.
However, John Wiegley reports problems, where the APPENDUID would
be None, and we attempt to go this path (it seems that imaplib2
returns [None] if there is no such reply, so our test here for "!="
might fail. Given that this is an undocumented imaplib2 function
anyway, and we do fetch gmail capabilities after authentication,
this hack should no longer be necessary.
We had problems there earlier, where imapobj.response() would
return [None] although we had received a APPENDUID response from
the server, this might need more debugging and testing.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Somehow we failed if no dry-run setting had been specified in the config
file. This got caught thanks to extending the test suite with a stock
configuration.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
By default OfflineImap propagates new folders in both
directions. Sometimes this is not what you want. E.g. you might want
new folders on your IMAP server to propagate to your local MailDir,
but not the other way around. The 'readonly' setting on a repository
will not help here, as it prevents any change from occuring on that
repository. This is what the `createfolders` setting is for. By
default it is `True`, meaning that new folders can be created on this
repository. To prevent folders from ever being created on a
repository, set this to `False`. If you set this to False on the
REMOTE repository, you will not have to create the `Reverse
nametrans`_ rules on the LOCAL repository.
Also implement a test for this
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Reported by sharat87 in https://github.com/spaetz/offlineimap/pull/38,
he would often get an unhandled Exception when trying to
releaseconnection() a connection that was not in the pool of
connections.
The reason this could happen is that when folder.IMAP.quickchanged()
raises an Exception in select(), we would release the connection in the
"except" handling, and than release the same connection in the "finally"
clause, which led to the error. The right thing is to only release the
connection once, of course.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
We use python's SafeConfigParser, and this is a built in "feature" used
for interpolating variables. But it imples that '%' needs encoding as
'%%'.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
As reported by James Cook, we would not check the fingerprint of the SSL
server, as we were looking for the 'ssl' module in locals() rather than
globals(). Ooops!
Rather than using globals() though, I simply remove the by-now
superfluous check. We now rely on python2.6 and we unconditionally
import the SSL module in any case, so it needs to be there.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
The warn() method tries to set the color to red. This leads to a garbled
tty after endwin() has been called. So lets simply use the UIBase
implementation.
Signed-off-by: Christoph Höger <christoph.hoeger@tu-berlin.de>
Somehow we failed if no dry-run setting had been specified in the config
file. This got caught thanks to extending the test suite with a stock
configuration.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
OfflineIMAP v6.5.2-rc1 (2012-01-09)
===================================
Commits v6.5.1.1 - v6.5.2-rc1:
note: Proper Changelog still in Changelog-draft.rst
d72bb88 Improve error message
3284e01 Revert "use .response() rather _get_untagged_response()"
81f194a mbnames should write out local and not nametransformed box names
7184ec2 Sanity check return value of UIDVALIDTY response
50de217 Allow to pass 'force' arg to selectro() to enforce a new select
ed71805 Changelog entry about "realdelete" option
0a275b9 Add scary warnings about "realdelete" option
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Blinkenlights UI 6.5.0 regression fixes only.
* Sleep led to crash ('abort_signal' not existing)
* Make exit via 'q' key work again cleanly
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
