This reverts commit 71a76d9a61.
Zak Smith reported a problem at:
self.processmessagesflags('+', uidlist, flags)
File "/usr/lib/python2.5/site-packages/offlineimap/folder/IMAP.py",
line 372, in processmessagesflags
myrights = imapobj.myrights(self.getfullname())[1][0].split()[1]
IndexError: list index out of range
Conflicts:
offlineimap/folder/IMAP.py
Added WrappedIMAP4_SSL class to help fix up performance of SSL
Standard imaplib.py is really bad with this, since it reads one
character at a time.
Reported by Aaron Kaplan at
http://lists.complete.org/offlineimap@complete.org/2008/01/msg00012.html.gz
He wrote:
I just noticed that the version of offlineimap I've been using
(3.99.17) is well over four years old. How time flies. I haven't
had any problems with it, but out of curiosity I decided to pull in
5.99.2 from the fedora repository. It turns out to take
consistently over twice as long as the old version to sync the same
account. Is this expected?
He tracked it down at
http://lists.complete.org/offlineimap@complete.org/2008/02/msg00012.html.gz
The following changeset is the one responsible for the difference in
speed I was noticing between the imaplib.py that was packaged with
older versions of offlineimap and the one that comes with python:
* /offlineimap/head: changeset 169
More optimizations -- this time fix readline() to not work
character-by-character!
closes#22
from pistore in OfflineIMAP #22:
When an IMAP flag update is performed for multiple messages, some IMAP
servers (e.g. Exchange) return the UID attribute only for some of the
FETCH untagged responses, as shown in the following log:
21:19.04 > DCKF8 UID STORE 66050,50613,52164,40043,40055,25874 +FLAGS
(\Deleted)
21:19.36 < * 35 FETCH (FLAGS (\Seen \Deleted) UID 25874)
21:19.36 < * 321 FETCH (FLAGS (\Seen \Deleted))
21:19.57 < * 322 FETCH (FLAGS (\Seen \Deleted))
21:19.57 < * 560 FETCH (FLAGS (\Seen \Deleted))
21:19.57 < * 581 FETCH (FLAGS (\Seen \Deleted) UID 52164)
21:19.62 < * 1022 FETCH (FLAGS (\Seen \Deleted))
21:19.62 < DCKF8 OK STORE completed.
Function IMAPFolder.processmessagesflags is able to manage the servers
which return the UID and the servers which do not return it, but is
not able to deal with the mixed behavior shown above.
The problem is that the fragment of function
IMAPFolder.processmessagesflags that handles the responses with UID
attribute uses variable flags to store the list of flags of the
message in the IMAP format ("flags = attributehashFLAGS?"), while the
fragment that handles the responses without UID expects variable
"flags" to contain the list of modified flags passed to the function
in Maildir format ("self.messagelist[uid]flags?.append(flag)").
As a consequence, the wrong list of flags is used for the messages
without UID, leading to the addition of "strange" flags to the Maildir
messages:
Syncing messages IMAP[INBOX] -> Maildir[.]
Adding flags to 4 messages on Maildir[.]
Adding flags e to 4 messages on Maildir[.]
Adding flags d to 4 messages on Maildir[.]
Adding flags ) to 4 messages on Maildir[.]
Adding flags ( to 4 messages on Maildir[.]
Adding flags l to 4 messages on Maildir[.]
Adding flags n to 4 messages on Maildir[.]
Adding flags t to 4 messages on Maildir[.]
Adding flags \ to 4 messages on Maildir[.]
Adding flags D to 4 messages on Maildir[.]
Deleting flags T to 4 messages on Maildir[.]
Adding flags to 4 messages on LocalStatus[.]
Adding flags e to 4 messages on LocalStatus[.]
Adding flags d to 4 messages on LocalStatus[.]
Adding flags ) to 4 messages on LocalStatus[.]
Adding flags ( to 4 messages on LocalStatus[.]
Adding flags l to 4 messages on LocalStatus[.]
Adding flags n to 4 messages on LocalStatus[.]
Adding flags t to 4 messages on LocalStatus[.]
Adding flags \ to 4 messages on LocalStatus[.]
Adding flags D to 4 messages on LocalStatus[.]
Deleting flags T to 4 messages on LocalStatus[.]
Fix: use a different variable to store IMAP flags when managing
messages corresponding to responses with UID attribute, e.g.:
*** IMAP.py.orig Wed Aug 22 18:23:17 2007
--- IMAP.py Wed Aug 22 18:22:38 2007
*************** class IMAPFolder(BaseFolder):
*** 340,348 ****
if not ('UID' in attributehash and 'FLAGS' in
attributehash):
# Compensate for servers that don't return a UID
attribute.
continue
! flags = attributehash['FLAGS']
uid = long(attributehash['UID'])
! self.messagelist[uid]['flags'] =
imaputil.flagsimap2maildir(flags)
try:
needupdate.remove(uid)
except ValueError: # Let it slide if it's not
in the list
--- 340,348 ----
if not ('UID' in attributehash and 'FLAGS' in
attributehash):
# Compensate for servers that don't return a UID
attribute.
continue
! lflags = attributehash['FLAGS']
uid = long(attributehash['UID'])
! self.messagelist[uid]['flags'] =
imaputil.flagsimap2maildir(lflags)
try:
needupdate.remove(uid)
except ValueError: # Let it slide if it's not
in the list
02/03/08 14:04:35 changed by js
* attachment flags-fix.patch added.
Delete 02/03/08 14:05:24 changed by js
Unfortunately I have to fetch some of my mail from an Exchange server
(Microsoft Exchange Server 2003 IMAP4rev1 server version 6.5.7638.1)
and I can confirm that the analysis of the problem is correct, and the
patch given here fixes the problem.
Looking at the code of the processmessagesflags() method I think it
generally is a bug that the "flags" parameter is reused as a local
variable, since the final "for uid in needupdate:" loop needs the
original value of "flags". This only worked by accident.
I'm attaching a unidiff version of the patch which applies cleanly
against Debian unstable's offlineimap 5.99.4.
New repository/folder classes to support "real deletion" of messages
thorugh Gmail's IMAP interface: to really delete a message in Gmail,
one has to move it to the Trash folder, rather than EXPUNGE it.
This patch maneuvers around python imaplib's mysterious read-only detection
algorithm and correctly calls the UI's deletetoreadonly(), when trying to
delete/expunge in a mailbox without having the necessary rights.
This involves several changes at different places:
- syncfoldersto() takes statusfolder as an argument, and returns the
list of new folders and the list of folders that should be ingnored,
typically those that were deleted. Warns the user about folders that
are present only on one side and are not synced.
- syncfoldersto() is called both ways, and on folder creation
forgetfolders() is used to rebuild the list and take the new creation
into account. Probably not the most efficient, since it involves
talking to the IMAP server again, but it will rarely be used anyway.
- Locally created folders are treated separately in the synchronization,
namely the local messages are uploaded and then the normal sync still
occurs. If the same folder is created on both sides and contains
messages on both sides, a two-way sync occurs.
fixes deb#439384
From: martin f krafft
Subject: race condition in Maildir writing
The offlineimap Maildir code checks for file existence and then
opens a file. That's open to a race condition. It's better to open
the file and fail if it already exists. The following patch does
this. It catches OSError 17 (file exists) and re-raises all others.
I'll leave it up to you to decide whether this is appropriate.
This involves several changes at different places:
- syncfoldersto() takes statusfolder as an argument, and returns the
list of new folders and the list of folders that should be ingnored,
typically those that were deleted. Warns the user about folders that
are present only on one side and are not synced.
- syncfoldersto() is called both ways, and on folder creation
forgetfolders() is used to rebuild the list and take the new creation
into account. Probably not the most efficient, since it involves
talking to the IMAP server again, but it will rarely be used anyway.
- Locally created folders are treated separately in the synchronization,
namely the local messages are uploaded and then the normal sync still
occurs. If the same folder is created on both sides and contains
messages on both sides, a two-way sync occurs.
From:
Ben Kibbey
> > Attached is a patch to enable evaluation of account credentials with the
> > remotehosteval, remoteusereval and remotepasseval configuration options.
> > I needed this because rather than change all my other programs
> > configuration settings when I change, say a password, I store them in a
> > file. So I call a function in pythonfile which parses the credential
> > file and returns the wanted info. Not really very well tested, but not
> > complex either. Offlineimap is great, thanks.
[ this is the doc for that patch which was already applied ]
From: "Mark A. Hershberger"
https://bugs.launchpad.net/ubuntu/+source/offlineimap/+bug/96710
the locked() method isn't implemented for non-interactive UIs, so
exceptions are thrown on cron jobs. Ubuntu's new apport catches these
and ? well, you get the idea.
patch provided.
