Rather than passing in huge lists of continuous numbers which eventually
overflow the maximum command line length, we coalesce number ranges
before passing the UID sequence to SEARCH. This should do away with the
error that has been reported with busy mailing lists and 'maxage'.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
It is not needed. list(ALIST) will create a new copy of the list just
fine.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Make sure that when a connection is dropped during append, we really
discard the broken connection and get a new one, retrying. We retry
indefinitely on the specific abort() Exception, as this is what imaplib2
suggests us to do.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
If a connection is broken, we want to have it really dropped and not be
reused. So far, we are checking the .Terminate attribute for this, but
according to the imaplib2 author, it is only set on normal shutdown and
it is an undocumented attribute whose meaning could change any time.
This patch introduces the parameter drop_conn which allows to tell
releaseconnection() that we really want to connection being dropped from
the pool of available connections and properly destroy it.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
When invoked with FETCH 1:* (UID), imaplib returns [None] for empty
folders. We need to protect against this case and simply 'continue' here.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
If a connection is dropped for some reason while fetching a message, the
imapobj.uid command throws an imapbj.abort() Exception which means we are
supposed to retry. Implement a fail loop that drops the connection, gets a
new one and attempts the command another time.
Remove obsolete comment that we need to catch nonexisting messages. We do
now.
GMail seems to drop connections left and right. This patch is a response to
the reported mail "4E5F8D8C.1020005@gmail.com" by zeek
<ezekiel.das@gmail.com>.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Handle the case gracefully where a server has closed an IMAP connection
that we want to use for IDLEing. Simply have it dropped and get a new one
in this case. THis should get rid of the errors reported by John Wiegley
in mail id:"m2sjohd16t.fsf@gmail.com".
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Imapserver.acquireconnection will succeed even whent the server connection
has been terminated and the first IMAP operation will throw an exception.
Often this is the folder SELECT operation (e.g. after an idle timeout), as
has been reported by John Wiegley. Catch this case and throw an
OfflineImapError with severity FOLDER_RETRY to notify consumers that they
are supposed to retry.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
In cases where processing a folder failed, but a retry might well succeed
e.g. when the server connection had simply timed out and was disconnected,
we can throw a FOLDER_RETRY (which is less severe than FOLDER).
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Name parameter that hands us a Status Repository 'status_repo' and not
'copyfolders' as was before:
a) make it clear that we pass in a repository and not folder
instances. That was very confusing before.
b) We were always only using one 'copyfolders' item anyway, so let us
not make it a list.
Go through the list and make the variable nameing consistent:
dst_repo rather than dest (is it a folder?) to match the status_repo
naming scheme.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
To preserve previous behavior and save a few CPU cycles, we were not sorting UID
lists and only collapsed them if they were alreay sorted. Vincent pointed out
that this is not always the case and unsorted lists lead to non-optimally
collapsing.
Force lists to numeric types and sort them before collapsing.
Reported-by: Beffara <vbeffara@ens-lyon.fr>
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
We were using self.getfolderbasename(self.name) but the API is simply
getfolderbasename(). Fix this glitch.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
SafeConfigParser is very similar to the currently used ConfigParser but
it supports interpolation. This means values can contain format strings
which refer to other values in the same section, or values in a special
DEFAULT section. For example:
[My Section]
foodir: %(dir)s/whatever
dir=frob
would resolve the %(dir)s to the value of dir (frob in this case). All reference expansions are done on demand.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
I included a wrong change in commit dcfdf2ade7 writen by Sebastian.
As he said:
> I think the first colon should be an equals sign... :-)
Actually no, this was on purpose.
http://docs.python.org/library/configparser.html:
[My Section]
foodir: %(dir)s/whatever
dir=frob
long: this value continues
in the next line
Reported-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
We really wanted nametrans here, also simplify the lambda per
Dan Christensen's suggestion.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Make it contain real use cases and more explanations. We probably need
to add more of the FAQ entries to the MANUAL and point to relevant FAQ
entries from the manual.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Both have been in a stable release and we have never gotten negative
feedback about them. I have been using sqlite exclusively for some time
and people seem actively to use the IDLE folders without major problems.
This patch removes the scary: "THIS IS EXPERIMENTAL" warning, that I
know has kept some people from using it. Do note that the plaintext
backend is still the default even with this patch.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
The text was confusing and not very helpful. Do away with the bad
folderfilters examples and describe a bit more what they are for.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
as that method doesn't exist on sets. Rather call the inbuilt
sorted(flags). This fixes Exceptions being thrown when using the sqlite
backend.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
We were omitting an '%' where we needed it. Also include the traceback
information where it belongs in the new ui.error infrastructure.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
443 is of course the https and not the IMAPS standard port. Fix.
Thanks to Daniel Shahaf <d.s@daniel.shahaf.name> for the heads up.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Commit e023f190b0 changed the storing of
file paths in the messagelist variable to be relative paths, but we were
using the full absolute path anyway as we missed one spot.
Adapt this and construct the full file path in the one place where we
need it.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Rather than output the full list of messages, coalesce it into number
ranges wherever possible.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
This function was badly named and completely undocumented. Rework it to
avoid copying the full UID list using an iterator. Make it possible to
hand it a list of UIDs as strings rather than implicitely relying on the
fact that they are numeric already. Document the code.
The behavior off the function itself remained otherwise unchanged.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
As this is essentially what it is, a set of values. This allows as
to do set arithmetics to see, e.g. the intersection of 2 flag sets
rather than clunkily having to do:
for flag in newflags:
if flag not in oldflags:
oldflags.append(flag)
Also some more code documenting.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Empty foldernames (as they could be created through nametrans) were
failing as the uidvalidity and status files names as determined by
folder/Base.py:getfolderbasename() lead to invalid file names ''.
Fix this by handling empty file names and translating them to '.' which
leads to the special file name 'dot'. (this special value existed before
and was not invented by this patch)
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
I changed the API to pass in a folder rather than a list of folders, but
used getnicename() on the wrong object. It is not used on the folder but
on the ui object. Fix this and give the variable somewhat better names.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
It works by fetching all headers of new messages from IMAP server and
searching for our X-OfflineIMAP marker by using regular expression.
Signed-off-by: Vladimir Marek <vlmarek@volny.cz>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
We only copy to a single folder anyway, so clean up the code to only
pass in a single folder.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
imaplib2 always attempts to verify a certificate if a verification
callback function is passed in, even the certificate is None
specified. Disable the verification excplictly by setting the
verification function to None in that case.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
we need errno.CONNREFUSED, but through some merging mishaps(?) the part
that actually imported errno was missing. Import the errno module.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
imaplib renamed self.sslobj to self.sock and our overriden open()
functions were failing for that reason when updating imaplib2 to
v2.28. It turns out that all of our custom initializations are being
done by stock imaplib2 now anyway, so there is no need to override them
anymore. This lets us simplify the code we have to worry about.
Move the verifycert() function to the imapserver.py file, it is now a
callback function that is being handed to imaplib from there, so it
makes sense to also define it in our imapserver function...
(this also lets us easily make use of the verifycert function in the
starttls case in the future)
TODO: we need to examine if and why we still need to override the
select() function, it is the only reason why we still wrap the IMAP4
classes.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Imaplib2 2.28 can deal with ID sequences, such as 1:*, so we need to
bump upstream in order to make use of these features.
Note that this revision will not run correctly as it requires
adaptations to our code, which happens in the next commit.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
We were not including the full server reply into our error message. Fix
that so we get better error logs.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
This is a bug fix on several levels. 1) We were lacking the import of
OfflineImapError. 2) OfflineImap.ERROR.MESSAGE was misspelled as ERROR.Message.
3) COntinuing with the next message only worked in single-thread mode
(using debug) and not in multi-thread mode. The reason is that we were
invoking a new thread and catching Exceptions in the main thread. But
python immediately aborts if an Exception bubbles up to a thread start.
This was fixed by catching exceptions directly in copymessageto() which
is the new thread, rather than catching them in the main thread.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Allowing to specify the char to use in the BLinkenlights is a bit over
the top and bloats our default offlineimap.conf. The dot is just fine,
so let us settle for it and cut the example config file by an unneeded
section.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
append() raises an Exception, in case the IMAP server replies with 'BAD'
(but not when it responds with 'NO') but we were not catching that. Do
catch the situation and also raise an OfflineImapError at MESSAGE
severity, so that we can continue with the next message.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
