1) Differentiate error messages between imaplib.abort and imaplib.error
exceptions in the log.
2) Drop connections in the case of imapobj.error, it also might denote a
broken connection.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
During cleanup we often call releaseconnection in a finally: block. But
in cases of error, we might have dropped the connection earlier already
and set it to "None". In this case don't fail releaseconnection() but
make it a NOOP.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
The default parameter value was "None", and we were comparing that
directly to the imaplib2 value of is_readonly which is False or True, so
the comparison always returned "False".
Fix this by setting the default parameter to "False" and not
"None". Also convert all users of that function to use False/True.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Conflicts:
Changelog.draft.rst
If port is None, we would try to format an empty string with %d wich
fails. Fix it by using %s.
Reported-by: Iain Dalton <iain.dalton@gmail.com>
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
It happens quick, and clutters the log. So we can usually skip this. We
will output a log entry when we actually create a new folder anyway.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
If we create a new folder we would previously not update our folder
list, which led to us skipping the synchronization of those new folders
during the initial run (subsequent runs would pick it up).
Invalidate the folder cache when we create a folder during folder
structure sync. Regetting the whole list from an IMAP server might be
slightly suboptimal from a performance point, but it is easy and will
lead to consistent results. Hopefully we will not have to create new
folders on each new run.
Reported-by: Daniel Shahaf <d.s@daniel.shahaf.name>
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Modify the UI:acct and acctdone functions to keep tab of the time
inbetween. Put self.ui.acct() and acctdone() at the right places in
accounts.py so that the timing happens at the right places.
While modifying that loop, flatten the nested try: try: except: finally:
constructs, we require python 2.5 now which copes with that.
At the end of each account sync you will now see something like:
*** Finished account 'test' in 0:05
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Thread names are used to determine the logging header in the TTY ui. A
recent change made them too terse (basically only changing the account
name and not the folder names). Unbreak.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Output (2 of 500) when logging message copying. This required moving of
self.ui.copyingmessage into a different function where we actually have
the information about the progress handy.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Rather than setting a global threadutil/profiledir variable, we make
set_profiledir a class function that sets the class variable profiledir.
While touching theprofiledir code, add warning to the user if the
profile directory existed before.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Registering a thread (associating it with a certain account name) would
fail if it was already registered. However, as we a) never unregister most
threads (bad) and b) single-threaded mode reuses threads, we failed when
syncing multiple accounts in single-threading mode.
This commit cleans up the functions to not make re-registering a thread
fatal (it could be legitimate, however it *should* not occur). Future
work needs to be done to unregister new threads at the appropriate places.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
When checking for the IMAP4.abort() exception, we need of course to
perform:
except imapobj.abort:
and not
except imapobj.abort():
Thanks to Johannes Stezenbach <js@sig21.net> for pointing to the glitch.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
apply() has been deprecated since Python 2.3, and won't be working in
python 3 anymore. Use the functional equivalent throughout.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Dave identified a case where our new dropped connection handling did
not work out correctly: we use the retry_left variable to signify
success (0=success if no exception occured).
However, we were decrementing the variable AFTER all the exception
checks, so if there was one due to a dropped connection, it
could well be that we 1) did not raise an exception (because we want to
retry), and 2) then DECREMENTED retry_left, which indicated "all is
well, no need to retry".
The code then continued to check() the append, which failed with the
above message (because we obtained a new connection which had not even
selected the current folder and we were still in mode AUTH). The fix is
of course, to fix our logic: Decrement retry_left first, THEN decide
whether to raise() (retry_left==0) or retry (retry_left>0) which would
then correctly attempt another loop. I am sorry for this newbie type of
logic error. The retry count loop was too hastily slipped in, it seems.
Reported-by: Dave Abrahams <dave@boostpro.com>
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
If APPEND raises abort(), the (typ, dat) variables will not be set, so
we should not be using it for the OfflineImapError Exception
string. Fixing and prettifying the string formatting a bit at the same
time.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
When syncfolder() fails, we output an error message containing the
foldername per the localfolder variable. However, the localfolder
variable is assigned inside our try: block and when the error occurs
there, we will have no localfolder variable to use for output. This
caused the errormsg to cause an Exception itself which unhelpfully
distracts from the root cause of the error.
Reconstruct the folder name in a bit more complex way, but in a way so
it is guaranteed to work (by relying on parameters passed in to the
function).
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
A report by Dave Abrahams showed that the dequote() function failed when
invoked with an empty string. This fixes the function to be more robust.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
nametrans rules can lead to different visiblename names for the
top-level directory, specifically both '.' and '' (the latter was
recently introduced). However, we need to be able to compare folder
names to see if we need to create a new directory or whether a directory
already exists, so we need to be able to compare a repositories
visiblename (=transposed via nametrans rule) with another folder.
To make the top-level directory comparison happen, we enforce a
top-level name of '', so that comparisons work.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Some Webservers (I am looking at you Gmail) send different capabilities
before and after login, so they can tailor their server capabilities to
the user. While legal, this is uncommon and we were not updating our
server capabilities. Doing so allows us to detect that Gmail actually
supports the UIDPLUS extension, and we will stop mangling headers when
uploading to Gmail. This could lead to some performance gains when we
upload many messages to Gmail.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Commit b0e88622c4 changed dst_hash[folder.visiblename] to
dst_hash[folder.name] but we did not adapt all places where it is needed
to use visiblename again. This led to attempting to create a name on
REMOTE ignoring the nametrans setting on the LOCAL repository.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Previously, we only checked if a LOCAL folder falls under the local
repositories folderfilter rule when deciding whether a folder should be
created on REMOTE.
However, we also do not want to create the folder on REMOTE if it would
fall under a folderfilter rule there. This patch prevents us from doing
so.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
It is not easy to think through when to use visiblenames() and whatnot. It seems I managed to not think it through properly. Which might be fixed now.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
The thread ID is not really useful and looks ugly. It also makes lines
longer than needed, there is more useful information we can put in the
log. So do away with it.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
This will ignore any nametrans rules, so we might want to limit this
only to cases where no nametrans has been specified, or we might want to
use the nametrans setting of the dest repo.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
getvisiblename() was only defined on IMAP(derived) foldertypes, but we
want it on eg. Maildirs too, so we define it centrally in Folder.Base.py
rather than only in folder.IMAP.py.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Currently we only filtered IMAP repositories, this patch enables filtering
for Maildir repositories too.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
We want to have these functions available for Maildir folders too, so we
can folderfilter a Maildir repository too (which is currently not possible)
This commit only move the corresponding functions from the IMAP to the Base
implementation. It should not change behavior in any way yet.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Also remove the removed parameters in the Gmail folder
initialization. This is one spot where I had forgotten to also strip the
parameters.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
If nametrans translates to an empty directory we want to find the
top-level directory by name '' and not by name '.'. This unbreaks
nametrans rules that result in empty folder names.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Previously, getfolder() would always construct new MaildirFolder()
objects, independent of whether the folder exists or not. Improve the
function to:
1) Scan and cache the folders if not already done
2) Return the same cached object if we ask for the same foldername twice
3) Reduce a tiny bit of code duplication
This is important because we handle stuff like folderfilter in the
scandir function and if we discard the scanned dir and create a new
object on folderget(), we will lose the folderfilter information.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Output a debug log line whenever we create a new folder on an IMAP
server. Also raise an OfflineImap Error in case we failed to create it.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
This variable shows if this folder should be synced or is disabled due to
a folderfilter statement. This lets us distinguish between a non-existent
folder and one that has been filtered out. Previously any filtered folder
would simply appear to be non-existing.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
The Message UID is already the key to self.messagelist, so we have that
information. It is redundant to save the UID again as
self.messagelist[uid]{'uid': uid} and we never made use of the
information anyway.
The same thing should be done with the other 2 backends.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
A more pythonic and less verbose way to do the same. Add a comment what the
variable is all about.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Warn the user and abort when we attempt a plaintext login, but the
server has explicitly disabled plaintext logins.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Use the ui.error infrastructure that has been put in place and use
ui.terminate even if we received an Exception, so that we can output the
list of errors that we have. This does away with 2 now unused functions
in ui/UIBase.py
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
We simply lock OfflineImap the same global way that we have always done
in addition to the previously implemented per-account lock. We can keep
both systems in parallel and then after a few stable releases, drop the
old-style global lock. by reverting this patch
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
The next commit will make use of OfflineImapError but is transient (the
old-style lock). The commit is supposed to be reverted after a few
releases. So add the new import in a separate commit, because we might
need this even when reverting the commit.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Previously, we were simply locking offlineimap whenever it was
running. Howver there is no reason why we shouldn't be able to invoke it
in parallel, e.g. to synchronize several accounts in one offlineimap
each.
This patch implements the locking per-account, so that it is possible to
sync different accounts at the same time. If in refresh mode, we will
attempt to loop three times before giving up.
This also fixes Debian bug #586655
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
open() and os.open() lead to different file permissions by default, and
while we have not changed the os.open that had been used, some code
changes led to these permissions slipping through. Fix this by setting
the permissions explicitly to 0666 (minus the users umask).
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
