Commit Graph

2730 Commits

Author SHA1 Message Date
Sebastian Spaeth
105012957f Merge branch 'next' 2011-09-29 11:21:38 +02:00
Sebastian Spaeth
c1120c9158 6.4.0 release
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-29 11:20:57 +02:00
Sebastian Spaeth
4bfc1e8227 except imapobj.abort() -> except imapobj.abort
When checking for the IMAP4.abort() exception, we need of course to
perform:

except imapobj.abort:

and not

except imapobj.abort():

Thanks to Johannes Stezenbach <js@sig21.net> for pointing to the glitch.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-27 14:08:20 +02:00
Sebastian Spaeth
ae8a1cb79f Remove deprecated calls to apply()
apply() has been deprecated since Python 2.3, and won't be working in
python 3 anymore. Use the functional equivalent throughout.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-27 13:00:12 +02:00
Sebastian Spaeth
e145beb394 Fix "command CHECK illegal in state AUTH"
Dave identified a case where our new dropped connection handling did
not work out correctly: we use the retry_left variable to signify
success (0=success if no exception occured).

However, we were decrementing the variable AFTER all the exception
checks, so if there was one due to a dropped connection, it
could well be that we 1) did not raise an exception (because we want to
retry), and 2) then DECREMENTED retry_left, which indicated "all is
well, no need to retry".

The code then continued to check() the append, which failed with the
above message (because we obtained a new connection which had not even
selected the current folder and we were still in mode AUTH). The fix is
of course, to fix our logic: Decrement retry_left first, THEN decide
whether to raise() (retry_left==0) or retry (retry_left>0) which would
then correctly attempt another loop. I am sorry for this newbie type of
logic error. The retry count loop was too hastily slipped in, it seems.

Reported-by: Dave Abrahams <dave@boostpro.com>
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-26 16:13:52 +02:00
Sebastian Spaeth
953c58a9c9 Robustify (&fix) error throwing on APPEND
If APPEND raises abort(), the (typ, dat) variables will not be set, so
we should not be using it for the  OfflineImapError Exception
string. Fixing and prettifying the string formatting a bit at the same
time.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-26 15:57:35 +02:00
Sebastian Spaeth
1b6d76345a Robustify error msg against more failure
When syncfolder() fails, we output an error message containing the
foldername per the localfolder variable. However, the localfolder
variable is assigned inside our try: block and when the error occurs
there, we will have no localfolder variable to use for output. This
caused the errormsg to cause an Exception itself which unhelpfully
distracts from the root cause of the error.

Reconstruct the folder name in a bit more complex way, but in a way so
it is guaranteed to work (by relying on parameters passed in to the
function).

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-26 15:32:24 +02:00
Sebastian Spaeth
248f23afd6 Do not fail calling dequote() with empty string
A report by Dave Abrahams showed that the dequote() function failed when
invoked with an empty string. This fixes the function to be more robust.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-26 15:16:59 +02:00
Thomas Kahle
3d00a8bb4a MANUAL: Improve documentation on using python code in the config
Hi, the following patch updates the documentation.  It was a bit out of
context since Sebastian originally just copied it from my webpage.
Also, the code for the helper functions is available there under
"Download".  I did not include it because it would leave larger chunks
of python code in the manual which seems weird.

Signed-off-by: Thomas Kahle <tomka@gentoo.org>
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-23 15:01:31 +02:00
Sebastian Spaeth
e30ae53b2a Folder: Force top-level folder visiblename to ''
nametrans rules can lead to different visiblename names for the
top-level directory, specifically both '.' and '' (the latter was
recently introduced). However, we need to be able to compare folder
names to see if we need to create a new directory or whether a directory
already exists, so we need to be able to compare a repositories
visiblename (=transposed via nametrans rule) with another folder.

To make the top-level directory comparison happen, we enforce a
top-level name of '', so that comparisons work.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-23 14:31:49 +02:00
Sebastian Spaeth
275dbfa3fb Release 6.3.5-rc3
Update Changelog and __VERSION__ for 6.3.5-rc3.
Also add in the notes for rc2 to the Changelog.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-21 02:32:50 +02:00
Sebastian Spaeth
347e1eaa32 update CAPABILITIES after login
Some Webservers (I am looking at you Gmail) send different capabilities
before and after login, so they can tailor their server capabilities to
the user. While legal, this is uncommon and we were not updating our
server capabilities. Doing so allows us to detect that Gmail actually
supports the UIDPLUS extension, and we will stop mangling headers when
uploading to Gmail. This could lead to some performance gains when we
upload many messages to Gmail.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-21 02:32:45 +02:00
Sebastian Spaeth
db28cb77e7 Fix another visiblename() glitch in determining the foldername to be created
Commit b0e88622c4 changed dst_hash[folder.visiblename] to
dst_hash[folder.name] but we did not adapt all places where it is needed
to use visiblename again. This led to attempting to create a name on
REMOTE ignoring the nametrans setting on the LOCAL repository.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-20 22:25:45 +02:00
Sebastian Spaeth
1c58ebe348 Do not create folder on REMOTE if it would fall in REMOTE's folderfilter
Previously, we only checked if a LOCAL folder falls under the local
repositories folderfilter rule when deciding whether a folder should be
created on REMOTE.

However, we also do not want to create the folder on REMOTE if it would
fall under a folderfilter rule there. This patch prevents us from doing
so.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-19 21:25:50 +02:00
Sebastian Spaeth
b0e88622c4 Fix think on how to compare folder names on src and dest
It is not easy to think through when to use visiblenames() and whatnot. It seems I managed to not think it through properly. Which might be fixed now.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-19 21:06:54 +02:00
Sebastian Spaeth
ee82cd135b Fix typo atter->latter
Simply typo in user error message.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-19 20:07:56 +02:00
Sebastian Spaeth
48fdb14418 Don't output thread ID in log
The thread ID is not really useful and looks ugly. It also makes lines
longer than needed, there is more useful information we can put in the
log. So do away with it.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-19 19:59:39 +02:00
Sebastian Spaeth
050bbb3d2d Reset draft Changelog post-release
Empty it, so new stuff can pour in.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-19 15:21:27 +02:00
Sebastian Spaeth
eeef8f4bab Don't ask for hostname if using a tunnel
In tunnel mode, we don't need to ask for a hostname because there is no
need for one. Fix this.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-19 15:07:34 +02:00
Sebastian Spaeth
b92e453f0e sanity check so that nametrans does not lead to infinite folder creation 2011-09-19 14:23:08 +02:00
Sebastian Spaeth
fd6261a50f Create new folders on srcrepo if needed
This will ignore any nametrans rules, so we might want to limit this
only to cases where no nametrans has been specified, or we might want to
use the nametrans setting of the dest repo.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-19 14:23:08 +02:00
Sebastian Spaeth
6b2ec956cf Apply nametrans to all Foldertypes
getvisiblename() was only defined on IMAP(derived) foldertypes, but we
want it on eg. Maildirs too, so we define it centrally in Folder.Base.py
rather than only in folder.IMAP.py.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-19 14:23:08 +02:00
Sebastian Spaeth
7d34060217 Enable folderfiltering for MailDir repositories too
Currently we only filtered IMAP repositories, this patch enables filtering
for Maildir repositories too.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-19 14:23:08 +02:00
Sebastian Spaeth
9c5f73d3b3 Move self.folderfilter|nametrans|sort from IMAPFolder to BaseFolder
We want to have these functions available for Maildir folders too, so we
can folderfilter a Maildir repository too (which is currently not possible)

This commit only move the corresponding functions from the IMAP to the Base
implementation. It should not change behavior in any way yet.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-19 14:23:08 +02:00
Sebastian Spaeth
3157a8d793 repository.Gmail.getfolder()
Also remove the removed parameters in the Gmail folder
initialization. This is one spot where I had forgotten to also strip the
parameters.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-19 14:22:03 +02:00
Sebastian Spaeth
a279aa7307 Maildir: Call top-level directory '', not '.'
If nametrans translates to an empty directory we want to find the
top-level directory by name '' and not by name '.'. This unbreaks
nametrans rules that result in empty folder names.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-19 14:14:44 +02:00
Sebastian Spaeth
792243c78f Improve repository/Maildir.getfolder() to use cached objects
Previously, getfolder() would always construct new MaildirFolder()
objects, independent of whether the folder exists or not. Improve the
function to:

1) Scan and cache the folders if not already done
2) Return the same cached object if we ask for the same foldername twice
3) Reduce a tiny bit of code duplication

This is important because we handle stuff like folderfilter in the
scandir function and if we discard the scanned dir and create a new
object on folderget(), we will lose the folderfilter information.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-19 13:24:42 +02:00
Sebastian Spaeth
c7420e6ad4 Debug log/error proof the creation of new IMAP folders
Output a debug log line whenever we create a new folder on an IMAP
server. Also raise an OfflineImap Error in case we failed to create it.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-19 11:36:03 +02:00
Sebastian Spaeth
8ba17c5bd1 add sync_this variable to all Folder() instances
This variable shows if this folder should be synced or is disabled due to
a folderfilter statement. This lets us distinguish between a non-existent
folder and one that has been filtered out. Previously any filtered folder
would simply appear to be non-existing.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-19 11:35:52 +02:00
Sebastian Spaeth
32ca20d0da Folder.Maildir: No need to store 'uid' in messagelist dict.
The Message UID is already the key to self.messagelist, so we have that
information. It is redundant to save the UID again as
self.messagelist[uid]{'uid': uid} and we never made use of the
information anyway.

The same thing should be done with the other 2 backends.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-19 10:01:31 +02:00
Sebastian Spaeth
dfdc4d457b Folder.Maildir: Simplify getmessagetime
No need to use os.stat()['st_mtime'] when there is os.path.getmtime()

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-19 09:59:04 +02:00
Sebastian Spaeth
a43677a3e6 Shorten self.infosep assignment
A more pythonic and less verbose way to do the same. Add a comment what the
variable is all about.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-19 09:58:20 +02:00
Sebastian Spaeth
1ac9dc7fb4 Implement RFC 2595 LOGINDISABLED
Warn the user and abort when we attempt a plaintext login, but the
server has explicitly disabled plaintext logins.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-19 09:55:27 +02:00
Sebastian Spaeth
19ff636390 Properly output errors when the main thread receives some
Use the ui.error infrastructure that has been put in place and use
ui.terminate even if we received an Exception, so that we can output the
list of errors that we have. This does away with 2 now unused functions
in ui/UIBase.py

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-19 09:34:25 +02:00
Sebastian Spaeth
0d95651417 Perform legacy global lock in addition to new per-account lock
We simply lock OfflineImap the same global way that we have always done
in addition to the previously implemented per-account lock. We can keep
both systems in parallel and then after a few stable releases, drop the
old-style global lock. by reverting this patch

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-19 09:34:25 +02:00
Sebastian Spaeth
89c705bb26 init.py: Import OfflineImapError
The next commit will make use of OfflineImapError but is transient (the
old-style lock). The commit is supposed to be reverted after a few
releases. So add the new import in a separate commit, because we might
need this even when reverting the commit.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-19 09:34:25 +02:00
Sebastian Spaeth
c7938dc081 Per-account locking
Previously, we were simply locking offlineimap whenever it was
running. Howver there is no reason why we shouldn't be able to invoke it
in parallel, e.g. to synchronize several accounts in one offlineimap
each.

This patch implements the locking per-account, so that it is possible to
sync different accounts at the same time. If in refresh mode, we will
attempt to loop three times before giving up.

This also fixes Debian bug #586655

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-19 09:34:25 +02:00
Sebastian Spaeth
42c9e04966 Merge branch 'pending/FixGetUsername' into next 2011-09-19 09:30:05 +02:00
Sebastian Spaeth
f5366343b9 Fix default Maildir File permissions.
open() and os.open() lead to different file permissions by default, and
while we have not changed the os.open that had been used, some code
changes led to these permissions slipping through. Fix this by setting
the permissions explicitly to 0666 (minus the users umask).

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-19 09:25:48 +02:00
Sebastian Spaeth
82e47896cf Don't ask for username in the preauthtunnel case
repos.getuesr() asks for a username if none is specified, but in the
case of a tunnel connection, we don't need one, so we need to skip the
repos.getuser() call here.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-19 08:31:14 +02:00
Sebastian Spaeth
0d3303ec12 Remove visiblename as parameter to IMAPFolder creation
IMAPFolder has the repository and foldername values so it can get the
transposed (aka visiblename) of a folder itself just fine. There is no
need to pass it in as an separate parameter.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-16 12:54:29 +02:00
Sebastian Spaeth
80e87d0d99 Don't pass in 'root' as para to LocalStatusFolders
They have the Repository() which contains the root, so no need to pass
it in as an extra parameter. Rename repository.LocalStatus()'s
self.directory to self.root for consistency with other backends.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-16 12:54:12 +02:00
Sebastian Spaeth
ee75e0921f Remove 'config' as parameter from BaseFolder & derivatives
It is possible to get the config parameter from the Repository() which is
set in BaseFolder, so we set self.config there and remove the various
methods and 'config' parameters that are superfluous.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-16 12:54:12 +02:00
Sebastian Spaeth
410e2d35e9 Set accountname in BaseFolder, and don't pass it in initialization
We passed in the accountname to all derivatives of BaseFolder, such as
IMAPFolder(...,repository,...,accountname), although it is perfectly
possible to get the accountname from the Repository(). So remove this
unneeded parameter. Each backend had to define getaccountname() (although
the function is hardly used and most accessed .accountname directly).

On the other hand BaseFolder was using getaccountname but it never defined
the function. So make the sane thing, remove all definitions from backends
and define accountname() once in Basefolder. It was made a property and not
just a (public) attribute, so it will show up in our developer
documentation as public API.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-16 12:54:12 +02:00
Sebastian Spaeth
c93cd9bb1a BaseFolder(): Save name and repository
As all Folders share these parameters, we can safely handle them in
BaseFolder. This makes sense, as BaseFolder has a getname() function
that returns self.name but nothing actually set self.name.

It also saves a few lines of code.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-16 12:54:12 +02:00
Sebastian Spaeth
7941ea7e7d folder.IMAP: Make use of the new connection discarding
In getmessage() we were releaseing a connection when we detected a
dropped connection, but it turns out that this was not enough, we need
to explicitely discard it when we detect a dropped one. So add the
drop_conn=True parameter that was recently introduced to force the
discarding of the dead conection.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-16 12:53:00 +02:00
Sebastian Spaeth
fe4f385e2c folder.IMAP: Improve dropped connection handling in quickchanged()
The quickchanged() function was not handling dropped connections yet. If
IMAP4.select() throws a FOLDER_RETRY error, we will now discard the
connection, reconnect and retry.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-16 12:53:00 +02:00
Sebastian Spaeth
5bcfbc1525 MaildirRepository: Beautify restore_atime code
Beauty of code is probably a subjective measure, but this patch hopefully
is an improvement over the previous incarnation without changing
functionality.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-15 19:28:52 +02:00
Sebastian Spaeth
ee1706fa90 documentation: Clarify restoreatime setting
Better document what this actually does and that most people won't be
needing it.

(Especially as mount setting such as relatime|noatime now reduce the
amount of atime changes anyway)

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-15 19:28:48 +02:00
Sebastian Spaeth
c1a2b1559d repository: Simplify restore_atime code
repository.BaseRepository().restore_atime() was testing in complex ways
that it only operates on a Maildir and that the 'restoreatime' setting
is set. This is unecessary, we can simply make the base implementation a
NoOp, and move the implementation to MaildirRepository().

This will save a tad of work for everyone doing IMAP<->IMAP
synchronization and simplify the code. Also document the functions.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-15 19:28:42 +02:00