Commit Graph

858 Commits

Author SHA1 Message Date
Sebastian Spaeth
82e47896cf Don't ask for username in the preauthtunnel case
repos.getuesr() asks for a username if none is specified, but in the
case of a tunnel connection, we don't need one, so we need to skip the
repos.getuser() call here.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-09-19 08:31:14 +02:00
Sebastian Spaeth
0d3303ec12 Remove visiblename as parameter to IMAPFolder creation
IMAPFolder has the repository and foldername values so it can get the
transposed (aka visiblename) of a folder itself just fine. There is no
need to pass it in as an separate parameter.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-16 12:54:29 +02:00
Sebastian Spaeth
80e87d0d99 Don't pass in 'root' as para to LocalStatusFolders
They have the Repository() which contains the root, so no need to pass
it in as an extra parameter. Rename repository.LocalStatus()'s
self.directory to self.root for consistency with other backends.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-16 12:54:12 +02:00
Sebastian Spaeth
ee75e0921f Remove 'config' as parameter from BaseFolder & derivatives
It is possible to get the config parameter from the Repository() which is
set in BaseFolder, so we set self.config there and remove the various
methods and 'config' parameters that are superfluous.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-16 12:54:12 +02:00
Sebastian Spaeth
410e2d35e9 Set accountname in BaseFolder, and don't pass it in initialization
We passed in the accountname to all derivatives of BaseFolder, such as
IMAPFolder(...,repository,...,accountname), although it is perfectly
possible to get the accountname from the Repository(). So remove this
unneeded parameter. Each backend had to define getaccountname() (although
the function is hardly used and most accessed .accountname directly).

On the other hand BaseFolder was using getaccountname but it never defined
the function. So make the sane thing, remove all definitions from backends
and define accountname() once in Basefolder. It was made a property and not
just a (public) attribute, so it will show up in our developer
documentation as public API.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-16 12:54:12 +02:00
Sebastian Spaeth
c93cd9bb1a BaseFolder(): Save name and repository
As all Folders share these parameters, we can safely handle them in
BaseFolder. This makes sense, as BaseFolder has a getname() function
that returns self.name but nothing actually set self.name.

It also saves a few lines of code.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-16 12:54:12 +02:00
Sebastian Spaeth
7941ea7e7d folder.IMAP: Make use of the new connection discarding
In getmessage() we were releaseing a connection when we detected a
dropped connection, but it turns out that this was not enough, we need
to explicitely discard it when we detect a dropped one. So add the
drop_conn=True parameter that was recently introduced to force the
discarding of the dead conection.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-16 12:53:00 +02:00
Sebastian Spaeth
fe4f385e2c folder.IMAP: Improve dropped connection handling in quickchanged()
The quickchanged() function was not handling dropped connections yet. If
IMAP4.select() throws a FOLDER_RETRY error, we will now discard the
connection, reconnect and retry.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-16 12:53:00 +02:00
Sebastian Spaeth
5bcfbc1525 MaildirRepository: Beautify restore_atime code
Beauty of code is probably a subjective measure, but this patch hopefully
is an improvement over the previous incarnation without changing
functionality.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-15 19:28:52 +02:00
Sebastian Spaeth
c1a2b1559d repository: Simplify restore_atime code
repository.BaseRepository().restore_atime() was testing in complex ways
that it only operates on a Maildir and that the 'restoreatime' setting
is set. This is unecessary, we can simply make the base implementation a
NoOp, and move the implementation to MaildirRepository().

This will save a tad of work for everyone doing IMAP<->IMAP
synchronization and simplify the code. Also document the functions.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-15 19:28:42 +02:00
Sebastian Spaeth
d2af21d57d Simplify testing for 'subscribedonly' setting
We only explicitly tested for 'yes' when we have a nice function to get
boolean settings which also works with Treu/False/NO, etc...

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-15 19:27:01 +02:00
Sebastian Spaeth
fe57140224 Fix repository 'readonly' configuration
The readonly feature was introduced to safeguard repositories from
accidental modifications. Unfortunately, my patch treated the readonly
setting as a string and not as a boolean, so if *anything* was set in
the configuration file as 'readonly', this value evaluated to True

Fortunately this was safe, we never treated a repository that we wanted
read-only as read-write. We always treated them readonly if something
was configured as "readonly=..." even if that was False.

The fix is simply to use getconfboolean() rather than getconf() which
checks for True/False/On/Off/yes/no/1/0 and hands back the correct boolean.

Reported-by: Tomasz Nowak <nowak2000@poczta.onet.pl>
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-15 19:20:27 +02:00
Sebastian Spaeth
7c83d505f8 IMAP cachefolder: Fix returning None on select
We rely on the number of mails being returned by the imapobj.select()
call, however that only happens if we "force" a real select() to occur.
Pass in the force parameter that I dropped earlier (we did not make use
of the return value when I dropped it, that is how it slipped through).

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-13 18:31:28 +02:00
Sebastian Spaeth
24db42916c IMAP savemessage(): Don't loop indefinitely on failure
We were retrying indefinitely on imapobj.abort() (as that is what
imaplib2 suggests), but if the failure occurs repeatedly, we'll never
quit this loop. So implement a counter that errs out after unsuccessful
retries.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-13 18:29:22 +02:00
Nicolas Sebrecht
1a4b7c337c v6.3.5-rc1
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-12 20:10:44 +02:00
Nicolas Sebrecht
78a67ac73c Merge branch 'ss/cleanup-idle' into next 2011-09-12 20:02:26 +02:00
Sebastian Spaeth
ce8471a011 IMAP IDLE cleanup(5): Really discard connections when they are dropped
Finally, actually discard dropped connections when we detect them as an
imapobj.abort() has been thrown. In this case, invoke releaseconnection
with drop_conn=True.

We don't need the self.aborted attribute to get signified of dropped
connections. An Execption during the noop will do.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-12 19:55:14 +02:00
Sebastian Spaeth
59753fc06f IMAP IDLE cleanup(4): Simplify code
while True: if a: return

is equivalent to

   while not a:

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-12 19:55:14 +02:00
Sebastian Spaeth
6ad0de08ef IMAP IDLE cleanup(3): Rename self.event to self.stop_sig
Variable name 'event' is as bad as it gets. Rename it to something that
actually describes what it is about.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-12 19:55:14 +02:00
Sebastian Spaeth
0bebd65ba0 IMAP IDLE cleanup(2): Add code documentation
Add code documentation throughout the idle() function.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-12 19:55:14 +02:00
Sebastian Spaeth
f369961a87 IMAP IDLE cleanup(1): Move idle callback out of loop
Don't redefine the idle callback function on every run in the while
loop, define it once when we enter the function.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-12 19:55:13 +02:00
Sebastian Spaeth
bc10e05600 Check for SSL module existence and only do fingerprint check then
Python 2.5 has no ssl module, and we can therefor not get the server
certificate for fingerprint verification. Add a check that disables
fingerprint verification for python 2.5.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-12 19:45:16 +02:00
Sebastian Spaeth
eafea0c880 IMAP4.sslobj -> sock
imaplib2 has changed internally to use self.sock for its ssl socket when
it used to be sslobj. Reflect that change.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-12 19:45:16 +02:00
Sebastian Spaeth
8800fa37a3 Implement Server SSL fingerprint check
If we connect to a SSL server (not STARTTLS) and no CA cert has been
specified for verification, we check the configured SSL fingerprint and
bail out in case it has not been set yet, or it does not match.

This means one more mandatory option for SSL configuration, but it
improves security a lot.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-12 19:45:15 +02:00
Sebastian Spaeth
5cbec30b3e Sanity check for maxage setting
If maxage is set too large, we would even SEARCH for negative
years. With devastating results. So implement some sanity check and err
out in case the year does not make sense.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-07 19:32:43 +02:00
Sebastian Spaeth
135f8c45cf Simplify constructing the SEARCH date
We can use Imaplib's monthnames and shorten the construction of the date
by using them rather than hardcoding them again.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-07 19:32:08 +02:00
Sebastian Spaeth
1b99c019e5 Fix handling the search results
Results are delivered in a 1-element list, and somehow I managed to drop
a [0] in the previous patches. We need to look at the element of course,
or our string splitting will fail horribly. Sorry this somehow slipped
through.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-07 19:31:37 +02:00
Sebastian Spaeth
26721c60d4 Don't cache empty IMAP folders
If a folder is empty, most servers will return EXISTS 0 and imaplib2
passes back ['0'] as return value to a select(). It returns [None] if
no EXISTS response was given by the server at all.

Attempting to fetch the UIDs of 0 emails which leads to
various error messages (One server responds with "NO No matching
messages", Gmail seems to say "BAD Bad message sequence 1:*" for some
(although it is working fine for me with Gmail, so it might behave
different for different people).

In case we get an None or 0 back, we simply stop caching messages as the
folder is empty. This should fix the various error reports that have
popped up.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-07 19:13:13 +02:00
Nicolas Sebrecht
67863017e2 Merge branch 'ss/better-error-throwing-and-id-sequence' into next
Conflicts:
	offlineimap/folder/IMAP.py
	offlineimap/imaputil.py

Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-06 20:15:05 +02:00
Sebastian Spaeth
4c558c1b69 Error proof IMAP.APPEND against dropped connections
Make sure that when a connection is dropped during append, we really
discard the broken connection and get a new one, retrying. We retry
indefinitely on the specific abort() Exception, as this is what imaplib2
suggests us to do.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-06 20:03:33 +02:00
Sebastian Spaeth
971ed3adac Allow Imapserver.releaseconnection() to drop a connection
If a connection is broken, we want to have it really dropped and not be
reused. So far, we are checking the .Terminate attribute for this, but
according to the imaplib2 author, it is only set on normal shutdown and
it is an undocumented attribute whose meaning could change any time.

This patch introduces the parameter drop_conn which allows to tell
releaseconnection() that we really want to connection being dropped from
the pool of available connections and properly destroy it.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-06 20:03:33 +02:00
Sebastian Spaeth
1917be8e83 Shorten list of messages to be deleted in UI output
Rather than output the full list of messages, coalesce it into number
ranges wherever possible.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-06 20:03:33 +02:00
Sebastian Spaeth
fb8017991c Rework undocumented listjoin to create UID sequences
This function was badly named and completely undocumented. Rework it to
avoid copying the full UID list using an iterator. Make it possible to
hand it a list of UIDs as strings rather than implicitely relying on the
fact that they are numeric already. Document the code.

The behavior off the function itself remained otherwise unchanged.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-06 20:03:33 +02:00
Sebastian Spaeth
3302940382 Proper error handling for SEARCH and FETCH failures from the server
SEARCH and FETCH were never checking that the IMAP server actually
returned OK. Throw OfflineImapErrors at severity FOLDER in case one of
them fails.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-06 20:03:33 +02:00
Sebastian Spaeth
f755c8b423 Use range 1:* if we want to examine all messages in a folder
Some code cleanup. If we want to examine all messages of a folder, don't
try to find out how many there are and request a long list of all of them,
but simply request 1:*. This obliviates us from the need to force a select
even if we already had the folder selected and it requires us to send a
few less bytes over the wire.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-06 20:03:33 +02:00
Sebastian Spaeth
8532a69458 Clean up the maxsize maxage code
Do away with the wrapping of this code in a try...except KeyError, as
this code cannot conceivably throw a KeyError. Even if it could, it
should be documented why we should simply return() in this case.

Shorten some of the variable names and minor code cleanup while taking
the git blame anyway.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-06 20:03:33 +02:00
Sebastian Spaeth
06bfff7c04 Coalesce SEARCH uid list into sequence set
Rather than passing in huge lists of continuous numbers which eventually
overflow the maximum command line length, we coalesce number ranges
before passing the UID sequence to SEARCH. This should do away with the
error that has been reported with busy mailing lists and 'maxage'.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-06 20:03:33 +02:00
Nicolas Sebrecht
a21bb949af Merge branch 'ss/improve-collapsing' into next 2011-09-06 19:59:28 +02:00
Nicolas Sebrecht
8c730cf509 Merge branch 'ss/folder.imap-dont-import-copy' into next
Conflicts:
	offlineimap/folder/IMAP.py

Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-06 19:57:02 +02:00
Sebastian Spaeth
63b9dcd896 folder.IMAP: Don't import copy
It is not needed. list(ALIST) will create a new copy of the list just
fine.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-06 19:53:44 +02:00
Sebastian Spaeth
2cf6155282 Error proof IMAP.APPEND against dropped connections
Make sure that when a connection is dropped during append, we really
discard the broken connection and get a new one, retrying. We retry
indefinitely on the specific abort() Exception, as this is what imaplib2
suggests us to do.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-06 19:51:49 +02:00
Sebastian Spaeth
9c678c9d6b Allow Imapserver.releaseconnection() to drop a connection
If a connection is broken, we want to have it really dropped and not be
reused. So far, we are checking the .Terminate attribute for this, but
according to the imaplib2 author, it is only set on normal shutdown and
it is an undocumented attribute whose meaning could change any time.

This patch introduces the parameter drop_conn which allows to tell
releaseconnection() that we really want to connection being dropped from
the pool of available connections and properly destroy it.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-06 19:51:49 +02:00
Sebastian Spaeth
0906d0db70 IMAP.cachemessagelist(): Protect against empty folders
When invoked with FETCH 1:* (UID), imaplib returns [None] for empty
folders. We need to protect against this case and simply 'continue' here.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-06 19:07:54 +02:00
Nicolas Sebrecht
ff9b941f42 Merge branch 'ss/protect-select-for-idle' into next 2011-09-06 19:01:11 +02:00
Sebastian Spaeth
dc103ab9ce Protect IMAP.getmessage() against dropped connections
If a connection is dropped for some reason while fetching a message, the
imapobj.uid command throws an imapbj.abort() Exception which means we are
supposed to retry. Implement a fail loop that drops the connection, gets a
new one and attempts the command another time.

Remove obsolete comment that we need to catch nonexisting messages. We do
now.

GMail seems to drop connections left and right. This patch is a response to
the reported mail "4E5F8D8C.1020005@gmail.com" by zeek
<ezekiel.das@gmail.com>.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-06 19:00:48 +02:00
Sebastian Spaeth
f1555ec893 Catch terminated connections on the IDLE SELECT operation
Handle the case gracefully where a server has closed an IMAP connection
that we want to use for IDLEing. Simply have it dropped and get a new one
in this case. THis should get rid of the errors reported by John Wiegley
in mail id:"m2sjohd16t.fsf@gmail.com".

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-06 19:00:21 +02:00
Sebastian Spaeth
61fffd5c5f imaplibutil: Have SELECT throw an OfflineImapError when the connection died
Imapserver.acquireconnection will succeed even whent the server connection
has been terminated and the first IMAP operation will throw an exception.

Often this is the folder SELECT operation (e.g. after an idle timeout), as
has been reported by John Wiegley. Catch this case and throw an
OfflineImapError with severity FOLDER_RETRY to notify consumers that they
are supposed to retry.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-06 19:00:21 +02:00
Sebastian Spaeth
7bc45507cb Introduce new error level FOLDER_RETRY
In cases where processing a folder failed, but a retry might well succeed
e.g. when the server connection had simply timed out and was disconnected,
we can throw a FOLDER_RETRY (which is less severe than FOLDER).

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-06 19:00:21 +02:00
Sebastian Spaeth
fe1391084b Improve repo.Base.py:syncfoldersto parameter/var names
Name parameter that hands us a Status Repository 'status_repo' and not
'copyfolders' as was before:

a) make it clear that we pass in a repository and not folder
instances. That was very confusing before.
b) We were always only using one 'copyfolders' item anyway, so let us
not make it a list.

Go through the list and make the variable nameing consistent:
dst_repo rather than dest (is it a folder?) to match the status_repo
naming scheme.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-03 17:16:42 +02:00
Sebastian Spaeth
54d3307499 imaputil: make uid_sequence sort all items to improve collapsing
To preserve previous behavior and save a few CPU cycles, we were not sorting UID
lists and only collapsed them if they were alreay sorted.  Vincent pointed out
that this is not always the case and unsorted lists lead to non-optimally
collapsing.

Force lists to numeric types and sort them before collapsing.

Reported-by: Beffara <vbeffara@ens-lyon.fr>
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-02 22:02:51 +02:00
Sebastian Spaeth
f0869e5c0f Fix sqlite upgrade code to use correct API call
We were using self.getfolderbasename(self.name) but the API is simply
getfolderbasename(). Fix this glitch.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-09-02 21:59:35 +02:00
Sebastian Spaeth
8cce178fb8 Improve developer documentation
Well, this patch... improves it. :)

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-30 22:19:16 +02:00
Sebastian Spaeth
4db5913492 Use SafeConfigParser for the configuration
SafeConfigParser is very similar to the currently used ConfigParser but
it supports interpolation. This means values can contain format strings
which refer to other values in the same section, or values in a special
DEFAULT section. For example:

[My Section]
foodir: %(dir)s/whatever
dir=frob

would resolve the %(dir)s to the value of dir (frob in this case). All reference expansions are done on demand.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-30 22:16:35 +02:00
Sebastian Spaeth
dcc50efc4f Don't call set().sort()
as that method doesn't exist on sets. Rather call the inbuilt
sorted(flags). This fixes Exceptions being thrown when using the sqlite
backend.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-23 20:59:48 +02:00
Sebastian Spaeth
a6480d4959 Fix string formatting
We were omitting an '%' where we needed it. Also include the traceback
information where it belongs in the new ui.error infrastructure.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-23 20:58:55 +02:00
Sebastian Spaeth
2b9b6be6be Fix standard port for SSL/TLS
443 is of course the https and not the IMAPS standard port. Fix.

Thanks to Daniel Shahaf <d.s@daniel.shahaf.name> for the heads up.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-23 20:57:38 +02:00
Vladimir Marek
9ecac29aaa Maildir relative paths change was not complete
Commit e023f190b0 changed the storing of
file paths in the messagelist variable to be relative paths, but we were
using the full absolute path anyway as we missed one spot.

Adapt this and construct the full file path in the one place where we
need it.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-23 20:56:24 +02:00
Sebastian Spaeth
6295d64d54 Shorten list of messages to be deleted in UI output
Rather than output the full list of messages, coalesce it into number
ranges wherever possible.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-23 20:55:28 +02:00
Sebastian Spaeth
0a25408199 Rework undocumented listjoin to create UID sequences
This function was badly named and completely undocumented. Rework it to
avoid copying the full UID list using an iterator. Make it possible to
hand it a list of UIDs as strings rather than implicitely relying on the
fact that they are numeric already. Document the code.

The behavior off the function itself remained otherwise unchanged.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-23 20:55:28 +02:00
Vladimir Marek
8de326b29f Catch correct type of exception
Signed-off-by: Vladimir Marek <vlmarek@volny.cz>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-19 19:05:16 +02:00
Vladimir Marek
df62bb61a5 Create exception when file rename fails
Signed-off-by: Vladimir Marek <vlmarek@volny.cz>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-19 19:02:47 +02:00
Sebastian Spaeth
466ded04d9 Make flags a set rather than a list
As this is essentially what it is, a set of values. This allows as
to do set arithmetics to see, e.g. the intersection of 2 flag sets
rather than clunkily having to do:

for flag in newflags:
  if flag not in oldflags:
    oldflags.append(flag)

Also some more code documenting.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-19 18:56:10 +02:00
Sebastian Spaeth
373e7cdbc1 Allow empty foldernames
Empty foldernames (as they could be created through nametrans) were
failing as the uidvalidity and status files names as determined by
folder/Base.py:getfolderbasename() lead to invalid file names ''.

Fix this by handling empty file names and translating them to '.' which
leads to the special file name 'dot'. (this special value existed before
and was not invented by this patch)

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-19 18:37:00 +02:00
Sebastian Spaeth
4f1cd05fdc Fix copyingmessage() in Machine UI
I changed the API to pass in a folder rather than a list of folders, but
used getnicename() on the wrong object. It is not used on the folder but
on the ui object. Fix this and give the variable somewhat better names.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-19 18:25:04 +02:00
Vladimir Marek
c2fc81dd3d Make syncfoldersto to accept a single folder only
It is just historic relict

Signed-off-by: Vladimir Marek <vlmarek@volny.cz>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-17 21:02:21 +02:00
Vladimir Marek
b6ac1aecb1 Another way of locating UID of just saved message
It works by fetching all headers of new messages from IMAP server and
searching for our X-OfflineIMAP marker by using regular expression.

Signed-off-by: Vladimir Marek <vlmarek@volny.cz>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-17 20:58:41 +02:00
Sebastian Spaeth
6f9b171ffd Don't pass a list to ui.copyingmessage()
We only copy to a single folder anyway, so clean up the code to only
pass in a single folder.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-17 20:41:32 +02:00
Vladimir Marek
38b1d7b085 Replaced tabs with spaces to unify python sources
Signed-off-by: Vladimir Marek <vlmarek@volny.cz>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-16 22:33:19 +02:00
Sebastian Spaeth
8d858a0b78 Disable certificate verification if there is no certificate
imaplib2 always attempts to verify a certificate if a verification
callback function is passed in, even the certificate is None
specified. Disable the verification excplictly by setting the
verification function to None in that case.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-16 22:22:47 +02:00
Nicolas Sebrecht
82bf23ee57 Merge branch 'ss/import-errno' into next
Conflicts:
	offlineimap/imapserver.py
2011-08-15 14:46:52 +02:00
Sebastian Spaeth
4a0d17b566 Import errno
we need errno.CONNREFUSED, but through some merging mishaps(?) the part
that actually imported errno was missing. Import the errno module.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-15 13:58:44 +02:00
Nicolas Sebrecht
2749374a93 Merge branch 'ss/imaplib2-2.28' into next
Conflicts:
	Changelog.draft.rst
2011-08-15 13:57:37 +02:00
Sebastian Spaeth
3a91e296f0 Adapt the code to work with the new imaplib2
imaplib renamed self.sslobj to self.sock and our overriden open()
functions were failing for that reason when updating imaplib2 to
v2.28. It turns out that all of our custom initializations are being
done by stock imaplib2 now anyway, so there is no need to override them
anymore. This lets us simplify the code we have to worry about.

Move the verifycert() function to the imapserver.py file, it is now a
callback function that is being handed to imaplib from there, so it
makes sense to also define it in our imapserver function...
(this also lets us easily make use of the verifycert function in the
starttls case in the future)

TODO: we need to examine if and why we still need to override the
select() function, it is the only reason why we still wrap the IMAP4
classes.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-15 12:21:38 +02:00
Sebastian Spaeth
1c0c19ad86 imaplib2: bump to upstream version 2.28 (rev 8228a0f)
Imaplib2 2.28 can deal with ID sequences, such as 1:*, so we need to
bump upstream in order to make use of these features.

Note that this revision will not run correctly as it requires
adaptations to our code, which happens in the next commit.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-15 12:21:38 +02:00
Sebastian Spaeth
fe388400c4 Better error message when FETCH fails.
We were not including the full server reply into our error message. Fix
that so we get better error logs.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-15 12:16:29 +02:00
Sebastian Spaeth
9f23fea74e Fix error handling in folder.Base.copymessageto()
This is a bug fix on several levels. 1) We were lacking the import of
OfflineImapError. 2) OfflineImap.ERROR.MESSAGE was misspelled as ERROR.Message.
3) COntinuing with the next message only worked in single-thread mode
(using debug) and not in multi-thread mode. The reason is that we were
invoking a new thread and catching Exceptions in the main thread. But
python immediately aborts if an Exception bubbles up to a thread start.
This was fixed by catching exceptions directly in copymessageto() which
is the new thread, rather than catching them in the main thread.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-15 12:16:28 +02:00
Sebastian Spaeth
634b6cd49e Don't make Blinkenlight statuschar configurable
Allowing to specify the char to use in the BLinkenlights is a bit over
the top and bloats our default offlineimap.conf. The dot is just fine,
so let us settle for it and cut the example config file by an unneeded
section.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-15 12:10:20 +02:00
Sebastian Spaeth
194aa1db3c Catch 'BAD' replies on append()ing a message
append() raises an Exception, in case the IMAP server replies with 'BAD'
(but not when it responds with 'NO') but we were not catching that. Do
catch the situation and also raise an OfflineImapError at MESSAGE
severity, so that we can continue with the next message.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-15 12:05:08 +02:00
Nicolas Sebrecht
b896e6f13b Merge branch 'ss/remove-processmessagesflags-function' into next 2011-08-14 17:23:49 +02:00
Nicolas Sebrecht
21b10bbf55 Merge branch 'ss/remove-mustquote' into next 2011-08-14 17:23:44 +02:00
Sebastian Spaeth
51629b8ffe Update example in code documentation of ui.error()
sys.exc_traceback is long deprecated and is seems removed in python2.7,
so document the legitimate use of sys.exc_info()[2] instead.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-14 17:18:51 +02:00
Sebastian Spaeth
3c89a72be9 remove unused 'mustquote' regex
We set an imapobj.mustquote which apparently was used in previous
incarnations of imaplib or imaplib2, however, nothing in our codebase
makes use of that. So let us remove it.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-14 17:15:38 +02:00
Sebastian Spaeth
306f584c86 Remove custom Gmail/folder/processmessagesflags()
This function was overridden as the IMAP version apparently had been
using imapobj.myrights() at some point in time, which was not
implemented in the Gmail version. However, IMAP is not using myrights()
anymore, and as that is an extension that needs to be advertised in
CAPABILITIES we should not unconditionally use it anyway.

So remove the function that is identical to it's ancestor's
function.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-14 17:14:35 +02:00
Sebastian Spaeth
97dbd18e12 LocalStatus.py: Fix getfolders()
This code was unused and broken. It is still unused but this commit
fixes it. (We should retain the method in case we ever start calling
getfolders() on LocalStatus.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-12 18:31:31 +02:00
Nicolas Sebrecht
a38c6da49d Merge branch 'ss/improved-folderfilter-errors' into next
Conflicts:
	Changelog.draft.rst
2011-08-12 18:29:23 +02:00
Sebastian Spaeth
fa18968642 Catch folderfilter errors in repository.IMAP.getfolders()
Rather than throwing ValueError, we now properly throw OfflineImapError
when selecting a folder in folderincludes. So we also need to catch
OfflineImapErrors here. If they are of severity FOLDER, just ignore the
invalid folder and continue. If the error is more severe, bubble it up.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-12 18:28:49 +02:00
Sebastian Spaeth
131298c2b1 Remove superfluous class ConfigedIMAPServer
Remove a level of wrapper abstraction that is not needed. Just use
IMAPserver and be done with it.

We do this by passing in the IMAPRepository() instance rather than a
long list of single paramters to the IMAPServer instanciation. This way
we can retrieve all repository parameters ourselves, rather than passing
a dozen paramters into IMAPServer. Also, this enables us to pass the
repository() object into our WrappedIMAP4() instance, so that it can
query, e.g. the SSL fingerprint configuration.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-12 18:18:26 +02:00
Sebastian Spaeth
f6b9c68333 IMAP: Don't use assert() in folder.savemessage()
We simply assert()ed that APPENDing a message returned OK, but in some
cases (e.g. Google chat messages) APPEND might return BAD or NO too. We
should be throwing an OfflineImapError here at MESSAGE level, so that we
can continue to sync all other messages, and still give the user some
details on what went wrong at the end of the sync run.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-11 19:14:28 +02:00
Sebastian Spaeth
b47bc44783 accounts.py: Use ui.error when raising exceptions
Rather than using ui.warn, use ui.error() which outputs Exceptions to
the error log, saving them to a stack, so we get notified again at the
end of the sync run.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-11 19:14:28 +02:00
Sebastian Spaeth
f937a86571 Implement ui.error() and output them at end of offlinimap sync
Output all raised Exceptions error strings to the error log. If we are
in debug mode, we also output the traceback of the exception.

Save all exceptions that occur during the run time to a Queue and output
them all again when the offlineimap sync is finished.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-11 19:14:28 +02:00
Nicolas Sebrecht
61e50b3b1a v6.3.4
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-08-10 21:44:58 +02:00
Vladimir Marek
e58399ac0b Fixed typo
Signed-off-by: Vladimir Marek <vlmarek@volny.cz>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-07-27 18:54:53 +02:00
Vladimir Marek
d5cbdc4c0e Handle when UID can't be found on saved messages
Message was stored to dstfolder, but we can't find it's UID. This means we can't
link current message to the one created in IMAP. So we just delete local message
and on next run we'll sync it back. Also fixed imap.savemessage description.

This was broken by e20d8b9679.

Signed-off-by: Vladimir Marek <vlmarek@volny.cz>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-07-27 18:54:53 +02:00
Vladimir Marek
86e9c7442b Include message header at better place
It's not enough to place header after first newline, since this might break
multiline rfc0822 folded long header lines. Those are difined as CRLF followed
by white space. Instead we'll search for two successive CRLF sequences which
mark end of mail headers and place our header just before that.

Signed-off-by: Vladimir Marek <vlmarek@volny.cz>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-07-27 18:54:52 +02:00
Nicolas Sebrecht
86cfd31ce9 v6.3.4-rc4
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-07-27 18:53:46 +02:00
Vladimir Marek
6995eeb92e Support maildir for windows
That makes OfflineIMAP to use exclamation mark (!) instead of colon for storing
messages. Such files can be written to windows partitions. But you will probably
loose compatibility with other programs trying to read the same Maildir.

Signed-off-by: Vladimir Marek <vladimir.marek@oracle.com>
Reviewed-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-07-12 21:53:33 +02:00
Nicolas Sebrecht
129b703bf2 v6.3.4-rc3
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-07-07 18:43:18 +02:00
Sebastian Spaeth
c293e64119 Fix caching inconsistencies in getselectedfolder()
getselectedfolder was using a cached variable that we were setting in
select(), but sometimes the IMAP4 instance got into the SELECTED state
without explicitely select()ing, it seems, and our variable was unset.

Let us just use the self.mailbox variable that imaplib2 is setting when
select()ing rather than doing our own caching. Also remove the part
where we were setting the cache.

Just access self.state rather than looking up self.state via
self.getstate() every time, it is just an unnecessary layer of
redirection.

Original-patch-by: Arnaud Fontaine <arnau@debian.org>
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-07-07 18:33:45 +02:00
Arnaud Fontaine
09ba269576 Add missing import of SSLError exception.
In commit 89cbdc9, usage of SSLError was dropped but later reintroduced
without importing SSLError exception.

Signed-off-by: Arnaud Fontaine <arnau@debian.org>
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-07-07 18:31:38 +02:00
Sebastian Spaeth
aec35eebd3 Reduce initial license blurb
Rather than the extremly verbose NO WARRANTY blurb, we output a somewhat
smaller initial text which should still make the GPL happy.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
2011-07-04 18:08:51 +02:00