Commit Graph

2004 Commits

Author SHA1 Message Date
Sebastian Spaeth
de537dc09c Implement change_message_uid
Previously, assigning a new UID to a mapped IMAP or Maildir repository
was done by loading the "local" item, saving it under a new UID and
deleting the old one. This involved lots of disk activity for nothing
more than an effective file rename in Maildirs, and lots of network
usage in the MappedUID cases.

We do this on every upload from a local to a remote item, so that can
potentially be quite expensive. This patch lets backends that support it
(Maildir, MappedUID) efficiently rename the file rather than having to
read the mail content, write it out as a new file and delete the old
file. This speeds up uploads from Maildir and the MappedUID server.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2012-01-06 19:08:48 +01:00
Sebastian Spaeth
09ce56c594 Factor out creating a Maildir message filename
Various functions (such as change_message_uid) will want to construct
maildir filenames, so factor out the code into a helper.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2012-01-06 19:08:48 +01:00
Sebastian Spaeth
6fe808338c Refactor parsing out maildirs filename components
Create a helper function that retrieves the UID, folder MD5, and Flags from
a message filename.

We need these items when we simply want to rename (=new UID) a Maildir
message file later. The new function can give us these components.

Rework, so we cache the calculation of the folder's md5 value once, it
never changes and we call it a lot.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2012-01-06 19:08:45 +01:00
Sebastian Spaeth
9d79bd7b01 Fix regression of MachineUI
The recent UI overhaul led to the breakage of the machineui. This bug
fixes that.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2012-01-06 14:06:48 +01:00
Sebastian Spaeth
de4f8c8605 Remove unneeded import in MachineUI
The urllib module is not needed anymore.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2012-01-06 14:06:03 +01:00
Sebastian Spaeth
d75a7a862f Fix regression of MachineUI
The recent UI overhaul led to the breakage of the machineui. This bug
fixes that.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2012-01-06 14:02:22 +01:00
Sebastian Spaeth
b569ef551f Merge branch 'next'
Conflicts:
	Changelog.rst
2012-01-06 13:24:50 +01:00
Sebastian Spaeth
69838b81a5 Release 6.4.4
This is a bugfix release that fixes regressions since 6.4.0.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2012-01-06 13:22:34 +01:00
Sebastian Spaeth
d5666ce91d Don't create invalid maildir names with lower case maildir flags
If someone had a custom :2,a flag, adding a new flag would lead to the
invalid maildir filename ...a:2,... due to regex deficiencies not coping
with this. Fix this so we alway produce valid maildir names.

Note that custom flags are still problematic: as the syncing to the
remote IMAP server will fail, the next sync will assume that they have
been removed from the remote IMAP side and they will be removed from the
local Maildir then. We will need to think about how to handle this. At
least, with this patch we won't lose standard flags and won't produce
invalid maildir names.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2012-01-06 13:05:08 +01:00
Sebastian Spaeth
64f5e557bc Fix (harmless) regex flaw when determining maildir flags
The regex for catching Maildir message flags was
self.infosep + '.*2,([A-Z]+)' (infosep being ':').

The .* is bogus, as there is nothing between the : and the 2, per
maildir name specification, so remove that unneeded piece.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2012-01-05 14:39:09 +01:00
Sebastian Spaeth
a280a905ca Fix "getfolders() asked to get nonexisting folder" bug
When a new remote folder was detected, we tried to create the folder
locally on the Maildir and called repository.forgetfolders() to force a
new scanning of the Maildir. However, that implementation used the
inherited base function that did nothing. We simply needed to implement
forgetfolders() to set self.folder=None, so we would force a new read in
of the updated local folder structure.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2012-01-05 14:23:33 +01:00
Sebastian Spaeth
3e28073f98 Do not create folders on readonly repositories
1) Rename the unintuitive repository.syncfoldersto() to
sync_folder_structure()

2) We were checking if the local repository is readonly and then turning
off any folder creation. But as we can create folders on a remote
repository too, we need to be more fine grained here. Just don't create
a folder on the repository that is marked readonly=True.

This still does not do away with the error message that one currently
gets on missing local folders.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2012-01-05 14:05:51 +01:00
Sebastian Spaeth
6f361c4d9a Don't output "Finished in x seconds" in quiet ui
The quiet UI should only output errors, and the final "Finished account
X in 2 seconds" clearly is none, so the message debug level needed to be
reduced to INFO to suppress it in the quiet ui.

Fixes https://github.com/spaetz/offlineimap/issues/6
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2012-01-05 13:21:08 +01:00
Sebastian Spaeth
04a1403f18 Merge branch 'next'
Conflicts:
	Changelog.draft.rst
	Changelog.rst
	offlineimap/__init__.py

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2012-01-04 19:38:55 +01:00
Sebastian Spaeth
9a7c700248 Release 6.4.3
Merge in the new logging mechanism, and provide an --info feature that
will help with debugging. There is still some unstableness, so there
will be another release soon, but this should be no worse than 6.4.2 at
least...

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2012-01-04 19:34:13 +01:00
Sebastian Spaeth
0ccf06d5e6 Implement clean CTRL-C termination
Previously, we would simply bail out in an ugly way, potentially leaving
temporary files around etc, or while writing status files. Hand SIGINT
and SIGTERM as an event to the Account class, and make that bail out
cleanly at predefined points. Stopping on ctrl-c can take a few seconds
(it will e.g. finish to transfer the ongoing message), but it will shut
down cleanly.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2012-01-04 19:31:27 +01:00
Sebastian Spaeth
8ec6980c96 Don't fail on empty LocalStatus cache files
As reported in https://github.com/spaetz/offlineimap/pull/2, we would
fail when files are empty because file.read() would throw attribute
errors.

Fix this by removing the superfluous read() check and additionally log
some warning message.

Reported-by: Ralf Schmitt
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-12-01 23:57:54 +01:00
Sebastian Spaeth
5f3ac2c4a2 Release v6.4.2
This is a bugfix release over 6.4.1. Upgrading is recommended.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-12-01 17:21:09 +01:00
Sebastian Spaeth
8c44115807 Init folder list early enough
We need the list of folders and the folder delimiter, but it was not
always retrieved early enough. E.g. when doing IMAP<->IMAP sync and the
local IMAP being readonly, we would bunk out with a mysterious error
message become repository.getsel() would still return None.

This commit fixes this error.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-12-01 17:21:09 +01:00
Sebastian Spaeth
50e78a8d41 Release v6.4.2
This is a bugfix release over 6.4.1. Upgrading is recommended.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-12-01 17:14:52 +01:00
Sebastian Spaeth
416b1fe551 Add changelog entry for bug fixed in previous commit
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-12-01 17:06:26 +01:00
Sebastian Spaeth
3bc68ecd65 Don't append trailing slash to maildir foldernames
When sep='/' in a Maildir, we were doing a os.path.join(dirname,'') on
the top level maildir, which results in a "dirname/", so all our maildir
folder names had slashes appended. Which is pretty much wrong, so this
fixes it by only using os.path.join when we actually have something to
append.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-12-01 17:04:23 +01:00
Sebastian Spaeth
9a654968fc Don't append trailing slash to maildir foldernames
When sep='/' in a Maildir, we were doing a os.path.join(dirname,'') on
the top level maildir, which results in a "dirname/", so all our maildir
folder names had slashes appended. Which is pretty much wrong, so this
fixes it by only using os.path.join when we actually have something to
append.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-12-01 17:01:43 +01:00
Sebastian Spaeth
c93f8710a3 Init folder list early enough
We need the list of folders and the folder delimiter, but it was not
always retrieved early enough. E.g. when doing IMAP<->IMAP sync and the
local IMAP being readonly, we would bunk out with a mysterious error
message become repository.getsel() would still return None.

This commit fixes this error.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-12-01 10:31:43 +01:00
Sebastian Spaeth
bf4127c2d6 Remove unused imapserver getdelim()
imapserver.getdelim() was not used at all, so remove this function. The
folder delimiter is available via the repository.getsep() call.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-12-01 10:12:54 +01:00
Sebastian Spaeth
20f2edfcec Sanity check to notify us when we call IMAPRepository.getsep() too early
The folder delimiter is only initialized after a call to
acquireconnection(), so we must never call this function too
early. Include an assert() to make sure we get notified when we do.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-12-01 09:49:09 +01:00
Sebastian Spaeth
464342e1dd Fix version number typo in Changelog
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-11-17 09:04:14 +01:00
Sebastian Spaeth
51164c4974 Fix version number typo in Changelog
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-11-17 09:04:03 +01:00
Sebastian Spaeth
98e82db505 Release current master as 6.4.1
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>

Conflicts:

	Changelog.draft.rst
2011-11-17 09:00:58 +01:00
Sebastian Spaeth
1a4b536a50 Release current master as 6.4.1
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-11-17 08:55:32 +01:00
Sebastian Spaeth
a93c80292d Curses UI: Simplify text buffer handling
Rather than keeping a separate queue of all logged lines in memory, we
rely on the curses window scrolling functionality to scroll lines. On
resizing the terminal this means, we'll clear the screen and start
filling it afresh, but that should be acceptable.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-11-08 13:51:36 +01:00
Sebastian Spaeth
f8d5f1890c Make sleep abort request working again for Curses UI
1) Rework the sleep abort request to set the skipsleep configuration
   setting that the sleep() code checks.
2) Only output 15 rather than 50 debug messages on abort...

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-11-03 14:21:25 +01:00
Sebastian Spaeth
f4a32bafd6 Pass ui.registerthread an Account() and not a name as string
This way, we can use all the account functions such as set_abort_event()
from the ui if needed.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-11-03 13:45:44 +01:00
Sebastian Spaeth
ab184d84e2 Reduce parameter list to account.syncfolder call
The remote|local|statusrepo is an anttribute of each SyncableAccount()
anyway, so we don't need to pass it in, we can simply get it from the
Account().

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-11-03 13:29:43 +01:00
Sebastian Spaeth
ef28d5dac0 More Blinkenlights UI cleanup
Rename some variables, simplify the hotkeys treatment. Refresh/exit
signals still don't work as of yet, but will come.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-11-03 12:25:55 +01:00
Sebastian Spaeth
70125d58e6 Curses UI: make resize behave better
Resizing a Blinkenlights terminal doesn't crash anymore, and actually
seems to be changing the size, with this patch.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-11-02 17:03:40 +01:00
Sebastian Spaeth
bfbd378025 Add FAQ item on "Mailbox already exists" error
IMAP servers treating folder names as case insensitive can lead to that
error.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-11-02 15:56:33 +01:00
Sebastian Spaeth
212e50eb4b Fix sleeping in the Blinkenlights UI
We were still referring to s.gettf() in sleeping(self, ...) causing each
attempt to sleep to crash. Fix this, and the CursesAccountFrame.sleeping()
method. I am sure, there is still wrong and broken but we are getting there.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-11-02 12:48:22 +01:00
Sebastian Spaeth
8dbf62cfdb Add a TODO comment
This function can IMHO lead to possible deadlocks when waiting for the
connectionlock. Do add a comment to that regard, this will need to audit.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-11-02 11:56:04 +01:00
Sebastian Spaeth
74d580bc68 Exit "infinite" monitorloop when SyncRunner thread exits
The huge UI rework patch removed some obscure logic and special handling of
thread exit messages. It turns out that this was in fact still needed as a
specific exit message of the SyncRunner thread signified the threatmonitor
to quit.

We will want a nicer machinery for this in the future I guess, but fix the
eternal hang on exit by reintroducing a special exit message for the
SyncRunner thread, and return from the infinite monitor loop if SyncRunner
finishes.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-11-02 11:55:05 +01:00
Sebastian Spaeth
52ca66f055 Reduce log verbosity while scanning Maildir
no need to be that verbose here...

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-11-02 11:37:29 +01:00
Sebastian Spaeth
ccfa747ce6 Use lock with 'with' statement
To make sure, the lock gets released even if we raise an exception between
acquire() and release()

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-11-02 11:30:16 +01:00
Sebastian Spaeth
d54859a931 Don't setDaemon explicitly, it's done inherently
All ExitNotifyThreads and InstanceLimitThreads are setDaemon(True) in their
constructor, so there is no need to do that again in the code.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-11-02 11:29:23 +01:00
Sebastian Spaeth
4eeb88dd8f Append, don't truncate, log file
The new logging handler was truncating the log file on each start.
Append by default.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-11-02 10:56:42 +01:00
Sebastian Spaeth
54117e702d Bump bundled imaplib2 to 2.29
It has fixed some bugs, so update to current upstream.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-11-02 10:55:42 +01:00
Sebastian Spaeth
dd02c307db Drop connection if it might be bad on APPEND
1) Differentiate error messages between imaplib.abort and imaplib.error
exceptions in the log.

2) Drop connections in the case of imapobj.error, it also might denote a
   broken connection.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-11-02 10:37:31 +01:00
Sebastian Spaeth
3aded16ed4 Make releaseconnection a NOOP when conn is None
During cleanup we often call releaseconnection in a finally: block. But
in cases of error, we might have dropped the connection earlier already
and set it to "None". In this case don't fail releaseconnection() but
make it a NOOP.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-11-02 10:37:17 +01:00
Sebastian Spaeth
656405616d Drop connection if it might be bad on APPEND
1) Differentiate error messages between imaplib.abort and imaplib.error
exceptions in the log.

2) Drop connections in the case of imapobj.error, it also might denote a
   broken connection.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-11-02 10:36:30 +01:00
Sebastian Spaeth
006f19aba9 Make releaseconnection a NOOP when conn is None
During cleanup we often call releaseconnection in a finally: block. But
in cases of error, we might have dropped the connection earlier already
and set it to "None". In this case don't fail releaseconnection() but
make it a NOOP.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
2011-11-02 10:36:30 +01:00
Sebastian Spaeth
8b3ed8b004 Fix mbox.select(foldername) readonly parameter comparison
The default parameter value was "None", and we were comparing that
directly to the imaplib2 value of is_readonly which is False or True, so
the comparison always returned "False".

Fix this by setting the default parameter to "False" and not
"None". Also convert all users of that function to use False/True.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>

Conflicts:

	Changelog.draft.rst
2011-11-02 09:00:11 +01:00