doautosave was a useless variable before (it was *always* 1). So we
remove the self.dofsync variable and store in doautosave whether we
should fsync as often as possible (which really hurts performance).
The sqlite backend could (at one point) use the doautosave variable to
determine if it should autocommit after each modification.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Based on patches by Stewart Smith, updated by Rob Browning.
plus:
- Inherit LocalStatusSQLFolder from LocalStatusFolder
This lets us remove all functions that are available via our ancestors
classes and are not needed.
- Don't fail if pysql import fails. Fail rather at runtime when needed.
- When creating the db file, create a metadata table which contains the
format version info, so we can upgrade nicely to other formats.
- Create an upgrade_db() function which allows us to upgrade from any
previous file format to the current one (including plain text)
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
When our LocalStatus cache is corrupt, ie e.g. it contains lines not in
the form number:number, we would previously just raise a ValueError
stating things like "too many values". In case we encounter clearly
corrupt LocalStatus cache entries, clearly raise an exception stating
the filename and the line, so that people can attempt to repair it.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
This was in the code for a very long time, it seems.
Remove one instance, no functional changes.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Earlier we would ignore *ALL* Exceptions that could occur during the
fsyncing of our LocalStatus database. Ignoring all Exceptions is not the
right thing here though. A recent commit improved the situation by
raising at least KeyboardInterrupt Exceptions, but that is still not
optimal.
os.fsync() is available on Unix, and Windows starting in python
2.2.3. so it should always work. If it doesn't, something is wrong.
It has been suggested to only catch EnvironmentError (ie SystemError and
OSError) here, but even those should be thrown. Something *is* wrong if
this fails and we should not ignore it.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Previously we did not catch KeyboardInterrupts explicitly as all of the
code was executed in forked child threads which would never receive
Ctrl-c exceptions. With the upcoming single threaded modus, this code
can be run in the main thread however, so we need to take care of
KeyboardInterrupts explicitly.
This was done wherever we would catch *ALL* exceptions universally and
print out an error message.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
After tens of thousands of messages on the IMAP server were deleted it
takes offlineimap extremely long time (several hours of high CPU usage)
to delete them locally. It spends almost all the time modifying
LocalStatus. It processes the messages one by one, rewriting the
folder's status file in LocalStatus after each message.
It is much more efficient to save the status file only once, after
removing all the messages from the messagelist.
Deleting lots of messages now takes seconds instead of hours.
This should solve Debian bug #518093:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=518093
Signed-off-by: Michal Schmidt <mschmidt@redhat.com>
I created new IMAP folders on my server. In some moment I found offlineimap
crashed. The reason were empty LocalStatus files. From looking at the code I
have no explanation how those files came into existence.
fsync the Maildir file, its final directory when writing a new message.
fsync the localstatus file and its final directory when writing the
local status cache.
This should reduce duplication in the event of hardware trouble.
fixes#8
see thread at http://lists.complete.org/offlineimap@complete.org/2007/03/threads.html.gz
The attached patch adds syncing the INTERNALDATE of IMAP folders with
the mtime of messages in maildir folders.
I want this to happen, because I'm running a dovecot over the maildirs
synced by offlineimap, and that uses the mtime as the INTERNALDATE.
When using mutt to view messages I generally sort based on the received
date, which for IMAP folders is the INTERNALDATE.
Since this is the first real coding I've done in Python the patch may
need to be cleaned up some, but it's working pretty well for me. I've
added new messages to each side, and the received date has been
preserved going both ways.
