Commit Graph

66 Commits

Author SHA1 Message Date
Peter Colberg
3299b2240c Applied patch from Peter Colberg to remove usage of hard linking
Closes: #535160

Bug#535160: offlineimap fails on encfs (FUSE) due to hard linking

Package: offlineimap
Version: 6.0.3
Severity: normal
Tags: patch

Hi,

if the localrepository is stored on an encfs, offlineimap fails as follows:

    OfflineIMAP 6.0.3
    Copyright (C) 2002 - 2008 John Goerzen <jgoerzen@complete.org>
    This software comes with ABSOLUTELY NO WARRANTY; see the file
    COPYING for details.  This is free software, and you are welcome
    to distribute it under the conditions laid out in COPYING.
    ***** Processing account pleione
    Copying folder structure from IMAP to Maildir
    Establishing connection to tunnel:eval `keychain --quiet --eval` && ssh -q mymailhost /usr/sbin/dovecot --exec-mail imap.
    Syncing Drafts: IMAP -> Maildir
    Syncing INBOX: IMAP -> Maildir
    ----------8<----------
    Syncing net: IMAP -> Maildir
    Copy message 360 IMAP[net] -> Maildir[net], LocalStatus[net]
    Syncing personal: IMAP -> Maildir
    Thread 'Copy message 360 from net' terminated with exception:
    Traceback (most recent call last):
      File "/var/lib/python-support/python2.4/offlineimap/threadutil.py", line 149, in run
	Thread.run(self)
      File "threading.py", line 422, in run
	self.__target(*self.__args, **self.__kwargs)
      File "/var/lib/python-support/python2.4/offlineimap/folder/Base.py", line 282, in copymessageto
	newuid = object.savemessage(uid, message, flags, rtime)
      File "/var/lib/python-support/python2.4/offlineimap/folder/Maildir.py", line 198, in savemessage
	os.path.join(tmpdir, messagename))
    OSError: [Errno 1] Operation not permitted

    Last 4 debug messages logged for Copy message 360 from net prior to exception:
    ----------8<----------

As encfs does not support hard linking, the rename of the temporary
message file fails.

This is the only occurrence of hard linking in offlineimap and can be
fixed by using an atomic `os.rename` instead of `os.link`+`os.unlink`.
I have been using the attached patch for more than 1.5 years and have
never again had any problems with offlineimap on encfs.

Regards,
Peter
2009-06-30 00:03:37 -05:00
Paul Hinze
c5886074ee Fix md5 import error
Paul Hinze <paul.t.hinze@gmail.com> on 2008-12-22 at 19:16:
> I did a bit of debugging and asking around in #python and it turns out
> that because Maildir.py uses a conditional import of md5 from either
> hashlib or the md5 module, the md5.new() call is not always correct.
> The attached patch should fix this problem.

Thanks to Daniel Rall for pointing out that the attachment got stripped
off my last message.  Patch follows.

Paul

----

From e01fdfbf62c61c31b514e826e6cc00fb98d88808 Mon Sep 17 00:00:00 2001
From: Paul Hinze <paul.t.hinze@gmail.com>
Date: Mon, 22 Dec 2008 18:42:44 -0600
Subject: [PATCH] Fixing md5 import when hashlib is available

Thanks to habnabit and Rhamphoryncus in #python for help debugging
2009-01-09 15:46:38 -06:00
Loui Chang
c6d95bd471 Import md5 from hashlib if available.
The md5 module is deprecated and hashlib is available
since python 2.5.

Signed-off-by: Loui Chang <louipc.ist@gmail.com>
2008-12-11 00:28:09 -06:00
John Goerzen
0da6bedaa0 Eliminate one more fsync 2008-08-02 15:31:43 -05:00
John Goerzen
fbcb157ca4 Fix syntax error in nofsync support 2008-08-02 15:11:23 -05:00
John Goerzen
2b23657db0 Added ability to disable fsync()
Passed config to LocalStatus and Maildir folders so they can look
up the fsync status
2008-08-02 14:55:08 -05:00
John Goerzen
24cdba3221 Undo 'Fix Maildir race' patch
This was causing OfflineIMAP to hang
2007-10-19 20:29:34 +01:00
John Goerzen
6caaea36e0 Fix Maildir race
fixes deb#439384

From: martin f krafft
Subject: race condition in Maildir writing

The offlineimap Maildir code checks for file existence and then
opens a file. That's open to a race condition. It's better to open
the file and fail if it already exists. The following patch does
this. It catches OSError 17 (file exists) and re-raises all others.
I'll leave it up to you to decide whether this is appropriate.
2007-10-19 01:06:18 +01:00
John Goerzen
3305d8cd4d Daniel Jacobowitz patches
fixes deb#433732

Date: Sun, 30 Sep 2007 13:54:56 -0400
From: Daniel Jacobowitz <drow@false.org>
To: offlineimap@complete.org
Subject: Assorted patches

Here's the result of a lazy Sunday hacking on offlineimap.  Sorry for
not breaking this into multiple patches.  They're mostly logically
independent so just ask if that would make a difference.
First, a new -q (quick) option.  The quick option means to only update
folders that seem to have had significant changes.  For Maildir, any
change to any message UID or flags is significant, because checking
the flags doesn't add a significant cost.  For IMAP, only a change to
the total number of messages or a change in the UID of the most recent
message is significant.  This should catch everything except for
flags changes.

The difference in bandwidth is astonishing: a quick sync takes 80K
instead of 5.3MB, and 28 seconds instead of 90.

There's a configuration variable that lets you say every tenth sync
should update flags, but let all the intervening ones be lighter.


Second, a fix to the UID validity problems many people have been
reporting with Courier.  As discussed in Debian bug #433732, I changed
the UID validity check to use SELECT unless the server complains that
the folder is read-only.  This avoids the Courier bug (see the Debian
log for more details).  This won't fix existing validity errors, you
need to remove the local status and validity files by hand and resync.


Third, some speedups in Maildir checking.  It's still pretty slow
due to a combination of poor performance in os.listdir (never reads
more than 4K of directory entries at a time) and some semaphore that
leads to lots of futex wake operations, but at least this saves
20% or so of the CPU time running offlineimap on a single folder:

Time with quick refresh and md5 in loop: 4.75s user 0.46s system 12%
cpu 41.751 total
Time with quick refresh and md5 out of loop: 4.38s user 0.50s system
14% cpu 34.799 total
Time using string compare to check folder: 4.11s user 0.47s system 13%
cpu 34.788 total


And fourth, some display fixes for Curses.Blinkenlights.  I made
warnings more visible, made the new quick sync message cyan, and
made all not explicitly colored messages grey.  That last one was
really bugging me.  Any time OfflineIMAP printed a warning in
this UI, it had even odds of coming out black on black!


Anyway, I hope these are useful.  I'm happy to revise them if you see
a problem.

-- 
Daniel Jacobowitz
CodeSourcery
2007-10-01 22:20:37 +01:00
Asheesh Laroia
35e7250187 only-write-once-to-cur-or-new.patch
I have tested this and Dovecot no longer beats offlineimap
to the punch. (-:

I achieved that by keeping the renames in tmp/ until it finally does
one last rename in new/ or cur/.
2007-06-13 04:42:15 +01:00
John Goerzen
4f54887265 Improve filesystem flushing semantics
fsync the Maildir file, its final directory when writing a new message.

fsync the localstatus file and its final directory when writing the 
local status cache.

This should reduce duplication in the event of hardware trouble.

fixes #8

see thread at http://lists.complete.org/offlineimap@complete.org/2007/03/threads.html.gz
2007-03-28 21:23:18 +01:00
John Goerzen
aa019172cb Handle rtime being Null when writing to Maildir
fixes #2
debian #401290
2006-12-02 21:54:26 +01:00
John Goerzen
c31f60f8df Update copyright date in Maildir.py 2006-12-02 21:54:15 +01:00
John Goerzen
03488ba81b Sync INTERNALDATE <-> mtime
The attached patch adds syncing the INTERNALDATE of IMAP folders with
the mtime of messages in maildir folders.
I want this to happen, because I'm running a dovecot over the maildirs
synced by offlineimap, and that uses the mtime as the INTERNALDATE.
When using mutt to view messages I generally sort based on the received
date, which for IMAP folders is the INTERNALDATE.

Since this is the first real coding I've done in Python the patch may
need to be cleaned up some, but it's working pretty well for me.  I've
added new messages to each side, and the received date has been
preserved going both ways.
2006-08-22 02:09:36 +01:00
John Goerzen
39a18fef60 Update FSF address 2006-08-12 05:15:55 +01:00
John Goerzen
d839be3c61 Step 2 of SVN to arch tree conversion 2005-04-16 20:33:35 +01:00