Merge branch 'next'

Changelog.draft.rst

@@ -18,8 +18,3 @@ Changes
   
 Bug Fixes
 ---------
-
-Pending for the next major release
-==================================
-
-* UIs get shorter and nicer names. (API changing)

Changelog.rst

@@ -11,6 +11,65 @@ ChangeLog
   on releases. And because I'm lazy, it will also be used as a draft for the
   releases announces.
 
+OfflineIMAP v6.4.0 (2011-09-29)
+===============================
+
+This is the first stable release to support the forward-compatible per-account locks and remote folder creation that has been introduced in the 6.3.5 series.
+
+* Various regression and bug fixes from the last couple of RCs
+
+OfflineIMAP v6.3.5-rc3 (2011-09-21)
+===================================
+
+Changes
+-------
+
+* Refresh server capabilities after login, so we know that Gmail
+  supports UIDPLUS (it only announces that after login, not
+  before). This prevents us from adding custom headers to Gmail uploads.
+  
+Bug Fixes
+---------
+
+* Fix the creation of folders on remote repositories, which was still
+  botched on rc2.
+
+OfflineIMAP v6.3.5-rc2 (2011-09-19)
+===================================
+
+New Features
+------------
+
+* Implement per-account locking, so that it will possible to sync
+  different accounts at the same time. The old global lock is still in
+  place for backward compatibility reasons (to be able to run old and
+  new versions of OfflineImap concurrently) and will be removed in the
+  future. Starting with this version, OfflineImap will be
+  forward-compatible with the per-account locking style.
+
+* Implement RFC 2595 LOGINDISABLED. Warn the user and abort when we
+  attempt a plaintext login but the server has explicitly disabled
+  plaintext logins rather than crashing.
+
+* Folders will now also be automatically created on the REMOTE side of
+  an account if they exist on the local side. Use the folderfilters
+  setting on the local side to prevent some folders from migrating to
+  the remote side.  Also, if you have a nametrans setting on the remote
+  repository, you might need a nametrans setting on the local repository
+  that leads to the original name (reverse nametrans).
+
+Changes
+-------
+
+* Documentation improvements concerning 'restoreatime' and some code cleanup
+
+* Maildir repositories now also respond to folderfilter= configurations.
+
+Bug Fixes
+---------
+
+* New emails are not created with "-rwxr-xr-x" but as "-rw-r--r--"
+  anymore, fixing a regression in 6.3.4.
 
 OfflineIMAP v6.3.5-rc1 (2011-09-12)
 ===================================

docs/MANUAL.rst

@@ -476,7 +476,8 @@ To only get the All Mail folder from a Gmail account, you would e.g. do::
 Another nametrans transpose example
 -----------------------------------
 
-Put everything in a GMX. subfolder except for the boxes INBOX, Draft, and Sent which should keep the same name::
+Put everything in a GMX. subfolder except for the boxes INBOX, Draft,
+and Sent which should keep the same name::
 
      nametrans: lambda folder: folder if folder in ['INBOX', 'Drafts', 'Sent'] \
                                else re.sub(r'^', r'GMX.', folder)
@@ -484,7 +485,9 @@ Put everything in a GMX. subfolder except for the boxes INBOX, Draft, and Sent w
 2 IMAP using name translations
 ------------------------------
 
-Synchronizing 2 IMAP accounts to local Maildirs that are "next to each other", so that mutt can work on both. Full email setup described by Thomas Kahle at `http://dev.gentoo.org/~tomka/mail.html`_
+Synchronizing 2 IMAP accounts to local Maildirs that are "next to each
+other", so that mutt can work on both. Full email setup described by
+Thomas Kahle at `http://dev.gentoo.org/~tomka/mail.html`_
 
 offlineimap.conf::
 
@@ -534,11 +537,25 @@ offlineimap.conf::
     ssl = yes
     maxconnections = 2
 
-One of the coolest things about offlineimap is that you can inject arbitrary python code. The file specified with::
+One of the coolest things about offlineimap is that you can call
+arbitrary python code from your configuration.  To do this, specify a
+pythonfile with::
 
     pythonfile=~/bin/offlineimap-helpers.py
 
-contains python functions that I used for two purposes: Fetching passwords from the gnome-keyring and translating folder names on the server to local foldernames. The python file should contain all the functions that are called here. get_username and get_password are part of the interaction with gnome-keyring and not printed here. Find them in the example file that is in the tarball or here. The folderfilter is a lambda term that, well, filters which folders to get. `oimaptransfolder_acc2` translates remote folders into local folders with a very simple logic. The `INBOX` folder will simply have the same name as the account while any other folder will have the account name and a dot as a prefix. offlineimap handles the renaming correctly in both directions::
+Your pythonfile needs to contain implementations for the functions
+that you want to use in offflineimaprc.  The example uses it for two
+purposes: Fetching passwords from the gnome-keyring and translating
+folder names on the server to local foldernames.  An example
+implementation of get_username and get_password showing how to query
+gnome-keyring is contained in
+`http://dev.gentoo.org/~tomka/mail-setup.tar.bz2`_ The folderfilter is
+a lambda term that, well, filters which folders to get. The function
+`oimaptransfolder_acc2` translates remote folders into local folders
+with a very simple logic. The `INBOX` folder will have the same name
+as the account while any other folder will have the account name and a
+dot as a prefix. This is useful for hierarchichal display in mutt.
+Offlineimap handles the renaming correctly in both directions::
 
     import re
     def oimaptransfolder_acc1(foldername):

offlineimap.conf

@@ -285,12 +285,12 @@ localfolders = ~/Test
 
 sep = .
 
-# Some users on *nix platforms may not want the atime (last access
+# Some users may not want the atime (last access time) of folders to be
-# time) to be modified by OfflineIMAP.  In these cases, they would
+# modified by OfflineIMAP.  If 'restoreatime' is set to yes, OfflineIMAP
-# want to set restoreatime to yes.  OfflineIMAP will make an effort
+# will restore the atime of the "new" and "cur" folders in each maildir
-# to not touch the atime if you do that.
+# folder to their original value after each sync.
 #
-# In most cases, the default of no should be sufficient.
+# In nearly all cases, the default should be fine.
 
 restoreatime = no
 

offlineimap/CustomConfig.py

@@ -24,26 +24,25 @@ class CustomConfigParser(SafeConfigParser):
         """Same as config.get, but returns the "default" option if there
         is no such option specified."""
         if self.has_option(section, option):
-            return apply(self.get, [section, option] + list(args), kwargs)
+            return self.get(*(section, option) + args, **kwargs)
         else:
             return default
     
     def getdefaultint(self, section, option, default, *args, **kwargs):
         if self.has_option(section, option):
-            return apply(self.getint, [section, option] + list(args), kwargs)
+            return self.getint (*(section, option) + args, **kwargs)
         else:
             return default
 
     def getdefaultfloat(self, section, option, default, *args, **kwargs):
         if self.has_option(section, option):
-            return apply(self.getfloat, [section, option] + list(args), kwargs)
+            return self.getfloat(*(section, option) + args, **kwargs)
         else:
             return default
 
     def getdefaultboolean(self, section, option, default, *args, **kwargs):
         if self.has_option(section, option):
-            return apply(self.getboolean, [section, option] + list(args),
+            return self.getboolean(*(section, option) + args, **kwargs)
-                         kwargs)
         else:
             return default
 
@@ -91,9 +90,9 @@ class ConfigHelperMixin:
     def _confighelper_runner(self, option, default, defaultfunc, mainfunc):
         """Return config value for getsection()"""
         if default == CustomConfigDefault:
-            return apply(mainfunc, [self.getsection(), option])
+            return mainfunc(*[self.getsection(), option])
         else:
-            return apply(defaultfunc, [self.getsection(), option, default])
+            return defaultfunc(*[self.getsection(), option, default])
 
 
     def getconf(self, option,

offlineimap/__init__.py

@@ -1,7 +1,7 @@
 __all__ = ['OfflineImap']
 
 __productname__ = 'OfflineIMAP'
-__version__     = "6.3.5-rc1"
+__version__     = "6.4.0"
 __copyright__   = "Copyright 2002-2011 John Goerzen & contributors"
 __author__      = "John Goerzen"
 __author_email__= "john@complete.org"

offlineimap/accounts.py

@@ -25,6 +25,11 @@ import os
 from sys import exc_info
 import traceback
 
+try:
+    import fcntl
+except:
+    pass # ok if this fails, we can do without
+
 def getaccountlist(customconfig):
     return customconfig.getsectionlist('Account')
 
@@ -159,6 +164,36 @@ class SyncableAccount(Account):
     functions :meth:`syncrunner`, :meth:`sync`, :meth:`syncfolders`,
     used for syncing."""
 
+    def __init__(self, *args, **kwargs):
+        Account.__init__(self, *args, **kwargs)
+        self._lockfd = None
+        self._lockfilepath = os.path.join(self.config.getmetadatadir(),
+                                          "%s.lock" % self)
+
+    def lock(self):
+        """Lock the account, throwing an exception if it is locked already"""
+        # Take a new-style per-account lock
+        self._lockfd = open(self._lockfilepath, 'w')
+        try:
+            fcntl.lockf(self._lockfd, fcntl.LOCK_EX|fcntl.LOCK_NB)
+        except NameError:
+            #fcntl not available (Windows), disable file locking... :(
+            pass
+        except IOError:
+            self._lockfd.close()
+            raise OfflineImapError("Could not lock account %s." % self,
+                                   OfflineImapError.ERROR.REPO)
+
+    def unlock(self):
+        """Unlock the account, deleting the lock file"""
+        #If we own the lock file, delete it
+        if self._lockfd and not self._lockfd.closed:
+            self._lockfd.close()
+            try:
+                os.unlink(self._lockfilepath)
+            except OSError:
+                pass #Failed to delete for some reason.
+
     def syncrunner(self):
         self.ui.registerthread(self.name)
         self.ui.acct(self.name)
@@ -175,6 +210,7 @@ class SyncableAccount(Account):
         while looping:
             try:
                 try:
+                    self.lock()
                     self.sync()
                 except (KeyboardInterrupt, SystemExit):
                     raise
@@ -194,6 +230,7 @@ class SyncableAccount(Account):
                     if self.refreshperiod:
                         looping = 3
             finally:
+                self.unlock()
                 if looping and self.sleeper() >= 2:
                     looping = 0                    
                 self.ui.acctdone(self.name)
@@ -231,12 +268,16 @@ class SyncableAccount(Account):
             localrepos = self.localrepos
             statusrepos = self.statusrepos
             # replicate the folderstructure from REMOTE to LOCAL
-            if not localrepos.getconf('readonly', False):
+            if not localrepos.getconfboolean('readonly', False):
                 self.ui.syncfolders(remoterepos, localrepos)
                 remoterepos.syncfoldersto(localrepos, statusrepos)
 
             # iterate through all folders on the remote repo and sync
             for remotefolder in remoterepos.getfolders():
+                if not remotefolder.sync_this:
+                    self.ui.debug('', "Not syncing filtered remote folder '%s'"
+                                  "[%s]" % (remotefolder, remoterepos))
+                    continue # Filtered out remote folder
                 thread = InstanceLimitedThread(\
                     instancename = 'FOLDER_' + self.remoterepos.getname(),
                     target = syncfolder,
@@ -286,7 +327,9 @@ class SyncableAccount(Account):
 def syncfolder(accountname, remoterepos, remotefolder, localrepos,
                statusrepos, quick):
     """This function is called as target for the
-    InstanceLimitedThread invokation in SyncableAccount."""
+    InstanceLimitedThread invokation in SyncableAccount.
+
+    Filtered folders on the remote side will not invoke this function."""
     ui = getglobalui()
     ui.registerthread(accountname)
     try:
@@ -294,6 +337,14 @@ def syncfolder(accountname, remoterepos, remotefolder, localrepos,
         localfolder = localrepos.\
                       getfolder(remotefolder.getvisiblename().\
                                 replace(remoterepos.getsep(), localrepos.getsep()))
+
+        #Filtered folders on the remote side will not invoke this
+        #function, but we need to NOOP if the local folder is filtered
+        #out too:
+        if not localfolder.sync_this:
+            ui.debug('', "Not syncing filtered local folder '%s'" \
+                         % localfolder)
+            return
         # Write the mailboxes
         mbnames.add(accountname, localfolder.getvisiblename())
 
@@ -345,7 +396,7 @@ def syncfolder(accountname, remoterepos, remotefolder, localrepos,
                              remotefolder.getmessagecount())
 
         # Synchronize remote changes.
-        if not localrepos.getconf('readonly', False):
+        if not localrepos.getconfboolean('readonly', False):
             ui.syncingmessages(remoterepos, remotefolder, localrepos, localfolder)
             remotefolder.syncmessagesto(localfolder, statusfolder)
         else:
@@ -353,7 +404,7 @@ def syncfolder(accountname, remoterepos, remotefolder, localrepos,
                          % localrepos.getname())
         
         # Synchronize local changes
-        if not remoterepos.getconf('readonly', False):
+        if not remoterepos.getconfboolean('readonly', False):
             ui.syncingmessages(localrepos, localfolder, remoterepos, remotefolder)
             localfolder.syncmessagesto(remotefolder, statusfolder)
         else:
@@ -369,8 +420,15 @@ def syncfolder(accountname, remoterepos, remotefolder, localrepos,
         if e.severity > OfflineImapError.ERROR.FOLDER:
             raise
         else:
-            ui.error(e, exc_info()[2], msg = "Aborting folder sync '%s' "
+            #if the initial localfolder assignement bailed out, the localfolder var will not be available, so we need
-                     "[acc: '%s']" % (localfolder, accountname))
+            ui.error(e, exc_info()[2], msg = "Aborting sync, folder '%s' "
+                     "[acc: '%s']" % (
+                    remotefolder.getvisiblename().\
+                        replace(remoterepos.getsep(), localrepos.getsep()),
+                    accountname))
+                    # we reconstruct foldername above rather than using
+                    # localfolder, as the localfolder var is not
+                    # available if assignment fails.
     except Exception, e:
         ui.error(e, msg = "ERROR in syncfolder for %s folder %s: %s" % \
                 (accountname,remotefolder.getvisiblename(),

offlineimap/folder/Base.py

@@ -28,8 +28,23 @@ except NameError:
     from sets import Set as set
 
 class BaseFolder(object):
-    def __init__(self):
+    def __init__(self, name, repository):
+        """
+        :para name: Path & name of folder minus root or reference
+        :para repository: Repository() in which the folder is.
+        """
+        self.sync_this = True
+        """Should this folder be included in syncing?"""
         self.ui = getglobalui()
+        self.name = name
+        self.repository = repository
+        self.visiblename = repository.nametrans(name)
+        # In case the visiblename becomes '.' (top-level) we use '' as
+        # that is the name that e.g. the Maildir scanning will return
+        # for the top-level dir.
+        if self.visiblename == '.':
+            self.visiblename = ''
+        self.config = repository.getconfig()
 
     def getname(self):
         """Returns name"""
@@ -38,6 +53,11 @@ class BaseFolder(object):
     def __str__(self):
         return self.name
 
+    @property
+    def accountname(self):
+        """Account name as string"""
+        return self.repository.accountname
+
     def suggeststhreads(self):
         """Returns true if this folder suggests using threads for actions;
         false otherwise.  Probably only IMAP will return true."""
@@ -55,7 +75,8 @@ class BaseFolder(object):
         return 1
 
     def getvisiblename(self):
-        return self.name
+        """The nametrans-transposed name of the folder's name"""
+        return self.visiblename
 
     def getrepository(self):
         """Returns the repository object that this folder is within."""
@@ -233,7 +254,7 @@ class BaseFolder(object):
         # self.getmessage().  So, don't call self.getmessage unless
         # really needed.
         if register: # output that we start a new thread
-            self.ui.registerthread(self.getaccountname())
+            self.ui.registerthread(self.accountname)
 
         try:
             message = None
@@ -289,7 +310,7 @@ class BaseFolder(object):
             self.ui.error(e, exc_info()[2])
         except Exception, e:
             self.ui.error(e, "Copying message %s [acc: %s]:\n %s" %\
-                              (uid, self.getaccountname(),
+                              (uid, self.accountname,
                                traceback.format_exc()))
             raise    #raise on unknown errors, so we can fix those
 
@@ -437,5 +458,5 @@ class BaseFolder(object):
                 self.ui.error(e, exc_info()[2])
             except Exception, e:
                 self.ui.error(e, exc_info()[2], "Syncing folder %s [acc: %s]" %\
-                                  (self, self.getaccountname()))
+                                  (self, self.accountname))
                 raise # raise unknown Exceptions so we can fix them

offlineimap/folder/Gmail.py

@@ -33,13 +33,12 @@ class GmailFolder(IMAPFolder):
       http://mail.google.com/support/bin/answer.py?answer=77657&topic=12815
     """
 
-    def __init__(self, imapserver, name, visiblename, accountname, repository):
+    def __init__(self, imapserver, name, repository):
+        super(GmailFolder, self).__init__(imapserver, name, repository)
         self.realdelete = repository.getrealdelete(name)
         self.trash_folder = repository.gettrashfolder(name)
         #: Gmail will really delete messages upon EXPUNGE in these folders
         self.real_delete_folders =  [ self.trash_folder, repository.getspamfolder() ]
-        IMAPFolder.__init__(self, imapserver, name, visiblename, \
-                            accountname, repository)
 
     def deletemessages_noconvert(self, uidlist):
         uidlist = [uid for uid in uidlist if uid in self.messagelist]

offlineimap/folder/IMAP.py

@@ -32,19 +32,15 @@ except NameError:
 
 
 class IMAPFolder(BaseFolder):
-    def __init__(self, imapserver, name, visiblename, accountname, repository):
+    def __init__(self, imapserver, name, repository):
-        self.config = imapserver.config
+        name = imaputil.dequote(name)
+        super(IMAPFolder, self).__init__(name, repository)
         self.expunge = repository.getexpunge()
-        self.name = imaputil.dequote(name)
         self.root = None # imapserver.root
         self.sep = imapserver.delim
         self.imapserver = imapserver
         self.messagelist = None
-        self.visiblename = visiblename
-        self.accountname = accountname
-        self.repository = repository
         self.randomgenerator = random.Random()
-        BaseFolder.__init__(self)
         #self.ui is set in BaseFolder
 
     def selectro(self, imapobj):
@@ -61,9 +57,6 @@ class IMAPFolder(BaseFolder):
         except imapobj.readonly:
             imapobj.select(self.getfullname(), readonly = 1)
 
-    def getaccountname(self):
-        return self.accountname
-
     def suggeststhreads(self):
         return 1
 
@@ -73,9 +66,6 @@ class IMAPFolder(BaseFolder):
     def getcopyinstancelimit(self):
         return 'MSGCOPY_' + self.repository.getname()
 
-    def getvisiblename(self):
-        return self.visiblename
-
     def getuidvalidity(self):
         imapobj = self.imapserver.acquireconnection()
         try:
@@ -89,10 +79,22 @@ class IMAPFolder(BaseFolder):
         # An IMAP folder has definitely changed if the number of
         # messages or the UID of the last message have changed.  Otherwise
         # only flag changes could have occurred.
+        retry = True # Should we attempt another round or exit?
+        while retry:
+            retry = False
             imapobj = self.imapserver.acquireconnection()
             try:
-            # Primes untagged_responses
+                # Select folder and get number of messages
-            imaptype, imapdata = imapobj.select(self.getfullname(), readonly = 1, force = 1)
+                restype, imapdata = imapobj.select(self.getfullname(), True,
+                                                   True)
+            except OfflineImapError, e:
+                # retry on dropped connections, raise otherwise
+                self.imapserver.releaseconnection(imapobj, True)
+                if e.severity == OfflineImapError.ERROR.FOLDER_RETRY:
+                    retry = True
+                else: raise
+            finally:
+                self.imapserver.releaseconnection(imapobj)
         # 1. Some mail servers do not return an EXISTS response
         # if the folder is empty.  2. ZIMBRA servers can return
         # multiple EXISTS replies in the form 500, 1000, 1500,
@@ -102,13 +104,9 @@ class IMAPFolder(BaseFolder):
         maxmsgid = 0
         for msgid in imapdata:
             maxmsgid = max(long(msgid), maxmsgid)
-
         # Different number of messages than last time?
         if maxmsgid != statusfolder.getmessagecount():
             return True      
-
-        finally:
-            self.imapserver.releaseconnection(imapobj)
         return False
 
     def cachemessagelist(self):
@@ -120,7 +118,7 @@ class IMAPFolder(BaseFolder):
 
         imapobj = self.imapserver.acquireconnection()
         try:
-            res_type, imapdata = imapobj.select(self.getfullname(), True)
+            res_type, imapdata = imapobj.select(self.getfullname(), True, True)
             if imapdata == [None] or imapdata[0] == '0':
                 # Empty folder, no need to populate message list
                 return
@@ -211,9 +209,9 @@ class IMAPFolder(BaseFolder):
                     res_type, data = imapobj.uid('fetch', str(uid),
                                                  '(BODY.PEEK[])')
                     fails_left = 0
-                except imapobj.abort(), e:
+                except imapobj.abort, e:
                     # Release dropped connection, and get a new one
-                    self.imapserver.releaseconnection(imapobj)
+                    self.imapserver.releaseconnection(imapobj, True)
                     imapobj = self.imapserver.acquireconnection()
                     self.ui.error(e, exc_info()[2])
                     fails_left -= 1
@@ -495,11 +493,10 @@ class IMAPFolder(BaseFolder):
             self.savemessageflags(uid, flags)
             return uid
 
+        retry_left = 2 # succeeded in APPENDING?
         imapobj = self.imapserver.acquireconnection()
         try:
-            success = False # succeeded in APPENDING?
+            while retry_left:
-            while not success:
-
                 # UIDPLUS extension provides us with an APPENDUID response.
                 use_uidplus = 'UIDPLUS' in imapobj.capabilities
 
@@ -536,20 +533,26 @@ class IMAPFolder(BaseFolder):
                     (typ, dat) = imapobj.append(self.getfullname(),
                                        imaputil.flagsmaildir2imap(flags),
                                        date, content)
-                    success = True
+                    retry_left = 0 # Mark as success
                 except imapobj.abort, e:
                     # connection has been reset, release connection and retry.
-                    self.ui.error(e, exc_info()[2])
+                    retry_left -= 1
                     self.imapserver.releaseconnection(imapobj, True)
                     imapobj = self.imapserver.acquireconnection()
-                except imapobj.error, e:
+                    if not retry_left:
-                    # If the server responds with 'BAD', append() raise()s directly.
+                        raise OfflineImapError("Saving msg in folder '%s', "
-                    # So we need to prepare a response ourselves.
+                              "repository '%s' failed. Server reponded: %s\n"
-                    typ, dat = 'BAD', str(e)
+                              "Message content was: %s" %
-                    if typ != 'OK': #APPEND failed
+                              (self, self.getrepository(), str(e), dbg_output),
-                        raise OfflineImapError("Saving msg in folder '%s', repository "
+                                               OfflineImapError.ERROR.MESSAGE)
-                                               "'%s' failed. Server reponded; %s %s\nMessage content was:"
+                    self.ui.error(e, exc_info()[2])
-                                               " %s" % (self, self.getrepository(), typ, dat, dbg_output),
+
+                except imapobj.error, e: # APPEND failed
+                    # If the server responds with 'BAD', append()
+                    # raise()s directly.  So we catch that too.
+                    raise OfflineImapError("Saving msg folder '%s', repo '%s'"
+                        "failed. Server reponded: %s\nMessage content was: "
+                        "%s" % (self, self.getrepository(), str(e), dbg_output),
                                            OfflineImapError.ERROR.MESSAGE)
             # Checkpoint. Let it write out stuff, etc. Eg searches for
             # just uploaded messages won't work if we don't do this.

offlineimap/folder/LocalStatus.py

@@ -26,22 +26,15 @@ except NameError:
 magicline = "OFFLINEIMAP LocalStatus CACHE DATA - DO NOT MODIFY - FORMAT 1"
 
 class LocalStatusFolder(BaseFolder):
-    def __init__(self, root, name, repository, accountname, config):
+    def __init__(self, name, repository):
-        self.name = name
+        super(LocalStatusFolder, self).__init__(name, repository)
-        self.root = root
         self.sep = '.'
-        self.config = config
+        self.filename = os.path.join(self.getroot(), self.getfolderbasename())
-        self.filename = os.path.join(root, self.getfolderbasename())
         self.messagelist = {}
-        self.repository = repository
         self.savelock = threading.Lock()
-        self.doautosave = config.getdefaultboolean("general", "fsync", False)
+        self.doautosave = self.config.getdefaultboolean("general", "fsync",
+                                                        False)
         """Should we perform fsyncs as often as possible?"""
-        self.accountname = accountname
-        super(LocalStatusFolder, self).__init__()
-
-    def getaccountname(self):
-        return self.accountname
 
     def storesmessages(self):
         return 0
@@ -53,7 +46,7 @@ class LocalStatusFolder(BaseFolder):
         return self.name
 
     def getroot(self):
-        return self.root
+        return self.repository.root
 
     def getsep(self):
         return self.sep

offlineimap/folder/LocalStatusSQLite.py

@@ -46,12 +46,8 @@ class LocalStatusSQLiteFolder(LocalStatusFolder):
     #current version of our db format
     cur_version = 1
 
-    def __init__(self, root, name, repository, accountname, config):
+    def __init__(self, name, repository):
-        super(LocalStatusSQLiteFolder, self).__init__(root, name, 
+        super(LocalStatusSQLiteFolder, self).__init__(name, repository)       
-                                                      repository, 
-                                                      accountname,
-                                                      config)       
-
         # dblock protects against concurrent writes in same connection
         self._dblock = Lock()
         #Try to establish connection, no need for threadsafety in __init__

offlineimap/folder/Maildir.py

@@ -58,34 +58,23 @@ def gettimeseq():
         timelock.release()
 
 class MaildirFolder(BaseFolder):
-    def __init__(self, root, name, sep, repository, accountname, config):
+    def __init__(self, root, name, sep, repository):
-        self.name = name
+        super(MaildirFolder, self).__init__(name, repository)
-        self.config = config
+        self.dofsync = self.config.getdefaultboolean("general", "fsync", True)
-        self.dofsync = config.getdefaultboolean("general", "fsync", True)
         self.root = root
         self.sep = sep
         self.messagelist = None
-        self.repository = repository
-        self.accountname = accountname
 
         self.wincompatible = self.config.getdefaultboolean(
             "Account "+self.accountname, "maildir-windows-compatible", False)
 
-        if self.wincompatible == False:
+        self.infosep = '!' if self.wincompatible else ':'
-            self.infosep = ':'
+        """infosep is the separator between maildir name and flag appendix"""
-        else:
-            self.infosep = '!'
-
         self.flagmatchre = re.compile(self.infosep + '.*2,([A-Z]+)')
-
-        BaseFolder.__init__(self)
         #self.ui is set in BaseFolder.init()
         # Cache the full folder path, as we use getfullname() very often
         self._fullname = os.path.join(self.getroot(), self.getname())
 
-    def getaccountname(self):
-        return self.accountname
-
     def getfullname(self):
         """Return the absolute file path to the Maildir folder (sans cur|new)"""
         return self._fullname
@@ -176,10 +165,8 @@ class MaildirFolder(BaseFolder):
                 flags = set(flagmatch.group(1))
             else:
                 flags = set()
-            # 'filename' is 'dirannex/filename', e.g. cur/123_U=1_FMD5=1:2,S
+            # 'filename' is 'dirannex/filename', e.g. cur/123,U=1,FMD5=1:2,S
-            retval[uid] = {'uid': uid,
+            retval[uid] = {'flags': flags, 'filename': file}
-                           'flags': flags,
-                           'filename': file}
         return retval
 
     def quickchanged(self, statusfolder):
@@ -216,8 +203,7 @@ class MaildirFolder(BaseFolder):
     def getmessagetime(self, uid):
         filename = self.messagelist[uid]['filename']
         filepath = os.path.join(self.getfullname(), filename)
-        st = os.stat(filepath)
+        return os.path.getmtime(filepath)
-        return st.st_mtime
 
     def savemessage(self, uid, content, flags, rtime):
         # This function only ever saves to tmp/,
@@ -246,7 +232,7 @@ class MaildirFolder(BaseFolder):
         # open file and write it out
         try:
             fd = os.open(os.path.join(tmpdir, messagename),
-                           os.O_EXCL|os.O_CREAT|os.O_WRONLY)
+                           os.O_EXCL|os.O_CREAT|os.O_WRONLY, 0666)
         except OSError, e:
             if e.errno == 17: 
                 #FILE EXISTS ALREADY
@@ -267,7 +253,7 @@ class MaildirFolder(BaseFolder):
         if rtime != None:
             os.utime(os.path.join(tmpdir, messagename), (rtime, rtime))
 
-        self.messagelist[uid] = {'uid': uid, 'flags': set(),
+        self.messagelist[uid] = {'flags': set(),
                                  'filename': os.path.join('tmp', messagename)}
         # savemessageflags moves msg to 'cur' or 'new' as appropriate
         self.savemessageflags(uid, flags)

offlineimap/imapserver.py

@@ -1,6 +1,5 @@
 # IMAP server support
-# Copyright (C) 2002 - 2007 John Goerzen
+# Copyright (C) 2002 - 2011 John Goerzen & contributors
-# <jgoerzen@complete.org>
 #
 #    This program is free software; you can redistribute it and/or modify
 #    it under the terms of the GNU General Public License as published by
@@ -56,11 +55,11 @@ class IMAPServer:
         self.config = repos.getconfig()
         self.tunnel = repos.getpreauthtunnel()
         self.usessl = repos.getssl()
-        self.username = repos.getuser()
+        self.username = None if self.tunnel else repos.getuser()
         self.password = None
         self.passworderror = None
         self.goodpassword = None
-        self.hostname = repos.gethost()
+        self.hostname = None if self.tunnel else repos.gethost()
         self.port = repos.getport()
         if self.port == None:
             self.port = 993 if self.usessl else 143
@@ -262,6 +261,12 @@ class IMAPServer:
                                 except imapobj.error, val:
                                     self.plainauth(imapobj)
                             else:
+                                # Use plaintext login, unless
+                                # LOGINDISABLED (RFC2595)
+                                if 'LOGINDISABLED' in imapobj.capabilities:
+                                    raise OfflineImapError("Plaintext login "
+                                       "disabled by server. Need to use SSL?",
+                                        OfflineImapError.ERROR.REPO)
                                 self.plainauth(imapobj)
                         # Would bail by here if there was a failure.
                         success = 1
@@ -270,6 +275,11 @@ class IMAPServer:
                         self.passworderror = str(val)
                         raise
 
+            # update capabilities after login, e.g. gmail serves different ones
+            typ, dat = imapobj.capability()
+            if dat != [None]:
+                imapobj.capabilities = tuple(dat[-1].upper().split())
+
             if self.delim == None:
                 listres = imapobj.list(self.reference, '""')[1]
                 if listres == [None] or listres == None:
@@ -539,7 +549,7 @@ class IdleThread(object):
             try:
                 # End IDLE mode with noop, imapobj can point to a dropped conn.
                 imapobj.noop()
-            except imapobj.abort():
+            except imapobj.abort:
                 self.ui.warn('Attempting NOOP on dropped connection %s' % \
                                  imapobj.identifier)
                 self.parent.releaseconnection(imapobj, True)

offlineimap/imaputil.py

@@ -34,16 +34,15 @@ def debug(*args):
     getglobalui().debug('imap', " ".join(msg))
 
 def dequote(string):
-    """Takes a string which may or may not be quoted and returns it, unquoted.
+    """Takes string which may or may not be quoted and unquotes it.
-    This function does NOT consider parenthised lists to be quoted.
-    """
 
-    if not (string[0] == '"' and string[-1] == '"'):
+    It only considers double quotes. This function does NOT consider
-        return string
+    parenthised lists to be quoted.
-    string = string[1:-1]               # Strip off quotes.
+    """
+    if string and string.startswith('"') and string.endswith('"'):
+        string = string[1:-1]  # Strip off the surrounding quotes.
         string = string.replace('\\"', '"')
         string = string.replace('\\\\', '\\')
-    debug("dequote() returning:", string)
     return string
 
 def flagsplit(string):

offlineimap/init.py

@@ -24,20 +24,17 @@ import signal
 import socket
 import logging
 from optparse import OptionParser
+try:
+    import fcntl
+except ImportError:
+    pass #it's OK
 import offlineimap
 from offlineimap import accounts, threadutil, syncmaster
+from offlineimap.error import OfflineImapError
 from offlineimap.ui import UI_LIST, setglobalui, getglobalui
 from offlineimap.CustomConfig import CustomConfigParser
 
 
-try:
-    import fcntl
-    hasfcntl = 1
-except:
-    hasfcntl = 0
-
-lockfd = None
-
 class OfflineImap:
     """The main class that encapsulates the high level use of OfflineImap.
 
@@ -46,17 +43,6 @@ class OfflineImap:
       oi = OfflineImap()
       oi.run()
     """
-    def lock(self, config, ui):
-        global lockfd, hasfcntl
-        if not hasfcntl:
-            return
-        lockfd = open(config.getmetadatadir() + "/lock", "w")
-        try:
-            fcntl.flock(lockfd, fcntl.LOCK_EX | fcntl.LOCK_NB)
-        except IOError:
-            ui.locked()
-            ui.terminate(1)
-    
     def run(self):
         """Parse the commandline and invoke everything"""
 
@@ -253,7 +239,6 @@ class OfflineImap:
                     config.set(section, "folderfilter", folderfilter)
                     config.set(section, "folderincludes", folderincludes)
 
-        self.lock(config, ui)
         self.config = config
     
         def sigterm_handler(signum, frame):
@@ -330,6 +315,18 @@ class OfflineImap:
             #various initializations that need to be performed:
             offlineimap.mbnames.init(config, syncaccounts)
 
+            #TODO: keep legacy lock for a few versions, then remove.
+            self._legacy_lock = open(self.config.getmetadatadir() + "/lock",
+                                       'w')
+            try:
+                fcntl.lockf(self._legacy_lock, fcntl.LOCK_EX|fcntl.LOCK_NB)
+            except NameError:
+                #fcntl not available (Windows), disable file locking... :(
+                pass
+            except IOError:
+                raise OfflineImapError("Could not take global lock.",
+                                       OfflineImapError.ERROR.REPO)
+
             if options.singlethreading:
                 #singlethreaded
                 self.sync_singlethreaded(syncaccounts, config)
@@ -349,8 +346,9 @@ class OfflineImap:
             return
         except (SystemExit):
             raise
-        except:
+        except Exception, e:
-            ui.mainException()
+            ui.error(e)
+            ui.terminate()
 
     def sync_singlethreaded(self, accs, config):
         """Executed if we do not want a separate syncmaster thread

offlineimap/repository/Base.py

@@ -16,19 +16,23 @@
 #    along with this program; if not, write to the Free Software
 #    Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301 USA
 
+import re
 import os.path
 import traceback
+from sys import exc_info
 from offlineimap import CustomConfig
 from offlineimap.ui import getglobalui
+from offlineimap.error import OfflineImapError
 
 class BaseRepository(object, CustomConfig.ConfigHelperMixin):
+
     def __init__(self, reposname, account):
         self.ui = getglobalui()
         self.account = account
         self.config = account.getconfig()
         self.name = reposname
         self.localeval = account.getlocaleval()
-        self.accountname = self.account.getname()
+        self._accountname = self.account.getname()
         self.uiddir = os.path.join(self.config.getmetadatadir(), 'Repository-' + self.name)
         if not os.path.exists(self.uiddir):
             os.mkdir(self.uiddir, 0700)
@@ -39,17 +43,30 @@ class BaseRepository(object, CustomConfig.ConfigHelperMixin):
         if not os.path.exists(self.uiddir):
             os.mkdir(self.uiddir, 0700)
 
-    # The 'restoreatime' config parameter only applies to local Maildir
+        self.nametrans = lambda foldername: foldername
-    # mailboxes.
+        self.folderfilter = lambda foldername: 1
+        self.folderincludes = []
+        self.foldersort = cmp
+        if self.config.has_option(self.getsection(), 'nametrans'):
+            self.nametrans = self.localeval.eval(
+                self.getconf('nametrans'), {'re': re})
+        if self.config.has_option(self.getsection(), 'folderfilter'):
+            self.folderfilter = self.localeval.eval(
+                self.getconf('folderfilter'), {'re': re})
+        if self.config.has_option(self.getsection(), 'folderincludes'):
+            self.folderincludes = self.localeval.eval(
+                self.getconf('folderincludes'), {'re': re})
+        if self.config.has_option(self.getsection(), 'foldersort'):
+            self.foldersort = self.localeval.eval(
+                self.getconf('foldersort'), {'re': re})
+
     def restore_atime(self):
-        if self.config.get('Repository ' + self.name, 'type').strip() != \
+        """Sets folders' atime back to their values after a sync
-                'Maildir':
-            return
 
-        if not self.config.has_option('Repository ' + self.name, 'restoreatime') or not self.config.getboolean('Repository ' + self.name, 'restoreatime'):
+        Controlled by the 'restoreatime' config parameter (default
-            return
+        False), applies only to local Maildir mailboxes and does nothing
-
+        on all other repository types."""
-        return self.restore_folder_atimes()
+        pass
 
     def connect(self):
         """Establish a connection to the remote, if necessary.  This exists
@@ -75,15 +92,17 @@ class BaseRepository(object, CustomConfig.ConfigHelperMixin):
     def __str__(self):
         return self.name
 
+    @property
+    def accountname(self):
+        """Account name as string"""
+        return self._accountname
+
     def getuiddir(self):
         return self.uiddir
 
     def getmapdir(self):
         return self.mapdir
 
-    def getaccountname(self):
-        return self.accountname
-
     def getsection(self):
         return 'Repository ' + self.name
 
@@ -117,7 +136,11 @@ class BaseRepository(object, CustomConfig.ConfigHelperMixin):
     def syncfoldersto(self, dst_repo, status_repo):
         """Syncs the folders in this repository to those in dest.
 
-        It does NOT sync the contents of those folders."""
+        It does NOT sync the contents of those folders. nametrans rules
+        in both directions will be honored, but there are NO checks yet
+        that forward and backward nametrans actually match up!
+        Configuring nametrans on BOTH repositories therefore could lead
+        to infinite folder creation cycles."""
         src_repo = self
         src_folders = src_repo.getfolders()
         dst_folders = dst_repo.getfolders()
@@ -130,33 +153,66 @@ class BaseRepository(object, CustomConfig.ConfigHelperMixin):
                     src_repo.getsep(), dst_repo.getsep())] = folder
         dst_hash = {}
         for folder in dst_folders:
-            dst_hash[folder.getvisiblename()] = folder
+            dst_hash[folder.name] = folder
 
-        #
+        # Find new folders on src_repo.
-        # Find new folders.
+        for src_name, src_folder in src_hash.iteritems():
-        for key in src_hash.keys():
+            if src_folder.sync_this and not src_name in dst_hash:
-            if not key in dst_hash:
                 try:
-                    dst_repo.makefolder(key)
+                    dst_repo.makefolder(src_name)
-                    status_repo.makefolder(key.replace(dst_repo.getsep(),
+                except OfflineImapError, e:
-                                                      status_repo.getsep()))
+                    self.ui.error(e, exc_info()[2],
-                except (KeyboardInterrupt):
+                                  "Creating folder %s on repository %s" %\
+                                      (src_name, dst_repo))
                     raise
-                except:
+                status_repo.makefolder(src_name.replace(dst_repo.getsep(),
-                    self.ui.warn("ERROR Attempting to create folder " \
+                                                   status_repo.getsep()))
-                        + key + ":"  +traceback.format_exc())
+        # Find new folders on dst_repo.
+        for dst_name, dst_folder in dst_hash.iteritems():
+            if dst_folder.sync_this and not dst_name in src_hash:
+                # nametrans sanity check!
+                # Does nametrans back&forth lead to identical names?
+                #src_name is the unmodified full src_name that would be created
+                newsrc_name = dst_folder.getvisiblename().replace(
+                    dst_repo.getsep(),
+                    src_repo.getsep())
+                folder = self.getfolder(newsrc_name)
+                # would src repo filter out the new folder name? In this
+                # case don't create it on it:
+                if not self.folderfilter(newsrc_name):
+                    self.ui.debug('', "Not creating folder '%s' (repository '%s"
+                        "') as it would be filtered out on that repository." %
+                                  (newsrc_name, self))
+                    continue
+                # apply reverse nametrans to see if we end up with the same name
+                newdst_name = folder.getvisiblename().replace(
+                    src_repo.getsep(), dst_repo.getsep())
+                if dst_name != newdst_name:
+                    raise OfflineImapError("INFINITE FOLDER CREATION DETECTED! "
+                        "Folder '%s' (repository '%s') would be created as fold"
+                        "er '%s' (repository '%s'). The latter becomes '%s' in "
+                        "return, leading to infinite folder creation cycles.\n "
+                        "SOLUTION: 1) Do set your nametrans rules on both repos"
+                        "itories so they lead to identical names if applied bac"
+                        "k and forth. 2) Use folderfilter settings on a reposit"
+                        "ory to prevent some folders from being created on the "
+                        "other side." % (dst_name, dst_repo, newsrc_name,
+                                         src_repo, newdst_name),
+                                           OfflineImapError.ERROR.REPO)
+                # end sanity check, actually create the folder
 
-        #
+                try:
+                    src_repo.makefolder(newsrc_name)
+                except OfflineImapError, e:
+                    self.ui.error(e, exc_info()[2],
+                                  "Creating folder %s on repository %s" %\
+                                      (src_name, dst_repo))
+                    raise
+                status_repo.makefolder(newsrc_name.replace(
+                                src_repo.getsep(), status_repo.getsep()))
         # Find deleted folders.
-        #
         # We don't delete folders right now.
 
-        #for key in desthash.keys():
-        #    if not key in srchash:
-        #        dest.deletefolder(key)
-        
-    ##### Keepalive
-
     def startkeepalive(self):
         """The default implementation will do nothing."""
         pass

offlineimap/repository/Gmail.py

@@ -59,8 +59,7 @@ class GmailRepository(IMAPRepository):
 
     def getfolder(self, foldername):
         return self.getfoldertype()(self.imapserver, foldername,
-                                    self.nametrans(foldername),
+                                    self)
-                                    self.accountname, self)
 
     def getfoldertype(self):
         return folder.Gmail.GmailFolder

offlineimap/repository/IMAP.py

@@ -21,7 +21,6 @@ from offlineimap import folder, imaputil, imapserver, OfflineImapError
 from offlineimap.folder.UIDMaps import MappedIMAPFolder
 from offlineimap.threadutil import ExitNotifyThread
 from threading import Event
-import re
 import types
 import os
 from sys import exc_info
@@ -36,23 +35,6 @@ class IMAPRepository(BaseRepository):
         self._host = None
         self.imapserver = imapserver.IMAPServer(self)
         self.folders = None
-        self.nametrans = lambda foldername: foldername
-        self.folderfilter = lambda foldername: 1
-        self.folderincludes = []
-        self.foldersort = cmp
-        localeval = self.localeval
-        if self.config.has_option(self.getsection(), 'nametrans'):
-            self.nametrans = localeval.eval(self.getconf('nametrans'),
-                                            {'re': re})
-        if self.config.has_option(self.getsection(), 'folderfilter'):
-            self.folderfilter = localeval.eval(self.getconf('folderfilter'),
-                                               {'re': re})
-        if self.config.has_option(self.getsection(), 'folderincludes'):
-            self.folderincludes = localeval.eval(self.getconf('folderincludes'),
-                                                 {'re': re})
-        if self.config.has_option(self.getsection(), 'foldersort'):
-            self.foldersort = localeval.eval(self.getconf('foldersort'),
-                                             {'re': re})
 
     def startkeepalive(self):
         keepalivetime = self.getkeepalive()
@@ -259,9 +241,7 @@ class IMAPRepository(BaseRepository):
 
 
     def getfolder(self, foldername):
-        return self.getfoldertype()(self.imapserver, foldername,
+        return self.getfoldertype()(self.imapserver, foldername, self)
-                                    self.nametrans(foldername),
-                                    self.accountname, self)
 
     def getfoldertype(self):
         return folder.IMAP.IMAPFolder
@@ -280,8 +260,7 @@ class IMAPRepository(BaseRepository):
         imapobj = self.imapserver.acquireconnection()
         # check whether to list all folders, or subscribed only
         listfunction = imapobj.list
-        if self.config.has_option(self.getsection(), 'subscribedonly'):
+        if self.getconfboolean('subscribedonly', False):
-          if self.getconf('subscribedonly') == "yes":
             listfunction = imapobj.lsub
         try:
             listresult = listfunction(directory = self.imapserver.reference)[1]
@@ -298,13 +277,14 @@ class IMAPRepository(BaseRepository):
             if '\\noselect' in flaglist:
                 continue
             foldername = imaputil.dequote(name)
-            if not self.folderfilter(foldername):
-                self.ui.debug('imap',"Filtering out '%s' due to folderfilter" %\
-                                  foldername)
-                continue
             retval.append(self.getfoldertype()(self.imapserver, foldername,
-                                               self.nametrans(foldername),
+                                               self))
-                                               self.accountname, self))
+            # filter out the folder?
+            if not self.folderfilter(foldername):
+                self.ui.debug('imap', "Filtering out '%s'[%s] due to folderfilt"
+                              "er" % (foldername, self))
+                retval[-1].sync_this = False
+        # Add all folderincludes
         if len(self.folderincludes):
             imapobj = self.imapserver.acquireconnection()
             try:
@@ -320,26 +300,36 @@ class IMAPRepository(BaseRepository):
                         continue
                     retval.append(self.getfoldertype()(self.imapserver,
                                                        foldername,
-                                                       self.nametrans(foldername),
+                                                       self))
-                                                       self.accountname, self))
             finally:
                 self.imapserver.releaseconnection(imapobj)
                 
         retval.sort(lambda x, y: self.foldersort(x.getvisiblename(), y.getvisiblename()))
         self.folders = retval
-        return retval
+        return self.folders
 
     def makefolder(self, foldername):
+        """Create a folder on the IMAP server
+
+        :param foldername: Full path of the folder to be created."""
+        #TODO: IMHO this existing commented out code is correct and
+        #should be enabled, but this would change the behavior for
+        #existing configurations who have a 'reference' set on a Mapped
+        #IMAP server....:
         #if self.getreference() != '""':
         #    newname = self.getreference() + self.getsep() + foldername
         #else:
         #    newname = foldername
-        newname = foldername
         imapobj = self.imapserver.acquireconnection()
         try:
-            result = imapobj.create(newname)
+            self.ui._msg("Creating new IMAP folder '%s' on server %s" %\
+                              (foldername, self))
+            result = imapobj.create(foldername)
             if result[0] != 'OK':
-                raise RuntimeError, "Repository %s could not create folder %s: %s" % (self.getname(), foldername, str(result))
+                raise OfflineImapError("Folder '%s'[%s] could not be created. "
+                                       "Server responded: %s" % \
+                                           (foldername, self, str(result)),
+                                       OfflineImapError.ERROR.FOLDER)
         finally:
             self.imapserver.releaseconnection(imapobj)
             

offlineimap/repository/LocalStatus.py

@@ -25,14 +25,14 @@ import re
 class LocalStatusRepository(BaseRepository):
     def __init__(self, reposname, account):
         BaseRepository.__init__(self, reposname, account)
-        self.directory = os.path.join(account.getaccountmeta(), 'LocalStatus')
+        # Root directory in which the LocalStatus folders reside
-
+        self.root = os.path.join(account.getaccountmeta(), 'LocalStatus')
         # statusbackend can be 'plain' or 'sqlite'
         backend = self.account.getconf('status_backend', 'plain')
         if backend == 'sqlite':
             self._backend = 'sqlite'
             self.LocalStatusFolderClass = LocalStatusSQLiteFolder
-            self.directory += '-sqlite'
+            self.root += '-sqlite'
         elif backend == 'plain':
             self._backend = 'plain'
             self.LocalStatusFolderClass = LocalStatusFolder
@@ -40,8 +40,8 @@ class LocalStatusRepository(BaseRepository):
             raise SyntaxWarning("Unknown status_backend '%s' for account '%s'" \
                                 % (backend, account.name))
 
-        if not os.path.exists(self.directory):
+        if not os.path.exists(self.root):
-            os.mkdir(self.directory, 0700)
+            os.mkdir(self.root, 0700)
 
         # self._folders is a list of LocalStatusFolders()
         self._folders = None
@@ -60,7 +60,7 @@ class LocalStatusRepository(BaseRepository):
         # replace with literal 'dot' if final path name is '.' as '.' is
         # an invalid file name.
         basename = re.sub('(^|\/)\.$','\\1dot', basename)
-        return os.path.join(self.directory, basename)
+        return os.path.join(self.root, basename)
 
     def makefolder(self, foldername):
         """Create a LocalStatus Folder
@@ -82,9 +82,7 @@ class LocalStatusRepository(BaseRepository):
 
     def getfolder(self, foldername):
         """Return the Folder() object for a foldername"""
-        return self.LocalStatusFolderClass(self.directory, foldername,
+        return self.LocalStatusFolderClass(foldername, self)
-                                           self, self.accountname,
-                                           self.config)
 
     def getfolders(self):
         """Returns a list of all cached folders."""
@@ -92,7 +90,7 @@ class LocalStatusRepository(BaseRepository):
             return self._folders
 
         self._folders = []
-        for folder in os.listdir(self.directory):
+        for folder in os.listdir(self.root):
             self._folders.append(self.getfolder(folder))
         return self._folders
 

offlineimap/repository/Maildir.py

@@ -19,6 +19,7 @@
 from Base import BaseRepository
 from offlineimap import folder
 from offlineimap.ui import getglobalui
+from offlineimap.error import OfflineImapError
 import os
 from stat import *
 
@@ -39,21 +40,25 @@ class MaildirRepository(BaseRepository):
             os.mkdir(self.root, 0700)
 
     def _append_folder_atimes(self, foldername):
+        """Store the atimes of a folder's new|cur in self.folder_atimes"""
         p = os.path.join(self.root, foldername)
         new = os.path.join(p, 'new')
         cur = os.path.join(p, 'cur')
-        f = p, os.stat(new)[ST_ATIME], os.stat(cur)[ST_ATIME]
+        atimes = (p, os.path.getatime(new), os.path.getatime(cur))
-        self.folder_atimes.append(f)
+        self.folder_atimes.append(atimes)
 
-    def restore_folder_atimes(self):
+    def restore_atime(self):
-        if not self.folder_atimes:
+        """Sets folders' atime back to their values after a sync
-            return
 
-        for f in self.folder_atimes:
+        Controlled by the 'restoreatime' config parameter."""
-            t = f[1], os.stat(os.path.join(f[0], 'new'))[ST_MTIME]
+        if not self.getconfboolean('restoreatime', False):
-            os.utime(os.path.join(f[0], 'new'), t)
+            return # not configured to restore
-            t = f[2], os.stat(os.path.join(f[0], 'cur'))[ST_MTIME]
+
-            os.utime(os.path.join(f[0], 'cur'), t)
+        for (dirpath, new_atime, cur_atime) in self.folder_atimes:
+            new_dir = os.path.join(dirpath, 'new')
+            cur_dir = os.path.join(dirpath, 'cur')
+            os.utime(new_dir, (new_atime, os.path.getmtime(new_dir)))
+            os.utime(cur_dir, (cur_atime, os.path.getmtime(cur_dir)))
 
     def getlocalroot(self):
         return os.path.expanduser(self.getconf('localfolders'))
@@ -110,11 +115,19 @@ class MaildirRepository(BaseRepository):
         self.ui.warn("NOT YET IMPLEMENTED: DELETE FOLDER %s" % foldername)
 
     def getfolder(self, foldername):
-        if self.config.has_option('Repository ' + self.name, 'restoreatime') and self.config.getboolean('Repository ' + self.name, 'restoreatime'):
+        """Return a Folder instance of this Maildir
-            self._append_folder_atimes(foldername)
+
-        return folder.Maildir.MaildirFolder(self.root, foldername,
+        If necessary, scan and cache all foldernames to make sure that
-                                            self.getsep(), self, 
+        we only return existing folders and that 2 calls with the same
-                                            self.accountname, self.config)
+        name will return the same object."""
+        # getfolders() will scan and cache the values *if* necessary
+        folders = self.getfolders()
+        for folder in folders:
+            if foldername == folder.name:
+                return folder
+        raise OfflineImapError("getfolder() asked for a nonexisting "
+                               "folder '%s'." % foldername,
+                               OfflineImapError.ERROR.FOLDER)
 
     def _getfolders_scandir(self, root, extension = None):
         """Recursively scan folder 'root'; return a list of MailDirFolder
@@ -133,7 +146,7 @@ class MaildirRepository(BaseRepository):
         self.debug("  toppath = %s" % toppath)
 
         # Iterate over directories in top & top itself.
-        for dirname in os.listdir(toppath) + ['.']:
+        for dirname in os.listdir(toppath) + ['']:
             self.debug("  *** top of loop")
             self.debug("  dirname = %s" % dirname)
             if dirname in ['cur', 'new', 'tmp']:
@@ -154,19 +167,19 @@ class MaildirRepository(BaseRepository):
                 os.path.isdir(os.path.join(fullname, 'tmp'))):
                 # This directory has maildir stuff -- process
                 self.debug("  This is maildir folder '%s'." % foldername)
-
+                if self.getconfboolean('restoreatime', False):
-                if self.config.has_option('Repository %s' % self,
-                                          'restoreatime') and \
-                    self.config.getboolean('Repository %s' % self,
-                                           'restoreatime'):
                     self._append_folder_atimes(foldername)
                 retval.append(folder.Maildir.MaildirFolder(self.root,
                                                            foldername,
                                                            self.getsep(),
-                                                           self,
+                                                           self))
-                                                           self.accountname,
+                # filter out the folder?
-                                                           self.config))
+                if not self.folderfilter(foldername):
-            if self.getsep() == '/' and dirname != '.':
+                    self.debug("Filtering out '%s'[%s] due to folderfilt"
+                               "er" % (foldername, self))
+                    retval[-1].sync_this = False
+
+            if self.getsep() == '/' and dirname != '':
                 # Recursively check sub-directories for folders too.
                 retval.extend(self._getfolders_scandir(root, foldername))
         self.debug("_GETFOLDERS_SCANDIR RETURNING %s" % \

offlineimap/threadutil.py

@@ -214,8 +214,7 @@ def initInstanceLimit(instancename, instancemax):
 class InstanceLimitedThread(ExitNotifyThread):
     def __init__(self, instancename, *args, **kwargs):
         self.instancename = instancename
-                                                   
+        super(InstanceLimitedThread, self).__init__(*args, **kwargs)
-        apply(ExitNotifyThread.__init__, (self,) + args, kwargs)
 
     def start(self):
         instancelimitedsems[self.instancename].acquire()

offlineimap/ui/Curses.py

@@ -66,7 +66,7 @@ class CursesUtil:
         """Perform an operation with full locking."""
         self.lock()
         try:
-            apply(target, args, kwargs)
+            target(*args, **kwargs)
         finally:
             self.unlock()
 

offlineimap/ui/UIBase.py

@@ -104,11 +104,10 @@ class UIBase:
            ui.error(exc, sys.exc_info()[2], msg="While syncing Folder %s in "
                                                 "repo %s")
         """
-        cur_thread = threading.currentThread()
         if msg:
-            self._msg("ERROR [%s]: %s\n  %s" % (cur_thread, msg, exc))
+            self._msg("ERROR: %s\n  %s" % (msg, exc))
         else:
-            self._msg("ERROR [%s]: %s" % (cur_thread, exc))
+            self._msg("ERROR: %s" % (exc))
 
         if not self.debuglist:
             # only output tracebacks in debug mode
@@ -344,14 +343,6 @@ class UIBase:
         s.delThreadDebugLog(thread)
         s.terminate(100)
 
-    def getMainExceptionString(s):
-        return "Main program terminated with exception:\n%s\n" %\
-               traceback.format_exc() + \
-               s.getThreadDebugLog(threading.currentThread())
-
-    def mainException(s):
-        s._msg(s.getMainExceptionString())
-
     def terminate(self, exitstatus = 0, errortitle = None, errormsg = None):
         """Called to terminate the application."""
         #print any exceptions that have occurred over the run