Merge branch 'next'

2014-07-08 11:45:04 +04:00 · 2014-07-08 11:45:04 +04:00 · d786fdb492
commit d786fdb492
parent 6bd76fed5a ffd1b1d691
4 changed files with 118 additions and 85 deletions
--- a/Changelog.rst
+++ b/Changelog.rst
@ -15,6 +15,8 @@ OfflineIMAP v6.5.6.1 (YYYY-MM-DD)
 * Add missing version bump for 6.5.6 (it was released with
  6.5.5 in setup.py and other places).

+* Various fixes in documentation.
+

 OfflineIMAP v6.5.6 (2014-05-14)
 ===============================
--- a/README.md
+++ b/README.md
@ -39,18 +39,18 @@ detailed information on how to install and configure OfflineImap.
 Quick Start
 ===========

-First, install OfflineIMAP. See docs/INSTALL.rst or read
-http://docs.offlineimap.org/en/latest/INSTALL.html.
-(hint: `sudo python setup.py install`)
+First, install OfflineIMAP. See `docs/INSTALL.rst` or read
+<http://docs.offlineimap.org/en/latest/INSTALL.html>
+(hint: `sudo python setup.py install`).

 Second, set up your configuration file and run it! The distribution
 includes offlineimap.conf.minimal (Debian users may find this at
-``/usr/share/doc/offlineimap/examples/offlineimap.conf.minimal``) that
+`/usr/share/doc/offlineimap/examples/offlineimap.conf.minimal`) that
 provides you with the bare minimum of setting up OfflineIMAP.  You can
 simply copy this file into your home directory and name it
-``.offlineimaprc``.  A command such as ``cp offlineimap.conf.minimal
-~/.offlineimaprc`` will do it.  Or, if you prefer, you can just copy
-this text to ``~/.offlineimaprc``:
+`.offlineimaprc`.  A command such as `cp offlineimap.conf.minimal
+~/.offlineimaprc` will do it.  Or, if you prefer, you can just copy
+this text to `~/.offlineimaprc`:

    [general]
    accounts = Test
@ -69,17 +69,17 @@ this text to ``~/.offlineimaprc``:
    remoteuser = jgoerzen


-Now, edit the ``~/.offlineimaprc`` file with your favorite editor.  All you have
-to do is specify a directory for your folders to be in (on the localfolders
-line), the host name of your IMAP server (on the remotehost line), and your
-login name on the remote (on the remoteuser line).  That's it!
+Now, edit the `~/.offlineimaprc` file with your favorite editor.  All you have
+to do is specify a directory for your folders to be in (on the `localfolders`
+line), the host name of your IMAP server (on the `remotehost` line), and your
+login name on the remote (on the `remoteuser` line).  That's it!

-If you prefer to be XDG-compatible,
-  http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html
-then substitute the above ``~/.offlineimaprc'' with
-``$XDG\_CONFIG\_HOME/offlineimap/config'' and don't forget to set
-XDG\_CONFIG\_HOME properly if you want it to be different from
-the default ``$HOME/.config'' for any reason.
+If you prefer to be compatible with the [XDG Base Directory
+spec](http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html),
+then substitute the above `~/.offlineimaprc` with
+`$XDG_CONFIG_HOME/offlineimap/config` and don't forget to set
+`XDG_CONFIG_HOME` properly if you want it to be different from
+the default `$HOME/.config` for any reason.

 To run OfflineIMAP, you just have to say `offlineimap` ― it will fire
 up, ask you for a login password if necessary, synchronize your folders,
@ -97,12 +97,12 @@ Mailing list & bug reporting

 The user discussion, development and all exciting stuff take place in the
 OfflineImap mailing list at
-http://lists.alioth.debian.org/mailman/listinfo/offlineimap-project. You do not
+<http://lists.alioth.debian.org/mailman/listinfo/offlineimap-project>. You do not
 need to subscribe to send emails.

 Bugs, issues and contributions should be reported to the mailing list. Bugs can
 also be reported in the issue tracker at
-https://github.com/OfflineIMAP/offlineimap/issues.
+<https://github.com/OfflineIMAP/offlineimap/issues>.

 Configuration Examples
 ======================
@ -117,22 +117,22 @@ Multiple Accounts with Mutt
 This example shows you how to set up OfflineIMAP to synchronize multiple
 accounts with the mutt mail reader.

-Start by creating a directory to hold your folders by running ``mkdir ~/Mail``.
-Then, in your ``~/.offlineimaprc``, specify:
+Start by creating a directory to hold your folders by running `mkdir ~/Mail`.
+Then, in your `~/.offlineimaprc`, specify:

    accounts = Personal, Work


-Make sure that you have both an [Account Personal] and an [Account Work]
-section.  The local repository for each account must have different localfolder
-path names.  Also, make sure to enable [mbnames].
+Make sure that you have both an `[Account Personal]` and an `[Account Work]`
+section.  The local repository for each account must have different `localfolder`
+path names.  Also, make sure to enable `[mbnames]`.

 In each local repository section, write something like this:

    localfolders = ~/Mail/Personal


-Finally, add these lines to your ``~/.muttrc``:
+Finally, add these lines to your `~/.muttrc`:

    source ~/path-to-mbnames-muttrc-mailboxes
    folder-hook Personal set from="youremail@personal.com"
@ -149,10 +149,10 @@ UW-IMAPD and References
 -----------------------

 Some users with a UW-IMAPD server need to use OfflineIMAP's "reference" feature
-to get at their mailboxes, specifying a reference of ``~/Mail`` or ``#mh/``
+to get at their mailboxes, specifying a reference of `~/Mail` or `#mh/`
 depending on the configuration.  The below configuration from (originally from
-docwhat@gerf.org) shows using a reference of Mail, a nametrans that strips the
-leading Mail/ off incoming folder names, and a folderfilter that limits the
+docwhat@gerf.org) shows using a reference of Mail, a `nametrans` that strips the
+leading `Mail/` off incoming folder names, and a `folderfilter` that limits the
 folders synced to just three:

    [Account Gerf]
@ -191,16 +191,16 @@ configuration file options that are Python expressions.  This example is based
 on one supplied by Tommi Virtanen for this feature.


-In ~/.offlineimaprc, he adds these options:
+In `~/.offlineimaprc`, he adds these options:

    [general]
    pythonfile=~/.offlineimap.py
    [Repository foo]
    foldersort=mycmp

-Then, the ~/.offlineimap.py file will contain:
+Then, the `~/.offlineimap.py` file will contain:

-  prioritized = ['INBOX', 'personal', 'announce', 'list']
+    prioritized = ['INBOX', 'personal', 'announce', 'list']

    def mycmp(x, y):
        for prefix in prioritized:
@ -221,5 +221,5 @@ Then, the ~/.offlineimap.py file will contain:
        print folders


-This code snippet illustrates how the foldersort option can be customized with a
-Python function from the pythonfile to always synchronize certain folders first.
+This code snippet illustrates how the `foldersort` option can be customized with a
+Python function from the `pythonfile` to always synchronize certain folders first.
--- a/offlineimap/folder/Base.py
+++ b/offlineimap/folder/Base.py
@ -386,69 +386,93 @@ class BaseFolder(object):
        for uid in uidlist:
            self.deletemessagelabels(uid, labels)

-
-    """
-    Illustration of all cases for addmessageheader().
-    '+' means the added contents.
-
-Case 1: No '\n\n', leading '\n'
-+X-Flying-Pig-Header: i am here\n
-\n
-This is the body\n
-next line\n
-
-Case 2: '\n\n' at position 0
-+X-Flying-Pig-Header: i am here\n
-\n
-\n
-This is the body\n
-next line\n
-
-Case 3: No '\n\n', no leading '\n'
-+X-Flying-Pig-Header: i am here\n
-+\n
-This is the body\n
-next line\n
-
-Case 4: '\n\n' at non-zero position
-Subject: Something wrong with OI\n
-From: some@person.at+\n
-X-Flying-Pig-Header: i am here\n <-- orig '\n'
-\n
-This is the body\n
-next line\n
-
-    """
-
-    def addmessageheader(self, content, crlf, headername, headervalue):
+    def addmessageheader(self, content, linebreak, headername, headervalue):
        """
        Adds new header to the provided message.

        Arguments:
        - content: message content, headers and body as a single string
-        - crlf: string that carries line ending
+        - linebreak: string that carries line ending
        - headername: name of the header to add
        - headervalue: value of the header to add

+        This has to deal with strange corner cases where the header is
+        missing or empty.  Here are illustrations for all the cases,
+        showing where the header gets inserted and what the end result
+        is.  In each illustration, '+' means the added contents.  Note
+        that these examples assume LF for linebreak, not CRLF, so '\n'
+        denotes a linebreak and '\n\n' corresponds to the transition
+        between header and body.  However if the linebreak parameter
+        is set to '\r\n' then you would have to substitute '\r\n' for
+        '\n' in the below examples.
+
+          * Case 1: No '\n\n', leading '\n'
+
+            +X-Flying-Pig-Header: i am here\n
+            \n
+            This is the body\n
+            next line\n
+
+          * Case 2: '\n\n' at position 0
+
+            +X-Flying-Pig-Header: i am here
+            \n
+            \n
+            This is the body\n
+            next line\n
+
+          * Case 3: No '\n\n', no leading '\n'
+
+            +X-Flying-Pig-Header: i am here\n
+            +\n
+            This is the body\n
+            next line\n
+
+          * Case 4: '\n\n' at non-zero position
+
+            Subject: Something wrong with OI\n
+            From: some@person.at
+            +\nX-Flying-Pig-Header: i am here
+            \n
+            \n
+            This is the body\n
+            next line\n
        """
        self.ui.debug('',
                 'addmessageheader: called to add %s: %s' % (headername,
                                                             headervalue))
-        prefix = crlf
-        suffix = ''
-        insertionpoint = content.find(crlf + crlf)
-        if insertionpoint == 0 or insertionpoint == -1:
-            prefix = ''
-            suffix = crlf
+
+        insertionpoint = content.find(linebreak * 2)
        if insertionpoint == -1:
+            self.ui.debug('', 'addmessageheader: headers were missing')
+        else:
+            self.ui.debug('', 'addmessageheader: headers end at position %d' % insertionpoint)
+            mark = '==>EOH<=='
+            contextstart = max(0,            insertionpoint - 100)
+            contextend   = min(len(content), insertionpoint + 100)
+            self.ui.debug('', 'addmessageheader: header/body transition context (marked by %s): %s' %
+                          (mark, repr(content[contextstart:insertionpoint]) + \
+                          mark + repr(content[insertionpoint:contextend])))
+
+        # Hoping for case #4
+        prefix = linebreak
+        suffix = ''
+        # Case #2
+        if insertionpoint == 0:
+            prefix = ''
+            suffix = ''
+        # Either case #1 or #3
+        elif insertionpoint == -1:
+            prefix = ''
+            suffix = linebreak
            insertionpoint = 0
-            # When body starts immediately, without preceding '\n'
+            # Case #3: when body starts immediately, without preceding '\n'
            # (this shouldn't happen with proper mail messages, but
            # we seen many broken ones), we should add '\n' to make
            # new (and the only header, in this case) to be properly
            # separated from the message body.
-            if content[0:len(crlf)] != crlf:
-                suffix = suffix + crlf
+            if content[0:len(linebreak)] != linebreak:
+                suffix = suffix + linebreak

        self.ui.debug('', 'addmessageheader: insertionpoint = %d' % insertionpoint)
        headers = content[0:insertionpoint]
@ -478,6 +502,8 @@ next line\n
    def getmessageheader(self, content, name):
        """
        Searches for the given header and returns its value.
+        Header name is case-insensitive.
+
        Arguments:
        - contents: message itself
        - name: name of the header to be searched
@ -491,7 +517,7 @@ next line\n
        headers = content[0:eoh]
        self.ui.debug('', 'getmessageheader: headers = %s' % repr(headers))

-        m = re.search('^%s:(.*)$' % name, headers, flags = re.MULTILINE)
+        m = re.search('^%s:(.*)$' % name, headers, flags = re.MULTILINE | re.IGNORECASE)
        if m:
            return m.group(1).strip()
        else:
--- a/offlineimap/folder/IMAP.py
+++ b/offlineimap/folder/IMAP.py
@ -517,6 +517,11 @@ class IMAPFolder(BaseFolder):
        # get the date of the message, so we can pass it to the server.
        date = self.__getmessageinternaldate(content, rtime)

+        # Message-ID is handy for debugging messages
+        msg_id = self.getmessageheader(content, "message-id")
+        if not msg_id:
+            msg_id = '[unknown message-id]'
+
        retry_left = 2 # succeeded in APPENDING?
        imapobj = self.imapserver.acquireconnection()
        # NB: in the finally clause for this try we will release
@ -569,9 +574,9 @@ class IMAPFolder(BaseFolder):
                        # In this case, we should immediately abort the repository sync
                        # and continue with the next account.
                        msg = \
-                            "Saving msg in folder '%s', repository '%s' failed (abort). " \
+                            "Saving msg (%s) in folder '%s', repository '%s' failed (abort). " \
                            "Server responded: %s %s\n" % \
-                            (self, self.getrepository(), typ, dat)
+                            (msg_id, self, self.getrepository(), typ, dat)
                        raise OfflineImapError(msg, OfflineImapError.ERROR.REPO)
                    retry_left = 0 # Mark as success
                except imapobj.abort as e:
@ -580,10 +585,10 @@ class IMAPFolder(BaseFolder):
                    self.imapserver.releaseconnection(imapobj, True)
                    imapobj = self.imapserver.acquireconnection()
                    if not retry_left:
-                        raise OfflineImapError("Saving msg in folder '%s', "
+                        raise OfflineImapError("Saving msg (%s) in folder '%s', "
                              "repository '%s' failed (abort). Server responded: %s\n"
                              "Message content was: %s" %
-                              (self, self.getrepository(), str(e), dbg_output),
+                              (msg_id, self, self.getrepository(), str(e), dbg_output),
                                               OfflineImapError.ERROR.MESSAGE)
                    self.ui.error(e, exc_info()[2])
                except imapobj.error as e: # APPEND failed
@ -592,9 +597,9 @@ class IMAPFolder(BaseFolder):
                    # drop conn, it might be bad.
                    self.imapserver.releaseconnection(imapobj, True)
                    imapobj = None
-                    raise OfflineImapError("Saving msg folder '%s', repo '%s'"
+                    raise OfflineImapError("Saving msg (%s) folder '%s', repo '%s'"
                        "failed (error). Server responded: %s\nMessage content was: "
-                        "%s" % (self, self.getrepository(), str(e), dbg_output),
+                        "%s" % (msg_id, self, self.getrepository(), str(e), dbg_output),
                                           OfflineImapError.ERROR.MESSAGE)
            # Checkpoint. Let it write out stuff, etc. Eg searches for
            # just uploaded messages won't work if we don't do this.