Revamped documentation structure and some doc fixes

`make` in the `docs` dir or `make doc` in the root dir will now
create the 1) man page and 2) the user documentation using sphinx
(requiring python-doctools, and sphinx).

The resulting user docs are in `docs/html`. You can also
only create the man pages with `make man` in the `docs` dir.

Also fixed all .rst conversion errors as requested by Gentoo downstream.

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>

.gitignore

@@ -1,9 +1,10 @@
 # Generated files
-/docs/dev-doc/
+/docs/html/
 /build/
 *.pyc
 offlineimap.1
 # backups
 .*.swp
 .*.swo
+*.html
 *~

Changelog.draft.rst

@@ -33,5 +33,11 @@ Changes
 * Revert "* Slight performance enhancement uploading mails to an IMAP
   server in the common case." It might have led to instabilities.
 
+* Revamped documentation structure. `make` in the `docs` dir or `make
+  doc` in the root dir will now create the 1) man page and 2) the user
+  documentation using sphinx (requiring python-doctools, and
+  sphinx). The resulting user docs are in `docs/html`. You can also
+  only create the man pages with `make man` in the `docs` dir.
+
 Bug Fixes
 ---------

Makefile

@@ -46,7 +46,6 @@ man:
 doc:
 	@$(MAKE) -C docs
 	$(RST2HTML) README.rst readme.html
-	$(RST2HTML) SubmittingPatches.rst SubmittingPatches.html
 	$(RST2HTML) Changelog.rst Changelog.html
 
 targz: ../$(TARGZ)

README.rst

@@ -1,26 +1,13 @@
 .. -*- coding: utf-8 -*-
-
 .. _mailing list: http://lists.alioth.debian.org/mailman/listinfo/offlineimap-project
 
 ======
 README
 ======
 
-.. contents::
-.. sectnum::
-
-
 Description
 ===========
 
-Welcome to the official OfflineIMAP project.
-
-*NOTICE:* this software was written by John Goerzen, who retired from
-maintaining. It is now maintained by Nicolas Sebrecht at
-https://github.com/nicolas33/offlineimap. Thanks to John for his great job and
-to have share this project with us.
-
-
 OfflineIMAP is a tool to simplify your e-mail reading. With OfflineIMAP, you can
 read the same mailbox from multiple computers. You get a current copy of your
 messages on each computer, and changes you make one place will be visible on all
@@ -29,75 +16,42 @@ it will appear deleted on your work computer as well. OfflineIMAP is also useful
 if you want to use a mail reader that does not have IMAP support, has poor IMAP
 support, or does not provide disconnected operation.
 
-OfflineIMAP works on pretty much any POSIX operating system, such as Linux, BSD
+OfflineIMAP does not require additional python dependencies (although python-sqlite is strongly recommended)
-operating systems, MacOS X, Solaris, etc.
 
 OfflineIMAP is a Free Software project licensed under the GNU General Public
 License. You can download it for free, and you can modify it. In fact, you are
 encouraged to contribute to OfflineIMAP, and doing so is fast and easy.
 
+Documentation
+=============
 
-OfflineIMAP is FAST; it synchronizes my two accounts with over 50 folders in 3
+The documentation (in .rst format) is included in the `docs`
-seconds.  Other similar tools might take over a minute, and achieve a
+directory. Read it directly or generate nicer html (python-sphinx
-less-reliable result.  Some mail readers can take over 10 minutes to do the same
+needed) or the man page (python-docutils needed) via::
-thing, and some don't even support it at all.  Unlike other mail tools,
-OfflineIMAP features a multi-threaded synchronization algorithm that can
-dramatically speed up performance in many situations by synchronizing several
-different things simultaneously.
 
-OfflineIMAP is FLEXIBLE; you can customize which folders are synced via regular
+    'make doc' (user docs), 'make man' (man page only) or 'make' (both)
-expressions, lists, or Python expressions; a versatile and comprehensive
-configuration file is used to control behavior; two user interfaces are
-built-in; fine-tuning of synchronization performance is possible; internal or
-external automation is supported; SSL and PREAUTH tunnels are both supported;
-offline (or "unplugged") reading is supported; and esoteric IMAP features are
-supported to ensure compatibility with the widest variety of IMAP servers.
 
-OfflineIMAP is SAFE; it uses an algorithm designed to prevent mail loss at all
+    (`make html` will simply create html versions of all *.rst files in /docs)
-costs.  Because of the design of this algorithm, even programming errors should
-not result in loss of mail.  I am so confident in the algorithm that I use my
-own personal and work accounts for testing of OfflineIMAP pre-release,
-development, and beta releases.  Of course, legally speaking, OfflineIMAP comes
-with no warranty, so I am not responsible if this turns out to be wrong.
-
-
-Method of Operation
-===================
-
-OfflineIMAP traditionally operates by maintaining a hierarchy of mail folders in
-Maildir format locally.  Your own mail reader will read mail from this tree, and
-need never know that the mail comes from IMAP.  OfflineIMAP will detect changes
-to the mail folders on your IMAP server and your own computer and
-bi-directionally synchronize them, copying, marking, and deleting messages as
-necessary.
-
-With OfflineIMAP 4.0, a powerful new ability has been introduced ― the program
-can now synchronize two IMAP servers with each other, with no need to have a
-Maildir layer in-between.  Many people use this if they use a mail reader on
-their local machine that does not support Maildirs.  People may install an IMAP
-server on their local machine, and point both OfflineIMAP and their mail reader
-of choice at it.  This is often preferable to the mail reader's own IMAP support
-since OfflineIMAP supports many features (offline reading, for one) that most
-IMAP-aware readers don't.  However, this feature is not as time-tested as
-traditional syncing, so my advice is to stick with normal methods of operation
-for the time being.
 
+The resulting user documentation will be in `docs/html`. The full user
+docs are also at: `<http://docs.offlineimap.org>`_. Please see there for
+detailed information on how to install and configure OfflineImap.
 
 Quick Start
 ===========
 
-If you have already installed OfflineIMAP system-wide, or your system
+First, install OfflineIMAP. See docs/INSTALL.rst or read
-administrator has done that for you, your task for setting up OfflineIMAP for
+`<http://docs.offlineimap.org/en/latest/INSTALL.html>`_. 
-the first time is quite simple.  You just need to set up your configuration
+(hint: `sudo python setup.py install`)
-file, make your folder directory, and run it!
 
-You can quickly set up your configuration file.  The distribution includes a
+Second, set up your configuration file and run it! The distribution
-file offlineimap.conf.minimal (Debian users may find this at
+includes offlineimap.conf.minimal (Debian users may find this at
-``/usr/share/doc/offlineimap/examples/offlineimap.conf.minimal``) that is a
+``/usr/share/doc/offlineimap/examples/offlineimap.conf.minimal``) that
-basic example of setting of OfflineIMAP.  You can simply copy this file into
+provides you with the bare minimum of setting up OfflineIMAP.  You can
-your home directory and name it ``.offlineimaprc`` (note the leading period).  A
+simply copy this file into your home directory and name it
-command such as ``cp offlineimap.conf.minimal ~/.offlineimaprc`` will do it.
+``.offlineimaprc``.  A command such as ``cp offlineimap.conf.minimal
-Or, if you prefer, you can just copy this text to ``~/.offlineimaprc``::
+~/.offlineimaprc`` will do it.  Or, if you prefer, you can just copy
+this text to ``~/.offlineimaprc``::
 
   [general]
   accounts = Test
@@ -121,30 +75,23 @@ to do is specify a directory for your folders to be in (on the localfolders
 line), the host name of your IMAP server (on the remotehost line), and your
 login name on the remote (on the remoteuser line).  That's it!
 
-To run OfflineIMAP, you just have to say offlineimap ― it will fire up, ask you
+To run OfflineIMAP, you just have to say `offlineimap` ― it will fire
-for a login password if necessary, synchronize your folders, and exit.  See?
+up, ask you for a login password if necessary, synchronize your
+folders, and exit.  See?
 
-You can just throw away the rest of this finely-crafted, perfectly-honed manual!
+You can just throw away the rest of the finely-crafted,
-Of course, if you want to see how you can make OfflineIMAP FIVE TIMES FASTER FOR
+perfectly-honed user manual!  Of course, if you want to see how you can
-JUST $19.95 (err, well, $0), you have to read on!
+make OfflineIMAP FIVE TIMES FASTER FOR JUST $19.95 (err, well, $0),
-
+you have to read on our full user documentation and peruse the sample
-
+offlineimap.conf (which includes all available options) for further
-Documentation
+tweaks!
-=============
-
-If you are reading this file on github, you can find more documentations in the
-`docs` directory.
-
-Using your git repository, you can generate documentation with::
-
-	$ make doc
 
 
 Mailing list
 ============
 
 The user discussion, development and all exciting stuff take place in the
-`mailing list`_. You're *NOT* supposed to subscribe to send emails.
+`mailing list`_. You do *NOT* need to subscribe to send emails.
 
 
 Reporting bugs and contributions
@@ -154,9 +101,6 @@ Bugs
 ----
 
 Bugs, issues and contributions should be reported to the `mailing list`_.
-**Please, don't use the github features (messages, pull requests, etc) at all.
-It would most likely be discarded or ignored.**
-
 
 ========
 Examples

docs/HACKING.rst

@@ -183,6 +183,5 @@ branch. ,-)
 API
 ===
 
-API is documented in the dev-doc-src directory using the sphinx tools (also used
+The API is documented in the user documentation in the docs/ directory and browsable at `<http://docs.offlineimap.org>`_. This is a WIP. Contributions in this area would be very
-for python itself). This is a WIP. Contributions in this area would be very
 appreciated.

docs/MANUAL.rst

@@ -2,17 +2,14 @@
  OfflineIMAP Manual
 ====================
 
+.. _OfflineIMAP: http://offlineimap.org
+
 --------------------------------------------------------
 Powerful IMAP/Maildir synchronization and reader support
 --------------------------------------------------------
 
 :Author: John Goerzen <jgoerzen@complete.org> & contributors
-:Date: 2011-01-15
+:Date: 2012-02-23
-:Copyright: GPL v2
-:Manual section: 1
-
-.. TODO: :Manual group:
-
 
 DESCRIPTION
 ===========
@@ -45,7 +42,7 @@ OfflineImap is well suited to be frequently invoked by cron jobs, or can run in
 
 The documentation is included in the git repository and can be created by 
 issueing `make dev-doc` in the `doc` folder (python-sphinx required), or it can
-be viewed online at `http://docs.offlineimap.org`_.
+be viewed online at http://docs.offlineimap.org.
 
 .. _configuration:
 
@@ -72,7 +69,7 @@ OPTIONS
 
 The command line options are described by issueing `offlineimap --help`.
 Details on their use can be found either in the sample offlineimap.conf file or
-in the user docs at `http://docs.offlineimap.org`_.
+in the user docs at http://docs.offlineimap.org.
 
 User Interfaces
 ===============
@@ -306,6 +303,8 @@ as Man-In-The-Middle attacks which cause you to connect to the wrong
 server and pretend to be your mail server. DO NOT RELY ON STARTTLS AS A
 SAFE CONNECTION GUARANTEEING THE AUTHENTICITY OF YOUR IMAP SERVER!
 
+.. _UNIX signals:
+
 UNIX Signals
 ============
 
@@ -445,7 +444,7 @@ and Sent which should keep the same name::
 
 Synchronizing 2 IMAP accounts to local Maildirs that are "next to each
 other", so that mutt can work on both. Full email setup described by
-Thomas Kahle at `http://dev.gentoo.org/~tomka/mail.html`_
+Thomas Kahle at `<http://dev.gentoo.org/~tomka/mail.html>`_
 
 offlineimap.conf::
 
@@ -507,7 +506,7 @@ purposes: Fetching passwords from the gnome-keyring and translating
 folder names on the server to local foldernames.  An example
 implementation of get_username and get_password showing how to query
 gnome-keyring is contained in
-`http://dev.gentoo.org/~tomka/mail-setup.tar.bz2`_ The folderfilter is
+`<http://dev.gentoo.org/~tomka/mail-setup.tar.bz2>`_ The folderfilter is
 a lambda term that, well, filters which folders to get. The function
 `oimaptransfolder_acc2` translates remote folders into local folders
 with a very simple logic. The `INBOX` folder will have the same name

docs/Makefile

@@ -9,7 +9,7 @@ RST2HTML=`type rst2html >/dev/null 2>&1 && echo rst2html || echo rst2html.py`
 RST2MAN=`type rst2man >/dev/null 2>&1 && echo rst2man || echo rst2man.py`
 SPHINXBUILD = sphinx-build
 
-all: html dev-doc
+all: man doc
 
 html: $(HTML_TARGETS)
 
@@ -22,12 +22,12 @@ offlineimap.1: MANUAL.rst
 	$(RST2MAN) MANUAL.rst offlineimap.1
 	cp -f offlineimap.1 ..
 
-dev-doc:
+doc:
-	$(SPHINXBUILD) -b html -d dev-doc/doctrees dev-doc-src dev-doc/html
+	$(SPHINXBUILD) -b html -d html/doctrees doc-src html
 
 clean:
 	$(RM) -f $(HTML_TARGETS)
 	$(RM) -f offlineimap.1 ../offlineimap.1
-	$(RM) -rf dev-doc/*
+	$(RM) -rf html/*
 
-.PHONY: dev-doc
+.PHONY: clean doc

docs/dev-doc-src/FAQ.rst

@@ -1 +0,0 @@
-../FAQ.rst

docs/doc-src/FAQ.rst

@@ -203,9 +203,10 @@ How is OfflineIMAP conformance?
 Can I force OfflineIMAP to sync a folder right now?
 ---------------------------------------------------
 
-Yes, 
+Yes:
-  1) if you use the `Blinkenlights` UI.  That UI shows the active accounts
+
-as follows::
+1) if you use the `Blinkenlights` UI.  That UI shows the active
+accounts as follows::
 
    4: [active]      *Control: .
    3: [  4:36]      personal:
@@ -216,8 +217,9 @@ as follows::
    resync that account immediately.  This will be ignored if a resync is
    already in progress for that account.
 
-  2) while in sleep mode, you can also send a SIGUSR1. See the `Signals
+2) while in sleep mode, you can also send a SIGUSR1. See the :ref:`UNIX
-    on UNIX`_ section in the MANUAL for details.
+   signals` section in the MANUAL for details.
+
 
 I get a "Mailbox already exists" error
 --------------------------------------
@@ -291,14 +293,17 @@ certificates chain) in PEM format.  (See the documentation of
 `ssl.wrap_socket`_'s `certfile` parameter for the gory details.)  You can use either openssl or gnutls to create a certificate file in the required format.
 
 #. via openssl::
+
     openssl s_client -CApath /etc/ssl/certs -connect ${hostname}:imaps -showcerts \
        | perl -ne 'print if /BEGIN/../END/; print STDERR if /return/' > $sslcacertfile
     ^D
 
+
 #. via gnutls::
     gnutls-cli --print-cert -p imaps ${host} </dev/null | sed -n \
     |     '/^-----BEGIN CERT/,/^-----END CERT/p' > $sslcacertfile
 
+
 The path `/etc/ssl/certs` is not standardized; your system may store
 SSL certificates elsewhere.  (On some systems it may be in
 `/usr/local/share/certs/`.)

docs/doc-src/features.rst

@@ -0,0 +1,66 @@
+Description
+===========
+
+OfflineIMAP is a tool to simplify your e-mail reading. With OfflineIMAP, you can
+read the same mailbox from multiple computers. You get a current copy of your
+messages on each computer, and changes you make one place will be visible on all
+other systems. For instance, you can delete a message on your home computer, and
+it will appear deleted on your work computer as well. OfflineIMAP is also useful
+if you want to use a mail reader that does not have IMAP support, has poor IMAP
+support, or does not provide disconnected operation.
+
+OfflineIMAP works on pretty much any POSIX operating system, such as Linux, BSD
+operating systems, MacOS X, Solaris, etc.
+
+OfflineIMAP is a Free Software project licensed under the GNU General Public
+License. You can download it for free, and you can modify it. In fact, you are
+encouraged to contribute to OfflineIMAP, and doing so is fast and easy.
+
+OfflineIMAP is FAST; it synchronizes my two accounts with over 50 folders in 3
+seconds.  Other similar tools might take over a minute, and achieve a
+less-reliable result.  Some mail readers can take over 10 minutes to do the same
+thing, and some don't even support it at all.  Unlike other mail tools,
+OfflineIMAP features a multi-threaded synchronization algorithm that can
+dramatically speed up performance in many situations by synchronizing several
+different things simultaneously.
+
+OfflineIMAP is FLEXIBLE; you can customize which folders are synced via regular
+expressions, lists, or Python expressions; a versatile and comprehensive
+configuration file is used to control behavior; two user interfaces are
+built-in; fine-tuning of synchronization performance is possible; internal or
+external automation is supported; SSL and PREAUTH tunnels are both supported;
+offline (or "unplugged") reading is supported; and esoteric IMAP features are
+supported to ensure compatibility with the widest variety of IMAP servers.
+
+OfflineIMAP is SAFE; it uses an algorithm designed to prevent mail loss at all
+costs.  Because of the design of this algorithm, even programming errors should
+not result in loss of mail.  I am so confident in the algorithm that I use my
+own personal and work accounts for testing of OfflineIMAP pre-release,
+development, and beta releases.  Of course, legally speaking, OfflineIMAP comes
+with no warranty, so I am not responsible if this turns out to be wrong.
+
+.. note: OfflineImap was written by John Goerzen, who retired from
+    maintaining.  It is now maintained by Nicolas Sebrecht & Sebastian
+    Spaeth at https://github.com/spaetz/offlineimap. Thanks to John
+    for his great job and to have share this project with us.
+
+Method of Operation
+===================
+
+OfflineIMAP traditionally operates by maintaining a hierarchy of mail folders in
+Maildir format locally.  Your own mail reader will read mail from this tree, and
+need never know that the mail comes from IMAP.  OfflineIMAP will detect changes
+to the mail folders on your IMAP server and your own computer and
+bi-directionally synchronize them, copying, marking, and deleting messages as
+necessary.
+
+With OfflineIMAP 4.0, a powerful new ability has been introduced ― the program
+can now synchronize two IMAP servers with each other, with no need to have a
+Maildir layer in-between.  Many people use this if they use a mail reader on
+their local machine that does not support Maildirs.  People may install an IMAP
+server on their local machine, and point both OfflineIMAP and their mail reader
+of choice at it.  This is often preferable to the mail reader's own IMAP support
+since OfflineIMAP supports many features (offline reading, for one) that most
+IMAP-aware readers don't.  However, this feature is not as time-tested as
+traditional syncing, so my advice is to stick with normal methods of operation
+for the time being.

docs/doc-src/index.rst

@@ -15,6 +15,7 @@ If you just want to get started with minimal fuzz, have a look at our `online qu
 More information on specific topics can be found on the following pages:
 
 **User documentation**
+  * :doc:`Overview and features <features>`
   * :doc:`installation/uninstall <INSTALL>`
   * :doc:`user manual/Configuration <MANUAL>`
   * :doc:`Folder filtering & name transformation guide <nametrans>`
@@ -28,6 +29,7 @@ More information on specific topics can be found on the following pages:
 .. toctree::
    :hidden:
 
+   features
    INSTALL
    MANUAL
    nametrans

docs/doc-src/nametrans.rst

@@ -48,7 +48,7 @@ at the end when required by Python syntax)  For instance::
 Usually it suffices to put a `folderfilter`_ setting in the remote repository section. You might want to put a folderfilter option on the local repository if you want to prevent some folders on the local repository to be created on the remote one. (Even in this case, folder filters on the remote repository will prevent that)
 
 folderincludes
-^^^^^^^^^^^^^^
+--------------
 
 You can specify `folderincludes`_ to manually include additional folders to be synced, even if they had been filtered out by a folderfilter setting. `folderincludes`_ should return a Python list.
 
@@ -92,8 +92,9 @@ locally?  Try this::
     this rule will result in undefined behavior. See also *Sharing a
     maildir with multiple IMAP servers* in the :ref:`pitfalls` section.
 
+
 Reverse nametrans
-^^^^^^^^^^^^^^^^^^
++++++++++++++++++
 
 Since 6.4.0, OfflineImap supports the creation of folders on the remote repository and that complicates things. Previously, only one nametrans setting on the remote repository was needed and that transformed a remote to a local name. However, nametrans transformations are one-way, and OfflineImap has no way using those rules on the remote repository to back local names to remote names.
 
@@ -162,7 +163,8 @@ What folder separators do I need to use in nametrans rules?
   in nametrans rules?::
 
       nametrans = lambda f: "INBOX/" + f
-or::
+
+  or::
       nametrans = lambda f: "INBOX." + f
 
 **A:** Generally use the folder separator as defined in the repository