tobias/docker-offlineimap
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

docs: full refactoring of the MANUAL

Browse Source 
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
This commit is contained in:
Nicolas Sebrecht 2015-03-10 14:55:54 +01:00
parent 7c7e7f92b1
commit 0b43418911
9 changed files with 592 additions and 912 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

623
docs/MANUAL.rst
View File

@ -1,623 +0,0 @@
====================
OfflineIMAP Manual
====================
.. _OfflineIMAP: http://offlineimap.org
--------------------------------------------------------
Powerful IMAP/Maildir synchronization and reader support
--------------------------------------------------------
:Author: John Goerzen <jgoerzen@complete.org> & contributors
:Date: 2012-02-23
DESCRIPTION
===========
OfflineImap operates on a REMOTE and a LOCAL repository and synchronizes
emails between them, so that you can read the same mailbox from multiple
computers. The REMOTE repository is some IMAP server, while LOCAL can be
either a local Maildir or another IMAP server.
Missing folders will be automatically created on both sides if
needed. No folders will be deleted at the moment.
Configuring OfflineImap in basic mode is quite easy, however it provides
an amazing amount of flexibility for those with special needs. You can
specify the number of connections to your IMAP server, use arbitrary
python functions (including regular expressions) to limit the number of
folders being synchronized. You can transpose folder names between
repositories using any python function, to mangle and modify folder
names on the LOCAL repository. There are six different ways to hand the
IMAP password to OfflineImap from console input, specifying in the
configuration file, .netrc support, specifying in a separate file, to
using arbitrary python functions that somehow return the
password. Finally, you can use IMAPs IDLE infrastructure to always keep
a connection to your IMAP server open and immediately be notified (and
synchronized) when a new mail arrives (aka Push mail).
Most configuration is done via the configuration file. However, any setting can
also be overriden by command line options handed to OfflineIMAP.
OfflineImap is well suited to be frequently invoked by cron jobs, or can run in
daemon mode to periodically check your email (however, it will exit in some
error situations).
The documentation is included in the git repository and can be created by
issueing `make doc` in the `doc` folder (python-sphinx required), or it can
be viewed online at http://docs.offlineimap.org.
.. _configuration:
Configuration
=============
`OfflineIMAP`_ is regulated by a configuration file that is normally stored in
`~/.offlineimaprc`. `OfflineIMAP`_ ships with a file named `offlineimap.conf`
that you should copy to that location and then edit. This file is vital to
proper operation of the system; it sets everything you need to run
`OfflineIMAP`_. Full documentation for the configuration file is included
within the sample file.
`OfflineIMAP`_ also ships a file named `offlineimap.conf.minimal` that you can
also try. It's useful if you want to get started with the most basic feature
set, and you can read about other features later with `offlineimap.conf`.
Check out the `Use Cases`_ section for some example configurations.
If you want to be XDG-compatible, you can put your configuration file into
`$XDG_CONFIG_HOME/offlineimap/config`.
OPTIONS
=======
The command line options are described by issueing `offlineimap --help`.
Details on their use can be found either in the sample offlineimap.conf file or
in the user docs at http://docs.offlineimap.org.
User Interfaces
===============
OfflineIMAP has various user interfaces that let you choose how the
program communicates information to you. The 'ui' option in the
configuration file specifies the user interface. The -u command-line
option overrides the configuration file setting. The available values
for the configuration file or command-line are described in this
section.
Blinkenlights
---------------
Blinkenlights is an interface designed to be sleek, fun to watch, and
informative of the overall picture of what OfflineIMAP is doing.
Blinkenlights contains a row of "LEDs" with command buttons and a log.
The log shows more detail about what is happening and is color-coded to match
the color of the lights.
Each light in the Blinkenlights interface represents a thread of execution --
that is, a particular task that OfflineIMAP is performing right now. The colors
indicate what task the particular thread is performing, and are as follows:
* Black:
indicates that this light's thread has terminated; it will light up again
later when new threads start up. So, black indicates no activity.
* Red (Meaning 1):
is the color of the main program's thread, which basically does nothing but
monitor the others. It might remind you of HAL 9000 in 2001.
* Gray:
indicates that the thread is establishing a new connection to the IMAP
server.
* Purple:
is the color of an account synchronization thread that is monitoring the
progress of the folders in that account (not generating any I/O).
* Cyan:
indicates that the thread is syncing a folder.
* Green:
means that a folder's message list is being loaded.
* Blue:
is the color of a message synchronization controller thread.
* Orange:
indicates that an actual message is being copied. (We use fuchsia for fake
messages.)
* Red (meaning 2):
indicates that a message is being deleted.
* Yellow / bright orange:
indicates that message flags are being added.
* Pink / bright red:
indicates that message flags are being removed.
* Red / Black Flashing:
corresponds to the countdown timer that runs between synchronizations.
The name of this interfaces derives from a bit of computer history. Eric
Raymond's Jargon File defines blinkenlights, in part, as:
Front-panel diagnostic lights on a computer, esp. a dinosaur. Now that
dinosaurs are rare, this term usually refers to status lights on a modem,
network hub, or the like.
This term derives from the last word of the famous blackletter-Gothic sign in
mangled pseudo-German that once graced about half the computer rooms in the
English-speaking world. One version ran in its entirety as follows:
| ACHTUNG! ALLES LOOKENSPEEPERS!
|
| Das computermachine ist nicht fuer gefingerpoken und mittengrabben.
| Ist easy schnappen der springenwerk, blowenfusen und poppencorken
| mit spitzensparken. Ist nicht fuer gewerken bei das dumpkopfen.
| Das rubbernecken sichtseeren keepen das cotten-pickenen hans in das
| pockets muss; relaxen und watchen das blinkenlichten.
TTYUI
------
TTYUI interface is for people running in terminals. It prints out basic
status messages and is generally friendly to use on a console or xterm.
Basic
------
Basic is designed for situations in which OfflineIMAP will be run non-attended
and the status of its execution will be logged. This user interface is not
capable of reading a password from the keyboard; account passwords must be
specified using one of the configuration file options. For example, it will not
print periodic sleep announcements and tends to be a tad less verbose, in
general.
Quiet
-----
It will output nothing except errors and serious warnings. Like Basic,
this user interface is not capable of reading a password from the
keyboard; account passwords must be specified using one of the
configuration file options.
MachineUI
---------
MachineUI generates output in a machine-parsable format. It is designed
for other programs that will interface to OfflineIMAP.
Synchronization Performance
===========================
By default, we use fairly conservative settings that are safe for
syncing but that might not be the best performing one. Once you got
everything set up and running, you might want to look into speeding up
your synchronization. Here are a couple of hints and tips on how to
achieve this.
1) Use maxconnections > 1. By default we only use one connection to an
IMAP server. Using 2 or even 3 speeds things up considerably in most
cases. This setting goes into the [Repository XXX] section.
2) Use folderfilters. The quickest sync is a sync that can ignore some
folders. I sort my inbox into monthly folders, and ignore every
folder that is more than 2-3 months old, this lets me only inspect a
fraction of my Mails on every sync. If you haven't done this yet, do
it :). See the folderfilter section the example offlineimap.conf.
3) The default status cache is a plain text file that will write out
the complete file for each single new message (or even changed flag)
to a temporary file. If you have plenty of files in a folder, this
is a few hundred kilo to megabytes for each mail and is bound to
make things slower. I recommend to use the sqlite backend for
that. See the status_backend = sqlite setting in the example
offlineimap.conf. You will need to have python-sqlite installed in
order to use this. This will save you plenty of disk activity. Do
note that the sqlite backend is still considered experimental as it
has only been included recently (although a loss of your status
cache should not be a tragedy as that file can be rebuilt
automatically)
4) Use quick sync. A regular sync will request all flags and all UIDs
of all mails in each folder which takes quite some time. A 'quick'
sync only compares the number of messages in a folder on the IMAP
side (it will detect flag changes on the Maildir side of things
though). A quick sync on my smallish account will take 7 seconds
rather than 40 seconds. Eg, I run a cron script that does a regular
sync once a day, and does quick syncs (-q) only synchronizing the
"-f INBOX" in between.
5) Turn off fsync. In the [general] section you can set fsync to True
or False. If you want to play 110% safe and wait for all operations
to hit the disk before continueing, you can set this to True. If you
set it to False, you lose some of that safety, trading it for speed.
Upgrading from plain text cache to SQLITE based cache
=====================================================
OfflineImap uses a cache to store the last know status of mails (flags etc).
Historically that has meant plain text files, but recently we introduced
sqlite-based cache, which helps with performance and CPU usage on large folders.
Here is how to upgrade existing plain text cache installations to sqlite based
one:
1) Sync to make sure things are reasonably similar
2) Change the account section to status_backend = sqlite
3) A new sync will convert your plain text cache to an sqlite cache
(but leave the old plain text cache around for easy reverting) This
should be quick and not involve any mail up/downloading.
4) See if it works :-)
5) If it does not work, go back to the old version or set
status_backend=plain
6) Or, once you are sure it works, you can delete the
.offlineimap/Account-foo/LocalStatus folder (the new cache will be
in the LocalStatus-sqlite folder)
Security and SSL
================
Some words on OfflineImap and its use of SSL/TLS. By default, we will
connect using any method that openssl supports, that is SSLv2, SSLv3, or
TLSv1. Do note that SSLv2 is notoriously insecure and deprecated.
Unfortunately, python2 does not offer easy ways to disable SSLv2. It is
recommended you test your setup and make sure that the mail server does
not use an SSLv2 connection. Use e.g. "openssl s_client -host
mail.server -port 443" to find out the connection that is used by
default.
Certificate checking
--------------------
Unfortunately, by default we will not verify the certificate of an IMAP
TLS/SSL server we connect to, so connecting by SSL is no guarantee
against man-in-the-middle attacks. While verifying a server certificate
fingerprint is being planned, it is not implemented yet. There is
currently only one safe way to ensure that you connect to the correct
server in an encrypted manner: You can specify a 'sslcacertfile' setting
in your repository section of offlineimap.conf pointing to a file that
contains (among others) a CA Certificate in PEM format which validating
your server certificate. In this case, we will check that: 1) The server
SSL certificate is validated by the CA Certificate 2) The server host
name matches the SSL certificate 3) The server certificate is not past
its expiration date. The FAQ contains an entry on how to create your own
certificate and CA certificate.
StartTLS
--------
If you have not configured your account to connect via SSL anyway,
OfflineImap will still attempt to set up an SSL connection via the
STARTTLS function, in case the imap server supports it. Do note, that
there is no certificate or fingerprint checking involved at all, when
using STARTTLS (the underlying imaplib library does not support this
yet). This means that you will be protected against passively listening
eavesdroppers and they will not be able to see your password or email
contents. However, this will not protect you from active attacks, such
as Man-In-The-Middle attacks which cause you to connect to the wrong
server and pretend to be your mail server. DO NOT RELY ON STARTTLS AS A
SAFE CONNECTION GUARANTEEING THE AUTHENTICITY OF YOUR IMAP SERVER!
.. _UNIX signals:
UNIX Signals
============
OfflineImap listens to the unix signals SIGUSR1, SIGUSR2, SIGTERM,
SIGINT, SIGHUP, SIGQUIT:
If sent a SIGUSR1 it will abort any current (or next future) sleep of all
accounts that are configured to "autorefresh". In effect, this will trigger a
full sync of all accounts to be performed as soon as possible.
If sent a SIGUSR2, it will stop "autorefresh mode" for all accounts. That is,
accounts will abort any current sleep and will exit after a currently running
synchronization has finished. This signal can be used to gracefully exit out of
a running offlineimap "daemon".
SIGTERM, SIGINT, SIGHUP are all treated to gracefully terminate as
soon as possible. This means it will finish syncing the current folder
in each account, close keep alive connections, remove locks on the
accounts and exit. It may take up to 10 seconds, if autorefresh option
is used.
SIGQUIT dumps stack traces for all threads and tries to dump process
core.
Folder filtering and nametrans
==============================
OfflineImap offers flexible (and complex) ways of filtering and transforming
folder names. Please see the docs/doc-src/nametrans.rst document about details
how to use folder filters and name transformations. The documentation will be
autogenerated by a "make doc" in the docs directory. It is also viewable at
:ref:`folder_filtering_and_name_translation`.
KNOWN ISSUES
============
* SSL3 write pending:
users enabling SSL may hit a bug about "SSL3 write pending". If so, the
account(s) will stay unsynchronised from the time the bug appeared. Running
OfflineIMAP again can help. We are still working on this bug. Patches or
detailed bug reports would be appreciated. Please check you're running the
last stable version and send us a report to the mailing list including the
full log.
* IDLE support is incomplete and experimental. Bugs may be encountered.
* No hook exists for "run after an IDLE response". Email will
show up, but may not be processed until the next refresh cycle.
* nametrans may not be supported correctly.
* IMAP IDLE <-> IMAP IDLE doesn't work yet.
* IDLE may only work "once" per refresh. If you encounter this bug,
please send a report to the list!
* Maildir support in Windows drive
Maildir uses colon caracter (:) in message file names. Colon is however
forbidden character in windows drives. There are several workarounds for
that situation:
* Use "maildir-windows-compatible = yes" account OfflineIMAP configuration.
- That makes OfflineIMAP to use exclamation mark (!) instead of colon for
storing messages. Such files can be written to windows partitions. But
you will probably loose compatibility with other programs trying to
read the same Maildir.
- Exclamation mark was chosen because of the note in
http://docs.python.org/library/mailbox.html
- If you have some messages already stored without this option, you will
have to re-sync them again
* Enable file name character translation in windows registry (not tested)
- http://support.microsoft.com/kb/289627
* Use cygwin managed mount (not tested)
- not available anymore since cygwin 1.7
* OfflineIMAP confused after system suspend.
When resuming a suspended session, OfflineIMAP does not cleanly handles the
broken socket(s) if socktimeout option is not set.
You should enable this option with a value like 10.
.. _pitfalls:
PITFALLS & ISSUES
=================
Sharing a maildir with multiple IMAP servers
--------------------------------------------
Generally a word of caution mixing IMAP repositories on the same
Maildir root. You have to be careful that you *never* use the same
maildir folder for 2 IMAP servers. In the best case, the folder MD5
will be different, and you will get a loop where it will upload your
mails to both servers in turn (infinitely!) as it thinks you have
placed new mails in the local Maildir. In the worst case, the MD5 is
the same (likely) and mail UIDs overlap (likely too!) and it will fail to
sync some mails as it thinks they are already existent.
I would create a new local Maildir Repository for the Personal Gmail and
use a different root to be on the safe side here. You could e.g. use
`~/mail/Pro` as Maildir root for the ProGmail and
`~/mail/Personal` as root for the personal one.
If you then point your local mutt, or whatever MUA you use to `~/mail/`
as root, it should still recognize all folders. (see the 2 IMAP setup
in the `Use Cases`_ section.
USE CASES
=========
Sync from GMail to another IMAP server
--------------------------------------
This is an example of a setup where "TheOtherImap" requires all folders to be under INBOX::
[Repository Gmailserver-foo]
#This is the remote repository
type = Gmail
remotepass = XXX
remoteuser = XXX
# The below will put all GMAIL folders as sub-folders of the 'local' INBOX,
# assuming that your path separator on 'local' is a dot.
nametrans = lambda x: 'INBOX.' + x
[Repository TheOtherImap]
#This is the 'local' repository
type = IMAP
remotehost = XXX
remotepass = XXX
remoteuser = XXX
#Do not use nametrans here.
Sync from Gmail to a local Maildir with labels
----------------------------------------------
This is an example of a setup where GMail gets synced with a local Maildir.
It also keeps track of GMail labels, that get embedded into the messages
under the header configured in the labelsheader setting, and syncs them back
and forth the same way as flags. This is particularly useful with an email
client that indexes your email and recognizes the labels header, so that you
can sync a single "All Mail" folder, and navigate your email via searches.
The header used to store the labels depends on the email client you plan to use.
Some choices that may be recognized by email clients are X-Keywords
(the default) or X-Labels. Note that if you need to change the label header
after the labels have already been synced, you will have to change the header
manually on all messages, otherwise offlineimap will not pick up the labels under
the old header.
The first time OfflineIMAP runs with synclabels enabled on a large repository it
may take some time as the labels are read / embedded on every message.
Afterwards local label changes are detected using modification times, which is
much faster::
[Account Gmail-mine]
localrepository = Gmaillocal-mine
remoterepository = Gmailserver-mine
synclabels = yes
# This header is where labels go. Usually you will be fine
# with default value (X-Keywords), but in case you want it
# different, here we go:
labelsheader = X-Keywords
[Repository Gmailserver-mine]
#This is the remote repository
type = Gmail
remotepass = XXX
remoteuser = XXX
[Repository Gmaillocal-mine]
#This is the 'local' repository
type = GmailMaildir
There are some labels, that gmail treats in a special way. All internal gmail
labels start with "\\". Those labels include: \\Drafts, \\Important, \\Inbox,
\\Sent, \\Junk, \\Flagged, \\Trash. You can add and remove those labels
locally, and when synced, will have special actions on the gmail side. For instance,
adding the label \Trash to an email will move it to the trash, and be permanantly
deleted after some time. This is relevant, since gmail's IMAP prevents from removing
messages from the "All Mail" folder the usual way.
Selecting only a few folders to sync
------------------------------------
Add this to the remote gmail repository section to only sync mails which are in a certain folder::
folderfilter = lambda folder: folder.startswith('MyLabel')
To only get the All Mail folder from a Gmail account, you would e.g. do::
folderfilter = lambda folder: folder.startswith('[Gmail]/All Mail')
Another nametrans transpose example
-----------------------------------
Put everything in a GMX. subfolder except for the boxes INBOX, Draft,
and Sent which should keep the same name::
nametrans: lambda folder: folder if folder in ['INBOX', 'Drafts', 'Sent'] \
else re.sub(r'^', r'GMX.', folder)
2 IMAP using name translations
------------------------------
Synchronizing 2 IMAP accounts to local Maildirs that are "next to each
other", so that mutt can work on both. Full email setup described by
Thomas Kahle at `<http://dev.gentoo.org/~tomka/mail.html>`_
offlineimap.conf::
[general]
accounts = acc1, acc2
maxsyncaccounts = 2
ui = ttyui
pythonfile=~/bin/offlineimap-helpers.py
socktimeout = 90
[Account acc1]
localrepository = acc1local
remoterepository = acc1remote
autorefresh = 2
[Account acc2]
localrepository = acc2local
remoterepository = acc2remote
autorefresh = 4
[Repository acc1local]
type = Maildir
localfolders = ~/Mail/acc1
[Repository acc2local]
type = Maildir
localfolders = ~/Mail/acc2
[Repository acc1remote]
type = IMAP
remotehost = imap.acc1.com
remoteusereval = get_username("imap.acc1.net")
remotepasseval = get_password("imap.acc1.net")
nametrans = oimaptransfolder_acc1
ssl = yes
maxconnections = 2
# Folders to get:
folderfilter = lambda foldername: foldername in [
'INBOX', 'Drafts', 'Sent', 'archiv']
[Repository acc2remote]
type = IMAP
remotehost = imap.acc2.net
remoteusereval = get_username("imap.acc2.net")
remotepasseval = get_password("imap.acc2.net")
nametrans = oimaptransfolder_acc2
ssl = yes
maxconnections = 2
One of the coolest things about offlineimap is that you can call
arbitrary python code from your configuration. To do this, specify a
pythonfile with::
pythonfile=~/bin/offlineimap-helpers.py
Here is a basic content sample::
import commands
def get_password(account_name):
cmd = "security find-internet-password -w -a '%s'"% account_name
(status, output) = commands.getstatusoutput(cmd)
return output
From this sample, replace the cmd line with whatever can retrieve your password.
Your pythonfile needs to contain implementations for the functions
that you want to use in offflineimaprc. The example uses it for two
purposes: Fetching passwords from the gnome-keyring and translating
folder names on the server to local foldernames. An example
implementation of get_username and get_password showing how to query
gnome-keyring is contained in
`<http://dev.gentoo.org/~tomka/mail-setup.tar.bz2>`_ The folderfilter is
a lambda term that, well, filters which folders to get. The function
`oimaptransfolder_acc2` translates remote folders into local folders
with a very simple logic. The `INBOX` folder will have the same name
as the account while any other folder will have the account name and a
dot as a prefix. This is useful for hierarchichal display in mutt.
Offlineimap handles the renaming correctly in both directions::
import re
def oimaptransfolder_acc1(foldername):
if(foldername == "INBOX"):
retval = "acc1"
else:
retval = "acc1." + foldername
retval = re.sub("/", ".", retval)
return retval
def oimaptransfolder_acc2(foldername):
if(foldername == "INBOX"):
retval = "acc2"
else:
retval = "acc2." + foldername
retval = re.sub("/", ".", retval)
return retval

10
docs/Makefile
View File

@ -16,11 +16,13 @@ html: $(HTML_TARGETS)
$(HTML_TARGETS): %.html : %.rst
$(RST2HTML) $? $@
man: offlineimap.1
man: offlineimap.1 offlineimapui.7
offlineimap.1: MANUAL.rst
$(RST2MAN) MANUAL.rst offlineimap.1
cp -f offlineimap.1 ..
offlineimap.1: offlineimap.txt
a2x -v -f manpage $?
offlineimapui.7: offlineimapui.txt
a2x -v -f manpage $?
doc:
$(SPHINXBUILD) -b html -d html/doctrees doc-src html

201
docs/conf.py
View File

@ -1,201 +0,0 @@
# -*- coding: utf-8 -*-
#
# pyDNS documentation build configuration file, created by
# sphinx-quickstart on Tue Feb 2 10:00:47 2010.
#
# This file is execfile()d with the current directory set to its containing dir.
#
# Note that not all possible configuration values are present in this
# autogenerated file.
#
# All configuration values have a default; values that are commented out
# serve to show the default.
import sys, os
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
sys.path.insert(0,os.path.abspath('../..'))
from offlineimap import __version__,__author__
# -- General configuration -----------------------------------------------------
# Add any Sphinx extension module names here, as strings. They can be extensions
# coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
extensions = ['sphinx.ext.autodoc', 'sphinx.ext.doctest',
'sphinx.ext.intersphinx', 'sphinx.ext.todo', 'sphinx.ext.viewcode']
autoclass_content = "both"
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# The suffix of source filenames.
source_suffix = '.rst'
# The encoding of source files.
#source_encoding = 'utf-8'
# The master toctree document.
master_doc = 'index'
# General information about the project.
project = u'OfflineImap'
copyright = u'2002-2010, ' + __author__
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
version = __version__
# The full version, including alpha/beta/rc tags.
release = __version__
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#language = None
# There are two options for replacing |today|: either, you set today to some
# non-false value, then it is used:
#today = ''
# Else, today_fmt is used as the format for a strftime call.
#today_fmt = '%B %d, %Y'
# List of documents that shouldn't be included in the build.
#unused_docs = []
# List of directories, relative to source directory, that shouldn't be searched
# for source files.
exclude_trees = []
# The reST default role (used for this markup: `text`) to use for all documents.
#default_role = None
# If true, '()' will be appended to :func: etc. cross-reference text.
#add_function_parentheses = True
# If true, the current module name will be prepended to all description
# unit titles (such as .. function::).
add_module_names = False
# If true, sectionauthor and moduleauthor directives will be shown in the
# output. They are ignored by default.
#show_authors = False
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
# A list of ignored prefixes for module index sorting.
#modindex_common_prefix = []
# -- Options for HTML output ---------------------------------------------------
# The theme to use for HTML and HTML Help pages. Major themes that come with
# Sphinx are currently 'default' and 'sphinxdoc'.
html_theme = 'default'
#html_style = ''
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
#html_theme_options = {}
# Add any paths that contain custom themes here, relative to this directory.
#html_theme_path = []
# The name for this set of Sphinx documents. If None, it defaults to
# "<project> v<release> documentation".
#html_title = None
# A shorter title for the navigation bar. Default is the same as html_title.
#html_short_title = None
# The name of an image file (relative to this directory) to place at the top
# of the sidebar.
#html_logo = None
# The name of an image file (within the static path) to use as favicon of the
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
# pixels large.
#html_favicon = None
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
#html_static_path = ['html']
# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
# using the given strftime format.
#html_last_updated_fmt = '%b %d, %Y'
# If true, SmartyPants will be used to convert quotes and dashes to
# typographically correct entities.
#html_use_smartypants = True
# Custom sidebar templates, maps document names to template names.
#html_sidebars = {}
# Additional templates that should be rendered to pages, maps page names to
# template names.
#html_additional_pages = {}
# If false, no module index is generated.
html_use_modindex = False
# If false, no index is generated.
#html_use_index = True
# If true, the index is split into individual pages for each letter.
#html_split_index = False
# If true, links to the reST sources are added to the pages.
#html_show_sourcelink = True
# If true, an OpenSearch description file will be output, and all pages will
# contain a <link> tag referring to it. The value of this option must be the
# base URL from which the finished HTML is served.
#html_use_opensearch = ''
# If nonempty, this is the file name suffix for HTML files (e.g. ".xhtml").
#html_file_suffix = ''
# Output file base name for HTML help builder.
htmlhelp_basename = 'dev-doc'
# -- Options for LaTeX output --------------------------------------------------
# The paper size ('letter' or 'a4').
#latex_paper_size = 'letter'
# The font size ('10pt', '11pt' or '12pt').
#latex_font_size = '10pt'
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title, author, documentclass [howto/manual]).
latex_documents = [
('index', 'offlineimap.tex', u'OfflineImap Documentation',
u'OfflineImap contributors', 'manual'),
]
# The name of an image file (relative to this directory) to place at the top of
# the title page.
#latex_logo = None
# For "manual" documents, if this is true, then toplevel headings are parts,
# not chapters.
#latex_use_parts = False
# Additional stuff for the LaTeX preamble.
#latex_preamble = ''
# Documents to append as an appendix to all manuals.
#latex_appendices = []
# If false, no module index is generated.
#latex_use_modindex = True
# Example configuration for intersphinx: refer to the Python standard library.
intersphinx_mapping = {'http://docs.python.org/': None}

1
docs/doc-src/MANUAL.rst
View File

@ -1 +0,0 @@
../MANUAL.rst

13
docs/doc-src/index.rst
View File

@ -2,20 +2,13 @@
.. _OfflineIMAP: http://offlineimap.github.io
Welcome to OfflineIMAP's documentation
======================================
**Configuration**
* :doc:`user manual/Configuration <MANUAL>`
* :doc:`Folder filtering & name transformation guide <nametrans>`
* :doc:`command line options <offlineimap>`
Welcome to OfflineIMAP's developer documentation
================================================
**Developer documentation**
* :doc:`Advanced Git <GitAdvanced>`
* :doc:`API documentation <API>` for internal details on the
:mod:`offlineimap` module
**OfflineIMAP APIs**
**Documented APIs**
.. toctree::
API

408
docs/offlineimap.txt Normal file
View File

@ -0,0 +1,408 @@
offlineimap(1)
==============
NAME
----
offlineimap - Synchronize mailboxes and Maildirs
SYNOPSIS
--------
[verse]
'offlineimap' (options)
DESCRIPTION
-----------
Synchronize the accounts configured in the configuration file via IMAP. Each
account has two sides.
One of the side must be an IMAP server.
The other side can either be a Maildir or another IMAP server.
OPTIONS
-------
-h::
--help::
Display summary of options.
--version::
Output version.
--dry-run::
Run in dry run mode.
+
Do not actually modify any store but check and print what synchronization
actions would be taken if a sync would be performed. It will not precisely
give the exact information what will happen. If e.g. we need to create a
folder, it merely outputs 'Would create folder X', but not how many and which
mails it would transfer.
--info::
Output information on the configured email repositories.
+
Useful for debugging and bug reporting. Use in conjunction with the -a option
to limit the output to a single account. This mode will prevent any actual
sync to occur and exits after it outp ut the debug information.
-1::
Limit multithreading operations and run solely a single-thread sync.
+
This effectively sets the maxsyncaccounts and all maxconnections configuration
file variables to 1. This is 1, the number.
-P <directory>::
Set OfflineIMAP into profile mode.
+
The program will create DIR (it must not already exist). As it runs, Python
profiling information about each thread is logged into profiledir. Please
note: This option is present for debugging and optimization only, and should
NOT be used unless you have a specific reason to do so. It will significantly
decrease program performance, may reduce reliability, and can generate huge
amounts of data. This option implies the -1 option.
-a <account1[,account2[,...]]>::
Overrides the accounts section in the config file.
+
Allows to specify a particular account or set of accounts to sync without
having to edit the config file.
-c <path/to/configuration_file>::
Specifies a configuration file to use.
-d <type1[,type2[,...]]>::
Enables debugging for OfflineIMAP.
+
This is useful if you are to track down a malfunction or figure out what is
going on under the hood. This option requires one or more debugtypes,
separated by commas. These define what exactly will be debugged, and so far
include two options: imap, thread, maildir or ALL. The imap option will enable
IMAP protocol stream and parsing debugging. Note that the output may contain
passwords, so take care to remove that from the debugging output before
sending it to anyone else. The maildir option will enable debugging for
certain Maildir operations. The use of any debug option (unless 'thread' is
included), implies the single-thread option -1.
-l <path/to/file.log>::
Send logs to <file.log>.
-f <folder1[,folder1[,...]]>::
Only sync the specified folders.
+
The folder names are the untranslated foldernames of the remote repository.
This command-line option overrides any 'folderfilter' and 'folderincludes'
options in the configuration file.
-k <[section:]option=value::
Override any configuration file option.
+
If "section" is omitted, it defaults to "general". Any underscores in the
section name are replaced with spaces: for instance, to override option
"autorefresh" in the "[Account Personal]" section in the config file one would
use "-k Account_Personal:autorefresh=30". Repeat this option as much as
necessary to redefine multiple options.
-o::
Run only once.
+
Ignore any autorefresh setting in the configuration file.
-q::
Run only quick synchronizations.
+
Ignore any flag updates on IMAP servers. If a flag on the remote IMAP changes,
and we have the message locally, it will be left untouched in a quick run.
-u <UI>::
Specifies an alternative user interface to use.
+
This overrides the default specified in the configuration file. The UI
specified with -u will be forced to be used, even if checks determine that it
is not usable. Possible interface choices are: quiet, basic, ttyui,
blinkenlights, machineui.
--column[=<options>]::
--no-column::
Display branch listing in columns. See configuration variable
column.branch for option syntax.`--column` and `--no-column`
without options are equivalent to 'always' and 'never' respectively.
+
This option is only applicable in non-verbose mode.
Synchronization Performance
---------------------------
By default, we use fairly conservative settings that are safe for syncing but
that might not be the best performing one. Once you got everything set up and
running, you might want to look into speeding up your synchronization. Here
are a couple of hints and tips on how to achieve this.
1. Use maxconnections > 1.
+
By default we only use one connection to an IMAP server. Using 2 or even 3
speeds things up considerably in most cases. This setting goes into the
[Repository XXX] section.
2. Use folderfilters.
+
The quickest sync is a sync that can ignore some folders. I sort my inbox into
monthly folders, and ignore every folder that is more than 2-3 months old,
this lets me only inspect a fraction of my Mails on every sync. If you haven't
done this yet, do it :). See the folderfilter section the example
offlineimap.conf.
3. The cache.
+
The default status cache is a plain text file that will write out the complete
file for each single new message (or even changed flag) to a temporary file.
If you have plenty of files in a folder, this is a few hundred kilo to
megabytes for each mail and is bound to make things slower. I recommend to use
the sqlite backend for that. See the status_backend = sqlite setting in the
example offlineimap.conf. You will need to have python-sqlite installed in
order to use this. This will save you plenty of disk activity. Do note that
the sqlite backend is still considered experimental as it has only been
included recently (although a loss of your status cache should not be a
tragedy as that file can be rebuilt automatically)
4. Use quick sync.
+
A regular sync will request all flags and all UIDs of all mails in each folder
which takes quite some time. A 'quick' sync only compares the number of
messages in a folder on the IMAP side (it will detect flag changes on the
Maildir side of things though). A quick sync on my smallish account will take
7 seconds rather than 40 seconds. Eg, I run a cron script that does a regular
sync once a day, and does quick syncs (-q) only synchronizing the "-f INBOX"
in between.
5. Turn off fsync.
+
In the [general] section you can set fsync to True or False. If you want to
play 110% safe and wait for all operations to hit the disk before continueing,
you can set this to True. If you set it to False, you lose some of that
safety, trading it for speed.
Upgrading from plain text to SQLite cache format
------------------------------------------------
OfflineImap uses a cache to store the last know status of mails (flags etc).
Historically that has meant plain text files, but recently we introduced
sqlite-based cache, which helps with performance and CPU usage on large
folders. Here is how to upgrade existing plain text cache installations to
sqlite based one:
1. Sync to make sure things are reasonably similar.
2. Change the account section to "status_backend = sqlite".
3. Run a new sync.
+
This will convert your plain text cache to an sqlite cache (but leave
the old plain text cache around for easy reverting). This must be quick and
not involve any mail up/downloading.
4. See if it works! :-)
5. If it does not work, go back to the old version or set "status_backend = plain"
6. Delete the old text cache files.
Once you are sure it works, you can delete the
~/.offlineimap/Account-foo/LocalStatus folder (the new cache will be in the
LocalStatus-sqlite folder)
Security and SSL
----------------
By default, OfflineIMAP will connect using any method that 'openssl' supports,
that is SSLv2, SSLv3, or TLSv1.
Do note that SSLv2 is notoriously insecure and deprecated. Unfortunately,
python2 does not offer easy ways to disable SSLv2. It is recommended you test
your setup and make sure that the mail server does not use an SSLv2
connection. Use e.g. "openssl s_client -host mail.server -port 443" to find
out the connection that is used by default.
* Certificate checking
+
Unfortunately, by default we will not verify the certificate of an IMAP
TLS/SSL server we connect to, so connecting by SSL is no guarantee against
man-in-the-middle attacks. While verifying a server certificate fingerprint is
being planned, it is not implemented yet. There is currently only one safe way
to ensure that you connect to the correct server in an encrypted manner: you
can specify a 'sslcacertfile' setting in your repository section of
offlineimap.conf pointing to a file that contains (among others) a CA
Certificate in PEM format which validating your server certificate. In this
case, we will check that:
1. The server SSL certificate is validated by the CA Certificate.
2. The server host name matches the SSL certificate.
3. The server certificate is not past its expiration date.
The FAQ has an entry on how to create your own certificate and CA certificate.
* StartTLS
+
If you have not configured your account to connect via SSL anyway, OfflineImap
will still attempt to set up an SSL connection via the STARTTLS function, in
case the imap server supports it.
+
There is no certificate or fingerprint checking involved at all, when using
STARTTLS (the underlying imaplib library does not support this yet). This
means that you will be protected against passively listening eavesdroppers and
they will not be able to see your password or email contents. However, this
will not protect you from active attacks, such as Man-In-The-Middle attacks
which cause you to connect to the wrong server and pretend to be your mail
server.
+
DO NOT RELY ON STARTTLS AS A SAFE CONNECTION GUARANTEEING THE AUTHENTICITY OF
YOUR IMAP SERVER!
Unix Signals
------------
OfflineImap listens to the unix signals SIGUSR1, SIGUSR2, SIGTERM, SIGINT,
SIGHUP, SIGQUIT.
* If sent a SIGUSR1 it will abort any current (or next future) sleep of all
accounts that are configured to "autorefresh". In effect, this will trigger a
full sync of all accounts to be performed as soon as possible.
* If sent a SIGUSR2, it will stop "autorefresh mode" for all accounts. That
is, accounts will abort any current sleep and will exit after a currently
running synchronization has finished. This signal can be used to gracefully
exit out of a running offlineimap "daemon".
* SIGTERM, SIGINT, SIGHUP are all treated to gracefully terminate as soon as
possible. This means it will finish syncing the current folder in each
account, close keep alive connections, remove locks on the accounts and exit.
+
It may take up to 10 seconds, if autorefresh option is used.
* If sent SIGQUIT, dumps stack traces for all threads and tries to dump
process core.
Known Issues
------------
* SSL3 write pending.
+
Users enabling SSL may hit a bug about "SSL3 write pending". If so, the
account(s) will stay unsynchronised from the time the bug appeared. Running
OfflineIMAP again can help. We are still working on this bug. Patches or
detailed bug reports would be appreciated. Please check you're running the
last stable version and send us a report to the mailing list including the
full log.
* IDLE support is incomplete and experimental. Bugs may be encountered.
* No hook exists for "run after an IDLE response".
+
Email will show up, but may not be processed until the next refresh cycle.
* nametrans may not be supported correctly.
* IMAP IDLE <-> IMAP IDLE doesn't work yet.
* IDLE may only work "once" per refresh.
+
If you encounter this bug, please send a report to the list!
* Maildir support in Windows drive.
+
Maildir uses colon caracter (:) in message file names. Colon is however
forbidden character in windows drives. There are several workarounds for that
situation:
* Use "maildir-windows-compatible = yes" account OfflineIMAP configuration.
- That makes OfflineIMAP to use exclamation mark (!) instead of colon for
storing messages. Such files can be written to windows partitions. But
you will probably loose compatibility with other programs trying to
read the same Maildir.
- Exclamation mark was chosen because of the note in
http://docs.python.org/library/mailbox.html
- If you have some messages already stored without this option, you will
have to re-sync them again
* Enable file name character translation in windows registry (not tested).
- http://support.microsoft.com/kb/289627
* Use cygwin managed mount (not tested).
- not available anymore since cygwin 1.7
* OfflineIMAP confused after system suspend.
+
When resuming a suspended session, OfflineIMAP does not cleanly handles the
broken socket(s) if socktimeout option is not set.
You should enable this option with a value like 10.
* Sharing a maildir with multiple IMAP servers.
+
Generally a word of caution mixing IMAP repositories on the same Maildir root.
You have to be careful that you *never* use the same maildir folder for 2 IMAP
servers. In the best case, the folder MD5 will be different, and you will get
a loop where it will upload your mails to both servers in turn (infinitely!)
as it thinks you have placed new mails in the local Maildir. In the worst
case, the MD5 is the same (likely) and mail UIDs overlap (likely too!) and it
will fail to sync some mails as it thinks they are already existent.
+
I would create a new local Maildir Repository for the Personal Gmail and
use a different root to be on the safe side here. You could e.g. use
`~/mail/Pro` as Maildir root for the ProGmail and
`~/mail/Personal` as root for the personal one.
+
If you then point your local mutt, or whatever MUA you use to `~/mail/`
as root, it should still recognize all folders.
Authors
-------
John Goerzen, Sebastian Spaetz, Eygene Ryabinkin, Nicolas Sebrecht.
See Also
--------
offlineimapui(7), openssl(1), signal(7), sqlite3(1).
http://offlineimap.github.io

140
docs/offlineimapui.txt Normal file
View File

@ -0,0 +1,140 @@
offlineimapui(7)
================
NAME
----
offlineimapui - The User Interfaces
DESCRIPTION
-----------
OfflineIMAP comes with differents UI, each aiming its own purpose.
TTYUI
------
TTYUI interface is for people running in terminals. It prints out basic
status messages and is generally friendly to use on a console or xterm.
Basic
------
Basic is designed for situations in which OfflineIMAP will be run non-attended
and the status of its execution will be logged.
+
This user interface is not capable of reading a password from the keyboard;
account passwords must be specified using one of the configuration file
options. For example, it will not print periodic sleep announcements and tends
to be a tad less verbose, in general.
Blinkenlights
-------------
Blinkenlights is an interface designed to be sleek, fun to watch, and
informative of the overall picture of what OfflineIMAP is doing.
Blinkenlights contains a row of "LEDs" with command buttons and a log. The
log shows more detail about what is happening and is color-coded to match the
color of the lights.
Each light in the Blinkenlights interface represents a thread of execution --
that is, a particular task that OfflineIMAP is performing right now. The
colors indicate what task the particular thread is performing, and are as
follows:
* Black
indicates that this light's thread has terminated; it will light up again
later when new threads start up. So, black indicates no activity.
* Red (Meaning 1)
is the color of the main program's thread, which basically does nothing but
monitor the others. It might remind you of HAL 9000 in 2001.
* Gray
indicates that the thread is establishing a new connection to the IMAP
server.
* Purple
is the color of an account synchronization thread that is monitoring the
progress of the folders in that account (not generating any I/O).
* Cyan
indicates that the thread is syncing a folder.
* Green
means that a folder's message list is being loaded.
* Blue
is the color of a message synchronization controller thread.
* Orange
indicates that an actual message is being copied. (We use fuchsia for fake
messages.)
* Red (meaning 2)
indicates that a message is being deleted.
* Yellow / bright orange
indicates that message flags are being added.
* Pink / bright red
indicates that message flags are being removed.
* Red / Black Flashing
corresponds to the countdown timer that runs between synchronizations.
The name of this interfaces derives from a bit of computer history. Eric
Raymond's Jargon File defines blinkenlights, in part, as:
Front-panel diagnostic lights on a computer, esp. a dinosaur. Now that
dinosaurs are rare, this term usually refers to status lights on a modem,
network hub, or the like.
This term derives from the last word of the famous blackletter-Gothic sign in
mangled pseudo-German that once graced about half the computer rooms in the
English-speaking world. One version ran in its entirety as follows:
ACHTUNG! ALLES LOOKENSPEEPERS!
Das computermachine ist nicht fuer gefingerpoken und mittengrabben.
Ist easy schnappen der springenwerk, blowenfusen und poppencorken
mit spitzensparken. Ist nicht fuer gewerken bei das dumpkopfen.
Das rubbernecken sichtseeren keepen das cotten-pickenen hans in das
pockets muss; relaxen und watchen das blinkenlichten.
Quiet
-----
It will output nothing except errors and serious warnings. Like Basic, this
user interface is not capable of reading a password from the keyboard; account
passwords must be specified using one of the configuration file options.
MachineUI
---------
MachineUI generates output in a machine-parsable format. It is designed
for other programs that will interface to OfflineIMAP.
See Also
--------
offlineima(1)

28
offlineimap.conf
View File

@ -2,9 +2,27 @@
# This file documents *all* possible options and can be quite scary.
# Looking for a quick start? Take a look at offlineimap.conf.minimal.
# More details can be found in the included user documention, which is
# also available at: http://docs.offlineimap.org/en/latest/
# More details can be found at http://offlineimap.github.io.
##################################################
# Overview
##################################################
# The default configuration file is "~/.offlineimaprc".
#
# OfflineIMAP ships with a file named "offlineimap.conf" that you should copy to
# that location and then edit.
#
# OfflineIMAP also ships a file named "offlineimap.conf.minimal" that you can
# also try. It's useful if you want to get started with the most basic feature
# set, and you can read about other features later with "offlineimap.conf".
#
# If you want to be XDG-compatible, you can put your configuration file into
# "$XDG_CONFIG_HOME/offlineimap/config".
##################################################
# General definitions
##################################################
# NOTE 1: Settings generally support python interpolation. This means
# values can contain python format strings which refer to other values
@ -31,10 +49,6 @@
# as it coincides with typical shell expansion strategy.
##################################################
# General definitions
##################################################
[general]
# This specifies where OfflineIMAP is to store its metadata.
@ -84,6 +98,8 @@ accounts = Test
# MachineUI -- Interactive interface suitable for machine
# parsing.
#
# See also offlineimapui(7)
#
# You can override this with a command-line option -u.
#
#ui = basic

80
offlineimap/init.py
View File

@ -58,108 +58,54 @@ class OfflineImap:
parser.add_option("--dry-run",
action="store_true", dest="dryrun",
default=False,
help="Do not actually modify any store but check and print "
"what synchronization actions would be taken if a sync would be"
" performed. It will not precisely give the exact information w"
"hat will happen. If e.g. we need to create a folder, it merely"
" outputs 'Would create folder X', but not how many and which m"
"ails it would transfer.")
help="dry run mode")
parser.add_option("--info",
action="store_true", dest="diagnostics",
default=False,
help="Output information on the configured email repositories"
". Useful for debugging and bug reporting. Use in conjunction wit"
"h the -a option to limit the output to a single account. This mo"
"de will prevent any actual sync to occur and exits after it outp"
"ut the debug information.")
help="output information on the configured email repositories")
parser.add_option("-1",
action="store_true", dest="singlethreading",
default=False,
help="Disable all multithreading operations and use "
"solely a single-thread sync. This effectively sets the "
"maxsyncaccounts and all maxconnections configuration file "
"variables to 1.")
help="disable all multithreading operations")
parser.add_option("-P", dest="profiledir", metavar="DIR",
help="Sets OfflineIMAP into profile mode. The program "
"will create DIR (it must not already exist). "
"As it runs, Python profiling information about each "
"thread is logged into profiledir. Please note: "
"This option is present for debugging and optimization "
"only, and should NOT be used unless you have a "
"specific reason to do so. It will significantly "
"decrease program performance, may reduce reliability, "
"and can generate huge amounts of data. This option "
"implies the -1 option.")
help="sets OfflineIMAP into profile mode.")
parser.add_option("-a", dest="accounts", metavar="ACCOUNTS",
help="Overrides the accounts section in the config file. "
"Lets you specify a particular account or set of "
"accounts to sync without having to edit the config "
"file. You might use this to exclude certain accounts, "
"or to sync some accounts that you normally prefer not to.")
help="overrides the accounts section in the config file")
parser.add_option("-c", dest="configfile", metavar="FILE",
default=None,
help="Specifies a configuration file to use")
help="specifies a configuration file to use")
parser.add_option("-d", dest="debugtype", metavar="type1,[type2...]",
help="Enables debugging for OfflineIMAP. This is useful "
"if you are to track down a malfunction or figure out what is "
"going on under the hood. This option requires one or more "
"debugtypes, separated by commas. These define what exactly "
"will be debugged, and so far include two options: imap, thread, "
"maildir or ALL. The imap option will enable IMAP protocol "
"stream and parsing debugging. Note that the output may contain "
"passwords, so take care to remove that from the debugging "
"output before sending it to anyone else. The maildir option "
"will enable debugging for certain Maildir operations. "
"The use of any debug option (unless 'thread' is included), "
"implies the single-thread option -1.")
help="enables debugging for OfflineIMAP.")
parser.add_option("-l", dest="logfile", metavar="FILE",
help="Log to FILE")
help="log to FILE")
parser.add_option("-f", dest="folders", metavar="folder1,[folder2...]",
help="Only sync the specified folders. The folder names "
"are the *untranslated* foldernames of the remote repository. "
"This command-line option overrides any 'folderfilter' "
"and 'folderincludes' options in the configuration file.")
help="only sync the specified folders")
parser.add_option("-k", dest="configoverride",
action="append",
metavar="[section:]option=value",
help=
"""Override configuration file option. If"section" is
omitted, it defaults to "general". Any underscores
in the section name are replaced with spaces:
for instance, to override option "autorefresh" in
the "[Account Personal]" section in the config file
one would use "-k Account_Personal:autorefresh=30".""")
help="override configuration file option")
parser.add_option("-o",
action="store_true", dest="runonce",
default=False,
help="Run only once, ignoring any autorefresh setting "
"in the configuration file.")
help="run only once")
parser.add_option("-q",
action="store_true", dest="quick",
default=False,
help="Run only quick synchronizations. Ignore any "
"flag updates on IMAP servers (if a flag on the remote IMAP "
"changes, and we have the message locally, it will be left "
"untouched in a quick run.")
help="run only quick synchronizations")
parser.add_option("-u", dest="interface",
help="Specifies an alternative user interface to "
"use. This overrides the default specified in the "
"configuration file. The UI specified with -u will "
"be forced to be used, even if checks determine that it is "
"not usable. Possible interface choices are: %s " %
", ".join(UI_LIST.keys()))
help="specifies an alternative user interface")
(options, args) = parser.parse_args()
globals.set_options (options)