Merge branch 'next'

Conflicts:
	Changelog.draft.rst
	docs/MANUAL.rst

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
This commit is contained in:
Sebastian Spaeth 2012-04-02 23:50:58 +02:00
commit 4b323f7532
57 changed files with 1482 additions and 989 deletions

1
.gitignore vendored
View File

@ -6,4 +6,5 @@ offlineimap.1
# backups # backups
.*.swp .*.swp
.*.swo .*.swo
*.html
*~ *~

10
COPYING
View File

@ -1,3 +1,13 @@
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
GNU GENERAL PUBLIC LICENSE GNU GENERAL PUBLIC LICENSE
Version 2, June 1991 Version 2, June 1991

View File

@ -1,17 +0,0 @@
offlineimap Mail syncing software
Copyright (C) 2002 - 2009 John Goerzen
<jgoerzen@complete.org>
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA

View File

@ -1,24 +0,0 @@
=========
ChangeLog
=========
Users should ignore this content: **it is draft**.
Contributors should add entries here in the following section, on top of the
others.
`WIP (coming releases)`
=======================
New Features
------------
Changes
-------
* internal code changes to prepare for Python3
* Improve user documentation of nametrans/folderfilter
Bug Fixes
---------

View File

@ -5,11 +5,52 @@ ChangeLog
:website: http://offlineimap.org :website: http://offlineimap.org
**NOTE FROM THE MAINTAINER:** WIP (add new stuff for the next release)
Contributors should use the `WIP` section in Changelog.draft.rst in order to ========================================
add changes they are working on. I will use it to make the new changelog entry
on releases. And because I'm lazy, it will also be used as a draft for the New Features
releases announces. ------------
Changes
-------
Bug Fixes
---------
OfflineIMAP v6.5.3 (2012-04-02)
===============================
* --dry-run mode protects us from performing any actual action. It will
not precisely give the exact information what will happen. If e.g. it
would need to create a folder, it merely outputs "Would create folder
X", but not how many and which mails it would transfer.
* internal code changes to prepare for Python3
* Improve user documentation of nametrans/folderfilter
* Fixed some cases where invalid nametrans rules were not caught and
we would not propagate local folders to the remote repository.
(now tested in test03)
* Revert "* Slight performance enhancement uploading mails to an IMAP
server in the common case." It might have led to instabilities.
* Revamped documentation structure. `make` in the `docs` dir or `make
doc` in the root dir will now create the 1) man page and 2) the user
documentation using sphinx (requiring python-doctools, and
sphinx). The resulting user docs are in `docs/html`. You can also
only create the man pages with `make man` in the `docs` dir.
* -f command line option only works on the untranslated remote
repository folder names now. Previously folderfilters had to match
both the local AND remote name which caused unwanted behavior in
combination with nametrans rules. Clarify in the help text.
* Some better output when using nonsensical configuration settings
* Improve compatability of the curses UI with python 2.6
OfflineIMAP v6.5.2.1 (2012-04-04) OfflineIMAP v6.5.2.1 (2012-04-04)
===================================== =====================================
@ -514,9 +555,8 @@ I'd like to thank reporters who involved in this cycle:
- Pan Tsu - Pan Tsu
- Vincent Beffara - Vincent Beffara
- Will Styler - Will Styler
(my apologies if I forget somebody)
...and all active developers, of course! (my apologies if I forget somebody) ...and all active developers, of course!
The imaplib2 migration looks to go the right way to be definetly released but The imaplib2 migration looks to go the right way to be definetly released but
still needs more tests. So, here we go... still needs more tests. So, here we go...
@ -595,9 +635,10 @@ OfflineIMAP v6.3.2 (2010-02-21)
Notes Notes
----- -----
First of all I'm really happy to announce our new official `website`_. Most of First of all I'm really happy to announce our new official `website
the work started from the impulse of Philippe LeCavalier with the help of <http://offlineimap.org>`_. Most of the work started from the impulse
Sebastian Spaeth and other contributors. Thanks to everybody. of Philippe LeCavalier with the help of Sebastian Spaeth and other
contributors. Thanks to everybody.
In this release, we are still touched by the "SSL3 write pending" but I think In this release, we are still touched by the "SSL3 write pending" but I think
time was long enough to try to fix it. We have our first entry in the "KNOWN time was long enough to try to fix it. We have our first entry in the "KNOWN

View File

@ -45,8 +45,6 @@ man:
doc: doc:
@$(MAKE) -C docs @$(MAKE) -C docs
$(RST2HTML) README.rst readme.html
$(RST2HTML) SubmittingPatches.rst SubmittingPatches.html
$(RST2HTML) Changelog.rst Changelog.html $(RST2HTML) Changelog.rst Changelog.html
targz: ../$(TARGZ) targz: ../$(TARGZ)

212
README Normal file
View File

@ -0,0 +1,212 @@
OfflineImap README
==================
Description
-----------
OfflineIMAP is a tool to simplify your e-mail reading. With OfflineIMAP, you can
read the same mailbox from multiple computers. You get a current copy of your
messages on each computer, and changes you make one place will be visible on all
other systems. For instance, you can delete a message on your home computer, and
it will appear deleted on your work computer as well. OfflineIMAP is also useful
if you want to use a mail reader that does not have IMAP support, has poor IMAP
support, or does not provide disconnected operation. It's homepage at http://offlineimap.org contains more information, source code, and online documentation.
OfflineIMAP does not require additional python dependencies beyond python >=2.6
(although python-sqlite is strongly recommended).
OfflineIMAP is a Free Software project licensed under the GNU General Public
License version 2 (or later). You can download it for free, and you can modify
it. In fact, you are encouraged to contribute to OfflineIMAP.
Documentation
-------------
The documentation is included (in .rst format) in the `docs` directory.
Read it directly or generate nice html docs (python-sphinx needed) and/or
the man page (python-docutils needed) while being in the `docs` dir via::
'make doc' (user docs), 'make man' (man page only) or 'make' (both)
(`make html` will simply create html versions of all *.rst files in /docs)
The resulting user documentation will be in `docs/html`. The full user
docs are also at: http://docs.offlineimap.org. Please see there for
detailed information on how to install and configure OfflineImap.
Quick Start
===========
First, install OfflineIMAP. See docs/INSTALL.rst or read
http://docs.offlineimap.org/en/latest/INSTALL.html.
(hint: `sudo python setup.py install`)
Second, set up your configuration file and run it! The distribution
includes offlineimap.conf.minimal (Debian users may find this at
``/usr/share/doc/offlineimap/examples/offlineimap.conf.minimal``) that
provides you with the bare minimum of setting up OfflineIMAP. You can
simply copy this file into your home directory and name it
``.offlineimaprc``. A command such as ``cp offlineimap.conf.minimal
~/.offlineimaprc`` will do it. Or, if you prefer, you can just copy
this text to ``~/.offlineimaprc``::
[general]
accounts = Test
[Account Test]
localrepository = Local
remoterepository = Remote
[Repository Local]
type = Maildir
localfolders = ~/Test
[Repository Remote]
type = IMAP
remotehost = examplehost
remoteuser = jgoerzen
Now, edit the ``~/.offlineimaprc`` file with your favorite editor. All you have
to do is specify a directory for your folders to be in (on the localfolders
line), the host name of your IMAP server (on the remotehost line), and your
login name on the remote (on the remoteuser line). That's it!
To run OfflineIMAP, you just have to say `offlineimap` ― it will fire
up, ask you for a login password if necessary, synchronize your folders,
and exit. See?
You can just throw away the rest of the finely-crafted, perfectly-honed user
manual! Of course, if you want to see how you can make OfflineIMAP
FIVE TIMES FASTER FOR JUST $19.95 (err, well, $0), you have to read on our
full user documentation and peruse the sample offlineimap.conf (which
includes all available options) for further tweaks!
Mailing list & bug reporting
----------------------------
The user discussion, development and all exciting stuff take place in the
OfflineImap mailing list at http://lists.alioth.debian.org/mailman/listinfo/offlineimap-project. You do not need to subscribe to send emails.
Bugs, issues and contributions should be reported to the mailing list. Bugs can also be reported in the issue tracker at https://github.com/spaetz/offlineimap/issues.
Configuration Examples
======================
Here are some example configurations for various situations. Please e-mail any
other examples you have that may be useful to me.
Multiple Accounts with Mutt
---------------------------
This example shows you how to set up OfflineIMAP to synchronize multiple
accounts with the mutt mail reader.
Start by creating a directory to hold your folders by running ``mkdir ~/Mail``.
Then, in your ``~/.offlineimaprc``, specify::
accounts = Personal, Work
Make sure that you have both an [Account Personal] and an [Account Work]
section. The local repository for each account must have different localfolder
path names. Also, make sure to enable [mbnames].
In each local repository section, write something like this::
localfolders = ~/Mail/Personal
Finally, add these lines to your ``~/.muttrc``::
source ~/path-to-mbnames-muttrc-mailboxes
folder-hook Personal set from="youremail@personal.com"
folder-hook Work set from="youremail@work.com"
set mbox_type=Maildir
set folder=$HOME/Mail
spoolfile=+Personal/INBOX
That's it!
UW-IMAPD and References
-----------------------
Some users with a UW-IMAPD server need to use OfflineIMAP's "reference" feature
to get at their mailboxes, specifying a reference of ``~/Mail`` or ``#mh/``
depending on the configuration. The below configuration from (originally from
docwhat@gerf.org) shows using a reference of Mail, a nametrans that strips the
leading Mail/ off incoming folder names, and a folderfilter that limits the
folders synced to just three::
[Account Gerf]
localrepository = GerfLocal
remoterepository = GerfRemote
[Repository GerfLocal]
type = Maildir
localfolders = ~/Mail
[Repository GerfRemote]
type = IMAP
remotehost = gerf.org
ssl = yes
remoteuser = docwhat
reference = Mail
# Trims off the preceeding Mail on all the folder names.
nametrans = lambda foldername: \
re.sub('^Mail/', '', foldername)
# Yeah, you have to mention the Mail dir, even though it
# would seem intuitive that reference would trim it.
folderfilter = lambda foldername: foldername in [
'Mail/INBOX',
'Mail/list/zaurus-general',
'Mail/list/zaurus-dev',
]
maxconnections = 1
holdconnectionopen = no
pythonfile Configuration File Option
-------------------------------------
You can have OfflineIMAP load up a Python file before evaluating the
configuration file options that are Python expressions. This example is based
on one supplied by Tommi Virtanen for this feature.
In ~/.offlineimaprc, he adds these options::
[general]
pythonfile=~/.offlineimap.py
[Repository foo]
foldersort=mycmp
Then, the ~/.offlineimap.py file will contain::
prioritized = ['INBOX', 'personal', 'announce', 'list']
def mycmp(x, y):
for prefix in prioritized:
xsw = x.startswith(prefix)
ysw = y.startswith(prefix)
if xsw and ysw:
return cmp(x, y)
elif xsw:
return -1
elif ysw:
return +1
return cmp(x, y)
def test_mycmp():
import os, os.path
folders=os.listdir(os.path.expanduser('~/data/mail/tv@hq.yok.utu.fi'))
folders.sort(mycmp)
print folders
This code snippet illustrates how the foldersort option can be customized with a
Python function from the pythonfile to always synchronize certain folders first.

View File

@ -1,296 +0,0 @@
.. -*- coding: utf-8 -*-
.. _mailing list: http://lists.alioth.debian.org/mailman/listinfo/offlineimap-project
======
README
======
.. contents::
.. sectnum::
Description
===========
Welcome to the official OfflineIMAP project.
*NOTICE:* this software was written by John Goerzen, who retired from
maintaining. It is now maintained by Nicolas Sebrecht at
https://github.com/nicolas33/offlineimap. Thanks to John for his great job and
to have share this project with us.
OfflineIMAP is a tool to simplify your e-mail reading. With OfflineIMAP, you can
read the same mailbox from multiple computers. You get a current copy of your
messages on each computer, and changes you make one place will be visible on all
other systems. For instance, you can delete a message on your home computer, and
it will appear deleted on your work computer as well. OfflineIMAP is also useful
if you want to use a mail reader that does not have IMAP support, has poor IMAP
support, or does not provide disconnected operation.
OfflineIMAP works on pretty much any POSIX operating system, such as Linux, BSD
operating systems, MacOS X, Solaris, etc.
OfflineIMAP is a Free Software project licensed under the GNU General Public
License. You can download it for free, and you can modify it. In fact, you are
encouraged to contribute to OfflineIMAP, and doing so is fast and easy.
OfflineIMAP is FAST; it synchronizes my two accounts with over 50 folders in 3
seconds. Other similar tools might take over a minute, and achieve a
less-reliable result. Some mail readers can take over 10 minutes to do the same
thing, and some don't even support it at all. Unlike other mail tools,
OfflineIMAP features a multi-threaded synchronization algorithm that can
dramatically speed up performance in many situations by synchronizing several
different things simultaneously.
OfflineIMAP is FLEXIBLE; you can customize which folders are synced via regular
expressions, lists, or Python expressions; a versatile and comprehensive
configuration file is used to control behavior; two user interfaces are
built-in; fine-tuning of synchronization performance is possible; internal or
external automation is supported; SSL and PREAUTH tunnels are both supported;
offline (or "unplugged") reading is supported; and esoteric IMAP features are
supported to ensure compatibility with the widest variety of IMAP servers.
OfflineIMAP is SAFE; it uses an algorithm designed to prevent mail loss at all
costs. Because of the design of this algorithm, even programming errors should
not result in loss of mail. I am so confident in the algorithm that I use my
own personal and work accounts for testing of OfflineIMAP pre-release,
development, and beta releases. Of course, legally speaking, OfflineIMAP comes
with no warranty, so I am not responsible if this turns out to be wrong.
Method of Operation
===================
OfflineIMAP traditionally operates by maintaining a hierarchy of mail folders in
Maildir format locally. Your own mail reader will read mail from this tree, and
need never know that the mail comes from IMAP. OfflineIMAP will detect changes
to the mail folders on your IMAP server and your own computer and
bi-directionally synchronize them, copying, marking, and deleting messages as
necessary.
With OfflineIMAP 4.0, a powerful new ability has been introduced ― the program
can now synchronize two IMAP servers with each other, with no need to have a
Maildir layer in-between. Many people use this if they use a mail reader on
their local machine that does not support Maildirs. People may install an IMAP
server on their local machine, and point both OfflineIMAP and their mail reader
of choice at it. This is often preferable to the mail reader's own IMAP support
since OfflineIMAP supports many features (offline reading, for one) that most
IMAP-aware readers don't. However, this feature is not as time-tested as
traditional syncing, so my advice is to stick with normal methods of operation
for the time being.
Quick Start
===========
If you have already installed OfflineIMAP system-wide, or your system
administrator has done that for you, your task for setting up OfflineIMAP for
the first time is quite simple. You just need to set up your configuration
file, make your folder directory, and run it!
You can quickly set up your configuration file. The distribution includes a
file offlineimap.conf.minimal (Debian users may find this at
``/usr/share/doc/offlineimap/examples/offlineimap.conf.minimal``) that is a
basic example of setting of OfflineIMAP. You can simply copy this file into
your home directory and name it ``.offlineimaprc`` (note the leading period). A
command such as ``cp offlineimap.conf.minimal ~/.offlineimaprc`` will do it.
Or, if you prefer, you can just copy this text to ``~/.offlineimaprc``::
[general]
accounts = Test
[Account Test]
localrepository = Local
remoterepository = Remote
[Repository Local]
type = Maildir
localfolders = ~/Test
[Repository Remote]
type = IMAP
remotehost = examplehost
remoteuser = jgoerzen
Now, edit the ``~/.offlineimaprc`` file with your favorite editor. All you have
to do is specify a directory for your folders to be in (on the localfolders
line), the host name of your IMAP server (on the remotehost line), and your
login name on the remote (on the remoteuser line). That's it!
To run OfflineIMAP, you just have to say offlineimap ― it will fire up, ask you
for a login password if necessary, synchronize your folders, and exit. See?
You can just throw away the rest of this finely-crafted, perfectly-honed manual!
Of course, if you want to see how you can make OfflineIMAP FIVE TIMES FASTER FOR
JUST $19.95 (err, well, $0), you have to read on!
Documentation
=============
If you are reading this file on github, you can find more documentations in the
`docs` directory.
Using your git repository, you can generate documentation with::
$ make doc
Mailing list
============
The user discussion, development and all exciting stuff take place in the
`mailing list`_. You're *NOT* supposed to subscribe to send emails.
Reporting bugs and contributions
================================
Bugs
----
Bugs, issues and contributions should be reported to the `mailing list`_.
**Please, don't use the github features (messages, pull requests, etc) at all.
It would most likely be discarded or ignored.**
========
Examples
========
Here are some example configurations for various situations. Please e-mail any
other examples you have that may be useful to me.
Multiple Accounts with Mutt
===========================
This example shows you how to set up OfflineIMAP to synchronize multiple
accounts with the mutt mail reader.
Start by creating a directory to hold your folders by running ``mkdir ~/Mail``.
Then, in your ``~/.offlineimaprc``, specify::
accounts = Personal, Work
Make sure that you have both an [Account Personal] and an [Account Work]
section. The local repository for each account must have different localfolder
path names. Also, make sure to enable [mbnames].
In each local repository section, write something like this::
localfolders = ~/Mail/Personal
Finally, add these lines to your ``~/.muttrc``::
source ~/path-to-mbnames-muttrc-mailboxes
folder-hook Personal set from="youremail@personal.com"
folder-hook Work set from="youremail@work.com"
set mbox_type=Maildir
set folder=$HOME/Mail
spoolfile=+Personal/INBOX
That's it!
UW-IMAPD and References
=======================
Some users with a UW-IMAPD server need to use OfflineIMAP's "reference" feature
to get at their mailboxes, specifying a reference of ``~/Mail`` or ``#mh/``
depending on the configuration. The below configuration from (originally from
docwhat@gerf.org) shows using a reference of Mail, a nametrans that strips the
leading Mail/ off incoming folder names, and a folderfilter that limits the
folders synced to just three::
[Account Gerf]
localrepository = GerfLocal
remoterepository = GerfRemote
[Repository GerfLocal]
type = Maildir
localfolders = ~/Mail
[Repository GerfRemote]
type = IMAP
remotehost = gerf.org
ssl = yes
remoteuser = docwhat
reference = Mail
# Trims off the preceeding Mail on all the folder names.
nametrans = lambda foldername: \
re.sub('^Mail/', '', foldername)
# Yeah, you have to mention the Mail dir, even though it
# would seem intuitive that reference would trim it.
folderfilter = lambda foldername: foldername in [
'Mail/INBOX',
'Mail/list/zaurus-general',
'Mail/list/zaurus-dev',
]
maxconnections = 1
holdconnectionopen = no
pythonfile Configuration File Option
====================================
You can have OfflineIMAP load up a Python file before evaluating the
configuration file options that are Python expressions. This example is based
on one supplied by Tommi Virtanen for this feature.
In ~/.offlineimaprc, he adds these options::
[general]
pythonfile=~/.offlineimap.py
[Repository foo]
foldersort=mycmp
Then, the ~/.offlineimap.py file will contain::
prioritized = ['INBOX', 'personal', 'announce', 'list']
def mycmp(x, y):
for prefix in prioritized:
xsw = x.startswith(prefix)
ysw = y.startswith(prefix)
if xsw and ysw:
return cmp(x, y)
elif xsw:
return -1
elif ysw:
return +1
return cmp(x, y)
def test_mycmp():
import os, os.path
folders=os.listdir(os.path.expanduser('~/data/mail/tv@hq.yok.utu.fi'))
folders.sort(mycmp)
print folders
This code snippet illustrates how the foldersort option can be customized with a
Python function from the pythonfile to always synchronize certain folders first.
Signals
=======
OfflineIMAP writes its current PID into ``~/.offlineimap/pid`` when it is
running. It is not guaranteed that this file will not exist when OfflineIMAP is
not running.
<!-- not done yet
You can send SIGINT to OfflineIMAP using this file to kill it. SIGUSR1 will
force an immediate resync of all accounts. This will be ignored for all
accounts for which a resync is already in progress.
-->

View File

@ -1,188 +0,0 @@
.. -*- coding: utf-8 -*-
.. _OfflineIMAP: https://github.com/nicolas33/offlineimap
===================
Hacking OfflineIMAP
===================
Welcome to the `OfflineIMAP`_ project. You'll find here all the information you
need to start hacking OfflineIMAP. Be aware there are a lot of very usefull tips
in the mailing list. You may want to subscribe if you didn't, yet. This is
where you'll get help.
.. contents::
.. sectnum::
=================================
Git: Branching Model And Workflow
=================================
Introduction
============
In order to involve into OfflineIMAP you need some knowledges about Git and our
workflow. Don't be afraid if you don't know much, we would be pleased to help
you.
You can find the API docs autogenerated on http://docs.offlineimap.org.
Release cycles
==============
We use a classical cycle based workflow:
1. A stable release is out.
2. Feature topics are sent, discussed and merged.
3. When enough work was merged, we start the freeze cycle: the first release
candidate is out.
4. During the freeze cycle, no more features are merged. It's time to test
OfflineIMAP. New candidates version are released. The more we are late in -rc
releases the less patches are merged but bug fixes.
5. When we think a release is stable enough, we restart from step 1.
Branching model
===============
The branching model with use in OfflineIMAP is very near from the Git project.
We use a topic oriented workflow. A topic may be one or more patches.
The branches you'll find in the official repository are:
* gh-pages
* master
* next
* pu
* maint
gh-pages
--------
This comes from a feature offered by Github. We maintain the online home github
page using this branch.
master
------
If you're not sure what branch you should use, this one is for you. This is the
mainline. Simple users should use this branch to follow OfflineIMAP's evolution.
Usually, patches submitted to the mailing list should start off of this branch.
next
----
Patches recently merged are good candidates for this branch. The content of next
is merged into the mainline (master) at release time for both stable and -rc
releases.
When patches are sent to the mailing list, contributors discuss about them. Once
done and when patches looks ready for mainline, patches are first merged into
next. Advanced users and testers use this branch to test last merged patches
before they hit the mainline. This helps not introducing strong breackages
directly in master.
pu
--
pu stands for "proposed updates". If a topic is not ready for master nor next,
it may be merged into pu. This branch only help developers to work on someone
else topic or an earlier pending topic.
This branch is **not intended to be checkouted**; never. Even developers don't
do that. Due to the way pu is built you can't expect content there to work in
any way... unless you clearly want to run into troubles.
Developers can extract a topic from this branch to work on it. See the following
section "Extract a topic from pu" in this documentation.
maint
-----
This is the maintenance branch. It gets its own releases starting from an old
stable release. It helps both users having troubles with last stable releases
and users not wanting latest features or so to still benefit from strong bug
fixes and security fixes.
Working with Git
================
Extract a topic from pu
-----------------------
pu is built this way::
git checkout pu
git reset --keep next
git merge --no-ff -X theirs topic1
git merge --no-ff -X theirs topic2
git merge --no-ff -X theirs blue
git merge --no-ff -X theirs orange
...
As a consequence:
1. Each topic merged uses a merge commit. A merge commit is a commit having 2
ancestors. Actually, Git allows more than 2 parents but we don't use this
feature. It's intended.
2. Paths in pu may mix up multiple versions if all the topics don't use the same
base commit. This is very often the case as topics aren't rebased: it guarantees
each topic is strictly identical to the last version sent to the mailing list.
No surprise.
What you need to extract a particular topic is the sha1 of the tip of that
branch (the last commit of the topic). Assume you want the branch of the topic
called 'blue'. First, look at the log given by this command::
git log --reverse --merges --parents origin/next..origin/pu
With this command you ask for the log:
* from next to pu
* in reverse order (older first)
* merge commits only
* with the sha1 of the ancestors
In this list, find the topic you're looking for, basing you search on the lines
like::
Merge branch 'topic/name' into pu
By convention, it has the form <author_initials>/<brief_title>. When you're at
it, pick the topic ancestor sha1. It's always the last sha1 in the line starting
by 'commit'. For you to know:
* the first is the sha1 of the commit you see: the merge commit
* the following sha1 is the ancestor of the branch checkouted at merge time
(always the previous merged topic or the ancien next in our case)
* last is the branch merged
Giving::
commit sha1_of_merge_commit sha1_of_ancient_pu sha1_of_topic_blue
Then, you only have to checkout the topic from there::
git checkout -b blue sha1_of_topic_blue
and you're done! You've just created a new branch called "blue" with the blue
content. Be aware this topic is almostly not updated against current next
branch. ,-)
===
API
===
API is documented in the dev-doc-src directory using the sphinx tools (also used
for python itself). This is a WIP. Contributions in this area would be very
appreciated.

View File

@ -2,17 +2,14 @@
OfflineIMAP Manual OfflineIMAP Manual
==================== ====================
.. _OfflineIMAP: http://offlineimap.org
-------------------------------------------------------- --------------------------------------------------------
Powerful IMAP/Maildir synchronization and reader support Powerful IMAP/Maildir synchronization and reader support
-------------------------------------------------------- --------------------------------------------------------
:Author: John Goerzen <jgoerzen@complete.org> & contributors :Author: John Goerzen <jgoerzen@complete.org> & contributors
:Date: 2011-01-15 :Date: 2012-02-23
:Copyright: GPL v2
:Manual section: 1
.. TODO: :Manual group:
DESCRIPTION DESCRIPTION
=========== ===========
@ -43,6 +40,10 @@ Most configuration is done via the configuration file. However, any setting can
OfflineImap is well suited to be frequently invoked by cron jobs, or can run in daemon mode to periodically check your email (however, it will exit in some error situations). OfflineImap is well suited to be frequently invoked by cron jobs, or can run in daemon mode to periodically check your email (however, it will exit in some error situations).
The documentation is included in the git repository and can be created by
issueing `make dev-doc` in the `doc` folder (python-sphinx required), or it can
be viewed online at http://docs.offlineimap.org.
.. _configuration: .. _configuration:
Configuration Configuration
@ -66,96 +67,9 @@ Check out the `Use Cases`_ section for some example configurations.
OPTIONS OPTIONS
======= =======
The command line options are described by issueing `offlineimap --help`.
-1 Disable most multithreading operations Details on their use can be found either in the sample offlineimap.conf file or
in the user docs at http://docs.offlineimap.org.
Use solely a single-connection sync. This effectively sets the
maxsyncaccounts and all maxconnections configuration file variables to 1.
-P profiledir
Sets OfflineIMAP into profile mode. The program will create profiledir (it
must not already exist). As it runs, Python profiling information about each
thread is logged into profiledir. Please note: This option is present for
debugging and optimization only, and should NOT be used unless you have a
specific reason to do so. It will significantly slow program performance, may
reduce reliability, and can generate huge amounts of data. You must use the
-1 option when you use -P.
-a accountlist
Overrides the accounts option in the general section of the configuration
file. You might use this to exclude certain accounts, or to sync some
accounts that you normally prefer not to. Separate the accounts by commas,
and use no embedded spaces.
-c configfile
Specifies a configuration file to use in lieu of the default,
``~/.offlineimaprc``.
-d debugtype[,...]
Enables debugging for OfflineIMAP. This is useful if you are trying to track
down a malfunction or figure out what is going on under the hood. I suggest
that you use this with -1 to make the results more sensible.
-d requires one or more debugtypes, separated by commas. These define what
exactly will be debugged, and include three options: imap, maildir, and
thread. The imap option will enable IMAP protocol stream and parsing
debugging. Note that the output may contain passwords, so take care to remove
that from the debugging output before sending it to anyone else. The maildir
option will enable debugging for certain Maildir operations. And thread will
debug the threading model.
-f foldername[,foldername]
Only sync the specified folders. The foldernames are the untranslated
foldernames. This command-line option overrides any folderfilter and
folderincludes options in the configuration file.
-k [section:]option=value
Override configuration file option. If "section" is omitted, it defaults to
general. Any underscores "_" in the section name are replaced with spaces:
for instance, to override option autorefresh in the "[Account Personal]"
section in the config file one would use "-k Account_Personal:autorefresh=30".
You may give more than one -k on the command line if you wish.
-l filename
Enables logging to filename. This will log everything that goes to the screen
to the specified file. Additionally, if any debugging is specified with -d,
then debug messages will not go to the screen, but instead to the logfile
only.
-o Run only once,
ignoring all autorefresh settings in the configuration file.
-q Run only quick synchronizations.
Ignore any flag updates on IMAP servers.
-h|--help Show summary of options.
-u interface
Specifies an alternative user interface module to use. This overrides the
default specified in the configuration file. The pre-defined options are
listed in the User Interfaces section. The interface name is case insensitive.
User Interfaces User Interfaces
=============== ===============
@ -389,6 +303,8 @@ as Man-In-The-Middle attacks which cause you to connect to the wrong
server and pretend to be your mail server. DO NOT RELY ON STARTTLS AS A server and pretend to be your mail server. DO NOT RELY ON STARTTLS AS A
SAFE CONNECTION GUARANTEEING THE AUTHENTICITY OF YOUR IMAP SERVER! SAFE CONNECTION GUARANTEEING THE AUTHENTICITY OF YOUR IMAP SERVER!
.. _UNIX signals:
UNIX Signals UNIX Signals
============ ============
@ -528,7 +444,7 @@ and Sent which should keep the same name::
Synchronizing 2 IMAP accounts to local Maildirs that are "next to each Synchronizing 2 IMAP accounts to local Maildirs that are "next to each
other", so that mutt can work on both. Full email setup described by other", so that mutt can work on both. Full email setup described by
Thomas Kahle at `http://dev.gentoo.org/~tomka/mail.html`_ Thomas Kahle at `<http://dev.gentoo.org/~tomka/mail.html>`_
offlineimap.conf:: offlineimap.conf::
@ -590,7 +506,7 @@ purposes: Fetching passwords from the gnome-keyring and translating
folder names on the server to local foldernames. An example folder names on the server to local foldernames. An example
implementation of get_username and get_password showing how to query implementation of get_username and get_password showing how to query
gnome-keyring is contained in gnome-keyring is contained in
`http://dev.gentoo.org/~tomka/mail-setup.tar.bz2`_ The folderfilter is `<http://dev.gentoo.org/~tomka/mail-setup.tar.bz2>`_ The folderfilter is
a lambda term that, well, filters which folders to get. The function a lambda term that, well, filters which folders to get. The function
`oimaptransfolder_acc2` translates remote folders into local folders `oimaptransfolder_acc2` translates remote folders into local folders
with a very simple logic. The `INBOX` folder will have the same name with a very simple logic. The `INBOX` folder will have the same name

View File

@ -9,7 +9,7 @@ RST2HTML=`type rst2html >/dev/null 2>&1 && echo rst2html || echo rst2html.py`
RST2MAN=`type rst2man >/dev/null 2>&1 && echo rst2man || echo rst2man.py` RST2MAN=`type rst2man >/dev/null 2>&1 && echo rst2man || echo rst2man.py`
SPHINXBUILD = sphinx-build SPHINXBUILD = sphinx-build
all: html dev-doc all: man doc
html: $(HTML_TARGETS) html: $(HTML_TARGETS)
@ -22,12 +22,12 @@ offlineimap.1: MANUAL.rst
$(RST2MAN) MANUAL.rst offlineimap.1 $(RST2MAN) MANUAL.rst offlineimap.1
cp -f offlineimap.1 .. cp -f offlineimap.1 ..
dev-doc: doc:
$(SPHINXBUILD) -b html -d dev-doc/doctrees dev-doc-src dev-doc/html $(SPHINXBUILD) -b html -d html/doctrees doc-src html
clean: clean:
$(RM) -f $(HTML_TARGETS) $(RM) -f $(HTML_TARGETS)
$(RM) -f offlineimap.1 ../offlineimap.1 $(RM) -f offlineimap.1 ../offlineimap.1
$(RM) -rf dev-doc/* $(RM) -rf html/*
.PHONY: dev-doc .PHONY: clean doc

View File

@ -1,26 +0,0 @@
Upgrading to 4.0
----------------
If you are upgrading from a version of OfflineIMAP prior to 3.99.12, you will
find that you will get errors when OfflineIMAP starts up (relating to
ConfigParser or AccountHashGenerator) and the configuration file. This is
because the config file format had to change to accommodate new features in 4.0.
Fortunately, it's not difficult to adjust it to suit.
First thing you need to do is stop any running OfflineIMAP instance, making sure
first that it's synced all your mail. Then, modify your `~/.offlineimaprc` file.
You'll need to split up each account section (make sure that it now starts with
"Account ") into two Repository sections (one for the local side and another for
the remote side.) See the files offlineimap.conf.minimal and offlineimap.conf
in the distribution if you need more assistance.
OfflineIMAP's status directory area has also changed. Therefore, you should
delete everything in `~/.offlineimap` as well as your local mail folders.
When you start up OfflineIMAP 4.0, it will re-download all your mail from the
server and then you can continue using it like normal.

View File

@ -1 +0,0 @@
../FAQ.rst

View File

@ -2,8 +2,10 @@
.. currentmodule:: offlineimap .. currentmodule:: offlineimap
Welcome to :mod:`offlineimaps`'s documentation .. _API docs:
==============================================
:mod:`offlineimap's` API documentation
======================================
Within :mod:`offlineimap`, the classes :class:`OfflineImap` provides the high-level functionality. The rest of the classes should usually not needed to be touched by the user. Email repositories are represented by a :class:`offlineimap.repository.Base.BaseRepository` or derivatives (see :mod:`offlineimap.repository` for details). A folder within a repository is represented by a :class:`offlineimap.folder.Base.BaseFolder` or any derivative from :mod:`offlineimap.folder`. Within :mod:`offlineimap`, the classes :class:`OfflineImap` provides the high-level functionality. The rest of the classes should usually not needed to be touched by the user. Email repositories are represented by a :class:`offlineimap.repository.Base.BaseRepository` or derivatives (see :mod:`offlineimap.repository` for details). A folder within a repository is represented by a :class:`offlineimap.folder.Base.BaseFolder` or any derivative from :mod:`offlineimap.folder`.

View File

@ -203,9 +203,10 @@ How is OfflineIMAP conformance?
Can I force OfflineIMAP to sync a folder right now? Can I force OfflineIMAP to sync a folder right now?
--------------------------------------------------- ---------------------------------------------------
Yes, Yes:
1) if you use the `Blinkenlights` UI. That UI shows the active accounts
as follows:: 1) if you use the `Blinkenlights` UI. That UI shows the active
accounts as follows::
4: [active] *Control: . 4: [active] *Control: .
3: [ 4:36] personal: 3: [ 4:36] personal:
@ -216,8 +217,9 @@ as follows::
resync that account immediately. This will be ignored if a resync is resync that account immediately. This will be ignored if a resync is
already in progress for that account. already in progress for that account.
2) while in sleep mode, you can also send a SIGUSR1. See the `Signals 2) while in sleep mode, you can also send a SIGUSR1. See the :ref:`UNIX
on UNIX`_ section in the MANUAL for details. signals` section in the MANUAL for details.
I get a "Mailbox already exists" error I get a "Mailbox already exists" error
-------------------------------------- --------------------------------------
@ -291,14 +293,17 @@ certificates chain) in PEM format. (See the documentation of
`ssl.wrap_socket`_'s `certfile` parameter for the gory details.) You can use either openssl or gnutls to create a certificate file in the required format. `ssl.wrap_socket`_'s `certfile` parameter for the gory details.) You can use either openssl or gnutls to create a certificate file in the required format.
#. via openssl:: #. via openssl::
openssl s_client -CApath /etc/ssl/certs -connect ${hostname}:imaps -showcerts \ openssl s_client -CApath /etc/ssl/certs -connect ${hostname}:imaps -showcerts \
| perl -ne 'print if /BEGIN/../END/; print STDERR if /return/' > $sslcacertfile | perl -ne 'print if /BEGIN/../END/; print STDERR if /return/' > $sslcacertfile
^D ^D
#. via gnutls:: #. via gnutls::
gnutls-cli --print-cert -p imaps ${host} </dev/null | sed -n \ gnutls-cli --print-cert -p imaps ${host} </dev/null | sed -n \
| '/^-----BEGIN CERT/,/^-----END CERT/p' > $sslcacertfile | '/^-----BEGIN CERT/,/^-----END CERT/p' > $sslcacertfile
The path `/etc/ssl/certs` is not standardized; your system may store The path `/etc/ssl/certs` is not standardized; your system may store
SSL certificates elsewhere. (On some systems it may be in SSL certificates elsewhere. (On some systems it may be in
`/usr/local/share/certs/`.) `/usr/local/share/certs/`.)

View File

@ -1,14 +1,134 @@
.. -*- coding: utf-8 -*- .. -*- coding: utf-8 -*-
.. _OfflineIMAP: http://offlineimap.org
.. _mailing list: http://lists.alioth.debian.org/mailman/listinfo/offlineimap-project
.. _commits mailing list: http://lists.offlineimap.org/listinfo.cgi/commits-offlineimap.org .. _commits mailing list: http://lists.offlineimap.org/listinfo.cgi/commits-offlineimap.org
.. _mailing list: http://lists.alioth.debian.org/mailman/listinfo/offlineimap-project
================================================= Hacking OfflineIMAP
Checklist (and a short version for the impatient) ===================
=================================================
Commits In this section you'll find all the information you need to start
======= hacking `OfflineIMAP`_. Be aware there are a lot of very usefull tips
in the mailing list. You may want to subscribe if you didn't,
yet. This is where you will get help.
.. contents:: :depth: 2
API
---
:ref:`OfflineImap's API <API docs>` documentation is included in the user
documentation (next section) and online browsable at
`<http://docs.offlineimap.org>`_. It is mostly auto-generated from the
source code and is a work in progress. Contributions in this area
would be very appreciated.
Following new commits
---------------------
You can follow upstream commits on
- `CIA.vc <http://cia.vc/stats/project/offlineimap>`,
- `Ohloh <http://www.ohloh.net/p/offlineimap>`,
- `GitHub <https://github.com/spaetz/offlineimap/commits/>`,
- or on the `commits mailing list`_.
Git: OfflineImap's branching Model And Workflow
===============================================
Introduction
------------
This optional section provides you with information on how we use git
branches and do releases. You will need to know very little about git
to get started.
For the impatient, see the :ref:`contribution checklist` below.
Git Branching model
--------------------
OfflineIMAP uses the following branches:
* master
* next
* maint
* (pu)
* & several topic oriented feature branches. A topic may consist of
one or more patches.
master
++++++
If you're not sure what branch you should use, this one is for you.
This is the mainline. Simple users should use this branch to follow
OfflineIMAP's evolution.
Usually, patches submitted to the mailing list should start off of
this branch.
next
++++
Patches recently merged are good candidates for this branch. The content of next
is merged into the mainline (master) at release time for both stable and -rc
releases.
When patches are sent to the mailing list, contributors discuss about them. Once
done and when patches looks ready for mainline, patches are first merged into
next. Advanced users and testers use this branch to test last merged patches
before they hit the mainline. This helps not introducing strong breackages
directly in master.
pu
+++
pu stands for "proposed updates". If a topic is not ready for master nor next,
it may be merged into pu. This branch only help developers to work on someone
else topic or an earlier pending topic.
This branch is **not intended to be checkouted**; never. Even developers don't
do that. Due to the way pu is built you can't expect content there to work in
any way... unless you clearly want to run into troubles.
Developers can extract a topic from this branch to work on it. See the following
section "Extract a topic from pu" in this documentation.
maint
+++++
This is the maintenance branch. It gets its own releases starting from an old
stable release. It helps both users having troubles with last stable releases
and users not wanting latest features or so to still benefit from strong bug
fixes and security fixes.
Release cycles
--------------
A typical release cycle works like this:
1. A stable release is out.
2. Feature topics are sent, discussed and merged.
3. When enough work was merged, we start the freeze cycle: the first release
candidate is out.
4. During the freeze cycle, no more features are merged. It's time to test
OfflineIMAP. New candidates version are released. The more we are late in -rc
releases the less patches are merged but bug fixes.
5. When we think a release is stable enough, we restart from step 1.
.. _contribution checklist:
Contribution Checklist (and a short version for the impatient)
===============================================================
Create commits
--------------
* make commits of logical units * make commits of logical units
* check for unnecessary whitespace with ``git diff --check`` * check for unnecessary whitespace with ``git diff --check``
@ -28,8 +148,9 @@ Commits
* make sure that you have tests for the bug you are fixing * make sure that you have tests for the bug you are fixing
* make sure that the test suite passes after your commit * make sure that the test suite passes after your commit
Patch
===== Export commits as patches
-------------------------
* use ``git format-patch -M`` to create the patch * use ``git format-patch -M`` to create the patch
* do not PGP sign your patch * do not PGP sign your patch
@ -52,9 +173,9 @@ Patch
* see below for instructions specific to your mailer * see below for instructions specific to your mailer
============
Long version Long version
============ ------------
I started reading over the SubmittingPatches document for Git, primarily because I started reading over the SubmittingPatches document for Git, primarily because
I wanted to have a document similar to it for OfflineIMAP to make sure people I wanted to have a document similar to it for OfflineIMAP to make sure people
@ -64,8 +185,8 @@ But the patch submission requirements are a lot more relaxed here on the
technical/contents front, because the OfflineIMAP is a lot smaller ;-). So here technical/contents front, because the OfflineIMAP is a lot smaller ;-). So here
is only the relevant bits. is only the relevant bits.
Decide what to base your work on Decide what branch to base your work on
================================ +++++++++++++++++++++++++++++++++++++++
In general, always base your work on the oldest branch that your In general, always base your work on the oldest branch that your
change is relevant to. change is relevant to.
@ -92,13 +213,12 @@ master..pu`` and look for the merge commit. The second parent of this
commit is the tip of the topic branch. commit is the tip of the topic branch.
Make separate commits for logically separate changes Make separate commits for logically separate changes
==================================================== ++++++++++++++++++++++++++++++++++++++++++++++++++++
Unless your patch is really trivial, you should not be sending Unless your patch is really trivial, you should not be sending your
out a patch that was generated between your working tree and changes in a single patch. Instead, always make a commit with
your commit head. Instead, always make a commit with complete complete commit message and generate a series of small patches from
commit message and generate a series of patches from your your repository.
repository. It is a good discipline.
Describe the technical detail of the change(s). Describe the technical detail of the change(s).
@ -115,7 +235,7 @@ but for code.
Generate your patch using git tools out of your commits Generate your patch using git tools out of your commits
------------------------------------------------------- +++++++++++++++++++++++++++++++++++++++++++++++++++++++
git based diff tools (git, Cogito, and StGIT included) generate git based diff tools (git, Cogito, and StGIT included) generate
unidiff which is the preferred format. unidiff which is the preferred format.
@ -133,7 +253,7 @@ that is fine, but please mark it as such.
Sending your patches Sending your patches
==================== ++++++++++++++++++++
People on the mailing list need to be able to read and People on the mailing list need to be able to read and
comment on the changes you are submitting. It is important for comment on the changes you are submitting. It is important for
@ -205,7 +325,7 @@ necessary.
Sign your work Sign your work
============== ++++++++++++++
To improve tracking of who did what, we've borrowed the To improve tracking of who did what, we've borrowed the
"sign-off" procedure from the Linux kernel project on patches "sign-off" procedure from the Linux kernel project on patches
@ -218,7 +338,7 @@ the right to pass it on as a open-source patch**. The rules are
pretty simple: if you can certify the below: pretty simple: if you can certify the below:
**Developer's Certificate of Origin 1.1** **Developer's Certificate of Origin 1.1**
----------------------------------------- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
By making a contribution to this project, I certify that: By making a contribution to this project, I certify that:
@ -321,12 +441,6 @@ Know the status of your patch after submission
tell you if your patch is merged in pu if you rebase on top of tell you if your patch is merged in pu if you rebase on top of
master). master).
* You can follow upstream commits on
`CIA.vc <http://cia.vc/stats/project/offlineimap>`,
`Ohloh <http://www.ohloh.net/p/offlineimap>`,
`GitHub <https://github.com/spaetz/offlineimap/commits/>`,
or on the `commits mailing list`_.
.. * Read the git mailing list, the maintainer regularly posts messages .. * Read the git mailing list, the maintainer regularly posts messages
entitled "What's cooking in git.git" and "What's in git.git" giving entitled "What's cooking in git.git" and "What's in git.git" giving
the status of various proposed changes. the status of various proposed changes.
@ -601,3 +715,69 @@ Just make sure to disable line wrapping in the email client (GMail web
interface will line wrap no matter what, so you need to use a real interface will line wrap no matter what, so you need to use a real
IMAP client). IMAP client).
Working with Git
================
Extract a topic from pu
-----------------------
pu is built this way::
git checkout pu
git reset --keep next
git merge --no-ff -X theirs topic1
git merge --no-ff -X theirs topic2
git merge --no-ff -X theirs blue
git merge --no-ff -X theirs orange
...
As a consequence:
1. Each topic merged uses a merge commit. A merge commit is a commit having 2
ancestors. Actually, Git allows more than 2 parents but we don't use this
feature. It's intended.
2. Paths in pu may mix up multiple versions if all the topics don't use the same
base commit. This is very often the case as topics aren't rebased: it guarantees
each topic is strictly identical to the last version sent to the mailing list.
No surprise.
What you need to extract a particular topic is the sha1 of the tip of that
branch (the last commit of the topic). Assume you want the branch of the topic
called 'blue'. First, look at the log given by this command::
git log --reverse --merges --parents origin/next..origin/pu
With this command you ask for the log:
* from next to pu
* in reverse order (older first)
* merge commits only
* with the sha1 of the ancestors
In this list, find the topic you're looking for, basing you search on the lines
like::
Merge branch 'topic/name' into pu
By convention, it has the form <author_initials>/<brief_title>. When you're at
it, pick the topic ancestor sha1. It's always the last sha1 in the line starting
by 'commit'. For you to know:
* the first is the sha1 of the commit you see: the merge commit
* the following sha1 is the ancestor of the branch checkouted at merge time
(always the previous merged topic or the ancien next in our case)
* last is the branch merged
Giving::
commit sha1_of_merge_commit sha1_of_ancient_pu sha1_of_topic_blue
Then, you only have to checkout the topic from there::
git checkout -b blue sha1_of_topic_blue
and you're done! You've just created a new branch called "blue" with the blue
content. Be aware this topic is almostly not updated against current next
branch. ,-)

66
docs/doc-src/features.rst Normal file
View File

@ -0,0 +1,66 @@
Description
===========
OfflineIMAP is a tool to simplify your e-mail reading. With OfflineIMAP, you can
read the same mailbox from multiple computers. You get a current copy of your
messages on each computer, and changes you make one place will be visible on all
other systems. For instance, you can delete a message on your home computer, and
it will appear deleted on your work computer as well. OfflineIMAP is also useful
if you want to use a mail reader that does not have IMAP support, has poor IMAP
support, or does not provide disconnected operation.
OfflineIMAP works on pretty much any POSIX operating system, such as Linux, BSD
operating systems, MacOS X, Solaris, etc.
OfflineIMAP is a Free Software project licensed under the GNU General Public
License. You can download it for free, and you can modify it. In fact, you are
encouraged to contribute to OfflineIMAP, and doing so is fast and easy.
OfflineIMAP is FAST; it synchronizes my two accounts with over 50 folders in 3
seconds. Other similar tools might take over a minute, and achieve a
less-reliable result. Some mail readers can take over 10 minutes to do the same
thing, and some don't even support it at all. Unlike other mail tools,
OfflineIMAP features a multi-threaded synchronization algorithm that can
dramatically speed up performance in many situations by synchronizing several
different things simultaneously.
OfflineIMAP is FLEXIBLE; you can customize which folders are synced via regular
expressions, lists, or Python expressions; a versatile and comprehensive
configuration file is used to control behavior; two user interfaces are
built-in; fine-tuning of synchronization performance is possible; internal or
external automation is supported; SSL and PREAUTH tunnels are both supported;
offline (or "unplugged") reading is supported; and esoteric IMAP features are
supported to ensure compatibility with the widest variety of IMAP servers.
OfflineIMAP is SAFE; it uses an algorithm designed to prevent mail loss at all
costs. Because of the design of this algorithm, even programming errors should
not result in loss of mail. I am so confident in the algorithm that I use my
own personal and work accounts for testing of OfflineIMAP pre-release,
development, and beta releases. Of course, legally speaking, OfflineIMAP comes
with no warranty, so I am not responsible if this turns out to be wrong.
.. note: OfflineImap was written by John Goerzen, who retired from
maintaining. It is now maintained by Nicolas Sebrecht & Sebastian
Spaeth at https://github.com/spaetz/offlineimap. Thanks to John
for his great job and to have share this project with us.
Method of Operation
===================
OfflineIMAP traditionally operates by maintaining a hierarchy of mail folders in
Maildir format locally. Your own mail reader will read mail from this tree, and
need never know that the mail comes from IMAP. OfflineIMAP will detect changes
to the mail folders on your IMAP server and your own computer and
bi-directionally synchronize them, copying, marking, and deleting messages as
necessary.
With OfflineIMAP 4.0, a powerful new ability has been introduced ― the program
can now synchronize two IMAP servers with each other, with no need to have a
Maildir layer in-between. Many people use this if they use a mail reader on
their local machine that does not support Maildirs. People may install an IMAP
server on their local machine, and point both OfflineIMAP and their mail reader
of choice at it. This is often preferable to the mail reader's own IMAP support
since OfflineIMAP supports many features (offline reading, for one) that most
IMAP-aware readers don't. However, this feature is not as time-tested as
traditional syncing, so my advice is to stick with normal methods of operation
for the time being.

View File

@ -15,6 +15,7 @@ If you just want to get started with minimal fuzz, have a look at our `online qu
More information on specific topics can be found on the following pages: More information on specific topics can be found on the following pages:
**User documentation** **User documentation**
* :doc:`Overview and features <features>`
* :doc:`installation/uninstall <INSTALL>` * :doc:`installation/uninstall <INSTALL>`
* :doc:`user manual/Configuration <MANUAL>` * :doc:`user manual/Configuration <MANUAL>`
* :doc:`Folder filtering & name transformation guide <nametrans>` * :doc:`Folder filtering & name transformation guide <nametrans>`
@ -22,18 +23,21 @@ More information on specific topics can be found on the following pages:
* :doc:`Frequently Asked Questions <FAQ>` * :doc:`Frequently Asked Questions <FAQ>`
**Developer documentation** **Developer documentation**
* :doc:`HACKING HowTo & git workflows <HACKING>`
* :doc:`API documentation <API>` for internal details on the * :doc:`API documentation <API>` for internal details on the
:mod:`offlineimap` module :mod:`offlineimap` module
.. toctree:: .. toctree::
:hidden: :hidden:
features
INSTALL INSTALL
MANUAL MANUAL
nametrans nametrans
offlineimap offlineimap
FAQ FAQ
HACKING
API API
repository repository
ui ui

184
docs/doc-src/nametrans.rst Normal file
View File

@ -0,0 +1,184 @@
.. _folder_filtering_and_name_translation:
Folder filtering and Name translation
=====================================
OfflineImap provides advanced and potentially complex possibilities for
filtering and translating folder names. If you don't need any of this, you can
safely skip this section.
.. warning::
Starting with v6.4.0, OfflineImap supports the creation of folders on the remote repostory. This change means that people that only had a nametrans option on the remote repository (everyone) will need to have a nametrans setting on the local repository too that will reverse the name transformation. See section `Reverse nametrans`_ for details.
folderfilter
------------
If you do not want to synchronize all your filters, you can specify a `folderfilter`_ function that determines which folders to include in a sync and which to exclude. Typically, you would set a folderfilter option on the remote repository only, and it would be a lambda or any other python function.
The only parameter to that function is the folder name. If the filter
function returns True, the folder will be synced, if it returns False,
it. will be skipped. The folderfilter operates on the *UNTRANSLATED*
name (before any `nametrans`_ fudging takes place). Consider the
examples below to get an idea of what they do.
Example 1: synchronizing only INBOX and Sent::
folderfilter = lambda folder: folder in ['INBOX', 'Sent']
Example 2: synchronizing everything except Trash::
folderfilter = lambda folder: folder not in ['Trash']
Example 3: Using a regular expression to exclude Trash and all folders
containing the characters "Del"::
folderfilter = lambda folder: not re.search('(^Trash$|Del)', folder)
.. note::
If folderfilter is not specified, ALL remote folders will be
synchronized.
You can span multiple lines by indenting the others. (Use backslashes
at the end when required by Python syntax) For instance::
folderfilter = lambda foldername: foldername in
['INBOX', 'Sent Mail', 'Deleted Items',
'Received']
Usually it suffices to put a `folderfilter`_ setting in the remote repository section. You might want to put a folderfilter option on the local repository if you want to prevent some folders on the local repository to be created on the remote one. (Even in this case, folder filters on the remote repository will prevent that)
folderincludes
--------------
You can specify `folderincludes`_ to manually include additional folders to be synced, even if they had been filtered out by a folderfilter setting. `folderincludes`_ should return a Python list.
This can be used to 1) add a folder that was excluded by your
folderfilter rule, 2) to include a folder that your server does not specify
with its LIST option, or 3) to include a folder that is outside your basic
`reference`. The `reference` value will not be prefixed to this folder
name, even if you have specified one. For example::
folderincludes = ['debian.user', 'debian.personal']
This will add the "debian.user" and "debian.personal" folders even if you
have filtered out everything starting with "debian" in your folderfilter
settings.
nametrans
----------
Sometimes, folders need to have different names on the remote and the
local repositories. To achieve this you can specify a folder name
translator. This must be a eval-able Python expression that takes a
foldername arg and returns the new value. We suggest a lambda function,
but it could be any python function really. If you use nametrans rules, you will need to set them both on the remote and the local repository, see `Reverse nametrans`_ just below for details. The following examples are thought to be put in the remote repository section.
The below will remove "INBOX." from the leading edge of folders (great
for Courier IMAP users)::
nametrans = lambda folder: re.sub('^INBOX\.', '', folder)
Using Courier remotely and want to duplicate its mailbox naming
locally? Try this::
nametrans = lambda folder: re.sub('^INBOX\.*', '.', folder)
.. warning::
You MUST construct nametrans rules such that it NEVER returns the
same value for two folders, UNLESS the second values are filtered
out by folderfilter below. That is, two filters on one side may
never point to the same folder on the other side. Failure to follow
this rule will result in undefined behavior. See also *Sharing a
maildir with multiple IMAP servers* in the :ref:`pitfalls` section.
Reverse nametrans
+++++++++++++++++
Since 6.4.0, OfflineImap supports the creation of folders on the remote repository and that complicates things. Previously, only one nametrans setting on the remote repository was needed and that transformed a remote to a local name. However, nametrans transformations are one-way, and OfflineImap has no way using those rules on the remote repository to back local names to remote names.
Take a remote nametrans rule `lambda f: re.sub('^INBOX/','',f)` which cuts of any existing INBOX prefix. Now, if we parse a list of local folders, finding e.g. a folder "Sent", is it supposed to map to "INBOX/Sent" or to "Sent"? We have no way of knowing. This is why **every nametrans setting on a remote repository requires an equivalent nametrans rule on the local repository that reverses the transformation**.
Take the above examples. If your remote nametrans setting was::
nametrans = lambda folder: re.sub('^INBOX\.', '', folder)
then you will want to have this in your local repository, prepending "INBOX" to any local folder name::
nametrans = lambda folder: 'INBOX' + folder
Failure to set the local nametrans rule will lead to weird-looking error messages of -for instance- this type::
ERROR: Creating folder moo.foo on repository remote
Folder 'moo.foo'[remote] could not be created. Server responded: ('NO', ['Unknown namespace.'])
(This indicates that you attempted to create a folder "Sent" when all remote folders needed to be under the prefix of "INBOX.").
OfflineImap will make some sanity checks if it needs to create a new
folder on the remote side and a back-and-forth nametrans-lation does not
yield the original foldername (as that could potentially lead to
infinite folder creation cycles).
You can probably already see now that creating nametrans rules can be a pretty daunting and complex endeavour. Check out the Use cases in the manual. If you have some interesting use cases that we can present as examples here, please let us know.
Debugging folderfilter and nametrans
------------------------------------
Given the complexity of the functions and regexes involved, it is easy to misconfigure things. One way to test your configuration without danger to corrupt anything or to create unwanted folders is to invoke offlineimap with the `--info` option.
It will output a list of folders and their transformations on the screen (save them to a file with -l info.log), and will help you to tweak your rules as well as to understand your configuration. It also provides good output for bug reporting.
FAQ on nametrans
----------------
Where to put nametrans rules, on the remote and/or local repository?
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
If you never intend to create new folders on the LOCAL repository that
need to be synced to the REMOTE repository, it is sufficient to create a
nametrans rule on the remote Repository section. This will be used to
determine the names of new folder names on the LOCAL repository, and to
match existing folders that correspond.
*IF* you create folders on the local repository, that are supposed to be
automatically created on the remote repository, you will need to create
a nametrans rule that provides the reverse name translation.
(A nametrans rule provides only a one-way translation of names and in
order to know which names folders on the LOCAL side would have on the
REMOTE side, you need to specify the reverse nametrans rule on the local
repository)
OfflineImap will complain if it needs to create a new folder on the
remote side and a back-and-forth nametrans-lation does not yield the
original foldername (as that could potentially lead to infinite folder
creation cycles).
What folder separators do I need to use in nametrans rules?
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
**Q:** If I sync from an IMAP server with folder separator '/' to a
Maildir using the default folder separator '.' which do I need to use
in nametrans rules?::
nametrans = lambda f: "INBOX/" + f
or::
nametrans = lambda f: "INBOX." + f
**A:** Generally use the folder separator as defined in the repository
you write the nametrans rule for. That is, use '/' in the above
case. We will pass in the untranslated name of the IMAP folder as
parameter (here `f`). The translated name will ultimately have all
folder separators be replaced with the destination repositories'
folder separator.
So if 'f' was "Sent", the first nametrans yields the translated name
"INBOX/Sent" to be used on the other side. As that repository uses the
folder separator '.' rather than '/', the ultimate name to be used will
be "INBOX.Sent".
(As a final note, the smart will see that both variants of the above
nametrans rule would have worked identically in this case)

View File

@ -6,6 +6,15 @@ Offlineimap is invoked with the following pattern: `offlineimap [args...]`.
Where [args...] are as follows: Where [args...] are as follows:
Options: Options:
--dry-run This mode protects us from performing any actual action.
It will not precisely give the exact information what
will happen. If e.g. it would need to create a folder,
it merely outputs "Would create folder X", but not how
many and which mails it would transfer.
--info Output information on the configured email
repositories. Useful for debugging and bug reporting.
Use in conjunction with the -a option to limit the
output to a single account.
--version show program's version number and exit --version show program's version number and exit
-h, --help show this help message and exit -h, --help show this help message and exit
-1 Disable all multithreading operations and use solely a -1 Disable all multithreading operations and use solely a

View File

@ -503,13 +503,14 @@ remoteuser = username
# one. For example: # one. For example:
# folderincludes = ['debian.user', 'debian.personal'] # folderincludes = ['debian.user', 'debian.personal']
# You can specify foldersort to determine how folders are sorted. # You can specify 'foldersort' to determine how folders are sorted.
# This affects order of synchronization and mbnames. The expression # This affects order of synchronization and mbnames. The expression
# should return -1, 0, or 1, as the default Python cmp() does. The # should return -1, 0, or 1, as the default Python cmp() does. The two
# two arguments, x and y, are strings representing the names of the folders # arguments, x and y, are strings representing the names of the folders
# to be sorted. The sorting is applied *AFTER* nametrans, if any. # to be sorted. The sorting is applied *AFTER* nametrans, if any. The
# # default is to sort IMAP folders alphabetically
# To reverse the sort: # (case-insensitive). Usually, you should never have to modify this. To
# eg. reverse the sort:
# #
# foldersort = lambda x, y: -cmp(x, y) # foldersort = lambda x, y: -cmp(x, y)

View File

@ -1,5 +1,4 @@
# Copyright (C) 2003 John Goerzen # Copyright (C) 2003-2012 John Goerzen & contributors
# <jgoerzen@complete.org>
# #
# This program is free software; you can redistribute it and/or modify # This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by # it under the terms of the GNU General Public License as published by
@ -15,7 +14,10 @@
# along with this program; if not, write to the Free Software # along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
from ConfigParser import SafeConfigParser try:
from ConfigParser import SafeConfigParser
except ImportError: #python3
from configparser import SafeConfigParser
from offlineimap.localeval import LocalEval from offlineimap.localeval import LocalEval
import os import os
@ -49,7 +51,7 @@ class CustomConfigParser(SafeConfigParser):
def getmetadatadir(self): def getmetadatadir(self):
metadatadir = os.path.expanduser(self.getdefault("general", "metadata", "~/.offlineimap")) metadatadir = os.path.expanduser(self.getdefault("general", "metadata", "~/.offlineimap"))
if not os.path.exists(metadatadir): if not os.path.exists(metadatadir):
os.mkdir(metadatadir, 0700) os.mkdir(metadatadir, 0o700)
return metadatadir return metadatadir
def getlocaleval(self): def getlocaleval(self):
@ -69,6 +71,14 @@ class CustomConfigParser(SafeConfigParser):
return [x[len(key):] for x in self.sections() \ return [x[len(key):] for x in self.sections() \
if x.startswith(key)] if x.startswith(key)]
def set_if_not_exists(self, section, option, value):
"""Set a value if it does not exist yet
This allows to set default if the user has not explicitly
configured anything."""
if not self.has_option(section, option):
self.set(section, option, value)
def CustomConfigDefault(): def CustomConfigDefault():
"""Just a constant that won't occur anywhere else. """Just a constant that won't occur anywhere else.

View File

@ -1,7 +1,7 @@
__all__ = ['OfflineImap'] __all__ = ['OfflineImap']
__productname__ = 'OfflineIMAP' __productname__ = 'OfflineIMAP'
__version__ = "6.5.2.1" __version__ = "6.5.3"
__copyright__ = "Copyright 2002-2012 John Goerzen & contributors" __copyright__ = "Copyright 2002-2012 John Goerzen & contributors"
__author__ = "John Goerzen" __author__ = "John Goerzen"
__author_email__= "john@complete.org" __author_email__= "john@complete.org"

View File

@ -65,9 +65,11 @@ class Account(CustomConfig.ConfigHelperMixin):
self.name = name self.name = name
self.metadatadir = config.getmetadatadir() self.metadatadir = config.getmetadatadir()
self.localeval = config.getlocaleval() self.localeval = config.getlocaleval()
#Contains the current :mod:`offlineimap.ui`, and can be used for logging etc. # current :mod:`offlineimap.ui`, can be used for logging:
self.ui = getglobalui() self.ui = getglobalui()
self.refreshperiod = self.getconffloat('autorefresh', 0.0) self.refreshperiod = self.getconffloat('autorefresh', 0.0)
# should we run in "dry-run" mode?
self.dryrun = self.config.getboolean('general', 'dry-run')
self.quicknum = 0 self.quicknum = 0
if self.refreshperiod == 0.0: if self.refreshperiod == 0.0:
self.refreshperiod = None self.refreshperiod = None
@ -199,7 +201,8 @@ class SyncableAccount(Account):
pass pass
except IOError: except IOError:
self._lockfd.close() self._lockfd.close()
raise OfflineImapError("Could not lock account %s." % self, raise OfflineImapError("Could not lock account %s. Is another "
"instance using this account?" % self,
OfflineImapError.ERROR.REPO) OfflineImapError.ERROR.REPO)
def unlock(self): def unlock(self):
@ -216,7 +219,7 @@ class SyncableAccount(Account):
self.ui.registerthread(self) self.ui.registerthread(self)
accountmetadata = self.getaccountmeta() accountmetadata = self.getaccountmeta()
if not os.path.exists(accountmetadata): if not os.path.exists(accountmetadata):
os.mkdir(accountmetadata, 0700) os.mkdir(accountmetadata, 0o700)
self.remoterepos = Repository(self, 'remote') self.remoterepos = Repository(self, 'remote')
self.localrepos = Repository(self, 'local') self.localrepos = Repository(self, 'local')
@ -231,7 +234,7 @@ class SyncableAccount(Account):
self.sync() self.sync()
except (KeyboardInterrupt, SystemExit): except (KeyboardInterrupt, SystemExit):
raise raise
except OfflineImapError, e: except OfflineImapError as e:
# Stop looping and bubble up Exception if needed. # Stop looping and bubble up Exception if needed.
if e.severity >= OfflineImapError.ERROR.REPO: if e.severity >= OfflineImapError.ERROR.REPO:
if looping: if looping:
@ -239,7 +242,7 @@ class SyncableAccount(Account):
if e.severity >= OfflineImapError.ERROR.CRITICAL: if e.severity >= OfflineImapError.ERROR.CRITICAL:
raise raise
self.ui.error(e, exc_info()[2]) self.ui.error(e, exc_info()[2])
except Exception, e: except Exception as e:
self.ui.error(e, exc_info()[2], msg = "While attempting to sync" self.ui.error(e, exc_info()[2], msg = "While attempting to sync"
" account '%s'" % self) " account '%s'" % self)
else: else:
@ -311,6 +314,8 @@ class SyncableAccount(Account):
# wait for all threads to finish # wait for all threads to finish
for thr in folderthreads: for thr in folderthreads:
thr.join() thr.join()
# Write out mailbox names if required and not in dry-run mode
if not self.dryrun:
mbnames.write() mbnames.write()
localrepos.forgetfolders() localrepos.forgetfolders()
remoterepos.forgetfolders() remoterepos.forgetfolders()
@ -336,6 +341,8 @@ class SyncableAccount(Account):
return return
try: try:
self.ui.callhook("Calling hook: " + cmd) self.ui.callhook("Calling hook: " + cmd)
if self.dryrun: # don't if we are in dry-run mode
return
p = Popen(cmd, shell=True, p = Popen(cmd, shell=True,
stdin=PIPE, stdout=PIPE, stderr=PIPE, stdin=PIPE, stdout=PIPE, stderr=PIPE,
close_fds=True) close_fds=True)
@ -344,7 +351,7 @@ class SyncableAccount(Account):
self.ui.callhook("Hook return code: %d" % p.returncode) self.ui.callhook("Hook return code: %d" % p.returncode)
except (KeyboardInterrupt, SystemExit): except (KeyboardInterrupt, SystemExit):
raise raise
except Exception, e: except Exception as e:
self.ui.error(e, exc_info()[2], msg = "Calling hook") self.ui.error(e, exc_info()[2], msg = "Calling hook")
def syncfolder(account, remotefolder, quick): def syncfolder(account, remotefolder, quick):
@ -445,7 +452,7 @@ def syncfolder(account, remotefolder, quick):
localrepos.restore_atime() localrepos.restore_atime()
except (KeyboardInterrupt, SystemExit): except (KeyboardInterrupt, SystemExit):
raise raise
except OfflineImapError, e: except OfflineImapError as e:
# bubble up severe Errors, skip folder otherwise # bubble up severe Errors, skip folder otherwise
if e.severity > OfflineImapError.ERROR.FOLDER: if e.severity > OfflineImapError.ERROR.FOLDER:
raise raise
@ -459,7 +466,7 @@ def syncfolder(account, remotefolder, quick):
# we reconstruct foldername above rather than using # we reconstruct foldername above rather than using
# localfolder, as the localfolder var is not # localfolder, as the localfolder var is not
# available if assignment fails. # available if assignment fails.
except Exception, e: except Exception as e:
ui.error(e, msg = "ERROR in syncfolder for %s folder %s: %s" % \ ui.error(e, msg = "ERROR in syncfolder for %s folder %s: %s" % \
(account, remotefolder.getvisiblename(), (account, remotefolder.getvisiblename(),
traceback.format_exc())) traceback.format_exc()))

View File

@ -23,10 +23,7 @@ import os.path
import re import re
from sys import exc_info from sys import exc_info
import traceback import traceback
try: # python 2.6 has set() built in
set
except NameError:
from sets import Set as set
class BaseFolder(object): class BaseFolder(object):
def __init__(self, name, repository): def __init__(self, name, repository):
@ -211,6 +208,10 @@ class BaseFolder(object):
If the uid is > 0, the backend should set the uid to this, if it can. If the uid is > 0, the backend should set the uid to this, if it can.
If it cannot set the uid to that, it will save it anyway. If it cannot set the uid to that, it will save it anyway.
It will return the uid assigned in any case. It will return the uid assigned in any case.
Note that savemessage() does not check against dryrun settings,
so you need to ensure that savemessage is never called in a
dryrun mode.
""" """
raise NotImplementedException raise NotImplementedException
@ -223,27 +224,48 @@ class BaseFolder(object):
raise NotImplementedException raise NotImplementedException
def savemessageflags(self, uid, flags): def savemessageflags(self, uid, flags):
"""Sets the specified message's flags to the given set.""" """Sets the specified message's flags to the given set.
Note that this function does not check against dryrun settings,
so you need to ensure that it is never called in a
dryrun mode."""
raise NotImplementedException raise NotImplementedException
def addmessageflags(self, uid, flags): def addmessageflags(self, uid, flags):
"""Adds the specified flags to the message's flag set. If a given """Adds the specified flags to the message's flag set. If a given
flag is already present, it will not be duplicated. flag is already present, it will not be duplicated.
Note that this function does not check against dryrun settings,
so you need to ensure that it is never called in a
dryrun mode.
:param flags: A set() of flags""" :param flags: A set() of flags"""
newflags = self.getmessageflags(uid) | flags newflags = self.getmessageflags(uid) | flags
self.savemessageflags(uid, newflags) self.savemessageflags(uid, newflags)
def addmessagesflags(self, uidlist, flags): def addmessagesflags(self, uidlist, flags):
"""
Note that this function does not check against dryrun settings,
so you need to ensure that it is never called in a
dryrun mode."""
for uid in uidlist: for uid in uidlist:
self.addmessageflags(uid, flags) self.addmessageflags(uid, flags)
def deletemessageflags(self, uid, flags): def deletemessageflags(self, uid, flags):
"""Removes each flag given from the message's flag set. If a given """Removes each flag given from the message's flag set. If a given
flag is already removed, no action will be taken for that flag.""" flag is already removed, no action will be taken for that flag.
Note that this function does not check against dryrun settings,
so you need to ensure that it is never called in a
dryrun mode."""
newflags = self.getmessageflags(uid) - flags newflags = self.getmessageflags(uid) - flags
self.savemessageflags(uid, newflags) self.savemessageflags(uid, newflags)
def deletemessagesflags(self, uidlist, flags): def deletemessagesflags(self, uidlist, flags):
"""
Note that this function does not check against dryrun settings,
so you need to ensure that it is never called in a
dryrun mode."""
for uid in uidlist: for uid in uidlist:
self.deletemessageflags(uid, flags) self.deletemessageflags(uid, flags)
@ -251,21 +273,34 @@ class BaseFolder(object):
"""Change the message from existing uid to new_uid """Change the message from existing uid to new_uid
If the backend supports it (IMAP does not). If the backend supports it (IMAP does not).
:param new_uid: (optional) If given, the old UID will be changed :param new_uid: (optional) If given, the old UID will be changed
to a new UID. This allows backends efficient renaming of to a new UID. This allows backends efficient renaming of
messages if the UID has changed.""" messages if the UID has changed."""
raise NotImplementedException raise NotImplementedException
def deletemessage(self, uid): def deletemessage(self, uid):
"""
Note that this function does not check against dryrun settings,
so you need to ensure that it is never called in a
dryrun mode."""
raise NotImplementedException raise NotImplementedException
def deletemessages(self, uidlist): def deletemessages(self, uidlist):
"""
Note that this function does not check against dryrun settings,
so you need to ensure that it is never called in a
dryrun mode."""
for uid in uidlist: for uid in uidlist:
self.deletemessage(uid) self.deletemessage(uid)
def copymessageto(self, uid, dstfolder, statusfolder, register = 1): def copymessageto(self, uid, dstfolder, statusfolder, register = 1):
"""Copies a message from self to dst if needed, updating the status """Copies a message from self to dst if needed, updating the status
Note that this function does not check against dryrun settings,
so you need to ensure that it is never called in a
dryrun mode.
:param uid: uid of the message to be copied. :param uid: uid of the message to be copied.
:param dstfolder: A BaseFolder-derived instance :param dstfolder: A BaseFolder-derived instance
:param statusfolder: A LocalStatusFolder instance :param statusfolder: A LocalStatusFolder instance
@ -321,11 +356,11 @@ class BaseFolder(object):
OfflineImapError.ERROR.MESSAGE) OfflineImapError.ERROR.MESSAGE)
except (KeyboardInterrupt): # bubble up CTRL-C except (KeyboardInterrupt): # bubble up CTRL-C
raise raise
except OfflineImapError, e: except OfflineImapError as e:
if e.severity > OfflineImapError.ERROR.MESSAGE: if e.severity > OfflineImapError.ERROR.MESSAGE:
raise # buble severe errors up raise # buble severe errors up
self.ui.error(e, exc_info()[2]) self.ui.error(e, exc_info()[2])
except Exception, e: except Exception as e:
self.ui.error(e, "Copying message %s [acc: %s]:\n %s" %\ self.ui.error(e, "Copying message %s [acc: %s]:\n %s" %\
(uid, self.accountname, (uid, self.accountname,
exc_info()[2])) exc_info()[2]))
@ -341,6 +376,8 @@ class BaseFolder(object):
2) invoke copymessageto() on those which: 2) invoke copymessageto() on those which:
- If dstfolder doesn't have it yet, add them to dstfolder. - If dstfolder doesn't have it yet, add them to dstfolder.
- Update statusfolder - Update statusfolder
This function checks and protects us from action in ryrun mode.
""" """
threads = [] threads = []
@ -348,6 +385,10 @@ class BaseFolder(object):
statusfolder.uidexists(uid), statusfolder.uidexists(uid),
self.getmessageuidlist()) self.getmessageuidlist())
num_to_copy = len(copylist) num_to_copy = len(copylist)
if num_to_copy and self.repository.account.dryrun:
self.ui.info("[DRYRUN] Copy {} messages from {}[{}] to {}".format(
num_to_copy, self, self.repository, dstfolder.repository))
return
for num, uid in enumerate(copylist): for num, uid in enumerate(copylist):
# bail out on CTRL-C or SIGTERM # bail out on CTRL-C or SIGTERM
if offlineimap.accounts.Account.abort_NOW_signal.is_set(): if offlineimap.accounts.Account.abort_NOW_signal.is_set():
@ -374,12 +415,17 @@ class BaseFolder(object):
Get all UIDS in statusfolder but not self. These are messages Get all UIDS in statusfolder but not self. These are messages
that were deleted in 'self'. Delete those from dstfolder and that were deleted in 'self'. Delete those from dstfolder and
statusfolder.""" statusfolder.
This function checks and protects us from action in ryrun mode.
"""
deletelist = filter(lambda uid: uid>=0 \ deletelist = filter(lambda uid: uid>=0 \
and not self.uidexists(uid), and not self.uidexists(uid),
statusfolder.getmessageuidlist()) statusfolder.getmessageuidlist())
if len(deletelist): if len(deletelist):
self.ui.deletingmessages(deletelist, [dstfolder]) self.ui.deletingmessages(deletelist, [dstfolder])
if self.repository.account.dryrun:
return #don't delete messages in dry-run mode
# delete in statusfolder first to play safe. In case of abort, we # delete in statusfolder first to play safe. In case of abort, we
# won't lose message, we will just retransmit some unneccessary. # won't lose message, we will just retransmit some unneccessary.
for folder in [statusfolder, dstfolder]: for folder in [statusfolder, dstfolder]:
@ -392,6 +438,8 @@ class BaseFolder(object):
msg has a valid UID and exists on dstfolder (has not e.g. been msg has a valid UID and exists on dstfolder (has not e.g. been
deleted there), sync the flag change to both dstfolder and deleted there), sync the flag change to both dstfolder and
statusfolder. statusfolder.
This function checks and protects us from action in ryrun mode.
""" """
# For each flag, we store a list of uids to which it should be # For each flag, we store a list of uids to which it should be
# added. Then, we can call addmessagesflags() to apply them in # added. Then, we can call addmessagesflags() to apply them in
@ -425,11 +473,15 @@ class BaseFolder(object):
for flag, uids in addflaglist.items(): for flag, uids in addflaglist.items():
self.ui.addingflags(uids, flag, dstfolder) self.ui.addingflags(uids, flag, dstfolder)
if self.repository.account.dryrun:
continue #don't actually add in a dryrun
dstfolder.addmessagesflags(uids, set(flag)) dstfolder.addmessagesflags(uids, set(flag))
statusfolder.addmessagesflags(uids, set(flag)) statusfolder.addmessagesflags(uids, set(flag))
for flag,uids in delflaglist.items(): for flag,uids in delflaglist.items():
self.ui.deletingflags(uids, flag, dstfolder) self.ui.deletingflags(uids, flag, dstfolder)
if self.repository.account.dryrun:
continue #don't actually remove in a dryrun
dstfolder.deletemessagesflags(uids, set(flag)) dstfolder.deletemessagesflags(uids, set(flag))
statusfolder.deletemessagesflags(uids, set(flag)) statusfolder.deletemessagesflags(uids, set(flag))
@ -474,11 +526,28 @@ class BaseFolder(object):
action(dstfolder, statusfolder) action(dstfolder, statusfolder)
except (KeyboardInterrupt): except (KeyboardInterrupt):
raise raise
except OfflineImapError, e: except OfflineImapError as e:
if e.severity > OfflineImapError.ERROR.FOLDER: if e.severity > OfflineImapError.ERROR.FOLDER:
raise raise
self.ui.error(e, exc_info()[2]) self.ui.error(e, exc_info()[2])
except Exception, e: except Exception as e:
self.ui.error(e, exc_info()[2], "Syncing folder %s [acc: %s]" %\ self.ui.error(e, exc_info()[2], "Syncing folder %s [acc: %s]" %\
(self, self.accountname)) (self, self.accountname))
raise # raise unknown Exceptions so we can fix them raise # raise unknown Exceptions so we can fix them
def __eq__(self, other):
"""Comparisons work either on string comparing folder names or
on the same instance
MailDirFolder('foo') == 'foo' --> True
a = MailDirFolder('foo'); a == b --> True
MailDirFolder('foo') == 'moo' --> False
MailDirFolder('foo') == IMAPFolder('foo') --> False
MailDirFolder('foo') == MaildirFolder('foo') --> False
"""
if isinstance(other, basestring):
return other == self.name
return id(self) == id(other)
def __ne__(self, other):
return not self.__eq__(other)

View File

@ -18,7 +18,7 @@
"""Folder implementation to support features of the Gmail IMAP server. """Folder implementation to support features of the Gmail IMAP server.
""" """
from IMAP import IMAPFolder from .IMAP import IMAPFolder
class GmailFolder(IMAPFolder): class GmailFolder(IMAPFolder):
"""Folder implementation to support features of the Gmail IMAP server. """Folder implementation to support features of the Gmail IMAP server.

View File

@ -1,5 +1,5 @@
# IMAP folder support # IMAP folder support
# Copyright (C) 2002-2011 John Goerzen & contributors # Copyright (C) 2002-2012 John Goerzen & contributors
# #
# This program is free software; you can redistribute it and/or modify # This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by # it under the terms of the GNU General Public License as published by
@ -21,13 +21,9 @@ import binascii
import re import re
import time import time
from sys import exc_info from sys import exc_info
from Base import BaseFolder from .Base import BaseFolder
from offlineimap import imaputil, imaplibutil, OfflineImapError from offlineimap import imaputil, imaplibutil, OfflineImapError
from offlineimap.imaplib2 import MonthNames from offlineimap.imaplib2 import MonthNames
try: # python 2.6 has set() built in
set
except NameError:
from sets import Set as set
class IMAPFolder(BaseFolder): class IMAPFolder(BaseFolder):
@ -97,7 +93,7 @@ class IMAPFolder(BaseFolder):
# Select folder and get number of messages # Select folder and get number of messages
restype, imapdata = imapobj.select(self.getfullname(), True, restype, imapdata = imapobj.select(self.getfullname(), True,
True) True)
except OfflineImapError, e: except OfflineImapError as e:
# retry on dropped connections, raise otherwise # retry on dropped connections, raise otherwise
self.imapserver.releaseconnection(imapobj, True) self.imapserver.releaseconnection(imapobj, True)
if e.severity == OfflineImapError.ERROR.FOLDER_RETRY: if e.severity == OfflineImapError.ERROR.FOLDER_RETRY:
@ -190,7 +186,7 @@ class IMAPFolder(BaseFolder):
continue continue
messagestr = messagestr.split(' ', 1)[1] messagestr = messagestr.split(' ', 1)[1]
options = imaputil.flags2hash(messagestr) options = imaputil.flags2hash(messagestr)
if not options.has_key('UID'): if not 'UID' in options:
self.ui.warn('No UID in message with options %s' %\ self.ui.warn('No UID in message with options %s' %\
str(options), str(options),
minor = 1) minor = 1)
@ -219,7 +215,7 @@ class IMAPFolder(BaseFolder):
res_type, data = imapobj.uid('fetch', str(uid), res_type, data = imapobj.uid('fetch', str(uid),
'(BODY.PEEK[])') '(BODY.PEEK[])')
fails_left = 0 fails_left = 0
except imapobj.abort, e: except imapobj.abort as e:
# Release dropped connection, and get a new one # Release dropped connection, and get a new one
self.imapserver.releaseconnection(imapobj, True) self.imapserver.releaseconnection(imapobj, True)
imapobj = self.imapserver.acquireconnection() imapobj = self.imapserver.acquireconnection()
@ -314,7 +310,7 @@ class IMAPFolder(BaseFolder):
headervalue = imapobj._quote(headervalue) headervalue = imapobj._quote(headervalue)
try: try:
matchinguids = imapobj.uid('search', 'HEADER', headername, headervalue)[1][0] matchinguids = imapobj.uid('search', 'HEADER', headername, headervalue)[1][0]
except imapobj.error, err: except imapobj.error as err:
# IMAP server doesn't implement search or had a problem. # IMAP server doesn't implement search or had a problem.
self.ui.debug('imap', "savemessage_searchforheader: got IMAP error '%s' while attempting to UID SEARCH for message with header %s" % (err, headername)) self.ui.debug('imap', "savemessage_searchforheader: got IMAP error '%s' while attempting to UID SEARCH for message with header %s" % (err, headername))
return 0 return 0
@ -328,8 +324,9 @@ class IMAPFolder(BaseFolder):
self.ui.debug('imap', 'savemessage_searchforheader: matchinguids now ' + \ self.ui.debug('imap', 'savemessage_searchforheader: matchinguids now ' + \
repr(matchinguids)) repr(matchinguids))
if len(matchinguids) != 1 or matchinguids[0] == None: if len(matchinguids) != 1 or matchinguids[0] == None:
raise ValueError, "While attempting to find UID for message with header %s, got wrong-sized matchinguids of %s" % (headername, str(matchinguids)) raise ValueError("While attempting to find UID for message with "
matchinguids.sort() "header %s, got wrong-sized matchinguids of %s" %\
(headername, str(matchinguids)))
return long(matchinguids[0]) return long(matchinguids[0])
def savemessage_fetchheaders(self, imapobj, headername, headervalue): def savemessage_fetchheaders(self, imapobj, headername, headervalue):
@ -491,12 +488,16 @@ class IMAPFolder(BaseFolder):
This function will update the self.messagelist dict to contain This function will update the self.messagelist dict to contain
the new message after sucessfully saving it. the new message after sucessfully saving it.
See folder/Base for details. Note that savemessage() does not
check against dryrun settings, so you need to ensure that
savemessage is never called in a dryrun mode.
:param rtime: A timestamp to be used as the mail date :param rtime: A timestamp to be used as the mail date
:returns: the UID of the new message as assigned by the server. If the :returns: the UID of the new message as assigned by the server. If the
message is saved, but it's UID can not be found, it will message is saved, but it's UID can not be found, it will
return 0. If the message can't be written (folder is return 0. If the message can't be written (folder is
read-only for example) it will return -1.""" read-only for example) it will return -1."""
self.ui.debug('imap', 'savemessage: called') self.ui.savemessage('imap', uid, flags, self)
# already have it, just save modified flags # already have it, just save modified flags
if uid > 0 and self.uidexists(uid): if uid > 0 and self.uidexists(uid):
@ -538,14 +539,13 @@ class IMAPFolder(BaseFolder):
self.ui.msgtoreadonly(self, uid, content, flags) self.ui.msgtoreadonly(self, uid, content, flags)
return uid return uid
# Clean out existing APPENDUID responses and do APPEND #Do the APPEND
try: try:
imapobj.response('APPENDUID') # flush APPENDUID responses (typ, dat) = imapobj.append(self.getfullname(),
typ, dat = imapobj.append(self.getfullname(),
imaputil.flagsmaildir2imap(flags), imaputil.flagsmaildir2imap(flags),
date, content) date, content)
retry_left = 0 # Mark as success retry_left = 0 # Mark as success
except imapobj.abort, e: except imapobj.abort as e:
# connection has been reset, release connection and retry. # connection has been reset, release connection and retry.
retry_left -= 1 retry_left -= 1
self.imapserver.releaseconnection(imapobj, True) self.imapserver.releaseconnection(imapobj, True)
@ -557,7 +557,7 @@ class IMAPFolder(BaseFolder):
(self, self.getrepository(), str(e), dbg_output), (self, self.getrepository(), str(e), dbg_output),
OfflineImapError.ERROR.MESSAGE) OfflineImapError.ERROR.MESSAGE)
self.ui.error(e, exc_info()[2]) self.ui.error(e, exc_info()[2])
except imapobj.error, e: # APPEND failed except imapobj.error as e: # APPEND failed
# If the server responds with 'BAD', append() # If the server responds with 'BAD', append()
# raise()s directly. So we catch that too. # raise()s directly. So we catch that too.
# drop conn, it might be bad. # drop conn, it might be bad.
@ -567,38 +567,43 @@ class IMAPFolder(BaseFolder):
"failed (error). Server reponded: %s\nMessage content was: " "failed (error). Server reponded: %s\nMessage content was: "
"%s" % (self, self.getrepository(), str(e), dbg_output), "%s" % (self, self.getrepository(), str(e), dbg_output),
OfflineImapError.ERROR.MESSAGE) OfflineImapError.ERROR.MESSAGE)
# get the new UID, default to 0 (=unknown)
uid = 0
if use_uidplus:
# get new UID from the APPENDUID response, it could look
# like OK [APPENDUID 38505 3955] APPEND completed with
# 38505 being folder UIDvalidity and 3955 the new UID.
typ, resp = imapobj.response('APPENDUID')
if resp == [None] or resp == None:
self.ui.warn("Server supports UIDPLUS but got no APPENDUID "
"appending a message.")
else:
uid = long(resp[-1].split(' ')[1])
else:
# Don't support UIDPLUS
# Checkpoint. Let it write out stuff, etc. Eg searches for # Checkpoint. Let it write out stuff, etc. Eg searches for
# just uploaded messages won't work if we don't do this. # just uploaded messages won't work if we don't do this.
typ, dat = imapobj.check() (typ,dat) = imapobj.check()
assert(typ == 'OK') assert(typ == 'OK')
# get the new UID. Test for APPENDUID response even if the
# server claims to not support it, as e.g. Gmail does :-(
if use_uidplus or imapobj._get_untagged_response('APPENDUID', True):
# get new UID from the APPENDUID response, it could look
# like OK [APPENDUID 38505 3955] APPEND completed with
# 38505 bein folder UIDvalidity and 3955 the new UID.
# note: we would want to use .response() here but that
# often seems to return [None], even though we have
# data. TODO
resp = imapobj._get_untagged_response('APPENDUID')
if resp == [None]:
self.ui.warn("Server supports UIDPLUS but got no APPENDUID "
"appending a message.")
return 0
uid = long(resp[-1].split(' ')[1])
if uid == 0:
self.ui.warn("savemessage: Server supports UIDPLUS, but"
" we got no usable uid back. APPENDUID reponse was "
"'%s'" % str(resp))
else:
# we don't support UIDPLUS
uid = self.savemessage_searchforheader(imapobj, headername, uid = self.savemessage_searchforheader(imapobj, headername,
headervalue) headervalue)
# If everything failed up to here, search the message # See docs for savemessage in Base.py for explanation
# manually TODO: rather than inserting and searching for our # of this and other return values
# custom header, we should be searching the Message-ID and
# compare the message size...
if uid == 0: if uid == 0:
self.ui.debug('imap', 'savemessage: attempt to get new UID ' self.ui.debug('imap', 'savemessage: attempt to get new UID '
'UID failed. Search headers manually.') 'UID failed. Search headers manually.')
uid = self.savemessage_fetchheaders(imapobj, headername, uid = self.savemessage_fetchheaders(imapobj, headername,
headervalue) headervalue)
self.ui.warn('imap', "savemessage: Searching mails for new "
"Message-ID failed. Could not determine new UID.")
finally: finally:
self.imapserver.releaseconnection(imapobj) self.imapserver.releaseconnection(imapobj)
@ -609,7 +614,11 @@ class IMAPFolder(BaseFolder):
return uid return uid
def savemessageflags(self, uid, flags): def savemessageflags(self, uid, flags):
"""Change a message's flags to `flags`.""" """Change a message's flags to `flags`.
Note that this function does not check against dryrun settings,
so you need to ensure that it is never called in a
dryrun mode."""
imapobj = self.imapserver.acquireconnection() imapobj = self.imapserver.acquireconnection()
try: try:
try: try:

View File

@ -15,16 +15,13 @@
# along with this program; if not, write to the Free Software # along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
from Base import BaseFolder from .Base import BaseFolder
import os import os
import threading import threading
try: # python 2.6 has set() built in
set
except NameError:
from sets import Set as set
magicline = "OFFLINEIMAP LocalStatus CACHE DATA - DO NOT MODIFY - FORMAT 1" magicline = "OFFLINEIMAP LocalStatus CACHE DATA - DO NOT MODIFY - FORMAT 1"
class LocalStatusFolder(BaseFolder): class LocalStatusFolder(BaseFolder):
def __init__(self, name, repository): def __init__(self, name, repository):
self.sep = '.' #needs to be set before super.__init__() self.sep = '.' #needs to be set before super.__init__()
@ -79,7 +76,7 @@ class LocalStatusFolder(BaseFolder):
uid, flags = line.split(':') uid, flags = line.split(':')
uid = long(uid) uid = long(uid)
flags = set(flags) flags = set(flags)
except ValueError, e: except ValueError as e:
errstr = "Corrupt line '%s' in cache file '%s'" % \ errstr = "Corrupt line '%s' in cache file '%s'" % \
(line, self.filename) (line, self.filename)
self.ui.warn(errstr) self.ui.warn(errstr)
@ -114,6 +111,11 @@ class LocalStatusFolder(BaseFolder):
return self.messagelist return self.messagelist
def savemessage(self, uid, content, flags, rtime): def savemessage(self, uid, content, flags, rtime):
"""Writes a new message, with the specified uid.
See folder/Base for detail. Note that savemessage() does not
check against dryrun settings, so you need to ensure that
savemessage is never called in a dryrun mode."""
if uid < 0: if uid < 0:
# We cannot assign a uid. # We cannot assign a uid.
return uid return uid

View File

@ -17,16 +17,12 @@
import os.path import os.path
import re import re
from threading import Lock from threading import Lock
from LocalStatus import LocalStatusFolder from .LocalStatus import LocalStatusFolder
try: try:
import sqlite3 as sqlite import sqlite3 as sqlite
except: except:
pass #fail only if needed later on, not on import pass #fail only if needed later on, not on import
try: # python 2.6 has set() built in
set
except NameError:
from sets import Set as set
class LocalStatusSQLiteFolder(LocalStatusFolder): class LocalStatusSQLiteFolder(LocalStatusFolder):
"""LocalStatus backend implemented with an SQLite database """LocalStatus backend implemented with an SQLite database
@ -220,6 +216,11 @@ class LocalStatusSQLiteFolder(LocalStatusFolder):
# assert False,"getmessageflags() called on non-existing message" # assert False,"getmessageflags() called on non-existing message"
def savemessage(self, uid, content, flags, rtime): def savemessage(self, uid, content, flags, rtime):
"""Writes a new message, with the specified uid.
See folder/Base for detail. Note that savemessage() does not
check against dryrun settings, so you need to ensure that
savemessage is never called in a dryrun mode."""
if uid < 0: if uid < 0:
# We cannot assign a uid. # We cannot assign a uid.
return uid return uid

View File

@ -19,7 +19,7 @@ import socket
import time import time
import re import re
import os import os
from Base import BaseFolder from .Base import BaseFolder
from threading import Lock from threading import Lock
try: try:
@ -40,7 +40,7 @@ re_uidmatch = re.compile(',U=(\d+)')
re_timestampmatch = re.compile('(\d+)'); re_timestampmatch = re.compile('(\d+)');
timeseq = 0 timeseq = 0
lasttime = long(0) lasttime = 0
timelock = Lock() timelock = Lock()
def gettimeseq(): def gettimeseq():
@ -237,13 +237,18 @@ class MaildirFolder(BaseFolder):
uid, self._foldermd5, self.infosep, ''.join(sorted(flags))) uid, self._foldermd5, self.infosep, ''.join(sorted(flags)))
def savemessage(self, uid, content, flags, rtime): def savemessage(self, uid, content, flags, rtime):
"""Writes a new message, with the specified uid.
See folder/Base for detail. Note that savemessage() does not
check against dryrun settings, so you need to ensure that
savemessage is never called in a dryrun mode."""
# This function only ever saves to tmp/, # This function only ever saves to tmp/,
# but it calls savemessageflags() to actually save to cur/ or new/. # but it calls savemessageflags() to actually save to cur/ or new/.
self.ui.debug('maildir', 'savemessage: called to write with flags %s ' self.ui.savemessage('maildir', uid, flags, self)
'and content %s' % (repr(flags), repr(content)))
if uid < 0: if uid < 0:
# We cannot assign a new uid. # We cannot assign a new uid.
return uid return uid
if uid in self.messagelist: if uid in self.messagelist:
# We already have it, just update flags. # We already have it, just update flags.
self.savemessageflags(uid, flags) self.savemessageflags(uid, flags)
@ -256,8 +261,8 @@ class MaildirFolder(BaseFolder):
# open file and write it out # open file and write it out
try: try:
fd = os.open(os.path.join(tmpdir, messagename), fd = os.open(os.path.join(tmpdir, messagename),
os.O_EXCL|os.O_CREAT|os.O_WRONLY, 0666) os.O_EXCL|os.O_CREAT|os.O_WRONLY, 0o666)
except OSError, e: except OSError as e:
if e.errno == 17: if e.errno == 17:
#FILE EXISTS ALREADY #FILE EXISTS ALREADY
severity = OfflineImapError.ERROR.MESSAGE severity = OfflineImapError.ERROR.MESSAGE
@ -291,8 +296,11 @@ class MaildirFolder(BaseFolder):
"""Sets the specified message's flags to the given set. """Sets the specified message's flags to the given set.
This function moves the message to the cur or new subdir, This function moves the message to the cur or new subdir,
depending on the 'S'een flag.""" depending on the 'S'een flag.
Note that this function does not check against dryrun settings,
so you need to ensure that it is never called in a
dryrun mode."""
oldfilename = self.messagelist[uid]['filename'] oldfilename = self.messagelist[uid]['filename']
dir_prefix, filename = os.path.split(oldfilename) dir_prefix, filename = os.path.split(oldfilename)
# If a message has been seen, it goes into 'cur' # If a message has been seen, it goes into 'cur'
@ -313,7 +321,7 @@ class MaildirFolder(BaseFolder):
try: try:
os.rename(os.path.join(self.getfullname(), oldfilename), os.rename(os.path.join(self.getfullname(), oldfilename),
os.path.join(self.getfullname(), newfilename)) os.path.join(self.getfullname(), newfilename))
except OSError, e: except OSError as e:
raise OfflineImapError("Can't rename file '%s' to '%s': %s" % ( raise OfflineImapError("Can't rename file '%s' to '%s': %s" % (
oldfilename, newfilename, e[1]), oldfilename, newfilename, e[1]),
OfflineImapError.ERROR.FOLDER) OfflineImapError.ERROR.FOLDER)

View File

@ -15,7 +15,7 @@
# along with this program; if not, write to the Free Software # along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
from threading import Lock from threading import Lock
from IMAP import IMAPFolder from .IMAP import IMAPFolder
import os.path import os.path
class MappedIMAPFolder(IMAPFolder): class MappedIMAPFolder(IMAPFolder):
@ -82,7 +82,13 @@ class MappedIMAPFolder(IMAPFolder):
if dolock: self.maplock.release() if dolock: self.maplock.release()
def _uidlist(self, mapping, items): def _uidlist(self, mapping, items):
try:
return [mapping[x] for x in items] return [mapping[x] for x in items]
except KeyError as e:
raise OfflineImapError("Could not find UID for msg '{0}' (f:'{1}'."
" This is usually a bad thing and should be reported on the ma"
"iling list.".format(e.args[0], self),
OfflineImapError.ERROR.MESSAGE)
def cachemessagelist(self): def cachemessagelist(self):
self._mb.cachemessagelist() self._mb.cachemessagelist()
@ -94,7 +100,7 @@ class MappedIMAPFolder(IMAPFolder):
# summary that have been deleted from the folder. # summary that have been deleted from the folder.
for luid in self.diskl2r.keys(): for luid in self.diskl2r.keys():
if not reallist.has_key(luid): if not luid in reallist:
ruid = self.diskl2r[luid] ruid = self.diskl2r[luid]
del self.diskr2l[ruid] del self.diskr2l[ruid]
del self.diskl2r[luid] del self.diskl2r[luid]
@ -107,7 +113,7 @@ class MappedIMAPFolder(IMAPFolder):
self.l2r = self.diskl2r.copy() self.l2r = self.diskl2r.copy()
for luid in reallist.keys(): for luid in reallist.keys():
if not self.l2r.has_key(luid): if not luid in self.l2r:
ruid = nextneg ruid = nextneg
nextneg -= 1 nextneg -= 1
self.l2r[luid] = ruid self.l2r[luid] = ruid
@ -178,7 +184,12 @@ class MappedIMAPFolder(IMAPFolder):
If the uid is > 0, the backend should set the uid to this, if it can. If the uid is > 0, the backend should set the uid to this, if it can.
If it cannot set the uid to that, it will save it anyway. If it cannot set the uid to that, it will save it anyway.
It will return the uid assigned in any case. It will return the uid assigned in any case.
See folder/Base for details. Note that savemessage() does not
check against dryrun settings, so you need to ensure that
savemessage is never called in a dryrun mode.
""" """
self.ui.savemessage('imap', uid, flags, self)
# Mapped UID instances require the source to already have a # Mapped UID instances require the source to already have a
# positive UID, so simply return here. # positive UID, so simply return here.
if uid < 0: if uid < 0:
@ -211,6 +222,11 @@ class MappedIMAPFolder(IMAPFolder):
return None return None
def savemessageflags(self, uid, flags): def savemessageflags(self, uid, flags):
"""
Note that this function does not check against dryrun settings,
so you need to ensure that it is never called in a
dryrun mode."""
self._mb.savemessageflags(self.r2l[uid], flags) self._mb.savemessageflags(self.r2l[uid], flags)
def addmessageflags(self, uid, flags): def addmessageflags(self, uid, flags):

View File

@ -1,2 +1,2 @@
import Base, Gmail, IMAP, Maildir, LocalStatus from . import Base, Gmail, IMAP, Maildir, LocalStatus

View File

@ -19,6 +19,7 @@ import os
import fcntl import fcntl
import re import re
import socket import socket
import ssl
import time import time
import subprocess import subprocess
import threading import threading
@ -28,11 +29,6 @@ from offlineimap.ui import getglobalui
from offlineimap import OfflineImapError from offlineimap import OfflineImapError
from offlineimap.imaplib2 import IMAP4, IMAP4_SSL, zlib, IMAP4_PORT, InternalDate, Mon2num from offlineimap.imaplib2 import IMAP4, IMAP4_SSL, zlib, IMAP4_PORT, InternalDate, Mon2num
try:
import ssl
except ImportError:
#fails on python <2.6
pass
class UsefulIMAPMixIn(object): class UsefulIMAPMixIn(object):
def getselectedfolder(self): def getselectedfolder(self):
@ -53,7 +49,7 @@ class UsefulIMAPMixIn(object):
del self.untagged_responses[:] del self.untagged_responses[:]
try: try:
result = super(UsefulIMAPMixIn, self).select(mailbox, readonly) result = super(UsefulIMAPMixIn, self).select(mailbox, readonly)
except self.abort, e: except self.abort as e:
# self.abort is raised when we are supposed to retry # self.abort is raised when we are supposed to retry
errstr = "Server '%s' closed connection, error on SELECT '%s'. Ser"\ errstr = "Server '%s' closed connection, error on SELECT '%s'. Ser"\
"ver said: %s" % (self.host, mailbox, e.args[0]) "ver said: %s" % (self.host, mailbox, e.args[0])
@ -141,7 +137,7 @@ class WrappedIMAP4_SSL(UsefulIMAPMixIn, IMAP4_SSL):
"""Improved version of imaplib.IMAP4_SSL overriding select()""" """Improved version of imaplib.IMAP4_SSL overriding select()"""
def __init__(self, *args, **kwargs): def __init__(self, *args, **kwargs):
self._fingerprint = kwargs.get('fingerprint', None) self._fingerprint = kwargs.get('fingerprint', None)
if kwargs.has_key('fingerprint'): if 'fingerprint' in kwargs:
del kwargs['fingerprint'] del kwargs['fingerprint']
super(WrappedIMAP4_SSL, self).__init__(*args, **kwargs) super(WrappedIMAP4_SSL, self).__init__(*args, **kwargs)

View File

@ -18,7 +18,6 @@
from offlineimap import imaplibutil, imaputil, threadutil, OfflineImapError from offlineimap import imaplibutil, imaputil, threadutil, OfflineImapError
from offlineimap.ui import getglobalui from offlineimap.ui import getglobalui
from threading import Lock, BoundedSemaphore, Thread, Event, currentThread from threading import Lock, BoundedSemaphore, Thread, Event, currentThread
from thread import get_ident # python < 2.6 support
import offlineimap.accounts import offlineimap.accounts
import hmac import hmac
import socket import socket
@ -27,11 +26,7 @@ import time
import errno import errno
from sys import exc_info from sys import exc_info
from socket import gaierror from socket import gaierror
try: from ssl import SSLError, cert_time_to_seconds
from ssl import SSLError, cert_time_to_seconds
except ImportError:
# Protect against python<2.6, use dummy and won't get SSL errors.
SSLError = None
try: try:
# do we have a recent pykerberos? # do we have a recent pykerberos?
@ -153,7 +148,7 @@ class IMAPServer:
rc = kerberos.authGSSClientWrap(self.gss_vc, response, rc = kerberos.authGSSClientWrap(self.gss_vc, response,
self.username) self.username)
response = kerberos.authGSSClientResponse(self.gss_vc) response = kerberos.authGSSClientResponse(self.gss_vc)
except kerberos.GSSError, err: except kerberos.GSSError as err:
# Kerberos errored out on us, respond with None to cancel the # Kerberos errored out on us, respond with None to cancel the
# authentication # authentication
self.ui.debug('imap', '%s: %s' % (err[0][0], err[1][0])) self.ui.debug('imap', '%s: %s' % (err[0][0], err[1][0]))
@ -171,6 +166,7 @@ class IMAPServer:
self.semaphore.acquire() self.semaphore.acquire()
self.connectionlock.acquire() self.connectionlock.acquire()
curThread = currentThread()
imapobj = None imapobj = None
if len(self.availableconnections): # One is available. if len(self.availableconnections): # One is available.
@ -180,7 +176,7 @@ class IMAPServer:
imapobj = None imapobj = None
for i in range(len(self.availableconnections) - 1, -1, -1): for i in range(len(self.availableconnections) - 1, -1, -1):
tryobj = self.availableconnections[i] tryobj = self.availableconnections[i]
if self.lastowner[tryobj] == get_ident(): if self.lastowner[tryobj] == curThread.ident:
imapobj = tryobj imapobj = tryobj
del(self.availableconnections[i]) del(self.availableconnections[i])
break break
@ -188,7 +184,7 @@ class IMAPServer:
imapobj = self.availableconnections[0] imapobj = self.availableconnections[0]
del(self.availableconnections[0]) del(self.availableconnections[0])
self.assignedconnections.append(imapobj) self.assignedconnections.append(imapobj)
self.lastowner[imapobj] = get_ident() self.lastowner[imapobj] = curThread.ident
self.connectionlock.release() self.connectionlock.release()
return imapobj return imapobj
@ -232,7 +228,7 @@ class IMAPServer:
'Attempting GSSAPI authentication') 'Attempting GSSAPI authentication')
try: try:
imapobj.authenticate('GSSAPI', self.gssauth) imapobj.authenticate('GSSAPI', self.gssauth)
except imapobj.error, val: except imapobj.error as val:
self.gssapi = False self.gssapi = False
self.ui.debug('imap', self.ui.debug('imap',
'GSSAPI Authentication failed') 'GSSAPI Authentication failed')
@ -258,7 +254,7 @@ class IMAPServer:
try: try:
imapobj.authenticate('CRAM-MD5', imapobj.authenticate('CRAM-MD5',
self.md5handler) self.md5handler)
except imapobj.error, val: except imapobj.error as val:
self.plainauth(imapobj) self.plainauth(imapobj)
else: else:
# Use plaintext login, unless # Use plaintext login, unless
@ -271,7 +267,7 @@ class IMAPServer:
# Would bail by here if there was a failure. # Would bail by here if there was a failure.
success = 1 success = 1
self.goodpassword = self.password self.goodpassword = self.password
except imapobj.error, val: except imapobj.error as val:
self.passworderror = str(val) self.passworderror = str(val)
raise raise
@ -301,10 +297,10 @@ class IMAPServer:
self.connectionlock.acquire() self.connectionlock.acquire()
self.assignedconnections.append(imapobj) self.assignedconnections.append(imapobj)
self.lastowner[imapobj] = get_ident() self.lastowner[imapobj] = curThread.ident
self.connectionlock.release() self.connectionlock.release()
return imapobj return imapobj
except Exception, e: except Exception as e:
"""If we are here then we did not succeed in getting a """If we are here then we did not succeed in getting a
connection - we should clean up and then re-raise the connection - we should clean up and then re-raise the
error...""" error..."""
@ -323,7 +319,7 @@ class IMAPServer:
(self.hostname, self.repos) (self.hostname, self.repos)
raise OfflineImapError(reason, severity) raise OfflineImapError(reason, severity)
elif SSLError and isinstance(e, SSLError) and e.errno == 1: elif isinstance(e, SSLError) and e.errno == 1:
# SSL unknown protocol error # SSL unknown protocol error
# happens e.g. when connecting via SSL to a non-SSL service # happens e.g. when connecting via SSL to a non-SSL service
if self.port != 993: if self.port != 993:
@ -540,7 +536,7 @@ class IdleThread(object):
imapobj = self.parent.acquireconnection() imapobj = self.parent.acquireconnection()
try: try:
imapobj.select(self.folder) imapobj.select(self.folder)
except OfflineImapError, e: except OfflineImapError as e:
if e.severity == OfflineImapError.ERROR.FOLDER_RETRY: if e.severity == OfflineImapError.ERROR.FOLDER_RETRY:
# Connection closed, release connection and retry # Connection closed, release connection and retry
self.ui.error(e, exc_info()[2]) self.ui.error(e, exc_info()[2])

View File

@ -18,12 +18,8 @@
import re import re
import string import string
import types
from offlineimap.ui import getglobalui from offlineimap.ui import getglobalui
try: # python 2.6 has set() built in
set
except NameError:
from sets import Set as set
# find the first quote in a string # find the first quote in a string
quotere = re.compile( quotere = re.compile(
@ -58,7 +54,7 @@ def flagsplit(string):
['FLAGS,'(\\Seen Old)','UID', '4807'] ['FLAGS,'(\\Seen Old)','UID', '4807']
""" """
if string[0] != '(' or string[-1] != ')': if string[0] != '(' or string[-1] != ')':
raise ValueError, "Passed string '%s' is not a flag list" % string raise ValueError("Passed string '%s' is not a flag list" % string)
return imapsplit(string[1:-1]) return imapsplit(string[1:-1])
def options2hash(list): def options2hash(list):
@ -91,7 +87,7 @@ def imapsplit(imapstring):
['(\\HasNoChildren)', '"."', '"INBOX.Sent"']""" ['(\\HasNoChildren)', '"."', '"INBOX.Sent"']"""
if type(imapstring) != types.StringType: if not isinstance(imapstring, basestring):
debug("imapsplit() got a non-string input; working around.") debug("imapsplit() got a non-string input; working around.")
# Sometimes, imaplib will throw us a tuple if the input # Sometimes, imaplib will throw us a tuple if the input
# contains a literal. See Python bug # contains a literal. See Python bug
@ -186,13 +182,12 @@ def flagsimap2maildir(flagstring):
return retval return retval
def flagsmaildir2imap(maildirflaglist): def flagsmaildir2imap(maildirflaglist):
"""Convert set of flags ([DR]) into a string '(\\Draft \\Deleted)'""" """Convert set of flags ([DR]) into a string '(\\Deleted \\Draft)'"""
retval = [] retval = []
for imapflag, maildirflag in flagmap: for imapflag, maildirflag in flagmap:
if maildirflag in maildirflaglist: if maildirflag in maildirflaglist:
retval.append(imapflag) retval.append(imapflag)
retval.sort() return '(' + ' '.join(sorted(retval)) + ')'
return '(' + ' '.join(retval) + ')'
def uid_sequence(uidlist): def uid_sequence(uidlist):
"""Collapse UID lists into shorter sequence sets """Collapse UID lists into shorter sequence sets

View File

@ -52,6 +52,25 @@ class OfflineImap:
description="%s.\n\n%s" % description="%s.\n\n%s" %
(offlineimap.__copyright__, (offlineimap.__copyright__,
offlineimap.__license__)) offlineimap.__license__))
parser.add_option("--dry-run",
action="store_true", dest="dryrun",
default=False,
help="Do not actually modify any store but check and print "
"what synchronization actions would be taken if a sync would be"
" performed. It will not precisely give the exact information w"
"hat will happen. If e.g. we need to create a folder, it merely"
" outputs 'Would create folder X', but not how many and which m"
"ails it would transfer.")
parser.add_option("--info",
action="store_true", dest="diagnostics",
default=False,
help="Output information on the configured email repositories"
". Useful for debugging and bug reporting. Use in conjunction wit"
"h the -a option to limit the output to a single account. This mo"
"de will prevent any actual sync to occur and exits after it outp"
"ut the debug information.")
parser.add_option("-1", parser.add_option("-1",
action="store_true", dest="singlethreading", action="store_true", dest="singlethreading",
default=False, default=False,
@ -73,11 +92,11 @@ class OfflineImap:
"implies the -1 option.") "implies the -1 option.")
parser.add_option("-a", dest="accounts", metavar="ACCOUNTS", parser.add_option("-a", dest="accounts", metavar="ACCOUNTS",
help="""Overrides the accounts section in the config file. help="Overrides the accounts section in the config file. "
Lets you specify a particular account or set of "Lets you specify a particular account or set of "
accounts to sync without having to edit the config "accounts to sync without having to edit the config "
file. You might use this to exclude certain accounts, "file. You might use this to exclude certain accounts, "
or to sync some accounts that you normally prefer not to.""") "or to sync some accounts that you normally prefer not to.")
parser.add_option("-c", dest="configfile", metavar="FILE", parser.add_option("-c", dest="configfile", metavar="FILE",
default="~/.offlineimaprc", default="~/.offlineimaprc",
@ -85,29 +104,27 @@ class OfflineImap:
"%default.") "%default.")
parser.add_option("-d", dest="debugtype", metavar="type1,[type2...]", parser.add_option("-d", dest="debugtype", metavar="type1,[type2...]",
help="""Enables debugging for OfflineIMAP. This is useful help="Enables debugging for OfflineIMAP. This is useful "
if you are to track down a malfunction or figure out what is "if you are to track down a malfunction or figure out what is "
going on under the hood. This option requires one or more "going on under the hood. This option requires one or more "
debugtypes, separated by commas. These define what exactly "debugtypes, separated by commas. These define what exactly "
will be debugged, and so far include two options: imap, thread, "will be debugged, and so far include two options: imap, thread, "
maildir or ALL. The imap option will enable IMAP protocol "maildir or ALL. The imap option will enable IMAP protocol "
stream and parsing debugging. Note that the output may contain "stream and parsing debugging. Note that the output may contain "
passwords, so take care to remove that from the debugging "passwords, so take care to remove that from the debugging "
output before sending it to anyone else. The maildir option "output before sending it to anyone else. The maildir option "
will enable debugging for certain Maildir operations. "will enable debugging for certain Maildir operations. "
The use of any debug option (unless 'thread' is included), "The use of any debug option (unless 'thread' is included), "
implies the single-thread option -1.""") "implies the single-thread option -1.")
parser.add_option("-l", dest="logfile", metavar="FILE", parser.add_option("-l", dest="logfile", metavar="FILE",
help="Log to FILE") help="Log to FILE")
parser.add_option("-f", dest="folders", metavar="folder1,[folder2...]", parser.add_option("-f", dest="folders", metavar="folder1,[folder2...]",
help= help="Only sync the specified folders. The folder names "
"Only sync the specified folders. The folder names " "are the *untranslated* foldernames of the remote repository. "
"are the *untranslated* foldernames. This " "This command-line option overrides any 'folderfilter' "
"command-line option overrides any 'folderfilter' " "and 'folderincludes' options in the configuration file.")
"and 'folderincludes' options in the configuration "
"file.")
parser.add_option("-k", dest="configoverride", parser.add_option("-k", dest="configoverride",
action="append", action="append",
@ -142,13 +159,6 @@ class OfflineImap:
"not usable. Possible interface choices are: %s " % "not usable. Possible interface choices are: %s " %
", ".join(UI_LIST.keys())) ", ".join(UI_LIST.keys()))
parser.add_option("--info",
action="store_true", dest="diagnostics",
default=False,
help="Output information on the configured email repositories"
". Useful for debugging and bug reporting. Use in conjunction wit"
"h the -a option to limit the output to a single account")
(options, args) = parser.parse_args() (options, args) = parser.parse_args()
#read in configuration file #read in configuration file
@ -201,6 +211,12 @@ class OfflineImap:
logging.warning('Using old interface name, consider using one ' logging.warning('Using old interface name, consider using one '
'of %s' % ', '.join(UI_LIST.keys())) 'of %s' % ', '.join(UI_LIST.keys()))
if options.diagnostics: ui_type = 'basic' # enforce basic UI for --info if options.diagnostics: ui_type = 'basic' # enforce basic UI for --info
#dry-run? Set [general]dry-run=True
if options.dryrun:
dryrun = config.set('general','dry-run', "True")
config.set_if_not_exists('general','dry-run','False')
try: try:
# create the ui class # create the ui class
self.ui = UI_LIST[ui_type.lower()](config) self.ui = UI_LIST[ui_type.lower()](config)
@ -252,11 +268,9 @@ class OfflineImap:
account_section = 'Account ' + accountname account_section = 'Account ' + accountname
remote_repo_section = 'Repository ' + \ remote_repo_section = 'Repository ' + \
config.get(account_section, 'remoterepository') config.get(account_section, 'remoterepository')
local_repo_section = 'Repository ' + \ config.set(remote_repo_section, "folderfilter", folderfilter)
config.get(account_section, 'localrepository') config.set(remote_repo_section, "folderincludes",
for section in [remote_repo_section, local_repo_section]: folderincludes)
config.set(section, "folderfilter", folderfilter)
config.set(section, "folderincludes", folderincludes)
if options.logfile: if options.logfile:
sys.stderr = self.ui.logfile sys.stderr = self.ui.logfile
@ -351,7 +365,7 @@ class OfflineImap:
self.ui.terminate() self.ui.terminate()
except (SystemExit): except (SystemExit):
raise raise
except Exception, e: except Exception as e:
self.ui.error(e) self.ui.error(e)
self.ui.terminate() self.ui.terminate()

View File

@ -1,6 +1,5 @@
# Base repository support # Base repository support
# Copyright (C) 2002-2007 John Goerzen # Copyright (C) 2002-2012 John Goerzen & contributors
# <jgoerzen@complete.org>
# #
# This program is free software; you can redistribute it and/or modify # This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by # it under the terms of the GNU General Public License as published by
@ -24,7 +23,7 @@ from offlineimap import CustomConfig
from offlineimap.ui import getglobalui from offlineimap.ui import getglobalui
from offlineimap.error import OfflineImapError from offlineimap.error import OfflineImapError
class BaseRepository(object, CustomConfig.ConfigHelperMixin): class BaseRepository(CustomConfig.ConfigHelperMixin, object):
def __init__(self, reposname, account): def __init__(self, reposname, account):
self.ui = getglobalui() self.ui = getglobalui()
@ -35,18 +34,18 @@ class BaseRepository(object, CustomConfig.ConfigHelperMixin):
self._accountname = self.account.getname() self._accountname = self.account.getname()
self.uiddir = os.path.join(self.config.getmetadatadir(), 'Repository-' + self.name) self.uiddir = os.path.join(self.config.getmetadatadir(), 'Repository-' + self.name)
if not os.path.exists(self.uiddir): if not os.path.exists(self.uiddir):
os.mkdir(self.uiddir, 0700) os.mkdir(self.uiddir, 0o700)
self.mapdir = os.path.join(self.uiddir, 'UIDMapping') self.mapdir = os.path.join(self.uiddir, 'UIDMapping')
if not os.path.exists(self.mapdir): if not os.path.exists(self.mapdir):
os.mkdir(self.mapdir, 0700) os.mkdir(self.mapdir, 0o700)
self.uiddir = os.path.join(self.uiddir, 'FolderValidity') self.uiddir = os.path.join(self.uiddir, 'FolderValidity')
if not os.path.exists(self.uiddir): if not os.path.exists(self.uiddir):
os.mkdir(self.uiddir, 0700) os.mkdir(self.uiddir, 0o700)
self.nametrans = lambda foldername: foldername self.nametrans = lambda foldername: foldername
self.folderfilter = lambda foldername: 1 self.folderfilter = lambda foldername: 1
self.folderincludes = [] self.folderincludes = []
self.foldersort = cmp self.foldersort = None
if self.config.has_option(self.getsection(), 'nametrans'): if self.config.has_option(self.getsection(), 'nametrans'):
self.nametrans = self.localeval.eval( self.nametrans = self.localeval.eval(
self.getconf('nametrans'), {'re': re}) self.getconf('nametrans'), {'re': re})
@ -125,6 +124,7 @@ class BaseRepository(object, CustomConfig.ConfigHelperMixin):
raise NotImplementedError raise NotImplementedError
def makefolder(self, foldername): def makefolder(self, foldername):
"""Create a new folder"""
raise NotImplementedError raise NotImplementedError
def deletefolder(self, foldername): def deletefolder(self, foldername):
@ -154,49 +154,50 @@ class BaseRepository(object, CustomConfig.ConfigHelperMixin):
src_repo.getsep(), dst_repo.getsep())] = folder src_repo.getsep(), dst_repo.getsep())] = folder
dst_hash = {} dst_hash = {}
for folder in dst_folders: for folder in dst_folders:
dst_hash[folder.name] = folder dst_hash[folder.getvisiblename().replace(
dst_repo.getsep(), src_repo.getsep())] = folder
# Find new folders on src_repo. # Find new folders on src_repo.
for src_name, src_folder in src_hash.iteritems(): for src_name_t, src_folder in src_hash.iteritems():
# Don't create on dst_repo, if it is readonly # Don't create on dst_repo, if it is readonly
if dst_repo.getconfboolean('readonly', False): if dst_repo.getconfboolean('readonly', False):
break break
if src_folder.sync_this and not src_name in dst_hash: if src_folder.sync_this and not src_name_t in dst_folders:
try: try:
dst_repo.makefolder(src_name) dst_repo.makefolder(src_name_t)
dst_haschanged = True # Need to refresh list dst_haschanged = True # Need to refresh list
except OfflineImapError, e: except OfflineImapError as e:
self.ui.error(e, exc_info()[2], self.ui.error(e, exc_info()[2],
"Creating folder %s on repository %s" %\ "Creating folder %s on repository %s" %\
(src_name, dst_repo)) (src_name_t, dst_repo))
raise raise
status_repo.makefolder(src_name.replace(dst_repo.getsep(), status_repo.makefolder(src_name_t.replace(dst_repo.getsep(),
status_repo.getsep())) status_repo.getsep()))
# Find new folders on dst_repo. # Find new folders on dst_repo.
for dst_name, dst_folder in dst_hash.iteritems(): for dst_name_t, dst_folder in dst_hash.iteritems():
if self.getconfboolean('readonly', False): if self.getconfboolean('readonly', False):
# Don't create missing folder on readonly repo. # Don't create missing folder on readonly repo.
break break
if dst_folder.sync_this and not dst_name in src_hash: if dst_folder.sync_this and not dst_name_t in src_folders:
# nametrans sanity check! # nametrans sanity check!
# Does nametrans back&forth lead to identical names? # Does nametrans back&forth lead to identical names?
#src_name is the unmodified full src_name that would be created # 1) would src repo filter out the new folder name? In this
newsrc_name = dst_folder.getvisiblename().replace(
dst_repo.getsep(),
src_repo.getsep())
folder = self.getfolder(newsrc_name)
# would src repo filter out the new folder name? In this
# case don't create it on it: # case don't create it on it:
if not self.folderfilter(newsrc_name): if not self.folderfilter(dst_name_t):
self.ui.debug('', "Not creating folder '%s' (repository '%s" self.ui.debug('', "Not creating folder '%s' (repository '%s"
"') as it would be filtered out on that repository." % "') as it would be filtered out on that repository." %
(newsrc_name, self)) (dst_name_t, self))
continue continue
# get IMAPFolder and see if the reverse nametrans
# works fine TODO: getfolder() works only because we
# succeed in getting inexisting folders which I would
# like to change. Take care!
folder = self.getfolder(dst_name_t)
# apply reverse nametrans to see if we end up with the same name # apply reverse nametrans to see if we end up with the same name
newdst_name = folder.getvisiblename().replace( newdst_name = folder.getvisiblename().replace(
src_repo.getsep(), dst_repo.getsep()) src_repo.getsep(), dst_repo.getsep())
if dst_name != newdst_name: if dst_folder.name != newdst_name:
raise OfflineImapError("INFINITE FOLDER CREATION DETECTED! " raise OfflineImapError("INFINITE FOLDER CREATION DETECTED! "
"Folder '%s' (repository '%s') would be created as fold" "Folder '%s' (repository '%s') would be created as fold"
"er '%s' (repository '%s'). The latter becomes '%s' in " "er '%s' (repository '%s'). The latter becomes '%s' in "
@ -205,18 +206,18 @@ class BaseRepository(object, CustomConfig.ConfigHelperMixin):
"itories so they lead to identical names if applied bac" "itories so they lead to identical names if applied bac"
"k and forth. 2) Use folderfilter settings on a reposit" "k and forth. 2) Use folderfilter settings on a reposit"
"ory to prevent some folders from being created on the " "ory to prevent some folders from being created on the "
"other side." % (dst_name, dst_repo, newsrc_name, "other side." % (dst_folder.name, dst_repo, dst_name_t,
src_repo, newdst_name), src_repo, newdst_name),
OfflineImapError.ERROR.REPO) OfflineImapError.ERROR.REPO)
# end sanity check, actually create the folder # end sanity check, actually create the folder
try: try:
src_repo.makefolder(newsrc_name) src_repo.makefolder(dst_name_t)
src_haschanged = True # Need to refresh list src_haschanged = True # Need to refresh list
except OfflineImapError, e: except OfflineImapError as e:
self.ui.error(e, exc_info()[2], "Creating folder %s on " self.ui.error(e, exc_info()[2], "Creating folder %s on "
"repository %s" % (newsrc_name, src_repo)) "repository %s" % (dst_name_t, src_repo))
raise raise
status_repo.makefolder(newsrc_name.replace( status_repo.makefolder(dst_name_t.replace(
src_repo.getsep(), status_repo.getsep())) src_repo.getsep(), status_repo.getsep()))
# Find deleted folders. # Find deleted folders.
# TODO: We don't delete folders right now. # TODO: We don't delete folders right now.

View File

@ -20,7 +20,6 @@ from offlineimap import folder, imaputil, imapserver, OfflineImapError
from offlineimap.folder.UIDMaps import MappedIMAPFolder from offlineimap.folder.UIDMaps import MappedIMAPFolder
from offlineimap.threadutil import ExitNotifyThread from offlineimap.threadutil import ExitNotifyThread
from threading import Event from threading import Event
import types
import os import os
from sys import exc_info from sys import exc_info
import netrc import netrc
@ -34,6 +33,9 @@ class IMAPRepository(BaseRepository):
self._host = None self._host = None
self.imapserver = imapserver.IMAPServer(self) self.imapserver = imapserver.IMAPServer(self)
self.folders = None self.folders = None
if self.getconf('sep', None):
self.ui.info("The 'sep' setting is being ignored for IMAP "
"repository '%s' (it's autodetected)" % self)
def startkeepalive(self): def startkeepalive(self):
keepalivetime = self.getkeepalive() keepalivetime = self.getkeepalive()
@ -95,7 +97,7 @@ class IMAPRepository(BaseRepository):
host = self.getconf('remotehosteval') host = self.getconf('remotehosteval')
try: try:
host = self.localeval.eval(host) host = self.localeval.eval(host)
except Exception, e: except Exception as e:
raise OfflineImapError("remotehosteval option for repository "\ raise OfflineImapError("remotehosteval option for repository "\
"'%s' failed:\n%s" % (self, e), "'%s' failed:\n%s" % (self, e),
OfflineImapError.ERROR.REPO) OfflineImapError.ERROR.REPO)
@ -128,7 +130,7 @@ class IMAPRepository(BaseRepository):
try: try:
netrcentry = netrc.netrc().authenticators(self.gethost()) netrcentry = netrc.netrc().authenticators(self.gethost())
except IOError, inst: except IOError as inst:
if inst.errno != errno.ENOENT: if inst.errno != errno.ENOENT:
raise raise
else: else:
@ -137,7 +139,7 @@ class IMAPRepository(BaseRepository):
try: try:
netrcentry = netrc.netrc('/etc/netrc').authenticators(self.gethost()) netrcentry = netrc.netrc('/etc/netrc').authenticators(self.gethost())
except IOError, inst: except IOError as inst:
if inst.errno not in (errno.ENOENT, errno.EACCES): if inst.errno not in (errno.ENOENT, errno.EACCES):
raise raise
else: else:
@ -223,7 +225,7 @@ class IMAPRepository(BaseRepository):
# 4. read password from ~/.netrc # 4. read password from ~/.netrc
try: try:
netrcentry = netrc.netrc().authenticators(self.gethost()) netrcentry = netrc.netrc().authenticators(self.gethost())
except IOError, inst: except IOError as inst:
if inst.errno != errno.ENOENT: if inst.errno != errno.ENOENT:
raise raise
else: else:
@ -234,7 +236,7 @@ class IMAPRepository(BaseRepository):
# 5. read password from /etc/netrc # 5. read password from /etc/netrc
try: try:
netrcentry = netrc.netrc('/etc/netrc').authenticators(self.gethost()) netrcentry = netrc.netrc('/etc/netrc').authenticators(self.gethost())
except IOError, inst: except IOError as inst:
if inst.errno not in (errno.ENOENT, errno.EACCES): if inst.errno not in (errno.ENOENT, errno.EACCES):
raise raise
else: else:
@ -274,9 +276,9 @@ class IMAPRepository(BaseRepository):
self.imapserver.releaseconnection(imapobj) self.imapserver.releaseconnection(imapobj)
for string in listresult: for string in listresult:
if string == None or \ if string == None or \
(type(string) == types.StringType and string == ''): (isinstance(string, basestring) and string == ''):
# Bug in imaplib: empty strings in results from # Bug in imaplib: empty strings in results from
# literals. # literals. TODO: still relevant?
continue continue
flags, delim, name = imaputil.imapsplit(string) flags, delim, name = imaputil.imapsplit(string)
flaglist = [x.lower() for x in imaputil.flagsplit(flags)] flaglist = [x.lower() for x in imaputil.flagsplit(flags)]
@ -297,7 +299,7 @@ class IMAPRepository(BaseRepository):
for foldername in self.folderincludes: for foldername in self.folderincludes:
try: try:
imapobj.select(foldername, readonly = True) imapobj.select(foldername, readonly = True)
except OfflineImapError, e: except OfflineImapError as e:
# couldn't select this folderinclude, so ignore folder. # couldn't select this folderinclude, so ignore folder.
if e.severity > OfflineImapError.ERROR.FOLDER: if e.severity > OfflineImapError.ERROR.FOLDER:
raise raise
@ -310,7 +312,23 @@ class IMAPRepository(BaseRepository):
finally: finally:
self.imapserver.releaseconnection(imapobj) self.imapserver.releaseconnection(imapobj)
retval.sort(lambda x, y: self.foldersort(x.getvisiblename(), y.getvisiblename())) if self.foldersort is None:
# default sorting by case insensitive transposed name
retval.sort(key=lambda x: str.lower(x.getvisiblename()))
else:
# do foldersort in a python3-compatible way
# http://bytes.com/topic/python/answers/844614-python-3-sorting-comparison-function
def cmp2key(mycmp):
"""Converts a cmp= function into a key= function
We need to keep cmp functions for backward compatibility"""
class K:
def __init__(self, obj, *args):
self.obj = obj
def __cmp__(self, other):
return mycmp(self.obj, other.obj)
return K
retval.sort(key=cmp2key(self.foldersort))
self.folders = retval self.folders = retval
return self.folders return self.folders
@ -326,11 +344,11 @@ class IMAPRepository(BaseRepository):
foldername = self.getreference() + self.getsep() + foldername foldername = self.getreference() + self.getsep() + foldername
if not foldername: # Create top level folder as folder separator if not foldername: # Create top level folder as folder separator
foldername = self.getsep() foldername = self.getsep()
self.ui.makefolder(self, foldername)
if self.account.dryrun:
return
imapobj = self.imapserver.acquireconnection() imapobj = self.imapserver.acquireconnection()
try: try:
self.ui._msg("Creating new IMAP folder '%s' on server %s" %\
(foldername, self))
result = imapobj.create(foldername) result = imapobj.create(foldername)
if result[0] != 'OK': if result[0] != 'OK':
raise OfflineImapError("Folder '%s'[%s] could not be created. " raise OfflineImapError("Folder '%s'[%s] could not be created. "

View File

@ -16,9 +16,9 @@
# along with this program; if not, write to the Free Software # along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
from Base import BaseRepository
from offlineimap.folder.LocalStatus import LocalStatusFolder, magicline from offlineimap.folder.LocalStatus import LocalStatusFolder, magicline
from offlineimap.folder.LocalStatusSQLite import LocalStatusSQLiteFolder from offlineimap.folder.LocalStatusSQLite import LocalStatusSQLiteFolder
from offlineimap.repository.Base import BaseRepository
import os import os
import re import re
@ -41,7 +41,7 @@ class LocalStatusRepository(BaseRepository):
% (backend, account.name)) % (backend, account.name))
if not os.path.exists(self.root): if not os.path.exists(self.root):
os.mkdir(self.root, 0700) os.mkdir(self.root, 0o700)
# self._folders is a list of LocalStatusFolders() # self._folders is a list of LocalStatusFolders()
self._folders = None self._folders = None
@ -67,10 +67,11 @@ class LocalStatusRepository(BaseRepository):
Empty Folder for plain backend. NoOp for sqlite backend as those Empty Folder for plain backend. NoOp for sqlite backend as those
are created on demand.""" are created on demand."""
# Invalidate the cache.
self._folders = None
if self._backend == 'sqlite': if self._backend == 'sqlite':
return return # noop for sqlite which creates on-demand
if self.account.dryrun:
return # bail out in dry-run mode
filename = self.getfolderfilename(foldername) filename = self.getfolderfilename(foldername)
file = open(filename + ".tmp", "wt") file = open(filename + ".tmp", "wt")

View File

@ -16,10 +16,10 @@
# along with this program; if not, write to the Free Software # along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
from Base import BaseRepository
from offlineimap import folder from offlineimap import folder
from offlineimap.ui import getglobalui from offlineimap.ui import getglobalui
from offlineimap.error import OfflineImapError from offlineimap.error import OfflineImapError
from offlineimap.repository.Base import BaseRepository
import os import os
from stat import * from stat import *
@ -37,7 +37,7 @@ class MaildirRepository(BaseRepository):
# Create the top-level folder if it doesn't exist # Create the top-level folder if it doesn't exist
if not os.path.isdir(self.root): if not os.path.isdir(self.root):
os.mkdir(self.root, 0700) os.mkdir(self.root, 0o700)
def _append_folder_atimes(self, foldername): def _append_folder_atimes(self, foldername):
"""Store the atimes of a folder's new|cur in self.folder_atimes""" """Store the atimes of a folder's new|cur in self.folder_atimes"""
@ -81,7 +81,9 @@ class MaildirRepository(BaseRepository):
levels will be created if they do not exist yet. 'cur', levels will be created if they do not exist yet. 'cur',
'tmp', and 'new' subfolders will be created in the maildir. 'tmp', and 'new' subfolders will be created in the maildir.
""" """
self.debug("makefolder called with arg '%s'" % (foldername)) self.ui.makefolder(self, foldername)
if self.account.dryrun:
return
full_path = os.path.abspath(os.path.join(self.root, foldername)) full_path = os.path.abspath(os.path.join(self.root, foldername))
# sanity tests # sanity tests
@ -97,16 +99,16 @@ class MaildirRepository(BaseRepository):
# sub-folders may be created before higher-up ones. # sub-folders may be created before higher-up ones.
self.debug("makefolder: calling makedirs '%s'" % full_path) self.debug("makefolder: calling makedirs '%s'" % full_path)
try: try:
os.makedirs(full_path, 0700) os.makedirs(full_path, 0o700)
except OSError, e: except OSError as e:
if e.errno == 17 and os.path.isdir(full_path): if e.errno == 17 and os.path.isdir(full_path):
self.debug("makefolder: '%s' already a directory" % foldername) self.debug("makefolder: '%s' already a directory" % foldername)
else: else:
raise raise
for subdir in ['cur', 'new', 'tmp']: for subdir in ['cur', 'new', 'tmp']:
try: try:
os.mkdir(os.path.join(full_path, subdir), 0700) os.mkdir(os.path.join(full_path, subdir), 0o700)
except OSError, e: except OSError as e:
if e.errno == 17 and os.path.isdir(full_path): if e.errno == 17 and os.path.isdir(full_path):
self.debug("makefolder: '%s' already has subdir %s" % self.debug("makefolder: '%s' already has subdir %s" %
(foldername, subdir)) (foldername, subdir))

View File

@ -47,15 +47,15 @@ class Repository(object):
return LocalStatusRepository(name, account) return LocalStatusRepository(name, account)
else: else:
raise ValueError, "Request type %s not supported" % reqtype raise ValueError("Request type %s not supported" % reqtype)
config = account.getconfig() config = account.getconfig()
repostype = config.get('Repository ' + name, 'type').strip() repostype = config.get('Repository ' + name, 'type').strip()
try: try:
repo = typemap[repostype] repo = typemap[repostype]
except KeyError: except KeyError:
raise Exception, "'%s' repository not supported for %s repositories."%\ raise ValueError("'%s' repository not supported for %s repositories"
(repostype, reqtype) "." % (repostype, reqtype))
return repo(name, account) return repo(name, account)

View File

@ -1,4 +1,4 @@
# Copyright (C) 2002-2011 John Goerzen & contributors # Copyright (C) 2002-2012 John Goerzen & contributors
# Thread support module # Thread support module
# #
# This program is free software; you can redistribute it and/or modify # This program is free software; you can redistribute it and/or modify
@ -15,10 +15,12 @@
# along with this program; if not, write to the Free Software # along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
from threading import Lock, Thread, BoundedSemaphore from threading import Lock, Thread, BoundedSemaphore, currentThread
from Queue import Queue, Empty try:
from Queue import Queue, Empty
except ImportError: # python3
from queue import Queue, Empty
import traceback import traceback
from thread import get_ident # python < 2.6 support
import os.path import os.path
import sys import sys
from offlineimap.ui import getglobalui from offlineimap.ui import getglobalui
@ -149,7 +151,6 @@ class ExitNotifyThread(Thread):
def run(self): def run(self):
global exitthreads global exitthreads
self.threadid = get_ident()
try: try:
if not ExitNotifyThread.profiledir: # normal case if not ExitNotifyThread.profiledir: # normal case
Thread.run(self) Thread.run(self)
@ -164,8 +165,8 @@ class ExitNotifyThread(Thread):
except SystemExit: except SystemExit:
pass pass
prof.dump_stats(os.path.join(ExitNotifyThread.profiledir, prof.dump_stats(os.path.join(ExitNotifyThread.profiledir,
"%s_%s.prof" % (self.threadid, self.getName()))) "%s_%s.prof" % (self.ident, self.getName())))
except Exception, e: except Exception as e:
# Thread exited with Exception, store it # Thread exited with Exception, store it
tb = traceback.format_exc() tb = traceback.format_exc()
self.set_exit_exception(e, tb) self.set_exit_exception(e, tb)
@ -208,7 +209,7 @@ def initInstanceLimit(instancename, instancemax):
"""Initialize the instance-limited thread implementation to permit """Initialize the instance-limited thread implementation to permit
up to intancemax threads with the given instancename.""" up to intancemax threads with the given instancename."""
instancelimitedlock.acquire() instancelimitedlock.acquire()
if not instancelimitedsems.has_key(instancename): if not instancename in instancelimitedsems:
instancelimitedsems[instancename] = BoundedSemaphore(instancemax) instancelimitedsems[instancename] = BoundedSemaphore(instancemax)
instancelimitedlock.release() instancelimitedlock.release()

View File

@ -16,7 +16,6 @@
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
from threading import RLock, currentThread, Lock, Event from threading import RLock, currentThread, Lock, Event
from thread import get_ident # python < 2.6 support
from collections import deque from collections import deque
import time import time
import sys import sys
@ -282,7 +281,7 @@ class CursesLogHandler(logging.StreamHandler):
"""self.ui has been set to the UI class before anything is invoked""" """self.ui has been set to the UI class before anything is invoked"""
def emit(self, record): def emit(self, record):
log_str = super(CursesLogHandler, self).format(record) log_str = logging.StreamHandler.format(self, record)
color = self.ui.gettf().curses_color color = self.ui.gettf().curses_color
# We must acquire both locks. Otherwise, deadlock can result. # We must acquire both locks. Otherwise, deadlock can result.
# This can happen if one thread calls _msg (locking curses, then # This can happen if one thread calls _msg (locking curses, then

View File

@ -13,12 +13,15 @@
# You should have received a copy of the GNU General Public License # You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software # along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
from urllib import urlencode try:
from urllib import urlencode
except ImportError: # python3
from urllib.parse import urlencode
import sys import sys
import time import time
import logging import logging
from UIBase import UIBase
from threading import currentThread from threading import currentThread
from offlineimap.ui.UIBase import UIBase
import offlineimap import offlineimap
protocol = '7.0.0' protocol = '7.0.0'
@ -122,12 +125,11 @@ class MachineUI(UIBase):
"\f".join(flags), "\f".join(flags),
dest)) dest))
def threadException(s, thread): def threadException(self, thread):
print s.getThreadExceptionString(thread) self._printData('threadException', "%s\n%s" % \
s._printData('threadException', "%s\n%s" % \ (thread.getName(), self.getThreadExceptionString(thread)))
(thread.getName(), s.getThreadExceptionString(thread))) self.delThreadDebugLog(thread)
s.delThreadDebugLog(thread) self.terminate(100)
s.terminate(100)
def terminate(s, exitstatus = 0, errortitle = '', errormsg = ''): def terminate(s, exitstatus = 0, errortitle = '', errormsg = ''):
s._printData('terminate', "%d\n%s\n%s" % (exitstatus, errortitle, errormsg)) s._printData('terminate', "%d\n%s\n%s" % (exitstatus, errortitle, errormsg))

View File

@ -1,5 +1,5 @@
# Noninteractive UI # Noninteractive UI
# Copyright (C) 2002-2011 John Goerzen & contributors # Copyright (C) 2002-2012 John Goerzen & contributors
# #
# This program is free software; you can redistribute it and/or modify # This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by # it under the terms of the GNU General Public License as published by
@ -16,7 +16,7 @@
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
import logging import logging
from UIBase import UIBase from offlineimap.ui.UIBase import UIBase
class Basic(UIBase): class Basic(UIBase):
"""'Quiet' simply sets log level to INFO""" """'Quiet' simply sets log level to INFO"""

View File

@ -22,7 +22,10 @@ import sys
import os import os
import traceback import traceback
import threading import threading
from Queue import Queue try:
from Queue import Queue
except ImportError: #python3
from queue import Queue
from collections import deque from collections import deque
from offlineimap.error import OfflineImapError from offlineimap.error import OfflineImapError
import offlineimap import offlineimap
@ -45,6 +48,8 @@ def getglobalui():
class UIBase(object): class UIBase(object):
def __init__(self, config, loglevel = logging.INFO): def __init__(self, config, loglevel = logging.INFO):
self.config = config self.config = config
# Is this a 'dryrun'?
self.dryrun = config.getboolean('general', 'dry-run')
self.debuglist = [] self.debuglist = []
"""list of debugtypes we are supposed to log""" """list of debugtypes we are supposed to log"""
self.debugmessages = {} self.debugmessages = {}
@ -156,7 +161,7 @@ class UIBase(object):
def unregisterthread(self, thr): def unregisterthread(self, thr):
"""Unregister a thread as being associated with an account name""" """Unregister a thread as being associated with an account name"""
if self.threadaccounts.has_key(thr): if thr in self.threadaccounts:
del self.threadaccounts[thr] del self.threadaccounts[thr]
self.debug('thread', "Unregister thread '%s'" % thr.getName()) self.debug('thread', "Unregister thread '%s'" % thr.getName())
@ -172,7 +177,7 @@ class UIBase(object):
def debug(self, debugtype, msg): def debug(self, debugtype, msg):
cur_thread = threading.currentThread() cur_thread = threading.currentThread()
if not self.debugmessages.has_key(cur_thread): if not cur_thread in self.debugmessages:
# deque(..., self.debugmsglen) would be handy but was # deque(..., self.debugmsglen) would be handy but was
# introduced in p2.6 only, so we'll need to work around and # introduced in p2.6 only, so we'll need to work around and
# shorten our debugmsg list manually :-( # shorten our debugmsg list manually :-(
@ -203,9 +208,6 @@ class UIBase(object):
def invaliddebug(self, debugtype): def invaliddebug(self, debugtype):
self.warn("Invalid debug type: %s" % debugtype) self.warn("Invalid debug type: %s" % debugtype)
def locked(s):
raise Exception, "Another OfflineIMAP is running with the same metadatadir; exiting."
def getnicename(self, object): def getnicename(self, object):
"""Return the type of a repository or Folder as string """Return the type of a repository or Folder as string
@ -269,7 +271,7 @@ class UIBase(object):
def connecting(self, hostname, port): def connecting(self, hostname, port):
"""Log 'Establishing connection to'""" """Log 'Establishing connection to'"""
if not self.logger.isEnabledFor(logging.info): return if not self.logger.isEnabledFor(logging.INFO): return
displaystr = '' displaystr = ''
hostname = hostname if hostname else '' hostname = hostname if hostname else ''
port = "%s" % port if port else '' port = "%s" % port if port else ''
@ -296,6 +298,12 @@ class UIBase(object):
(src_repo, dst_repo)) (src_repo, dst_repo))
############################## Folder syncing ############################## Folder syncing
def makefolder(self, repo, foldername):
"""Called when a folder is created"""
prefix = "[DRYRUN] " if self.dryrun else ""
self.info("{}Creating folder {}[{}]".format(
prefix, foldername, repo))
def syncingfolder(self, srcrepos, srcfolder, destrepos, destfolder): def syncingfolder(self, srcrepos, srcfolder, destrepos, destfolder):
"""Called when a folder sync operation is started.""" """Called when a folder sync operation is started."""
self.logger.info("Syncing %s: %s -> %s" % (srcfolder, self.logger.info("Syncing %s: %s -> %s" % (srcfolder,
@ -337,8 +345,9 @@ class UIBase(object):
def deletingmessages(self, uidlist, destlist): def deletingmessages(self, uidlist, destlist):
ds = self.folderlist(destlist) ds = self.folderlist(destlist)
self.logger.info("Deleting %d messages (%s) in %s" % ( prefix = "[DRYRUN] " if self.dryrun else ""
len(uidlist), self.info("{}Deleting {} messages ({}) in {}".format(
prefix, len(uidlist),
offlineimap.imaputil.uid_sequence(uidlist), ds)) offlineimap.imaputil.uid_sequence(uidlist), ds))
def addingflags(self, uidlist, flags, dest): def addingflags(self, uidlist, flags, dest):
@ -400,10 +409,15 @@ class UIBase(object):
if conn: #release any existing IMAP connection if conn: #release any existing IMAP connection
repository.imapserver.close() repository.imapserver.close()
def savemessage(self, debugtype, uid, flags, folder):
"""Output a log line stating that we save a msg"""
self.debug(debugtype, "Write mail '%s:%d' with flags %s" %
(folder, uid, repr(flags)))
################################################## Threads ################################################## Threads
def getThreadDebugLog(self, thread): def getThreadDebugLog(self, thread):
if self.debugmessages.has_key(thread): if thread in self.debugmessages:
message = "\nLast %d debug messages logged for %s prior to exception:\n"\ message = "\nLast %d debug messages logged for %s prior to exception:\n"\
% (len(self.debugmessages[thread]), thread.getName()) % (len(self.debugmessages[thread]), thread.getName())
message += "\n".join(self.debugmessages[thread]) message += "\n".join(self.debugmessages[thread])
@ -459,6 +473,9 @@ class UIBase(object):
################################################## Hooks ################################################## Hooks
def callhook(self, msg): def callhook(self, msg):
if self.dryrun:
self.info("[DRYRUN] {}".format(msg))
else:
self.info(msg) self.info(msg)
################################################## Other ################################################## Other

View File

@ -29,7 +29,10 @@ from test.OLItest import TextTestRunner, TestLoader, OLITestLib
class TestCommand(Command): class TestCommand(Command):
"""runs the OLI testsuite""" """runs the OLI testsuite"""
description = "Runs the test suite" description = """Runs the test suite. In order to execute only a single
test, you could also issue e.g. 'python -m unittest
test.tests.test_01_basic.TestBasicFunctions.test_01_olistartup' on the
command line."""
user_options = [] user_options = []
def initialize_options(self): def initialize_options(self):

View File

@ -13,16 +13,24 @@
# You should have received a copy of the GNU General Public License # You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software # along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
import imaplib
import unittest import unittest
import logging import logging
import os import os
import re
import sys import sys
import shutil import shutil
import subprocess import subprocess
import tempfile import tempfile
from ConfigParser import SafeConfigParser import random
random.seed()
try:
from configparser import SafeConfigParser
except ImportError: # python 2
from ConfigParser import SafeConfigParser
from . import default_conf from . import default_conf
class OLITestLib(): class OLITestLib():
cred_file = None cred_file = None
testdir = None testdir = None
@ -50,30 +58,45 @@ class OLITestLib():
directory at a time. OLITestLib is not suited for running directory at a time. OLITestLib is not suited for running
several tests in parallel. The user is responsible for several tests in parallel. The user is responsible for
cleaning that up herself.""" cleaning that up herself."""
assert cls.cred_file != None
# creating temporary dir for testing in same dir as credentials.conf # creating temporary dir for testing in same dir as credentials.conf
cls.testdir = os.path.abspath( cls.testdir = os.path.abspath(
tempfile.mkdtemp(prefix='tmp_%s_'%suffix, tempfile.mkdtemp(prefix='tmp_%s_'%suffix,
dir=os.path.dirname(cls.cred_file))) dir=os.path.dirname(cls.cred_file)))
cls.create_config_file() cls.write_config_file()
return cls.testdir return cls.testdir
@classmethod @classmethod
def create_config_file(cls): def get_default_config(cls):
"""Creates a OLI configuration file """Creates a default ConfigParser file and returns it
It is created in testdir (so create_test_dir has to be called The returned config can be manipulated and then saved with
earlier) using the credentials information given (so they had to write_config_file()"""
be set earlier). Failure to do either of them will raise an #TODO, only do first time and cache then for subsequent calls?
AssertionException."""
assert cls.cred_file != None assert cls.cred_file != None
assert cls.testdir != None assert cls.testdir != None
config = SafeConfigParser() config = SafeConfigParser()
config.readfp(default_conf) config.readfp(default_conf)
default_conf.seek(0) # rewind config_file to start
config.read(cls.cred_file) config.read(cls.cred_file)
config.set("general", "metadata", cls.testdir) config.set("general", "metadata", cls.testdir)
return config
@classmethod
def write_config_file(cls, config=None):
"""Creates a OLI configuration file
It is created in testdir (so create_test_dir has to be called
earlier) using the credentials information given (so they had
to be set earlier). Failure to do either of them will raise an
AssertionException. If config is None, a default one will be
used via get_default_config, otherwise it needs to be a config
object derived from that."""
if config is None:
config = cls.get_default_config()
localfolders = os.path.join(cls.testdir, 'mail') localfolders = os.path.join(cls.testdir, 'mail')
config.set("Repository Maildir", "localfolders", localfolders) config.set("Repository Maildir", "localfolders", localfolders)
with open(os.path.join(cls.testdir, 'offlineimap.conf'), "wa") as f: with open(os.path.join(cls.testdir, 'offlineimap.conf'), "wt") as f:
config.write(f) config.write(f)
@classmethod @classmethod
@ -88,7 +111,7 @@ class OLITestLib():
def run_OLI(cls): def run_OLI(cls):
"""Runs OfflineImap """Runs OfflineImap
:returns: (rescode, stdout) :returns: (rescode, stdout (as unicode))
""" """
try: try:
output = subprocess.check_output( output = subprocess.check_output(
@ -96,8 +119,54 @@ class OLITestLib():
"-c%s" % os.path.join(cls.testdir, 'offlineimap.conf')], "-c%s" % os.path.join(cls.testdir, 'offlineimap.conf')],
shell=False) shell=False)
except subprocess.CalledProcessError as e: except subprocess.CalledProcessError as e:
return (e.returncode, e.output) return (e.returncode, e.output.decode('utf-8'))
return (0, output) return (0, output.decode('utf-8'))
@classmethod
def delete_remote_testfolders(cls, reponame=None):
"""Delete all INBOX.OLITEST* folders on the remote IMAP repository
reponame: All on `reponame` or all IMAP-type repositories if None"""
config = cls.get_default_config()
if reponame:
sections = ['Repository {}'.format(reponame)]
else:
sections = [r for r in config.sections() \
if r.startswith('Repository')]
sections = filter(lambda s: \
config.get(s, 'Type').lower() == 'imap',
sections)
for sec in sections:
# Connect to each IMAP repo and delete all folders
# matching the folderfilter setting. We only allow basic
# settings and no fancy password getting here...
# 1) connect and get dir listing
host = config.get(sec, 'remotehost')
user = config.get(sec, 'remoteuser')
passwd = config.get(sec, 'remotepass')
imapobj = imaplib.IMAP4(host)
imapobj.login(user, passwd)
res_t, data = imapobj.list()
assert res_t == 'OK'
dirs = []
for d in data:
m = re.search(br''' # Find last quote
"((?: # Non-tripple quoted can contain...
[^"] | # a non-quote
\\" # a backslashded quote
)*)" # closing quote
[^"]*$ # followed by no more quotes
''', d, flags=re.VERBOSE)
folder = bytearray(m.group(1))
#folder = folder.replace(br'\"', b'"') # remove quoting
dirs.append(folder)
# 2) filter out those not starting with INBOX.OLItest and del...
dirs = [d for d in dirs if d.startswith(b'INBOX.OLItest')]
for folder in dirs:
res_t, data = imapobj.delete(b'\"'+folder+b'\"')
assert res_t == 'OK', "Folder deletion of {} failed with error"\
":\n{} {}".format(folder.decode('utf-8'), res_t, data)
imapobj.logout()
@classmethod @classmethod
def create_maildir(cls, folder): def create_maildir(cls, folder):
@ -108,11 +177,41 @@ class OLITestLib():
maildir = os.path.join(cls.testdir, 'mail', folder) maildir = os.path.join(cls.testdir, 'mail', folder)
for subdir in ('','tmp','cur','new'): for subdir in ('','tmp','cur','new'):
try: try:
os.mkdir(os.path.join(maildir, subdir)) os.makedirs(os.path.join(maildir, subdir))
except OSError as e: except OSError as e:
if e.errno != 17: # 'already exists' is ok. if e.errno != 17: # 'already exists' is ok.
raise raise
@classmethod
def delete_maildir(cls, folder):
"""Delete maildir 'folder' in our test maildir
Does not fail if not existing"""
assert cls.testdir != None
maildir = os.path.join(cls.testdir, 'mail', folder)
shutil.rmtree(maildir, ignore_errors=True)
@classmethod
def create_mail(cls, folder, mailfile=None, content=None):
"""Create a mail in maildir 'folder'/new
Use default mailfilename if not given.
Use some default content if not given"""
assert cls.testdir != None
while True: # Loop till we found a unique filename
mailfile = '{}:2,'.format(random.randint(0,999999999))
mailfilepath = os.path.join(cls.testdir, 'mail',
folder, 'new', mailfile)
if not os.path.isfile(mailfilepath):
break
with open(mailfilepath,"wb") as mailf:
mailf.write(b'''From: test <test@offlineimap.org>
Subject: Boo
Date: 1 Jan 1980
To: test@offlineimap.org
Content here.''')
@classmethod @classmethod
def count_maildir_mails(cls, folder): def count_maildir_mails(cls, folder):
"""Returns the number of mails in maildir 'folder' """Returns the number of mails in maildir 'folder'
@ -130,3 +229,22 @@ class OLITestLib():
if dirpath.endswith(('/cur', '/new')): if dirpath.endswith(('/cur', '/new')):
mails += len(files) mails += len(files)
return boxes, mails return boxes, mails
# find UID in a maildir filename
re_uidmatch = re.compile(',U=(\d+)')
@classmethod
def get_maildir_uids(cls, folder):
"""Returns a list of maildir mail uids, 'None' if no valid uid"""
assert cls.testdir != None
mailfilepath = os.path.join(cls.testdir, 'mail', folder)
assert os.path.isdir(mailfilepath)
ret = []
for dirpath, dirs, files in os.walk(mailfilepath):
if not dirpath.endswith((os.path.sep + 'new', os.path.sep + 'cur')):
continue # only /new /cur are interesting
for file in files:
m = cls.re_uidmatch.search(file)
uid = m.group(1) if m else None
ret.append(uid)
return ret

View File

@ -30,5 +30,5 @@ banner = """%(__productname__)s %(__version__)s
import unittest import unittest
from unittest import TestLoader, TextTestRunner from unittest import TestLoader, TextTestRunner
from globals import default_conf from .globals import default_conf
from TestRunner import OLITestLib from .TestRunner import OLITestLib

View File

@ -14,7 +14,10 @@
# You should have received a copy of the GNU General Public License # You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software # along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
from cStringIO import StringIO try:
from cStringIO import StringIO
except ImportError: #python3
from io import StringIO
default_conf=StringIO("""[general] default_conf=StringIO("""[general]
#will be set automatically #will be set automatically
@ -33,5 +36,7 @@ localfolders =
[Repository IMAP] [Repository IMAP]
type=IMAP type=IMAP
# Don't hammer the server with too many connection attempts:
maxconnections=1
folderfilter= lambda f: f.startswith('INBOX.OLItest') folderfilter= lambda f: f.startswith('INBOX.OLItest')
""") """)

View File

@ -19,12 +19,19 @@ import logging
import os, sys import os, sys
from test.OLItest import OLITestLib from test.OLItest import OLITestLib
# Things need to be setup first, usually setup.py initializes everything.
# but if e.g. called from command line, we take care of default values here:
if not OLITestLib.cred_file:
OLITestLib(cred_file='./test/credentials.conf', cmd='./offlineimap.py')
def setUpModule(): def setUpModule():
logging.info("Set Up test module %s" % __name__) logging.info("Set Up test module %s" % __name__)
tdir = OLITestLib.create_test_dir(suffix=__name__) tdir = OLITestLib.create_test_dir(suffix=__name__)
def tearDownModule(): def tearDownModule():
logging.info("Tear Down test module") logging.info("Tear Down test module")
# comment out next line to keep testdir after test runs. TODO: make nicer
OLITestLib.delete_test_dir() OLITestLib.delete_test_dir()
#Stuff that can be used #Stuff that can be used
@ -52,13 +59,16 @@ class TestBasicFunctions(unittest.TestCase):
def test_01_olistartup(self): def test_01_olistartup(self):
"""Tests if OLI can be invoked without exceptions """Tests if OLI can be invoked without exceptions
It syncs all "OLItest* (specified in the default config) to our Cleans existing remote tet folders. Then syncs all "OLItest*
local Maildir at keeps it there.""" (specified in the default config) to our local Maildir. The
result should be 0 folders and 0 mails."""
OLITestLib.delete_remote_testfolders()
code, res = OLITestLib.run_OLI() code, res = OLITestLib.run_OLI()
self.assertEqual(res, "") self.assertEqual(res, "")
boxes, mails = OLITestLib.count_maildir_mails('') boxes, mails = OLITestLib.count_maildir_mails('')
logging.warn("%d boxes and %d mails" % (boxes, mails)) self.assertTrue((boxes, mails)==(0,0), msg="Expected 0 folders and 0 "
"mails, but sync led to {} folders and {} mails".format(
boxes, mails))
def test_02_createdir(self): def test_02_createdir(self):
"""Create local OLItest 1 & OLItest "1" maildir, sync """Create local OLItest 1 & OLItest "1" maildir, sync
@ -71,4 +81,49 @@ class TestBasicFunctions(unittest.TestCase):
#logging.warn("%s %s "% (code, res)) #logging.warn("%s %s "% (code, res))
self.assertEqual(res, "") self.assertEqual(res, "")
boxes, mails = OLITestLib.count_maildir_mails('') boxes, mails = OLITestLib.count_maildir_mails('')
logging.warn("%d boxes and %d mails" % (boxes, mails)) self.assertTrue((boxes, mails)==(2,0), msg="Expected 2 folders and 0 "
"mails, but sync led to {} folders and {} mails".format(
boxes, mails))
def test_03_nametransmismatch(self):
"""Create mismatching remote and local nametrans rules
This should raise an error."""
config = OLITestLib.get_default_config()
config.set('Repository IMAP', 'nametrans',
'lambda f: f' )
config.set('Repository Maildir', 'nametrans',
'lambda f: f + "moo"' )
OLITestLib.write_config_file(config)
code, res = OLITestLib.run_OLI()
#logging.warn("%s %s "% (code, res))
# We expect an INFINITE FOLDER CREATION WARNING HERE....
mismatch = "ERROR: INFINITE FOLDER CREATION DETECTED!" in res
self.assertEqual(mismatch, True, msg="Mismatching nametrans rules did "
"NOT trigger an 'infinite folder generation' error. Output was:\n"
"{}".format(res))
# Write out default config file again
OLITestLib.write_config_file()
def test_04_createmail(self):
"""Create mail in OLItest 1, sync, wipe folder sync
Currently, this will mean the folder will be recreated
locally. At some point when remote folder deletion is
implemented, this behavior will change."""
OLITestLib.delete_remote_testfolders()
OLITestLib.delete_maildir('') #Delete all local maildir folders
OLITestLib.create_maildir('INBOX.OLItest')
OLITestLib.create_mail('INBOX.OLItest')
code, res = OLITestLib.run_OLI()
#logging.warn("%s %s "% (code, res))
self.assertEqual(res, "")
boxes, mails = OLITestLib.count_maildir_mails('')
self.assertTrue((boxes, mails)==(1,1), msg="Expected 1 folders and 1 "
"mails, but sync led to {} folders and {} mails".format(
boxes, mails))
# The local Mail should have been assigned a proper UID now, check!
uids = OLITestLib.get_maildir_uids('INBOX.OLItest')
self.assertFalse (None in uids, msg = "All mails should have been "+ \
"assigned the IMAP's UID number, but {} messages had no valid ID "\
.format(len([None for x in uids if x==None])))

View File

@ -0,0 +1,71 @@
# Copyright (C) 2012- Sebastian Spaeth & contributors
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
import random
import unittest
import logging
import os, sys
from test.OLItest import OLITestLib
# Things need to be setup first, usually setup.py initializes everything.
# but if e.g. called from command line, we take care of default values here:
if not OLITestLib.cred_file:
OLITestLib(cred_file='./test/credentials.conf', cmd='./offlineimap.py')
def setUpModule():
logging.info("Set Up test module %s" % __name__)
tdir = OLITestLib.create_test_dir(suffix=__name__)
def tearDownModule():
logging.info("Tear Down test module")
OLITestLib.delete_test_dir()
#Stuff that can be used
#self.assertEqual(self.seq, range(10))
# should raise an exception for an immutable sequence
#self.assertRaises(TypeError, random.shuffle, (1,2,3))
#self.assertTrue(element in self.seq)
#self.assertFalse(element in self.seq)
class TestBasicFunctions(unittest.TestCase):
#@classmethod
#def setUpClass(cls):
#This is run before all tests in this class
# cls._connection = createExpensiveConnectionObject()
#@classmethod
#This is run after all tests in this class
#def tearDownClass(cls):
# cls._connection.destroy()
# This will be run before each test
#def setUp(self):
# self.seq = range(10)
def test_01_MappedImap(self):
"""Tests if a MappedIMAP sync can be invoked without exceptions
Cleans existing remote test folders. Then syncs all "OLItest*
(specified in the default config) to our local IMAP (Gmail). The
result should be 0 folders and 0 mails."""
pass #TODO
#OLITestLib.delete_remote_testfolders()
#code, res = OLITestLib.run_OLI()
#self.assertEqual(res, "")
#boxes, mails = OLITestLib.count_maildir_mails('')
#self.assertTrue((boxes, mails)==(0,0), msg="Expected 0 folders and 0"
# "mails, but sync led to {} folders and {} mails".format(
# boxes, mails))