From 0fe456226920361cbf9fee7948da69e6d0ba82f0 Mon Sep 17 00:00:00 2001 From: Nicolas Sebrecht Date: Sat, 5 Nov 2016 16:38:29 +0100 Subject: [PATCH] manual: add known issues entry about XOAUTH2 "invalid_grant" - Split the manual asciidoc source file. - Improve the Makefile to build the manuals. Github-ref: https://github.com/OfflineIMAP/offlineimap/issues/374 Signed-off-by: Nicolas Sebrecht --- docs/Makefile | 6 +- docs/offlineimap.known_issues.txt | 142 ++++++++++++++++++++++++++++++ docs/offlineimap.txt | 98 +-------------------- offlineimap.conf | 75 ++++++++-------- offlineimap/imapserver.py | 3 +- 5 files changed, 188 insertions(+), 136 deletions(-) create mode 100644 docs/offlineimap.known_issues.txt diff --git a/docs/Makefile b/docs/Makefile index aadfd66..0753cfe 100644 --- a/docs/Makefile +++ b/docs/Makefile @@ -18,11 +18,11 @@ $(HTML_TARGETS): %.html : %.rst man: offlineimap.1 offlineimapui.7 -offlineimap.1: offlineimap.txt - a2x -v -f manpage $? +offlineimap.1: offlineimap.txt offlineimap.known_issues.txt + a2x -v -d manpage -f manpage $< offlineimapui.7: offlineimapui.txt - a2x -v -f manpage $? + a2x -v -d manpage -f manpage $< api: $(SPHINXBUILD) -b html -d html/doctrees doc-src html diff --git a/docs/offlineimap.known_issues.txt b/docs/offlineimap.known_issues.txt new file mode 100644 index 0000000..c3a6639 --- /dev/null +++ b/docs/offlineimap.known_issues.txt @@ -0,0 +1,142 @@ + +* SSL3 write pending. ++ +Users enabling SSL may hit a bug about "SSL3 write pending". If so, the +account(s) will stay unsynchronised from the time the bug appeared. Running +OfflineIMAP again can help. We are still working on this bug. Patches or +detailed bug reports would be appreciated. Please check you're running the +last stable version and send us a report to the mailing list including the +full log. + +* IDLE support is incomplete and experimental. Bugs may be encountered. + + - No hook exists for "run after an IDLE response". ++ +Email will show up, but may not be processed until the next refresh cycle. + + - nametrans may not be supported correctly. + + - IMAP IDLE <-> IMAP IDLE doesn't work yet. + + - IDLE might stop syncing on a system suspend/resume. + + - IDLE may only work "once" per refresh. ++ +If you encounter this bug, please send a report to the list! + +* Maildir support in Windows drive. ++ +Maildir uses colon character (:) in message file names. Colon is however +forbidden character in windows drives. There are several workarounds for that +situation: + + . Enable file name character translation in windows registry (not tested). + - + + . Use cygwin managed mount (not tested). + - not available anymore since cygwin 1.7 + + . Use "maildir-windows-compatible = yes" account OfflineIMAP configuration. + - That makes OfflineIMAP to use exclamation mark (!) instead of colon for + storing messages. Such files can be written to windows partitions. But + you will probably loose compatibility with other programs trying to + read the same Maildir. ++ + - Exclamation mark was chosen because of the note in + http://docs.python.org/library/mailbox.html ++ + - If you have some messages already stored without this option, you will + have to re-sync them again + +* OfflineIMAP confused after system suspend. ++ +When resuming a suspended session, OfflineIMAP does not cleanly handles the +broken socket(s) if socktimeout option is not set. +You should enable this option with a value like 10. + +* OfflineIMAP confused when mails change while in a sync. ++ +When OfflineIMAP is syncing, some events happening since the invocation on +remote or local side are badly handled. OfflineIMAP won't track for changes +during the sync. + + +* Sharing a maildir with multiple IMAP servers. ++ +Generally a word of caution mixing IMAP repositories on the same Maildir root. +You have to be careful that you *never* use the same maildir folder for 2 IMAP +servers. In the best case, the folder MD5 will be different, and you will get +a loop where it will upload your mails to both servers in turn (infinitely!) +as it thinks you have placed new mails in the local Maildir. In the worst +case, the MD5 is the same (likely) and mail UIDs overlap (likely too!) and it +will fail to sync some mails as it thinks they are already existent. ++ +I would create a new local Maildir Repository for the Personal Gmail and +use a different root to be on the safe side here. You could e.g. use + + `~/mail/Pro' as Maildir root for the ProGmail and + `~/mail/Personal' as root for the personal one. ++ +If you then point your local mutt, or whatever MUA you use to `~/mail/' +as root, it should still recognize all folders. + + +* Edge cases with maxage causing too many messages to be synced. ++ +All messages from at most maxage days ago (+/- a few hours, depending on +timezones) are synced, but there are cases in which older messages can also be +synced. This happens when a message's UID is significantly higher than those of +other messages with similar dates, e.g. when messages are added to the local +folder behind offlineimap's back, causing them to get assigned a new UID, or +when offlineimap first syncs a pre-existing Maildir. In the latter case, it +could appear as if a noticeable and random subset of old messages are synced. + +* Passwords in netrc. ++ +Offlineimap doesn't know how to retrieve passwords when more than one account is +stored in the netrc file. See +. + +* XOAUTH2 ++ +XOAUTH2 might be a bit tricky to set up. Make sure you've followed the step to +step guide in 'offlineimap.conf'. The known bugs about Gmail are tracked at +. ++ +Sometimes, you might hit one of the following error: + +- [imap]: xoauth2handler: response "{u'error': u'invalid_grant'}" +- oauth2handler got: {u'error': u'invalid_grant'} + ++ +In such case, we had report that generating a new refesh token from the same +client ID and secret can help. ++ +.Google documentation on "invalid_grant" +---- +When you try to use a refresh token, the following returns you an +invalid_grant error: + + - Your server's clock is not in sync with network time protocol - NTP. + - The refresh token limit has been exceeded. +---- ++ +.Token expiration +---- +It is possible that a granted token might no longer work. A token might stop +working for one of these reasons: + + - The user has revoked access. + - The token has not been used for six months. + - The user changed passwords and the token contains Gmail scopes. + - The user account has exceeded a certain number of token requests. + +There is currently a limit of 50 refresh tokens per user account per client. If +the limit is reached, creating a new token automatically invalidates the oldest +token without warning. This limit does not apply to service accounts. +---- ++ +See +and +to know more. + diff --git a/docs/offlineimap.txt b/docs/offlineimap.txt index 48b55dc..fc4a966 100644 --- a/docs/offlineimap.txt +++ b/docs/offlineimap.txt @@ -335,103 +335,7 @@ process core. Known Issues ------------ -* SSL3 write pending. -+ -Users enabling SSL may hit a bug about "SSL3 write pending". If so, the -account(s) will stay unsynchronised from the time the bug appeared. Running -OfflineIMAP again can help. We are still working on this bug. Patches or -detailed bug reports would be appreciated. Please check you're running the -last stable version and send us a report to the mailing list including the -full log. - -* IDLE support is incomplete and experimental. Bugs may be encountered. - - - No hook exists for "run after an IDLE response". -+ -Email will show up, but may not be processed until the next refresh cycle. - - - nametrans may not be supported correctly. - - - IMAP IDLE <-> IMAP IDLE doesn't work yet. - - - IDLE might stop syncing on a system suspend/resume. - - - IDLE may only work "once" per refresh. -+ -If you encounter this bug, please send a report to the list! - -* Maildir support in Windows drive. -+ -Maildir uses colon caracter (:) in message file names. Colon is however -forbidden character in windows drives. There are several workarounds for that -situation: - - . Enable file name character translation in windows registry (not tested). - - - - . Use cygwin managed mount (not tested). - - not available anymore since cygwin 1.7 - - . Use "maildir-windows-compatible = yes" account OfflineIMAP configuration. - - That makes OfflineIMAP to use exclamation mark (!) instead of colon for - storing messages. Such files can be written to windows partitions. But - you will probably loose compatibility with other programs trying to - read the same Maildir. -+ - - Exclamation mark was chosen because of the note in - http://docs.python.org/library/mailbox.html -+ - - If you have some messages already stored without this option, you will - have to re-sync them again - -* OfflineIMAP confused after system suspend. -+ -When resuming a suspended session, OfflineIMAP does not cleanly handles the -broken socket(s) if socktimeout option is not set. -You should enable this option with a value like 10. - -* OfflineIMAP confused when mails change while in a sync. -+ -When OfflineIMAP is syncing, some events happening since the invocation on -remote or local side are badly handled. OfflineIMAP won't track for changes -during the sync. - - -* Sharing a maildir with multiple IMAP servers. -+ -Generally a word of caution mixing IMAP repositories on the same Maildir root. -You have to be careful that you *never* use the same maildir folder for 2 IMAP -servers. In the best case, the folder MD5 will be different, and you will get -a loop where it will upload your mails to both servers in turn (infinitely!) -as it thinks you have placed new mails in the local Maildir. In the worst -case, the MD5 is the same (likely) and mail UIDs overlap (likely too!) and it -will fail to sync some mails as it thinks they are already existent. -+ -I would create a new local Maildir Repository for the Personal Gmail and -use a different root to be on the safe side here. You could e.g. use - - `~/mail/Pro' as Maildir root for the ProGmail and - `~/mail/Personal' as root for the personal one. -+ -If you then point your local mutt, or whatever MUA you use to `~/mail/' -as root, it should still recognize all folders. - - -* Edge cases with maxage causing too many messages to be synced. -+ -All messages from at most maxage days ago (+/- a few hours, depending on -timezones) are synced, but there are cases in which older messages can also be -synced. This happens when a message's UID is significantly higher than those of -other messages with similar dates, e.g. when messages are added to the local -folder behind offlineimap's back, causing them to get assigned a new UID, or -when offlineimap first syncs a pre-existing Maildir. In the latter case, it -could appear as if a noticeable and random subset of old messages are synced. - -* Passwords in netrc. -+ -Offlineimap doesn't know how to retrieve passwords when more than one account is -stored in the netrc file. See -. +include::./offlineimap.known_issues.txt[] Main authors diff --git a/offlineimap.conf b/offlineimap.conf index dbd2d1c..1cc853a 100644 --- a/offlineimap.conf +++ b/offlineimap.conf @@ -10,10 +10,10 @@ # The default configuration file is "~/.offlineimaprc". # -# OfflineIMAP ships with a file named "offlineimap.conf" that you should copy to +# Offlineimap ships with a file named "offlineimap.conf" that you should copy to # that location and then edit. # -# OfflineIMAP also ships a file named "offlineimap.conf.minimal" that you can +# Offlineimap also ships a file named "offlineimap.conf.minimal" that you can # also try. It's useful if you want to get started with the most basic feature # set, and you can read about other features later with "offlineimap.conf". # @@ -56,7 +56,7 @@ [general] -# This specifies where OfflineIMAP is to store its metadata. +# This specifies where Offlineimap is to store its metadata. # This directory will be created if it does not already exist. # # Tilde and environment variable expansions will be performed. @@ -91,7 +91,7 @@ accounts = Test # This option stands in the [general] section. # -# You can specify one or more user interface. OfflineIMAP will try the first in +# You can specify one or more user interface. Offlineimap will try the first in # the list, and if it fails, the second, and so forth. # # The pre-defined options are: @@ -113,11 +113,11 @@ accounts = Test # This option stands in the [general] section. # # If you try to synchronize messages to a folder which the IMAP server -# considers read-only, OfflineIMAP will generate a warning. If you want +# considers read-only, Offlineimap will generate a warning. If you want # to suppress these warnings, set ignore-readonly to yes. Read-only # IMAP folders allow reading but not modification, so if you try to # change messages in the local copy of such a folder, the IMAP server -# will prevent OfflineIMAP from propagating those changes to the IMAP +# will prevent Offlineimap from propagating those changes to the IMAP # server. Note that ignore-readonly is UNRELATED to the "readonly" # setting which prevents a repository from being modified at all. # @@ -141,7 +141,7 @@ accounts = Test # This option is in the [general] section. # -# By default, OfflineIMAP will not exit due to a network error until the +# By default, Offlineimap will not exit due to a network error until the # operating system returns an error code. Operating systems can sometimes take # forever to notice this. Here you can activate a timeout on the socket. This # timeout applies to individual socket reads and writes, not to an overall sync @@ -157,10 +157,10 @@ accounts = Test # This option stands in the [general] section. # -# By default, OfflineIMAP will use fsync() to force data out to disk at +# By default, Offlineimap will use fsync() to force data out to disk at # opportune times to ensure consistency. This can, however, reduce performance. # Users where /home is on SSD (Flash) may also wish to reduce write cycles. -# Therefore, you can disable OfflineIMAP's use of fsync(). Doing so will come +# Therefore, you can disable Offlineimap's use of fsync(). Doing so will come # at the expense of greater risk of message duplication in the event of a system # crash or power loss. Default is true. Set it to false to disable fsync. # @@ -176,7 +176,7 @@ accounts = Test [mbnames] -# OfflineIMAP can record your mailbox names in a format you specify. +# Offlineimap can record your mailbox names in a format you specify. # You can define the header, each mailbox item, the separator, # and the footer. Here is an example for Mutt. # If enabled is yes, all settings except incremental must be specified, even if @@ -253,7 +253,7 @@ remoterepository = RemoteExample # This option stands in the [Account Test] section. # -# You can have OfflineIMAP continue running indefinitely, automatically syncing +# You can have Offlineimap continue running indefinitely, automatically syncing # your mail periodically. If you want that, specify how frequently to do that # (in minutes) here. Fractional minutes (ie, 3.25) is allowed. # @@ -310,7 +310,7 @@ remoterepository = RemoteExample # # If you have a limited amount of bandwidth available you can exclude larger # messages (e.g. those with large attachments etc). If you do this it will -# appear to OfflineIMAP that these messages do not exist at all. They will not +# appear to Offlineimap that these messages do not exist at all. They will not # be copied, have flags changed etc. For this to work on an IMAP server the # server must have server side search enabled. This works with Gmail and most # imap servers (e.g. cyrus etc) @@ -328,7 +328,7 @@ remoterepository = RemoteExample # messages later than that date will be synced. # # Messages older than the cutoff will not be synced, their flags will not be -# changed, they will not be deleted, etc. For OfflineIMAP it will be like these +# changed, they will not be deleted, etc. For Offlineimap it will be like these # messages do not exist. This will perform an IMAP search in the case of IMAP or # Gmail and therefore requires that the server support server side searching. # @@ -348,7 +348,7 @@ remoterepository = RemoteExample # # Maildir file format uses colon (:) separator between uniq name and info. # Unfortunatelly colon is not allowed character in windows file name. If you -# enable maildir-windows-compatible option, OfflineIMAP will be able to store +# enable maildir-windows-compatible option, Offlineimap will be able to store # messages on windows drive, but you will probably loose compatibility with # other programs working with the maildir. # @@ -396,7 +396,7 @@ remoterepository = RemoteExample # This option stands in the [Account Test] section. # -# OfflineIMAP can strip off some headers when your messages are propagated +# Offlineimap can strip off some headers when your messages are propagated # back to the IMAP server. This option carries the comma-separated list # of headers to trim off. Header name matching is case-sensitive. # @@ -432,7 +432,7 @@ type = Maildir # This option stands in the [Repository LocalExample] section. # # Specify local repository. Your IMAP folders will be synchronized -# to maildirs created under this path. OfflineIMAP will create the +# to maildirs created under this path. Offlineimap will create the # maildirs for you as needed. # localfolders = ~/Test @@ -479,7 +479,7 @@ localfolders = ~/Test # This option stands in the [Repository LocalExample] section. # # Some users may not want the atime (last access time) of folders to be -# modified by OfflineIMAP. If 'restoreatime' is set to yes, OfflineIMAP +# modified by Offlineimap. If 'restoreatime' is set to yes, Offlineimap # will restore the atime of the "new" and "cur" folders in each maildir # folder to their original value after each sync. # @@ -560,7 +560,7 @@ localfolders = ~/Test # some extra overhead which sometimes may be significant. We look for # modified tags in local messages by looking only to the files # modified since last run. This is usually rather fast, but the first -# time OfflineIMAP runs with synclabels enabled, it will have to check +# time Offlineimap runs with synclabels enabled, it will have to check # the contents of all individual messages for labels and this may take # a while. # @@ -635,7 +635,7 @@ remotehost = examplehost # # Note: be care to configure the 'remotehost' line with the domain name defined # in the certificate. E.g., if you trust your provider and want to use the -# certificate it provides on a shared server. Otherwise, OfflineIMAP will stop +# certificate it provides on a shared server. Otherwise, Offlineimap will stop # and say that the domain is not named in the certificate. # # Default is yes. @@ -671,10 +671,10 @@ remotehost = examplehost # # Tilde and environment variable expansions will be performed. # -# Special value OS-DEFAULT makes OfflineIMAP to automatically +# Special value OS-DEFAULT makes Offlineimap to automatically # determine system-wide location of standard trusted CA roots file # for known OS distributions and use the first bundle encountered -# (if any). If no system-wide CA bundle is found, OfflineIMAP +# (if any). If no system-wide CA bundle is found, Offlineimap # will refuse to continue; this is done to prevent creation # of false security expectations ("I had configured CA bundle, # thou certificate verification shalt be present"). @@ -688,15 +688,15 @@ remotehost = examplehost # This option stands in the [Repository RemoteExample] section. # # If you connect via SSL/TLS (ssl = yes) and you have no CA certificate -# specified, OfflineIMAP will refuse to sync as it connects to a server +# specified, Offlineimap will refuse to sync as it connects to a server # with an unknown "fingerprint". If you are sure you connect to the # correct server, you can then configure the presented server -# fingerprint here. OfflineIMAP will verify that the server fingerprint +# fingerprint here. Offlineimap will verify that the server fingerprint # has not changed on each connect and refuse to connect otherwise. # # You can also configure fingerprint validation in addition to # CA certificate validation above and it will check both: -# OfflineIMAP fill verify certificate first and if things will be fine, +# Offlineimap fill verify certificate first and if things will be fine, # fingerprint will be validated. # # Multiple fingerprints can be specified, separated by commas. @@ -795,7 +795,7 @@ remoteuser = username # This option stands in the [Repository RemoteExample] section. # # Specify which authentication/authorization mechanisms we should try and the -# order in which OfflineIMAP will try them. +# order in which Offlineimap will try them. # # NOTE: any given mechanism will be tried ONLY if it is supported by the remote # IMAP server. @@ -809,7 +809,7 @@ remoteuser = username # This option stands in the [Repository RemoteExample] section. # -# XOAuth2 authentication (for instance, to use with Gmail). +# XOAUTH2 authentication (for instance, to use with Gmail). # # This option was tested on Gmail only, but should work # with type = IMAP for compatible servers. @@ -859,11 +859,14 @@ remoteuser = username # If the type of the remote is IMAP, oauth2_request_url MUST be defined. # For Gmail, the default URL is https://accounts.google.com/o/oauth2/token. # +# If you're experiencing issues, please read the "Known issues" section of the +# manual. +# #oauth2_access_token = ACCESS_TOKEN #oauth2_request_url = https://accounts.google.com/o/oauth2/token #oauth2_refresh_token = REFRESH_TOKEN # -# The return values must be bytes. +# The returned values must be bytes. #oauth2_access_token_eval = get_access_token("accountname") #oauth2_refresh_token_eval = get_refresh_token("accountname") @@ -877,7 +880,7 @@ remoteuser = username # allows one entry per hostname. If there is no ~/.netrc file but # there is an /etc/netrc file, the password will instead be taken # from there. Otherwise you will be prompted for the password when -# OfflineIMAP starts when using a UI that supports this. +# Offlineimap starts when using a UI that supports this. # # 2. The remote password stored in this file with the remotepass # option. Save this file with the UTF-8 encoding if your server expect UTF-8 @@ -901,7 +904,7 @@ remoteuser = username # # 5. If you are using Kerberos and have the Python Kerberos package # installed, you should not specify a remotepass. If the user has a -# valid Kerberos TGT, OfflineIMAP will figure out the rest all by +# valid Kerberos TGT, Offlineimap will figure out the rest all by # itself, and fall back to password authentication if needed. # # 6. Using arbitrary python code. With this method, you invoke a @@ -981,7 +984,7 @@ remoteuser = username # This option stands in the [Repository RemoteExample] section. # -# In between synchronisations, OfflineIMAP can monitor mailboxes for new +# In between synchronisations, Offlineimap can monitor mailboxes for new # messages using the IDLE command. If you want to enable this, specify here the # folders you wish to monitor. IMAP protocol requires a separate connection for # each folder monitored in this way, so setting this option will force settings @@ -1005,7 +1008,7 @@ remoteuser = username # This option stands in the [Repository RemoteExample] section. # -# OfflineIMAP can use a compressed connection to the IMAP server. +# Offlineimap can use a compressed connection to the IMAP server. # This can result in faster downloads for some cases. # #usecompression = yes @@ -1013,7 +1016,7 @@ remoteuser = username # This option stands in the [Repository RemoteExample] section. # -# OfflineIMAP can use multiple connections to the server in order +# Offlineimap can use multiple connections to the server in order # to perform multiple synchronization actions simultaneously. # This may place a higher burden on the server. In most cases, # setting this value to 2 or 3 will speed up the sync, but in some @@ -1039,7 +1042,7 @@ remoteuser = username # This option stands in the [Repository RemoteExample] section. # -# OfflineIMAP normally closes IMAP server connections between refreshes if +# Offlineimap normally closes IMAP server connections between refreshes if # the global option autorefresh is specified. If you wish it to keep the # connection open, set this to true. If not specified, the default is # false. Keeping the connection open means a faster sync start the @@ -1061,8 +1064,8 @@ remoteuser = username # This option stands in the [Repository RemoteExample] section. # -# Normally, OfflineIMAP will expunge deleted messages from the server. You can -# disable that if you wish. This means that OfflineIMAP will mark them deleted +# Normally, Offlineimap will expunge deleted messages from the server. You can +# disable that if you wish. This means that Offlineimap will mark them deleted # on the server, but not actually delete them. You must use some other IMAP # client to delete them if you use this setting; otherwise, the messages will # just pile up there forever. Therefore, this setting is definitely NOT @@ -1248,6 +1251,8 @@ remoteuser = username # # http://mail.google.com/support/bin/answer.py?answer=78799&topic=12814 # +# This means ssl is enabled and must be configured correcly. +# # To enable GMail labels synchronisation, set the option "synclabels" in the # corresponding "Account" section. # diff --git a/offlineimap/imapserver.py b/offlineimap/imapserver.py index 3d349f4..483da54 100644 --- a/offlineimap/imapserver.py +++ b/offlineimap/imapserver.py @@ -415,7 +415,8 @@ class IMAPServer(object): "failed:\n\t%s"% msg, OfflineImapError.ERROR.REPO) if not tried_to_authn: - methods = ", ".join([x[5:] for x in [x for x in imapobj.capabilities if x[0:5] == "AUTH="]]) + methods = ", ".join([x[5:] for x in + [x for x in imapobj.capabilities if x[0:5] == "AUTH="]]) raise OfflineImapError(u"Repository %s: no supported " "authentication mechanisms found; configured %s, " "server advertises %s"% (self.repos,