imaplib2 is doing this code for strings:
if isinstance(message, str):
message = bytes(message, 'ASCII')
But our message is already encoded using 'utf-8'.
Then, we can set the message as bytes, encoded using 'utf-8'
in offlineimap and imaplib2 won't change our message.
This patch solves this problem:
WARNING:OfflineImap:
Traceback:
File "/home/kix/src/offlineimap3/offlineimap/folder/Base.py", line 1127, in syncmessagesto
action(dstfolder, statusfolder)
File "/home/kix/src/offlineimap3/offlineimap/folder/Base.py", line 955, in __syncmessagesto_copy
self.copymessageto(uid, dstfolder, statusfolder, register=0)
File "/home/kix/src/offlineimap3/offlineimap/folder/Base.py", line 855, in copymessageto
new_uid = dstfolder.savemessage(uid, message, flags, rtime)
File "/home/kix/src/offlineimap3/offlineimap/folder/IMAP.py", line 668, in savemessage
(typ, dat) = imapobj.append(self.getfullIMAPname(),
File "/usr/lib/python3/dist-packages/imaplib2.py", line 660, in append
message = bytes(message, 'ASCII')
This patch changes the __DEF_OS_LOCATIONS to an iterable value.
The reason is because a list is an iterable, but an string is an
iterable too, and this is a mistake.
The function get_os_sslcertfile() has a loop to iterate the return of
get_os_sslcertfile_searchpath(), that returns the value in the
__DEF_OS_LOCATIONS dictionary. When the value is an iterable, the "f"
variable is set to the iterable value and works fine.
If the value of "f" is an string, the for-loop iterates over every
character, so the test for os.path.exists is always false (is comparing
the path with a character, not with the full path), so this function
fails and return None.
To check this change, edit your .offlineimaprc file and change the
sslcacertfile to OS-DEFAULT:
sslcacertfile = OS-DEFAULT
And run offlineimap. If you are not using 'darwin0 (the only iterable)
it will fails. Now, apply this patch, and run offlineimap again. Problem
is solved.
Signed-off-by: Rodolfo García Peñas (kix) <kix@kix.es>
The call from platform.linux_distribution was depdrecated in Python 3.7.
This patch solves this problem, using the recomended package distro.
Signed-off-by: Rodolfo García Peñas (kix) <kix@kix.es>
The file utils/distro.py is renamed to utils/distro_utils.py
because we need import the distro file to update some calls.
The file is used in IMAP.py, I need change the import.
Emails received may not be UTF-8. Following error was observed on a specific
mail:
Traceback (most recent call last):
File "/home/tdescham/repo/offlineimap3/offlineimap/threadutil.py", line 146, in run
Thread.run(self)
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/home/tdescham/repo/offlineimap3/offlineimap/folder/Base.py", line 850, in copymessageto
message = self.getmessage(uid)
File "/home/tdescham/repo/offlineimap3/offlineimap/folder/IMAP.py", line 327, in getmessage
data = self._fetch_from_imap(str(uid), self.retrycount)
File "/home/tdescham/repo/offlineimap3/offlineimap/folder/IMAP.py", line 844, in _fetch_from_imap
ndata1 = data[0][1].decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa0 in position 10177: invalid start byte
This completely aborted offlineimap3, blocking further mail reception.
Instead, use the 'replace' error strategy in Python:
Replace with a suitable replacement character; Python will use the
official U+FFFD REPLACEMENT CHARACTER for the built-in Unicode codecs on
decoding and ‘?’ on encoding.
https://docs.python.org/2/library/codecs.html#codec-base-classes
ERROR: ERROR in syncfolder for gmail folder INBOX: Traceback (most recent call last):
File ".../offlineimap3/offlineimap/accounts.py", line 634, in syncfolder
cachemessagelists_upto_date(maxage)
File ".../offlineimap3/offlineimap/accounts.py", line 526, in cachemessagelists_upto_date
min_date=time.gmtime(time.mktime(date) + 24 * 60 * 60))
File ".../offlineimap3/offlineimap/folder/IMAP.py", line 277, in cachemessagelist
imapobj, min_date=min_date, min_uid=min_uid)
File ".../offlineimap3/offlineimap/folder/IMAP.py", line 259, in _msgs_to_fetch
search_result = search(search_cond)
File ".../offlineimap3/offlineimap/folder/IMAP.py", line 222, in search
if ' ' in res_data[0] or res_data[0] == '':
TypeError: a bytes-like object is required, not 'str'
When a hook is configured, the output is a bytes object.
This is then printed on the console/logfile as:
2020-10-12 08:36:17 INFO: Hook stdout: b'Processed 3 total files in almost no time.\nAdded 3 new messages to the database.\n'
Hook stderr:b''
Decode the output so that it is printed nicely, as:
2020-10-12 08:36:17 INFO: Hook stdout: Processed 3 total files in almost no time.
Added 3 new messages to the database.
Hook stderr:
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Following error is seen when parsing server responses for sent mail:
2020-10-12 08:19:11 WARNING: Can't parse FETCH response, we awaited string: b' UID 26855)'
2020-10-12 08:19:11 WARNING: savemessage: Searching mails for new Message-ID failed. Could not determine new UID on Sent.
The comparison with 'type("")' means comparing with 'string' type in Python
3, but the left-hand side is a bytes object.
In case a tuple was received (first case in the code), the input is already
decoded from bytes to strings, but in case a single input was received it
was not.
Note that the comparison with 'type("")' is a bit odd, a more logical way
seems to be:
if isinstance(item, bytes)
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
This patch allows using folders with atom-specials like
"(", ")", spaces,...
We need quotes the folder name if it includes this special
characters.
Closes#4
This patch remves the set() mutable argument as default value,
sets the default value to None and check if the argument is none
in the code to call set().
Error when using the XOAUTH2 token refresh logic:
("POST data should be bytes, an iterable of bytes, or a file object. It cannot be of type str. (configuration is: {....}", <traceback object at 0x7fc6e69f2b40>)
[' File ".../offlineimap3/offlineimap/accounts.py", line 298, in syncrunner
self.__sync()
', ' File ".../offlineimap3/offlineimap/accounts.py", line 374, in __sync
remoterepos.getfolders()
', ' File ".../offlineimap3/offlineimap/repository/IMAP.py", line 446, in getfolders
imapobj = self.imapserver.acquireconnection()
', ' File ".../offlineimap3/offlineimap/imapserver.py", line 579, in acquireconnection
self.__authn_helper(imapobj)
', ' File ".../offlineimap3/offlineimap/imapserver.py", line 443, in __authn_helper
if func(imapobj):
', ' File ".../offlineimap3/offlineimap/imapserver.py", line 377, in __authn_xoauth2
imapobj.authenticate(\'XOAUTH2\', self.__xoauth2handler)
', ' File ".../offlineimap3/venv/lib/python3.7/site-packages/imaplib2.py", line 681, in authenticate
typ, dat = self._simple_command(\'AUTHENTICATE\', mechanism.upper())
', ' File ".../offlineimap3/venv/lib/python3.7/site-packages/imaplib2.py", line 1674, in _simple_command
return self._command_complete(self._command(name, *args), kw)
', ' File ".../offlineimap3/venv/lib/python3.7/site-packages/imaplib2.py", line 1394, in _command
literal = literator(data, rqb)
', ' File ".../offlineimap3/venv/lib/python3.7/site-packages/imaplib2.py", line 2237, in process
ret = self.mech(self.decode(data))
', ' File ".../offlineimap3/offlineimap/imapserver.py", line 253, in __xoauth2handler
raise type(e)(msg, exc_info()[2])
']
Fix by encoding the data passed to urllib.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
This patch removes the library six, compatible with python2.
I need change these re-raise calls.
Signed-off-by: Rodolfo García Peñas (kix) <kix@kix.es>
This patch removes the library six, compatible with python2.
I need change these re-raise calls.
Signed-off-by: Rodolfo García Peñas (kix) <kix@kix.es>
This patch removes the library six, compatible with python2.
I need change these re-raise calls.
Signed-off-by: Rodolfo García Peñas (kix) <kix@kix.es>
This patch removes the library six, compatible with python2.
I need change these re-raise calls.
Signed-off-by: Rodolfo García Peñas (kix) <kix@kix.es>
This patch removes the library six, compatible with python2.
I need change these re-raise calls.
Signed-off-by: Rodolfo García Peñas (kix) <kix@kix.es>
This patch removes the library six, compatible with python2.
I need change these re-raise calls.
Signed-off-by: Rodolfo García Peñas (kix) <kix@kix.es>
This patch removes the library six, compatible with python2.
I need change these re-raise calls.
Signed-off-by: Rodolfo García Peñas (kix) <kix@kix.es>
This patch removes the library six, compatible with python2.
I need change these re-raise calls.
Signed-off-by: Rodolfo García Peñas (kix) <kix@kix.es>
This patch changes the function __savemessage_fetchheaders to decode the
bytes retunred by imaplib2.
We need a list of headers, with string values, but imapli2 is providing
a list with bytes. This change convert the values to str.
Signed-off-by: Rodolfo García Peñas (kix) <kix@kix.es>
Now, the server response is in a list of strings. We need the second
string, so we need read the [1].
Previously, was a list of tuples, so, we used [0][1].
This patch is like the patch IMAP.py Get the server response right, but
now for Gmail.
Signed-off-by: Rodolfo García Peñas (kix) <kix@kix.es>
In the patch "IMAP list folders" we updated the function getselectors
to return '""'. We use this variable in the folders, but also here.
Because the variable is never '', we need update it with this change, to
test that the variable has value, but is not empty.
Signed-off-by: Rodolfo García Peñas (kix) <kix@kix.es>
matchinguids variable is a list of UIDs, separated by spaces. You can
check it some lines later, using the split command.
We need decode the bytes value returned by imaplib2 and convert it to
sting.
Signed-off-by: Rodolfo García Peñas (kix) <kix@kix.es>
This patch changes the refernce to "", because now the empty string
is not saved as '""', is just '', so the fecth command fails.
Then, I changed the fech call to "" "*", using refernce "" and pattern
"*".
