When file_use_mail_timestamp or utime_from_header are enabled,
OfflineIMAP tries to parse the Date header in the email. If the header
is present but invalid -- it doesn't contain a valid date -- this will
cause email.message to raise an exception. This is all fine. However
when handling that exception, OfflineIMAP can't try to extract the date
again: it's clearly invalid, and raising the same exception a second
time while handling the first exception just causes the entire sync to
fail.
To avoid that happening, don't try to provide the invalid date string in
the error message. Instead, just give the user the UID of the email
that triggered the exception, and the exception text.
Ideally we'd instead fix the code to actually extract the header value
and provide it in the error message, but Python's email.message module
doesn't provide an easy way to get the raw text of the Date header from
an EmailMessage object; it's possible using private variables like
EmailMessage._headers, or by parsing the email using a custom
email.policy.EmailPolicy object that disables the module's attempts to
coerce the header value to a DateTime. However, a user should be able
to get the problematic Date header from the message directly anyway, so
it's not worth adding all that complexity for something that should be
rare and provides little value.
Fixes#134
Signed-off-by: Adam Dinwoodie <adam@dinwoodie.org>
This patch checks the exception raises by os.rename()
on Windows and provide the same behavior than Linux.
This patch is related to issue #37, issue 5.
This patch sets closes the issue 37.
closes#37
Moving the quoted boundary fix to the Base class so that it can be used
by any subclass that needs to read an email. Adding another utility to
extract message-id from a raw email.
unreachable due to an optimization in PR#56. Since message-id is more
useful to better pin point the correct message, removing dbg_output.
Also fixing https://github.com/OfflineIMAP/offlineimap3/issues/62 by
correcting broken multipart boundaries or raising an error if as_bytes()
fails. Related python bug submitted: https://bugs.python.org/issue43818
although this workaround should be sufficent in the interim.
Signed-off-by: Joseph Ishac <jishac@nasa.gov>
If synclabels is enabled then offlineimap is sending '1:*' to imaplib2,
and imaplib2 while creating the FETCH command is quoting the sequence
and the command becomes:
b"JFFJ10 FETCH '1:*' (FLAGS X-GM-LABELS UID)\r\n"
Remove the single-quotes to prevent that and also consider the response
as bytes.
Closes: #52
Signed-off-by: Sudip Mukherjee <sudipm.mukherjee@gmail.com>
a message from a string to an email object that is part of the built-in
email library. The allows for emails to be processed as bytes and
re-encoded properly if they are not UTF-8 or ascii encoded. Currently
these changes cover the Base, IMAP, and Maildir classes but not the
specialized GMAIL class yet.
This patch converts the search results from bytes to strings
I add a bit comment about it here:
In Py2, with IMAP, imaplib2 returned a list of one element string.
['1, 2, 3, ...'] -> in Py3 is [b'1 2 3,...']
In Py2, with Davmail, imaplib2 returned a list of strings.
['1', '2', '3', ...] -> in Py3 should be [b'1', b'2', b'3',...]
In my tests with Py3, I get a list with one element: [b'1 2 3 ...']
Then I convert the values to string and I get ['1 2 3 ...']
With Davmail, it should be [b'1', b'2', b'3',...]
When I convert the values to string, I get ['1', '2', '3',...]
imaplib2 is doing this code for strings:
if isinstance(message, str):
message = bytes(message, 'ASCII')
But our message is already encoded using 'utf-8'.
Then, we can set the message as bytes, encoded using 'utf-8'
in offlineimap and imaplib2 won't change our message.
This patch solves this problem:
WARNING:OfflineImap:
Traceback:
File "/home/kix/src/offlineimap3/offlineimap/folder/Base.py", line 1127, in syncmessagesto
action(dstfolder, statusfolder)
File "/home/kix/src/offlineimap3/offlineimap/folder/Base.py", line 955, in __syncmessagesto_copy
self.copymessageto(uid, dstfolder, statusfolder, register=0)
File "/home/kix/src/offlineimap3/offlineimap/folder/Base.py", line 855, in copymessageto
new_uid = dstfolder.savemessage(uid, message, flags, rtime)
File "/home/kix/src/offlineimap3/offlineimap/folder/IMAP.py", line 668, in savemessage
(typ, dat) = imapobj.append(self.getfullIMAPname(),
File "/usr/lib/python3/dist-packages/imaplib2.py", line 660, in append
message = bytes(message, 'ASCII')
Emails received may not be UTF-8. Following error was observed on a specific
mail:
Traceback (most recent call last):
File "/home/tdescham/repo/offlineimap3/offlineimap/threadutil.py", line 146, in run
Thread.run(self)
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/home/tdescham/repo/offlineimap3/offlineimap/folder/Base.py", line 850, in copymessageto
message = self.getmessage(uid)
File "/home/tdescham/repo/offlineimap3/offlineimap/folder/IMAP.py", line 327, in getmessage
data = self._fetch_from_imap(str(uid), self.retrycount)
File "/home/tdescham/repo/offlineimap3/offlineimap/folder/IMAP.py", line 844, in _fetch_from_imap
ndata1 = data[0][1].decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa0 in position 10177: invalid start byte
This completely aborted offlineimap3, blocking further mail reception.
Instead, use the 'replace' error strategy in Python:
Replace with a suitable replacement character; Python will use the
official U+FFFD REPLACEMENT CHARACTER for the built-in Unicode codecs on
decoding and ‘?’ on encoding.
https://docs.python.org/2/library/codecs.html#codec-base-classes
ERROR: ERROR in syncfolder for gmail folder INBOX: Traceback (most recent call last):
File ".../offlineimap3/offlineimap/accounts.py", line 634, in syncfolder
cachemessagelists_upto_date(maxage)
File ".../offlineimap3/offlineimap/accounts.py", line 526, in cachemessagelists_upto_date
min_date=time.gmtime(time.mktime(date) + 24 * 60 * 60))
File ".../offlineimap3/offlineimap/folder/IMAP.py", line 277, in cachemessagelist
imapobj, min_date=min_date, min_uid=min_uid)
File ".../offlineimap3/offlineimap/folder/IMAP.py", line 259, in _msgs_to_fetch
search_result = search(search_cond)
File ".../offlineimap3/offlineimap/folder/IMAP.py", line 222, in search
if ' ' in res_data[0] or res_data[0] == '':
TypeError: a bytes-like object is required, not 'str'
Following error is seen when parsing server responses for sent mail:
2020-10-12 08:19:11 WARNING: Can't parse FETCH response, we awaited string: b' UID 26855)'
2020-10-12 08:19:11 WARNING: savemessage: Searching mails for new Message-ID failed. Could not determine new UID on Sent.
The comparison with 'type("")' means comparing with 'string' type in Python
3, but the left-hand side is a bytes object.
In case a tuple was received (first case in the code), the input is already
decoded from bytes to strings, but in case a single input was received it
was not.
Note that the comparison with 'type("")' is a bit odd, a more logical way
seems to be:
if isinstance(item, bytes)
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
This patch allows using folders with atom-specials like
"(", ")", spaces,...
We need quotes the folder name if it includes this special
characters.
Closes#4
This patch remves the set() mutable argument as default value,
sets the default value to None and check if the argument is none
in the code to call set().
