Error while listing objects using boto3: botocore.parsers.ResponseParserError: - amazon-web-services

Question

Hi I am using boto3 for pulling data from s3.
result = s3.list_objects_v2(
Bucket = bucket,
Prefix ='1/abc/2/cde',
)
I am trying to list all the folder names after the "Prefix"
I am getting the following error
File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 357, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 648, in _make_api_call
operation_model, request_dict, request_context)
File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 667, in _make_request
return self._endpoint.make_request(operation_model, request_dict)
File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 102, in make_request
return self._send_request(request_dict, operation_model)
File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 135, in _send_request
request, operation_model, context)
File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 167, in _get_response
request, operation_model)
File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 218, in _do_get_response
response_dict, operation_model.output_shape)
File "/usr/local/lib/python3.7/site-packages/botocore/parsers.py", line 242, in parse
parsed = self._do_parse(response, shape)
File "/usr/local/lib/python3.7/site-packages/botocore/parsers.py", line 775, in _do_parse
self._parse_payload(response, shape, member_shapes, final_parsed)
File "/usr/local/lib/python3.7/site-packages/botocore/parsers.py", line 811, in _parse_payload
original_parsed = self._initial_body_parse(response['body'])
File "/usr/local/lib/python3.7/site-packages/botocore/parsers.py", line 897, in _initial_body_parse
return self._parse_xml_string_to_dom(xml_string)
File "/usr/local/lib/python3.7/site-packages/botocore/parsers.py", line 437, in _parse_xml_string_to_dom
"invalid XML received:\n%s" % (e, xml_string))
botocore.parsers.ResponseParserError: Unable to parse response (not well-formed (invalid token): line 1, column 0), invalid XML received:
b'{"bucket":"bucket","maxKeys":100,"versions":false,"prefix":"1\\/abc\\/2\\/cde","truncated":false,"results":[{"size":492574,"etag":"\\"BS6196892E3E0762DB660D395019F9E3\\"","lastModified":15634564256981,"version":"t4950050-afcf-11e9-bc48-e41d2d615800","key":"1\\/abc\\/2\\/cde\\/folder1\\/folder2\\/file.csv"}]}'
Is this because it is not able to parse "etag" ? Please help!
Thanks

Answer 1

For your case, the response type is JSON, where boto3 tries to parse it with XML format. You need to request explicitly to let list-objects reply with application/xml. One way that works for me is,
def add_xml_header(params,**kwargs):
params['headers']['Accept'] = 'application/xml'
client.meta.events.register('before-call.s3.ListObjects',add_xml_header)
Another common cause is,
From list_objects,
EncodingType (string) -- Requests Amazon S3 to encode the object keys in the response and specifies the encoding method to use. An object key may contain any Unicode character; however, XML 1.0 parser cannot parse some characters, such as characters with an ASCII value from 0 to 10. For characters that are not supported in XML 1.0, you can add this parameter to request that Amazon S3 encode the keys in the response.
In your case, your prefix contains ASCII value from 0 to 10.