couchbase python sdk ascii exception - python-2.7

Question

First of all, this is the exception
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\scrapy\middleware.py", line 62, in _process_chain
return process_chain(self.methods[methodname], obj, *args)
File "C:\Python27\lib\site-packages\scrapy\utils\defer.py", line 65, in process_chain
d.callback(input)
File "C:\Python27\lib\site-packages\twisted\internet\defer.py", line 383, in callback
self._startRunCallbacks(result)
File "C:\Python27\lib\site-packages\twisted\internet\defer.py", line 491, in _startRunCallbacks
self._runCallbacks()
--- <exception caught here> ---
File "C:\Python27\lib\site-packages\twisted\internet\defer.py", line 578, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "D:\ScrapyProjects\General_Spider_code_version_4\General_Spider_code_version_4\pipelines.py", line 14, in process_item
connection.set(fileName, dict(item)) #write the item to the couchbase database
File "C:\Python27\lib\site-packages\couchbase-1.2.5-py2.7-win-amd64.egg\couchbase\connection.py", line 331, in set
persist_to, replicate_to)
File "C:\Python27\lib\site-packages\couchbase-1.2.5-py2.7-win-amd64.egg\couchbase\_bootstrap.py", line 99, in _json_encode_wrapper
return json.dumps(*args, ensure_ascii=False, separators=(',', ':'))
File "C:\Python27\lib\json\__init__.py", line 250, in dumps
sort_keys=sort_keys, **kw).encode(obj)
File "C:\Python27\lib\json\encoder.py", line 210, in encode
return ''.join(chunks)
couchbase.exceptions.ValueFormatError: <Couldn't encode value, inner_cause='ascii' codec can't decode byte 0xe2 in position 5: ordinal not in range(128), C Source=(src\convert.c,131), OBJ={'bathrooms': 1.0, 'furnished': 'No', 'ad_title': 'Large Studio For Rent in IMPZ just 45K/4chqs(KK)', 'agent_fees': -1, 'size': 550.0, 'category': 'Apartment', 'company_rera_number': '12913', 'agent_company': 'AL ANAS REAL ESTATE BROKER', 'ded_licence_number': '700590', 'source': 'dubizzleproperty', 'location': 'UAE \xe2\x80\xaa>\xe2\x80\xaa Dubai \xe2\x80\xaa>\xe2\x80\xaa IMPZ International Media Production Zone ; 3.1 km from Meadows Town Centre \xc2\xa0', 'image_links': [u'http://87421a79fde09fda7e57-79445249ccb41a60f7b99c8ef6df8604.r12.cf3.rackcdn.com/4_async/2015/2/18/73ff34e2a38c7b104401c9e5c54b03628971053f/main.jpeg', u'http://87421a79fde09fda7e57-79445249ccb41a60f7b99c8ef6df8604.r12.cf3.rackcdn.com/4_async/2015/2/18/24ec831f6b4afb47fecc1c3e0991cf3090c90c24/main.jpeg', u'http://87421a79fde09fda7e57-79445249ccb41a60f7b99c8ef6df8604.r12.cf3.rackcdn.com/4_async/2015/2/18/77fee11394090aaea2d668cfe2754b92d6e36264/main.jpeg', u'http://87421a79fde09fda7e57-79445249ccb41a60f7b99c8ef6df8604.r12.cf3.rackcdn.com/4_async/2015/2/18/5d4113319ccbabcdd65b0ffe7302da59b374b5fe/main.jpeg', u'http://87421a79fde09fda7e57-79445249ccb41a60f7b99c8ef6df8604.r12.cf3.rackcdn.com/4_async/2015/2/18/8070689f309759d5860e97aa35d3f0eac425dc1d/main.jpeg', u'http://87421a79fde09fda7e57-79445249ccb41a60f7b99c8ef6df8604.r12.cf3.rackcdn.com/4_async/2015/2/18/8e86702847e69d485d147629dd2e48e1ad831e63/main.jpeg'], 'latitude': -1, 'description': 'Central A/C & Heating , Balcony , Shared Pool , Built in Wardrobes , Walk-in Closet , Shared Gym , Security , Built in Kitchen Appliances', 'bedrooms': 'Studio', 'rent_is_paid': 'Quarterly', 'action': 'Rent', 'link': 'http://dubai.dubizzle.com/property-for-rent/residential/apartmentflat/2015/2/18/large-studio-for-rent-in-impz-just-45k4chq-2/?back=ZHViYWkuZHViaXp6bGUuY29tL3Byb3BlcnR5LWZvci1yZW50L3Jlc2lkZW50aWFsL2FwYXJ0bWVudGZsYXQv&pos=1', 'longitude': -1, 'property_reference': '', 'yearly_cost': 45000.0, 'agent_mobile': -1, 'posting_date': '2015-02-19'}>
I am trying to store a dictionary on a couchbase. i am using this code
connection.set(fileName, dict(item))
to transfer the item to dictionary. as you see from the error message. i have a unicode values, which is according to python sdk couchbase is fine, could you help me please?

Answer 1

Your values are not unicode. Keep in mind that a str object containing valid unicode escape sequences does not automatically make it "Unicode" in Python parlance. You need to ensure the strings are properly Unicode.
This seems to work with the normal json.dumps() function (without any arguments); whereas the python client passes (by default) the ensure_ascii=False parameter to decrease the data size (JSON itself can be in UTF-8 encoding, and is not limited to ASCII).
Thus, a workaround may be to set your own encoding function for JSON which does not pass the ensure_ascii parameter; like so:
import json
import couchbase
couchbase.set_json_converters(json.dumps, json.loads)
Though this workaround is not recommended as it may inflate your document size slightly.