Function assertIn causes the UnicodeDecodeError - django

During the test for my Django 1.9 project I get an error:
Python swears on this code:
def test_students_list(self):
# make request to the server to get homepage page
response = self.client.get(self.url)
# do we have student name on a page?
self.assertIn('Vitaliy', response.content)
How to set the same encoding for the arguments in function assertIn?
I tried so:
self.assertIn(u"Vitaliy", response.content.decode('utf8'))
The result is the same...
P.S. I have Python 2.7.6 on Ubuntu 14.04

Did you try to define your Python source code encoding using:
# -- coding: utf-8 --
As suggest in PEP 0263.

See https://docs.djangoproject.com/en/1.9/ref/request-response/#django.http.HttpResponse.content
The HTTPResponse.content is noted in the docs as being a bytestring (https://github.com/django/django/blob/master/django/http/response.py#L225), it should be encoded as DEFAULT_CHARSET which by default is utf-8 but in both our cases this doesn't seem to get through to the test.
My solution is to tell Python request.content should have unicode encoding:
def test_students_list(self):
# make request to the server to get homepage page
response = self.client.get(self.url)
# do we have student name on a page?
self.assertIn('Vitaliy', unicode(response.content, encoding='utf-8'))

Use self.assertContains(response, 'Vitaliy') instead.

Related

My telegram bot does not support Persian language

I built a telegram bot with Python-Telegram-bot, and I want to send a bot to a user in Persian when the user sends /Start ;but the bot does not work.
My Code:
from telegram.ext import Updater,CommandHandler
updater = Updater(token='TOKEN')
def start_method(bot,update):
bot.sendMessage(update.message.chat_id,"سلام")
start_command = CommandHandler('start', start_method)
updater.dispatcher.add_handler(start_command)
updater.start_polling()
If you want to use unicode text in your code, you have to specify the file encoding according to PEP 263.
Place this comment at the beginning of your script:
#!/usr/bin/python
# -*- coding: utf-8 -*-
You can also use Python 3, which has much better unicode support in general and assumes utf-8 encoding for source files by default.
First, need to use a urllib. If your text is something like txt1, you need to quote it first and then send it as a message. like this:
from urllib.parse import quote
......
txt1 = 'سلام. خوش آمدید!'
txt = quote(txt1.encode('utf-8'))

Basic authentication error with urllib2 since python 2.7 update

This piece of python code worked fine an hour ago, before I ran an apt-get upgrade on my raspberry pi.
This is now my python version: Python 2.7.9 (default, Sep 17 2016, 20:26:04)
import urllib, urllib2
from PIL import Image
URL="http://server.local/picture.jpg"
headers = {'Authorization': 'Basic ' + base64.encodestring('Guess:Thepassword')}
req = urllib2.Request(URL, None, headers)
img=Image.open(urllib2.urlopen(req,timeout=1))
But now I get this error:
File "/usr/lib/python2.7/httplib.py", line 1017, in putheader
raise ValueError('Invalid header value %r' % (one_value,))
ValueError: Invalid header value 'Basic TGlvbjpSdW5SYWJiaXRSdW4=\n'
I assume something has changed, but can't figure out what..
You can't have a new line character \n at the end of your header. Instead of using base64.encodestring, use base64.b64encode.
I don't think this has anything to do with an update to Python, since this behaviour has been there since the base64 module was included back in Python 2.4 (see the bolded text):
Encode the string s, which can contain arbitrary binary data, and
return a string containing one or more lines of base64-encoded data.
encodestring() returns a string containing one or more lines of
base64-encoded data always including an extra trailing newline ('\n').
FYI - I believe the change in functionality can be traced to this security update:
https://launchpad.net/ubuntu/+source/python2.7/2.7.6-8ubuntu0.3:
SECURITY UPDATE: CRLF injection vulnerability in the
HTTPConnection.putheader
- debian/patches/CVE-2016-5699.patch: disallow newlines in
putheader() arguments when not followed by spaces or tabs in
Lib/httplib.py, add tests in Lib/test/test_httplib.py
- CVE-2016-5699

Encoding is not kept

I am using Python 2.7 and using google plus public API to get activity data in a file. I am encountering issues to maintain the json encoding in my file. Double quotes are coming as u'' in my file. Below is my code:
from apiclient import discovery
API_KEY = 'MY API KEY'
service = discovery.build("plus", "v1", developerKey=API_KEY)
activities_resource = service.activities()
request = activities_resource.search(query='India versus South Africa', maxResults=1, orderBy='best',)
while request!= None:
activities_document = request.execute()
if 'items' in activities_document:
with open("output.json", mode='a') as file:
data = str(activities_document['items'])
file.write(data +"\n\n")
request = service.activities().list_next(request, activities_document)
Output:
[{u'kind': u'plus#activity', u'provider': {u'title': u'Google+'}, u'titl.......
I am expecting [{"kind": "plus#activity", .....
I am running my code on windows and I have tried both on DOS and pycharm IDE. I have also run the code on ubuntu machine but same output. Please let me know what I am doing wrong.
The json module is used for generating JSON. Use it.

Downloading a file in Python 2.x vs 3.x

I'm working on a script that checks multiple domain names if they are registered or not. It loops through a list of the domains read from a file, and the registration check is done using enom's API. My problem is the code accessing the API in Python 2:
import urllib2
import xml.etree.ElementTree as ET
...
response = urllib2.urlopen(url)
content = ET.fromstring(response.read())
...
return content.find("RRPCode").text
generates the error: 'NoneType' object has no attribute 'text' in nearly 30% of the checks. While the Python 3 code:
import urllib.request
import xml.etree.ElementTree as ET
...
response = urllib.request.urlopen(url)
content = ET.fromstring(response.read())
...
return content.find("RRPCode").text
works fine. I should also mention that the number of errors returned are random and not related to specific domain names.
What could be the cause of these errors?
I am by the way using Python 2.7.3 and Python 3.2.3 on a VPS running Ubuntu 12.04 server.
Thanks.

Python script using requests to post files to Django app - Nothing in request.FILES

Using Django 1.5 and Python 2.7.3.
Simple example to demonstrate my problem. I can't send files to our Django webapp using Python + requests. I'm using a script that resides on my computer and testing using Django's built in server (also running locally).
I've followed the example here on how to post files but requests.FILES is empty.
script.py
import requests
import os
url="http://<myurl>/upload"
files={'file':('myfile.txt', open('myfile.txt', 'rb'))}
r=requests.post(url, files = files)
r.text # Number of files in request.FILES: 0
views.py
def upload(request):
return HttpResponse("Number of files in request.FILES: {0}".format(len(request.FILES)))
EDIT
Headers: {'date': 'Tue, 11 Jun 2013 15:10:13 GMT', 'content-type': 'text/html; charset=utf-8', 'server': 'WSGIServer/0.1 Python/2.7.3'}
The URL is correct (I can see the response text in my script). The script works when I point the URL to http://httpbin.org/post as in the example.
Is there something in my requests.post that I'm missing?
In a moment of slight embarrassment, it turns out you really NEED the trailing forward-slash in the URL.
url="http://<myurl>/upload/"