am using python and google app engine majorly on jinja2 templates
i could like when a user registers a new account, they get a popup indicating that their registration is successful of even any alert on the very interface before moving to the next registration step.
def post(self):
user = (str(users.get_current_user().email()))
userquery = Users.query(Users.email == user)
count = userquery.count()
if count == 0:
#test if user is admin or employee
qry = Users.query()
count = qry.count()
if count == 0:
privilage = 'admin'
db_put = Users(
f_name=self.request.get("f_name"),
l_name = self.request.get("l_name"),
org = self.request.get("org"),
email=users.get_current_user().email(),
privilage = privilage
)
db_put.put()
How are you calling this POST method? Are you sending a form there directly (use method 1) or is this being done with an AJAX call (use method 2)?
Method 1
You can redirect to a GET page where you render a template with a success or error message for Jinja to use. This would however involve a page change.
import webapp2
class MyHandler(webapp2.RequestHandler):
def get(self): # Let's assume /someurl is mapped to this handler.
template_values = {}
notification = self.request.get('notification')
if notification:
template_values['notification'] = notification
self.response.set_status(200)
self.response.headers['Content-Type'] = 'text/html; charset=utf-8'
# Need to get the template from jinja and set it as template variable.
self.response.out.write(template.render(template_values))
def post(self):
# Do all your stuff here.
self.redirect('/someurl?notification=Success')
Alternatively you can set the parameters directly on the request instead of passing them as URI parameters:
def post(self):
# Do all your stuff here.
self.redirect('/someurl, params={'notification': 'Success'})
Method 2
In this method you can send back a JSON response with a success or error message. The caller (whatever function in your javascript that submitted the request to the backend) can use that to render a butterbar message or other popup notification of your choosing:
import json
import webapp2
class MyHandler(webapp2.RequestHandler):
def post(self):
# Do all your stuff here.
self.response.set_status(200)
self.response.headers['Content-Type'] = 'application/json; charset=utf-8'
self.response.headers['Content-Disposition'] = 'attachment'
self.response.out.write(json.JsonEncoder(sort_keys=True).encode('Success'))
For the latter, make sure you think about cross-site scripting (XSS) vulnerabilities and perhaps add a JSON prefix.
So here's something I'm trying to figure out. I've got a method that is triggered by post_save
for this "Story" model. Works fine. What I need to do is figure out how to mock out the test, so I can fake the call and make assertions on my returns. I think I need to patch it somehow, but I've tried a couple different ways without much success. Best i can get is a object instance, but it ignores values I pass in.
I've commented in my test where my confusion lies. Any help would be welcome.
Here's my test:
from django.test import TestCase
from django.test.client import Client
from marketing.blog.models import Post, Tag
from unittest.mock import patch, Mock
class BlogTestCase(TestCase):
fixtures = [
'auth-test.json',
'blog-test.json',
]
def setUp(self):
self.client = Client()
def test_list(self):
# verify that we can load the list page
r = self.client.get('/blog/')
self.assertEqual(r.status_code, 200)
self.assertContains(r, "<h1>The Latest from Our Blog</h1>")
self.assertContains(r, 'Simple JavaScript Date Formatting')
self.assertContains(r, 'Page 1 of 2')
# loading a page out of range should redirect to last page
r = self.client.get('/blog/5/', follow=True)
self.assertEqual(r.redirect_chain, [
('http://testserver/blog/2/', 302)
])
self.assertContains(r, 'Page 2 of 2')
# verify that unpublished posts are not displayed
with patch('requests') as mock_requests:
# my futile attempt at mocking.
# creates <MagicMock> object but not able to call return_values
mock_requests.post.return_value = mock_response = Mock()
# this doesn't get to the magic mock object. Why?
mock_response.status_code = 201
p = Post.objects.get(id=5)
p.published = False
# post_save signal runs here and requests is called.
# Needs to be mocked.
p.save()
r = self.client.get('/blog/')
self.assertNotContains(r, 'Simple JavaScript Date Formatting')
Here's the model:
from django.db import models
from django.conf import settings
from django.db.models import signals
import requests
def update_console(sender, instance, raw, created, **kwargs):
# ignoring raw so that test fixture data can load without
# hitting this method.
if not raw:
update = instance
json_obj = {
'author': {
'alias': 'the_dude',
'token': 'the_dude'
},
'text': update.description,
}
headers = {'content-type': 'application/json'}
path = 'http://testserver.com:80/content/add/'
request = requests(path, 'POST',
json_obj, headers=headers,
)
if request.status_code < 299:
story_id = request.json().get('id')
if story_id:
# disconnect and reconnect signal so
# we don't enter recursion-land
signals.post_save.disconnect(
update_console,
sender = Story, )
update.story_id = story_id
update.save()
signals.post_save.connect(
update_console,
sender = Story, )
else:
raise AttributeError('Error Saving to console, '+ request.text)
class Story(models.Model):
"""Lets tell a story"""
story_id = models.CharField(
blank=True,
max_length=10,
help_text="This maps to the id of the post"
)
slug = models.SlugField(
unique=True,
help_text="This is used in URL and in code references.",
)
description = models.TextField(
help_text='2-3 short paragraphs about the story.',
)
def __str__(self):
return self.short_headline
# add/update this record as a custom update in console
signals.post_save.connect(update_console, sender = Story)
You need to patch requests in the module where it is actually used, i.e.
with patch('path.to.your.models.requests') as mock_requests:
mock_requests.return_value.status_code = 200
mock_requests.return_value.json.return_value = {'id': story_id'}
...
The documentation offers more detailed explanations on where to patch:
patch works by (temporarily) changing the object that a name points to with another one. There can be many names pointing to any individual object, so for patching to work you must ensure that you patch the name used by the system under test.
The basic principle is that you patch where an object is looked up, which is not necessarily the same place as where it is defined.
Here, you need to patch the name requests inside the models module, hence the need to provide its full path.
In my django app, I have a view which accomplishes file upload.The core snippet is like this
...
if (request.method == 'POST'):
if request.FILES.has_key('file'):
file = request.FILES['file']
with open(settings.destfolder+'/%s' % file.name, 'wb+') as dest:
for chunk in file.chunks():
dest.write(chunk)
I would like to unit test the view.I am planning to test the happy path as well as the fail path..ie,the case where the request.FILES has no key 'file' , case where request.FILES['file'] has None..
How do I set up the post data for the happy path?Can somebody tell me?
I used to do the same with open('some_file.txt') as fp: but then I needed images, videos and other real files in the repo and also I was testing a part of a Django core component that is well tested, so currently this is what I have been doing:
from django.core.files.uploadedfile import SimpleUploadedFile
def test_upload_video(self):
video = SimpleUploadedFile("file.mp4", "file_content", content_type="video/mp4")
self.client.post(reverse('app:some_view'), {'video': video})
# some important assertions ...
In Python 3.5+ you need to use bytes object instead of str. Change "file_content" to b"file_content"
It's been working fine, SimpleUploadedFile creates an InMemoryFile that behaves like a regular upload and you can pick the name, content and content type.
From Django docs on Client.post:
Submitting files is a special case. To POST a file, you need only
provide the file field name as a key, and a file handle to the file
you wish to upload as a value. For example:
c = Client()
with open('wishlist.doc') as fp:
c.post('/customers/wishes/', {'name': 'fred', 'attachment': fp})
I recommend you to take a look at Django RequestFactory. It's the best way to mock data provided in the request.
Said that, I found several flaws in your code.
"unit" testing means to test just one "unit" of functionality. So,
if you want to test that view you'd be testing the view, and the file
system, ergo, not really unit test. To make this point more clear. If
you run that test, and the view works fine, but you don't have
permissions to save that file, your test would fail because of that.
Other important thing is test speed. If you're doing something like
TDD the speed of execution of your tests is really important.
Accessing any I/O is not a good idea.
So, I recommend you to refactor your view to use a function like:
def upload_file_to_location(request, location=None): # Can use the default configured
And do some mocking on that. You can use Python Mock.
PS: You could also use Django Test Client But that would mean that you're adding another thing more to test, because that client make use of Sessions, middlewares, etc. Nothing similar to Unit Testing.
I do something like this for my own event related application but you should have more than enough code to get on with your own use case
import tempfile, csv, os
class UploadPaperTest(TestCase):
def generate_file(self):
try:
myfile = open('test.csv', 'wb')
wr = csv.writer(myfile)
wr.writerow(('Paper ID','Paper Title', 'Authors'))
wr.writerow(('1','Title1', 'Author1'))
wr.writerow(('2','Title2', 'Author2'))
wr.writerow(('3','Title3', 'Author3'))
finally:
myfile.close()
return myfile
def setUp(self):
self.user = create_fuser()
self.profile = ProfileFactory(user=self.user)
self.event = EventFactory()
self.client = Client()
self.module = ModuleFactory()
self.event_module = EventModule.objects.get_or_create(event=self.event,
module=self.module)[0]
add_to_admin(self.event, self.user)
def test_paper_upload(self):
response = self.client.login(username=self.user.email, password='foz')
self.assertTrue(response)
myfile = self.generate_file()
file_path = myfile.name
f = open(file_path, "r")
url = reverse('registration_upload_papers', args=[self.event.slug])
# post wrong data type
post_data = {'uploaded_file': i}
response = self.client.post(url, post_data)
self.assertContains(response, 'File type is not supported.')
post_data['uploaded_file'] = f
response = self.client.post(url, post_data)
import_file = SubmissionImportFile.objects.all()[0]
self.assertEqual(SubmissionImportFile.objects.all().count(), 1)
#self.assertEqual(import_file.uploaded_file.name, 'files/registration/{0}'.format(file_path))
os.remove(myfile.name)
file_path = import_file.uploaded_file.path
os.remove(file_path)
I did something like that :
from django.core.files.uploadedfile import SimpleUploadedFile
from django.test import TestCase
from django.core.urlresolvers import reverse
from django.core.files import File
from django.utils.six import BytesIO
from .forms import UploadImageForm
from PIL import Image
from io import StringIO
def create_image(storage, filename, size=(100, 100), image_mode='RGB', image_format='PNG'):
"""
Generate a test image, returning the filename that it was saved as.
If ``storage`` is ``None``, the BytesIO containing the image data
will be passed instead.
"""
data = BytesIO()
Image.new(image_mode, size).save(data, image_format)
data.seek(0)
if not storage:
return data
image_file = ContentFile(data.read())
return storage.save(filename, image_file)
class UploadImageTests(TestCase):
def setUp(self):
super(UploadImageTests, self).setUp()
def test_valid_form(self):
'''
valid post data should redirect
The expected behavior is to show the image
'''
url = reverse('image')
avatar = create_image(None, 'avatar.png')
avatar_file = SimpleUploadedFile('front.png', avatar.getvalue())
data = {'image': avatar_file}
response = self.client.post(url, data, follow=True)
image_src = response.context.get('image_src')
self.assertEquals(response.status_code, 200)
self.assertTrue(image_src)
self.assertTemplateUsed('content_upload/result_image.html')
create_image function will create image so you don't need to give static path of image.
Note : You can update code as per you code.
This code for Python 3.6.
from rest_framework.test import force_authenticate
from rest_framework.test import APIRequestFactory
factory = APIRequestFactory()
user = User.objects.get(username='#####')
view = <your_view_name>.as_view()
with open('<file_name>.pdf', 'rb') as fp:
request=factory.post('<url_path>',{'file_name':fp})
force_authenticate(request, user)
response = view(request)
As mentioned in Django's official documentation:
Submitting files is a special case. To POST a file, you need only provide the file field name as a key, and a file handle to the file you wish to upload as a value. For example:
c = Client()
with open('wishlist.doc') as fp:
c.post('/customers/wishes/', {'name': 'fred', 'attachment': fp})
More Information: How to check if the file is passed as an argument to some function?
While testing, sometimes we want to make sure that the file is passed as an argument to some function.
e.g.
...
class AnyView(CreateView):
...
def post(self, request, *args, **kwargs):
attachment = request.FILES['attachment']
# pass the file as an argument
my_function(attachment)
...
In tests, use Python's mock something like this:
# Mock 'my_function' and then check the following:
response = do_a_post_request()
self.assertEqual(mock_my_function.call_count, 1)
self.assertEqual(
mock_my_function.call_args,
call(response.wsgi_request.FILES['attachment']),
)
if you want to add other data with file upload then follow the below method
file = open('path/to/file.txt', 'r', encoding='utf-8')
data = {
'file_name_to_receive_on_backend': file,
'param1': 1,
'param2': 2,
.
.
}
response = self.client.post("/url/to/view", data, format='multipart')`
The only file_name_to_receive_on_backend will be received as a file other params received normally as post paramas.
In Django 1.7 there's an issue with the TestCase wich can be resolved by using open(filepath, 'rb') but when using the test client we have no control over it. I think it's probably best to ensure file.read() returns always bytes.
source: https://code.djangoproject.com/ticket/23912, by KevinEtienne
Without rb option, a TypeError is raised:
TypeError: sequence item 4: expected bytes, bytearray, or an object with the buffer interface, str found
from django.test import Client
from requests import Response
client = Client()
with open(template_path, 'rb') as f:
file = SimpleUploadedFile('Name of the django file', f.read())
response: Response = client.post(url, format='multipart', data={'file': file})
Hope this helps.
Very handy solution with mock
from django.test import TestCase, override_settings
#use your own client request factory
from my_framework.test import APIClient
from django.core.files import File
import tempfile
from pathlib import Path
import mock
image_mock = mock.MagicMock(spec=File)
image_mock.name = 'image.png' # or smt else
class MyTest(TestCase):
# I assume we want to put this file in storage
# so to avoid putting garbage in our MEDIA_ROOT
# we're using temporary storage for test purposes
#override_settings(MEDIA_ROOT=Path(tempfile.gettempdir()))
def test_send_file(self):
client = APIClient()
client.post(
'/endpoint/'
{'file':image_mock},
format="multipart"
)
I am using Python==3.8.2 , Django==3.0.4, djangorestframework==3.11.0
I tried self.client.post but got a Resolver404 exception.
Following worked for me:
import requests
upload_url='www.some.com/oaisjdoasjd' # your url to upload
with open('/home/xyz/video1.webm', 'rb') as video_file:
# if it was a text file we would perhaps do
# file = video_file.read()
response_upload = requests.put(
upload_url,
data=video_file,
headers={'content-type': 'video/webm'}
)
I am using django rest framework and I had to test the upload of multiple files.
I finally get it by using format="multipart" in my APIClient.post request.
from rest_framework.test import APIClient
...
self.client = APIClient()
with open('./photo.jpg', 'rb') as fp:
resp = self.client.post('/upload/',
{'images': [fp]},
format="multipart")
I am using GraphQL, upload for test:
with open('test.jpg', 'rb') as fp:
response = self.client.execute(query, variables, data={'image': [fp]})
code in class mutation
#classmethod
def mutate(cls, root, info, **kwargs):
if image := info.context.FILES.get("image", None):
kwargs["image"] = image
TestingMainModel.objects.get_or_create(
id=kwargs["id"],
defaults=kwargs
)
I'm trying to test my webapp2 handlers. To do this, I thought it would be a good idea to send a request to the handler e.g.:
request = webapp2.Request.blank('/')
# Get a response for that request.
response = request.get_response(main.app)
The problem is, response is mostly just a bunch of HTML etc.
I want to look at what was passed to my jinja2 template from the handler before it was turned into HTML.
I want my test to get at the state within the handler class code. I wan't to be able to see what certain variables looked like in the response handler, and then I want to see what the dict templates looks like before it was passed to render_to_response()
I want to test these variables have the correct values.
Here is my test code so far, but I'm stuck because response = request.get_response() just gives me a bunch of html and not the raw variables.
import unittest
import main
import webapp2
class DemoTestCase(unittest.TestCase):
def setUp(self):
pass
def tearDown(self):
pass
def testNothing(self):
self.assertEqual(42, 21 + 21)
def testHomeHandler(self):
# Build a request object passing the URI path to be tested.
# You can also pass headers, query arguments etc.
request = webapp2.Request.blank('/')
# Get a response for that request.
response = request.get_response(main.app)
# Let's check if the response is correct.
self.assertEqual(response.status_int, 200)
self.assertEqual(response.body, 'Hello, world!')
if __name__ == '__main__':
unittest.main()
and here is my handler:
class HomeHandler(BaseHandler):
def get(self, file_name_filter=None, category_filter=None):
file_names = os.listdir('blog_posts')
blogs = []
get_line = lambda file_: file_.readline().strip().replace("<!--","").replace("-->","")
for fn in file_names:
with open('blog_posts/%s' % fn) as file_:
heading = get_line(file_)
link_name = get_line(file_)
category = get_line(file_)
date_ = datetime.strptime(fn.split("_")[0], "%Y%m%d")
blog_dict = {'date': date_, 'heading': heading,
'link_name': link_name,
'category': category,
'filename': fn.replace(".html", ""),
'raw_file_name': fn}
blogs.append(blog_dict)
categories = Counter(d['category'] for d in blogs)
templates = {'categories': categories,
'blogs': blogs,
'file_name_filter': file_name_filter,
'category_filter': category_filter}
assert(len(file_names) == len(set(d['link_name'] for d in blogs)))
self.render_template('home.html', **templates)
and here is my basehandler:
class BaseHandler(webapp2.RequestHandler):
#webapp2.cached_property
def jinja2(self):
return jinja2.get_jinja2(app=self.app)
def render_template(self, filename, **kwargs):
#kwargs.update({})
#TODO() datastore caching here for caching of (handlername, handler parameters, changeable parameters, app_upload_date)
#TODO() write rendered page to its own html file, and just serve that whole file. (includes all posts). JQuery can show/hide posts.
self.response.write(self.jinja2.render_template(filename, **kwargs))
Perhaps I have got the wrong idea of how to do unit testing, or perhaps I should have written my code in a way that makes it easier to test? or is there some way of getting the state of my code?
Also if someone were to re-write the code and change the variable names, then the tests would break.
You can mock BaseHandler.render_template method and test its parameters.
See this question for a list of popular Python mocking frameworks.
Thanks to proppy's suggestion I ended up using a mock.
http://www.voidspace.org.uk/python/mock/
(mock is included as part or unittest.mock in python 3)
So here is my main.py code which is similar to what I have in webapp2:
note instead of BaseHandler.render_template i have BaseHandler.say_yo
__author__ = 'Robert'
print "hello from main"
class BaseHandler():
def say_yo(self,some_number=99):
print "yo"
return "sup"
class TheHandler(BaseHandler):
def get(self, my_number=42):
print "in TheHandler's get()"
print self.say_yo(my_number)
return "TheHandler's return string"
and atest.py:
__author__ = 'Robert'
import unittest
import main
from mock import patch
class DemoTestCase(unittest.TestCase):
def setUp(self):
pass
def tearDown(self):
pass
def testNothing(self):
self.assertEqual(42, 21 + 21)
def testSomeRequests(self):
print "hi"
bh = main.BaseHandler()
print bh.say_yo()
print "1111111"
with patch('main.BaseHandler.say_yo') as patched_bh:
print dir(patched_bh)
patched_bh.return_value = 'double_sup'
bh2 = main.BaseHandler()
print bh2.say_yo()
print "222222"
bh3 = main.BaseHandler()
print bh3.say_yo()
print "3333"
th = main.TheHandler()
print th.get()
print "44444"
with patch('main.BaseHandler.say_yo') as patched_bh:
patched_bh.return_value = 'last_sup'
th = main.TheHandler()
print th.get()
print th.get(123)
print "---"
print patched_bh.called
print patched_bh.call_args_list
print "555555"
if __name__ == '__main__':
unittest.main()
this code gives lots of output, here is a sample:
44444
in TheHandler's get()
last_sup
TheHandler's return string
in TheHandler's get()
last_sup
TheHandler's return string
---
True
[call(42), call(123)]
555555
The project I'm working on has some data that needs to get passed to every view, so we have a wrapper around render_to_response called master_rtr. Ok.
Now, I need our 404 pages to run through this as well. Per the instructions, I created a custom 404 handler (cleverly called custom_404) that calls master_rtr. Everything looks good, but our tests are failing, because we're receiving back a 200 OK.
So, I'm trying to figure out how to return a 404 status code, instead. There seems to be an HttpResponseNotFound class that's kinda what I want, but I'm not quite sure how to construct all of that nonsense instead of using render_to_response. Or rather, I could probably figure it out, but it seems like their must be an easier way; is there?
The appropriate parts of the code:
def master_rtr(request, template, data = {}):
if request.user.is_authenticated():
# Since we're only grabbing the enrollments to get at the courses,
# doing select_related() will save us from having to hit database for
# every course the user is enrolled in
data['courses'] = \
[e.course for e in \
Enrollment.objects.select_related().filter(user=request.user) \
if e.view]
else:
if "anonCourses" in request.session:
data['courses'] = request.session['anonCourses']
else:
data['courses'] = []
data['THEME'] = settings.THEME
return render_to_response(template, data, context_instance=RequestContext(request))
def custom_404(request):
response = master_rtr(request, '404.html')
response.status_code = 404
return response
The easy way:
def custom_404(request):
response = master_rtr(...)
response.status_code = 404
return response
But I have to ask: why aren't you just using a context processor along with a RequestContext to pass the data to the views?
Just set status_code on the response.
Into your application's views.py add:
# Imports
from django.shortcuts import render
from django.http import HttpResponse
from django.template import Context, loader
##
# Handle 404 Errors
# #param request WSGIRequest list with all HTTP Request
def error404(request):
# 1. Load models for this view
#from idgsupply.models import My404Method
# 2. Generate Content for this view
template = loader.get_template('404.htm')
context = Context({
'message': 'All: %s' % request,
})
# 3. Return Template for this view + Data
return HttpResponse(content=template.render(context), content_type='text/html; charset=utf-8', status=404)
The secret is in the last line: status=404
Hope it helped!