I've encountered some strange behavior in a Django unittest. Specifically, I'm using the APIClient module from rest_framework.test to simulate GET/POST requests from a unittest.
The issue occurs when updating/creating a new object in the Django ORM via a POST request (see the code below):
def test_something(self):
data = {
"name": 'unit testing',
"data": {}
}
response = self.api_client.post(reverse('save_model'), data=data, format='json')
self.assertEqual(response.status_code, 200)
#api_view(['GET', 'POST'])
def save_model(request):
obj, created = MyModel.objects.update_or_create(
user_id=request.user,
**request.data
)
return JsonResponse({
'id': obj.id,
'name': obj.name,
'user_id': obj.user_id.id
})
The error i receive when running the test case:
Error binding parameter 1 - probably unsupported type
Based on other stack posts involving this error, i would assume i have a type issue for the second parameter (the data field). However when the same exact data is used to store an object in Django shell, it works every time. Additionally, when the request is made from the client (with the same data) the request succeeds every time.
If i print the data in the unittest request i get the following:
(, u'{}')
(, u'unit testing')
Model code is below:
class MyModel(models.Model):
id = models.AutoField(primary_key=True)
name = models.CharField(max_length=255)
user_id = models.ForeignKey(AUTH_USER_MODEL)
data = JSONField()
So i thought this might be a unicode issue. But once again, storing the object with unicode data in the shell works just fine. One subtle difference to note, the django unittest will create a new test db for the models, whereas running in the shell does not.
Im out of answers, so if someone can shed some light on what's occuring here, that would be amazing.
Related
Our team is currently writing tests for our application. I am currently writing code to acces the views. These views are behind a login-screen, so our test first have to login and than peform the rest of the test. I've run into a very strange error. Basically My tests can only login once.
As you can see in the example below, both classes are doing the exact same thing, yet only one of them succeeds with the login, the other gives a '302 doest not equal 200' assertion error.
If I comment out the bottom one, the one at the top works, and vice versa.
Code that is testing different views also doesnt work, unless I comment out all other tests.
It doesnt matter if I login like shown below, or use a different variant (like self.client.login(username='test', password='password')).
Me and my team have no idea why Django is behaving this way and what we are doing wrong. Its almost as if the connection remains open and we would have to add code to close it. But the django-documentation doesnt mention any of this. DOes anyone know what we are doing wrong?
class FunctieListView_tests(TestCase):
"""Function listview only shows the data for the current_user / tenant"""
def setUp(self):
self.tenant = get_tenant()
self.function = get_function(self.tenant)
self.client = Client(HTTP_HOST='tc.tc:8000')
self.user = get_user(self.tenant)
def test_correct_function_context(self):
# Test if the view is only displaying the correct context data
self.client.post(settings.LOGIN_URL, {
'username': self.user.username,
'password': 'password'
}, HTTP_HOST='tc.tc:8000')
response = self.client.get(reverse('functie_list'))
self.assertEqual(response.status_code, 200)
self.assertTrue(response.context['functie_templates'] != None)
self.assertEqual(response.context['functie_templates'][0],
FunctieTemplate.objects.filter(linked_tenant=self.tenant)[0])
class FunctieListView_2_tests(TestCase):
"""Role Listview only shows the data for the current_user / tenant"""
def setUp(self):
self.tenant = get_tenant()
self.function = get_function(self.tenant)
self.client = Client(HTTP_HOST='tc.tc:8000')
self.user = get_user(self.tenant)
def test_correct_function_context_second(self):
#login
# Test if the view is only displaying the correct context data
self.client.post(settings.LOGIN_URL, {
'username': self.user.username,
'password': 'password'
}, HTTP_HOST='tc.tc:8000')
response = self.client.get(reverse('functie_list'))
self.assertEqual(response.status_code, 200)
self.assertTrue(response.context['functie_templates'] != None)
self.assertEqual(response.context['functie_templates'][0],
FunctieTemplate.objects.filter(linked_tenant=self.tenant)[0])
The users, tenants and functions are defined in a seperate utils file like so:
def get_user(tenant, name='test'):
u = User.objects.create_user(name, '{}#test.test'.format(name), 'password')
u.save()
u.profile.tenant = tenant
u.profile.tenant_role = generis.models.TENANT_OWNER
u.profile.save()
return u
def get_function(tenant):
userfunction = UserFunction.objects.create(name='test_functie', linked_tenant=tenant)
userfunction.save()
return userfunction
def get_tenant(slug_var='tc'):
f = elearning.models.FontStyle(font='foobar')
f.save()
c = elearning.models.ColorScheme(name='foobar', title='foo', text='fleeb', background='juice', block_background='schleem', box='plumbus')
c.save()
t = elearning.models.Tenant(name='tc', slug=slug_var, default_font_style=f, default_color_scheme=c)
t.save()
return t
My guess is that it happens because you are instantiating the Client yourself in setUp. Although it looks fine the outcome is obviously different from the regular behavior. I never had problems with login using the preinitialized self.client of django.test.TestCase.
Looking at django.test.client.Client, it says in the inline documentation:
Client objects are stateful - they will retain cookie (and thus session) details for the lifetime of the Client instance.
and a still existing cookie would explain the behavior you describe.
I cannot find HTTP_HOST in django.test.client.py, so I'm not sure whether you are really using that Client class at all. If you need access to a live server instance during tests, you could use Django's LiveServerTestCase.
I am doing unit testing of the rest Apis. I am using django rest framework.
Apis are saving data into and getting data from the database. Both of the operations are not working or if it is working i am not able to see that in the databases. Apis are also using django-fsm, because of which i need same data from the db for the other tests. Tests depends on previous tests due to django-fsm. There is always state changing with the api. But now i am not able to see any data in database during test runs. Don't know where it is saving the data or in which database.
Below is my test settings:-
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': join(PROJECT_ROOT, 'run', 'db_for_testing.sqlite3'),
'TEST': {
'NAME': 'test_db_for_testing',
},
},
}
below is my api:-
class DummyView(CreateAPIView):
def post(self, request, *args, **kwargs):
data = request.data.copy()
serializer = self.get_serializer(data=data)
serializer.is_valid(raise_exception=True)
order = self.model(book=serializer.data.get('book'))
order.save()
data = {
'status_code': 200,
'message': 'successfully.'
}
return Response(data, status=status.HTTP_200_OK)
As my tests depends on the previous test saving the data to db, so the other tests also fails. I am using APITestCase of rest_framework.
Help guys.
thanks in advance.
If I'm understanding your question correctly, Django "clear" database after each test (either rolling back or truncating.) So you need to write your tests accordingly.
See: https://docs.djangoproject.com/en/1.10/topics/testing/tools/#transactiontestcase
TL;DR - Solution: Use SimpleTestCase - See example below
Explanation
The thing is that the recommended test classes provided by Django for tests involving database queries, TransactionTestCase and the subclass TestCase, wraps every test in a transaction to speed up the process of resetting the database after each test. Source: Django TransactionTestCase docs
It is possible to avoid this behaviour by using SimpleTestCase which is the parent class of TransactionTestCase. You then have to specify explicitly that you want to allow database queries by setting allow_database_queries to True-
Also note that you are then responsible for any cleaning that needs to be done after the test. You can do that by overriding the tearDownClass class method. Similarly there's a setUpClass class method for any initialization prior to running the test. Remember to call the super methods. See full details in the docs
Example
from django.test import SimpleTestCase
class MyTestCase(SimpleTestCase):
allow_database_queries = True
#classmethod
def setUpClass(cls):
# Do your pre test initialization here.
super(MyTestCase, cls).setUpClass()
#classmethod
def tearDownClass(cls):
# Do your post test clean uphere.
super(MyTestCase, cls).tearDownClass()
def test_add_data_through_api(self):
# Add 'dat data
...
def test_work_with_data_from_previous_test(self):
# Work 'dat data
...
Use the --keepdb option when calling manage.py test:
https://docs.djangoproject.com/en/2.1/ref/django-admin/#cmdoption-test-keepdb
It's available since django 1.8.
Hope this helps.
Here my three tests are dependent with previous one. If I run them as separate test the previous test data deleted and later one failed for lost of that data.
So I make them in two different functions both are not test function. Finally call the dependent functions from one test function.
class test_profile(APITestCase):
def setUp(self):
super_user = default_service.create_super_user()
self.application = default_service.create_oath2_application(super_user.id)
self.content_type = "application/x-www-form-urlencoded"
self.username = "user#domain.com"
self.password = "pass123"
def create_profile(self):
url = reverse('EmailSignUp')
body = {
"email": self.username,
"password": self.password,
"fullname": "Mamun Hasan"
}
response = self.client.post(url, body, CONTENT_TYPE=self.content_type)
self.assertEqual(response.status_code, status.HTTP_200_OK)
data = (json.loads(response.content))['data']
# print("Profile", data)
self.assertEqual(data['username'], self.username)
self.assertEqual(data['fullname'], data['fullname'])
def get_access_token(self):
url = reverse('oauth2_provider:token')
body = {
"username": self.username,
"password": self.password,
"grant_type": self.application.authorization_grant_type,
"client_id": self.application.client_id,
"client_secret": self.application.client_secret,
}
response = self.client.post(url, body, CONTENT_TYPE=self.content_type)
self.assertEqual(response.status_code, status.HTTP_200_OK)
data = (json.loads(response.content))
# print("AccessToken", data)
self.assertEqual(data['token_type'], 'Bearer')
return data
def get_profile(self, oath2_token):
url = reverse('GetProfile')
authorization = oath2_token["token_type"] + ' ' + oath2_token["access_token"]
response = self.client.get(url, CONTENT_TYPE=self.content_type, HTTP_AUTHORIZATION=authorization)
self.assertEqual(response.status_code, status.HTTP_200_OK)
data = (json.loads(response.content))['data']
# print("Profile", data)
self.assertEqual(data['username'], self.username)
def test_dependent(self):
self.create_profile()
oath2_token = self.get_access_token()
self.get_profile(oath2_token)
I did not find any solution to commit the previous API data. If anyone knows please comment. So I have done it this way. I don't know this is the best solution but it works and tested.
Hi I am implementing test cases for my models.
I am using Mongoengine0.9.0 + Django 1.8
My models.py
class Project(Document):
# commented waiting for org-group to get finalize
project_name = StringField()
org_group = ListField(ReferenceField(OrganizationGroup, required=False))
My Serializers.py
class ProjectSerializer(DocumentSerializer):
class Meta:
model = Project
depth = 1
test.py file
def setUp(self):
# Every test needs access to the request factory.
self.factory = RequestFactory()
self.user = User.objects.create_user(
username='jacob', email='jacob#jacob.com', password='top_secret')
def test_post_put_project(self):
"""
Ensure we can create new clients in mongo database.
"""
org_group = str((test_utility.create_organization_group(self)).id)
url = '/project-management/project/'
data = {
"project_name": "googer",
"org_group": [org_group],
}
##import pdb; pdb.set_trace()
factory = APIRequestFactory()
user = User.objects.get(username='jacob')
view = views.ProjectList.as_view()
# Make an authenticated request to the view...
request = factory.post(url, data=data,)
force_authenticate(request, user=user)
response = view(request)
self.assertEqual(response.status_code, 200)
When I am running test cases I am getting this error
(Only lists and tuples may be used in a list field: ['org_group'])
The complete Stack Trace is
ValidationError: Got a ValidationError when calling Project.objects.create().
This may be because request data satisfies serializer validations but not Mongoengine`s.
You may need to check consistency between Project and ProjectSerializer.
If that is not the case, please open a ticket regarding this issue on https://github.com/umutbozkurt/django-rest-framework-mongoengine/issues
Original exception was: ValidationError (Project:None) (Only lists and tuples may be used in a list field: ['org_group'])
Not getting why we cant pass object like this.
Same thing when I am posting as an request to same method It is working for me but test cases it is failing
The tests should be running using multipart/form-data, which means that they don't support lists or nested data.
You can override this with the format argument, which I'm guessing you probably want to set to json. Most likely your front-end is using JSON, or a parser which supports lists, which explains why you are not seeing this.
I have a small web app with AngularJS front-end and Django ReST in the back. There's a strange hitch going on when I make POST request to the web service: the browser console clearly shows 3 parameters being sent, but the backend logging reports only 2 params received. The result is that the server throws a code 500 error due to a bad database lookup.
Here's the code:
Client
var b = newQuesForm.username.value;
$http.post('/myapp/questions/new', {username:b,title:q.title,description:q.description}).
success(function(data, status, headers, config) {
$http.get('/myapp/questions').success(function(data){
$scope.questions = data;
q = null;
$scope.newQuesForm.$setPristine();
}).error(function(data, status, headers, config) {
console.log(headers+data);
});
}).
error(function(data, status, headers, config) {
console.log(headers+data);
});
Both my manual logging and the dev console show a string like:
{"username":"admin","description":"What's your name?","title":"question 1"}
Server
class CreateQuestionSerializer(serializers.Serializer):
author = UserSerializer(required=False)
title = serializers.CharField(max_length=150)
description = serializers.CharField(max_length=350)
def create(self, data):
q= Question()
d = data
q.title = d.get('title')
q.description = d.get("description")
q.author = User.objects.get(username=d.get('username'))
q.save()
return q
Server-side logging shows the username parameter never succeeds in making the trip, and thus I end up with code 500 and error message:
User matching query does not exist. (No user with id=none)
What's causing some of the data to get lost?
So it turns out the problem was really with the serialization of fields, as #nikhiln began to point out. I followed his lead to refactor the code, moving the create() method to api.py, rather than serializers.py, and stopped relying altogether on the client-side data for the user's identity, something that was a bit silly in the first place (passing User to a hidden input in the view, and then harvesting the username from there and passing it back to the server in the AJAX params). Here's the new code, that works perfectly:
class QuestionCreate(generics.CreateAPIView):
model = Question
serializer_class = CreateQuestionSerializer
def create(self, request,*args,**kwargs):
q= Question()
d = request.data
q.title = d.get('title')
q.description = d.get("description")
q.author = request.user
q.save()
if q.pk:
return Response({'id':q.pk,'author':q.author.username}, status=status.HTTP_201_CREATED)
return Response({'error':'record not created'}, status=status.HTTP_400_BAD_REQUEST)
So here, I do it the right way: pull the User from the request param directly in the backend.
Full Disclosure: Cross posted to Tastypie Google Group
I have a situation where I have limited control over what is being sent to my api. Essentially there are two webservices that I need to be able to accept POST data from. Both use plain POST actions with urlencoded data (basic form submission essentially).
Thinking about it in "curl" terms it's like:
curl --data "id=1&foo=2" http://path/to/api
My problem is that I can't update records using POST. So I need to adjust the model resource (I believe) such that if an ID is specified, the POST acts as a PUT instead of a POST.
api.py
class urlencodeSerializer(Serializer):
formats = ['json', 'jsonp', 'xml', 'yaml', 'html', 'plist', 'urlencoded']
content_types = {
'json': 'application/json',
'jsonp': 'text/javascript',
'xml': 'application/xml',
'yaml': 'text/yaml',
'html': 'text/html',
'plist': 'application/x-plist',
'urlencoded': 'application/x-www-form-urlencoded',
}
# cheating
def to_urlencoded(self,content):
pass
# this comes from an old patch on github, it was never implemented
def from_urlencoded(self, data,options=None):
""" handles basic formencoded url posts """
qs = dict((k, v if len(v)>1 else v[0] )
for k, v in urlparse.parse_qs(data).iteritems())
return qs
class FooResource(ModelResource):
class Meta:
queryset = Foo.objects.all() # "id" = models.AutoField(primary_key=True)
resource_name = 'foo'
authorization = Authorization() # only temporary, I know.
serializer = urlencodeSerializer()
urls.py
foo_resource = FooResource
...
url(r'^api/',include(foo_resource.urls)),
)
In #tastypie on Freenode, Ghost[], suggested that I overwrite post_list() by creating a function in the model resource like so, however, I have not been successful in using this as yet.
def post_list(self, request, **kwargs):
if request.POST.get('id'):
return self.put_detail(request,**kwargs)
else:
return super(YourResource, self).post_list(request,**kwargs)
Unfortunately this method isn't working for me. I'm hoping the larger community could provide some guidance or a solution for this problem.
Note: I cannot overwrite the headers that come from the client (as per: http://django-tastypie.readthedocs.org/en/latest/resources.html#using-put-delete-patch-in-unsupported-places)
I had a similar problem on user creation where I wasn't able to check if the record already existed. I ended up creating a custom validation method which validated if the user didn't exist in which case post would work fine. If the user did exist I updated the record from the validation method. The api still returns a 400 response but the record is updated. It feels a bit hacky but...
from tastypie.validation import Validation
class MyValidation(Validation):
def is_valid(self, bundle, request=None):
errors = {}
#if this dict is empty validation passes.
my_foo = foo.objects.filter(id=1)
if not len(my_foo) == 0: #if object exists
foo[0].foo = 'bar' #so existing object updated
errors['status'] = 'object updated' #this will be returned in the api response
return errors
#so errors is empty if object does not exist and validation passes. Otherwise object
#updated and response notifies you of this
class FooResource(ModelResource):
class Meta:
queryset = Foo.objects.all() # "id" = models.AutoField(primary_key=True)
validation = MyValidation()
With Cathal's recommendation I was able to utilize a validation function to update the records I needed. While this does not return a valid code... it works.
from tastypie.validation import Validation
import string # wrapping in int() doesn't work
class Validator(Validation):
def __init__(self,**kwargs):
pass
def is_valid(self,bundle,request=None):
if string.atoi(bundle.data['id']) in Foo.objects.values_list('id',flat=True):
# ... update code here
else:
return {}
Make sure you specify the validation = Validator() in the ModelResource meta.