Django test Client can only login once? - django

Our team is currently writing tests for our application. I am currently writing code to acces the views. These views are behind a login-screen, so our test first have to login and than peform the rest of the test. I've run into a very strange error. Basically My tests can only login once.
As you can see in the example below, both classes are doing the exact same thing, yet only one of them succeeds with the login, the other gives a '302 doest not equal 200' assertion error.
If I comment out the bottom one, the one at the top works, and vice versa.
Code that is testing different views also doesnt work, unless I comment out all other tests.
It doesnt matter if I login like shown below, or use a different variant (like self.client.login(username='test', password='password')).
Me and my team have no idea why Django is behaving this way and what we are doing wrong. Its almost as if the connection remains open and we would have to add code to close it. But the django-documentation doesnt mention any of this. DOes anyone know what we are doing wrong?
class FunctieListView_tests(TestCase):
"""Function listview only shows the data for the current_user / tenant"""
def setUp(self):
self.tenant = get_tenant()
self.function = get_function(self.tenant)
self.client = Client(HTTP_HOST='tc.tc:8000')
self.user = get_user(self.tenant)
def test_correct_function_context(self):
# Test if the view is only displaying the correct context data
self.client.post(settings.LOGIN_URL, {
'username': self.user.username,
'password': 'password'
}, HTTP_HOST='tc.tc:8000')
response = self.client.get(reverse('functie_list'))
self.assertEqual(response.status_code, 200)
self.assertTrue(response.context['functie_templates'] != None)
self.assertEqual(response.context['functie_templates'][0],
FunctieTemplate.objects.filter(linked_tenant=self.tenant)[0])
class FunctieListView_2_tests(TestCase):
"""Role Listview only shows the data for the current_user / tenant"""
def setUp(self):
self.tenant = get_tenant()
self.function = get_function(self.tenant)
self.client = Client(HTTP_HOST='tc.tc:8000')
self.user = get_user(self.tenant)
def test_correct_function_context_second(self):
#login
# Test if the view is only displaying the correct context data
self.client.post(settings.LOGIN_URL, {
'username': self.user.username,
'password': 'password'
}, HTTP_HOST='tc.tc:8000')
response = self.client.get(reverse('functie_list'))
self.assertEqual(response.status_code, 200)
self.assertTrue(response.context['functie_templates'] != None)
self.assertEqual(response.context['functie_templates'][0],
FunctieTemplate.objects.filter(linked_tenant=self.tenant)[0])
The users, tenants and functions are defined in a seperate utils file like so:
def get_user(tenant, name='test'):
u = User.objects.create_user(name, '{}#test.test'.format(name), 'password')
u.save()
u.profile.tenant = tenant
u.profile.tenant_role = generis.models.TENANT_OWNER
u.profile.save()
return u
def get_function(tenant):
userfunction = UserFunction.objects.create(name='test_functie', linked_tenant=tenant)
userfunction.save()
return userfunction
def get_tenant(slug_var='tc'):
f = elearning.models.FontStyle(font='foobar')
f.save()
c = elearning.models.ColorScheme(name='foobar', title='foo', text='fleeb', background='juice', block_background='schleem', box='plumbus')
c.save()
t = elearning.models.Tenant(name='tc', slug=slug_var, default_font_style=f, default_color_scheme=c)
t.save()
return t

My guess is that it happens because you are instantiating the Client yourself in setUp. Although it looks fine the outcome is obviously different from the regular behavior. I never had problems with login using the preinitialized self.client of django.test.TestCase.
Looking at django.test.client.Client, it says in the inline documentation:
Client objects are stateful - they will retain cookie (and thus session) details for the lifetime of the Client instance.
and a still existing cookie would explain the behavior you describe.
I cannot find HTTP_HOST in django.test.client.py, so I'm not sure whether you are really using that Client class at all. If you need access to a live server instance during tests, you could use Django's LiveServerTestCase.

Related

Django rest framework: unit testing database issue

I am doing unit testing of the rest Apis. I am using django rest framework.
Apis are saving data into and getting data from the database. Both of the operations are not working or if it is working i am not able to see that in the databases. Apis are also using django-fsm, because of which i need same data from the db for the other tests. Tests depends on previous tests due to django-fsm. There is always state changing with the api. But now i am not able to see any data in database during test runs. Don't know where it is saving the data or in which database.
Below is my test settings:-
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': join(PROJECT_ROOT, 'run', 'db_for_testing.sqlite3'),
'TEST': {
'NAME': 'test_db_for_testing',
},
},
}
below is my api:-
class DummyView(CreateAPIView):
def post(self, request, *args, **kwargs):
data = request.data.copy()
serializer = self.get_serializer(data=data)
serializer.is_valid(raise_exception=True)
order = self.model(book=serializer.data.get('book'))
order.save()
data = {
'status_code': 200,
'message': 'successfully.'
}
return Response(data, status=status.HTTP_200_OK)
As my tests depends on the previous test saving the data to db, so the other tests also fails. I am using APITestCase of rest_framework.
Help guys.
thanks in advance.
If I'm understanding your question correctly, Django "clear" database after each test (either rolling back or truncating.) So you need to write your tests accordingly.
See: https://docs.djangoproject.com/en/1.10/topics/testing/tools/#transactiontestcase
TL;DR - Solution: Use SimpleTestCase - See example below
Explanation
The thing is that the recommended test classes provided by Django for tests involving database queries, TransactionTestCase and the subclass TestCase, wraps every test in a transaction to speed up the process of resetting the database after each test. Source: Django TransactionTestCase docs
It is possible to avoid this behaviour by using SimpleTestCase which is the parent class of TransactionTestCase. You then have to specify explicitly that you want to allow database queries by setting allow_database_queries to True-
Also note that you are then responsible for any cleaning that needs to be done after the test. You can do that by overriding the tearDownClass class method. Similarly there's a setUpClass class method for any initialization prior to running the test. Remember to call the super methods. See full details in the docs
Example
from django.test import SimpleTestCase
class MyTestCase(SimpleTestCase):
allow_database_queries = True
#classmethod
def setUpClass(cls):
# Do your pre test initialization here.
super(MyTestCase, cls).setUpClass()
#classmethod
def tearDownClass(cls):
# Do your post test clean uphere.
super(MyTestCase, cls).tearDownClass()
def test_add_data_through_api(self):
# Add 'dat data
...
def test_work_with_data_from_previous_test(self):
# Work 'dat data
...
Use the --keepdb option when calling manage.py test:
https://docs.djangoproject.com/en/2.1/ref/django-admin/#cmdoption-test-keepdb
It's available since django 1.8.
Hope this helps.
Here my three tests are dependent with previous one. If I run them as separate test the previous test data deleted and later one failed for lost of that data.
So I make them in two different functions both are not test function. Finally call the dependent functions from one test function.
class test_profile(APITestCase):
def setUp(self):
super_user = default_service.create_super_user()
self.application = default_service.create_oath2_application(super_user.id)
self.content_type = "application/x-www-form-urlencoded"
self.username = "user#domain.com"
self.password = "pass123"
def create_profile(self):
url = reverse('EmailSignUp')
body = {
"email": self.username,
"password": self.password,
"fullname": "Mamun Hasan"
}
response = self.client.post(url, body, CONTENT_TYPE=self.content_type)
self.assertEqual(response.status_code, status.HTTP_200_OK)
data = (json.loads(response.content))['data']
# print("Profile", data)
self.assertEqual(data['username'], self.username)
self.assertEqual(data['fullname'], data['fullname'])
def get_access_token(self):
url = reverse('oauth2_provider:token')
body = {
"username": self.username,
"password": self.password,
"grant_type": self.application.authorization_grant_type,
"client_id": self.application.client_id,
"client_secret": self.application.client_secret,
}
response = self.client.post(url, body, CONTENT_TYPE=self.content_type)
self.assertEqual(response.status_code, status.HTTP_200_OK)
data = (json.loads(response.content))
# print("AccessToken", data)
self.assertEqual(data['token_type'], 'Bearer')
return data
def get_profile(self, oath2_token):
url = reverse('GetProfile')
authorization = oath2_token["token_type"] + ' ' + oath2_token["access_token"]
response = self.client.get(url, CONTENT_TYPE=self.content_type, HTTP_AUTHORIZATION=authorization)
self.assertEqual(response.status_code, status.HTTP_200_OK)
data = (json.loads(response.content))['data']
# print("Profile", data)
self.assertEqual(data['username'], self.username)
def test_dependent(self):
self.create_profile()
oath2_token = self.get_access_token()
self.get_profile(oath2_token)
I did not find any solution to commit the previous API data. If anyone knows please comment. So I have done it this way. I don't know this is the best solution but it works and tested.

model_mommy - user to assignment relationship

I'm finally setting up testing for my Django app, but I'm having difficulties getting started. I'm using model_mommy to create dynamic data for my tests, but have the following problem:
The view I'm testing is supposed to show me all the assignments a particular user has to complete. To test this, I want to create 500 assignments, log into the app and check if they are shown. So far I have the following test cases:
class TestLogin(TestCase):
def setUp(self):
self.client = Client()
user = User.objects.create(username='sam')
user.set_password('samspassword')
user.save()
def test_login(self):
self.client.login(username='sam', password='samspassword')
response = self.client.get('/')
print (response.content)
self.assertEqual(response.status_code, 200)
and
class TestShowAssignments(TestCase):
def setUp(self):
user_recipe = Recipe(User, username='sam', password="samspassword")
self.assignment = Recipe(Assignment,
coders = related(user_recipe))
self.assignments = self.assignment.make(_quantity=500)
def test_assignments(self):
self.assertIsInstance(self.assignments[0],Assignment)
self.assertEqual(len(self.assignments),500)
The first test passes fine and does what it should: TestLogin logs the user in and shows his account page.
The trouble starts with TestShowAssignments, which creates 500 assignments but if I look at the assignments with print (self.assignments[0].coders), I get auth.User.None. So it doesn't add the user I defined as a relation to the assignments. What might be important here is that the coders field in the model is a m2m field, which I tried to address by using related, but that doesn't seem to work.
What also doesn't work is logging in: if I use the same code I use for logging in during TestLogin in TestShowAssignments, I can't log in and see the user page.
So, my question: How do I use model_mommy to create Assignments and add them to a specific user, so that I can log in as that user and see if the assignments are displayed properly?
Do you want 500 Assignments that all have User "sam" as a single entry in the 'coders' field? If so, try:
from model_mommy import mommy
...
class TestShowAssignments(TestCase):
def setUp(self):
self.user = mommy.make(User, username='sam', password='samspassword')
self.assignments = mommy.make(Assigment, coders=[self.user], _quantity=500)

Only lists and tuples may be used in a list field Validation Error

Hi I am implementing test cases for my models.
I am using Mongoengine0.9.0 + Django 1.8
My models.py
class Project(Document):
# commented waiting for org-group to get finalize
project_name = StringField()
org_group = ListField(ReferenceField(OrganizationGroup, required=False))
My Serializers.py
class ProjectSerializer(DocumentSerializer):
class Meta:
model = Project
depth = 1
test.py file
def setUp(self):
# Every test needs access to the request factory.
self.factory = RequestFactory()
self.user = User.objects.create_user(
username='jacob', email='jacob#jacob.com', password='top_secret')
def test_post_put_project(self):
"""
Ensure we can create new clients in mongo database.
"""
org_group = str((test_utility.create_organization_group(self)).id)
url = '/project-management/project/'
data = {
"project_name": "googer",
"org_group": [org_group],
}
##import pdb; pdb.set_trace()
factory = APIRequestFactory()
user = User.objects.get(username='jacob')
view = views.ProjectList.as_view()
# Make an authenticated request to the view...
request = factory.post(url, data=data,)
force_authenticate(request, user=user)
response = view(request)
self.assertEqual(response.status_code, 200)
When I am running test cases I am getting this error
(Only lists and tuples may be used in a list field: ['org_group'])
The complete Stack Trace is
ValidationError: Got a ValidationError when calling Project.objects.create().
This may be because request data satisfies serializer validations but not Mongoengine`s.
You may need to check consistency between Project and ProjectSerializer.
If that is not the case, please open a ticket regarding this issue on https://github.com/umutbozkurt/django-rest-framework-mongoengine/issues
Original exception was: ValidationError (Project:None) (Only lists and tuples may be used in a list field: ['org_group'])
Not getting why we cant pass object like this.
Same thing when I am posting as an request to same method It is working for me but test cases it is failing
The tests should be running using multipart/form-data, which means that they don't support lists or nested data.
You can override this with the format argument, which I'm guessing you probably want to set to json. Most likely your front-end is using JSON, or a parser which supports lists, which explains why you are not seeing this.

Django Modular Testing

I have an "ok" test suite now, but I'm wanting to improve it. What happens is that I'm having to repeat setting up (limiting models for an example) users, property, school, and city objects.
Here is an example of something I have now, which works (note: could be broken because of changes made to simplify the example, but the logic is what I'm after):
class MainTestSetup(TestCase):
def setUp(self):
self.manage_text = 'Manage'
User = get_user_model()
# set up all types of users to be used
self.staff_user = User.objects.create_user('staff_user', 'staff#gmail.com', 'testpassword')
self.staff_user.is_staff = True
self.staff_user.save()
self.user = User.objects.create_user('user', 'user#gmail.com', 'testpassword')
self.city = City.objects.create(name="Test Town", state="TX")
self.school = School.objects.create(city=self.city, name="RE Test University",
long=-97.1234123, lat=45.7801234)
self.company = Company.objects.create(name="Test Company", default_school=self.school)
def login(self):
self.client.login(username=self.user.username,
password='testpassword')
def login_admin(self):
self.client.login(username=self.staff_user, password="testpassword")
class MainViewTests(MainTestSetup):
def test_home(self):
url = reverse('home-list')
manage_url = reverse('manage-property')
anon_response = self.client.get(url)
self.assertEqual(anon_response.status_code, 200)
self.assertNotContains(anon_response, self.manage_text)
self.login_admin()
admin_response = self.client.get(url)
self.assertContains(admin_response, self.manage_text)
def test_search(self):
url = reverse('search')
response = self.client.get(url)
self.assertEqual(response.status_code, 200)
...more tests
As you can see the MainViewTest inherits the setUp and login functions from the MainTestSetup class. This works ok, but I have many apps and not all need to set up all models. What I've tried to do is set up a set of mixins to include things like User, School, Company only in the TestSetups that I need.
This MainTestSetup would turn into something like:
class SchoolMixin(object):
def setUp(self):
self.city = City.objects.create(name="Test Town", state="TX")
self.school = School.objects.create(city=self.city, name="RE Test University",
long=-97.1234123, lat=45.7801234)
class CompanyMixin(SchoolMixin):
def setUp(self):
self.company = Company.objects.create(name="Test Company", default_school=self.school)
class UserMixin(object):
def setUp(self):
User = get_user_model()
# set up all types of users to be used
self.staff_user = User.objects.create_user('staff_user', 'staff#gmail.com', 'testpassword')
self.staff_user.is_staff = True
self.staff_user.save()
self.user = User.objects.create_user('user', 'user#gmail.com', 'testpassword')
def login(self):
self.client.login(username=self.user.username,
password='testpassword')
def login_admin(self):
self.client.login(username=self.staff_user, password="testpassword")
class MainTestSetup(UserMixin, CompanyMixin, TestCase):
def setUp(self):
self.manage_text = 'Manage'
This would allow a lot more flexibility for my test suite - this is only a small example. It would allow me in other apps to only include the Mixins that are necessary. For example if company was not needed, I would include just the SchoolMixin from the above example.
I believe my problem here is with inhertance of the setUp function. I'm not sure how to inherit correctly (through super, or though something else?). I've tried using super but haven't been able to get it to work. I have to admit, I'm not that great with classes/mixins yet, so any help/pointers would be much appreciated.
You can simplify and reduce the amount of code you have by using 2 libraries: WebTest and FactoryBoy. You won't need these Mixins.
https://pypi.python.org/pypi/django-webtest
https://github.com/rbarrois/factory_boy
Do the change step by step:
1. Starts with WebTest so you can get rid of your login_ method (and you won't need to prepare the passwords as well). With WebTest, you can specify the logged-in user when you load a page. For instance you will replace:
self.login_admin()
admin_response = self.client.get(url)
with:
admin_response = = self.app.get(url, user=self.admin)
2. Then use factory_boy to create all the objects you need. For instance you will replace:
self.staff_user = User.objects.create_user('staff_user', 'staff#gmail.com', 'testpassword')
self.staff_user.is_staff = True
self.staff_user.save()
with:
self.staff_user = StaffFactory.create()
3. Mix it up. Get rid of self.admin. Replace it with:
admin = AdminFactory.create()
response = = self.app.get(url, user=admin)
Once you've done all that, your code is going to be a lot shorter and easier to read. You won't need these mixins at all. For example your SchoolMixin can be replaced like this:
self.city = City.objects.create(name="Test Town", state="TX")
self.school = School.objects.create(city=self.city, name="RE Test University",
long=-97.1234123, lat=45.7801234)
replaced with:
school = SchoolFactory.create()
That's because factories can automatically create related entities with "SubFactories".
Here is a complete really simple example of a test using factories: http://codeku.co/testing-in-django-1

Convert POST to PUT with Tastypie

Full Disclosure: Cross posted to Tastypie Google Group
I have a situation where I have limited control over what is being sent to my api. Essentially there are two webservices that I need to be able to accept POST data from. Both use plain POST actions with urlencoded data (basic form submission essentially).
Thinking about it in "curl" terms it's like:
curl --data "id=1&foo=2" http://path/to/api
My problem is that I can't update records using POST. So I need to adjust the model resource (I believe) such that if an ID is specified, the POST acts as a PUT instead of a POST.
api.py
class urlencodeSerializer(Serializer):
formats = ['json', 'jsonp', 'xml', 'yaml', 'html', 'plist', 'urlencoded']
content_types = {
'json': 'application/json',
'jsonp': 'text/javascript',
'xml': 'application/xml',
'yaml': 'text/yaml',
'html': 'text/html',
'plist': 'application/x-plist',
'urlencoded': 'application/x-www-form-urlencoded',
}
# cheating
def to_urlencoded(self,content):
pass
# this comes from an old patch on github, it was never implemented
def from_urlencoded(self, data,options=None):
""" handles basic formencoded url posts """
qs = dict((k, v if len(v)>1 else v[0] )
for k, v in urlparse.parse_qs(data).iteritems())
return qs
class FooResource(ModelResource):
class Meta:
queryset = Foo.objects.all() # "id" = models.AutoField(primary_key=True)
resource_name = 'foo'
authorization = Authorization() # only temporary, I know.
serializer = urlencodeSerializer()
urls.py
foo_resource = FooResource
...
url(r'^api/',include(foo_resource.urls)),
)
In #tastypie on Freenode, Ghost[], suggested that I overwrite post_list() by creating a function in the model resource like so, however, I have not been successful in using this as yet.
def post_list(self, request, **kwargs):
if request.POST.get('id'):
return self.put_detail(request,**kwargs)
else:
return super(YourResource, self).post_list(request,**kwargs)
Unfortunately this method isn't working for me. I'm hoping the larger community could provide some guidance or a solution for this problem.
Note: I cannot overwrite the headers that come from the client (as per: http://django-tastypie.readthedocs.org/en/latest/resources.html#using-put-delete-patch-in-unsupported-places)
I had a similar problem on user creation where I wasn't able to check if the record already existed. I ended up creating a custom validation method which validated if the user didn't exist in which case post would work fine. If the user did exist I updated the record from the validation method. The api still returns a 400 response but the record is updated. It feels a bit hacky but...
from tastypie.validation import Validation
class MyValidation(Validation):
def is_valid(self, bundle, request=None):
errors = {}
#if this dict is empty validation passes.
my_foo = foo.objects.filter(id=1)
if not len(my_foo) == 0: #if object exists
foo[0].foo = 'bar' #so existing object updated
errors['status'] = 'object updated' #this will be returned in the api response
return errors
#so errors is empty if object does not exist and validation passes. Otherwise object
#updated and response notifies you of this
class FooResource(ModelResource):
class Meta:
queryset = Foo.objects.all() # "id" = models.AutoField(primary_key=True)
validation = MyValidation()
With Cathal's recommendation I was able to utilize a validation function to update the records I needed. While this does not return a valid code... it works.
from tastypie.validation import Validation
import string # wrapping in int() doesn't work
class Validator(Validation):
def __init__(self,**kwargs):
pass
def is_valid(self,bundle,request=None):
if string.atoi(bundle.data['id']) in Foo.objects.values_list('id',flat=True):
# ... update code here
else:
return {}
Make sure you specify the validation = Validator() in the ModelResource meta.