As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I'm NOT new to testing, but got really confused with the mess of recommendations for testing different layers in Django.
Some recommend (and they are right) to avoid Doctests in the model as they are not maintainable...
Others say don't use fixtures, as they are less flexible than helper functions, for instance..
There are also two groups of people who fight for using Mock objects. The first group believe in using Mock and isolating the rest of the system, while another group prefer to Stop Mocking and start testing..
All I have mentioned above, were mostly in regards to testing models. Functional testing is an another story (using test.Client() VS webTest VS etc. )
Is there ANY maintainable, extandible and proper way for testing different layers??
UPDATE
I am aware of Carl Meyer's talk at PyCon 2012..
UPDATE 08-07-2012
I can tell you my practices for unit testing that are working pretty well for my own ends and I'll give you my reasons:
1.- Use Fixtures only for information that is necessary for testing but is not going to change, for example, you need a user for every test you do so use a base fixture to create users.
2.- Use a factory to create your objects, I personally love FactoryBoy (this comes from FactoryGirl which is a ruby library). I create a separate file called factories.py for every app where I save all these objects. This way I keep off the test files all the objects I need which makes it a lot more readable and easy to maintain. The cool thing about this approach is that you create a base object that can be modified if you want to test something else based on some object from the factory. Also it doesn't depend on django so when I migrated these objects when I started using mongodb and needed to test them, everything was smooth. Now after reading about factories it's common to say "Why would I want to use fixtures then". Since these fixtures should never change all the extra goodies from factories are sort of useless and django supports fixtures very well out of the box.
3.- I Mock calls to external services, because these calls make my tests very slow and they depend on things that have nothing to do with my code being right or wrong. for example, if I tweet within my test, I do test it to tweet rightly, copy the response and mock that object so it returns that exact response every time without doing the actual call. Also sometimes is good to test when things go wrong and mocking is great for that.
4.- I use an integration server (jenkins is my recommendation here) which runs the tests every time I push to my staging server and if they fail it sends me an email. This is just great since it happens to me a lot that I break something else in my last change and I forgot to run the tests. It also gives you other goodies like a coverage report, pylint/jslint/pep8 verifications and there exists a lot of plugins where you can set different statistics.
About your question for testing front end, django comes with some helper functions to handle this in a basic way.
This is what I personally use, you can fire gets, posts, login the user, etc. that's enough for me. I don't tend to use a complete front end testing engine like selenium since I feel it's an overkill to test anything else besides the business layer. I am sure some will differ and it always depends on what you are working on.
Besides my opinion, django 1.4 comes with a very handy integration for in-browser frameworks.
I'll set an example app where I can apply this practices so it is more understandable. Let's create a very basic blog app:
structure
blogger/
__init__.py
models.py
fixtures/base.json
factories.py
tests.py
models.py
from django.db import models
class Blog(models.Model):
user = models.ForeignKey(User)
text = models.TextField()
created_on = models.DateTimeField(default=datetime.now())
fixtures/base.json
[
{
"pk": 1,
"model": "auth.user",
"fields": {
"username": "fragilistic_test",
"first_name": "demo",
"last_name": "user",
"is_active": true,
"is_superuser": true,
"is_staff": true,
"last_login": "2011-08-16 15:59:56",
"groups": [],
"user_permissions": [],
"password": "IAmCrypted!",
"email": "test#email.com",
"date_joined": "1923-08-16 13:26:03"
}
}
]
factories.py
import factory
from blog.models import User, Blog
class BlogFactory(factory.Factory):
FACTORY_FOR = Blog
user__id = 1
text = "My test text blog of fun"
tests.py
class BlogTest(TestCase):
fixtures = ['base'] # loads fixture
def setUp(self):
self.blog = BlogFactory()
self.blog2 = BlogFactory(text="Another test based on the last one")
def test_blog_text(self):
self.assertEqual(Blog.objects.filter(user__id=1).count(), 2)
def test_post_blog(self):
# Lets suppose we did some views
self.client.login(username='user', password='IAmCrypted!')
response = self.client.post('/blogs', {'text': "test text", user='1'})
self.assertEqual(response.status, 200)
self.assertEqual(Blog.objects.filter(text='test text').count(), 1)
def test_mocker(self):
# We will mock the datetime so the blog post was created on the date
# we want it to
mocker = Mock()
co = mocker.replace('datetime.datetime')
co.now()
mocker.result(datetime.datetime(2012, 6, 12))
with mocker:
res = Blog.objects.create(user__id=1, text='test')
self.assertEqual(res.created_on, datetime.datetime(2012, 6, 12))
def tearDown(self):
# Django takes care of this but to be strict I'll add it
Blog.objects.all().delete()
Notice I am using some specific technology for the sake of the example (which haven't been tested btw).
I have to insist, this may not be the standard best practice (which I doubt it exists) but it is working pretty well for me.
I really like the suggestions from #Hassek and want to stress out what an excellent point he makes about the obvious lack of standard practices, which holds true for many of Django's aspects, not just testing, since all of us approach the framework with different concerns in mind, also adding to that the great degree of flexibility we have with designing our applications, we often end up with drastically different solutions that are applicable to the same problem.
Having said that, though, most of us still strive for many of the same goals when testing our applications, mainly:
Keeping our test modules neatly organized
Creating reusable assertion and helper methods, helper functions that reduce the LOC for test methods, to make them more compact and readable
Showing that there is an obvious, systematic approach to how the application components are tested
Like #Hassek, these are my preferences that may directly conflict with the practices that you may be applying, but I feel it's nice to share the things we've proven that work, if only in our case.
No test case fixtures
Application fixtures work great, in cases you have certain constant model data you'd like to guarantee to be present in the database, say a collection of towns with their names and post office numbers.
However, I see this as an inflexible solution for providing test case data. Test fixtures are very verbose, model mutations force you to either go through a lengthy process of reproducing the fixture data or to perform tedious manual changes and maintaining referential integrity is difficult to manually perform.
Additionally, you'll most likely use many kinds of fixtures in your tests, not just for models: you'd like to store the response body from API requests, to create fixtures that target NoSQL database backends, to write have fixtures that are used to populate form data, etc.
In the end, utilizing APIs to create data is concise, readable and it makes it much easier to spot relations, so most of us resort to using factories for dynamically creating fixtures.
Make extensive use of factories
Factory functions and methods are preferable to stomping out your test data. You can create helper factory module-level functions or test case methods that you may want to either reuse
across application tests or throughout the whole project. Particularly, factory_boy, that #Hassek mentions, provides you with the ability to inherit/extend fixture data and do automatic sequencing, which might look a bit clumsy if you'd do it by hand otherwise.
The ultimate goal of utilizing factories is to cut down on code-duplication and streamline how you create test data. I cannot give you exact metrics, but I'm sure if you go through your test methods with a discerning eye you will notice that a large portion of your test code is mainly preparing the data that you'll need to drive your tests.
When this is done incorrectly, reading and maintaining tests becomes an exhausting activity. This tends to escalate when data mutations lead to not-so-obvious test failures across the board, at which point you'll not be able to apply systematic refactoring efforts.
My personal approach to this problem is to start with a myproject.factory module that creates easy-to-access references to QuerySet.create methods for my models and also for any objects I might regularly use in most of my application tests:
from django.contrib.auth.models import User, AnonymousUser
from django.test import RequestFactory
from myproject.cars.models import Manufacturer, Car
from myproject.stores.models import Store
create_user = User.objects.create_user
create_manufacturer = Manufacturer.objects.create
create_car = Car.objects.create
create_store = Store.objects.create
_factory = RequestFactory()
def get(path='/', data={}, user=AnonymousUser(), **extra):
request = _factory.get(path, data, **extra)
request.user = user
return request
def post(path='/', data={}, user=AnonymousUser(), **extra):
request = _factory.post(path, data, **extra)
request.user = user
return request
This in turn allows me to do something like this:
from myproject import factory as f # Terse alias
# A verbose, albeit readable approach to creating instances
manufacturer = f.create_manufacturer(name='Foomobiles')
car1 = f.create_car(manufacturer=manufacturer, name='Foo')
car2 = f.create_car(manufacturer=manufacturer, name='Bar')
# Reduce the crud for creating some common objects
manufacturer = f.create_manufacturer(name='Foomobiles')
data = {name: 'Foo', manufacturer: manufacturer.id)
request = f.post(data=data)
view = CarCreateView()
response = view.post(request)
Most people are rigorous about reducing code duplication, but I actually intentionally introduce some whenever I feel it contributes to test comprehensiveness. Again, the goal with whichever approach you take to factories is to minimize the amount of brainfuck you introduce into the header of each test method.
Use mocks, but use them wisely
I'm a fan of mock, as I've developed an appreciation for the author's solution to what I believe was the problem he wanted to address. The tools provided by the package allow you to form test assertions by injecting expected outcomes.
# Creating mocks to simplify tests
factory = RequestFactory()
request = factory.get()
request.user = Mock(is_authenticated=lamda: True) # A mock of an authenticated user
view = DispatchForAuthenticatedOnlyView().as_view()
response = view(request)
# Patching objects to return expected data
#patch.object(CurrencyApi, 'get_currency_list', return_value="{'foo': 1.00, 'bar': 15.00}")
def test_converts_between_two_currencies(self, currency_list_mock):
converter = Converter() # Uses CurrencyApi under the hood
result = converter.convert(from='bar', to='foo', ammount=45)
self.assertEqual(4, result)
As you can see, mocks are really helpful, but they have a nasty side effect: your mocks clearly show your making assumptions on how it is that your application behaves, which introduces coupling. If Converter is refactored to use something other than the CurrencyApi, someone may not obviously understand why the test method is suddenly failing.
So with great power comes great responsibility--if your going to be a smartass and use mocks to avoid deeply rooted test obstacles, you may completely obfuscate the true nature of your test failures.
Above all, be consistent. Very very consistent
This is the most important point to be made. Be consistent with absolutely everything:
how you organize code in each of your test modules
how you introduce test cases for your application components
how you introduce test methods for asserting the behavior of those components
how you structure test methods
how you approach testing common components (class-based views, models, forms, etc.)
how you apply reuse
For most projects, the bit about how your collaboratively going to approach testing is often overlooked. While the application code itself looks perfect--adhering to style guides, use of Python idioms, reapplying Django's own approach to solving related problems, textbook use of framework components, etc.--no one really makes it an effort to figure out how to turn test code into a valid, useful communication tool and it's a shame if, perhaps, having clear guidelines for test code is all it takes.
I have a number of simple controller classes that use Doctrine's entity manager to retrieve data and pass it to a view.
public function indexAction() {
$pages = $this->em->getRepository('Model_Page')->findAll();
$this->view->pages = $pages;
}
What exactly should we be testing here?
I could test routing on the action to ensure that's configured properly
I could potentially test that the appropriate view variables are being set, but this is cumbersome
The findAll() method should probably be in a repository layer which can be tested using mock data, but then this constitutes a different type of test and brings us back to the question of
What should we be testing as part of controller tests?
Controller holds core logic for your application. Though simple "index" controller actions don't have any specific functions, those that verify/actively use data and generate viewmodels have pretty much the most functionality of the system.
For example, consider login form. Whenever the login data is posted, controller should validate login/password and return: 1) to index page whenever logins are good. Show welcome,{user} text. 2) to the login page saying that login is not found in db. 3) to the login page saying that password is not found in db.
These three types of outputs make perfect test cases. You should validate that correct viewmodels/views are being sent back to the client with the appropriate actions.
You shouldn't look at a controller like at something mysterious. It's just another code piece, and it's tested as any other code - any complicated logic that gives business-value to the user should be tested.
Also, I'd recommend using acceptance testing with framework like Cucumber to generate meaningful test cases.
Probably the controller is the hardest thing to test, as it has many dependencies. In theory you should test it in full isolation, but as you already seen - it has no sense.
Probably you should start with functional or acceptance test. It tests your controller action in a whole. I agree with previous answer, that you should try acceptance testing tools. But Cucumber is for Ruby, for PHP you can try using Codeception. It makes tests simple and meaningful.
Also on a Codeception page there is an article on how to test sample controllers.
I'm trying to write unit tests for parts of my Node app. I'm using Mongoose for my ORM.
I've searched a bunch for how to do testing with Mongoose and Node but not come with anything. The solutions/frameworks all seem to be full-stack or make no mention of mocking stuff.
Is there a way I can mock my Mongoose DB so I can return static data in my tests? I'd rather not have to set up a test DB and fill it with data for every unit test.
Has anyone else encountered this?
I too went looking for answers, and ended up here. This is what I did:
I started off using mockery to mock out the module that my models were in. An then creating my own mock module with each model hanging off it as a property. These properties wrapped the real models (so that child properties exist for the code under test). And then I override the methods I want to manipulate for the test like save. This had the advantage of mockery being able to undo the mocking.
but...
I don't really care enough about undoing the mocking to write wrapper properties for every model. So now I just require my module and override the functions I want to manipulate. I will probably run tests in separate processes if it becomes an issue.
In the arrange part of my tests:
// mock out database saves
var db = require("../../schema");
db.Model1.prototype.save = function(callback) {
console.log("in the mock");
callback();
};
db.Model2.prototype.save = function(callback) {
console.log("in the mock");
callback("mock staged an error for testing purposes");
};
I solved this by structuring my code a little. I'm keeping all my mongoose-related stuff in separate classes with APIs like "save", "find", "delete" and no other class does direct access to the database. Then I simply mock those in tests that rely on data.
I did something similar with the actual objects that are returned. For every model I have in mongoose, I have a corresponding class that wraps it and provides access-methods to fields. Those are also easily mocked.
Also worth mentioning:
mockgoose - In-memory DB that mocks Mongoose, for testing purposes.
monckoose - Similar, but takes a different approach (Implements a fake driver). Monckoose seems to be unpublished as of March 2015.
I am fairly new to unit testing. I am building an ASP.NET MVC3 application (although my question seems language agnostic) and am confused about a basic test.
I want to make a unit test that makes sure my "ValidatePassword" function works - It will take in a username and password, then hash the password and see if it matches the hash for a user in the database. If so, it returns true. The problem is that I am using a mock repository, so I will have to add the user to the db before running my test. I can't really create this user in my test setup because I don't know what the encrypted password will be until I actually run it through the function I am testing. Is the answer to run it through the Hash function once, write it down in my test, and then test with that?
Hope this is clear. Thanks!
I prefer to set up my test data where possible through the public interface of my code, rather than giving the test code knowledge of how the code is implemented. So personally I would not use a hardcoded encrypted password in the test code. Let me explain...
Presumably, you have a method to add a new user, which internally will create an new entry in the database with a hashed password. Then the test would look something like this:
AddNewUser("username", "passsword");
bool isValid = ValidateUser("username", "password");
Assert.IsTrue(isValid);
This of course would have to be complimented with invalid user/password tests:
test: ValidUser_InvalidPassword:
AddNewUser("username2", "pwd");
bool isValid = ValidateUser("username2", "wrongPassword");
Assert.IsFalse(isValid);
test: NonExistingUser:
bool isValid = ValidateUser("non_existing_user", "anyPassword");
Assert.IsFalse(isValid);
The argument against this would be that you are testing more than one unit in a single test. But personally I think this is better. Why?
Because the tests are not so brittle - i.e. if you make an internal change to the hashing algorithm the test is there to check that everything still works. You don't have to change the hard coded encrypted password in the test code.
That is one of the main benefits of unit tests: to check that we don't break anything when we refactor. So when we want to change the internal implementation for whatever reason (code cleanliness/performance or security improvements), the tests give us confidence that we have not broken the functionality.
An interesting article discussing the benefits of higher-level tests can be found in this Dr Dobbs article:
Yes, you could have your setup function add the user with an hardcoded encrypted password to the mock repository. When unit-testing, you should use known values so that the behavior of the tested functions can be predicted.
Let's say in my user model I have a ChangePassword method. Given an already initialised user model, it takes the new password as a parameter and does the database work to make the magic happen. The front end to this is a web form, where the user enters their current password and their desired new password. The controller then checks to see if the user's current password is correct. If so, it invokes the user model's ChangePassword method. If not, it displays an error to the user.
From what I hear you're supposed to unit test the smallest piece of code possible, but doing that in this case completely ignores the check to make sure the user entered the correct current password. So what should I do?
Should I:
A) Unit test only from the controller, effectively testing the model function too?
OR
B) Create 2 different tests; one for the controller and one for the model?
When in doubt, test both. If you only test the controller and the test fails, you don't know whether the issue is in the controller or the model. If you test both, then you know where the problem lies by looking at the model's test result - if it passes, the controller is at fault, if it fails, then the model is at fault.
A)
The test fails. You have a problem in either the model or the controller, or both and spend time searching through the model and controller.
B)
The model and controller tests fail... chances are you have a problem in the model.
Only the controller test fails... chances are better that the problem is not in the model, only in the controller.
Only the model test fails... hard to see this happening, but if it does somehow then you know the problem is in the model, not in the controller.
It's good to test both layers. It'll make finding the problem later that much easier.
There should be multiple tests here:
Verify the correct password was entered.
Validate the new password, e.g. doesn't match existing one, has minimum length, sufficient complexity, tests for errors thrown, etc.
Updating the database to the new password.
Don't forget that the tests can also help act as documentation of the code in a sense so that it becomes clear for what each part of the code is there.
You might want to consider another option: Mock objects. Using these, you can test the controller without the model, which can result in faster test execution and increased test robustness (if the model fails, you know that the controller still works). Now you have two proper unit tests (both testing only a single piece of code each), and you can still add an integration test if required.
Unit testing means to test every unit on its own, so in this case you would need to build two unit tests, one for the frontend and one for the backend.
To test the combination of both an integration test is needed, at least the ITSQB calls it like that.
If you code object oriented you usually build unit tests for every class as that is the smallest independent unit testable.
A) is not a unit test in my opinion since it uses more than one class (or layer). So you should really be unit-testing the model only.