Recently,I am studying about django test. Because I need to build unittest and integration test in my website.But I find that the tutorial of integration test in django is really less and it often occurs when I hit a link named "integration test" ,I only see the title "functional tests".So,are they the same? if not,how can I start a integrstion test in django?
You cannot compare these notions.
The opposite to integration testing is unit testing. Unit testing - is testing different isolated parts (usually small code blocks) of your system separately - it's very focused, integration testing is testing how these different parts of your system work together - for example, url routing, logic in views, logging, querying your models etc.
Functional testing is a type of black box testing that usually checks that some (usually documented) piece of functionality works as expected.
You may have all sorts of tests in your django project:
unit tests of different library, helper functions
view tests (this could be already called integration tests, because it may include dealing with models, logging etc)
ui tests (high-level tests, that could be called functional/integration/system)
..
If you don't have tests at all, I'd start with high-level tests. For example, I'd take selenium and django_selenium and write some in-browser tests that will go through pre-defined scenarios, like login->do smth->logoff, login with incorrect credentials->see error etc - and these tests would be called functional and system and integration and ui and etc - you got it I think.
See also:
Testing Django Applications
A Guide to Testing in Django
Carl Mayer's "Testing and Django" slides
Integration Testing in Python
Hope that helps.
Related
How do you write unit tests for Htmx?
Stack overflow isn't letting me post such a short question, so this paragraph says that I didn't see docs or a library about this.
EDIT: Specifically, I wish to unit test in Django.
There are a couple of solutions to this:
Many server side platforms offer the equivalent of rails "functional" controller tests, and those can be used to test your partial end points, depending on what platform you are using.
You can write client side javascript tests in the normal manner. The htmx test suite uses chai.js & mocha.js to create a test suite, and sinon.js to mock out the server side. If you can run the tests against your actual server, that will simplify things even more.
Broadly, for day to day unit tests, I would lean towards the first approach because it will be more stable and "functional" and then use the second approach for integration testing.
I guess you use some kind of framework. For example Django, Laravel, Ruby on Rails, ...
Just use the tools of your framework to test your http endpoints.
For e2e tests I would use Playwright. But keep in mind this rule: 80% unittests, 15% integration tests, 5% e2e tests.
I have one e2e test which checks to happy path.
And a lot of pytest-django based tests.
For forms I use this pattern: html_form_to_dict
Maybe this question gets better answers in the discord channel, since there is no clear answer.
Context: On a project (C#/.Net in backend and integration tests, TypeScript/React in frontend) I have complex integration tests that are more like "test scenarios": they test the business logic through the UI and somehow test the functionalities of the UI, for example: it ensures an input field is readonly after having done some modification in the page.
I mainly have 2 issues with those tests:
Finding DOM elements
Most of the tests describe a scenario but for some reason the testing framework has been designed to represent what is on the page of the frontend. For example, a test might go to a form page for edition and then click on the "Attachments" tab to get the attachments panel and then do some clicks to upload a file.
To do that, I've been asked to modify the working frontend to add a "data-id" attribute on the elements that should be clicked by the integration test. I can't say why but this sounds wrong to me. It feels like I'm going to introduce code (even simple) of the integration tests to the frontend application.
So my first question is: how does everybody usually find their DOM elements?
My concern is that if I actually look for the elements using an xpath or CSS selector I will end up with tests that have a strong dependency on the DOM generated by the frontend while using data-id would probably make it more flexible. So if you decide to change the location of an element, you will break easily the integration tests.
On the other hand, I have the feeling that the tests are describing a bit too well what they are doing and for example they should be simplified from a 3 steps logic Go to attachments > Select file > Click add attachment to a single step logic Call magic helper that will upload an attachment in order to have this logic in one single place.
Limit of the integration tests
We have backend unit tests, frontend unit tests and integration tests. Integration tests outnumber everything by a factor of 2 or 3 the rest. And I'm not sure that we put the right stuff in those integration tests.
For example all the different login methods are tested by integration test. The account/bad password locking mechanism is also tested by integration test. We don't have/need unit tests at all and most of the time they feel duplicated with the integration tests.
To get more into this example I think the logic of the account/bad password locking should be tested by unit tests while the integration tests should check only the login methods that require an integration with an external service (using Windows credentials for example or OAuth).
But I'm not sure of what exactly I would put where. So my question is: how do you determine if a test belongs in an integration test or backend/frontend unit test? Is there a list of questions you should ask yourself?
Some interesting reading on the subject: Frontend testing: what and how to test, and what tool to use?
Especially:
Don't write tests for every small corner case. These tests are expensive to write and take too long to run. You should focus on the cases that explore most of your functionality. If you write too many tests at this level you will probably test the same functionality that you have previously tested on your unit tests (supposing you have written them).
I'm trying to add unit-test and integration-test for my grails application but I have some trouble how to distinguish between both and when to use unit or integration to test my controllers actions and services.
The tutorial I found online is not very clean. I can't find complete example to follow up.
Can you please share helpful topics?
I follow the following guidelines:
Try writing as many unit tests as you can. They can be written for controllers, services, domain classes or any other groovy classes. The idea is that unit tests are friends for developers. Writing enough unit tests will
make sure that the developer makes lesser mistakes. As they execute
quickly, this means quick verification. But unit tests cannot test the following:
Criteria queries, HQL queries
Actual database Interactions (queries, transactional behaviour, updates, db constraints etc.)
Inter modular interactions
So we write the Integration tests as well
Integration tests take longer to execute. Writing Integration tests often need bootstrapping data. But they really are helpful to test functionalities end to end (excluding the actual user interactions through UI for which functional tests are written). So Integration Tests can be written for:
Testing all database interactions as unit tests actually do not test the database interactions. This also includes testing criteria, hql etc.,
Testing transactional behaviour (which is dependent on db)
Testing implementations end to end. So this will also test how two independently created modules interact with each other and make sure we have created them correctly.
One problem with integration tests is their speed. For me, integration tests take 15+ seconds to start up. In that time, certain things do slip out of mind focus.
I prefer to go with unit tests that start in no more then 2 sec and can be run several times in those 15 seconds.
One more argument for unit tests is they force you to decouple your code. Integration tests always tempt you to just rely on some other component existing and initialized.
Important links:
http://spockframework.org/spock/docs/1.0/interaction_based_testing.html
http://docs.grails.org/latest/guide/testing.html
Unfortunately it is not just a matter preference or speed. It is a huge subject, but I can give you some advice based on my experience.
If you expect to be covering your database access code (queries, transactional behaviour) by using unit tests, you are deluding yourself. You are testing how your queries comply with the in-memory implementation of GORM. Not hibernate, not your database.
I usually have two types of tests. Unit and functional tests. The functional tests will perform a full test, running against a real database, and stimulating the system like a user would (if it is a web site via Geb, if it is a REST api, via a REST client).
The functional tests will set up a startup state by executing some kind of fixture code first. This can be registering a user and logging them in, for example. Then the test will run, and then the postconditions are checked. Here, you can check the postconditions either by accessing the database through the GORM API, or by using production API calls (danger of covering a bug with another bug).
Sometimes, your system will interact with a third system. Here, if you can, you can mock the implementation of the third system, by injecting a mock implementation into the system under test.
You have also tools like Spring Cloud Contract, that allow you to create bock a mock server for your system under test, and a specification for your third-party system. See https://cloud.spring.io/spring-cloud-contract/spring-cloud-contract.html
The unit tests, I use to thoroughly test all execution paths of a given class. I will try to trigger all exception states, all secondary scenarios, to make sure that everything is covered. I don't think it is realistic to have 100% coverage by using functional or integration tests.
I am pretty confuse with unit and integration testing of traditional software development and web development. I have seen many different answer and explanation of it.
The Web Engineering textbook says about unit and integration testing for web application:
Unit testing: Testing on single web page as opposed to testing single function
Integration testing: Testing on flow of data from one web page to another (and linkage)
while the software engineering textbook defines unit testing and integration testing as followed.
unit testing: Testing on smallest unit
integration testing: Testing on interaction between unit or module
Hope someone can clarify to me which is the correct one.
The "web engineering" textbook is... likely wrong. That's not what those words mean to most folks.
Unit testing: testing the smallest possible bits of functionality, independently. For Java, something like the JUnit framework is used to do this. You often try to test just one class, and you may fake it's dependencies using something like Mockito, so you're really testing just one thing.
Integration Testing: testing several parts of the system together. This may be a small integration test (testing multiple classes without mocking), or something large, like making sure that your webserver is connecting to a database correctly.
End-to-End Testing: the biggest Integration test; this is basically standing up every part of your system and running scripts that look like fake users. Selenium is a tool used for this.
I've been ignoring the need to test my project for far to long.
So I spent more than a day looking for ways to implement tests for my current apps and trying to get some TDD going for new apps.
I found a lot of "tutorials" with the steps: "1. Install this 2. Install that 3. Install thisnthat 4. Done!",
but noone seems to talk about how to structure your tests, both file and code wise.
And noone ever talks about how to set up a CI server, or just integrate the testing with the deployment of your project.
A lot of people mention fabric, virtualenv and nose - but noone describes how they work with them together as a whole.
What I keep finding is detailed information about how you set up a proper Rails environment with testing and CI etc...
Does anyone else feel that the Django community lacks in this area, or is it just me? :)
Oh, and does anyone else have any suggestions on how to do it?
As I see it, there are several parts to the problem.
One thing you need are good unit tests. The primary characteristic of unit tests is that they are very fast, so that they can test the combinatorial possibilities of function inputs and branch coverage. To get their speed, and to maintain isolation between tests, even if they are running in parallel, unit tests should not touch the database or network or file system. Such tests are hard to write in Django projects, because the Django ORM makes it so convenient to scatter database access calls throughout your product code. Hence any tests of Django code will inevitably hit the database. Ideally, you should approach this by limiting the database access in your product code to a thin data access layer built on top of the django ORM, which exposes methods pertinent to your application. Another approach is for your tests to mock out the ORM calls. In the worst case, you will give up on this: Your unit tests become integration tests: They actually hit the database, cross multiple layers of your architecture, and take minutes to run, which discourages developers from running them frequently enough.
The implication of this is that writing integration tests is easy - the canonical style of Django tests covers this perfectly.
The final, and hardest part of the problem, is running your acceptance tests. The defining characteristic of acceptance tests is that they invoke your application end-to-end, as a user does in production, to prove that your application actually works. Canonical dhango tests using the django testrunner fall short of this. They do not issue actually HTTP requests (instead, they examine the url config to figure out what middleware and view would get called to handle a particular request, and then they call it, in process.) This means that such tests are not testing your webserver config, nor any javascript, or rendering in the browser, etc. To test this, you need something like selenium.
Additionally, we have many server-side processes, such as cron jobs, which use code from our Django project. Acceptance tests which involve these processes should invoke the jobs just like cron does, as a new process.
Both these scenarios have some problems. Firstly, you can't simply run such tests under the Django test runner. If you try to do so, then you will find that the test data you have written during the test setup (either using the django fixtures mechanism, or by simply calling "MyModel().save()" in a test) are in a transaction which your product code, running in a different process, is not party to. So your tests have to commit the changes they make before the product code can see them. This interferes with the clean-up between tests that Django's testrunner helpfully does, so you have to switch it into a different mode, which does explicit deletes rather than rolling back. Sadly, this is much slower. At last night's London Django user group, a couple of Django core developers assured me that this scenario also has other complications (which I confess I don't know what they are), which it is best to avoid by not running acceptance tests within the Django test runner at all, but creating them as a completely stand-alone set of tests.
If you do this, then your immediate problem is that you have lost the benefits the Django test runnner provides: Namely it creates a test database, and cleans it between each test. You will have to create some equivalent mechanism for yourself. You will need your product code to run against a test database if it is being invoked as part of a test. You need to be absolutely certain that if product code is run as part of a test, even on a production box, then it can NEVER accidentally touch the production database, so this mechanism has to be absolutely failsafe. Forgetting to set an environment variable in a test setup, for example, should not cause a blooper in this regard.
This is all before even considering the complications that arise from deployment, having parts of your project in different repos, dependent on each other, creating pip-installable packages, etc.
All in all, I'd love to hear from someone who feels they have found a good solution to this problem. It is far from a trivial issue as some commenters imply.
Harry Percival is creating a Django / TDD / Selenium tutorial (and accompanying workshop, if you live in London.) His book reads like a hands-on tutorial, and goes into great detail on the subject:
https://www.obeythetestinggoat.com/book/part1.harry.html
In my experience, fine-grained unit tests for web apps are not worth it, the setup/teardown is too expensive and the tests are too fragile. The only exception is isolated components, especially those with clear inputs & outputs and complicated algorithms. Do unit-test those to the smallest details.
I had the best testing experience using a semi-functional testing tool called testbrowser, which simulates browser actions in Python. For integration with Django, install the homophony app (disclaimer: I am the author of the app).
Testbrowser may be a little too coarse for test-driven development, but it's the best testing tool of the ones I have used so far. Most importantly, it scales up fairly well, whereas unit tests and browser-based functional test tools tend to become very brittle as your app grows in size.
As for a CI tool, go with Buildbot or Jenkins.
I use a combination of Django's excellent extension of the python unittest framework for testing api's / models / helper functions, and selenium for in browser testing. Selenium has great instructions for how to set it up and write tests in python.