I'm following TDD technique in my new Laravel project. Thus I have a set of tests which cover my controllers, model classes, services, etc. Most of these tests are HTTP tests, so I stored them in the /tests/Feature directory. Additionally I have few unit tests, which cover quite specific methods, which are not (easily) reachable from the HTTP tests.
If I understand correctly, each HTTP test is a functional tests, because it covers a lot of classes included Controller. Should I in that situation separately create unit tests for each method in my project even if it is already covered by HTTP tests? If yes, what benefit can I take from it.
Thank you in advance for explanations.
The philosophy of creating unit tests is to test a small piece of your code, for example, if you have an API, so it only returns the list of 10 recent posts then I guess it's no need to create a unit test for that and it can be covered with writing the functional or HTTP tests. But let's assume you have an endpoint that the user uses that endpoint to upgrade his account from regular type to golden type so he/she can access more posts or videos. There will be definitely a lot of things going on in that endpoint, so yeah you need to write Unit tests in addition to functional tests. Also, one more thing is that when you write functional tests you should see it from the QA perspective, I can categorize it like this
Functional/Http/Feature => [Validation Checks, Response Checks=>[The endpoint throw an error in different situations or return success if all goes well]]
Unit => [Write the test for the small functions that they get called by that endpoint Or write the test for that endpoint and mock everything so you can get what you expect]
Integration => [Write tests for the third party APIs or database persistence or caching persistence that you are using in your application]
So if you have an endpoint and you are writing tests for that endpoint and your functional tests has covered some part of that endpoint logic then I guess it's not necessary to write more unit tests for it.
Related
I am learning the art of Unit Testing and BDD and in my company there is no one who follows this approach. I try a lot to learn it by myself but get stuck somewhere and give up after trying for a few days. After some time I get inspiration again from someone and try to learn to swim in these waters again.
I have recently developed a Windows Service that started out small but ended up a big ball of messy business rules. Here is a brief overview of what the service does.
Log to database “Service starting…”
Get Data From DB that needs to be posted to another web service
If there is no data to post Log to database “No data to process…”
and exit service
If data contains duplicate values Log to database “Duplicate data
found, this record will be skipped.”
Update the status of the record for which duplicate data was found
to something e.g. 302
If the data is null, Log to database “Record contains null value and cannot be processed.”
Update the status of the record appropriately e.g. 310
If the database is unavailable or down due to some reason, Log in a file “Database is down…”
If the service is down where we have to post the data log to database “Receiver service is down.”
Log to the database “Exiting service…”
So my service basically retrieves some data from the Database, creates JSON request from it and post it to another service.
It also parses the response from that service and logs if the data was posted successfully or not. I have just entered some of the business rules that are currently implemented in the service to give you an idea of what lies under the hood. I am learning BDD and unit testing and would really love to know how an expert will write test cases that test these complex business rules?
From my understanding BDD does not need to focus internally on how the service is written, instead it will test scenarios that service should fulfill For example
When executing the windows service with duplicate data
It should log to database “Duplicate data found, this record will be
skipped.”
It should update the status of the record to 302
I can write multiple scenarios that test some functionality of the service. Is this the right approach or should I right very large sets of scenarios that test each business rule in every test?
Secondly as the service is communicating with the DB as well as a web service, how can I test the HttpRequest and HttpResponse that is sent and received by the service?
Finally how do I actually test something as complex as the business rules I have written above, If I simple assert that the service called some specific method of some class is that enough? How do we know that by simply calling some method it will perform the right task?
A few simple thoughts to help keep it in perspective:
You say learning...
Keep that in mind and don't get hung up on what is perfect, right, or proper. You're learning, feel free to make mistakes and know that you'll improve as you go. Keep at it, keep practicing, you'll get better and it will feel more natural the more you do and the more you think about it.
BDD tests behaviors.
You use this to say the system should behave in a specific way, and that implies that it must be the system. You may still stub in a couple dummy services, on occasion (like a fake credit-card processing service) but for the most part, you want this to prove the system works as needed. Think of them as more of integration tests.
Your BDD tests should drive your Unit Tests.
Write the BDD test to fail, and then let that dictate what unit tests should be written in order to get your system to behave as you expect. This essentially means that each BDD test of yours will introduce a set of Unit Tests as well.
In summary, let BDD drive TDD
and you'll have the right balance of tests. The starting point is with your first BDD test.
And in your scenario, if your system is supposed to alert the user that they are attempting to add a duplicate, that's a valid test.
The annoying thing with testing Http Requests and Responses are that you end up doing string comparisons, but that is doable. The BDD tests should just care that the system responded as you expect.
The unit tests should isolate in respect to what you are doing, so you would have unit tests inside your web service to make sure it responds correctly, but you would not have a unit test on the outside that calls the web service; extract it out instead.
It all can become pretty philosophical, and that probably gets into what makes a good unit test versus what makes a good behavioral test, but hopefully this helps you get started down the road.
I am trying to write some unit tests for testing my service layer which I am doing well I think, the service layer as a dependency on a repository so am mocking a repository using RhinoMocks, so I am testing the Service layer "WITHOUT" hitting the database which is great.
Now I need to test my repository layer, this has a direct connection to a database so I have to test it don't i? I have no other option but to test it?
If I test another implementation of the Repository that doesn't hit the database then this is no testing my implementation.
I have managed to mock out all lower layers so anything that depended on code that takes a while to run ie. The repository, then I mocked this out. The result is that all my tests for layers below the repository complete fast and do not hit the database.
The problem is what do I now do with the repository. I have to test it but it has a dependency on a SQL database.
Well, the general answer goes like this. I would write unit tests that verifies the logic of the repository layer and break out the sql dependency in a new class and mock it in the tests of the repo. If the repository layer contains only a sql connection and no logic there is nothing to unit test in my opinion. Then you are more suitable with integration tests with the database connected.
Thus mocking code you don't own is a bad practice, I think best option for you is to test repositories via acceptance/integration tests
You certainly can test your repository layer without talking to a database. Most ADO.net classes are mockable, if you are careful about how you create them and you are careful to couple to interfaces instead of concretions. Unfortunately, ADO.net was created before mocking was a very popular practice, and it is still a bit of a pain to do.
The real question in my mind is whether you should try to mock them. The benefits of mocking are twofold: they run faster, and they force you to encapsulate more details about your database (making it easier to switch out db technologies, if you ever want to do it). The benefits of functional tests are that they also test your database layer (stored procedures, etc), they are arguably easier to write, and they are easier to maintain in the sense that if a db change is made, integration tests notice automatically, instead of you hunting the mocked out tests down.
I would say the "best" approach would be to test them both with moqs and with the real database, since this gives you the best of both worlds. However, it is quite costly of course.
What are the best Junit tests for soap services.
Would connecting to the web service fall under a unit test or an integration test?
I need to pick useful unit tests and was looking for ideas on tests that would be automatable and repeatable.
Thanks.
For unit tests, you should test an isolated behavior of a single class, and therefore use mocks/stubs/test doubles as much as possible. If you do not mock, then you are not testing an isolated part of your code. Hence isolating the error becomes more difficult and the test might also not be repeatable (e.g. strange network behavior might occur unrepeatably).
You should also try to automate your integration tests, where you replace your mocked objects with the real implementations.
At this infoq site you can find a short introduction about mocking web services, and several links to tutorials/tools/whitepapers.
Should we unit test web service or really be looking to unit test the code that the web service is invoking for us and leave the web service alone or at least until integration testing, etc.... ?
EDIT: Further clarification / thought
My thought is that testing the web service is really integration testing not unit testing?? I ask because our web service at this point (under development) is coded in such a way there is no way to unit test the code it is invoking. So I am wondering if it is worth while / smart to refactor it now in order to be able to unit test the code free of the web service? I would like to know the general consensus on if it's that important to separate the two or if it's really OK to unit test the web service and call it good/wise.
If I separate them I would look to test both but I am just not sure if separation is worth it. My hunch is that I should.
Unit testing the code that the web service is invoking is definitely a good idea since it ensures the "inside" of your code is stable (and well designed). However, it's also a good idea to test the web service calls, especially if a few of them are called in succession to accomplish a certain task. This will ensure that the web services that you've provided are usable, as well as, work properly when called along with other web service calls.
(Not sure if you're writing these tests before or after writing your code, but you should really consider writing your web service tests before implementing the actual calls so that you ensure that they are usable in advance of writing the code behind them.)
Why not do both? You can unit test the web service code, as well as unit test it from the point of view of a client of the web service.
Under my concept, the WS is just a mere encapsulation of a Method of a Central Business Layer Object, in other words, the Web Method is just a "gate" to access methods deeper in the model.
With the former said, i do both operations:
Inside the Server, I create a Winform App who do Load Testing on the Business Layer Method.
Outside the Server (namely a Machine out of the LAN where the Web App "lives"), I create a Tester (Winform or Web) that consumes the WS, doing that way the Load Testing.
That Way I can evaluate the performance of my solution considering and discarding the "Web Effect" (i.e. The time for the data to travel and reach the WS, the WS object creation, etc).
All the above said is of course IMHO. At least that worked a lot for me!
Haj.-
We do both.
We unit test the various code elements.
Plus, we use the unit test framework to execute tests against the web service as a whole. This is rather complex because we have to create (and load) a database, start a server, and then execute requests against that server.
Testing the web service API is easy (it's got an API) and valuable. It's not a unit test, though - it's an "integration", "sub-system", or "system" test (depends on who you ask).
There's no need to delay the testing until some magical period called "integration testing" though, just get some simple tests now and reap the benefit early.
I like the idea of writing unit tests which call your web service through one of its public interfaces. For instance, a given WCF web service may expose HTTP, TCP, and "web" bindings. Such a unit tests proves that the web service can be called through a binding.
Integration testing would involve testing all of the bindings of the service, testing with particular client scenarios, and with particular client tools. For instance, it would be important to show that a Java client can be created with IBM's Rational Web Developer that can access the service when using WS-Security.
If you can, try consuming your web service using some of the development tools your customers will use (Delphi, C#, VB.Net, ColdFusion, hand crafted XML, etc...). Within reason, of course.
1) Different tools may have problems consuming your web service. Better for you to run in to this before your customers do.
2) If a customer runs in to a problem, you can easily prove that your web service is working as expected. In the past year or so, this has stopped finger pointing in its tracks at least a dozen times.
The worst was the developer in a different time zone who was hand crafting the XML for SOAP calls and parsing the responses. Every time he ran in to a problem, he would insist that it was on our end and demand (seriously) that we prove otherwise. I made a dead simple Delphi app to consume the web service, demonstrate that the methods worked as expected and even displayed the XML for each request and response.
re your updated question.
The integration testing and unit testing are only superficially similar, so yes, they should be done and thought of separately.
Refactoring existing code to make it testable can be risky. It's up to you to decide if the benefits outweigh the time and effort it will take. In my case, I'd certainly try, even if you do it a little bit at a time.
On the bright side, a web service has a defined interface, so you don't really have to change anything to add integration testing. Go crazy. If you can, try to do this before you roll the web service out to customers. There's a good chance that using the web service lead to changes in the interface, and you don't want this to mess up customers too much.
I am building a web application that uses the database for Users, Security/roles, and to store content.
It seems a little daunting to me to begin on the road of unit testing because I have to make sure my database has been initialized properly for my tests to run.
What are common practices to help in this regard?
i.e. while developing/testing, I might delete a user, but for my test to pass that user has to be in the database, along with his profile, security settings etc.
I know I can create a setup script, something to recreat the databas etc.
I don't want to end up spending my entire time maintaining my tests and ensuring my database is in sych
Or is that the cost of Unit Testing/TDD?
The solution is Mocking. Mocks "replace" the connection. The unit under test will "connect" to the Mock and executes its statement. The Mock returns normal resultsets o.s.e.
After the test, the mock can give you a list of all methods, that were called by the unit under test. Easymock.org
As the other said: DB connection aren't a unit test. So drop it and do it local with Mocking objects
It's not a unit test if you are testing more than one unit.
Usually you'll have one component (your page, or the business layer) talking to a data layer object that is responsible for actually connecting and querying the database. My recommendation is to develop a unit test for the first component, using dependency injection to pass in a mock version of the DataLayer (which acts on hardcoded data, or a List you pass in, etc). This way you are testing your higher level code in isolation from the other components.
Then you are free to develop other unit tests (and integration tests) for the data layer to ensure that it is handling it's job (writing to the database) correctly.
We use an in-memory database (hsqldb) for our unit tests. In the setup we populate it with test data and then before each test case we start a transaction and after each test case we roll it the transaction back. This means each testcase has a clean start of the db.
It sounds like you actually want functional/integration testing. For Web applications I recommend you look into Selenium or Canoo WebTest.
These are also great for automating tasks you do on the Web. I have a set-up suite and a tear-down suite that create business entities and testing users through the admin interface as well as tests for the customer-facing site.
Michael Feathers argues that tests that communicate with databases aren't unit tests by definition. The main reason for this is the point you bring up: unit tests should be simple and easy to run.
This isn't to say that you shouldn't test database code. But you don't want to consider them unit tests. Thus, if you do any database testing, you want to separate the tests from the rest of your unit tests.
Since I used Doctrine for my PHP database work, and since Doctrine has a query abstraction layer (called DQL), I can swap out back ends without having to worry too much about compatibility issues. So in this case for my unit tests I would just at the beginning of my tests load the schema and fixtures into a SQLlite db, test my models, and discard the SQLlite db at the end of testing.
This way I've tested my models and data access to make sure their queries are formed correctly.
Now testing the specific database instance to make sure the current schema is correct is a different story and IMHO probably doesn't belong in your Unit Tests, so much as it belongs in your deployment task list.
The cost of unit testing/TDD is that you have to alter your design so that you have a clean separation between the database and the domain layer, so that you can create a fake that will allow you to create tests that don't hit the database.
But that cleaner design is just the beginning of the cost. After that you have to create tests that both help you make the code work right the first time and alert you when anyone breaks something that used to work.
And after you have a good fundamental design with tests that protect your existing functionality, you'll find yourself cleaning up code to make it easier to work with, with confidence you aren't breaking things along the way.
And so on and so on... The costs of unit testing/TDD just keep piling up over time.
For Java, you may also want to look into dbunit. http://www.dbunit.org/