What are the best ways to test servlet oriented web applications. With possibly jdbc backed backends.
Front-end = Struts2 and some servlets
Back-end = hibernate, some basic jdbc
use mocks, easymock is one of the best mock frameworks(or you could use jmock if you like the name better). spring-mock has nice support for mocking servlet classes.
you could also do integration testing with dbunit(to tests your sqls and hqls), but its slow and take much longer to write and maintain. i think you should not do it unless you had some problems with people writing bad queries or database schema is changed a lot.
Most important thing : design for testability.
It means trying to have small independent components (which is almost always a good thing, not only for testability) and test components separately. Inversion of Control (with or without an IoC framework like Spring) can help you a lot in that area. Once you have small components, it's pretty easy to test them with jUnit, easymock or any other standard test utilities.
Testing the frontend, is the most tedious and boring task. Selenium can help you a bit, but there is no silver bullets that I know of.
Related
What is the advantage of mocking the container services in Unit testing of EJB 3.1?
The probable answers I get when I think about it are,
It improves the performance of the tests.
It does not abide by the rules of Unit testing as there is a lot of interactions with other APIs. (Please provide your views on this)
Other than these, do you think there are other advantages?
As many of you may know, it is possible to test some of the services provided by the container, like persistence, transaction management (eg. using Bitronix), messaging (eg. using Apache ActiveMQ and in-memory JNDI) out of the container in your own JVM. Still there is an argument that it is integration testing and unit testing should not be done that way.
According to me, if you can have a good performance in your tests, it is fine to use these third party implementations for unit testing because you do not have to spend too much time in mocking and mocking is heavily subject to developer errors. If a developer does not have a good understanding of mocking, he might end up mocking everything or in other words misusing mocking to turn the tests "green". Is this right? (Please provide your views on this)
After all, I never got any solid definition of unit testing :-). It depends on the author. Some define "unit" as the smallest unit that can be tested and some define "Depending on the context, these could be the individual subprograms or a larger component made of tightly related units."
Thanks.
If you have code which uses container services, then to test it, you will need to either mock those services, or use a real implementation. You have to do one or the other: without some implementation of the services, your code will not run, and so cannnot be tested.
Sometimes, you can refactor your code to remove the direct dependency on the container services, which will also remove the need to mock those services. But not always.
Mocking the container services provides more isolation than using a real implementation. It also gives you more control over and insight into the execution of your code. However, it also involves writing more code, and with it, more risk of introducing bugs (bugs in mocks, that can translate directly into bugs in application code).
There are some times when mocking definitely makes sense. For example, if you want to write a test that checks that your code is making the correct calls on UserTransaction, then that it much easier to do by mocking than by trying to instrument a real transaction monitor. If you want to write a test that checks that your code handles a particular SQLException correctly, then that is almost impossible to do without a mock.
Beyond those cases, as you point out, it is possible to write tests using the real services, or mocking them. As i think you have realised, the orthodox unit testing approach would be to mock them, or, in fact, to wrap them, and then mock the wrappers.
Whether this is actually necessary, or a good idea, is very much open to debate.
StackOverflow is not supposed to be for subjective questions, or for debates or discussions, so i hesitate to go into my opinion on this. Suffice to say that it is the same as i suspect yours to be - that the orthodox 'mock everything that moves' approach is unnecessary and harmful, and we would be much better off writing tests with less mocking, covering larger areas of real code. After all, real code is what we're going to ship to users, so why not test it?
I'm thinking of designing my own Test library (framework) ->in c++
I am wondering if some of you have already designed there own (and what good advices, documentation they could offer me), decided to not do that (and why), What critics (and argue) you have against differents existing testing frameworks.
I want to no more about testing framework design.
In fact I have some pretty differents things to test :
simple unit test
MVC and signal slot
data, (escpecially for audio and DSP)
performance
compatibility
"So much things ... and so few time "
No really I need to test a lot of different things.
So I checkout how is designed XUnit, and the Addison and W XUnit related book, also the Advanced Unit Testing related article on code project....
And different articles, discuss this with coworkers ...
And at the end, I want to design my own.
Why :
specific needs,
like the do it yourself way (and learn why it's done this way in existing frameworks and that I'm not a genious ... ^^)
Thank you all.
I remember having read some discussions about Cppunit 2 design on the sourceforge wiki. I'd start from here. Also, Noel Llopis explored the C++ unit-testing framework jungle.
But, you're saying you want to re-create another framework and you only have few time left. I'd suggest picking one framework fitting your need for the unit tests, see if it can be used fore your MVC and data testing. Moreover unit testing framework are not designed to run performance tests. I'd recommend following the Unix philosophy here: simple little tools that do one thing and do it well.
Learn at least one existing framwork before you implement your own. My experience is that the framework is not the problem. Learning how to write good unit tests is the hard part.
I have used several framworks through the years including CxxTest, CppUnitLite and UnitTest++. But my recommendation is Google Test together with Google Mock (Google Mock comes with a copy of Google Test bundled).
Recently there has been quite some hype around all the different mocking frameworks in the .NET world. I still haven't quite grasped what is so great about them. It doesn't seem to be to hard to write the mocking objects I need myself. Especially with the help of Visual Studio I quickly can write a class that implements the interface I want to mock (it auto-generates almost everything for me) and then write an implementation for the method(s) I need for my test. Done! Why going through the hassle of understanding a mocking framework for the sole purpose of saving a few lines of code. Or is a mocking framework not only about saving lines of code?
Once I finally got the hang of mock objects, I realized that they're essential for unit testing for the same reason that double blind testing or control groups are essential for scientific trials: they isolate what you're actually testing.
If you're testing a class which has quite a bit of interaction via other interfaces, you not only save the lines of code on having to mock each and every interface, but you also gain the ability to do things like "throw an exception if an unexpected method is called" or "exception if these methods are called out of order". You can get remarkably sophisticated with mock frameworks, and though I'll quickly admit there's a large learning curve, when you get up to speed they'll help make your unit tests more thorough without being bloated.
You actually identified one of the key points of a mock framework in your question. The fact that you code the mocks yourself is not something the developer should be concerned with. The mocking frameworks give you implementations of interfaces programatically, plus they are functional (based on your setup of the mock).
What do you do if you are testing an ICustomerDAO, for example, and you want to test some method 14 times each with different outcomes? Implement 14 different classes manually? I doubt that anyone would want to do that.
Mocks give you the power to define what will happen with parts of your classes when you are not concerned with whether or not they will actually work, like throwing exceptions whenever you want them to, returning zero results and making sure you handle that correctly, etc...
They are a great unit testing tool.
Previous questions that may help:
What is a mock and when should you use it?
Mockist vs classical TDD
I find that using a mocking framework allows me to generate tests a lot faster and with better verification that what I expect to happen in the test actually is happengin. I have in the past implemented stubs or fakes myself. I found that I needed to generate stubs specific to the test that I wanted and this took a lot of time. I can create the same test much faster using a mocking framework. The good ones support the generation of fakes, stubs or mocks with straightforward syntax.
It takes a while to get the hang of it, I avoided it for a while but now wouldn't try to work without a mocking framework for the reasons #Chamelaeon states.
Roy Osherove had a poll about Mock Frameworks and down in the comment section, there is a discussion (albeit brief) about whether one needs a Mock Framework or not.
I personally have been manually doing exactly as you stated and it has worked well enough, but this has mainly been out of habit rather than a closely-held opinion on mock frameworks in general.
Well I certainly don't think that you NEED a mocking framework. It's a framework like any other, and it's ultimately designed to save you some time and effort. You can also do things like roll your own common data structures like stacks and queues, but isn't it generally easier to just use the ones built into the class libraries that ship with the compiler/IDE of your language of choice?
I'm sure there are other compelling reasons for using mocking frameworks, though I'd leave it to the TDD and unit testing gurus to answer.
For the same reason you wouldn't try to write unit tests without NUnit. A mocking framework will assist you in verifying state and behavior over hundreds of unit tests. It's worth the 2 weeks or so of pain to get up to speed and really helps you focus on what needs to be tested.
One thing that troubles me about a mocking framework is that "what a function should o/p given an i/p" via
when(mock.someMethod("some arg")).thenReturn("something");
statement is spread across many unit test classes.
Let me elaborate with an example. Lets say there was a DAO Interface function getEmp(int EmpID) which was returning an Employee Object when passed an Employee ID as a parameter. Assume that this function was being mocked by 10 different unit test classes. Now if in the future, this function were changed to return a newer version of the Employee Object, one would have to go to each of the 10 different classes to update this change.
The disadvantages are as follows...
a) I don't know how to figure out all the classes which mock this function so that I can go update this change.
b) My existing test cases which consumes the mock DAO object continue to be blissfully unaware of the changes that have happened to the DAO Interface because the mock has not changed and hence continue to be green.
Ideally, if I were to have coded a single mock class myself and consumed it everywhere, I would have just one place to update for the newer version of the Employee object. Also, once I update at this one place, all my existing test cases which consume the mock would break and I would then know exactly what places I need to go and do an update for the new Employee Object.
Any thoughts on my views..
One of the good things about a mocking framework is that it allows setting expectations on the objects being mocked. With the expectations I can then set up all sorts of conditions to exercise the code thats being tested.
An isolation framework or mocking framework allows you to test the code you want, without its dependencies. It makes for short running tests, allows you to debug quickly, and easily build a safety net of tests around the code. Different frameworks have different features, and as said before - it's a tool, and you should select the right tool for the job.
I've use rhino mocks for a mocking framework. I and 5 other developers used it on a large enterprise application that was an 8 month project. We used tdd on the project. Was it worth it? I guess. Was there such a massive huge selling point to using mocks that I have to use it on every project? In my opinion, no. It is not something that is necessary, it is just a tool that you can use if you want to try it out. Some projects you can roll out your own mock classes as some here say they prefer - it is easier. Other projects are larger and may require a mocking framework. The key word (in my opinion) is MAY require... how much code coverage do you require? To me, that is another consideration to using mocks. The project I did with tdd/rhino mocks we were required to have 80% code coverage so the mocks helped us attain that. If our code coverage requirements were less, for example 40%, we probably would have not used a mocking framework and just wrote our own mock classes as others mention they do.
I am curious about how many of you folks incorporate mocking of objects (frameworks like JMock, NMock, RhinoMocks hand in hand with unit testing frameworks) into your daily development process. What are your experiences?
You see, I develop on top of a GIS (geographic information systems) platform, in which most of work pertains to some way of handling data. As its data object model is quite complex (many many classes and interfaces, all COM-based), it is also quite difficult and cumbersome to mock. In this case, mocking incurs a great deal of overhead when writing test suites. I wonder if there are people in similar situation, or just, how does mocking (in whatever situation you are in) work for you.
On a recent project that I worked on we used mock objects extensively in our unit testing approach. The project was 100% Java and moderately sized (about 100,000 lines of non-commented code). It was a Swing-based desktop application - and the only effective way that we found to test the user interface logic was through an MVC variant design that allowed us to use mock objects to substitute for the actual Swing user interface classes for the automated testing. We also used mocking extensively in the testing of our data access layer (Hibernate/DAOs).
In the user inteface use, the Mocks were easy and straightforward to build. And the design of the application (Fowler Passive View) easily incorporated mocks. This was not the case for the mocks used in testing the data access layer. But I can say that it was clearly worth the effort. In fact, most of the 'effort' really focused on coming up with a reusable solution that minimized the work that a developer had to do to create each individual mock. I'd recommend taking the time to dig into and discover an approach for your situation that allows you to easily mock up your GIS data layer. That - or just manually mock up each class. Either way the ability to run the automated unit tests that rely on the mocks is worthwhile...
In my situation mocks work really nice. But I'm using Python, which is so dynamic that it makes many things involving testing much, much easier.
In situation like yours, when application is mainly data-driven (as far as I see), mocks may not be as useful. Just passing data in and watching it come out should be enough for testing. I would just make sure that application is modularized enough, so this approach can be applied to reasonably small components.
Mocking can be useful in some kind of project. But, sometimes mocking is very time consuming and the ROI of it is low.
Trying to test Sharepoint it seems that mocking is the only way, and only typemock will let you mock sealed classes.
Mocking is used very extensively in my case. Mocks are usually for the classes that has external dependencies, e.g. network, database, filesystem. Any of these can introduce flakiness in the tests if mocks are not used.
If the mocks that you find costly to write because there are a lot of fake data to populate, you could set some pre-populated data objects as constants and use them or slightly modified copies in your test. If such data objects has external dependencies, then maybe refactor it in a way you can separate the two concerns.
There is an initiative started by Dave Bouman to try and build a community library of Mocks for use in ArcObjects related unit testing. His blog and this svn repository have great information related to unit testing GIS systems
http://blog.davebouwman.net/CategoryView,category,Unit%2BTesting.aspx
http://svn2.assembla.com/svn/arcdeveloper/TestingUtilities/trunk/
Which scenarios, areas of an application/system, etc. are best suited for 'classic' state based testing versus using mock objects?
I'm going to tackle it from a TDD/BDD perspective, where the tests are driving the designs.
First off it depends what style of design you buy into, because remember this is about design first. As Martin Fowler discusses in this excellent article Mocks Aren't Stubs there are two school of thoughts and they do produce different types of designs.
If you want to buy into the mockist approach then I highly suggest you start by looking at the mockobjects site and their article Mock Roles, Not Objects. They also have a book coming out.
The truth is even if you do believe that the mock-first style of design is not for you, or that you don't want to do it right through your application (e.g. not when testing your domain/service layer) then you will still want to use test doubles. xUnit Test Patterns explains the different types of test doubles and their purposes.
Personally I never mock domain classes, so never mocking entities/value objects. However I've been trying out the mock objects style approach a lot recently, it does change my designs a bit and I find the working style comfortable. Working in this style in an MVC app I'll probably start with an automated acceptance test, then I'll write a controller test that mocks out any non-domain objects (e.g. repositories/services), then I'll move down to testing those repositories/services again mocking out their dependencies. I stop when I reach a class with no troublesome dependencies such as a domain entities/value object. I could go on and test against specific role interfaces which are then implemented by my domain classes, which is what the mockobjects guys would recommend, but I don't currently see a lot of value in that approach.
Obviously it is worth adding that design for test is important here, but remember that although 90% of IoC/mocking/DIP examples show interface-implementation pairs (ICustomerRepository/CustomerRepository) there is a lot of value in instead looking for role interfaces.
You should be using mocks for dependencies. I don't think that its an either-or; Usually you will create mocks for dependencies, set expectations (whether it is calls or state) on them, then run the unit under test. Then you would check its state, and verify the expectations on the mocks, afterwards.
Using mock objects doesnt mean you're not doing state based testing.
When using services, whether my own or third party, I design to interfaces as a matter of course, so I tend to focus on the interactions there. This encourages me to design minimal interfaces.
I check state on anything focusing on value objects, just as simple calculations, lookups, and the like.
The next time you find yourself designing something that typically follows Model/View/Controller-or-Presenter, I highly recommend trying the Presenter First approach (Google it) using interfaces for the Model and View. This will give you a great feel for how to use stubs/mocks effectively.
Its a matter of style.. Mockist vs Classic TDDers.
Personally.. I'd go testing real classes as far as possible. Tone down on Mocks as much as possible; only to decouple things like IO (filesystems, DB Connections, network), Third party components, etc. things that are slow/difficult to get under test.
As an experienced TDD'er, this is a question that I'm frequently asked by other developers. For me, it's a mistake to get sucked into a Mockist vs. Classicist debate, as such a discussion is misleading. State-based and behaviuour-based unit testing are two different tools in your toolbox, and there's no reason why they should be mutually exclusive.
State-based unit testing is suitable when you want to query the internal properties of an object after talking to its external interface. If one or more of its collaborators involve potentially expensive calls to other objects, then by all means stub those calls and simply disregard collaborators.
Behaviour-based unit testing is a good idea when you want to consider the "how" of a unit test and focus upon discoving relationships between objects, as opposed to the traditional "what" questions of a state-based unit test. If you wish to assert that collaborators are used in a certain way and/or sequential order, use mocks - stubs that provide assertions.
I suggest you focus upon standard unit testing practices - exercise as little production code as possible, and have one assertion per test. This will force you to think "what exactly is it that I want to exercise and assert in this test?", and the answer to that question will help you choose the correct unit testing tools.