I'm relatively new to the testing part of rails. I've used unit testing in other languages. but with rails I have noticed those some examples using mocking to test controllers and others using factories, What is the best practise here?
Example using mocks: http://solnic.eu/2012/02/02/yes-you-should-write-controller-tests.html
Example using factories: http://everydayrails.com/2012/04/07/testing-series-rspec-controllers.html
In my research I've noticed this question:
Rails Model Testing - Mocking vs. Factories
Is the answer the same for controllers?
I am leaning towards mocking, mainly to speed things up and test in isolation, but I have seen comments that imply mocks cause tests to be brittle.
Mocks are best at letting you isolate your tests. If you're testing what messages your controller sends, which you probably are, there's no reason to send those messages to a real instantiated factory created object. You can send them to a mock.
Also if you look at the second example. Mostly what you're doing with factories is using them to generate a valid set of attributes to send to a POST or UPDATE request. Using attributes_for not the whole object anyway.
One of the really interesting things I found that struggling with this kind of question about controller testing did for me, was highlight how the traditional way we build and use our Rails controllers is sometimes flawed and encourages a lot of very non-OO practices.
An example of an alternative approach is here: https://www.youtube.com/watch?v=CGN4RFkhH2M
This would be very easily tested using mocks and following Sandi Metz's approach laid out in Practical Object Oriented Design in Ruby - and this talk: http://www.confreaks.com/videos/2452-railsconf2013-the-magic-tricks-of-testing
I include the two links as you appear to be keen to learn more on the topic.
Related
I'm trying to test a simple WebForms (asp.net) based UI, and follow the MVP pattern to allow my UI to be more testable.
As I follow the TDD methodology for backend algorithms, I find that there are some unit test refactorings that happen in the spirit of the DRY principle (Don't Repeat Yourself). As I try to apply this to the UI using Rhino Mocks to verify my interactions, I see many commonalities in the Controller tests when setting up the view or model expectations.
My question is: how far do you typically take this refactoring, if at all? I'm curious to see how other TDDer's test their MVC/MVP based UIs.
I would not refactor tests like standard code. Tests start to become more obscure as you refactor things into common base classes, helper methods, etc. Tests should be sufficiently clear on their own.
DRY is not a test concern.
That said, there are many plumbing things that are commonly done, and those should be abstracted away.
I use MVP, and on my tests I try to apply most of the refactoring I would in standard code. It normally doesn't work quite as well on the tests, due to the slight variations needed to test different scenarios, but within parts there can be commonality, and when possible I do consolidate. This does ease the needed changes later as the project evolves; just like in your standard code it is easier to change one place instead of 20.
I'd prefer to treat unit test as pure functional programs, to avoid to have to test them. If an operation is enough common in between tests, then I would evaluate it for the standard codebase, but even then I'd avoid refactoring tests, because I tend to have lots of them, specially for gui driven BL.
I use selenium for functional testing
and I'm using JUnit to test my controllers.
I'll mock out services or resources used by the controller and test to see what URI the controller is redirecting to, etc...
The only thing I'm not really testing at this point are the views. But I have employed functional testing to compensate.
Recently there has been quite some hype around all the different mocking frameworks in the .NET world. I still haven't quite grasped what is so great about them. It doesn't seem to be to hard to write the mocking objects I need myself. Especially with the help of Visual Studio I quickly can write a class that implements the interface I want to mock (it auto-generates almost everything for me) and then write an implementation for the method(s) I need for my test. Done! Why going through the hassle of understanding a mocking framework for the sole purpose of saving a few lines of code. Or is a mocking framework not only about saving lines of code?
Once I finally got the hang of mock objects, I realized that they're essential for unit testing for the same reason that double blind testing or control groups are essential for scientific trials: they isolate what you're actually testing.
If you're testing a class which has quite a bit of interaction via other interfaces, you not only save the lines of code on having to mock each and every interface, but you also gain the ability to do things like "throw an exception if an unexpected method is called" or "exception if these methods are called out of order". You can get remarkably sophisticated with mock frameworks, and though I'll quickly admit there's a large learning curve, when you get up to speed they'll help make your unit tests more thorough without being bloated.
You actually identified one of the key points of a mock framework in your question. The fact that you code the mocks yourself is not something the developer should be concerned with. The mocking frameworks give you implementations of interfaces programatically, plus they are functional (based on your setup of the mock).
What do you do if you are testing an ICustomerDAO, for example, and you want to test some method 14 times each with different outcomes? Implement 14 different classes manually? I doubt that anyone would want to do that.
Mocks give you the power to define what will happen with parts of your classes when you are not concerned with whether or not they will actually work, like throwing exceptions whenever you want them to, returning zero results and making sure you handle that correctly, etc...
They are a great unit testing tool.
Previous questions that may help:
What is a mock and when should you use it?
Mockist vs classical TDD
I find that using a mocking framework allows me to generate tests a lot faster and with better verification that what I expect to happen in the test actually is happengin. I have in the past implemented stubs or fakes myself. I found that I needed to generate stubs specific to the test that I wanted and this took a lot of time. I can create the same test much faster using a mocking framework. The good ones support the generation of fakes, stubs or mocks with straightforward syntax.
It takes a while to get the hang of it, I avoided it for a while but now wouldn't try to work without a mocking framework for the reasons #Chamelaeon states.
Roy Osherove had a poll about Mock Frameworks and down in the comment section, there is a discussion (albeit brief) about whether one needs a Mock Framework or not.
I personally have been manually doing exactly as you stated and it has worked well enough, but this has mainly been out of habit rather than a closely-held opinion on mock frameworks in general.
Well I certainly don't think that you NEED a mocking framework. It's a framework like any other, and it's ultimately designed to save you some time and effort. You can also do things like roll your own common data structures like stacks and queues, but isn't it generally easier to just use the ones built into the class libraries that ship with the compiler/IDE of your language of choice?
I'm sure there are other compelling reasons for using mocking frameworks, though I'd leave it to the TDD and unit testing gurus to answer.
For the same reason you wouldn't try to write unit tests without NUnit. A mocking framework will assist you in verifying state and behavior over hundreds of unit tests. It's worth the 2 weeks or so of pain to get up to speed and really helps you focus on what needs to be tested.
One thing that troubles me about a mocking framework is that "what a function should o/p given an i/p" via
when(mock.someMethod("some arg")).thenReturn("something");
statement is spread across many unit test classes.
Let me elaborate with an example. Lets say there was a DAO Interface function getEmp(int EmpID) which was returning an Employee Object when passed an Employee ID as a parameter. Assume that this function was being mocked by 10 different unit test classes. Now if in the future, this function were changed to return a newer version of the Employee Object, one would have to go to each of the 10 different classes to update this change.
The disadvantages are as follows...
a) I don't know how to figure out all the classes which mock this function so that I can go update this change.
b) My existing test cases which consumes the mock DAO object continue to be blissfully unaware of the changes that have happened to the DAO Interface because the mock has not changed and hence continue to be green.
Ideally, if I were to have coded a single mock class myself and consumed it everywhere, I would have just one place to update for the newer version of the Employee object. Also, once I update at this one place, all my existing test cases which consume the mock would break and I would then know exactly what places I need to go and do an update for the new Employee Object.
Any thoughts on my views..
One of the good things about a mocking framework is that it allows setting expectations on the objects being mocked. With the expectations I can then set up all sorts of conditions to exercise the code thats being tested.
An isolation framework or mocking framework allows you to test the code you want, without its dependencies. It makes for short running tests, allows you to debug quickly, and easily build a safety net of tests around the code. Different frameworks have different features, and as said before - it's a tool, and you should select the right tool for the job.
I've use rhino mocks for a mocking framework. I and 5 other developers used it on a large enterprise application that was an 8 month project. We used tdd on the project. Was it worth it? I guess. Was there such a massive huge selling point to using mocks that I have to use it on every project? In my opinion, no. It is not something that is necessary, it is just a tool that you can use if you want to try it out. Some projects you can roll out your own mock classes as some here say they prefer - it is easier. Other projects are larger and may require a mocking framework. The key word (in my opinion) is MAY require... how much code coverage do you require? To me, that is another consideration to using mocks. The project I did with tdd/rhino mocks we were required to have 80% code coverage so the mocks helped us attain that. If our code coverage requirements were less, for example 40%, we probably would have not used a mocking framework and just wrote our own mock classes as others mention they do.
I am curious about how many of you folks incorporate mocking of objects (frameworks like JMock, NMock, RhinoMocks hand in hand with unit testing frameworks) into your daily development process. What are your experiences?
You see, I develop on top of a GIS (geographic information systems) platform, in which most of work pertains to some way of handling data. As its data object model is quite complex (many many classes and interfaces, all COM-based), it is also quite difficult and cumbersome to mock. In this case, mocking incurs a great deal of overhead when writing test suites. I wonder if there are people in similar situation, or just, how does mocking (in whatever situation you are in) work for you.
On a recent project that I worked on we used mock objects extensively in our unit testing approach. The project was 100% Java and moderately sized (about 100,000 lines of non-commented code). It was a Swing-based desktop application - and the only effective way that we found to test the user interface logic was through an MVC variant design that allowed us to use mock objects to substitute for the actual Swing user interface classes for the automated testing. We also used mocking extensively in the testing of our data access layer (Hibernate/DAOs).
In the user inteface use, the Mocks were easy and straightforward to build. And the design of the application (Fowler Passive View) easily incorporated mocks. This was not the case for the mocks used in testing the data access layer. But I can say that it was clearly worth the effort. In fact, most of the 'effort' really focused on coming up with a reusable solution that minimized the work that a developer had to do to create each individual mock. I'd recommend taking the time to dig into and discover an approach for your situation that allows you to easily mock up your GIS data layer. That - or just manually mock up each class. Either way the ability to run the automated unit tests that rely on the mocks is worthwhile...
In my situation mocks work really nice. But I'm using Python, which is so dynamic that it makes many things involving testing much, much easier.
In situation like yours, when application is mainly data-driven (as far as I see), mocks may not be as useful. Just passing data in and watching it come out should be enough for testing. I would just make sure that application is modularized enough, so this approach can be applied to reasonably small components.
Mocking can be useful in some kind of project. But, sometimes mocking is very time consuming and the ROI of it is low.
Trying to test Sharepoint it seems that mocking is the only way, and only typemock will let you mock sealed classes.
Mocking is used very extensively in my case. Mocks are usually for the classes that has external dependencies, e.g. network, database, filesystem. Any of these can introduce flakiness in the tests if mocks are not used.
If the mocks that you find costly to write because there are a lot of fake data to populate, you could set some pre-populated data objects as constants and use them or slightly modified copies in your test. If such data objects has external dependencies, then maybe refactor it in a way you can separate the two concerns.
There is an initiative started by Dave Bouman to try and build a community library of Mocks for use in ArcObjects related unit testing. His blog and this svn repository have great information related to unit testing GIS systems
http://blog.davebouwman.net/CategoryView,category,Unit%2BTesting.aspx
http://svn2.assembla.com/svn/arcdeveloper/TestingUtilities/trunk/
What are the best ways to test servlet oriented web applications. With possibly jdbc backed backends.
Front-end = Struts2 and some servlets
Back-end = hibernate, some basic jdbc
use mocks, easymock is one of the best mock frameworks(or you could use jmock if you like the name better). spring-mock has nice support for mocking servlet classes.
you could also do integration testing with dbunit(to tests your sqls and hqls), but its slow and take much longer to write and maintain. i think you should not do it unless you had some problems with people writing bad queries or database schema is changed a lot.
Most important thing : design for testability.
It means trying to have small independent components (which is almost always a good thing, not only for testability) and test components separately. Inversion of Control (with or without an IoC framework like Spring) can help you a lot in that area. Once you have small components, it's pretty easy to test them with jUnit, easymock or any other standard test utilities.
Testing the frontend, is the most tedious and boring task. Selenium can help you a bit, but there is no silver bullets that I know of.
Which scenarios, areas of an application/system, etc. are best suited for 'classic' state based testing versus using mock objects?
I'm going to tackle it from a TDD/BDD perspective, where the tests are driving the designs.
First off it depends what style of design you buy into, because remember this is about design first. As Martin Fowler discusses in this excellent article Mocks Aren't Stubs there are two school of thoughts and they do produce different types of designs.
If you want to buy into the mockist approach then I highly suggest you start by looking at the mockobjects site and their article Mock Roles, Not Objects. They also have a book coming out.
The truth is even if you do believe that the mock-first style of design is not for you, or that you don't want to do it right through your application (e.g. not when testing your domain/service layer) then you will still want to use test doubles. xUnit Test Patterns explains the different types of test doubles and their purposes.
Personally I never mock domain classes, so never mocking entities/value objects. However I've been trying out the mock objects style approach a lot recently, it does change my designs a bit and I find the working style comfortable. Working in this style in an MVC app I'll probably start with an automated acceptance test, then I'll write a controller test that mocks out any non-domain objects (e.g. repositories/services), then I'll move down to testing those repositories/services again mocking out their dependencies. I stop when I reach a class with no troublesome dependencies such as a domain entities/value object. I could go on and test against specific role interfaces which are then implemented by my domain classes, which is what the mockobjects guys would recommend, but I don't currently see a lot of value in that approach.
Obviously it is worth adding that design for test is important here, but remember that although 90% of IoC/mocking/DIP examples show interface-implementation pairs (ICustomerRepository/CustomerRepository) there is a lot of value in instead looking for role interfaces.
You should be using mocks for dependencies. I don't think that its an either-or; Usually you will create mocks for dependencies, set expectations (whether it is calls or state) on them, then run the unit under test. Then you would check its state, and verify the expectations on the mocks, afterwards.
Using mock objects doesnt mean you're not doing state based testing.
When using services, whether my own or third party, I design to interfaces as a matter of course, so I tend to focus on the interactions there. This encourages me to design minimal interfaces.
I check state on anything focusing on value objects, just as simple calculations, lookups, and the like.
The next time you find yourself designing something that typically follows Model/View/Controller-or-Presenter, I highly recommend trying the Presenter First approach (Google it) using interfaces for the Model and View. This will give you a great feel for how to use stubs/mocks effectively.
Its a matter of style.. Mockist vs Classic TDDers.
Personally.. I'd go testing real classes as far as possible. Tone down on Mocks as much as possible; only to decouple things like IO (filesystems, DB Connections, network), Third party components, etc. things that are slow/difficult to get under test.
As an experienced TDD'er, this is a question that I'm frequently asked by other developers. For me, it's a mistake to get sucked into a Mockist vs. Classicist debate, as such a discussion is misleading. State-based and behaviuour-based unit testing are two different tools in your toolbox, and there's no reason why they should be mutually exclusive.
State-based unit testing is suitable when you want to query the internal properties of an object after talking to its external interface. If one or more of its collaborators involve potentially expensive calls to other objects, then by all means stub those calls and simply disregard collaborators.
Behaviour-based unit testing is a good idea when you want to consider the "how" of a unit test and focus upon discoving relationships between objects, as opposed to the traditional "what" questions of a state-based unit test. If you wish to assert that collaborators are used in a certain way and/or sequential order, use mocks - stubs that provide assertions.
I suggest you focus upon standard unit testing practices - exercise as little production code as possible, and have one assertion per test. This will force you to think "what exactly is it that I want to exercise and assert in this test?", and the answer to that question will help you choose the correct unit testing tools.