It is good practice to use the main method to test a java/.net class?
I've seen it reccommended in some text books, but to me it seems like using a unit testing framework would make more sense...
The main method gives you one point of entry to the class and you can test one aspect of the classes functionality. You could I guess test many but it doesn't seem to make sense as much as using Junit or Nunit.
One obvious advantage seems to be that you can whitebox test the class. That is, you can test the internals of it (private methods for example). You can't do that with unit-tests, nor would you wan't that, they are primarily there to test the interface and the behavior from the users perspective.
I think it could be useful to develop integration tests that are invoked from a main method -- like a test runner -- that tests suites of integration tests. I wouldn't do unit testing this way as unit testing frameworks provide a much better mechanism to do this.
[EDIT] To clarify, I'm not suggesting that each class have static main method to be used for integration tests, but rather that you could write an integration test program with a static main method that would run your suite of integration tests.
The main method can be useful for certain situations, but using a debugger and then writing a unit test (to provide some insurance against regressions) is a more robust solution.
In Java it's accepted to have multiple main methods and use them for testing however .NET doesn't allow this, if you have two mains in the same program you will get compiler error CS0017, and tells you to Compile with /main to specify the type that contains the entry point.
I've got to say the Java way makes more sense to me.
Related
I have a more general question:
Assuming I have a web application, for example using the Struts2 Framework.
Therefore it becomes quite complicated to write Unit tests for functions, as you have to mock every aspect of the Framework.
The Database+Connection, The Session, a LDAP-Connection or what ever else is needed, which I do not have written on my own
It would be much easier to write the unit Tests so, that they run in a WebInterface inside the Base-Application, as all these things then already would exist.
The question:
Would you guys still consider this as unit testing?
Some thoughts..
The question is very general. My suggestion is that you still want to write some sort of Unit Tests for number of reasons. Firstly you can run them as an automated test suite so if something breaks you know quickly. Secondly you get a better designed system - Your objects are loosely coupled. You get more confident on the code you write.
If you have a framework harder to test,
a. Try abstracting away some dependencies, so they code can be injected without interfering with real instances.
b. Use a testing framework that can break any tightly coupled harder dependencies.
Harder to provide a comprehensive answer, but this is the general direction, which I would suggest.
You should consider what you really want to test first. A framework, for its definition, will use the classes you provide to do some "magic". Do you want to test that has already been tested "magic" or the business core of the app you programmed?.
Also, something you should consider, is where to stop testing. You probably don't want to test the connection to the database (considering what you wrote) so just mock it.
Take in consideration that you will have to test just one functionality at the time, don't think of having, for example, the database connection and the ldap in the same test, it wouldn't be unit testing.
Take a look at this tutorial also :http://tutorials.jenkov.com/java-unit-testing/index.html
I'm moderately new to test frameworks and I have been reading up on Moq and this introductory post used a way of organising tests that I had not seen before. Further research shows that we in the .Net world tend to meld the terms BDD and Context/Specification (CS) Testing. I don't want to get into that argument - I am primarily interested in achieving this style of writing test classes.
This article shows the approach again and makes explicit the use of a base class that allows us to construct our specification through the test framework.
This issue I have is that I cannot see an instance method under MSTest that would allow a test fixture to be initialised just once for each test. The best I can see is the constructor of the test class but that feels a bit wrong. In NUnit one could use [TestFixtureSetup]. Is there an equivalent using Visual Studio's built in test framework?
Edit
I've subsequently moved to NUnit which provides the flexibility I require.
After some investigation I have moved to using NUnit. There are several reasons for preferring NUnit over MSTest but, with regards to this particular issue, the compelling reason is that the only class-wide initialisation method supported by MSTest (that doesn't run for every test) is for static methods which is not what I'm looking for.
As Jason points out [ClassInitialise] will provide this static, class wide initialisation in MSTest. There isn't really a way of mimicking the behavior of [TextFixtureSetup] found in NUnit such that an instance method is run once before any test in the fixture is run.
To replicate NUnit's [TestFixtureSetUp] for MSTest, try [ClassInitialize]. Here's the link where I found that information. This is another useful blog article that might help you.
If someone has created a small diagnostic, internal web app that uses (unit) tests as its logic, is there ever a valid reason to do that? Keep in mind that Nunit has to be deployed as well where ever this website goes.
I'm of the view that programs should contain their own logic and possibly reusable parts (if available) but not wrap tests for their logic. Tests serve the purpose of validation for code logic. If you than say tests are going to be code logic, shouldn't you need to write tests to validate the tests? Why is that fundamentally wrong?
Hint: Because now you are stringing all of these tests together and interrelating them, which means they are no longer dependent(?).
Using a Unit test framework for something else than unit tests is usually not the most proper path. You don't have to write tests for your unit tests, since you write them first and see them fail. That's how you know they're working properly. I'm guessing testing code written within a unit test framework is nontrivial, and if I had a diagnostics app for a critical piece of software, I would really like to be certain it worked as it should.
Edit: It seems that you've already made up your mind but need support in expressing why the current strategy is less than ideal to, perhaps, other project members. If that's the case, I suggest you put your code where your mouth is, and throw together a small sample app designed differently. If utilizing a unit test framework in this specific case was a bad design decision, then that would make it clear as sunshine.
I'm pretty sure if you look at this question TDD Anti-patters catalogue, you will find that you are commiting sever anti-patterns.
Code is code. Just because it's labeled a test doesn't mean it's not also a useful application.
Suppose we need to write a lot of verifications of beahviour. Why is using test framework a bad idea? SHoudl we instead write a new framework with identical function and call it something different?
Take an external view. This applciation claims to do certain things. Does it do them correctly? reliably? Can it be maintained, enhanced? Understood?
If so, why do you care that it happens to use a test framework in its implementation. If its behaviour or structure are defective then we criticise.
One lovely thing about great technology is that it has unexpected applications.
The purpose of unit testing is to make sure your class works the way it should and according to the interface. So if you are using your unit tests for some testing app it seems like using assert in programm logic.
If you need some testing app - implement it and use WITH unit tests. It maybe to configure something or to get some user interaction.
One of the other reasons I see - unit test are written according to assumptions on how everything works. If your assumption is right tests should pass. When adding functionality unit tests let you feel confident that all assumptions are still there. So every test should be kept as simple as you can keep it. that`s why no need to test test code and no need to use testing code in any testing applications.
I have an existing VS 2005 Std .NET Compact Framework application that I want to do some major refactorings on. Currently there is no unit testing in place, but I want to add this before messing with the code. I have no practical experience with unit testing, even though I know the theory (just never got around actually implementing it; I know: shame on me :-))
Here are some questions I am pondering at the moment:
a) As a beginner, should I use NUnit or NUnitLite (which claims to be easier to use)?
b) Should I aim for running the tests on the mobile device or on the desktop (except for device-specific code of course)? Currently the desktop looks more appealing, especially for including the tests in automated builds...
c) How is the class that I want to test usually included in the test project? My application is a .EXE file, i.e. I can not just reference it like a .DLL assembly from the test project (or can I? Never tried this ...). I checked various NUnit tutorials but either found no mention of that, or one tutorial suggested to copy and paste the class that I want to test into the test project (yuk!). Should I link to the original source code file in my test project? What about private methods, or dependencies on other classes?
d) Should I start modifying my original code to allow better testability, e.g. make private methods public, decouple etc.? This is a bit like refactoring before being able to test, which does not sound good to me ... Or is it better practice to not touch the original code at all in the beginning, even if this means less code coverage etc.?
e) Should I look into any other tools or addons that most people use?
Thanks in advance for any answers (I also appreciate answers if they are only to one or some of the above items).
First, I would recommend you a good book on unit testing: Pragmatic Unit Testing in C#.
It will introduce you to NUnit, but what's more important, the author will provide you a lot of advices how to write good unit tests in general. The xUnit test frameworks are not very complex and you'll get used to their API/workflow very quick. Challenging is the actual process of identifying boundary conditions, decrease coupling and design for testability. It's available as an eBook (PDF) or a printed copy.
Regarding your actual questions (the book will give you some answers, too):
#a) I've no experience with NUnit lite, thus I cannot give you any advice on this point.
#b) Unit tests are supposed to be very local in regard of their dependencies. You aim to test classes independent of each other, thus there would be no need to deploy it on a mobile device first. You won't run the full app, just test components in isolation. Hence, I would recommend to use your desktop machine as the target for your unit test environment. You'll get better turn-around times, too.
#c) You have to reference the assembly that contains the classes you want to test in your test project. The test project will be an assembly itself (DLL). A test runner executes this assembly and uses the stored meta information to run the contained test cases.
#d) It depends a lot on the state and design of your software. But in general I would use a divide and conquer strategy: Introduce interfaces between classes and start to refactor step by step. Write unit tests before you start to change the implementation. The interfaces keep the contracts up and running, but you can change the underlying implementation if necessary. Don't make private methods public just to make them testable. Private methods are internal helpers of the class that support public methods in doing their job. Since you test your public methods you'll assert that your private methods do the right thing.
#e) A helpful add in for Visual Studio is TestDriven.Net. It allows you to run NUnit tests directly from the IDE without changing to NUnit's GUI or console runner.
#c) I haven't tried it, but I would think VisualStudio would let you add a project reference from your test assembly to your actual code assembly, even if its an exe.
As for private methods and such, generally I don't test private methods. In theory, all the private stuff should be used by public or internal methods, so testing those should indirectly test the privates.
I do test public and internals though. One thing I find very helpful is the InternalsVisibleTo attribute:
[assembly:InternalsVisibleTo("MyTestAssembly")]
which you can use to make the internals of one assembly visible to another. I use this attribute to expose the internals to my test assembly, so that it can directly reference them.
Recently there has been quite some hype around all the different mocking frameworks in the .NET world. I still haven't quite grasped what is so great about them. It doesn't seem to be to hard to write the mocking objects I need myself. Especially with the help of Visual Studio I quickly can write a class that implements the interface I want to mock (it auto-generates almost everything for me) and then write an implementation for the method(s) I need for my test. Done! Why going through the hassle of understanding a mocking framework for the sole purpose of saving a few lines of code. Or is a mocking framework not only about saving lines of code?
Once I finally got the hang of mock objects, I realized that they're essential for unit testing for the same reason that double blind testing or control groups are essential for scientific trials: they isolate what you're actually testing.
If you're testing a class which has quite a bit of interaction via other interfaces, you not only save the lines of code on having to mock each and every interface, but you also gain the ability to do things like "throw an exception if an unexpected method is called" or "exception if these methods are called out of order". You can get remarkably sophisticated with mock frameworks, and though I'll quickly admit there's a large learning curve, when you get up to speed they'll help make your unit tests more thorough without being bloated.
You actually identified one of the key points of a mock framework in your question. The fact that you code the mocks yourself is not something the developer should be concerned with. The mocking frameworks give you implementations of interfaces programatically, plus they are functional (based on your setup of the mock).
What do you do if you are testing an ICustomerDAO, for example, and you want to test some method 14 times each with different outcomes? Implement 14 different classes manually? I doubt that anyone would want to do that.
Mocks give you the power to define what will happen with parts of your classes when you are not concerned with whether or not they will actually work, like throwing exceptions whenever you want them to, returning zero results and making sure you handle that correctly, etc...
They are a great unit testing tool.
Previous questions that may help:
What is a mock and when should you use it?
Mockist vs classical TDD
I find that using a mocking framework allows me to generate tests a lot faster and with better verification that what I expect to happen in the test actually is happengin. I have in the past implemented stubs or fakes myself. I found that I needed to generate stubs specific to the test that I wanted and this took a lot of time. I can create the same test much faster using a mocking framework. The good ones support the generation of fakes, stubs or mocks with straightforward syntax.
It takes a while to get the hang of it, I avoided it for a while but now wouldn't try to work without a mocking framework for the reasons #Chamelaeon states.
Roy Osherove had a poll about Mock Frameworks and down in the comment section, there is a discussion (albeit brief) about whether one needs a Mock Framework or not.
I personally have been manually doing exactly as you stated and it has worked well enough, but this has mainly been out of habit rather than a closely-held opinion on mock frameworks in general.
Well I certainly don't think that you NEED a mocking framework. It's a framework like any other, and it's ultimately designed to save you some time and effort. You can also do things like roll your own common data structures like stacks and queues, but isn't it generally easier to just use the ones built into the class libraries that ship with the compiler/IDE of your language of choice?
I'm sure there are other compelling reasons for using mocking frameworks, though I'd leave it to the TDD and unit testing gurus to answer.
For the same reason you wouldn't try to write unit tests without NUnit. A mocking framework will assist you in verifying state and behavior over hundreds of unit tests. It's worth the 2 weeks or so of pain to get up to speed and really helps you focus on what needs to be tested.
One thing that troubles me about a mocking framework is that "what a function should o/p given an i/p" via
when(mock.someMethod("some arg")).thenReturn("something");
statement is spread across many unit test classes.
Let me elaborate with an example. Lets say there was a DAO Interface function getEmp(int EmpID) which was returning an Employee Object when passed an Employee ID as a parameter. Assume that this function was being mocked by 10 different unit test classes. Now if in the future, this function were changed to return a newer version of the Employee Object, one would have to go to each of the 10 different classes to update this change.
The disadvantages are as follows...
a) I don't know how to figure out all the classes which mock this function so that I can go update this change.
b) My existing test cases which consumes the mock DAO object continue to be blissfully unaware of the changes that have happened to the DAO Interface because the mock has not changed and hence continue to be green.
Ideally, if I were to have coded a single mock class myself and consumed it everywhere, I would have just one place to update for the newer version of the Employee object. Also, once I update at this one place, all my existing test cases which consume the mock would break and I would then know exactly what places I need to go and do an update for the new Employee Object.
Any thoughts on my views..
One of the good things about a mocking framework is that it allows setting expectations on the objects being mocked. With the expectations I can then set up all sorts of conditions to exercise the code thats being tested.
An isolation framework or mocking framework allows you to test the code you want, without its dependencies. It makes for short running tests, allows you to debug quickly, and easily build a safety net of tests around the code. Different frameworks have different features, and as said before - it's a tool, and you should select the right tool for the job.
I've use rhino mocks for a mocking framework. I and 5 other developers used it on a large enterprise application that was an 8 month project. We used tdd on the project. Was it worth it? I guess. Was there such a massive huge selling point to using mocks that I have to use it on every project? In my opinion, no. It is not something that is necessary, it is just a tool that you can use if you want to try it out. Some projects you can roll out your own mock classes as some here say they prefer - it is easier. Other projects are larger and may require a mocking framework. The key word (in my opinion) is MAY require... how much code coverage do you require? To me, that is another consideration to using mocks. The project I did with tdd/rhino mocks we were required to have 80% code coverage so the mocks helped us attain that. If our code coverage requirements were less, for example 40%, we probably would have not used a mocking framework and just wrote our own mock classes as others mention they do.