I'm working on a variation of this stack overflow answer that provides reliable cleanup of tests. How do you write unit tests for NUnit addins?
Examining how NUnit self tests, I have determined:
You can write tests, that pass, that verify correct behavior of NUnit for failing tests.
You write unit tests against test fixtures in a separate assembly (otherwise the fixtures under test will execute with your unit tests)
Use NUnit.TestUtilities.TestBuilder to create fixtures and call TestSuite.Run method.
What I don't see are any tests of the add-in process. I've got errors occurring sometime between install and execution. How would I unit test implementations the following?
IAddin.Install
ITestDecorator.Decorate
Here's an article by someone who hacked a way to do it: manipulating some of the singletons in the NUnit add-in implementation to swap his add-in in and out.
http://www.bryancook.net/2009/09/testing-nunit-addins-from-within-nunit.html
Sometimes, the easiest thing to do is run integration tests. It been a while since I played with the NUnit add-in API, so I can't really say regarding any existing unit tests for the extensibility mechanism. If you have looked through NUnit source code and haven't found any, then I guess that is not something that was tested or even written using TDD.
Like I said, sometimes it's easier to just run integration tests. Have your addon, for example, print something to the output stream, and have your test verify that the exact message was written. This way you could test that both the installation and initialization of your plugin succeeded.
Hope that helps...
Related
We use SWTBot for writing of functional tests. To test some cases is very difficult and some programmers use classes and their methods directly from implementation (for example call methods from class AddUserDialog etc.). Is this good approach? And why?
And next qustion please. It is SWTBot enough for testing of eclipse RCP based application? Is is necessary to write unit tests please?
Note: We are scrum team.
SWTBot and JUnit serve two different purposes.
JUnit
As the name implies, JUnit is meant for unit testing. Unit tests should be small and fast to execute. They test only a single unit of code and the above mentioned attributes allow them to be executed often while developing the unit under test.
But there is more to (good) units tests. You may want to read one of the following posts for further attributes of unit tests:
Key qualities of a good unit test
What attribute should a good Unit-Test have?
I would go one step further and say that unit tests only make sense in TDD, that is you write the test before the production code. Otherwise you neglect the tests. Who want's to do the extra effort of writing tests for something that already works. And even if you have the discipline to write the tests afterwards, they merely manifest the state of your production code. Whereas, like in TDD, writing tests beforehand leads to lean production code that only does what is required by the tests.
But I guess that's something not everyone will agree on.
In an RCP setting, unit tests would ideally be able to run without starting the platform (which takes rather long). I.e. they would not require to be run as PDE JUnit Tests but as plain JUnit Tests instead. Therefore the unit under test should be isolated from the RCP APIs.
On a related note, see also this question: How to efficiently JUnit test Eclipse RCP Plugins
SWTBot
While SWTBot uses the JUnit runtime to execute the tests, it is rather meant as a utility to create integration or functional tests. SWTBot, when used with RCP, starts the entire workbench and runs all tests within the same instance. Therefore great care should be taken to ensure that each test leaves the environment in the same state as it was before the test started. Specialized Rules may help here to set up and tear down a particular recurring scenario.
It is perfectly valid in order to setup an SWTBot test to call methods from your application. For example, you could programmatically open the wizard and then use SWTBot to simulate a user that enters data and presses the OK button. There is no need to use SWTBot to laboriously open the wizard itself.
In my experience, SWTBot is even too much for simple use cases. Consider a test that should enter some data into a dialog and then press OK. If you already have the dialog opened programmatically you can as well continue without SWTBot:
dialog.textField.setText( "data" );
dialog.okButton.notifyListeners( SWT.Selection, null );
assertThat( dialog.getEnteredData() ).isEqualTo( "data" );
Use Both
The best bet is to have both, unit tests that ensure the behavior of the respective units and functional tests that make sure that the particular units play together as desired.
Not sure if that answers the question, if you have further concerns please leave a comment.
I'm writing unit tests using NUnit.Framework for a project I have up live. I have read few things about Ncover and it looks like an alternative of NUnit. Could someone please explain what it really is and what it does or post a link where I could find my answer.
NCover is a code-coverage tool that allows you to assess how much of your source code is "covered" by unit tests. The idea is that while you're running your unit tests with nUnit, nCover will be operating in the background, keeping track of each line of code that's being executed. After the tests are finished, nCover will show a visual report showing exactly which parts of your source code are and are not covered by existing unit tests.
It's a tool to help you determine if you have any functionality that isn't assessed by unit tests. While it's not perfect, it does give a good visual representation if you're missing out on some potential tests.
I am working on a project in where I decided to use unit tests. This was new to me but after researching I feel pretty confident I am doing it correctly creating mock objects and testing that the correct methods are called. This is working great but now I would like to actually run some tests that use the actual database and external components. How should I go about testing the actual execution of code? I do not want these tests to run when I run all tests. Is there a way to accomplish this using built in testing in vs2012?
Not that I'm aware of.
What you could do is create a separate project for your integration tests and then in the test explorer, separate your tests by class. Or some other logical seperation.
the current implementation that exists seems to specifically have unit testing mainly in mind, but that should change with 2013 and a stronger push towards TDD and Agile development.
Have a look here :http://msdn.microsoft.com/en-us/library/ms243147(v=vs.80).aspx
Scroll down to "Attributes for Identifying and Sorting Tests". There might be something there that's useful that you can use.
I am new to testing in general and am working on a Grails application.
I want to write a test that says "when this action is called, the correct view is returned". I don't know how to go about deciding if I should make something like this a unit test or an integration test. Either test would show me what I want - how do I decide?
One problem with integration tests is their speed. For me, integration tests take 15+ seconds to start up. In that time, certain things do slip out of mind focus.
I prefer to go with unit tests that start in no more then 2 sec and can be run several times in those 15 seconds. Especially with mockDomain(). Especially with Grails 2.0 implementing criteria and named queries in unit tests.
One more argument for unit tests is they force you to decouple your code. Integration tests always tempt you to just rely on some other component existing and initialized.
From Grails Docs section 9.1
Unit testing are tests at the "unit" level. In other words you are
testing individual methods or blocks of code without considering for
surrounding infrastructure. In Grails you need to be particularity
aware of the difference between unit and integration tests because in
unit tests Grails does not inject any of the dynamic methods present
during integration tests and at runtime.
From Grails Docs section 9.2
Integration tests differ from unit tests in that you have full access
to the Grails environment within the test. Grails will use an
in-memory HSQLDB database for integration tests and clear out all the
data from the database in between each test.
What this means is that a unit test is completely isolated from the Grails environment whereas an integration test is not. According to Scott Davis, author of this article, it is acceptable to write only integration tests...
Unit vs. integration tests
As I mentioned earlier, Grails supports two basic types of tests: unit
and integration. There's no syntactical difference between the two —
both are written as a GroovyTestCase using the same assertions. The
difference is the semantics. A unit test is meant to test the class in
isolation, whereas the integration test allows you to test the class
in a full, running environment.
Quite frankly, if you want to write all of your Grails tests as
integration tests, that's just fine with me. All of the Grails
create-* commands generate corresponding integration tests, so most
folks simply use what is already there. As you'll see in just a
moment, most of the things you want to test require the full
environment to be up and running anyway, so integration tests are a
pretty good default. If you have noncore Grails classes that you'd
like to test, unit tests are perfectly fine.
First go through this chapter of the grails guide http://grails.org/doc/latest/guide/9.%20Testing.html
It talks about testing controllers and ability to get controller response like so :
controller.response.contentAsString
Now deciding on which test is more of an art rather than science. I prefer unit tests cause they are faster to run :)
Its a really interesting and challenging question to answer, but the truth is it really depends on what exactly you are testing.
Take the following test: "saving a book to the database". The hints are in the description. We are saying we need a book and we need a database, so in this case a unit test wont do because we need the integrated database.
My advice is write the full test description down and break it down like I did above. It will give you the hints to help you decide.
This is made easier with spock where you can use strings for test names.
I have been doing a lot a reading about unit testing.
Unit testing seems all well and good.
But it seems to miss a lot of the fundamentals of how the web works. User Interaction.
I have not seen any way a unit test could test for unexpected input, or test to make sure that an ajax call works etc.
Am I missing something here or is unit testing not really designed well for web development?
You are not missing anything.
Ideally unit testing is about testing a small piece of code, e.g. a class in isolation. For this you may want to use a unit testing tools such as JUnit or NUnit. Some people refer to this type of tests as developer tests.
In contrast to that you may want to test web applications as a whole. Some call this acceptance testing. For the latter you could use a tool such as Selenium. Tools like Selenium can test Ajax and other JavaScript as well.
You have even more options if you take a look at a tool like WebDriver as you will find that you can implement Selenium-based tests using a unit testing tool.
Take a look at Selenium.