Unit Testing show output - unit-testing

I searched for unit testing tool and I found the suitable one is NUnit and I think it good but my problem that this tool only show test method result only (pass or fail) and I need to show not only pass or fail also the output .How can I show the output using NUnit or if there another unit testing tool its also good ?If its not supported please suggest me how can I solve it.
All ideas are welcomed

Piping the output of System.Console will work for NUnit, but it's not your best option.
For passing tests, you shouldn't need to review the console output to verify that the tests have passed. If you are, you're doing it wrong. Tests should be automated and repeatable without human intervention. Verifying by hand doesn't scale and creates false positives.
On the other hand, having console output for failing tests is helpful, but it will only provide information that could otherwise be inferred from attaching a debugger. That's a lot of extra effort to add console logging to your application for little benefit.
Instead, make sure that your error messages are meaningful. When writing your tests make sure your assertions are explicit. Always try to use the assertion that closely fits the object you are asserting and provide a failure message that explains why the test is important.
For example:
// very bad
Assert.IsTrue( collection.Count == 23 );
The above assertion doesn't really provide much help when the test fails. As NUnit formats the output of the assertions, this assertion won't help you as it will state something like "expecting <True> but was <False>".
A more appropriate assert will provide more meaningful test failures.
// much better
Assert.AreEqual(23, collection.Count,
"There should be a minimum of 23 items by default.");
This provides a much more meaningful failure message: "Expecting <23> but was <0>: There should be a minimum of 23 items by default."

On the bottom bar of NUnit you can click Text Output and that shows all debug and console output.

It depends where are you want to output the data from a test.
I believe you mentioned something another from File, Log, Console, Debug output.
As an alternative NUnit allow to output any message in the regular tests output stream, just use following utility methods:
For successful test
Assert.Pass( string message, object[] parms );
For failed test
Assert.Fail( string message, object[] parms );
More details see here

This post is loooooong after the question was asked, but wanted to chime in. Yes, you can accomplish a lot in unit/integration tests and probably do most of what you need. So, I agree, do as much as you can in your test methods.
But sometimes, providing some output is useful. Especially if you need to further verify the results and that verification cannot be accomplished via your unit test. Think of an external system that your dev/test environment has no or limited access to.
As an example, let's say you are hitting a webapi to CREATE a claim and the response is the new claim number. But the api does not expose methods to GET a claim, and you need to verify some other data that was created when you made the webapi call. In this case, you could use the outputted claim numbers to manually check the remote system.
FWIW

Related

How to test BizTalk-Orchestrations

As the title of this post says I want to test a BizTalk-Orchestration. In past I did something like that by using BizUnit. But I was only able to test the Input, let the BizTalk-solution run and test the output. BizUnit helped me by automatizing this process.
So the question is:
How can I test every step in an Orchestration (for example the output of Transform component). How can I read the messages in the MessageBox while I'm testing that?
Does anyone know a great tutorial?
I agree with Johns-305 and would suggest that its not probably required to test every step of the Orchestration.
Still if you want to, you could put temporary Send shapes after every transformation and send them out to a FILE location so that you can understand what messages are being generated.
Also, you can make use of Orchestration Debugger if you want to understand the flow of the Orchestration.
You wont be able to see any messages that are created within the orchestration but are to published to the message box.
You can debug the orchestration in the BizTalk administrator console.
Just stop your orchestration. then in the BizTalk group you can attach debugger.
Then you can resume in debug mode
The first question you need to ask yourself is: Do I really need to do this? Hint, the answer is no.
The Unit Test fad has long past and was never a useful thing with BizTalk apps.
Instead, focus on developing, with the business owners, a set of test cases that validate the entire app. You then use these to test everything, not just Orchestrations.
During DEV, 97% of the time, testing in Visual Studio is all you need for Maps.

Is there an auto test framework that tracks passing asserts?

I'm looking for a unit test framework that tracks every assert in the code, pass or fail. I looked into Google Test, which is based on xUnit, and it only tracks failures. I need this because I work in a company that makes medical devices and we must keep evidence of validation that can be audited by the FDA. We want a test report that tells you what the test did, not just that it passed. Also, the framework would have to be usable with POSIX C++.
Ideally what I would like to have is something like this (using Google Test syntax):
EXPECT_EQ(1, x, "checking x value");
and the test would generate a report that has the following for every assert: a description, the expected value, the actual value, the comparison type, and a pass/fail status.
It looks like I'll have to create my own test framework to accomplish this. I stepped into the code of Google Test to verify it really does nothing for a passing assert. I wanted to see if there were other ideas, such as a framework that could accomplish this or be modified to accomplish this before creating my own.
Why not simply generate a json/xml/html report as part of your build process and then check that file into some kind of source control?

Can rules be used to run certain test methods

I have question related to creating test methods in my coded UI test.
Is it possible to create like rules or like an if else statement tree that would execute certain test methods to run when certain things happening or when i'm on a certain counter?.
I don't know if this is the correct way to do this. I was going to do it in a 1 huge block of code but i dont really like the direction that is going in, since the application i'm testing has to account for different paths.
I want to create and have test methods run based on these if else statements code blocks.
If anyone has done this any help would be greatly appreciated. Thanks
I think you want your tests to be as small as possible and test only one thing.
For multiple reasons you want to test in isolation and not make tests depended on each other. What you are trying todo sounds very unmaintainable on the long run.
Also when a single test steps fails, you might need to run a long test to see the actual fail, making debuging harder and slower.
I would follow the AAA pattern in your tests to guide you
Arrange: Setup the state of the application including data
Act: The action you want to test. (e.g fill in a form and submit)
Assert: Verify the action was correct by checking one thing
It is possible to group tests and run only a subset, for details see: https://msdn.microsoft.com/en-us/library/dd286683.aspx

What is the best way to test using grails using IDEA?

I am seriously having a very non-pleasant time testing using Grails. I will describe my experience, and I'd like to know if there's a better way.
The first problem I have with testing is that Grails doesn't give immediate feedback to the developer when .save() fails inside of an integration test. So let's say you have a domain class with 12 fields, and 1 of them is violating a constraint and you don't know it when you create the instance... it just doesn't save. Naturally, the test code afterward is going to fail.
This is most troublesome because the thingy under test is probably fine... and the real risk and pain is the setup code for the test itself.
So, I've tried to develop the habit of using .save(failOnError: true) to avoid this problem, but that's not something that can be easily enforced by everyone working on the project... and it's kind of bloaty. It'd be nice to turn this on for code that is running as part of a unit test automatically.
Integration Tests run slow. I cannot understand how 1 integration test that saves 1 object takes 15-20 seconds to run. With some careful test planning, I've been able to get 1000 tests talking to an actual database and doing dbunit dumps after every test to happen in about the same time! This is dumb.
It is hard to run all the unit tests and not integration tests in IDEA.
Integration tests are a massive pain. Idea actually shows a GREEN BAR when integration tests fail. The output given by grails indicates that something failed, but it doesn't say what it was. It says to look in the test reports... which forces the developer to launch up their file system to hunt the stupid html file down. What a pain.
Then once you got the html file and click to the failing test, it'll tell you a line number. Since these reports are not in the IDE, you can't just click the stack trace to go to that line of code... you gotta go back and find it yourself. ARGGH!#!#!
Maybe people put up with this, but I refuse. Testing should not be this painful. It should be fast and painless, or people won't do it.
Please help. What is the solution? Rails instead of Grails? Something else entirely? I love the Grails framework, but they never demo their testing for a reason. They have a snazzy framework, but the testing is painful.
After having used Scala for the last 1.5 months, and being totally spoiled by ScalaTest... I can't go back to this.
You can set this property in your config file:
grails.gorm.failOnError=true
That will make it a system wide default for save (which you can override with .save(failOnError: false) if you want).
If you only want this behavior in the test, you can put it in that environment specific stanza in Config.groovy. I actually like this as a project wide behavior.
I'm sure theres a way that you could turn failOnError on/off within a defined scope, but I haven't investigated how to do it yet (might be a good blog post, I'll update this if I write one).
I'm not sure what you've got misconfigured in IDEA, but it shows me a red bar when my tests fail and I can click on the lines in the stacktrace and get right to the issues. The latest version of intellij even collapses down the majority of metaclass cruft that isn't interesting when trying to fix issues.
If you haven't done this already to generate your project, I'd try wiping away your existing .ipr/.iml/.iws/.idea files and running this command to have grails regenerate your configuration:
grails integrate-with --intellij
Then run the .ipr file that gets generated.

unit tests for screen-scraping?

I'm new to unit testing so I'd like to get the opinion of some who are a little more clued-in.
I need to write some screen-scraping code shortly. The target system is a web ui where there'll be copious HTML parsing and similar volatile goodness involved. I'll never be notified of any changes by the target system (e.g. they put a redesign on their site or otherwise change functionality). So I anticipate my code breaking regularly.
So I think my real question is, how much, if any, of my unit testing should worry about or deal with the interface (the website I'm scraping) changing?
I think unit tests or not, I'm going to need to test heavily at runtime since I need to ensure the data I'm consuming is pristine. Even if I ran unit tests prior to every run, the web UI could still change between tests and runtime.
So do I focus on in-code testing and exception handling? Does that mean to draw a line in the sand and exclude this kind of testing from unit tests altogether?
Thanks
Unit testing should always be designed to have repeatable known results.
Therefore, to unit test a screen-scraper, you should be writing the test against a known set of HTML (you may use a mock object to represent this)
The sort of thing you are talking about doesn't really sound like a scenario for unit-testing to me - if you want to ensure your code runs as robustly as possible, then it is more, as you say, about in-code testing and exception handling.
I would also include some alerting code, so they system made you aware of any occasions when the HTML does not get parsed as expected.
You should try to separate your tests as much as possible. Test the data handling with low level tests that execute the actual code (i.e. not via a simulated browser).
In the simulated browser, just make sure that the right things happen when you click on buttons, when you submit forms, and when you follow links.
Never try to test whether the layout is correct.
I think the thing unit tests might be useful for here is if you have a build server they will give you an early warning the code no longer works. You can't write a unit test to prove that screenscraping will still work if the site changes its HTML (because you can't tell what they will change).
You might be able to write a unit test to check that something useful is returned from your efforts.