I'm building unit testing using Pex. my problem is not all code branches are being tested, Pex keep generating parameter values that fails the same condition which make all the code after that condition not to run.
my method goes something like this:
public void SetUp(DbSyncScopeDescription SyncScopeDesc, BasicInfo info, string dbContext = "MyDBContext")
{
// <pex>
Contracts validation
// </pex>
string localDbConnStr = string.Empty;
//this condition never get a parameter that results in true
if (IsContextExist(dbContext))
{
localDbConnStr = ConfigurationManager.ConnectionStrings[dbContext + "Context"].ConnectionString;
}
else
{
throw new MissingFieldException("dbcontext does not exist");
}
// This part is never being reached
ProvisionLocalScope(SyncScopeDesc, info.FarmId, localDbConnStr);
info.Tables = GetSyncTablesAsSyncTableInfo(SyncScopeDesc);
AdminOrm.Create(info.ToORM(), String.Format("name={0}AdminEntities", dbContext));
}
I wonder if its possible to tell Pex to pass that test so all the code will be reached.
if this is not possible is it possible to make Pex take the default values of the function parameters for one of the tests (i think this will be a good feature if its not present).
Thank you
Related
Before starting, I have used Moq to mock things in unit tests for years. This should be a simple mock verify, but for whatever reason moq is not matching the invocation on the Mock when it occurs. I've manually tested, it is hit. I've debugged the test and compared actual vs. expected values (they match), I've scoured SO and its multitudes of people doing blatantly wrong things and I cannot figure out why this isn't working. Help appreciated.
The unit test is a very simple test of checking if BulkInsert is called at the end of a void returning function. Sample code:
Code:
public interface IDependencyService
{
void BulkInsert(IList<T> items);
}
public class MyServiceClass
{
private readonly IDependencyService _service;
/* ctor and all that jazz */
public void Run()
{
/* do things to the data */
_service.BulkInsert(items); // where items is an IList<T>
}
}
Test:
public class ServiceTests
{
[Fact]
public void ServiceRun_Calls_DependencyBulkInsert()
{
var dependencyMock = new Mock<IDependencyService>();
List<T> expected = /* somehow build expected values */
dependencyMock
.Setup(mock => mock.BulkInsert(It.IsAny<IList<T>>()));
var sut = new MyServiceClass(dependencyMock.Object);
sut.Run();
dependencyMock.Verify(mock => mock.BulkInsert(expected), Times.Once());
}
}
Error message:
Expected invocation on the mock once, but was 0 times: mock => mock.BulkInsert([ThresholdCheck])
Performed invocations:
Mock<IThresholdCheckHandler:1> (mock):
IThresholdCheckHandler.GetQueuedChecks()
IThresholdCheckHandler.BulkInsert([ThresholdCheck])
If I change expected to It.IsAny<T>() in the Verify call, test passes. This leads me to believe that maybe somehow the objects passed from expected are somehow different from the objects generated when running the program. However as stated I've went through with the debugger and manually compared every value in the actual list to the expected list of values in the test and they are exactly the same.
This then leads me to believe that I'm just a stupid bipedal monkey clicking at a keyboard and that the problem is right in front of me and I'm just not seeing it. Any help or set of eyes is appreciated.
As per NKosi's comment, Verify operates via. reference when using reference types. I was confused as I'd used Verify in the past for simple things like ints and strings but was unaware of that quirk. One would assume with Verify that it would check the equivalency of actual vs. expected, but no.
Regardless then, as per Quercus's comment I adjusted my test to this in order to continue with my day:
public class ServiceTests
{
[Fact]
public void ServiceRun_Calls_DependencyBulkInsert()
{
var dependencyMock = new Mock<IDependencyService>();
List<T> actual = new List<T>();
List<T> expected = /* somehow build expected values */
dependencyMock
.Setup(mock => mock.BulkInsert(It.IsAny<IList<T>>()))
.Callback<List<T>>(l => actual = l);
var sut = new MyServiceClass(dependencyMock.Object);
sut.Run();
actual.Should().BeEquivalentTo(expected);
}
}
and this solution works for me. thanks to you both for helping me realize my mistake.
CGrunddaten m_grdDaten;
[SetUp]
public void Init()
{
m_grdDaten = new CGrunddaten();
m_grdDaten.m_cwdGeoH.m_dW = 325.0;
m_grdDaten.m_cwd_tl.m_dW = 15;
}
[Test]
public void TestMethod()
{
m_grdDaten.RechGrdDaten();
Assert.That(m_grdDaten.m_cwd_pl.m_dW, Is.EqualTo(93344).Within(.1),"Außenluftdruck");
Assert.That(m_grdDaten.m_cwd_pl_neb.m_dW, Is.EqualTo(93147.3).Within(.1), "Außenluftdruck Nebenluftberechnung");
Assert.That(m_grdDaten.m_cwd_pl_pmax.m_dW, Is.EqualTo(92928.2).Within(.1), "Außenluftdruck max. zul. Unterdruck");
Assert.That(m_grdDaten.m_cwdRho_l.m_dW, Is.EqualTo(1.124).Within(.001), "Dichte Außenluft");
Assert.That(m_grdDaten.m_cwdRho_l_neb.m_dW, Is.EqualTo(1.184).Within(.001), "Dichte Außenluft Nebenluftberechnung");
Assert.That(m_grdDaten.m_cwdRho_l_pmax.m_dW, Is.EqualTo(1.249).Within(.001), "Dichte Außenluft max. zul. Unterdruck");
}
Is there a way to get this in a TestCase or TestCaseSource, so that I have only one Assert-line ?
I'm talking about this:
m_grdDaten.m_cwd_pl.m_dW, 93344
m_grdDaten.m_cwd_pl_neb.m_dW, 93147.3
m_grdDaten.m_cwd_pl_pmax.m_dW, 92928.2
....
I know that TestCase and TestCaseSource are static.... but is there another way?
The best way to do this test would be using the not-yet-implemented multiple asserts feature, so that all the asserts would run even if some failed.
Since that's not available yet, I can understand your wanting to make this into multiple tests, where each gets reported separately. Using test cases makes this possible, of course, even though this is really logically just one test.
The fact that a test case source method must be static doesn't prevent it from creating an instance of your CGrunddaten class. The tests themselves are all just comparing two doubles for equality and don't need to know anything about that class.
You could write something like this:
private static IEnumerable<TestCaseData> GrundDatenDaten
{
var gd = new CGrunddaten();
gd.m_cwdGeoH.m_dW = 325.0;
gd.m_cwd_tl.m_dW = 15;
gd.RechGrdDaten();
yield return new TestCaseData(gd.m_cwd_pl.m_dW, 93344, .1, "Außenluftdruck");
// und so weiter
}
[TestCaseSource("GrundDatenDaten")]
public void testMethod(object actual, object expected, object tolerance, string label)
{
Assert.That(actual, Is.EqualTo(expected).Within(tolerance), label);
}
However, I don't like that very much as it hides the true function of the test in the data source. I think your original formulation is the best way to do it for now and leaves you with the ability to include the code in an Assert.Multiple block once that feature is implemented.
I'm just setting up some first unit tests, and I can't quite see how I'm trying to achieve (with my current test structure) can be done, which means I'm not sure whether my approach to the tests is incorrect, or it's just a limitation on xUnit.
I'm testing my MVC Controllers, and want to ensure that they all provide a ArgumentNullException if they are constructed passing null across as the arguments (they get resolved via Castle in the real world).
So, I've a private field on the Test class:
private IEnumerable<Type> ControllerTypes = typeof(MyBaseController).Assembly.GetTypes().Where(t => IsController(t));
Then, my test method:
[Fact]
public void EnsureControllersThrowIfProvidedWithNull() {
foreach (var controller in ControllerTypes) {
var ctrs = GetConstructorsForType(controller);
if (null == ctrs || !ctrs.Any()) { //if the controller has no constructors, that's fine, we just skip over it
continue;
}
var ctr = ctrs.ElementAt(0);
var ctrParamsAsNull = ctr.GetParameters().Select(p => (object)null);
Assert.Throws<ArgumentNullException>(() => {
ctr.Invoke(ctrParamsAsNull.ToArray());
});
}
}
So this is all working fine, I run the test runner, and one of my Controllers doesn't throw an ArgumentNullException when passed null, great, my test fails, but I don't know which controller it was, from the given output.
I do know how I can debug through the test to see which it is that fails, and can manually go through all my controllers to check which it is, but it would be useful to know which controller it was that failed.
Or am I just using a unit test wrong here?
(Side note, there's another test which ensures there's only 1 public constructor for each controller, so I can be sure I'm targeting the correct constructor when this fires, as long as that first test passed).
Thanks
Note:
There's a flaw in the logic for the test, which means it doesn't fully cover what I was expecting it too, as long as it throws an ArgumentNullException for at least 1 of the arguments, then it will pass the test, which isn't right. However as the arguments are interfaces I can't instantiate a new instance of them. So anyone looking to copy the code for the test, I wouldn't do so. Not looking for a solution to that issue here.
Assert.Throws is only helper method that executes delegate inside try catch block. You don't have to use it and you can replace it with your own implementation. Something like:
[Fact]
public void EnsureControllersThrowIfProvidedWithNull() {
foreach (var controller in ControllerTypes) {
var ctrs = GetConstructorsForType(controller);
if (null == ctrs || !ctrs.Any()) { //if the controller has no constructors, that's fine, we just skip over it
continue;
}
var ctr = ctrs.ElementAt(0);
var ctrParamsAsNull = ctr.GetParameters().Select(p => (object)null);
book ok = false;
try
{
ctr.Invoke(ctrParamsAsNull.ToArray());
}
catch(ArgumentNullException)
{
//you get exception you expected so continue
ok = true;
}
if(!ok)
{
// you didn't get exception so throw your own exception with message that contains controller type name
throw new Exception(String.Format("Ctor on type {0} did not throw ArgumentNullException",controller.Name);
}
}
}
This is only as idea to work on. You can refactor that inside your own static assertion method...
I am still trying to get the hang of unit testing, I have a simple question. Today I wanted to write a test for a very simple function. This function was doing just this:
void OnSomething()
{
increment++;
if (increment == 20)
SaveIt();
}
I said, this function could be testable. I could write a test that calls it 20 times and then verifies that SaveIt has been called.
Then my doubt arose. How can I test that SaveIt has been called? My first answer was to add a boolean, but then I thought: is it correct to add class features just to make it testable?
Please advise. Thank you.
I would suggest having SaveIt return a success or failure result, this just makes it easier to test overall. You could do something as simple as having it return a bool, or you could create a generic result class that contains the ability to set messages as well, if you ever need to report whether it passed or failed.
A simple example example
public class Result
{
public bool IsSuccess;
public List<string> Messages;
}
In the unit test you're trying to test only the OnSomething behavior though -- what happens inside "SaveIt" should not be tested. So ideally you'd want SaveIt() to occur in another class so you can mock its response.
I use Moq for this purpose. Moq is free, you can get it here: http://code.google.com/p/moq/
my method would then become
Result OnSomething()
{
Result result=null;
increment++;
if(increment == 20)
{
result = saver.SaveIt();
}
return result;
}
Your class constructor would take an object that implements ISaver interface (defining SaveIt() method) (ideally injected by a DI framework but you could generate it manually if you had to).
Now in your unit test you would create a mock version of ISaver and tell it what to return when it gets called:
Mock<ISaver> mock = new Mock<ISaver>();
mock.Setup(x=> x.SaveIt()).Returns(new Result{IsSuccess=true});
You'd instantiate your class passing mock.Object in the constructor ISaver parameter.
ex.
MyClass myClass = new MyClass(mock.Object);
//(assuming it didn't have other parameters)
Then, you could Assert whether result is null or not -- if it never got called, it would be null because the setup you did above would never trigger.
(in nunit)
Result result = myClass.OnSomething();
Assert.IsNotNull(result);
If you really didn't want OnSomething() to return a result, or it couldn't because it's an event, then I would have OnSomething() call a method to do the work for you:
void OnSomething()
{
Result result = DoTheWork();
}
Result DoTheWork()
{
Result result=null;
increment++;
if(increment == 20)
{
result = saver.SaveIt();
}
return result;
}
And then run your unit test on DoTheWork() instead of OnSomething().
Definitely not! Production code should not depend on tests at all, but the tests should verify the correct behaviour of the actual code. This can be achieved by several methods, such as IOC, and using mocks. You can take a look at some existing frameworks which simplify your life a lot:
http://code.google.com/p/mockito/
http://code.google.com/p/jmockit/
http://www.easymock.org/
I have a DTO which I'm populating from the request object, and the request object has many fields. I want to write a test to check if the populateDTO() method is putting values in the right places or not. If I follow the rule of one assert per test, I would have to write a large number of tests, to test each field. The other approach would be to write multiple asserts in a single test. Is it really recommended to follow one assert per test rule or can we relax in these cases. How do I approach this problem?
Keep them separate. A unit test is supposed to tell you which unit failed. Keeping them separate also allows you to isolate the problem quickly w/o requiring you to go through a lengthy debug cycle.
Is it really recommended to have only one assert per unit test? Yes it is, there are people who make that recommendation. Are they right? I don't think so. I find it hard to believe such people have actually worked on real code for a long time.
So, imangine you have a mutator method you want to unit test. The mutator has some kind of effect, or effects, which you want to check. Typically the expected effect of a mutator are few in number, because many effects suggests an overly complicated design for the mutator. With one assert per effect and one test case per assert, you will not need many test cases per mutator, so the recommendation does not seem so bad.
But the flaw in this reasoning is that those tests are looking at only the expected effects of the mutator. But if the mutator has a bug in it, it might have unexpected faulty side effects. The tests are making the foolish assumption that the code does not have a whole class of bugs, and that no future refactoring will introduce such bugs. When the method was originally written it might be obvious to the author that particular side effects were impossible, but refactoring and addition of new functionality might make such side effects possible.
The only safe way to test long lived code is to check that the mutators do not have unexpected side effects. But how can you test for those? Most classes have some invariants: things that no mutator can ever change. The size method of a container will never return a negative value, for example. Each invariant is, in effect, a post condition for every mutator (and also the constructor). Each mutator also typically has a set of invariants that describe what kind of changes it does not make. A sort method does not change the length of the container, for example. The class and mutator invariants are, in effect, post conditions for every mutator call. Adding assertions for all them is the only way of checking for unexpected side effects.
So, just add more test cases? In practice the number of invariants multiplied by the number of mutators to test is large, so one assertion per test leads to many test cases. And the information about your invariants is scattered over many test cases. A design change to tweak one invariant will require alteration of many test cases. It becomes impractical. Its better to have parameterised test cases for a mutator, which check several invariants for the mutator, using several assertions.
And the authors of JUnit5 seem to agree. They provide an assertAll for checking several assertions in one test-case.
this construction help you to have 1 big assert (with small asserts inside)
import static org.junit.jupiter.api.Assertions.assertAll;
assertAll(
() -> assertThat(actual1, is(expected)),
() -> assertThat(actual2, is(expected))
);
You can have a parameterized test where the 1st parameter is the propertyname and the second the expected value.
Is that rule extended to being in a loop? Consider this
Collection expectedValues = // populate expected values
populateDTO();
for(DTO dto : myDtoContainer)
assert_equal(dto, expectedValues.get(someIndexRelatedToDto))
Now I'm not so big on the exact syntax, but this is just the notion I'm looking at.
EDIT:
After the comments...
The answer is ... Nope!
The reason the principle exists is so you can identify which parts of the object fail. If you have them in one method, you're going to run into only one assertion, then the next, then the next, and you won't see them all.
So you can have it one of two ways:
One method, less boilerplate code.
Many methods, better reporting on the test run
It's up to you, both have ups and downs.
3. List item
[caveat: I'm very "unfluent" in Java/JUnit, so beware of errors in the details below]
There's a couple of ways to do this:
1) Write multiple assertions in the same test. This should be ok if you are only testing the DTO generation once. You could start here, and move to another solution when this starts to hurt.
2) Write a helper assertion, e.g. assertDtoFieldsEqual, passing in the expected and actual DTO. Inside the helper assertion you assert each field separately. This at least gives you the illusion of only one assert per test and will make things clearer if you test DTO generation for multiple scenarios.
3) Implement equals for the object that check each property and implement toString so that you at least can inspect the assertion result manually to find out what part is incorrect.
4) For each scenario where the DTO is generated, create a separate test fixture that generates the DTO and initializes the expected properties in the setUp method. The create a separate test for testing each of the properties. This also results in a lot of tests, but they will at least be one-liners only. Example in pseudo-code:
public class WithDtoGeneratedFromXxx : TestFixture
{
DTO dto = null;
public void setUp()
{
dto = GenerateDtoFromXxx();
expectedProp1 = "";
...
}
void testProp1IsGeneratedCorrectly()
{
assertEqual(expectedProp1, dto.prop1);
}
...
}
If you need to test the DTO generation under different scenarios and choose this last method it could soon become tedious to write all those tests. If this is the case you could implement an abstract base fixture that leaves out the details on how to create the DTO and setup the expected properties to derived classes. Pseudo-code:
abstract class AbstractDtoTest : TestFixture
{
DTO dto;
SomeType expectedProp1;
abstract DTO createDto();
abstract SomeType getExpectedProp1();
void setUp()
{
dto = createDto();
...
}
void testProp1IsGeneratedCorrectly()
{
assertEqual(getExpectedProp1(), dto.prop1);
}
...
}
class WithDtoGeneratedFromXxx : AbstractDtoTest
{
DTO createDto() { return GenerateDtoFromXxx(); }
abstract SomeType getExpectedProp1() { return new SomeType(); }
...
}
Or you can do some workaround.
import junit.framework.Assert;
import org.junit.After;
import org.junit.AfterClass;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Test;
public class NewEmptyJUnitTest {
public NewEmptyJUnitTest() {
}
#BeforeClass
public static void setUpClass() throws Exception {
}
#AfterClass
public static void tearDownClass() throws Exception {
}
#Before
public void setUp() {
}
#After
public void tearDown() {
}
#Test
public void checkMultipleValues() {
String errMessages = new String();
try{
this.checkProperty1("someActualResult", "someExpectedResult");
} catch (Exception e){
errMessages += e.getMessage();
}
try{
this.checkProperty2("someActualResult", "someExpectedResult");
} catch (Exception e){
errMessages += e.getMessage();
}
try{
this.checkProperty3("someActualResult", "someExpectedResult");
} catch (Exception e){
errMessages += e.getMessage();
}
Assert.assertTrue(errMessages, errMessages.isEmpty());
}
private boolean checkProperty1(String propertyValue, String expectedvalue) throws Exception{
if(propertyValue == expectedvalue){
return true;
}else {
throw new Exception("Property1 has value: " + propertyValue + ", expected: " + expectedvalue);
}
}
private boolean checkProperty2(String propertyValue, String expectedvalue) throws Exception{
if(propertyValue == expectedvalue){
return true;
}else {
throw new Exception("Property2 has value: " + propertyValue + ", expected: " + expectedvalue);
}
}
private boolean checkProperty3(String propertyValue, String expectedvalue) throws Exception{
if(propertyValue == expectedvalue){
return true;
}else {
throw new Exception("Property3 has value: " + propertyValue + ", expected: " + expectedvalue);
}
}
}
Maybe not the best approach and if overused than can confuse... but it is a possibility.