Evaluate all properties in one assertion with the FluentAssertions framework - unit-testing

Context:
We need to assert object responses with many properties which have many permutations and quite a few of those properties are dynamic (generated GUIDs etc).
Example Scenario
When using FluentAssertions ...Should().BeEquivalentTo(...) one is able to get a list of all non matching fields in one evaluation.
So given the (C#) code:
using System;
using FluentAssertions;
public class Program
{
public class HouseResponse
{
public int Windows { get; set; }
public int Bedrooms { get; set; }
public int Doors { get; set; }
public int Bathrooms { get; set; }
}
public static readonly HouseResponse ExpectedHouseResponse = new HouseResponse
{
Windows = 10,
Bedrooms = 5,
Doors = 2,
Bathrooms = 2
};
public static readonly HouseResponse ActualHouseResponse = new HouseResponse
{
Windows = 10,
Bedrooms = 5,
Doors = 3,
Bathrooms = 3
};
public static void Main()
{
ActualHouseResponse
.Should()
.BeEquivalentTo(ExpectedHouseResponse);
}
}
where there are 2 properties that do not match, the output for the single assertion is:
Unhandled exception. FluentAssertions.Execution.AssertionFailedException: Expected property root.Doors to be 2, but found 3.
Expected property root.Bathrooms to be 2, but found 3.
which is very handy as you get all failures in one error message.
But for partial matches, say where we expect the number of doors to differ but always be a valid number > 0, we would have to do this instead:
public static void Main()
{
ActualHouseResponse
.Should()
.BeEquivalentTo(ExpectedHouseResponse, config =>
config.Excluding(x => x.Doors));
ActualHouseResponse.Doors.Should().BeGreaterThan(0);
}
which wouldn't actually hit the ActualHouseResponse.Doors.Should().BeGreaterThan(0); assertion as we are already failing on .Should().BeEquivalentTo because .Bathrooms isn't a match.
So the goal is to be able to evaluate all properties in one go. Which will:
Enforce that all properties are evaluated.
Allow us to get a summary of all failing properties in one test run (rather than having to fix one property, run the test and then see where the next one fails etc)
Something along the lines of:
public static void Main()
{
ActualHouseResponse
.Should()
.BeEquivalentTo(ExpectedHouseResponse, config =>
config.OverideEvaluation(x => x.Doors, doors => doors > 0));
}
Does anyone have any ideas or has perhaps stumbled upon a bit of the FluentAssertions documentation I may have missed?
P.S I know this can be accomplished with a custom RuleBuilder and am familiar with FluentValidation but would like to keep that as a last resort.

You can use the Using/When combination to instruct the equivalency engine how to compare certain properties.
ActualHouseResponse.Should().BeEquivalentTo(ExpectedHouseResponse, opt => opt
.Using<int>(ctx => ctx.Subject.Should().BeGreaterThan(0))
.When(e => e.Path.EndsWith(nameof(HouseResponse.Doors)))
);
https://fluentassertions.com/objectgraphs/#equivalency-comparison-behavior

I see one option is to use the AssertionScope
public static void Main()
{
using (new AssertionScope())
{
ActualHouseResponse
.Should()
.BeEquivalentTo(ExpectedHouseResponse, config =>
config.Excluding(x => x.Doors));
ActualHouseResponse.Doors.Should().BeGreaterThan(0);
}
}
Which will run all the assertions before halting execution.
But this doesn't enforce that all properties of ActualHouseResponse are evaluated.

For those interested, this is the solution I went with based on Jonas's answer
public static class EquivalencyAssertionOptionsExtensions
{
public static EquivalencyAssertionOptions<TSource> Override<TSource, TProperty>(
this EquivalencyAssertionOptions<TSource> options,
Expression<Func<TSource, TProperty>> propertyAccessor,
Action<TProperty> assertion)
{
var memberExpression = (MemberExpression) propertyAccessor.Body;
var propertyName = memberExpression.Member.Name;
return options
.Using<TProperty>(ctx => assertion(ctx.Subject))
.When(x => x.SelectedMemberPath.Equals(propertyName));
}
}
which lets me do
ActualHouseResponse.Should().BeEquivalentTo(ExpectedHouseResponse, opt => opt
.Override(x => x.Doors, doors => doors.Should().BeGreaterThan(0))
.Override(x => x.Windows, windows => windows.Should().BeLessThan(10))
);

Related

Akka.net Testkit does not mark test case failed despite ActorInitializationException exception

Following is the actor, I've defined (trying to get my head around persistent actor!!)
public class Country : ReceivePersistentActor
{
public override string PersistenceId => GetType().Name + state.Id;
private CountryState state;
public Country()
{
Command<CreateCountry>(CreateCountry);
}
private bool CreateCountry(CreateCountry cmd)
{
Persist(new CountryCeated
{
Id = cmd.Id,
Code = cmd.Code,
Description = cmd.Description,
Active = cmd.Active
}, evt =>
{
state = new CountryState
{
Id = evt.Id,
Code = evt.Code,
Description = evt.Description,
Active = evt.Active
};
});
return true;
}
}
Following is unit test case that I've defined:
[TestClass]
public class CountrySpec : TestKit
{
[TestMethod]
public void CountryActor_Should_Create_A_Country()
{
var country = Sys.ActorOf(Props.Create(() => new Country()), "Country");
country.Tell(new CreateCountry(Guid.NewGuid(), "UK", "United Kingdom", true));
ExpectNoMsg();
}
}
When I run the test case, there is an exception that I can see in the output window of the test case
[ERROR][25/08/2016 08:25:07][Thread 0007][akka://test/user/Country] Object reference not set to an instance of an object.
Cause: [akka://test/user/Country#552449332]: Akka.Actor.ActorInitializationException: Exception during creation ---> System.NullReferenceException: Object reference not set to an instance of an object.
at Domain.Country.get_PersistenceId() in Y:\Spikes\StatefulActors\Domain\Country.cs:line 9
at Akka.Persistence.Eventsourced.StartRecovery(Recovery recovery)
at Akka.Persistence.Eventsourced.AroundPreStart()
at Akka.Actor.ActorCell.<>c__DisplayClass154_0.<Create>b__0()
at Akka.Actor.ActorCell.UseThreadContext(Action action)
at Akka.Actor.ActorCell.Create(Exception failure)
--- End of inner exception stack trace ---
at Akka.Actor.ActorCell.Create(Exception failure)
at Akka.Actor.ActorCell.SysMsgInvokeAll(EarliestFirstSystemMessageList messages, Int32 currentState)
but the test case is marked as success
Is there any way/settings in the TestKit, where it can be set such that for any exception, mark the test case failed?
By default, any exceptions inside actors are encapsulated - that means they don't bubble up, blowing the rest of the system.
Actors come in systems, and can be tested by observing the way they communicate with each other. Usually it comes up to providing inputs and asserting outputs from actor system - in your case test has passed, since you haven't validated any output. From the perspective of your test, this actor could be dead and it wouldn't make a difference.
Validating an outputs (either by assertion inside actor itself or i.e. using a custom test journal) is the best way to work with tests.
If for some reason you still have to catch exceptions inside actors, you could create supervision strategy bound to i.e. TestActor, where all exceptions could be forwarded:
public class TestingStrategy : OneForOneStrategy
{
protected TestingStrategy(IActorRef probe) : base(exception =>
{
probe.Tell(exception);
return DefaultDecider.Decide(exception);
}) { }
}

How can I override the test method name that appears on the TestNG report?

How can I override the test name that appears on the TestNG report? I want to override the name that appears in the middle column (currently shows as the method name). Is this even possible?
I tried to do it like this, but it didn't work.
public class EchApiTest1 extends TestBase {
...
#BeforeTest
public void setUp() {
restClient = new RestClientPost();
this.setTestName( "ech: XXXXXX" );
}
And, the base class:
import org.testng.ITest;
public class TestBase implements ITest {
String testName = "";
#Override
public String getTestName() {
return this.testName;
}
public void setTestName( String name ) {
this.testName = name;
}
}
NOTE: The above code does work when I am viewing the report detail in the Jenkins TestNG plugin report, which shows the overridden test name as a string called "Instance Name:" at the beginning of the Reporter log output. Why, in this case, WHY does a "setTestName()" method alter a string labeled "Instance Name" in the report?
One answer I found had a suggestion like this but I don't know how to pass an ITestResult arg to a AfterMethod method:
#AfterMethod
public void setResultTestName( ITestResult result ) {
try {
BaseTestMethod bm = (BaseTestMethod)result.getMethod();
Field f = bm.getClass().getSuperclass().getDeclaredField("m_methodName");
f.setAccessible(true);
f.set( bm, bm.getMethodName() + "." + your_customized_name );
} catch ( Exception ex ) {
Reporter.log( "ex" + ex.getMessage() );
}
Thoughts?
Please find following code for set custom name of testcase in TestNG reports.
Following features are available in this code.
Dynamic execution on same test-case in multiple time
Set custom test-case name for reports
Set parallel execution of multiple test-cases execution
import java.lang.reflect.Field;
import org.testng.ITest;
import org.testng.ITestResult;
import org.testng.Reporter;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Factory;
import org.testng.annotations.Test;
import org.testng.internal.BaseTestMethod;
import com.test.data.ServiceProcessData;
public class ServiceTest implements ITest {
protected ServiceProcessData serviceProcessData;
protected String testCaseName = "";
#Test
public void executeServiceTest() {
System.out.println(this.serviceProcessData.toString());
}
#Factory(dataProvider = "processDataList")
public RiskServiceTest(ServiceProcessData serviceProcessData) {
this.serviceProcessData = serviceProcessData;
}
#DataProvider(name = "processDataList", parallel = true)
public static Object[] getProcessDataList() {
Object[] serviceProcessDataList = new Object[0];
//Set data in serviceProcessDataList
return serviceProcessDataList;
}
#Override
public String getTestName() {
this.testCaseName = "User custom testcase name";
// this.testCaseName = this.serviceProcessData.getTestCaseCustomName();
return this.testCaseName;
}
#AfterMethod(alwaysRun = true)
public void setResultTestName(ITestResult result) {
try {
BaseTestMethod baseTestMethod = (BaseTestMethod) result.getMethod();
Field f = baseTestMethod.getClass().getSuperclass().getDeclaredField("m_methodName");
f.setAccessible(true);
f.set(baseTestMethod, this.testCaseName);
} catch (Exception e) {
ErrorMessageHelper.getInstance().setErrorMessage(e);
Reporter.log("Exception : " + e.getMessage());
}
}}
Thanks
I found a "workaround" but I am hoping for a better answer. I want to be able to show this "test name" OR "instance name" value on the HTML report (not just within the Reporter.log output) and I am starting to think its not possible :
#Test(dataProvider = "restdata2")
public void testGetNameFromResponse( TestArguments testArgs ) {
this.setTestName( "ech: " + testArgs.getTestName() );
Reporter.log( getTestName() ); // this magic shows test name on report
....
With this workaround, the user can now identify which test it was by looking at the Reporter.log output but I still wish the name was more prominant.
I suspect the answer lies in writing a TestListenerAdapter that somehow overrides the ITestResult.getTestNameMethod() method? That is the holy grail I am looking for.
The ‘result’ object will automatically pass in the method setResultTestName( ITestResult result )
Make sure you put alwaysRun=true like the following when you have groups defined in your test class otherwise “AfterMethod” will not be excuted.
#AfterMethod (alwaysRun=true)

When using Moq Verify() method invocation count, have failing test's error message contain actual method invocation count using Moq

Consider the following, where I am testing that an injected dependency's method is called a specific number of times:
[Fact]
public void WhenBossTalksEmployeeBlinksTwice()
{
// arrange
var employee = new Mock<IEmployee>();
employee.Setup(e => e.Blink());
var boss = new Boss(employee.Object);
// act
boss.Talk();
// assert
employee.Verify(e => e.Blink(), Times.Exactly(2)); // Passes as expected
employee.Verify(e => e.Blink(), Times.Exactly(1)); // Fails as expected
}
When I force the failing test, the output is:
Moq.MockException: Invocation was not performed on the mock 1 times: e
=> e.Blink()
What would be better is something like:
Moq.MockException: Invocation was unexpectedly performed 2 times, not 1 time: e
=> e.Blink()
Here are the items involved with the test:
public interface IEmployee { void Blink(); }
public class Boss {
private readonly IEmployee _employee;
public Boss(IEmployee employee) { _employee = employee; }
public void Talk() {
_employee.Blink();
_employee.Blink();
}
}
Is it possible to harvest and display the actual number of times the dependency's method was called, in the failing test's error message?
I'm not sure that it matters, but I'm using Moq v3.1.416.3 (not the latest, I know, but another library I'm using hasn't updated to Moq 4.x yet…)
I don't know of a straight way to harvest the information in Moq3.
What I would do is use a callback on the setup of Blink.
int count = 0;
employee.Setup(e => e.Blink()).Callback(() => count++);
...
employee.Verify(e => e.Blink(), Times.Exactly(1), "Moq.MockException: Invocation was unexpectedly performed " + count + " times, not 1 time: e => e.Blink()"); // Fails as expected

Code Contracts throws MethodAccess Exception when unit testing

I'm not quite sure why, but when I enable runtime contract checking, I'm getting a MethodAccessException during unit testing. I use the Machine.Specifications test framework and the ReSharper/dotCover test runner. When I test my assembly containing code contracts, I get this MethodAccessException:
Machine.Specifications.SpecificationException Should be of type System.ArgumentNullException but is of type System.MethodAccessException
at BusinessLogic.Specifications.When_testing_whether_a_set_of_cart_items_contains_a_product_and_a_null_set_is_passed.<.ctor>b__3()
in CartItemQuerySpecs.cs: line 66
The specification looks like this:
[Subject(typeof(ShoppingCartQueries), "Cart contains product")]
public class When_testing_whether_a_set_of_cart_items_contains_a_product_and_a_null_set_is_passed
: with_fake_shopping_cart_repository
{
Establish context = () => QueryResult = CartItems.AsQueryable().ForUser(UserWithNoProductsInCart);
Because of = () => Thrown = Catch.Exception(() => result = QueryResult.ContainsProduct(100));
It should_throw = () => Thrown.ShouldBeOfType<ArgumentNullException>();
static bool result;
}
The unit under test looks like this:
[Pure]
public static bool ContainsProduct(this IQueryable<CartItem> items, int id)
{
Contract.Requires<ArgumentNullException>(items != null);
return (items.Any(item => item.ProductId.Equals(id)));
}
Why would I get a MethodAccessException?

Using Moq to mock a repository that returns a value

How do I set up my test method on that mocks a repository which accepts an object?
This is what I have so far:
Service.cs
public int AddCountry(string countryName)
{
Country country = new Country();
country.CountryName = countryName;
return geographicsRepository.SaveCountry(country).CountryId;
}
test.cs
[Test]
public void Insert_Country()
{
//Setup
var geographicsRepository = new Mock<IGeographicRepository>();
geographicsRepository.Setup(x => x.SaveCountry(It.Is<Country>(c => c.CountryName == "Jamaica"))); //How do I return a 1 here?
GeographicService geoService = new GeographicService(geographicsRepository.Object);
int id = geoService.AddCountry("Jamaica");
Assert.AreEqual(1, id);
}
SaveCountry(Country country); returns an int.
I need to do 2 things:
First test, I need to tell the setup to return an int of 1.
I need to create a second test Insert_Duplicate_Country_Throws_Exception(). In my Setup, how do I tell the repository to throw an error when I do:
int id = geoService.AddCountry("Jamaica");
int id = geoService.AddCountry("Jamaica");
Framework:
NUnit.
Moq.
ASP.NET MVC - repository pattern.
Your first test should look something like this:
[Test]
public void Insert_Country()
{
Mock<IGeographicRepository> geographicsRepository = new Mock<IGeographicRepository>();
GeographicService geoService = new GeographicService(geographicsRepository.Object);
// Setup Mock
geographicsRepository
.Setup(x => x.SaveCountry(It.IsAny<Country>()))
.Returns(1);
var id = geoService.AddCountry("Jamaica");
Assert.IsInstanceOf<Int32>(id);
Assert.AreEqual(1, id);
geographicsRepository.VerifyAll();
}
The second test should look like this:
[Test]
public void Insert_Duplicate_Country_Throws_Exception()
{
Mock<IGeographicRepository> geographicsRepository = new Mock<IGeographicRepository>();
GeographicService geoService = new GeographicService(geographicsRepository.Object);
// Setup Mock
geographicsRepository
.Setup(x => x.SaveCountry(It.IsAny<Country>()))
.Throws(new MyException());
try
{
var id = geoService.AddCountry("Jamaica");
Assert.Fail("Exception not thrown");
}
catch (MyException)
{
geographicsRepository.VerifyAll();
}
}
I think maybe you are slightly misunderstanding the purpose of testing with mocks in the two scenarios you have supplied.
In the first scenario, you wish to test that 1 is returned when you pass in "Jamaica". This is not a mock test case but a test case for real behaviour as you wish to test a specific input against an expected output i.e. "Jamaica" -> 1. In this situation mocking is more useful to ensure that internally your service calls SaveCountry on the repository with the expected country, and that it returns the value from the call.
Setting up your "SaveCountry" case and then calling "VerifyAll" on your mock is the key. This will assert that "SaveCountry" was indeed called with country "Jamaica", and that the expected value is returned. In this way you have confidence that your service is wired up to your repository as expected.
[Test]
public void adding_country_saves_country()
{
const int ExpectedCountryId = 666;
var mockRepository = new Mock<IGeographicRepository>();
mockRepository.
Setup(x => x.SaveCountry(It.Is<Country>(c => c.CountryName == "Jamaica"))).
Returns(ExpectedCountryId);
GeographicService service= new GeographicService(mockRepository.Object);
int id = service.AddCountry(new Country("Jamaica"));
mockRepo.VerifyAll();
Assert.AreEqual(ExpectedCountryId, id, "Expected country id.");
}
In the second scenario you wish to test that an exception is raised when you attempt to add a duplicate country. There's not much point in doing this with a mock as all you will test is that your mock has behaviour when adding duplicates, not your real implementation.