Test runners inconsistent with HttpClient and Mocking HttpMessageRequest XUnit - unit-testing

So let me start by saying I've seen all the threads over the wars between creating a wrapper vs mocking the HttpMethodRequest. In the past, I've done the wrapper method with great success, but I thought I'd go down the path of Mocking the HttpMessageRequest.
For starters here is an example of the debate: Mocking HttpClient in unit tests. I want to add that's not what this is about.
What I've found is that I have tests upon tests that inject an HttpClient. I've been doing a lot of serverless aws lambdas, and the basic flow is like so:
//some pseudo code
public class Functions
{
public Functions(HttpClient client)
{
_httpClient = client;
}
public async Task<APIGatewayResponse> GetData(ApiGatewayRequest request, ILambdaContext context)
{
var result = await _client.Get("http://example.com");
return new APIGatewayResponse
{
StatusCode = result.StatusCode,
Body = await result.Content.ReadStringAsAsync()
};
}
}
...
[Fact]
public void ShouldDoCall()
{
var requestUri = new Uri("http://example.com");
var mockResponse = new HttpResponseMessage(HttpStatusCode.OK) { Content = new StringContent(expectedResponse) };
var mockHandler = new Mock<HttpClientHandler>();
mockHandler
.Protected()
.Setup<Task<HttpResponseMessage>>(
"SendAsync",
It.IsAny<HttpRequestMessage>(),
It.IsAny<CancellationToken>())
.ReturnsAsync(mockResponse);
var f = new Functions(new HttpClient(handler.Object);
var result = f.GetData().Result;
handlerMock.Protected().Verify(
"SendAsync",
Times.Exactly(1), // we expected a single external request
ItExpr.Is<HttpRequestMessage>(req =>
req.Method == HttpMethod.Get &&
req.RequestUri == expectedUri // to this uri
),
ItExpr.IsAny<CancellationToken>()
);
Assert.Equal(200, result.StatusCode);
}
So here's where I have the problem!
When all my tests run in NCrunch they pass, and pass fast!
When I run them all manually with Resharper 2018, they fail.
Equally, when they get run within the CI/CD platform, which is a docker container with the net core 2.1 SDK on a Linux distro, they too fail.
These tests should not be run in parallel (read the tests default this way). I have about 30 tests around these methods combined, and each one randomly fails on the moq verify portion. Sometimes they pass, sometimes they fail. If I break down the tests per test class and on run the groups that way, instead of all in one, then these will all pass in chunks. I'll also add that I have even gone through trying to isolate the variables per test method to make sure there is no overlap.
So, I'm really lost with trying to handle this through here and make sure this is testable.
Are there different ways to approach the HttpClient where it can consistently pass?

After lots of back n forth. I found two of situations from this.
I couldn't get parallel processing disabled within the docker setup, which is where I thought the issue was (I even made it do thread sleep between tests to slow it down (It felt really icky to me)
I found that all the tests l locally ran through the test runners were telling me they passed when about 1/2 failed on the docker test runner. What ended up being the issue was a magic string area when seeing and getting environment variables.
Small caveat to call out, Amazon updated their .NET Core lambda tools to install via dotnet cli, so this was updated in our docker image.

Related

ReSharper not supporting Assert.That

I'm in the process of returning to ReSharper recently, using trial of the newest version - 2022.2.3. I've got quite surprised when one of my nUnit tests failed in a weird way, when run by Resharper's built in Unit Test runner. Something that has never happened to me with a Test Explorer.
As long as the Asserts pass, it's all fine - green, all tests are listed. However, when the assert fails, it says One or more child tests had errors. Exception doesn't have a stacktrace
Not only there is no mention of actual values that weren't correct, but the whole failing test seems to be gone!
This happens only when I use the 'modern' approach with Assert.That. So
Assert.That(httpContext.Response.StatusCode, Is.EqualTo(200));
is causing issues, meanwhile, the more classic:
Assert.AreEqual(200, httpContext.Response.StatusCode);
works as expected. Is that something that is a known bug, or maybe some attributes are required? JetBrains claims they have full support of nUnit out of the box, so that is a bit surprising.
NOTE: the tests methods are async, awaiting result and returning Tasks, beside this nothing unusual.
EDIT: The test code is as follows, ApiKeyMiddleware is any middleware that returns response with 200 here.
[TestFixture]
public class ApiKeyMiddlewareTests
{
[Test]
public async Task Invoke_ActiveKey_Authorized()
{
var httpContext = new DefaultHttpContext();
httpContext.Request.Headers.Add("XXXXX", "xxxx");
var configuration = Options.Create(new AccessConfiguration { ActiveApiKeys = new List<string> { "xxxx" } });
var middleware = new ApiKeyMiddleware(GetEmptyRequest(), configuration);
await middleware.Invoke(httpContext);
Assert.That(httpContext.Response.StatusCode, Is.EqualTo(200)); //change to anything else than 200 and it fails + vanishes
}
}

How best to unit test a ServiceStack service that uses IServiceGateway to call other internal services

I've been following the guidelines here - https://docs.servicestack.net/testing
I'm trying to do unit testing rather than integration, just to cut down the level of mocking and other complexities.
Some of my services call some of my other services, via the recommended IServiceGateway API, e.g. Gateway.Send(MyRequest).
However when running tests i'm getting System.NotImplementedException: 'Unable to resolve service 'GetMyContentRequest''.
I've used container.RegisterAutoWired() which is the service that handles this request.
I'm not sure where to go next. I really don't want to have to start again setting up an integration test pattern.
You're likely going to continually run into issues if you try to execute Service Integrations as unit tests instead of Integration tests which would start in a verified valid state.
But for Gateway Requests, they're executed using an IServiceGateway which you can choose to override by implementing GetServiceGateway() in your custom AppHost with a custom implementation, or by registering an IServiceGatewayFactory or IServiceGateway in your IOC, here's the default implementation:
public virtual IServiceGateway GetServiceGateway(IRequest req)
{
if (req == null)
throw new ArgumentNullException(nameof(req));
var factory = Container.TryResolve<IServiceGatewayFactory>();
return factory != null ? factory.GetServiceGateway(req)
: Container.TryResolve<IServiceGateway>()
?? new InProcessServiceGateway(req);
}
Based on the discussion in the answer by #mythz, this is my solution:
Use case like OP: Test the "main" service, and mock the "sub service", and like OP I'd want to do that with Unit test (so BasicAppHost), because it's quicker, and I believe it is easier to mock services that way (side note: for AppHost based integration test, SS will scan assemblies for (real) Services, so how to mock? Unregister from "container" and replace w. mock?).
Anyway, for the unit test:
My "main" service is using another service, via the IServiceGateway (which is the officially recommended way):
public MainDtoResponse Get(MainDto request) {
// do some stuff
var subResponse = Gateway.Send(new SubDto { /* params */ });
// do some stuff with subResponse
}
In my test setup:
appHost = new BasicAppHost().Init();
var mockGateway = new Mock<IServiceGateway>(); // using Moq
mockGateway.Setup(x => x.Send<SubDtoResponse>(It.IsAny<SubDto>()))
.Returns(new SubDtoResponse { /* ... */ });
container.Register(mockGateway.Object);
So the IServiceGateway must be mocked, and then the Send method is the important one. What I was doing wrong was to mock the service, when I should have mocked the Gateway.
Then call the main service (under test) in the normal fashion for a Unit Test, like in the docs:
var s = appHost.Container.Resolve<MainService>(); // must be populated in DI manually earlier in code
s.Get(new MainDto { /* ... */ })
PS: The mockGateway.Setup can be used inside each test, not necessarily in the OneTimeSetUp.

What is a good practice to write unit-test on .net core Ihostedservice?

I have a background task initiated in .net core 2.0 startup, inherits from backgroundservice, implementing StartAsync, StopAsync and ExecuteAsync. This task is to update some data in database table periodically based on some business logic.
While I can run the backgroundtask as an application and test using logs, db check and with the help of other tools, can the unit-testing is necessary for testing the backgroundtask? If so how to register the task as a service with dependencies and trigger the start and stop methods to assert the actual vs expected? Appreciate some basic sample unit-test method on testing timer based .net core ihostedservice backgroundtask.
Here is my basic test start just for sample, but not completed yet. Having said that, this is just a thought but not the exact working test. Here is what need some help from the community. Can also add some more asserts i.e. Assert.Verify()?
[Fact]
public async void Run_background_task_success()
{
//Arrange
IServiceCollection services = new ServiceCollection();
services.AddHostedService<BackgroundManagerTask>();
var serviceProvider = services.BuildServiceProvider();
var service = serviceProvider.GetService<IHostedService>() as BackgroundManagerTask;
var isExecuted = false;
if(await service.StartAsync(CancellationToken.None))
{
isExecuted = true;
}
await Task.Delay(10000);
Assert.True(isExecuted);
await service.StopAsync(CancellationToken.None);
}
Here's how I usually do it. You mention you are going to the database to update some data, so I'm assuming you are expecting that as a dependency from BackgroundManager
[Fact]
public void BackgroundManagerUpdatingDataTest()
{
// Arrange
Mock<IDataAccess> dbMock = new Mock<IDataAccess>();
dbMock.Setup(x => x.UpdateSomethingInDB(It.IsAny<BusinessObject>())).Returns(1); // One row updated from the DML in UpdateSomethingInDB from the BusinessObject
BackgroundManager sut = new BackgroundManager(dbMock.Object); // System under test.
// Act
await sut.StartAsync(CancellationToken.None);
await Task.Delay(500); // Give the test some time to execute.
await sut.StopAsync(CancellationToken.None); // Stop the Background Service.
// Assert
dbMock.Verify(x => x.UpdateSomethingInDB(It.IsAny<BusinessObject>()), Times.Exactly(1));
}
Above, we are plainly testing the update to the database occurred by Mocking the data access call and verifying that it was called exactly once.
You could of course Mock any other dependency out using Moq and Assert on anything else you want to verify.

How can I unit test a MassTransit consumer that builds and executes a routing slip?

In .NET Core 2.0 I have a fairly simple MassTransit routing slip that contains 2 activities. This is built and executed in a consumer and it all ties back to an automatonymous state machine. It all works great albeit with a few final clean tweaks needed.
However, I can't quite figure out the best way to write unit tests for my consumer as it builds a routing slip. I have the following code in my consumer:
public async Task Consumer(ConsumerContext<ProcessRequest> context)
{
var builder = new RoutingSlipBuilder(NewId.NextGuid());
SetupRoutingSlipActivities(builder, context);
var routingSlip = builder.Build();
await context.Execute(routingSlip).ConfigureAwait(false);
}
I created the SetupRoutingSlipActivities method as I thought it would help me write tests to make sure the right activities were being added and it simply looks like:
public void SetupRoutingSlipActivities(RoutingSlipBuilder builder, ConsumeContext<IProcessCreateLinkRequest> context)
{
builder.AddActivity(
nameof(ActivityOne),
new Uri("execute_activity_one_example_address"),
new ActivityOneArguments(
context.Message.Id,
context.Message.Name)
);
builder.AddActivity(
nameof(ActivityTwo),
new Uri("execute_activity_two_example_address"),
new ActivityTwoArguments(
context.Message.AnotherId,
context.Message.FileName)
);
}
I tried to just write tests for the SetupRoutingSlipActivities by using a Moq mock builder and a MassTransit InMemoryTestHarness but I found that the AddActivity method is not virtual so I can't verify it as such:
aRoutingSlipBuilder.Verify(x => x.AddActivity(
nameof(ActivityOne),
new Uri("execute_activity_one_example_address"),
It.Is<ActivityOne>(y => y.Id == 1 && y.Name == "A test name")));
Please ignore some of the weird data in the code examples as I just put up a simplified version.
Does anyone have any recommendations on how to do this? I also wanted to test to make sure the RoutingSlipBuilder was created but as that instance is created in the Consume method I wasn't sure how to do it! I've searched a lot online and through the MassTransit repo but nothing stood out.
Look at how the Courier tests are written, there are a number of test fixtures available to test routing slip activities. While they aren't well documented, the unit tests are a working testament to how the testing is used.
https://github.com/MassTransit/MassTransit/blob/develop/src/MassTransit.Tests/Courier/TwoActivityEvent_Specs.cs

Unit Testing Azure Functions with HttpRequestMessage

I'm running into a few issues when trying to unit test HTTP Trigger Azure Functions in Visual Studio. I've created a GitHub repo (https://github.com/ericdboyd/TestingAzureFunctions) with an example solution that contains both an Azure Function project and a Unit Test project that demonstrates the issues.
First, when I bring in Microsoft.AspNet.WebApi.Core it creates a conflict between System.Web.Http and Microsoft.AspNetCore.Mvc.WebApiCompatShim when trying to use IContentNegotiator. The only way around that was to alias WebApiCompatShim in the csproj file for the test project using the following:
<Target Name="ChangeAliasesOfStrongNameAssemblies" BeforeTargets="FindReferenceAssembliesForReferences;ResolveReferences">
<ItemGroup>
<ReferencePath Condition="'%(FileName)' == 'Microsoft.AspNetCore.Mvc.WebApiCompatShim'">
<Aliases>webapicompatshim</Aliases>
</ReferencePath>
</ItemGroup>
Once I got past that error, I run into this issue which I haven't been able to get past. Using the HttpRequestMessage.CreateResponse extension method to return a response in the Azure Function, I get "No service for type 'System.Net.Http.Formatting.IContentNegotiator' has been registered." when I try to test it. I have tried to build a HttpRequestMessage that I think should work with that extension method using the following code which can also be found in the GitHub repo, but it fails, and I have worked on trying to get past this for several hours now.
IServiceCollection services = new ServiceCollection();
services.AddOptions();
services.AddSingleton(typeof(IContentNegotiator), typeof(DefaultContentNegotiator));
IServiceProvider serviceProvider = services.BuildServiceProvider();
var httpContext = new DefaultHttpContext {RequestServices = serviceProvider};
var httpConfiguration = new HttpConfiguration();
HttpRequestMessage request = new HttpRequestMessage
{
Method = HttpMethod.Post,
RequestUri = new Uri(url),
Content = new StringContent(content, Encoding.UTF8, "application/json"),
Properties =
{
{ HttpPropertyKeys.HttpConfigurationKey, httpConfiguration },
{ nameof(HttpContext), httpContext}
}
};
If I don't use that CreateResponse extension method, and just create HttpResponseMessage objects, it works fine.
Just to set some additional context, I know this is not the best way to unit test the code being executed by the Azure Function. I'm unit testing that code much more granularly. But I want to be able to unit test the Azure Function that is performing the mapping between the http request and response to that logic.
There are two Azure Functions and two unit tests in the GitHub repo, one set with the extension method, one without to demonstrate the issue. But everything else is the same.
Not a direct answer to your problem, but it should help you go forward.
You are using Azure Functions v2 - .NET Standard version. This version is currently in beta, so it's a bit shady territory: the documentation is missing and some issues exist.
In V2 you are advised to use HttpRequest class and IActionResult instead of 'classic' HttpRequestMessage and HttpResponseMessage. The default template has a signature like this:
public static IActionResult Run(
[HttpTrigger(AuthorizationLevel.Function, "get")] HttpRequest req, TraceWriter log)
This should enable you to get rid of your shim and to unit test the functions similar to ASP.NET Core way.
Specify the jsonformatter while responding as below
req.CreateResponse(HttpStatusCode.OK, $"Hello, {name}", new JsonMediaTypeFormatter())