I am new to Nunit and have some tests that perform API calls for a .NET 6 Web API app. I want to convert these into true unit tests by removing those external dependencies. I was looking into mocking and found Mockoon.
How I would go about mocking API calls? Can I do it with Mockoon?
I wasn't sure how I could change the API calls programmatically to point to a mock API for testing only.
The ultimate goal is to automate the testing in GitHub Actions but since the tests have external dependencies, these need to be mocked first. Mockoon provides a CLI to deploy in GitHub Actions.
My project was created in VS Code so I'm not sure if Visual Studio utilities would help with configurations.
If you need any more information, please let me know. Thanks!
It's a bit of a broad question for SO I think. But here are some thoughts.
In my opinion, pure unit testing should mock APIs programmatically and not depend on external applications like Mockoon.
What you want to do seems to be more like integration testing, where your code is calling an external API, except that instead of maintaining an instance of the original API specifically for test/QA purposes, you maintain a mock (with Mockoon).
Then, the general idea is more or less the following:
you prepare the mock API with Mockoon locally with the UI.
you save Mockoon's data file in the repository.
During the GitHub Actions workflow, you install Mockoon CLI and start it targeting the data file (relevant tutorial).
You run your application tests and modify the environment variable containing the API URL to point to were the CLI is running (probably something like http://localhost:3000).
Related
I am using AWS API Gateway with Lambda/ECS for compute and Cognito for users. But I find it really hard to test such applications. With AWS SAM Local I maybe able to test simple Lambda and API gateway functionality but if I use things like API Gateway authorizers I find it hard to test these end to end.
Looks like to test such applications, I need an entire new setup just for testing? I mean like a separate API Gateway with Lambda/ECS cluster/Cognito user pool just to enable testing? This seems very slow, and I think I will not be able to get things like a code coverage report anymore?
Disclaimer: I'm fairly new to AWS Lambda/ECS/Cognito so take this with a grain of salt.
Unit Tests: SAM Local or some other local docker hosting with a unit testing library (mocha) would be good for this because:
Speed. All your tests should execute quickly against a lambda function
Example : wildrydes with mocha
Integration Tests: Once you stage your changes, there's a bunch of options calling the API. I'd start off with postman to run the API tests and you can chain them together or run them in command line if needed.
End to End (E2E) tests: If the API is your front end then there might not be any difference between E2E and API tests. UI, Voice, Chat front ends differ significantly as do the options so I'll suggest some options:
UI : Selenium (has the most support and options available to you including docker images: Selenium Hub or standalone)
Voice: Suggestions?
Text: Suggestions?
Step functions :
allows you to visualize each step
retries when there are errors
allows you to diagnose and debug problems
X-Ray: collects data about requests that your app serves, and provides tools you can use to view
As for code coverage, I'm not sure how you currently do code coverage. Something like this npm run coverage, maybe?
I am assuming you are using cloudformation for deploying such an extensive stack and the following answer is based on that assumption.
Thus in addition to the #lloyd's answer, I would like to add that you can add custom resources within your cloudformation template for testing each individual lambdas or even api endpoints.
Also for lambda, you can use Deployment Preferences Hooks to test your serverless lambdas before and after moving your lambda to the new version.
https://github.com/awslabs/serverless-application-model/blob/release/v1.8.0/docs/safe_lambda_deployments.rst
In webmethods (Software AG), is there a way to Mock object during unit testing?
or any available tool to test flow service.
You could have a look at the Open Source http://www.wmaop.org test framework that allows general mocking and unit testing along with a host of other functionality. The framework allows you to:
Create mocks of IS services
Apply conditions to mocks so that they only execute when the pipeline contents meet that condition
Raise an exception based on a condition or in place of a service
Capture the pipeline to file before or after a service is called
Modify or insert content into the pipeline
Have a series of conditions for a mocked service with a default if none of the conditions match
Create assertions that can apply before or after a service so that its possible to prove a service has been executed. Assertions can also have conditions to verify that the pipeline had the expected content.
Return either random or sequenced content from a mock to very its output every time its called
Create mocks using RESTful calls so you can use alternative test tools, such as SOAPui, to create them as part of your integrations test
Use the JBehave functionality for Behaviour Driven Unit Testing within Designer and execute tests with the in-built JUnit.
WmTestSuite could be a good tool for you (Why reinvent the wheel), your company chose webMethods to speedup devs, i advice you to keep going.
What wmTestSuite does:
Create unit tests Graphically for you flows in the Designer
Generate the related TestUnit class (you can complete it to add some asserts)
Add a hook the Integration server to "register" data to create test data
Mock endpoints to ease tests (db, ws...)
I got this slide from a SoftwareAG guy. From the version 9.10 (April 2016) you should be able to download it from empower.
You cannot define mocks in webMethods directly, as it requires you to hook into the invoke chain. This is a set of methods that are called between every flow or java service invocation. They take care of things like access control, input/output validation, updating statistics, auditing etc.
There are various tools and products available that leverage this internal mechanism and let you create mocks (or stubs) for your unit or system test cases:
IwTest, commercial, from IntegrationWise
WmTestSuite, commercial, from SoftwareAG
CATE, commercial, from Cloudgensys
WmAOP, open source, www.wmaop.org
With all four you can create test cases for webMethods flow/java services and define mocks for services that access external systems. All four provide ways to define assertions that the results should satisfy.
By far the easiest to work with is IwTest as it lets you generate test suites, including mocks (or stubs), based on input/output pipelines that it records for you. In addition to this it also supports pub/sub (asynchronous) scenario's.
Ask your Software AG liaison about webMethods Test Suite (WmTestSuite), which plugs into the Eclipse-based Designer and provides basic Unit testing capabilities.
Mocks per se are lightweight services that can be configured in the WmTestSuite dialog alongside the (test) input and (expected) output pipelines.
Within a short-time period I'm going to start a project based on Windows Azure. And I was wondering what are the experiences with testing for Windows Azure projects (in continuous intergration (with a TFS build server))? (Eventual using TDD)
Some things I was wondering:
Do you use mocking (in your own written wrapper class)?
Do you use the storage emulator?
Do you deploy the services to Azure and run the tests from the build server to the cloud? (what about costs)?
Thnaks in advance!
The same good practices for writing unit tests for applications outside of Windows Azure apply. If you have an external dependency to what you are actually testing, that dependency should be mocked and injected for your granular unit test.
For example, when I'm using Windows Azure Storage Queues I will have an interface that I use to interact with the queue itself, so in my code consuming the queue service I can mock the subsystem using the interface and use dependency injection to inject the mock. This removes the necessity to actually deal with the emulator during unit tests. For the most part the actual concrete implementation of the code working with the queue is not much more than a very thin wrapper.
I personally don't shoot for 100% test coverage, so I may not have direct unit tests that utilize the concrete implementation of the wrappers. In many cases I try to have integration tests that will exercise these wrappers and exercise multiple aspects of the system working together. In some cases I can run the integration tests in the emulator (for Storage operations for example), but in some cases they simply have to be run with access to the Windows Azure environment (in the case of usage of ACS or Service Bus).
Ideally you'd like to have a set of scripts that can be run to spin up a minimum set of test servers in Azure, deploy your solution and exercise the integration tests that can't be done on premises. Then get the results of that and have the script shut everything down (or optionally leave it running if you need that). Then run the integration tests suite that utilizes these scripts often enough to detect issues, but you certainly don't need to run them every time you check something in unless you are happy with running the test environment all the time. If you okay with the cost of a semi-permanent test environment running in Azure then just make sure to have the scripts to an update deployment rather than a delete and redeploy to cut down on cost a bit (savings would be relative to how often the deploy occurs).
I believe this question is a very subjective one as you're likely to get several different opinions.
I was told the best solution to run unit tests for code that uses App Engine services such as datastore or memcache was to run the development server in a child process, but I'm not sure how. Anybody successfully ran this kind of test and can share a solution?
App Engine SDK for Go uses the Python dev_appserver; see this thread.
You should check out Google App Engine Go testing library by Josh Marsh.
An interesting development, as of 1.8.6 using service stubs has been integrated into the SDK through the "appengine/aetest" package. More info
I know the questioner wants to build a testbed and needs to do this, but I think there's another approach worth mentioning here.
Besides using a testbed for the GAE services, Go's interesting nature also opens up another possibility: write your application to just require objects that have the interfaces that you use (they'd be a subset of the official APIs) and mock them out when testing. This requires you to be doing some amount of dependency injection of some sort, but that's a really good idea anyway.
Once the interfaces the interfaces are written, you can mock them using a library like gomock.
I have an application that sends messages to an external web service. I build and deploy this application using MSBuild and Cruisecontrol.NET. As CCNET build and deploys the app it also runs a set of test using NUnit. I'd now like to test the web service communication as well.
My idea is that as part of the build process a web service should be generated (based on the external web services WSDL) and deployed to the build servers local web server. All the web service should do is to receive the message and place it on the file system so I then can check it using ordinary NUnit for example. This would also make development easier as new developers would only have to run the build script and be up and running (not have to spend time to set up a connection to the third party service).
Are there any existing utilities out there that easily mock a web service based on a WSDL? Anyone done something similar using MSBuild?
Are there other ways of testing this scenario?
I just started looking into http://www.soapui.org/ and it seems like it will work nicely for testing web services.
Also, maybe look at adding an abstraction layer in your web service, each service call would directly call a testable method (outside of the web scope)? I just did this with a bigger project I'm working on, and it's testability is working nicely.
In general, a very good way to test things like this is to use mock objects.
At work, we use the product TypeMock to test things like Web Service communication and other outside dependencies. It costs money, so for that reason it may not be suitable for your needs, but I think it's a fantastic product. I can tell you from personal experience that it integrates very well with NUnit and CCNet.
It's got a really simple syntax where you basically say "when this method/property is called, I want you to return this value instead." It's great for testing things like network failures, files not being present, and of course, web services.
Take a look at NMock2. It's a open-source mocking product and allows you to create "virtual" implementations for interfaces that support rich and deep interaction.
For example, if your WS interface is called IService and has a Data GetData() method, you can create a mock that requires the method to be called once and returns a new Data object:
var testService = mockery.NewMock<IService>();
Expect
.Once
.On(testService)
.Method("GetService")
.WithNoArguments()
.Will(
Return.Value(new Data());
At the end of the test, call mockery.VerifyAllExpectationsHaveBeenMet() to assure that the GetData method was actually called.
P.S.: don't confuse the "NMock2" project with the "NMock RC2", which is also called "nmock2" on sourceforge. NMock2-the-project seems to have superseded NMock.
This might also be something - MockingBird. Look useful.
At my work place we are using Typemock and nUnit for our unit testing.