We have Github actions set for PRs to automatically run Unit tests and reject merge if any test fails.
I added an additional environment variable in a new PR, and the Github action does not pick this new environment variable as it doesn't exist in the main branch.
It got me thinking since Unit tests should run as independently as possible, do we explicitly set the $_ENV values, somewhere in the setUp possibly?
Related
My unit test needs a remote server address to startup. The server address is not fixed.
If I put the address in my .go test source, I will change them everytime when I run it.
If I put them in system environment variable, it is very inconvenience to change it in VSCode GUI. (I mean I will start the test in VSCode menu.)
I known I can put environment variable in launch.json to setup before run or debug my program. But here I just want to run unit test.
Is there a good way to change the parameters without restarting the VSCode?
You can add following snippets to VSCode settings.json to specify environment variables just for go test runs:
Defining variables directly:
"go.testEnvVars": {
"MY_VAR": "my value"
},
Or using dedicated file (in my example called test.env in root of project workspace) containing the environment variables in MY_VAR="my value" format with one variable per line:
"go.testEnvFile": "${workspaceFolder}/test.env",
Also note that unit tests (as name suggests they test one unit of code) should generally not depend on any external services or resources. Everything except the logic under test should be provided in form of mocks.
I have a test project I'm using to practice deploying to Azure
Located:
https://github.com/EdLichtman/HelloAzureCI
When I use Resharper to run the NUnit tests, All of them pass except for the Environment-Specific test case, as should be expected.
However when I run deploy.cmd on my local computer All 4 tests fail because "Object reference not set to an instance of an object."
One of my unit tests is "Assert.AreEqual(1,1)" and that throws a nullReference exception, which leads me to think that Assert is not an instance of an object.
Why is this such a problem? Can anyone else recreate?
There are a few odd things here, but the main one is that you are trying to run NUnit 3.7.1 tests using an NUnit V2 console runner (2.6.2). This is never going to work. My suggestion is that whenever you have trouble with running NUnit in a remote environment or using a third-party runner, you fall back to using the console runner locally. Even if that's not your preferred mode of working, you will usually be able to figure out what's wrong more easily if you eliminate as many middlemen as possible.
If you actually want to run under vsconsole, then you need to install the nunit3-vs-adapter nuget package and point to it's location in your command-line. Note that the adapter, even though it is ours, constitutes another middleman, so debugging using nunit3-console is still a good choice.
Does jest automatically restore mocked modules between test files? For example, if I call jest.mock('some_module') in one file, do I need to ensure I call jest.unmock('some_module') after all the tests are run in that file?
It's not clear to me whether that happens in the documentation.
You don't have to reset the mocks, as the test are run in parallel, every test file run in its own sandboxed thread. Even mocking JavaScript globals like Date or Math.random only affects the actual test file.
The only problem we had so far was mocking process.env.NODE_ENV which affected other test that run at the same time. But reseting this after the test run solved the problem.
The code I'm unit testing refers to an appsetting in the app.config file. To cater for this, I've added an app.config file to my unit tests project. If I click the "Run All Tests" icon in the Unit Test Sessions window, all my tests pass.
I have mapped the "ReSharper.ReSharper_UnitTest_RunSolution" command to Ctrl+Shift+Alt+U. If I run the tests by pressing this combination, the tests all run, but they fail to find the appsetting, which comes through as null.
I'm assuming this means that the button click runs under the context of the test project, whilst the command does not, but I can't quite work out what the command is doing.
Have I mapped the wrong command?
EDIT 1: I've also tried using the keyboard shortcut Alt-RUN (Resharper > Unit Tests > Run All), as well as clicking the menus manually, and found that this also causes all unit tests to not find the appsetting and therefore fail. Clicking the Run All Tests icon in Unit Test Sessions (the double green arrow) continues to work fine.
EDIT 2: I realised I should probably be mocking a separate class that fetches appsettings from the config file anyway, so this is what I'm now doing. So now there is no dependence on the config file when unit testing.
There are two things going on here. Firstly, the Run All Tests icon in the sessions window runs all tests in the session, while the Run All Tests menu item runs all tests in the solution. Slightly confusing that they've got the same name, but it does make sense given the context. This is why they give different results.
Secondly, when running all tests in a solution, the app setting might not get found. This is due to an optimisation the test runner makes which runs all tests in the same AppDomain. This avoids the overhead of creating a new AppDomain for each assembly, but has the downside that only one app.config will be used for all assemblies. If it picks the wrong one, your app setting is lost.
You can disable this by unchecking ReSharper » Options » Unit Testing » "Use separate AppDomain for each assembly with tests". Ideally, it should be disabled if any project has an app.config - I've added a feature request you can vote for and track: https://youtrack.jetbrains.com/issue/RSRP-428958
I've got some unit tests (c++) running in the Visual Studio 2012 test framework.
From what I can tell, the tests are running in parallel. In this case the tests are stepping on each other - I do not want to run them in parallel!
For example, I have two tests in which I have added breakpoints and they are hit in the following order:
Test1 TEST_CLASS_INITIALIZE
Test2 TEST_CLASS_INITIALIZE
Test2 TEST_METHOD
Test1 TEST_METHOD
If the init for Test1 runs first then all of its test methods should run to completion before anything related to Test2 is launched!
After doing some internet searches I am sufficiently confused. Everything I am reading says Visual Studio 2012 does not run tests concurrently by default, and you have to jump through hoops to enable it. We certainly have not enabled it in our project.
Any ideas on what could be happening? Am I missing something fundamental here?
Am I missing something fundamental here?
Yes.
Your should never assume that another test case will work as expected. This means that it should never be a concern if the tests execute synchronously or asynchronously.
Of course there are test cases that expect some fundamental part code to work, this might be own code or a part of the framework/library you work with. When it comes to this, the programmer should know what data or object to expect as a result.
This is where Mock Objects come into play. Mock objects allow you to mimic a part of code and assure that the object provides exactly what you expect, so you don't rely on other (time consuming) services, such as HTTP requests, file stream etc.
You can read more here.
When project becomes complex, the setup takes a fair number of lines and code starts duplicating. Solution to this are Setup and TearDown methods. The naming convention differs from framework to framework, Setup might be called beforeEach or TestInitialize and TearDown can also appear as afterEach or TestCleanup. Names for NUnit, MSTest and xUnit.net can be found on xUnit.net codeplex page.
A simple example application:
it should read a config file
it should verify if config file is valid
it should update user's config
The way I would go about building and testing this:
have a method to read config and second one to verify it
have a getter/setter for user's settings
test read method if it returns desired result (object, string or however you've designed it)
create mock config which you're expecting from read method and test if method accepts it
at this point, you should create multiple mock configs, which test all possible scenarios to see if it works for all possible scenarios and fix it accordingly. This is also called code coverage.
create mock object of accepted config and use the setter to update user's config, then use to check if it was set correctly
This is a basic principle of Test-Driven Development (TDD).
If the test suite is set up as described and all tests pass, all these parts, connected together, should work perfectly. Additional test, for example End-to-End (E2E) testing isn't necessarily needed, I use them only to assure that whole application flow works and to easily catch the error (e.g. http connection error).