I'm writing and running my tests on VS2010 with mstest. When I want to run a test, or entire tests in a class, sometimes Visual Studio decides to run all my tests.
Why is that? How can I avoid it? Any tips?
May be it is this bug?
http://connect.microsoft.com/VisualStudio/feedback/details/524288/ctrl-r-t-runs-all-tests-the-first-time
Might this be the same problem VS2008 seems to have, where if you use run context (Ctrl+R [Ctrl+]T) or run class (Ctrl+R [Ctrl+]C) it acts as if you used run all (Ctrl+R [Ctrl+]A) the first time after opening a solution? Or at least, it seems so to me. In VS2008 I simply stop the test run when I see this happening and restart the same operation. Then it usually works "as advertised".
No idea if and how this might be relevant to VS2010 at all at this stage...
right click in the body of a single test and select 'run tests' should only run that test
I have the same problem.
My temporary solution at the moment is:
Open Test List Editor.
Type in your test method name in the filter box and filter it.
Check the tests that you want to run.
Right click and select "Run checked tests"
But I prefer the "should-run" as advertised if it works of course. I'm open to other better solutions.
Not quite sure, if this applies to VS2010. In VS2008 running tests in the current context sometimes runs all tests in the solution. It seems, that context is defined by what element was last activated (usually through mouse click). If the last window selected was the code windows containing the unit test source code, then all tests in that class/file are executed. Same with selecting a (unit test-)file in the solution explorer. If however, another windows (e.g. breakpoints) was last selected, then the context is the whole solution.
Related
On running an individual test in VS2012, a window is shown at the bottom of Test Explorer that includes (assuming failure) a red icon with "Test Failed" next to it. There follows the failure message with "elapsed time" directly beneath.
I would like to know simply whether there is a way to clear this window. For instance if I right-click my test and select "Debug Selected Tests", it is somewhat confusing as I step through the test that this test-results window still shows the failure from a past test-run.
Actually, there is a way - clean and then rebuild your solution.
Previous test run results will clear right up.
Switch the build to a different configuration - eg if in debug, switch to release. Then switch back to debug. This should cause it reload the tests. If vs fails to reload the tests on switching back - just do a build (not a rebuild) as this will trigger it to reload the tests
You can't do that. Instead, you can filter which tests are shown to you, if that suits your needs. You can find more information about tests here: http://msdn.microsoft.com/en-us/library/hh270865.aspx
You could make your tests sleep for a second when invoked:
[ClassInitialize()]
public static void Init(TestContext ctx)
{
System.Threading.Thread.Sleep(1000);
}
This is only a workaround, but it will at least give you the chance to see the Test Explorer progress bar moving, and won't leave you wondering whether anything has actually happened:
In VS 2019 you can unload/reload the project with the tests in to clear the results.
Delete the folder that contains the test results. This will have different names depending on which test suite you use.
The easiest way is to search all of the files under your solution to look for the name(s) of the test(s) that you want to remove. In my case there's a directory called TestStore. I delete that and my test explorer is now empty. Running Clean on your solution will also clear this directory.
There is also a sanctioned method that removes the test runs via the UI but I find it way to cumbersome. But you may like it. How to: Delete Test Results
How do I stop a running MSTest unit test in VisualStudio 2012?
This seems silly, but I can't seem to find a way. I accidentally wrote a unit test that gets stuck at some point and waits forever.
there IS a Cancel/Cancel all button in the TestExplorer window.
(There is a new row of buttons between the search bar and the list of tests, which apparently is easily overlooked.)
Thanks Tyler!
I'm just starting to use QTestLib. I have gone through the manual and tutorial. Although I understand how to create tests, I'm just not getting how to make those tests convenient to run. My unit test background is NUnit and MSTest. In those environments, it was trivial (using a GUI, at least) to alternate between running a single test, or all tests in a single test class, or all tests in the entire project, just by clicking the right button.
All I'm seeing in QTestLib is either you use the QTEST_MAIN macro to run the tests in a single class, then compile and test each file separately; or use QTest::qExec() in main() to define which objects to test, and then manually change that and recompile when you want to add/remove test classes.
I'm sure I'm missing something. I'd like to be able to easily:
Run a single test method
Run the tests in an entire class
Run all tests
Any of those would call the appropriate setup / teardown functions.
EDIT: Bounty now available. There's got to be a better way, or a GUI test runner that handles it for you or something. If you are using QtTest in a test-driven environment, let me know what is working for you. (Scripts, test runners, etc.)
You can run only selected test cases (test methods) by passing test names as command line arguments :
myTests.exe myCaseOne myCaseTwo
It will run all inits/cleanups too. Unfortunately there is no support for wildcards/pattern matching, so to run all cases beginning with given string (I assume that's what you mean by "running the tests in an entire class"), you'd have to create script (windows batch/bash/perl/whatever) that calls:
myTests.exe -functions
parses the results and runs selected tests using first syntax.
To run all cases, just don't pass any parameter:
myTests.exe
The three features requested by the OP, are nowadays integrated in to the Qt Creator.
The project will be automatically scanned for tests and they apear on the Test pane. Bottom left in the screenshot:
Each test and corresponding data can be enabled by clicking the checkbox.
The context menu allows to run all tests, all tests of a class, only the selected or only one test.
As requested.
The test results will be available from the Qt Creator too. A color indicator will show pass/fail for each test, along additional information like debug messages.
In combination with the Qt Creator, the use of the QTEST_MAIN macro for each test case will work well, as each compiled executable is invoked by the Qt Creator automatically.
For a more detailed overview, refer to the Running Autotests section of the Qt Creator Manual.
I have noticed that if I have a set of regression tests and decide to change a property on one of my objects (DTO) from int to decimal for example - i make all the other changes and the tests pass like normal. But if this project is under source control (VSS specifically) this small change will cause something strange to happen...
Similar to this question
Testing in Visual Studio Succeeds Individually, Fails in a Set
But a little different. I can make this change, and try to run my tests and any test that has an assert around this new data type will fail, but if I then click "debug checked tests" and it then runs through the previously failed tests - they pass. No changes to the test code /etc
Does anyone know why this might be happening? I hate to work outside of source control but if my tests are not reliable ... why have them at all in this case ... and I live for testing code :P
Given the age of the question, I doubt it's still an issue for you, but I wonder if you have a bin or obj folders under source control or an assembly that is in them?
If they are then when you compile the app (before MSTest runs) the source controlled assemblies are going to be in read-only mode and won't get overridden by the compiler and thus your tests will be against out of date binaries.
I'm trying to setup a post-build event in .NET 3.5 that will run a suite of unit tests w/ MS test. I found this post that shows how to call a bat file using MbUnit but I'm wanting to see if anyone has done this type of thing w/ MS Test?
If so, I would be interested in a sample of what the bat file would look like
We were using NUnit in the same style and decided to move to MSTest. When doing so, we just added the following to our Post-Build event of the applicable MSTest project:
CD $(TargetDir)
"$(DevEnvDir)MSTEST.exe" /testcontainer:$(TargetFileName)
The full set of MSTest command line options can be found at the applicable MSDN site.
Personally I would not recomment running unit tests as a part of the compilation process. Instead, consider something like ReSharper (+ appropriate Unit Test Runner or how do they call these nowadays) or some other GUI runner.
Instead of a doing it in a post build event, that will happen every time you compile, I would look at setting up a Continuous Integration Server like CruiseControl.Net. It'll provide you a tight feedback cycle, but not block your work with running tests every time you build your application.
If you are wanting to run the set of tests you are currently developing, Anton's suggestion of using ReSharper will work great. You can create a subset of tests to execute when you wish and it's smart enough to compile for you if it needs to. While you're there picking up the demo, if you don't already have a license, pick up Team City. It is another CI server that has some promise.
If you are wanting to use this method to control the build quality, you'll probably find that as the number of tests grow, you no longer want to wait for 1000 tests to run each time you press F5 to test a change.