I followed the steps from Specflow Getting Started but the Test Explorer in VS2017 shows only "SpecFlow Evaluation" instead of the tests.
https://specflow.org/getting-started/
Here is the Test Output window:
Unable to fetch symbols for class: ...SpecRun Evaluation
Unable to fetch source Information for test method: ...::SpecRun Evaluation.Execution delay for SpecRun evaluation mode contained in project: ...
Update: I installed a Trial license key for SpecFlow+ and now the Test Explorer shows no tests at all, not even "SpecFlow Evaluation".
Related
This question is related to the problem where my unit test procedure is crashing but I don't know on which unit test.
I realise this question is a duplicate of How can I find out which test method in a batch of test methods fails to run? but I need to try anyway:
I'm running some unit tests (about 118 of them), but one of them seems to make the Visual Studio unit test environment crash. This is what I see in the "output" window's "Tests" tab:
[5/02/2018 11:13:18 Informational] ------ Run test started ------
[5/02/2018 11:13:38 Error] The active Test Run was aborted because the execution process exited unexpectedly. The test execution process crashed while running the tests. To investigate further, open file:///C:/Users/DominiqueDS/AppData/Local/CrashDumps/vstest.executionengine.x86.exe.19136.dmp file in Visual Studio and choose "Debug in mixed mode".
[5/02/2018 11:13:38 Informational] ========== Run test finished: 65 run (0:00:19,6926824) ==========
The reason that I have that dump file is thanks to my Windows configuration which automatically creates such dumpfiles in case of a crashing application. (The procedure for this configuration is found under this URL)
Examining the dump file gives me more information on the test, which is failing: I have a hunch where I can find it, and in another thread, I find a function, calling a method which seems to contain the name of the unit test I'm running.
Although I know which test is failing, it's based on hunches and on dumpfiles I should even not generate, so I'm looking for another solution:
In the function TEST_METHOD in the file "c:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Auxiliary\VS\UnitTest\include\CppUnitTest.h", I've tried to add the following lines:
cout << "Test : ";\
cout << methodName;\
cout << "\n";\
I was hoping that this would show every test I wanted to run, but instead I get the error message that this is wrong (also using std::cout and OutputDebugString() function seems not to be allowed).
Hence my question: does anybody know a way to find out, in case one of my unit tests makes my test environment crash, how can I find out which test this is, without needing to generate and debug the Visual Studio dumpfile?
For your information: looking in the "Test Explorer" is not helping: I have a list of disabled tests, a list of successful ones, and a list of some which are not finished, and my failing test is not the first of the "not finished" ones.
Found it: I've added the following line in the definition of TEST_METHOD():
Microsoft::VisualStudio::CppUnitTestFramework::Logger::WriteMessage(L#methodName);\
I am using Nunit4TeamBuild to try and publish the results of some unit tests back to TFS 2010.
When I try to publish a new test results XML the build is successful and it says the test results have been published. However there are no test results anywhere to be found in the build summary/log.
It even says "No Test Results" in the build summary beneath the compilation step.
So even though it was successful, it doesn't look like the test results actually get published.
But if I run another build definition that just takes a test result XML and publish it, I get this error:
Test run already uploaded.
So it would seem that the test results were, in fact, published to TFS. But where? How can I see them? Why don't they show up in the build summary/log like it's meant to? I cannot find the results anywhere.
I'm using tfs 2017 to run CodedUI ordered unit tests. These are my build steps:
These are my "Run Functional Tests" configurations:
And these are the "Publish Test Results" (I'm not sure they are correct):
The TestAgent is deployed and the tests are running fine, The problem is that the test results appear as only one result and I can't see detailed result for each test. This is how my tests results looks like (The attachments are screen shots I take for each test):
I can reproduce your situation and TFS will treat the tests as one in the log:
DistributedTests: Total Tests : 1, Passed Tests : 0
It's a know issue, please refer below link:
How to display test results of individual test in the ordered test
suite in TFS Web
This is a limitation of the run functional test task. You can
publish the .trx file using the "publish test results" task and it
will show you all the tests but you wont know which ".orderedtest"
they were associated with etc.
You need either open *.trx file in Visual Studio or use Publish Test Results task(need to check continue on error).
Besides also change the outcome from failed to all in the test result page.
I am new to testing in Xcode. I am following the developer documentation "iOS Developement Guide" Unit Testing Applications.
I have successfully added unit test bundles to my application with one failing test.
When I build the test target the output shows 1 error as expected but the editor does not show the expected error message under the failed test.
Does anyone know any configurations that need to be set to enable this feature?
Apples docs say:
"If the unit-test bundle is configured correctly, the build fails and Xcode displays an error message in the text editor."
TIA
Vital Clue:
- One thing I have noticed in my output window is the message "Command /bin/sh failed with exit code 1"
Try to follow this guide
Be double careful about when to set application target, and when to set Unit test target. It works fine for me though.
If Unit Test fail, there should be some messages shown in the console.
I have a strange problem,
my MSBuild runs tests, code-coverage and publishing fine (part of the build.txt shown):
Results Top Level Tests
------- ---------------
Passed BuildTestProject.UnitTest1.TestMethod1
Passed BuildTestProject.UnitTest1.TestMethod2
2/2 test(s) Passed
...
Results file: W:\BuildWorkspace\XXX\Test Release\TestResults\XXX_XXX 2009-08-20 11_47_09_Any CPU_Release.trx
Run Configuration: Local Test Run
Waiting to publish...
Publishing results of test run XXX#XXX 2009-08-20 11:47:09_Any CPU_Release to http://XXX:8080/Build/v1.0/PublishTestResultsBuildService2.asmx...
....Publish completed successfully.
When I import these testresults on my local machine i get to see the code coverage-data as expected. But the code-coverage details are not shown in the details of the build that Visual Studio shows when you expand the 'results details'.
Any tips?
Finally found the solution today, found out that my TeamFoundationServer itself had problems with publishing the results to my buildAgent. I read somewhere (after searching for CoverAn.exe) that it is installed as a service, then i checked the credentials for the service 'Code Coverage Analysis Service'.
These were running under TFSMachine\NETWORK instead of our service-account for the TeamFoundationServer. Changed this, reran a build with tests in it and it now publishes the results.
W00t!
Check that the .testrunconfig used by the server has coverage enabled. It may be different than your local .testrunconfig.