I am using WPToolkitTestFX in Windows Phone 8 project. So I am running tests
with special page, where in constructor I got:
this.Content = UnitTestSystem.CreateTestPage();
But if I want to run application, I need to change Navigation Page in WMAppManifest.xml. It is not good, because I often forgetting to change it back and pushing that WMAppManifest to source control system.
Is it possible to create difference run configuration for WP8 project? One for application, another for tests?
Put your unit tests in a different project with its own WMAppManifest.
Related
I have an application where there is a mess of code that has a bunch of non-isolated components. This makes things difficult in terms of doing some unit testing. So along with some unit tests in their own separate test DLL, I'm trying to also create some tests within the application DLL. The application DLL is normally invoked from an application EXE.
For some background, this code is 20+ years old written in native C++. I cannot execute the tests in the DLL directly as the framework is not setup, so any calls executed within the DLL will not execute correctly. I've unsuccessfully tried to do this already, but maybe I need a more fundamental understanding of the MFC framework to do this.
A colleague suggested that maybe it might be possible to have the vstest.console somehow run the tests through the EXE where the framework can be brought up, run the tests through the EXE, which are then forwarded to the DLL, and then have the test results returned back through the EXE to vstest.console, effectively making the EXE a proxy of sorts.
I'm thinking that this might be a longshot, but I'm at a loss as how I can run the tests in the DLL properly. Could it be done? Is there a better way?
For legacy EXE, you may use Generic Test (for console app) or Coded UI Test (for GUI app). Technically, Generic Test or Coded UI Test is System Level Test. You can still get Code Coverage for the two tests.
More on Generic Test
Use a generic test to wrap a console app or test harness console that
• Runs from a command line
• Returns error code: 0 <- Pass; Nonzero <- Fail
• Positive tests only for a console app; A test harness may include negative tests within.
Visual Studio treats generic test just like other tests, but
• Add Generic Tests into Unit Test Project type
• Command line must run .GenericTest file instead of UnitTest1.dll
• vstest.console GenericTest1.GenericTest
NOTE: Set the “run duration” long enough for your EXE.
The code I'm unit testing refers to an appsetting in the app.config file. To cater for this, I've added an app.config file to my unit tests project. If I click the "Run All Tests" icon in the Unit Test Sessions window, all my tests pass.
I have mapped the "ReSharper.ReSharper_UnitTest_RunSolution" command to Ctrl+Shift+Alt+U. If I run the tests by pressing this combination, the tests all run, but they fail to find the appsetting, which comes through as null.
I'm assuming this means that the button click runs under the context of the test project, whilst the command does not, but I can't quite work out what the command is doing.
Have I mapped the wrong command?
EDIT 1: I've also tried using the keyboard shortcut Alt-RUN (Resharper > Unit Tests > Run All), as well as clicking the menus manually, and found that this also causes all unit tests to not find the appsetting and therefore fail. Clicking the Run All Tests icon in Unit Test Sessions (the double green arrow) continues to work fine.
EDIT 2: I realised I should probably be mocking a separate class that fetches appsettings from the config file anyway, so this is what I'm now doing. So now there is no dependence on the config file when unit testing.
There are two things going on here. Firstly, the Run All Tests icon in the sessions window runs all tests in the session, while the Run All Tests menu item runs all tests in the solution. Slightly confusing that they've got the same name, but it does make sense given the context. This is why they give different results.
Secondly, when running all tests in a solution, the app setting might not get found. This is due to an optimisation the test runner makes which runs all tests in the same AppDomain. This avoids the overhead of creating a new AppDomain for each assembly, but has the downside that only one app.config will be used for all assemblies. If it picks the wrong one, your app setting is lost.
You can disable this by unchecking ReSharper » Options » Unit Testing » "Use separate AppDomain for each assembly with tests". Ideally, it should be disabled if any project has an app.config - I've added a feature request you can vote for and track: https://youtrack.jetbrains.com/issue/RSRP-428958
I'm new to Xcode (and Macs in general) and am trying to port some of my code base over to run on both OS X and iOS. I have a large set of unit tests written against the Google C++ Testing Framework (Google Test). I successfully compiled the framework and I can run some tests, but I'm unsure how to view the colorized output from within Xcode.
I'm used to hitting "Run" in Visual Studio and immediately seeing a console window (with colors) letting me know at a glance if the tests passed or failed.
I've managed to set up a "Run Script" "Build Phase" but that seems to only output to the Log Navigator which obliterates the colors and even the fixed-width output making it very difficult to see at a glance if the tests pass. I also can't find a way to display the log after running the tests. When I do this nothing appears in the "All Output" window.
I've played around with XcodeColors but that doesn't seem to work with scripts that use the ANSI color codes.
At this point I wouldn't be surprised if this simply can't be done within Xcode. That would be ideal, but if it isn't, is it possible to create a "Run Script" that will run the tests in an independent Terminal window? Colors work fine there.
Thanks for any help!
Here are links to a tool that colorizes the text in the Log window. It's free and the source is in github so you can figure out how it works. The first link says that it just uses simple ANSI codes to do the job.
http://deepitpro.com/en/articles/XcodeColors/info
https://github.com/robbiehanson/XcodeColors#readme
To kick off the execution from within Xcode, you will probably need to add a new target to your project. To add a Target, click on your project and then there is an Add Target button on the bottom of the screen. I don't know exactly what you're executing but here are my best guesses based on your question:
MacOSX/Application/Cocoa-AppleScript or Command Line Tool - Create a simple script or program that will execute your units tests.
MacOSX/Other/External Build System - Allows for execution of an external "make" program with args.
Once you have a way to execute your unit tests, you just need to figure out how to route the output from the unit tests to the Log window. If you can edit the Google Test project and make it use NSLog(), that would seem to be the easiest solution. You could create your own logging method, perform the ANSI colorization, and then send the final text to NSLog().
ADDED: OK. Interesting findings... Read all before starting. Here's what to do:
Start AppleScript Editor. This is in LaunchPad. Paste the following script into it:
tell application "Terminal"
activate
do script "<your commands>" in window 1
end tell
You can repeat the "do script" line as needed. Use this to execute your unit tests. In Script Editor, do Save As.../File Format=Script and save it to a safe location for now like your Documents directory. This will create a file like "UnitTests.scpt".
Now go to your iOS project. Select the project at the top-left. Select the Build Phases tab top-middle. Click the Add Build Phase button on the bottom right. Here's the interesting part.
Leave Shell as is ("/bin/sh"). Add one line:
osascript ~/Documents/UnitTests.scpt
That will execute the script after every build.
But here's the interesting part I found. Click on Build Settings (top-middle). Make sure All is selected (not Basic). Scroll down the list to find Unit Testing. Open Test Host. Hit the + next to Debug. You can also put the above osascript command here. You might be able to execute your unit tests here and if you can, the output will likely show up in the Log! Let me know what happens.
I am familiar in Java: JUnit + JCodecoverage, at mobile applications: Android and iPhone I was to lazy to develop with TDD, but if I would like to start than :
I would create a Hello Word app, with JUnitTesting options turned on:
Include Unit Test checked
That will create a Test App / target whatever, and you will be able to run that.
The same thing it is at Android too: you have to create a "test project"
Once I did and forgot how is working, but, there are other stuff too:
- long press the Play button on Xcode ( 4.4 ) and you will have a dropdown menu with: Run, Test, Profile,Analyze.
I can't present those, because if I press the Shift+ Cmd + 4 to screenshot it it is changing, but here it look like the changed menu:
IMHO: for banking, forex, other financial or military (high security software) I would use test driven development, with over 99% code coverage, but those simple 3-4 web-service call mobile apps, which display public data available in browsers are just waste of time to develop tests and upkeep it!
Many times I need to test with internet connection and without.
to be worse case with WI-FI connection , but router doesn't give IP or let go out the phone, but if I ask the phone state: it is connected...
The GUI flow hard to test from unit testing, where is / would be usefully for me: the data got from web-service and synchronization with internal cache. As I see it is still cheaper to do it with manu testing.
I'm debugging a set of WCF services. Initially, I created some unit tests, but since I'm using threading I often receive "Aborted" or "Stopped" tests without any clear explanation why (this is a known bug in Visual Studio).
I found it extremely challenging to debug the services when I can't even read the log output, so I quickly wrote a custom Assert class and converted all unit tests to console applications. This way, I was able to fix a huge number of simple problems immediately that were hard to impossible before.
So I'm wondering if it is a good idea to write unit tests as (fully automated) console apps first and convert them to real (executes when launching unit tests in VS) tests later.
if you want to stick to the stand alone console app you can have a one fits all aproach: Change
the application type of the MsUnitTest (or NUnitTest) to "Console application"
add a public static void Main() that call your unittests you are interested in.
This exe is can run on its own or it runs in the unittest-ide.
I prefer a standalone consolerunner as described in how-do-i-use-mstest-without-visual-studio
I'm just starting to use QTestLib. I have gone through the manual and tutorial. Although I understand how to create tests, I'm just not getting how to make those tests convenient to run. My unit test background is NUnit and MSTest. In those environments, it was trivial (using a GUI, at least) to alternate between running a single test, or all tests in a single test class, or all tests in the entire project, just by clicking the right button.
All I'm seeing in QTestLib is either you use the QTEST_MAIN macro to run the tests in a single class, then compile and test each file separately; or use QTest::qExec() in main() to define which objects to test, and then manually change that and recompile when you want to add/remove test classes.
I'm sure I'm missing something. I'd like to be able to easily:
Run a single test method
Run the tests in an entire class
Run all tests
Any of those would call the appropriate setup / teardown functions.
EDIT: Bounty now available. There's got to be a better way, or a GUI test runner that handles it for you or something. If you are using QtTest in a test-driven environment, let me know what is working for you. (Scripts, test runners, etc.)
You can run only selected test cases (test methods) by passing test names as command line arguments :
myTests.exe myCaseOne myCaseTwo
It will run all inits/cleanups too. Unfortunately there is no support for wildcards/pattern matching, so to run all cases beginning with given string (I assume that's what you mean by "running the tests in an entire class"), you'd have to create script (windows batch/bash/perl/whatever) that calls:
myTests.exe -functions
parses the results and runs selected tests using first syntax.
To run all cases, just don't pass any parameter:
myTests.exe
The three features requested by the OP, are nowadays integrated in to the Qt Creator.
The project will be automatically scanned for tests and they apear on the Test pane. Bottom left in the screenshot:
Each test and corresponding data can be enabled by clicking the checkbox.
The context menu allows to run all tests, all tests of a class, only the selected or only one test.
As requested.
The test results will be available from the Qt Creator too. A color indicator will show pass/fail for each test, along additional information like debug messages.
In combination with the Qt Creator, the use of the QTEST_MAIN macro for each test case will work well, as each compiled executable is invoked by the Qt Creator automatically.
For a more detailed overview, refer to the Running Autotests section of the Qt Creator Manual.