I regularly use right mouse button > "Run/Debug Grails Command..." from within NetBeans.
When I do so, it's cumbersome b/c I have to wait for "Reloading Grails commands...", then I have to choose the command and manually type all parameters e.g. "unit:spock -coverage ExampleController".
I'll have to compose the commands everytime I restart NetBeans.
Is there a better solution to this?
Also, everytime I run "test-app" complete Grails restarts - is it possible to leave Grails running and just execute the tests in question via a click again, and again, and again ... ?
Thanks to the help of david, I can now answer my question #2:
When clicking right mouse button > "Run/Debug Grails Command..." from within NetBeans, simply double-click "interactive" from the list.
Then, in the new shell, type the test you want to run without grails. E.g. only test-app unit:spock -coverage ExampleController
Everytime you want to execute the test again just hit ENTER within the shell/console.
Note that Grails won't be able to handle certain changes correctly. In this case you willl most likely see unexpected exceptions and the like.
If that happens, just close the shell, clean the project, rinse & repeat.
Related
Context: I have an acceptance test for my ember-cli application, and the test passes just fine in Chrome. However, in phantomjs, my test fails -- the UI doesn't get created the same way, and I'm trying to work out why. (I think the test is broken because of https://github.com/ember-cli/ember-cli/issues/1763, but the general question of how to debug remains)
In Chrome, I can use the standard debugging tools on my tests and all is well -- but in phantomjs, I can't get at it with a debugger. I also don't see console.log() messages show up in the output -- all I get is a list of test results in my terminal window.
I can sort-of get diagnostic info by writing things like
equal(true, false, "This is a log message");
and then I get the message as details for the assertion that failed, or I can try and work out what's in the DOM with
equal(true, false, document.getElementsByClassName("my-class".innerHTML);
but both of those a:stop the test going any further, and b:only let me log information from the test itself, not my application.
Is there a way to run my tests outside of "ember test", or some way to attach to the running test processes? Alternatively, is there a way to get console.log() messages to show up in the output?
You can expose PhantomJS debug port and open it in browser then you can interact with context at your debugger breakpoints.
Debugging tests on PhantomJS using Testem test runner
In testem.json add "phantomjs_debug_port": 9000.
While you run your tests visit http://localhost:9000 in your browser and click the long link that shows up.
Source: cssugared
I had no luck with the other answers, so here's what I found out:
Add a return pauseTest(); at the point in your test where you want to be able to interact with the container in the browser. This is in the docs but I'm not sure it's in the guides.
To answer the part of my original question about "how do I get log messages to show up", if I use the TAP reporter, then console.log (in my app and in my tests) messages show up in the output; the xunit reporter doesn't pass console.log on, which was confusing me.
(I've also hit issues where running the tests on teamcity behaves differently than running locally; in that situation, combining the TAP reporter with https://github.com/aghassemi/tap-xunit (or the TAP teamcity plugin) lets me get log messages and also test counts)
The code I'm unit testing refers to an appsetting in the app.config file. To cater for this, I've added an app.config file to my unit tests project. If I click the "Run All Tests" icon in the Unit Test Sessions window, all my tests pass.
I have mapped the "ReSharper.ReSharper_UnitTest_RunSolution" command to Ctrl+Shift+Alt+U. If I run the tests by pressing this combination, the tests all run, but they fail to find the appsetting, which comes through as null.
I'm assuming this means that the button click runs under the context of the test project, whilst the command does not, but I can't quite work out what the command is doing.
Have I mapped the wrong command?
EDIT 1: I've also tried using the keyboard shortcut Alt-RUN (Resharper > Unit Tests > Run All), as well as clicking the menus manually, and found that this also causes all unit tests to not find the appsetting and therefore fail. Clicking the Run All Tests icon in Unit Test Sessions (the double green arrow) continues to work fine.
EDIT 2: I realised I should probably be mocking a separate class that fetches appsettings from the config file anyway, so this is what I'm now doing. So now there is no dependence on the config file when unit testing.
There are two things going on here. Firstly, the Run All Tests icon in the sessions window runs all tests in the session, while the Run All Tests menu item runs all tests in the solution. Slightly confusing that they've got the same name, but it does make sense given the context. This is why they give different results.
Secondly, when running all tests in a solution, the app setting might not get found. This is due to an optimisation the test runner makes which runs all tests in the same AppDomain. This avoids the overhead of creating a new AppDomain for each assembly, but has the downside that only one app.config will be used for all assemblies. If it picks the wrong one, your app setting is lost.
You can disable this by unchecking ReSharper » Options » Unit Testing » "Use separate AppDomain for each assembly with tests". Ideally, it should be disabled if any project has an app.config - I've added a feature request you can vote for and track: https://youtrack.jetbrains.com/issue/RSRP-428958
On running an individual test in VS2012, a window is shown at the bottom of Test Explorer that includes (assuming failure) a red icon with "Test Failed" next to it. There follows the failure message with "elapsed time" directly beneath.
I would like to know simply whether there is a way to clear this window. For instance if I right-click my test and select "Debug Selected Tests", it is somewhat confusing as I step through the test that this test-results window still shows the failure from a past test-run.
Actually, there is a way - clean and then rebuild your solution.
Previous test run results will clear right up.
Switch the build to a different configuration - eg if in debug, switch to release. Then switch back to debug. This should cause it reload the tests. If vs fails to reload the tests on switching back - just do a build (not a rebuild) as this will trigger it to reload the tests
You can't do that. Instead, you can filter which tests are shown to you, if that suits your needs. You can find more information about tests here: http://msdn.microsoft.com/en-us/library/hh270865.aspx
You could make your tests sleep for a second when invoked:
[ClassInitialize()]
public static void Init(TestContext ctx)
{
System.Threading.Thread.Sleep(1000);
}
This is only a workaround, but it will at least give you the chance to see the Test Explorer progress bar moving, and won't leave you wondering whether anything has actually happened:
In VS 2019 you can unload/reload the project with the tests in to clear the results.
Delete the folder that contains the test results. This will have different names depending on which test suite you use.
The easiest way is to search all of the files under your solution to look for the name(s) of the test(s) that you want to remove. In my case there's a directory called TestStore. I delete that and my test explorer is now empty. Running Clean on your solution will also clear this directory.
There is also a sanctioned method that removes the test runs via the UI but I find it way to cumbersome. But you may like it. How to: Delete Test Results
I'm new to Xcode (and Macs in general) and am trying to port some of my code base over to run on both OS X and iOS. I have a large set of unit tests written against the Google C++ Testing Framework (Google Test). I successfully compiled the framework and I can run some tests, but I'm unsure how to view the colorized output from within Xcode.
I'm used to hitting "Run" in Visual Studio and immediately seeing a console window (with colors) letting me know at a glance if the tests passed or failed.
I've managed to set up a "Run Script" "Build Phase" but that seems to only output to the Log Navigator which obliterates the colors and even the fixed-width output making it very difficult to see at a glance if the tests pass. I also can't find a way to display the log after running the tests. When I do this nothing appears in the "All Output" window.
I've played around with XcodeColors but that doesn't seem to work with scripts that use the ANSI color codes.
At this point I wouldn't be surprised if this simply can't be done within Xcode. That would be ideal, but if it isn't, is it possible to create a "Run Script" that will run the tests in an independent Terminal window? Colors work fine there.
Thanks for any help!
Here are links to a tool that colorizes the text in the Log window. It's free and the source is in github so you can figure out how it works. The first link says that it just uses simple ANSI codes to do the job.
http://deepitpro.com/en/articles/XcodeColors/info
https://github.com/robbiehanson/XcodeColors#readme
To kick off the execution from within Xcode, you will probably need to add a new target to your project. To add a Target, click on your project and then there is an Add Target button on the bottom of the screen. I don't know exactly what you're executing but here are my best guesses based on your question:
MacOSX/Application/Cocoa-AppleScript or Command Line Tool - Create a simple script or program that will execute your units tests.
MacOSX/Other/External Build System - Allows for execution of an external "make" program with args.
Once you have a way to execute your unit tests, you just need to figure out how to route the output from the unit tests to the Log window. If you can edit the Google Test project and make it use NSLog(), that would seem to be the easiest solution. You could create your own logging method, perform the ANSI colorization, and then send the final text to NSLog().
ADDED: OK. Interesting findings... Read all before starting. Here's what to do:
Start AppleScript Editor. This is in LaunchPad. Paste the following script into it:
tell application "Terminal"
activate
do script "<your commands>" in window 1
end tell
You can repeat the "do script" line as needed. Use this to execute your unit tests. In Script Editor, do Save As.../File Format=Script and save it to a safe location for now like your Documents directory. This will create a file like "UnitTests.scpt".
Now go to your iOS project. Select the project at the top-left. Select the Build Phases tab top-middle. Click the Add Build Phase button on the bottom right. Here's the interesting part.
Leave Shell as is ("/bin/sh"). Add one line:
osascript ~/Documents/UnitTests.scpt
That will execute the script after every build.
But here's the interesting part I found. Click on Build Settings (top-middle). Make sure All is selected (not Basic). Scroll down the list to find Unit Testing. Open Test Host. Hit the + next to Debug. You can also put the above osascript command here. You might be able to execute your unit tests here and if you can, the output will likely show up in the Log! Let me know what happens.
I am familiar in Java: JUnit + JCodecoverage, at mobile applications: Android and iPhone I was to lazy to develop with TDD, but if I would like to start than :
I would create a Hello Word app, with JUnitTesting options turned on:
Include Unit Test checked
That will create a Test App / target whatever, and you will be able to run that.
The same thing it is at Android too: you have to create a "test project"
Once I did and forgot how is working, but, there are other stuff too:
- long press the Play button on Xcode ( 4.4 ) and you will have a dropdown menu with: Run, Test, Profile,Analyze.
I can't present those, because if I press the Shift+ Cmd + 4 to screenshot it it is changing, but here it look like the changed menu:
IMHO: for banking, forex, other financial or military (high security software) I would use test driven development, with over 99% code coverage, but those simple 3-4 web-service call mobile apps, which display public data available in browsers are just waste of time to develop tests and upkeep it!
Many times I need to test with internet connection and without.
to be worse case with WI-FI connection , but router doesn't give IP or let go out the phone, but if I ask the phone state: it is connected...
The GUI flow hard to test from unit testing, where is / would be usefully for me: the data got from web-service and synchronization with internal cache. As I see it is still cheaper to do it with manu testing.
I'm just starting to use QTestLib. I have gone through the manual and tutorial. Although I understand how to create tests, I'm just not getting how to make those tests convenient to run. My unit test background is NUnit and MSTest. In those environments, it was trivial (using a GUI, at least) to alternate between running a single test, or all tests in a single test class, or all tests in the entire project, just by clicking the right button.
All I'm seeing in QTestLib is either you use the QTEST_MAIN macro to run the tests in a single class, then compile and test each file separately; or use QTest::qExec() in main() to define which objects to test, and then manually change that and recompile when you want to add/remove test classes.
I'm sure I'm missing something. I'd like to be able to easily:
Run a single test method
Run the tests in an entire class
Run all tests
Any of those would call the appropriate setup / teardown functions.
EDIT: Bounty now available. There's got to be a better way, or a GUI test runner that handles it for you or something. If you are using QtTest in a test-driven environment, let me know what is working for you. (Scripts, test runners, etc.)
You can run only selected test cases (test methods) by passing test names as command line arguments :
myTests.exe myCaseOne myCaseTwo
It will run all inits/cleanups too. Unfortunately there is no support for wildcards/pattern matching, so to run all cases beginning with given string (I assume that's what you mean by "running the tests in an entire class"), you'd have to create script (windows batch/bash/perl/whatever) that calls:
myTests.exe -functions
parses the results and runs selected tests using first syntax.
To run all cases, just don't pass any parameter:
myTests.exe
The three features requested by the OP, are nowadays integrated in to the Qt Creator.
The project will be automatically scanned for tests and they apear on the Test pane. Bottom left in the screenshot:
Each test and corresponding data can be enabled by clicking the checkbox.
The context menu allows to run all tests, all tests of a class, only the selected or only one test.
As requested.
The test results will be available from the Qt Creator too. A color indicator will show pass/fail for each test, along additional information like debug messages.
In combination with the Qt Creator, the use of the QTEST_MAIN macro for each test case will work well, as each compiled executable is invoked by the Qt Creator automatically.
For a more detailed overview, refer to the Running Autotests section of the Qt Creator Manual.