Context: I have an acceptance test for my ember-cli application, and the test passes just fine in Chrome. However, in phantomjs, my test fails -- the UI doesn't get created the same way, and I'm trying to work out why. (I think the test is broken because of https://github.com/ember-cli/ember-cli/issues/1763, but the general question of how to debug remains)
In Chrome, I can use the standard debugging tools on my tests and all is well -- but in phantomjs, I can't get at it with a debugger. I also don't see console.log() messages show up in the output -- all I get is a list of test results in my terminal window.
I can sort-of get diagnostic info by writing things like
equal(true, false, "This is a log message");
and then I get the message as details for the assertion that failed, or I can try and work out what's in the DOM with
equal(true, false, document.getElementsByClassName("my-class".innerHTML);
but both of those a:stop the test going any further, and b:only let me log information from the test itself, not my application.
Is there a way to run my tests outside of "ember test", or some way to attach to the running test processes? Alternatively, is there a way to get console.log() messages to show up in the output?
You can expose PhantomJS debug port and open it in browser then you can interact with context at your debugger breakpoints.
Debugging tests on PhantomJS using Testem test runner
In testem.json add "phantomjs_debug_port": 9000.
While you run your tests visit http://localhost:9000 in your browser and click the long link that shows up.
Source: cssugared
I had no luck with the other answers, so here's what I found out:
Add a return pauseTest(); at the point in your test where you want to be able to interact with the container in the browser. This is in the docs but I'm not sure it's in the guides.
To answer the part of my original question about "how do I get log messages to show up", if I use the TAP reporter, then console.log (in my app and in my tests) messages show up in the output; the xunit reporter doesn't pass console.log on, which was confusing me.
(I've also hit issues where running the tests on teamcity behaves differently than running locally; in that situation, combining the TAP reporter with https://github.com/aghassemi/tap-xunit (or the TAP teamcity plugin) lets me get log messages and also test counts)
Related
I am using Karma through Grunt. We have around 1000 tests and it is a bit painful to have them all run whenever we change a file (autoWatch = true).
This is what we are doing now:
Start Karma with singleRun=false, autoWatch=false.
Open the debug page and grep for a specific suite (using mocha html reporter).
Change a test or file related to that suite.
Refresh the debug page to run the set of tests again.
My changes in (3) haven't been picked up by Karma so the tests still behave as if nothing had changed.
This is what I need:
Start Karma with singleRun=false, magicOption=true.
Open the debug page and grep for a specific suite (using mocha html reporter).
Change a test or file related to that suite.
Refresh the debug page to run the set of tests again.
My changes are porperly picked up and only the grepped tests are run.
If I set autoWatch=true I get what I need but the whole suite of 1000 tests is run in the background whenever I change a file, which soon collapses my environment.
I don't think there is anything equivalent to magicOption according to Karma docs but, is there any way to achieve the same behaviour?
Thanks a lot.
I have continuous integration on drone.io for my dart projects, normally there aren't any issues with this, except for actual bugs in my code, but my latest tests are all passing and the test suite reports it completed successfully, but the drone.io test runner never exists it just keeps running until it times out and reports build failed. Has anyone else had anything something familiar to this? or no how to fix it? here is the build if you kick off a new build from the big-refactor-and-enhancement branch that is where it has this odd behaviour.
After a quick look at your code, I would bet that the server launched under the cover is not shut down. You should add a close() method on it and call it in _tearDown().
On running an individual test in VS2012, a window is shown at the bottom of Test Explorer that includes (assuming failure) a red icon with "Test Failed" next to it. There follows the failure message with "elapsed time" directly beneath.
I would like to know simply whether there is a way to clear this window. For instance if I right-click my test and select "Debug Selected Tests", it is somewhat confusing as I step through the test that this test-results window still shows the failure from a past test-run.
Actually, there is a way - clean and then rebuild your solution.
Previous test run results will clear right up.
Switch the build to a different configuration - eg if in debug, switch to release. Then switch back to debug. This should cause it reload the tests. If vs fails to reload the tests on switching back - just do a build (not a rebuild) as this will trigger it to reload the tests
You can't do that. Instead, you can filter which tests are shown to you, if that suits your needs. You can find more information about tests here: http://msdn.microsoft.com/en-us/library/hh270865.aspx
You could make your tests sleep for a second when invoked:
[ClassInitialize()]
public static void Init(TestContext ctx)
{
System.Threading.Thread.Sleep(1000);
}
This is only a workaround, but it will at least give you the chance to see the Test Explorer progress bar moving, and won't leave you wondering whether anything has actually happened:
In VS 2019 you can unload/reload the project with the tests in to clear the results.
Delete the folder that contains the test results. This will have different names depending on which test suite you use.
The easiest way is to search all of the files under your solution to look for the name(s) of the test(s) that you want to remove. In my case there's a directory called TestStore. I delete that and my test explorer is now empty. Running Clean on your solution will also clear this directory.
There is also a sanctioned method that removes the test runs via the UI but I find it way to cumbersome. But you may like it. How to: Delete Test Results
I'm new to Xcode (and Macs in general) and am trying to port some of my code base over to run on both OS X and iOS. I have a large set of unit tests written against the Google C++ Testing Framework (Google Test). I successfully compiled the framework and I can run some tests, but I'm unsure how to view the colorized output from within Xcode.
I'm used to hitting "Run" in Visual Studio and immediately seeing a console window (with colors) letting me know at a glance if the tests passed or failed.
I've managed to set up a "Run Script" "Build Phase" but that seems to only output to the Log Navigator which obliterates the colors and even the fixed-width output making it very difficult to see at a glance if the tests pass. I also can't find a way to display the log after running the tests. When I do this nothing appears in the "All Output" window.
I've played around with XcodeColors but that doesn't seem to work with scripts that use the ANSI color codes.
At this point I wouldn't be surprised if this simply can't be done within Xcode. That would be ideal, but if it isn't, is it possible to create a "Run Script" that will run the tests in an independent Terminal window? Colors work fine there.
Thanks for any help!
Here are links to a tool that colorizes the text in the Log window. It's free and the source is in github so you can figure out how it works. The first link says that it just uses simple ANSI codes to do the job.
http://deepitpro.com/en/articles/XcodeColors/info
https://github.com/robbiehanson/XcodeColors#readme
To kick off the execution from within Xcode, you will probably need to add a new target to your project. To add a Target, click on your project and then there is an Add Target button on the bottom of the screen. I don't know exactly what you're executing but here are my best guesses based on your question:
MacOSX/Application/Cocoa-AppleScript or Command Line Tool - Create a simple script or program that will execute your units tests.
MacOSX/Other/External Build System - Allows for execution of an external "make" program with args.
Once you have a way to execute your unit tests, you just need to figure out how to route the output from the unit tests to the Log window. If you can edit the Google Test project and make it use NSLog(), that would seem to be the easiest solution. You could create your own logging method, perform the ANSI colorization, and then send the final text to NSLog().
ADDED: OK. Interesting findings... Read all before starting. Here's what to do:
Start AppleScript Editor. This is in LaunchPad. Paste the following script into it:
tell application "Terminal"
activate
do script "<your commands>" in window 1
end tell
You can repeat the "do script" line as needed. Use this to execute your unit tests. In Script Editor, do Save As.../File Format=Script and save it to a safe location for now like your Documents directory. This will create a file like "UnitTests.scpt".
Now go to your iOS project. Select the project at the top-left. Select the Build Phases tab top-middle. Click the Add Build Phase button on the bottom right. Here's the interesting part.
Leave Shell as is ("/bin/sh"). Add one line:
osascript ~/Documents/UnitTests.scpt
That will execute the script after every build.
But here's the interesting part I found. Click on Build Settings (top-middle). Make sure All is selected (not Basic). Scroll down the list to find Unit Testing. Open Test Host. Hit the + next to Debug. You can also put the above osascript command here. You might be able to execute your unit tests here and if you can, the output will likely show up in the Log! Let me know what happens.
I am familiar in Java: JUnit + JCodecoverage, at mobile applications: Android and iPhone I was to lazy to develop with TDD, but if I would like to start than :
I would create a Hello Word app, with JUnitTesting options turned on:
Include Unit Test checked
That will create a Test App / target whatever, and you will be able to run that.
The same thing it is at Android too: you have to create a "test project"
Once I did and forgot how is working, but, there are other stuff too:
- long press the Play button on Xcode ( 4.4 ) and you will have a dropdown menu with: Run, Test, Profile,Analyze.
I can't present those, because if I press the Shift+ Cmd + 4 to screenshot it it is changing, but here it look like the changed menu:
IMHO: for banking, forex, other financial or military (high security software) I would use test driven development, with over 99% code coverage, but those simple 3-4 web-service call mobile apps, which display public data available in browsers are just waste of time to develop tests and upkeep it!
Many times I need to test with internet connection and without.
to be worse case with WI-FI connection , but router doesn't give IP or let go out the phone, but if I ask the phone state: it is connected...
The GUI flow hard to test from unit testing, where is / would be usefully for me: the data got from web-service and synchronization with internal cache. As I see it is still cheaper to do it with manu testing.
I regularly use right mouse button > "Run/Debug Grails Command..." from within NetBeans.
When I do so, it's cumbersome b/c I have to wait for "Reloading Grails commands...", then I have to choose the command and manually type all parameters e.g. "unit:spock -coverage ExampleController".
I'll have to compose the commands everytime I restart NetBeans.
Is there a better solution to this?
Also, everytime I run "test-app" complete Grails restarts - is it possible to leave Grails running and just execute the tests in question via a click again, and again, and again ... ?
Thanks to the help of david, I can now answer my question #2:
When clicking right mouse button > "Run/Debug Grails Command..." from within NetBeans, simply double-click "interactive" from the list.
Then, in the new shell, type the test you want to run without grails. E.g. only test-app unit:spock -coverage ExampleController
Everytime you want to execute the test again just hit ENTER within the shell/console.
Note that Grails won't be able to handle certain changes correctly. In this case you willl most likely see unexpected exceptions and the like.
If that happens, just close the shell, clean the project, rinse & repeat.