I have a long-running unit test job in hudson. If some tests fail I want to run them first and not wait for other tests to run before them (to see, have I fixed them, or not).
Is it possible to setup this in Hudson?
Thanks.
I have the same issue before, here is my solution.
You can write a standalone program to run a list of unit test cases. (In my case, I wrote a Java main class to run Junit manually.)
Create a job that can run with "Trigger builds remotely" and pass the list from the url
Use Selenium to grab the failure result from Hudson's "Test Result"
Use Selenium to trigger the job from "Trigger builds remotely" with the failure list.
By the way, you can also send a mail with the result when the rerun testing failure, and then you can just check mail if the test is real "failure".
Note that the Selenium is not necessary if you have another choice.
Don't think it's possible in Hudson, but if you're using Eclipse (sorry I'm assuming you're using Java), you can run the tests, and re-run them using 'Rerun Test - Failures First'.
Related
Recently we have had an issue where a unit test started to take 20+ minutes to run when on the build server.
I have now fixed that issue but I wondered if there was a way to fail the test when running a CI build on TFS server if a test reaches a certain time limit. I've looked at the Definition in TFS and the only timeout I can configure is the "Build job timeout in minutes" which is for the whole project. Currently this is 60 minutes.
What I am wanting is a "unit test timeout".
Can this be configured in TFS?
or do I need to set it in my test settings for the solution?
The feature of timeout a task has already existed in VSTS and new TFS15RC1. If you use VSTS or TFS15, you can specify the timeout for the test task directly in build definition, check the screenshot below:
If you use TFS 2015, there is not a direct way to set timeout for a task in build definition, but you can set the timeout for each TestMethod in your Unit Test project, for example:
[TestMethod(),Timeout(10000)]
I have Rails app which has an Rspec feature with selenium that always passes locally and periodically fails on travis. It fails on click_link("my link"), with a Net::ReadTimeout: error. The stack trace isn't all that helpful and It'd be nice if there was a way to tail the log (tail -f log/test.log), so see if that's helpful...or at least view the log output. Is this possible using travis ci? I'm already waiting for ajax to finish, which suggests something external, so ultimately I'm trying to find out what request it's getting hung up on.
I believe you can cat the logs to the console as the last step of your test task, or you can use Travis' artifact option to get them uploaded to S3 - see http://docs.travis-ci.com/user/uploading-artifacts/
I am using Django 1.8 and I have a management command that geocodes some items in my database, which requires an internet connection.
I have written a test for this management command. However, the test runs the script, so it also requires an internet connection.
After pushing the test to GitHub, my CI is broken, because Travis doesn't have an outside internet connection so it fails on this test.
I want to keep this test, and I'd like to continue to include it in python manage.py test when run locally.
However, is there a way I can explicitly tell Travis not to bother with this particular test?
Alternatively, is there some other clean way that I can keep this test as part of my main test suite, but stop it breaking Travis?
Maybe you could decorate your test with #unittest.skipIf(condition, reason) to test for the presence of a Travis CI specific environment variable to skip it or not. For example:
import os
...
#unittest.skipIf("TRAVIS" in os.environ and os.environ["TRAVIS"] == "true", "Skipping this test on Travis CI.")
def test_example(self):
...
If the external resource is an HTTP endpoint, you should consider using vcrpy to record and replay the HTTP requests/responses.
This way you can continue running the same test suite in different environments. It'll also speed this test up.
I need to completely disable the SwiftMailer spooling functionality for some of my unit tests.
I have methods that implement application-specific functionality on top of SwiftMailer and I have unit tests for them.
Unfortunately, it appears that SwiftMailer's listener that sends mail spooled to memory at the end of a request is not run during unit testing.
This means that messages spooled to memory are lost. If I spool to a file then I have to manually run the console/swiftmailer:spool:send command. Yes. I know that I could run that command from within my test, but that really doesn't seem very clean and is subject to failure if the syntax of the send command is ever changed.
I have tried removing the swiftmailer.spool configuration from config.yml and specifying it only in config_dev.yml and config_prod.yml, leaving it out of config_test.yml. This has had no effect.
In fact, I have been utterly unable to get rid of the default spool configuration.
I have been using the console config:debug swiftmailer --env=[whetever] to test after each change and the spool configuration is always there with type:memory unless I explicitly set the type to file.
Suggestions?
Many thanks.
This is pretty simple question. I am posting this because I couldn't get any satisfying answer. First the background: I have Jenkins job that builds and deploys a web application on to a server. The server takes some time (in the order of 5 to 10 minutes). I would like to setup a job (or modify the existing as required) to rig up the unit test case execution which will test the application. I am thinking of the following approaches. I would like you to validate or suggest any alternatives:
Have an Ant target that waits for a fixed time
Have a custom Ant target which pings the URL and checks for app availability
Thanks in advance for your help.
-Vadiraj.
Waiting for a fixed time has the problem that the time you choose is either to short (build fails) or to long (waste of build time). So I think it would be better to check if the app is available.
I have done something similar for my Selenium tests. I had to wait until the Selenium Remote Server has started. I used the waitfor element. For a detailled documentation see here.
Here is a stripped down version of my ant-Target:
<parallel>
<sequential>
... Start web application server ...
</sequential>
<sequential>
<waitfor maxwait="10" maxwaitunit="minute">
<socket server="localhost" port="8080" />
</waitfor>
<junit>
...
</junit>
</sequential>
</parallel>
If your server is available before the web app is deployed you can try to use the http condition instead of socket to check for a HTTP error code. The conditions are documented here.