I am using Django 1.8 and I have a management command that geocodes some items in my database, which requires an internet connection.
I have written a test for this management command. However, the test runs the script, so it also requires an internet connection.
After pushing the test to GitHub, my CI is broken, because Travis doesn't have an outside internet connection so it fails on this test.
I want to keep this test, and I'd like to continue to include it in python manage.py test when run locally.
However, is there a way I can explicitly tell Travis not to bother with this particular test?
Alternatively, is there some other clean way that I can keep this test as part of my main test suite, but stop it breaking Travis?
Maybe you could decorate your test with #unittest.skipIf(condition, reason) to test for the presence of a Travis CI specific environment variable to skip it or not. For example:
import os
...
#unittest.skipIf("TRAVIS" in os.environ and os.environ["TRAVIS"] == "true", "Skipping this test on Travis CI.")
def test_example(self):
...
If the external resource is an HTTP endpoint, you should consider using vcrpy to record and replay the HTTP requests/responses.
This way you can continue running the same test suite in different environments. It'll also speed this test up.
Related
I want to Automate and integrate the the Selenium web driver tests that we developed for a website in AWS code build environment. We want to run these tests automated, in the AWS Code-Build and then Release (AWS Code-Deploy) if all good.
For example, We wrote all of our test cases using node. Let's assume that I have a basic test case like below
npm install selenium-webdriver In a file called google_test.js
const webdriver = require('selenium-webdriver'),
By = webdriver.By,
until = webdriver.until;
const driver = new webdriver.Builder()
.forBrowser('firefox')
.build();
driver.get('http://www.google.com');
driver.findElement(By.name('q')).sendKeys('webdriver');
driver.sleep(1000).then(function() {
driver.findElement(By.name('q')).sendKeys(webdriver.Key.TAB);
});
driver.findElement(By.name('btnK')).click();
driver.sleep(2000).then(function() {
driver.getTitle().then(function(title) {
if(title === 'webdriver - Google Search') {
console.log('Test passed');
} else {
console.log('Test failed');
}
driver.quit();
});
});
Then as you would expect We run this test in the command line,
node google_test
This works fine and great in a manual environment,
however, our challenge is to automate this and deploy automatically if tests were successful,
I wonder how we can archive this in a the AWS code-build setup. Even after doing all the research Im still confused on what is the best way to achieve this, Many suggests many different ways all looks very hacky and unreliable,
Problems/Questions
Since in an Automated AWS code build enr. We dont have browser access, so how can we actually see the output to see if the tests were successful?
What is the way/How we can detect the tests ran correctly to proceed to the next step of code-deploy? what signals can be generated and how?
If this is not possible, what is the recommended way of doing this?
I have Rails app which has an Rspec feature with selenium that always passes locally and periodically fails on travis. It fails on click_link("my link"), with a Net::ReadTimeout: error. The stack trace isn't all that helpful and It'd be nice if there was a way to tail the log (tail -f log/test.log), so see if that's helpful...or at least view the log output. Is this possible using travis ci? I'm already waiting for ajax to finish, which suggests something external, so ultimately I'm trying to find out what request it's getting hung up on.
I believe you can cat the logs to the console as the last step of your test task, or you can use Travis' artifact option to get them uploaded to S3 - see http://docs.travis-ci.com/user/uploading-artifacts/
This is pretty simple question. I am posting this because I couldn't get any satisfying answer. First the background: I have Jenkins job that builds and deploys a web application on to a server. The server takes some time (in the order of 5 to 10 minutes). I would like to setup a job (or modify the existing as required) to rig up the unit test case execution which will test the application. I am thinking of the following approaches. I would like you to validate or suggest any alternatives:
Have an Ant target that waits for a fixed time
Have a custom Ant target which pings the URL and checks for app availability
Thanks in advance for your help.
-Vadiraj.
Waiting for a fixed time has the problem that the time you choose is either to short (build fails) or to long (waste of build time). So I think it would be better to check if the app is available.
I have done something similar for my Selenium tests. I had to wait until the Selenium Remote Server has started. I used the waitfor element. For a detailled documentation see here.
Here is a stripped down version of my ant-Target:
<parallel>
<sequential>
... Start web application server ...
</sequential>
<sequential>
<waitfor maxwait="10" maxwaitunit="minute">
<socket server="localhost" port="8080" />
</waitfor>
<junit>
...
</junit>
</sequential>
</parallel>
If your server is available before the web app is deployed you can try to use the http condition instead of socket to check for a HTTP error code. The conditions are documented here.
I have a long-running unit test job in hudson. If some tests fail I want to run them first and not wait for other tests to run before them (to see, have I fixed them, or not).
Is it possible to setup this in Hudson?
Thanks.
I have the same issue before, here is my solution.
You can write a standalone program to run a list of unit test cases. (In my case, I wrote a Java main class to run Junit manually.)
Create a job that can run with "Trigger builds remotely" and pass the list from the url
Use Selenium to grab the failure result from Hudson's "Test Result"
Use Selenium to trigger the job from "Trigger builds remotely" with the failure list.
By the way, you can also send a mail with the result when the rerun testing failure, and then you can just check mail if the test is real "failure".
Note that the Selenium is not necessary if you have another choice.
Don't think it's possible in Hudson, but if you're using Eclipse (sorry I'm assuming you're using Java), you can run the tests, and re-run them using 'Rerun Test - Failures First'.
So, I have a GWT client, which interacts with a Python Google App Engine server. The client makes request to server resources, the server responds in JSON. It is simple, no RPC or anything like that. I am using Eclipse to develop my GWT code.
I have GWTTestCase test that I would like to run. Unfortunately, I have no idea how to actually get the google app engine server running per test. I had the bright idea below of trying to start the app engine server from the command line, but of course this does not work, as Process and ProcessBuilder are not classes that the GWT Dev kit actually contains.
package com.google.gwt.sample.quizzer.client;
import java.io.IOException;
import java.lang.ProcessBuilder;
import java.lang.Process;
import com.google.gwt.junit.client.GWTTestCase;
public class QuizzerTest extends GWTTestCase {
public String getModuleName() {
return "com.google.gwt.sample.quizzer.Quizzer";
}
public void gwtSetUp(){
ProcessBuilder pb = new ProcessBuilder("dev_appserver.py",
"--clear_datastore",
"--port=9000",
"server_python");
try {
p = pb.start();
} catch (IOException e) {
System.out.println("Something happened when starting the app server!");
}
public void gwtTearDown(){ p.destroy(); }
public void testSimple() {
//NOTE: do some actual network testing from the GWT client to GAE here
assertTrue(true);}
}
I get the following errors when compiling this file:
[ERROR] Line 21: No source code is available for type java.lang.Process; did you forget to inherit a required module?
[ERROR] Line 30: No source code is available for type java.lang.ProcessBuilder; did you forget to inherit a required module?
As you can see below, I basically want it to be the case that per test it:
Starts a datastore-empty instance of my GAE server
runs the test across the network, against this server instance.
Stop the server
Of course, report the result of the test back to me.
Does anyone have a good way of doing this? Partial solutions are welcome! Hacks are fine as well. Maybe some progress on this problem could be made by editing the ".launch" config file? The only important criteria is that I would like to "unit test" portions of my GWT code against my actual GAE Python server.
Thank you.
I would recommend creating an Ant target for this - take a look at this page for the full ant build file for GWT.
Then, as the first line of the testing target, add an execution task to start the server. Look here for exec docs.
Then set up that ant task in your IDE. This way you get the server running before your tests irrespective of where you run the tests from, and it can be integrated into your build process if you want.