So i'm starting use Intern for functional tests, so far so good I did it all, unit and functional tests.
I followed their intern-tutorial
Whenever you need to run a full test against all platforms, use the test runner. When you are in the process of writing your tests and want to check them for correctness more quickly, you can either use just the Node.js client (for unit tests only) or create an alternate configuration file that only tests against a single local platform, like your local copy of Chrome or Firefox (for all tests, including functional tests).
I searched on their documentation, but I didn't find anything exactly about local "tunnels".
I'm using Intern with Gulp, my localhost is localhost:3000 and I want to test on my Chrome 54 on Mac.
Thank you
I guess NullTunnel is what you're looking for?
I found the answer. I had to change the tunnel to Local Selenium.
Download the latest version of ChromeDriver
Set tunnel to 'NullTunnel'
Run chromedriver --port=4444 --url-base=wd/hub
Set your environments capabilities to [ { browserName: 'chrome' } ]
Run the test runner
Obs:
Don't forget to copy the chromedriver file to your project root.
I had to run on my project root .\chromedriver --port=4444 --url-base=wd/hub
The test runner has to be run in a new command line/terminal/shell
Hope to help someone that had the same issue.
Related
I'm running the gradle test task on a Micronaut project and the imbedded server is getting started.
In local that is not much of an issue, but when running the unit tests in the CI environment there isn't a database available for the server to connect to. So ideally we would run the gradle test task without starting the server.
Is it possible to configure Micronaut to not do that?
Is it possible to configure Micronaut to not do that?
Yes. The embedded server is not started by default. It is only started if you write code in your test to start it or if you mark the test with #MicronautTest. If you don't do either of those things, the server shouldn't start.
I set up a new Flask Python server and I created a Dockerfile with all my codes. I've written some unit tests and I'm executing them locally. When should I execute them if I want to implement a CI/CD?
I also need to write integration tests (to test if I'm querying the database correctly, to understand if the endpoint is exposed correctly, and so on), when should I execute them in a CI/CD?
I was thinking to execute them during the docker build so to put the execution of the tests in the Dockerfile. Is it correct?
Unit tests: Outside of Docker, before you run your docker build. Within your CI pipeline, after checking out the source code and running any setup steps like installing package dependencies.
Integration tests: Launched from outside of Docker; depending on how complex your setup is, either late in your CI pipeline or as part of your CD pipeline.
This assumes a true "unit test" that has no external dependencies; it depends only on the application/library code, and where it needs things like databases, it either mocks out those dependencies or uses something like an embedded SQLite. (Some frameworks are especially bad at this workflow and make it impossible to start up the application at all if the database isn't available. But Rails doesn't run on Python.)
Running unit tests in a Dockerfile will last until it's midnight, you have a production outage, and either your quick fix that will bring the site back up happens to break one obscure unit test, or you can't wait the 5-minute cycle time to run the whole unit-test suite. Since there shouldn't be dependencies on the Docker-or-not environment in your unit tests, I'd just run them outside Docker.
Often you can stand up enough infrastructure to be able to run your application "for real" with a couple of docker run commands or a simple Docker Compose setup. In that case, it makes sense to run an integration test towards the end of your CI pipeline. With a more complex setup (maybe one involving Kubernetes) you might need to actually deploy into a test environment, and if you have separate CI and CD tools, this would turn into "test deploy", "integration test", "pre-production deploy".
As a developer I find having tools not-in-Docker vastly easier to manage than tools that only run in Docker. (I don't subscribe to the "any binary other than /usr/bin/docker is bad" philosophy.) I'd rather just run pytest or curl than remember the 4-line docker run invocation to do some specific task.
I have a project that I am usually debugging/running using the grails run-app command. I would like to run a test but without having the server run again only for the specific test.
I usually run the server in debug mode all the time in the background.
I've tried playing around with the run configurations in iteli-j, with latest try being grails test-app functional: className
In Grails 2.4.4 if you override the baseUrl you can run your tests against a server other than localhost. For example, we have a pre-production server hosted on AWS and we run a subset of our functional tests against it from Jenkins, post-deploy, as a smoke test.
grails -plain-output test-app -baseUrl=https://foo.bar.org/ -echoOut -echoErr functional:
That works, but test-app still starts the embedded tomcat server. However, with a bit of digging, I found that overriding the server host to point to a running instance will cause the tests to run without starting the embedded tomcat. There are a couple of ways to accomplish this:
Pass the value on the command line:
grails -plain-output test-app -Dgrails.server.host=foo.bar.org -baseUrl=https://foo.bar.org/ -echoOut -echoErr functional:
Or, overriding the value in Config.groovy for the specific environment should also work:
...
preProd {
...
grails.server.host = 'foo.bar.org'
...
}
...
It isn't documented under test-app, but it is mentioned under run-app and it turns out it works for test-app too.
This works because Grails determines if the embedded server should be started by trying to open a connection to the server host/port, and if successful skips the startup.
From trial and error I have discovered that I have the best results when I specify both grails.server.host and -baseUrl even though the base url becomes redundant information. Possibly this is because my case involves an ssl connection, but I tried running with -https instead of -baseUrl=... and the tests just hung.
This plugin maybe helps you.
Grails functional-test-development Plugin
This plugin aims to make developing functional tests for Grails more convenient by facilitating running your tests against an already running application. It utilises the improved functional testing support added in Grails 1.3.5 and does not work with earlier versions.
At work we fully test the GUI components. The problem arises from the fact that, while the testsuite is running, the various components pop up, stealing the focus or making it impossible to continue working. The first thing I thought of was Xnest, but I was wondering if there's a more elegant solution to this problem.
I think what you need to do here is have your tests run on a different Display than the one you're working on.
When we moved our TeamCity agents to EC2, we had to figure out a solution to running our UI unit tests on a headless Linux server. I found a way to do it in this blog post, which outlines how to use Xvfb.
For my case, all I had to do was:
yum install xorg-x11-server-Xvfb
Xvfb :100 -ac to run the server. I added this to my rc.local file on my EC2 agents to start it at machine startup.
Then I added env.DISPLAY :100 to my TeamCity build configuration
I'm wondering if anyone has any best practices for automating the testing of installers on various machines with potentially different hardware / software profiles and by specifying various options to the installer. The idea would be that I could write "unit test like" code to set up a machine, run the installer, then test that certain things are true. Tests might look similar to:
Test:
Boot Machine without IIS
Run Installer
Assert Installer Had Errors
Test:
Boot Machine with IIS
Run Installer
Assert Installer Ran
Test_Fixture:
SetUp:
Boot Machine with IIS
Test:
Run Installer without IIS install
Assert Website Not Installed
Test:
Run Installer with IIS install
Assert Website Installed
I know I could create lots of VMs, but waiting for a VM to boot for each functional test sounds like way more work than I want. What I really want is a way to virtualize the installer environment. Any suggestions?
We have created a set of VMs and find it is very easy to manage. We run the tests for 13 different Windows installers over night. The VMs we have created our very bare bones, so it is possible to run a number of tests in parallel.
If you have the installer runnable from the command line, it's easy to have a script to call it automatically.
Then you can use a web app testing tool to see it the install was successful, like this one http://seleniumhq.org/ For this you will need an unique way to test a new install - like a page with the current version.