I'm wondering if anyone has any best practices for automating the testing of installers on various machines with potentially different hardware / software profiles and by specifying various options to the installer. The idea would be that I could write "unit test like" code to set up a machine, run the installer, then test that certain things are true. Tests might look similar to:
Test:
Boot Machine without IIS
Run Installer
Assert Installer Had Errors
Test:
Boot Machine with IIS
Run Installer
Assert Installer Ran
Test_Fixture:
SetUp:
Boot Machine with IIS
Test:
Run Installer without IIS install
Assert Website Not Installed
Test:
Run Installer with IIS install
Assert Website Installed
I know I could create lots of VMs, but waiting for a VM to boot for each functional test sounds like way more work than I want. What I really want is a way to virtualize the installer environment. Any suggestions?
We have created a set of VMs and find it is very easy to manage. We run the tests for 13 different Windows installers over night. The VMs we have created our very bare bones, so it is possible to run a number of tests in parallel.
If you have the installer runnable from the command line, it's easy to have a script to call it automatically.
Then you can use a web app testing tool to see it the install was successful, like this one http://seleniumhq.org/ For this you will need an unique way to test a new install - like a page with the current version.
Related
I'm considering building app with Django for only one user that would run application locally on windows. I read about containerizing Django apps, but it's new for me.
My goal is to run Django server on one click like a standard windows app and connect to it on localhost and interact with web browser. It is even possible?
It is possible, but this may not be the best solution. If you want to release a Django app that can be installed on your client computer, you usually need to ensure all the dependencies are shipped with the app.
Containerising your application means it will depends on Docker runtime (or any container system you use). You will have to setup Docker with your app, or ensure your client has Docker on his machine to run it. If the destination machine runs Windows or macOS, you will need to setup docker-desktop which may be more complicated than standard Docker runtime (linux only).
But if you decide to ship your app without containerising it, it will only depends on a Python interpreter and some dependencies (Django, dateutil, etc.). In such case, using python tools like virtualenv, you may prepare a ready-to-run application by creating the venv and installing dependencies at "build time". Then, with a proper setup (MSI for Windows or DMG for macOS), you may be able to distribute the final application so the client can install and run it without any additional step (you do all the hard job yourself).
Django app can be convert into .exe but it wont work as local server while click .exe because runserver command and some of django service wont support on this way as per my experiment.
So i'm starting use Intern for functional tests, so far so good I did it all, unit and functional tests.
I followed their intern-tutorial
Whenever you need to run a full test against all platforms, use the test runner. When you are in the process of writing your tests and want to check them for correctness more quickly, you can either use just the Node.js client (for unit tests only) or create an alternate configuration file that only tests against a single local platform, like your local copy of Chrome or Firefox (for all tests, including functional tests).
I searched on their documentation, but I didn't find anything exactly about local "tunnels".
I'm using Intern with Gulp, my localhost is localhost:3000 and I want to test on my Chrome 54 on Mac.
Thank you
I guess NullTunnel is what you're looking for?
I found the answer. I had to change the tunnel to Local Selenium.
Download the latest version of ChromeDriver
Set tunnel to 'NullTunnel'
Run chromedriver --port=4444 --url-base=wd/hub
Set your environments capabilities to [ { browserName: 'chrome' } ]
Run the test runner
Obs:
Don't forget to copy the chromedriver file to your project root.
I had to run on my project root .\chromedriver --port=4444 --url-base=wd/hub
The test runner has to be run in a new command line/terminal/shell
Hope to help someone that had the same issue.
I have a build system, it makes me the build of my c# software, and then runs a build to make my setup and deploys it into a repository.
Now, what I need is, after this deployment, I need this setup installed on a virtual machine or sandbox.
How can I do that? Is there any software that does this for me?
For example, after the build of setup is done, this software with my own configuration, takes my setup and install in silent mode in this VM/ sandbox, and then if I want I can run also tests.
The main idea is, whenever a build is deployed, it's automatically installed in a machine for me, to quickly see the result/ run tests, otherwise I need always, after build, open machine, install and test.
You can look into the use of Vagrant with VirtualBox for such a setup.
Once you finish the build and have the installer ready, you can mount/copy it into a new VM you provision with Vagrant.
You then add the installation command to your Vagrantfile (file describing your VM configuration and setup).
This is a very common setup for automated build, install and test scenarios.
I hope this helps.
At work we fully test the GUI components. The problem arises from the fact that, while the testsuite is running, the various components pop up, stealing the focus or making it impossible to continue working. The first thing I thought of was Xnest, but I was wondering if there's a more elegant solution to this problem.
I think what you need to do here is have your tests run on a different Display than the one you're working on.
When we moved our TeamCity agents to EC2, we had to figure out a solution to running our UI unit tests on a headless Linux server. I found a way to do it in this blog post, which outlines how to use Xvfb.
For my case, all I had to do was:
yum install xorg-x11-server-Xvfb
Xvfb :100 -ac to run the server. I added this to my rc.local file on my EC2 agents to start it at machine startup.
Then I added env.DISPLAY :100 to my TeamCity build configuration
My complete working environment is Linux based (Ubuntu for desktop and server). I use Hudson and Selenium to execute tests on my EE6/ZK web application with any browser available for Ubuntu.
But how would I test my application with IE (Windows) without setting up a Selenium server in Windows. Thought I could run Windows in a VMWare or Virtualbox, but still it needs a "local" selenium and how would I trigger it ? Install hudson as well and pretty much replicate the complete CI in Windows ? How is MAC ?
Any tips or experience with cross-browser-platform-OS automated testing ?
Sven
You will need to use Selenium Grid. There is a Selenium plug-ins for Hudson available, and this article has a guide how to setup Continuous Integration with Hudson and Selenium Grid. And yes, you will have to install Windows in a virtual machine.