I wonder if you have any mobile test automation tool that you can recommend? After trying several tools I simply cannot find one that meets every criteria. Ranorex can be used on both devices but after facing too many issues I have chosen not to use it. The best tool I have encountered is robotium, although it only works on Android and therefore is not interesting.
Criteria:
Possible to CI integrate.
Can be used with Nunit or a similar structure.
Can be used on both iOS and Android.
Is stable.
Please do not recommend one of these recording tools, since they (in my opinion) do not test anything.
Have a nice day an thank you for all the help.
Automation of applications on real devices for Hybrid/Native/Mobile Web can be achieved by using Appium which is an open-source framework.
Appium allows you to create once and use everywhere capability for mobile web apps, e.g., same script would be running on Android as well as iOS.
Its rich function library support enables you to automate even complex mobile application gestures e.g., tap, pinch, precise tap, swipe etc.
For more details on appium, visit: http://appium.io/
Maybe Tosca Mobile+ would be of interest for you:
http://www.tricentis.com/tricentis-tosca-testsuite/tosca-mobile-plus/
Tosca follows a model based approach - it's not script based, and it's also not a Capture & Replay tool. The identification of the controls is done via technology-related properties and IMHO the most reliable way for a stable automation.
Beside of Android & iOS, Tosca does support many more technologies (UI and Non-UI).
Tosca can also be completely integrated into your CI (using e.g. Jenkins, TeamCity, QuickBuild or any other build tool).
For more details you can also look at the Tosca Mobile+ Documentation:
https://documentation.tricentis.com/en/840/index.htm#tbox/mobile.htm%3FTocPath%3DTBox%7CTosca%2520Mobile%252B%7C_____0
May be it is bit late to answer this question - you can try M-eux from Jamo Solutions. It works well for android and iOS. Supports native, hybrid and web apps on both android and iOS. Continuous integration is possible. By default there is no out of the box integration but you can write unit test cases in junit. Compile them and put them in CI using jenkins.
Other option could be seetest from Experitest. Same features are available with this tool as well. Both of these tools are not open source or freeware.
You could try out Calabash. It can be used with both IOS and android and is easily integrated into a CI pipeline.
Have you tried SIGOS App Experience (formerly Keynote Mobile Testing/DeviceAnywhere)?
It integrates with many of the leading CI/agile tools including Appium, HP UFT, Selenium, and SAP. They offer testing on real devices and you can choose what type of deployment you want-- public cloud, private cloud, on-premise.
SIGOS App Experience also offers app performance monitoring, which not all test automation platforms do. https://appexperience.sigos.com/
Related
I have an app with ParseServer back-end and Ionic2 front-end. I need to simulate multiple users to stress test the back-end.
What load testing tools would you recommend to use for such setup?
Thanks.
You need to split your process into 2 phases:
Server-side testing. You need to load test your backend to ensure that it is in position to simulate anticipated number of users. In fact any tool capable of sending HTTP Requests will fit, the most popular free and open source load testing solutions are JMeter, Grinder, Gatling, and Tsung. All of them come with record-and-replay functionality so you will be able to build your test by just interacting with your mobile application and using the load testing tool as a proxy. See Open Source Load Testing Tools: Which One Should You Use? article for main features highlighted and compared.
Client-side testing. Even if your server responds very fast, handles enormous loads, able to scale, etc. your application user experience may be not that good as client side performance also matters. You can go for Chrome Dev Tools Remote Debugging and/or Intel XDK which is capable of profiling existing applications.
You can try using ZebraTester and record the script for this test. The trial version allows you to have up to 20 Virtual Users and these can run multiple loops depending on the length of your test. The same tool can record the script as well as run the load test from your local machine.
I use to test parse server
in http://jmeter.apache.org/
is a free tool you can install your computer then start testing
I am a Qt/C++ developer. I would like to setup a continuous integration environment whereby after committing the source code, it triggers a build process that build the code for the 3 platforms I'm using:
Linux
OS X
Win32
If possible, how do I setup such environment. Any hints or links are welcome.
I've read around about Jenkins, but I can't find any good tutorial for it.
I also suggest Jenkins for several reasons:
It will run on all of the platforms you listed.
It can be configured to start a build when the repository is updated (hint: configure the Job to "Poll SCM" and you won't have to muck with your SCM tool to get it to tell Jenkins to start building).
It provides good support (mostly through plugins) for Unit Testing. [You're project is doing unit testing, right?]
The price is right
A bigger issue is going to have is that AFAIK, Qt doesn't really do cross-compiling for other platforms well. Using Jenkins (and the appropriate plugins), you should be able to solve this.
One method that comes quickly to mind is to have an instance of Jenkins on each platform. Each instance is responsible for building the version for its own platform. At the end of the build, the created artifacts are all put into a common, shared location.
Jenkins supports this feature via plugins for all major source control systems. If you seriously considering using Jenkins (and I would highly recommend it), consider buying John Ferguson Smart's Jenkins: The Definitive Guide.
Two solutions coming to my mind:
BuildBot
BuildBot is a highly customizable continuous integration system written in Python. The master component offers a nice web-based GUI to monitor and trigger builds; slave components are put on the target machines (usually virtual machines but they could be the Mac laptop of one of the developers). Docs are good enough to build up a basic system, customization could be a little tricky (at least it was for me). Using commit/push hooks provided by VC systems you can easily activate the master and trigger builds across the slaves. It also supports incremental builds (a must if your project is big).
CDash
Developed by the authors of CMake, CDash is a web application collecting builds coming from across the network, not exactly what you asked for but I think it's worth a try. Very powerful if you have a team of developers who could continuosly submit build result on their machines to the server (and if you use CMake it's almost transparent). You cannot trigger builds from the server as Buildbot does, but you could setup a bunch of VM with a cron which checks for changes and in case performs the build and sends results to CDash
Sure it's possible. Most of the version control systems are able to execute custom script on server side. Some of them (git, for example), has hooks to achieve the same locally. Have a look at git's post-commit hook.
All you need is to create a script that will trigger cross-platform builds.
Most version control systems allow post-commit hooks to allow you to kick off events like builds. Alternatively build systems can be configured to regularly poll a source control repository and manage their own build scheduling (this is how we use Jenkins).
Something to bear in mind is how long it will take to do a complete build across platforms and the typical number of check-ins in that interval. You might find batching check-ins a better way of doing continuous integration builds if you have an fair sized team or limited build server resources. Otherwise your build system could quickly end up trying to play catch up.
As for whether it is possible to build on all target platforms, that depends on your tool chain.
At the moment, we are running our automated (not CI as such) builds via FinalBuilder via a very simple homegrown Apache interface that just launches the FB scripts on our server. (I like FinalBuilder, and will keep it, but it's CI server, FinalBuilder Server just doesn't cut it IMHO -- especially it doesn't support any "agent" concept at the moment to distribute builds across machines.)
We are doing native C++ development on Windows with a bit .NET mixed in where it's needed and makes sense.
Our current FinalBuilder scripts do everything quite well, from creating nightly builds to full releases (build / automated translation / build / unit test / create setup / put created artifacts on a network share / ...), but our webinterface, queuing abilities, user traceability and reporting is pretty limited.
I have looked around and it seems that TeamCity and Bamboo tick similar boxes, but most descriptions I can find cover only Java and/or .NET simple builds.
So my specific question is, given
several (20-30) complicated FinalBuilder Scripts that work to my satisfaction and that I will have to integrate into ("call" from) the new automation/"CI" server
Native Windows C++ and .NET projects
The actual build (= compiler invocation(s)) is done via a few Visual Studio solution files at the moment
Currently one build server machine, wishing to scale to 2-3 atm.
Using JIRA as issue tracker
using AccuRev as SCM
which tool is better suited, and why: TeamCity (currently 6.5) or Bamboo (currently 3.1).
(Note that I also hope to get some highly subjective answers on the TeamCity and Bamboo forums.)
For TeamCity side, it integrates with Jira, has AccuRev plugin, and has a good support for VisualStudio/C++ projects. It can also run arbitrary scripts.
You can trigger a build and obtain some build results via HTTP-based API. In the UI, you can see which changes have been built and in which build configurations. Easily integrate any custom HTML reports into TeamCity UI (no coding), publish artifacts.
Probably, you should try both solutions and see which one is more suitable for you (with Teamcity, you can use full-functional server for free, the only limit is number of build agents and number of build configurations).
Disclaimer: I'm a TeamCity developer
I found Bamboo more credible than TeamCity. Here are my reasons:
Those Jira plugins for VS or Eclipse are Bamboo plug-ins too. :) no extra add-ins needed.
Better support for Jira integration.
Nice user interface, like the one you used for Jira.
Ability to better integration with other Atlassian tools, such as FishEye.
Cheaper. A 10$ license will suffice your company.
More add-ons on Bamboo than TeamCity, lots of plug-ins.
For completeness' sake: I ended up using Jenkins + Finalbuilder. :-)
I worked in a similar environment using FinalBuilder for build automation, AccuRev for source control and a native windows projects.
I ended up selecting Electric Commander as the best CI solution for the job. It is possible to reuse parts of the FinalBuilder scripts and call them from Electric Commander but simply calling the FB script as one build step would result in you missing out on some of the key advantages of using Electric Commander - realtime log file processing, the ability to parallelize right down to individual step levels in Electric Commander and data collection and reporting.
Electric Commander has an API that exposes all product functionality which can be used in combination with AccuRev triggers to achieve a very flexible solution.
Disclaimer - I liked Electric Commander so much I joined the company and am currently employed by Electric Cloud.
You can try Electric Commander by going to www.electric-cloud.com and clicking on "Try It!"
Is there a way to meet the following criteria in distributing a Web Service to Windows machines?
1) Automatic installation and configuration of the Web Server.
2) No configuration (or even awareness) of a Web Server required by the customer.
3) No prompts to download and install Java or .NET - especially anything after .NET 2.0; those installs / restarts can take forever!
In short, is there a way to deliver a single install process that installs the Web Server along with a simple web app without requiring lengthy installations of pre-requisites? Something for even the most non-technical of users?
.NET's WCF almost meets the requirements but getting .NET updated up to 3.0 / 3.5 is a lengthy process and can be a turn-off for customers, even if the install holds their hand through the whole thing.
Rubyscript2exe was also very close, but it is extremely touchy and out-dated.
I am open to any technology / programming language - just looking for the slickest distribution process for my customers that meets the above three criteria.
I've been doing quite a bit of research on this as it is extremely important to me that my users have a simple installation experience. Here are a few things that I've found:
UltiDev Cassini: Cassini is that convenient mini-server that runs when you debug your web apps from Visual Studio or Visual Web Developer. UltiDev Cassini builds on that and looks pretty promising. It offers support for all non-beta flavors of .Net and integrates right into Visual Studio. Most interesting to me is the ability to include as part of your installer. The only down side is that pesky .Net pre-requisite. I can handle helping users get installed up to 2.0, but the install process to move to 3.0 and 3.5 is way too heavy for the typical user.
RubyScript2Exe: I like the premise of an executable Rails app. However, I attempted to use this on a Mac and it is simply too outdated and requires too many workarounds for my tastes. It's too bad, because I love Ruby on Rails development.
Server2Go: This is my favorite of the three options. It is easily distributable (just send off a zip file) and has a lot of nice options. For example, you can configure it to leave the included Apache server running even after the browser closes - that is PERFECT for a nicely packaged web service. It can also provide a customizable icon in the task bar for shutting down the service if necessary. I think this best meets my needs for the time being.
Please, if you know of any other options, let me know.
Also, you may be wondering, "Why not just write a desktop app?". The simple answer is that I don't need much of a GUI, if any. I need a simple to install web service that can be consumed by various other applications (web, mobile, and desktop included).
My company uses a lot of different web services on daily bases. I find that I repeat same steps over and over again on daily bases.
For example, when I start a new project, I perform the following actions:
Create a new client & project in Liquid Planner.
Create a new client Freshbooks
Create a project in Github or Codebasehq
Developers to Codebasehq or Github who are going to be working on this project
Create tasks in Ticketing system on Codebasehq and tasks in Liquid Planner
This is just when starting new projects. When I have to track tasks, it gets even trickier because I have to monitor tasks in 2 different systems.
So my question is, is there a tool that I can use to create a web service that will automate some of these interactions? Ideally, it would be something that would allow me to graphically work with the web service API and produce an executable that I can run on a server.
I don't want to build it from scratch. I know, I can do it with Python or RoR, but I don't want to get that low level.
I would like to add my sources and pass data around from one service to another. What could I use? Any suggestions?
Progress DataXtend Semantic Integrator lets you build WebServices through an Eclipse based GUI.
It is a commercial product, and I happen to work for the company that makes it. In some respects I think it might be overkill for you, as it's really an enterprise-level data mapping tool for mapping disparate data sources (web services, databases, xml files, COBOL) to a common model, as opposed to a simple web services builder, and it doesn't really support your github bits, anymore than normal Eclipse plugins would.
That said, I do believe there are Mantis plugins for github to do task tracking, and I know there's a git plugin for Eclipse that works really well (jgit).
Couldn't you simply use Selenium to execute some of this tasks for you? Basically as long as you can do something from the browser, Selenium will also be able to do. Selenium comes with a language called "selenese", so you can even use it to programmatically create an "API" with your tasks.
I know this is a different approach to what you're originally looking for, but I've been using selenium for a number of tasks, and found it's even good to execute ANT tasks or unit tests.
Hope this helps you
What about Apache Camel?
Camel lets you create the Enterprise Integration Patterns to implement routing and mediation rules in either a Java based Domain Specific Language (or Fluent API), via Spring based Xml Configuration files or via the Scala DSL. This means you get smart completion of routing rules in your IDE whether in your Java, Scala or XML editor.
Apache Camel uses URIs so that it can easily work directly with any kind of Transport or messaging model such as HTTP, ActiveMQ, JMS, JBI, SCA, MINA or CXF Bus API together with working with pluggable Data Format options.