We're looking into automating our web UI using either Microsoft's coded UI tests or Telerik TestStudio framework. I would love to run our tests using a service like Sauce Labs however, Sauce Labs requires Selenium tests. Is anyone aware of some service similar to Sauce Labs to run web tests other than Selenium?
TFS in the cloud looks promising http://tfspreview.com/
Regardless of TFS service or Sauce Lab, be aware that a service only frees you the burden of maintaining a CI/Test server in house. It doesn't magically generate tests for you. It's still your responsibility to design and code many tests. That is where huge effort and learning curve will take place.
(Speaking of learning curve, Selenium is much more manageable than Ms coded ui test. Wait for chances to influence company decision...)
Sauce Labs in fact is able to run other (Non-Selenium & Appium) frameworks. There is a testrunner that is super easy to use called saucectl. You can test with CYpress, Test Cafe, Espresso, XCUITest and more. You can see the info in the docs.
TLDR
Install saucectl in your project with cURL or npm install -g saucectl
Setup the runner in your test project with saucectl init. choose your framework flavor and preferences
Run your tests with the
command saucectl run
TFS is not an alternative to Sauce. In fact their is now an integration to run Sauce test from Visual Studio Online formerly TFS. Sauce is a testing grid. The closest competitive solution is BrowserStack. But they are not as widely adopted. But it also is selenium based. Telerik TestStudio is nice, but it also is not the cloud you run on. Maintaining your own grid is a huge part of the value of a cloud-based solution. And selenium really is the best and most supported language for doing it.
Related
I'm using sfdx to deploy my components and classes to a developer edition org, but while deploying it doesn't check 75% test coverage of apex classes, is there any setting to enable test coverage while deployment or DE Org doesn't support test coverage checks while deployment?
You can deploy to developer edition / trailhead playground orgs without code coverage or edit apex straight in "prod" (in setup or developer console). Developer Editions have no sandboxes and concept of scratch orgs is relatively new so being able to work in "prod" was the only option. Plus well, it helps with rapid prototyping of ideas without the "hurdle" of unit tests.
If you'd be eventually making a managed package - don't worry. During process of packaging new version you'd have the usual checks (does everything compile, do all tests pass, 75% overall and at least 1% on triggers).
You can choose to run local tests while deploying but I don't think there's native way to enforce 75% threshold. Maybe something in your CI tool could help (GitHub actions or what have you)
I wanted to create UI interface for Business developer team which contains Test scripts which are written in my PC in selenium with c#.
Problem :-
Business developer team has no visual studio installed in their PC
because they don't have licence for it.Now if they wanted to show
demo of Application to client then if they have UI which contains
Test scripts which i wrote in selenium.so how could i display those
Test script on business developers laptop which don't have visual
studio?
Here for demo purpose to client ,Business developer team will
click on particular test case on UI and run that feature. For
example ,If Business developer wanted to show Login feature in UI at
Business developer laptop then that script should run.
how can i define connectivity between business developer's laptop
which don't have visual studio and source laptop which has visual
studio which contain Test script?
It's kind of R & D work,If anybody has idea about this let me know.
To do this, you are going to have to setup some kind of infrastructure to support this. You could achieve this through setting up a build server and making a build plan per collection of test scripts. This is going to let the BDs run the tests, but not look into the tests till the execution is done. You will require proper reporting in order to do so.
What you could also do is use a tool like browserstack. With (I am not affiliated with browserstack what so ever) browserstack you can make every executed test viewable through their recording function and every step of every test will be administrated. This way, you execute the test once, they login into browserstack and check every test, test step, expected results, actual results, screenshots and 30 day history recording of each and every test executed.
You don't need an IDE to run your tests. The person who wishes to run tests to show off some features of the site should be able to download your test codebase and run it locally from the command line. If they don't want to pass in a lot of flags at the command line, you could bundle the commands into batch or shell scripts.
We're looking at creating Selenium tests to help automate some of our testing.
Is there a tool/service/etc. for helping to manage those tests. Specifically, is there any kind of tool that helps keep them organized and documented?
If there isn't anything automated, are there any resources out there to help with this?
Ideally we'd like our testers to do the bulk of the work when it comes to these tests. Most of our testers have no programming background, so anything too intensive would be handled by the developers.
Thanks.
Generally for setting up automation framework a good rule of thumb is to use what your developers are using for unit and integration testing. Following the same model eliminates the need to maintain/learn multiple tools, plus developers will be happier helping with those tests if they are in their normal environment.
To begin with, you will need 4 components:
Source control (SVN, Git, etc.) for storing the source code of the tests. Some companies may not want the test code to be in the same repository as production code, so they may setup a separate server.
A project (or projects), that contain the test code. Starting from the typical bare-bone development project, adding selenium libraries/dependencies to it will give a good starting point.
Then you need an IDE to develop the tests. Again, ask developers what do they use.
And finally you need something that executes those tests on schedule or on single button click. This is where development's CI product (e.g. Jenkins, Bamboo or whatever they are using) can be useful. This tool will not only run tests for you, but will also give a fairly well-organized results report and history. And it can deploy, say, latest build, or reset test environment.
But to achieve all this, I think you need to accept that testers will deal with writing and debugging the code. And test code has to be of good quality, so you would have to hire or develop 1-2 good automation developers in your team. Or have some developers committed to creation of the tests, which testers will only draft with Selenium IDE.
Java EE is a new world for me, my experiences are on embedded systems, but I started a new job and I would like to know if there is a test process to follow for web applications based on Java EE. Which test strategy is usually adopted in this field?
Basic Unit test
Functional test
Integration test
System test, stress test, load test,....
....
and which is the scope of each test phase for web development? As server code and client code are both involved I don't know which is the best approach in this field. Also, several machines are involved: DB, buisness tier, presentation tier, load balancers, authentication with CAS, Active Directory,...
Which is the best test environment for each phases? When using the production CAS authentication, ...
Links, books, simple explanation or other kind of address is well appreciated.
The best test framework is Junit -for unit tests, in my opinion.
http://www.junit.org/
-for mocking objects, which you will need a lot, like to mock the database, mock services and other object in j2ee environment to be able to test in isolation .use http://www.jmock.org/ , http://code.google.com/p/mockito/, http://www.easymock.org/
-for acceptance and functional testing there is selenium http://seleniumhq.org/ this framework enables you to automate your tests.
I Advice you to read this books about testing in general and testing in j2ee evironment in particular.
http://www.manning.com/rainsberger/
http://www.amazon.com/Test-Driven-Development-By-Example/dp/0321146530
http://manning.com/massol/
http://manning.com/koskela/
First, whatever you plan to do as testing, take care of your build process (a good starting point is maven as build tool)
Junit (or testng) is almost good for everything (due to its simplicity)
Unit test:
For mock, I would prefer Mockito to jmock or easymock.
Acceptance test:
Regarding UI testing selenium is fine for web application (give a look at PageObject pattern if you plan to do a lot of UI testing).
For other interface testing (such as webservice), soapui is a nice starting point.
Integration testing:
You will face the middle ware problem, mainly solved in java by a container. Now it becomes fun :) If you run in "real" JEE, then it depends if it's prior to JEE6 or not as from JEE6 you have an embedded container (which really ease the testing). Otherwise, go for a dependency injection framework (Spring, Guice, ...).
Other hints for integration or acceptance testing:
you will may be need to mock some interface (give a look to MOCO to mock external service based on HTTP).
also think about some embedded servlet container (Jetty) to ease web the testing.
configuration and provisioning can be a problem too. ex.: for the DB you can automate this with "flyway" or "liquibase"
DB testing you have two approach: resetting data after each test (see DBUnit) or in transaction testing (see Spring test for an example)
At the moment, we are running our automated (not CI as such) builds via FinalBuilder via a very simple homegrown Apache interface that just launches the FB scripts on our server. (I like FinalBuilder, and will keep it, but it's CI server, FinalBuilder Server just doesn't cut it IMHO -- especially it doesn't support any "agent" concept at the moment to distribute builds across machines.)
We are doing native C++ development on Windows with a bit .NET mixed in where it's needed and makes sense.
Our current FinalBuilder scripts do everything quite well, from creating nightly builds to full releases (build / automated translation / build / unit test / create setup / put created artifacts on a network share / ...), but our webinterface, queuing abilities, user traceability and reporting is pretty limited.
I have looked around and it seems that TeamCity and Bamboo tick similar boxes, but most descriptions I can find cover only Java and/or .NET simple builds.
So my specific question is, given
several (20-30) complicated FinalBuilder Scripts that work to my satisfaction and that I will have to integrate into ("call" from) the new automation/"CI" server
Native Windows C++ and .NET projects
The actual build (= compiler invocation(s)) is done via a few Visual Studio solution files at the moment
Currently one build server machine, wishing to scale to 2-3 atm.
Using JIRA as issue tracker
using AccuRev as SCM
which tool is better suited, and why: TeamCity (currently 6.5) or Bamboo (currently 3.1).
(Note that I also hope to get some highly subjective answers on the TeamCity and Bamboo forums.)
For TeamCity side, it integrates with Jira, has AccuRev plugin, and has a good support for VisualStudio/C++ projects. It can also run arbitrary scripts.
You can trigger a build and obtain some build results via HTTP-based API. In the UI, you can see which changes have been built and in which build configurations. Easily integrate any custom HTML reports into TeamCity UI (no coding), publish artifacts.
Probably, you should try both solutions and see which one is more suitable for you (with Teamcity, you can use full-functional server for free, the only limit is number of build agents and number of build configurations).
Disclaimer: I'm a TeamCity developer
I found Bamboo more credible than TeamCity. Here are my reasons:
Those Jira plugins for VS or Eclipse are Bamboo plug-ins too. :) no extra add-ins needed.
Better support for Jira integration.
Nice user interface, like the one you used for Jira.
Ability to better integration with other Atlassian tools, such as FishEye.
Cheaper. A 10$ license will suffice your company.
More add-ons on Bamboo than TeamCity, lots of plug-ins.
For completeness' sake: I ended up using Jenkins + Finalbuilder. :-)
I worked in a similar environment using FinalBuilder for build automation, AccuRev for source control and a native windows projects.
I ended up selecting Electric Commander as the best CI solution for the job. It is possible to reuse parts of the FinalBuilder scripts and call them from Electric Commander but simply calling the FB script as one build step would result in you missing out on some of the key advantages of using Electric Commander - realtime log file processing, the ability to parallelize right down to individual step levels in Electric Commander and data collection and reporting.
Electric Commander has an API that exposes all product functionality which can be used in combination with AccuRev triggers to achieve a very flexible solution.
Disclaimer - I liked Electric Commander so much I joined the company and am currently employed by Electric Cloud.
You can try Electric Commander by going to www.electric-cloud.com and clicking on "Try It!"