I am Unit testing on the client side of a GWT+SmartGWT application.
First I tested with GwtTestCase. Too long for unit testing a huge application. GwtTestSuite doesn't help. It still takes too much time to execute. (more, it asked me to start a browser when it's testing)
Then gwt-test-utils : Great framework. Sadly, my javassist version is 3.5 and need at least the 3.11. Javassist is used by gilead so I can't touch this. So, no gwt-test-utils...
I saw Selenium. That's just great. With htmlunit driver, it's fast and useful. Simplest way to test a webapp. Problem here is SmartGWT generates it's own IDs when it generates the web page. So I can't get the TextItems and fill them since those IDs are constantly changing. I found that it could be solved by using setID() before the initialization of the widget. But that's the ID of the scLocator and not an HTML ID. Selenium doesn't want to work with scLocator.
=> Is there a simple way to accept scLocator with Selenium ?
(And when I say simple, it must be simple... I'm not a specialist in Java...)
Could someone help me to unit test the application ? It's coded in Java, it's huge and I have to cover ~70% of the code (25k lines of code)
Some more specs :
Maven is used to compile.
I'm not touching at the server side.
Tests must be faster than GwtTestCase and Suite :/
I hope my problem is clear, I'm not a native english so sorry for mistakes I may do :x
We provide Selenium extensions and a user guide for them in the SDK, under the "selenium" directory at top level.
If you download 3.1d (from smartclient.com/builds) there's even more documentation including some samples for JUnit.
Do not use ensureDebugId() (won't have an effect at all). Never try to work with DOM IDs (won't work).
Best practices information in the Selenium user guide explains when you should use setID().
I found that it could be solved by using setID() before the
initialization of the widget. But that's the ID of the scLocator and
not an HTML ID.
Why don't you try:
widget.ensureDebugId("some-id");
From the Java docs for ensureDebugId():
Ensure that the main Element for this UIObject has an ID property set,
which allows it to integrate with third-party libraries and test tools
< defaultUserExtensionsEnabled>true< /defaultUserExtensionsEnabled>
< userExtensions>[path to user-extensions.js]< /userExtensions>
There we go. I managed to make it work. (With the selenium-maven-plugin in the < configuration> tag)
Thanks for your help though.
Related
Unit tests are strongly connected to the existing code of the application, written by developers. But what about UI and API automated tests (integration tests)? Does anybody think, that it's acceptable to re-use code of the application in separate automation solution?
The answer would be no. UI tests follow the UI, go to that page, input that value in that textbox, press that button, I should see this text. You don't need anything code related for this. All this should be done against some acceptance criteria so you should already know what to expect, without looking at any code.
For the API integration tests you would call the endpoints, with some payloads and then check the results. You don't need references to any code for this. The API should be documented and explain very well what endpoints are available, what the payloads look like and what you can expect to get back.
I am not sure why you'd think about reusing application code inside an automation project.
Ok, so after clarifications, you're talking about reusing models only, not actual code. This is not a bad idea, it can actually help, as long as these nuget packages do not bring in any other dependencies.
Code Reusability is a great concept but it's very hard to get right in practice. Models typically come with adnotations which require other packages which are of course not needed in an automation project. So, if you can get nuget packages without extra dependencies, so literally data models only and nothing else then that does work. Anything more than that and it will create issues so I'd push back on that
After learning about TDD and unit testing, I'm really looking to write tests when I code, preferably before I code because I see the benefits coding in Python. I'm developing a website, and I'm trying to write tests according to the requirements, but its proving more difficult than expected.
I can see the benefits of writing tests when you're producing a library of code with a public interface for others to use. Developing a website, where there is really not much logic, and mostly Reading and Writing against a database seems a little harder to unit test. Mostly, I have to create/edit/delete rows in the database.
I'm using a framework (Kohana 3 for php), so 99% of all the libraries and helpers that I'm going to be using have already been tested (hopefully), so what else is their to write tests for?
I'm mostly talking about scripting languages, not about CSS or HTML, I already know about cross-browser testing.
How much can you really test when developing a web site, and how should you go about it?
Edit: Is the lack of activity on this question a sign? I understand that certain things MUST be tested, like security and the like, but how much can be written using unit tests and TDD is the question.
Thanks.
Developing a website, where there is really not much logic, and mostly Reading and Writing against a database seems a little harder to unit test. Mostly, I have to create/edit/delete rows in the database.
Not completely true.
You have data model processing. Does the validation work? Do the calculations on the reported rows from the database work?
You have control, sequence and navigation among pages -- do the links work? The test setup will provide a logged-in-user. The test will (1) do a GET or a POST to fetch a page, then (2) confirm the page actually loaded and has the right stuff.
You have authorization -- who can do what? Each distinct test setup will provide a different logged-in-user. The tests will (1) attempt a GET or POST to process a page. Some tests will (2) confirm they got directed to error-response pages. Some tests will (2) confitrm that the GET or POST worked.
You have content on the page -- what data was fetched? The test setup will provide a logged-in-user. The test will (1) do a GET or a POST to fetch a page, then (2) confirm the page actually loaded and has the right stuff.
Have you tried Selenium? It allows you to automatically do almost anything in a web browser. For example, you could have it go through and click all of the links and make sure that they go to the correct url.
It works with multiple languages, including python and allows for testing in chrome, firefox, ie, and other browsers.
If your site contains many forms, how do you write them? Do you write each view using plain HTML? Or do you write your own form helpers that generate forms just the way you want them? If you do that, you may find that unit-testing your form generators makes it easier to write them.
In general, if your program is mostly CRUD, look out for ways to automate CRUD management; write your own custom CRUD generator. Which does not mean write the CRUD framework that will end all frameworks; that would be too much work. Just write a generator for the small things you need for your current application. TDD will help you there.
I'm a complete selenium newbie. Where should I start if I want to learn how to use it?
I've used Selenium in two different ways:
Selenium Remote Control - Great at scripting tests that manipulate a large amount of data to hit or insert into a website because you can manipulate the Selenium commands from a real programming language (I used Ruby).
Selenium IDE - Excellent at making quick scripts to demonstrate problems and single path execution of a website. The Firefox plugin integration is excellent and easy to use. Non-developers can even create and submit scripts to demonstrate problems.
Pick one of the two ways and write a script to do something routine on the web (check for latest Hockey scores from NHL.com, etc). After you have that super-simple script running, try something more complex.
Download the firefox plugin to begin. You can record, replay some tests. There are some video tutorials.
Look at this intro
You can get this book Selenium 2 Testing Tools Beginner’s Guide.
Really helpful book, especially for a newbie. I had a good experience, learning Selenium with it.
I would suggest to start from an overview automated web testing with selenium presentation on scribd
Better idea, to start selenium IDE, which is record and playback method like other tools.
Once familiar, then can start using selenium RC, Grid, Webdriver.
Selenium IDE is firefox extension.
more help http://seleniumhq.org/docs/02_selenium_ide.html
Really depends on what you want to do with Selenium.
I definitely recommend getting Selenium IDE if you don't know what you're doing because it does most things automatically and gives you an intro to how you would use Selenium Remote Control.
Selenium Remote Control lets you use Selenium from your favourite language. From there it depends on what language you want to use for where you get help. I use Java/JUnit and I just use google/documentation/stackoverflow whenever there's something I'm not sure how to do and that Selenium IDE won't tell me how to do.
Aside from the other suggestions on this page, one of the most useful sites I've used to get up to speed with Selenium was this one: Selenium Training from theautomatedtester.com
He covers
Record & Playback,
Javascript & Variables,
XPaths,
Regular Expressions and more;
However, I think I only needed those first few ones.
Also, there are many extensions/addons/plug-ins for Selenium IDE, so be sure to check for any of those if you get stuck in some way.
For example, I thought I needed to start using the Selenium Remote Control (and code in C#) because I wanted to create tests with logical flows and loops...but there is a plug-in for incorporating that in the IDE (goto & while loops extension), so I didn't need to!
Hope this helps!
Where to start?
Go to: http://seleniumhq.org/docs/
Read the docs from the beginning to the end.
Estimated time 8 hours.
We develop custom survey web sites and I am looking for a way to automate the pattern testing of these sites. Surveys often contain many complex rules and branches which are triggered on how items are responded too. All surveys are rigorously tested before being released to clients. This testing results in a lot of manual work. I would like to learn of some options I could use to automate these tests by responding to questions and verifying the results in the database. The survey sites are produced by an engine which creates and writes asp pages and receives the responses to process into a database. So the only way I can determine to test the site is to interact with the web pages themselves. I guess in a way I need to build some type of bot; I really don't know much about the design behind them.
Could someone please provide some suggestions on how to achieve this? Thank you for your time.
Brett
Check out selenium: http://selenium.openqa.org/
Also, check out the answers to this other question: https://stackoverflow.com/questions/484/how-do-you-test-layout-design-across-multiple-browsersoss
You could also check out WatiN.
Sounds like your engine could generate a test script using something like Test::WWW::Mechanize
Usual test methodologies applies; white box and black box.
White box testing for you may mean instrumenting your application to be able to make it go into a particular state, then you can predict the the result you expect.
Black box may mean that you hit a page, then consider of the possible outcomes valid. Repeat and rinse till you get sufficient coverage.
Another thing we use is monitoring statistics for our service. Did we get the expected number of hits on this page. We routinely run a/b tests, and I have run a/b tests against refactored code to verify that nothing changed before rolling things out.
/Allan
I can think of a couple of good web application testing suites that should get the job done - one free/open source and one commercial:
Selenium (open source/cross platform)
TestComplete (commercial/Windows-based)
Both will let you create test suites by verifying database records based on interactions with the web app.
The fact that you're Windows/ASP based might mean that TestComplete will get you up and running faster, as it's native to Windows and .NET. You can download a free trial to see if it'll work for you before making the investment.
Check out the unit testing framework 'lime' that comes with the Symfony framework. http://www.symfony-project.org/book/1_0/15-Unit-and-Functional-Testing. You didn't mention you language, lime is php.
I would suggest the mechanize gem,available for ruby . It's pretty intuitive to use .
I use the QEngine(commerical) for the same purpose. I need to add a data and check the same in the UI. I write one script which does this and call that in a loop. the data can be passed via either csv or excel.
check that www.qengine.com , you can try Watir also.
My proposal is QA Agent (http://qaagent.com). It seems this is a new approach because you do not need to install anything. Just develop your web tests in the browser based ide. By the way you can develop your tests using jQuery and java script. Really cool!
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
The company I'm currently working for is using Selenium for Uniting-Testing our User Interface. What do you use to Unit-Test your Web UI and how effective do you find it?
I'm a huge fan of Selenium. Saying 'unit-testing your web ui' isn't exactly accurate as some of the comments have mentioned. However, I do find Selenium to be incredibly useful for performing those sort of acceptance and sanity tests on the UI.
A good way to get started is using Selenium IDE as part of your development. Ie, just have the IDE open as you're developing and write your test as you go to cut down on your dev time. (Instead of having to manually go through the UI to get to the point where you can test whatever you're working on, just hit a button and Selenium IDE will take care of that for you. It's a terrific time-saver!)
Most of my major use case scenarios have Selenium RC tests to back them up. You can't really think of them as unit-tests along the lines of an xUnit framework, but they are tests targetted to very specific functionality. They're quick to write (especially if you implement common methods for things like logging in or setting up your test cases), quick to run, and provide a very tight feedback loop. In those senses Selenium RC tests are very similar to unit-tests.
I think, like anything else, if you put the effort into properly learning a test tool (eg, Selenium), your effort will pay off in spades. You mention that your company already uses Selenium to do UI testing. This is great. Work with it. If you find Selenium hard to use, or confusing, stick with it. The learning curve really isn't all that steep once you learn the API a little bit.
If I'm working on a web app, its rare for me to write a significant amount of code without Selenium RC tests to back it up. That's how effective I find Selenium. :) (Hopefully that'll answer your question..)
We use Watin at my place of employment, we are a .net shop so this solution made a lot of sense. We actually started with Watir (the original ruby implementation) and switched after. It's been a pretty good solution for us so far
Well, if you've designed your application properly, you won't have scads of logic inside the UI anyway. It makes much more sense to separate the actual work getting done into units separate from the UI, and then test those.
If you do that, then the only code in the UI will be code that invokes the backend, so simply testing the backend is sufficient.
I have used NUnit ASP in the past (at my job), and if you insist on unit testing your UI, I would strongly advise you to use ANYTHING but NUnit ASP. It's a pain to work with, and tests tend to be invalidated (needing to be revised) after even the most minor UI changes (even if the subjects of the tests don't actually change).
We are using QuickTestPro. So far it is effective, but the browser selection is limited. The nicest part is the ability to record your browser's activity, and convert it into a scriptable set of steps. There is also a nice .Net addin so if you have any validation code you need to do for the different stages of your test, you can write methods in an assembly and call them from your script.
We use Visual Studio 2008 Tester Edition.
Pros:
Very good at capturing user interaction
Captures Ajax calls
It is very easy to map user input to a database, XML or CSV file
The captured test can be converted to C# for more control
The same tests can be used for load testing and code coverage
Cons:
VS2008 Tester Edition is a seperate SKU from the normal Developer Edition, which means extra cost
You may be alergic to Microsoft ;-)
We have used it very effectively on projects, however there a lot of effort involved in keeping tests up to date, every time you change a screen the test may need to be re-recorded
We tend to keep the tests short and sharp, do one thing and get out instead of recording 10 minutes worth of clicking around in a single test.
We have a few standard UI test types:
Menu Test: Log in as a specific user (or user type/role) and make sure all the required menu items are available
Validation Test: Open a page and click save without entering any data, ensure that all the validation warnings appear. Complete required fields one at a time and check that the warning messages disappear when they are supposed to.
Search Test: Search using data from your database or a data file and ensure the correct data is returned by the search
Data Entry Test: Create new recrords from a data file, cleanup the database to allow tests to run multiple times
UI Testing is quite time consuming but the comfort feeling you get when a few hundred tests pass before you release a new version is priceless.
We have been using JSunit for a while to do unit tests... it may not be the same kinds of tests you are talking about, but it is great for ensuring your JavaScript works as you expect.
You run it in the browser, and it can be set in an Ant build to be automatically run against a bunch of browsers on a bunch of platforms remotely (so you can ensure your code is cross-browser as well as ensure the logic is correct).
I don't think it replaces Selenium, but it complements it well.
We use Selenium Core, but are switching gradually to Selenium RC which is much nicer and easier to manage. We have written lots of custom code to make the tests run on our Continuous Integration servers, some of them in parallel suites to run faster.
One thing you'll find is that Selenium seems to restart the browser for each test (you can set it not to do this, but we got memory problems when we did that). This can be slow in Firefox, but is not too bad in IE (one time I'm thankful for Bill Gates's OS integraion).
I've used WATIR, which is pretty good. I liked it because it's Ruby and allows for testing interactivity, available elements and source code parsing. I haven't used it for a while but I assume it's gotten better.
It's supposedly being ported to Firefox and Safari, but that's been happening for a while now.
Check out Canoo Web Test. It is open source and built on the ANT framework.
I spent some time working with it for a graduate course on Software QA and it seems to be a pretty powerful testing tool.
Selenium Grid can run your web tests across multiple machines in parallel, which can speed up the web testing process
I mostly use CubicTest, which is an eclipse plugin that lets you define tests graphically. It can export/run tests through several libraries, including watir and selenium. Most people just use the Selenium runner though.
Full disclosure: I'm one of the developers, so I'm kind of biased :)
Take a closer look here: cubictest.openqa.org
-Erlend
Selenium is for Integration testing, not Unit testing. It's a subtle, but important difference. The usage I usually see is for sanity checking a build. i.e., have a test that logs in, a test that (for example) submits a story, makes a comment, etc.
The idea is that you're testing to see if the whole system is working together before deployment, rather than have a user discover that your site is broken.
We currently use Silk4J - a Java centric approach to testing Web UI. It can test Flash, Flex, AIR, Silver Light, Win32, HTML, and a few other applications.
Since Silk4J can control Win32 apps it can control browser dialogs directly, which is a step above what Selenium can control and is especially useful for download prompts.
We use WatiN for system testing, and QUnit for JavaScript unit testing.
Molybdenum is built over Selenium and has some additional features.