Currently we are generating HTML Reports for Automation, but those reports are not good enough to explain number of scenario which we cover in Automation, Is there anything we can use with Selenium to generate a proper reports which can give a complete overview and can easily understand by anyone
First Thing we can show a complete pie charts which cover number of test case passed and Failed.
Second thing we can show, what are test cases are there in this build.
You may also checkout the page http://seleniumtesting-nx.blogspot.de/2012/03/testng-xslt-report-generation.html
I found it useful. :)
It's easy to write your own TestNG reporter, look up IReporter in the doc.
Related
How to control output from Twisted-trial tests?
I have looked up for different solutions, but I'm quite new to testing, so I can't find a fitting solution or can't use it correctly.
In general, I try to make autotesting system for my project like BuildBot. But BuildBot doesn't fit me because it reacts only to "On change sources" hook from Mercurial and I want to use other hooks too.
On THIS page from BuildBot documentation I found this information:
One advantage of trial is that the Buildbot happens to know how to
parse trial output, letting it identify which tests passed and which
ones failed. The Buildbot can then provide fine-grained reports about
how many tests have failed, when individual tests fail when they had
been passing previously, etc.
Does it mean that there is no way but to parse information from test-output?
Other possible solutions?
Besides, I looked up in Twisted documentation and found this class IReporter.
Is it a solution and if it is, how can I use it?
If it isn't, are there any other solutions?
P.S. Please, note, that tests have already been written, so I can only start them and can't modify source code.
You can format output from trial arbitrarily by writing a reporter plugin. You found the interface for that plugin already - IReporter.
Once you write such a plugin, you'll be able to use it by adding --reporter=yourplugin to your trial command line arguments.
You can see the list of reporter plugins that are already available on your system using trial --help-reporters. If you have python-subunit installed then you'll see subunit which is a machine-parseable format that might already satisfy your requirements. Unfortunately it's still a subunit v1 reporter and subunit v2 is better in a number of ways. Still, it might suffice.
My Django app is in need of some automated testing.
Many of the views produce tabular data (from a generic list view). I have created fixtures to test some of the more complex cases that have been causing subtle bugs.
What should I be using to test the value in a specific table cell (or column)?
There seems to be a lot of testing tools / libraries out there django-test client, Selenium, Nose. A lot of things seem to be aimed at unit testing (while I am not finding so many bugs at this level). I am looking more to integration testing. Reading all the documentation for all the libraries is going to take a while to find what I want.
So can someone advise what libraries / tools I should use to check the final output values in my list view's tabular output? I would like to give a URL, and confirm that the page returned has a value in particular row / column is equal to my expected value.
There seems to be a lot of testing tools / libraries out there django-test client, Selenium, Nose. A lot of things seem to be aimed at unit testing (while I am not finding so many bugs at this level). I am looking more to integration testing.
It works very well for integration testing. Maybe you would have found out if you had tried ? Maybe it's time for you to (re?) read about the correct hacker attitude
Also, I've been having loads of fun with ghost.py
Reading all the documentation for all the libraries is going to take a while to find what I want.
It's your work to do research as well. Believe it or not it took me like 5 hours to check all solutions yesterday and decide to go with ghost.py and get it to work nicely with django (hence the gist upload !).
But yeah, if you don't want to learn anything new then you're stuck at "Not knowing out to do integration testing". If you want to learn "how to make integration testing" then you have to do research. There's no secret my friend :)
We're developing alot of enhancements to salesforce using Visualforce and Apex as part of a larger system, as part of our quality metrics we have to provide a report to management on our Code Coverage.
I'd like to get a report similar to the one produced by Run All Tests in the Force.com IDE but in HTML so I can display it easily via a web interface.
For the rest of our system we use Sonar http://www.sonarsource.org/ to produce the reports.
Does anybody know the best approach to this?
I've explored the API documentation but am unable to find out if the Coverage Percentages is stored against the classes so querying that isn't an option.
Any help or pointers would be greatly appreciated.
If you run the Apex tests yourself via the API there are objects returned indicating which lines hasn't been covered by the tests in that run. You can run the tests via either the synchronous or asynchronous methods.
Then you can use the data to create a report in a format that you require. For example, I've used it to create a basic report in the FuseIT SFDC Explorer (Windows based and free). I'm just dumping out line ranges that aren't covered.
You will probably want to run all the tests in one run to get the complete code coverage of all the tests. For example, in the screenshot above I only ran one out of a much greater number of test classes. As a result it looks like the code coverage was much lower than the cumulative tests give. It does however show which lines an individual test class reaches.
I've also heard good things about MavensMate for the Sublime Text Editor. Being open source you should be able to find how it integrates with the testing api and then generates the reports.
I want every steps to write a Nbehave testcase. I have gone through the concept, but i don't know how to start and where to start. If any one can give an example with all the steps then it would be a great help.
There's a full example on the NBehave documentation page.
http://nbehave.codeplex.com/wikipage?title=Getting%20started&referringTitle=Documentation
However even though I really like NBehave, if you need to fit in with other DotNet tools, such as test coverage tools or CI servers, I find there is better integration with SpecFlow, http://specflow.org/
We develop custom survey web sites and I am looking for a way to automate the pattern testing of these sites. Surveys often contain many complex rules and branches which are triggered on how items are responded too. All surveys are rigorously tested before being released to clients. This testing results in a lot of manual work. I would like to learn of some options I could use to automate these tests by responding to questions and verifying the results in the database. The survey sites are produced by an engine which creates and writes asp pages and receives the responses to process into a database. So the only way I can determine to test the site is to interact with the web pages themselves. I guess in a way I need to build some type of bot; I really don't know much about the design behind them.
Could someone please provide some suggestions on how to achieve this? Thank you for your time.
Brett
Check out selenium: http://selenium.openqa.org/
Also, check out the answers to this other question: https://stackoverflow.com/questions/484/how-do-you-test-layout-design-across-multiple-browsersoss
You could also check out WatiN.
Sounds like your engine could generate a test script using something like Test::WWW::Mechanize
Usual test methodologies applies; white box and black box.
White box testing for you may mean instrumenting your application to be able to make it go into a particular state, then you can predict the the result you expect.
Black box may mean that you hit a page, then consider of the possible outcomes valid. Repeat and rinse till you get sufficient coverage.
Another thing we use is monitoring statistics for our service. Did we get the expected number of hits on this page. We routinely run a/b tests, and I have run a/b tests against refactored code to verify that nothing changed before rolling things out.
/Allan
I can think of a couple of good web application testing suites that should get the job done - one free/open source and one commercial:
Selenium (open source/cross platform)
TestComplete (commercial/Windows-based)
Both will let you create test suites by verifying database records based on interactions with the web app.
The fact that you're Windows/ASP based might mean that TestComplete will get you up and running faster, as it's native to Windows and .NET. You can download a free trial to see if it'll work for you before making the investment.
Check out the unit testing framework 'lime' that comes with the Symfony framework. http://www.symfony-project.org/book/1_0/15-Unit-and-Functional-Testing. You didn't mention you language, lime is php.
I would suggest the mechanize gem,available for ruby . It's pretty intuitive to use .
I use the QEngine(commerical) for the same purpose. I need to add a data and check the same in the UI. I write one script which does this and call that in a loop. the data can be passed via either csv or excel.
check that www.qengine.com , you can try Watir also.
My proposal is QA Agent (http://qaagent.com). It seems this is a new approach because you do not need to install anything. Just develop your web tests in the browser based ide. By the way you can develop your tests using jQuery and java script. Really cool!