My Intention is to test the functional flows of the application. So i think best way is testing UI for the steps of the functional flow. (E.g : Add item, after search it and add pricing etc.). My concern is to find best automation tool which should support for desktop application. does anyone have handheld experience of doing this. As a coding guy i don't have experience in Testing.
Additionally: the application is huge application which have connected with oracle database. So it need to clear the database as startup of the testing. So tool should support this matter also.
Programming language : C#
I found Selenium (but this is only for web application), Renorex and IBM rational Functional Tester.
is my approach(test UI steps for Functional steps) correct? have any one work on above tools or any other best tools whcih support to C# windows application.
I think NUnit is best for text unit test in this project. am i correct? according to my research google i found this as result.
I'll try to keep it short. A good fit for you case
automation tool which should support for desktop application.
and
Programming language : C#
will be the VisualStudio's build in Coded UI test. All actions performed on the desktop will be recorded. Pause the recording if you are performing actions that may lead to sensitive data being included in the recording. When coded UI tests are automated, they have to interact with the desktop when you run them, unlike other automated tests.
As far as you try to keep the tests close to the Test pyramid
the QA effort should be acceptable and your
approach(test UI steps for Functional steps)
will be correct and feasible. In general try to avoid long and fragile tests, except for some E2E tests that will exercise the (most business valuable) end-user journeys.
With C# should be relatively easy to drop oracle database for your
need to clear the database as startup of the testing.
Such DB cleaning is often used Fixture strategy. Take a look at this Persistent Fixture. Usage of such DB fixtures can be put to work for all conditions that your UI tests expect/rely on (avoid UI setup for tests), example - create available and applicable bonuses.
Another way for your UI layer tests could be some of the .Net based Sikuli implementations:
SikuliSharp
SikuliIntegrator
sikuli4net
Related
I'm coming up to speed with the Moq framework and I have a button on a form that launches another form.
The client wants the form to launch whether the Click or DoubleClick event is fired off.
My understanding of the Moq framework is that it is useful when you want to simulate certain types of actions: (For brevity, I'm only listing a few types of actions)
Connect to a database
Write to a file
Determine network connectivity
But I'm not 100% certain on how Moq can interact with Windows controls and events.
Can anyone put me in the right direction with a simple example?
I wouldn't use Moq or unit testing for that scenario.
You have code under a visual control that launches another form. That's inherently not testable in isolation. You could automate integration tests with other tools if you wanted to.
Moq is designed for enabling unit testing. Unit testing tests functionality (often logic) in isolation. It's difficult to apply to user interfaces.
Normally what you'd do is go with one of the Model-View-Controller variants, separate your presentation layer from your business logic, use interfaces and Dependency Injection to break dependencies, then use an isolation framework such as Moq to create fake implementations of those interfaces so that you could test subsystems independently.
If this sounds complicated, it is. It's also critical to writing large, tested applications.
I'd recommend starting by reading the book The Art of Unit Testing with Examples in .NET by Roy Osherove. That covers a lot of the basics.
what is the different purpose of those both? I mean, in which condition I should do each of them?
as for the example condition. if you have the backend server and several front-end webs, which one you'll do?do-unit testing the backend server first or do-UI testing in the web UI first?
given the condition, the server and the front-end webs already exist, so it's not an iterative design to build along with (TDD)...
Unit testing aims to test small portions of your code (individual classes / methods) in isolation from the rest of the world.
UI testing may be a different name for system / functional / acceptance testing, where you test the whole system together to ensure it does what it is supposed to do under real life circumstances. (Unless by UI testing you mean usability / look & feel etc. testing, which is typically constrained to details on the UI.)
You need both of these in most of projects, but at different times: unit testing during development (ideally from the very beginning, TDD style), and UI testing somewhat later, once you actually have some complete end-to-end functionality to test.
If you already have the system running, but no tests, practically you have legacy code. Strive to get the best test coverage achievable with the least effort first, which means high level functional tests. Adding unit tests is needed too, but it takes much more effort and starts to pay back later.
Recommended reading: Working Effectively with Legacy Code.
Unit test should always be done. Unittests are there to provide proof that each UNIT (read: object) of your technical solution delivers the expected results. To put it very (maybe too) simple, user testing is there to verify that your system fulfills the needs and demands of the user.
Test pyramid [1] is important concept here, well described by Martin Fawler.
In short, tests that run end-to-end through the UI are: brittle and expensive to write. You may consider test recording tools [2] to speed recording and re-recording up. Disclaimer - I'm developer of such tool.
[1] https://martinfowler.com/articles/practical-test-pyramid.html
[2] https://anwendo.com
In addition to the accepted answer, today I just came up with this question of why not just programmatically trigger layout functions and then unit-test your logic around that as well?
The answer I got from a senior dev was: programmatically trigger layout functions will not be an absolute copy of the real user-experience. In the real world, the system will trigger many callbacks, like when the user of an app backgrounds or foregrounds the app. Obviously you can trigger such events manually and test again, but would you be sure you got all events in all sequences right?!
The real user-experience is one where user makes actual network calls, taps on screens, loads multiple screen on top of each other and at times you might get system callbacks. Callbacks which you forgot to mock that you didn't properly mock. In unit-tests you're mainly testing in isolation. In UI test, you setup the app, may have to login, etc. That stack you build is much more complex vs a unit-test. Hence it's better to not mix unit-testing with UI testing.
I've heard of unit testing, and written some tests myself just as tests but never used any testing frameworks. Now I'm writing a wxPython GUI for some in-house data analysis/visualisation libraries. I've read some of the obvious Google results, like http://wiki.wxpython.org/Unit%20Testing%20with%20wxPython and its link http://pywinauto.openqa.org/ but am still uncertain where to start.
Does anyone have experience or good references for someone who sort of knows the theory but has never used any of the frameworks and has no idea how it works with GUIs?
I am on a Windows machine developing a theoretically cross-platform application that uses NumPy, Matplotlib, Newville's MPlot package, and wxPython 2.8.11. Python 2.6 with plans for 3.1. I work for a bunch of scientists, so there is no in-house unit-testing policy.
If you want to unit-test your application, you haven't to focus on GUI testing techniques. It is much better to write the application using MVC, MVP, or other meta-pattern like these. So you get business logic and presentation layer separated.
It is much more important to cover the business layer with tests since this is your code. Presentation layer is tested already by wxWidgets developers. To test the business layer it will be enough just basic tools like standard unittest module and maybe nose.
To make sure the whole application behave correctly, you should add few acceptance tests that will test functionality from end to end. These will deal with GUI, but there will be few such tests comparing to number of unit-tests.
If you will limit yourself with acceptance tests only, you'll get low coverage, fragile and very slow test code base.
To unit test your application without requiring lots of mock objects/stubs, your GUI's event handlers should basically delegate to other method calls, passing in values from the Event object as parameters to the delegated method.
Otherwise you'll be unable to test your application without having to mock wx's objects.
Take a look at the PyPubSub project for a great module to help with MVC.
In one early project of mine I really test wxPython application using GUI layer. So tests really spin live wxApp object, pops up real windows and then starts messing with a real MainLoop(). Very soon I realize it was a wrong way to do testing. My tests was run very slow and unreliable. Much better way is to separate GUI stuff aside and test only the "model" level of your application. Note that you can actually create model for presentation level logic (model that represent some visual part of your application) and test it. But this model should not involve any "real" gui objects (windows, dialogs, widgets).
As part of a new job, I have to devise and implement a complete test strategy for the company's new product. So far, all I really know about it is that it is written in C++, uses an SQL database and has a web API which is used by a browser client written using GWT.
As far as I know, there isn't much of an existing strategy, except for using Python scripts to test the web API.
I need to develop and implement a suitable strategy for unit, system, regression and release testing, preferably a fully automated one.
I'm looking for good references for:
Devising the complete test strategy.
Testing the web API.
Testing the GWT based application.
Unit testing C++ code.
In addition, any suitable tools would be appreciated.
Testing Computer Software is a great soup-to-nuts book on the entire testing process. In addition to the items you mentioned, you'll need to think about other types of testing (performance, security, localization, stress testing, to name a few) and how to manage the test process; test plans, issue tracking, test data, test cases, in addition to the tools.
There's a lot there, and you can't do everything at once. I think a phased approach would be best, where you identify the gaps, weaknesses, and risks in the current process, prioritize them, then set up a high level plan to address them one by one.
Software QA Testing and Test Tool Resources is a good starting place for finding some tools to fit your process. StickyMinds is a nice web site dedicated to software testing, and the folks here at StackOverflow certainly know their stuff, so don't be afraid to ask.
Good luck :)
You can find a ton of excellent information on testing and developing a test strategy in general over on James Bach's blog. Specifically by searching through it for tips on testing strategies.
James is an excellent resource for information about how to do great software testing.
Best of luck.
There's a good conversation here on The Purpose of a Test Strategy.
As testing tools you could use Selenium for web testing and CppUnit for c++ unit testing.
What vb6 UI automated testing tools are available?
Additionally, would you rate the tools listed from 1-10, where ten means that its the most amazing tool ever and one means that its almost not worth listing.
It's worth looking at some of the other questions about automatic UI testing. I don't think there's anything particularly special about testing a VB6 UI.
Have you created a sustainable and successful test suite using UI automation on your Windows application?
User interface testing