I am looking to automate control clicks, listview clicks and so forth, is there any good framework for qt that can help? I need to perform these tests without the source code
Qt provides its own framework for unit testing: QTestLib
This library is part of every Qt SDK.
You can find a tutorial here, Especially the points 3 and 4 will show a way that might be sufficient for your GUI testing needs.
Obviously, starting from point 1 won't hurt.
For a full-blown UI testing framework, have a look at Squish. It supports scripting of UI interactions (with Python, tcl, javascript), property and screenshots comparisons. It requires you to purchase license(s), but it's well worth the money if you do commercial software development.
Related
I'm just starting out with my Marmalade career with 6.1.
I'm aware they discontinued their UI builder in favor of 3rd party solutions. However, "Marmalade Ui Builder" in google brings up nothing in the first 3 pages (past 3 pages is always too far).
Are there such 3rd party tools? Or are people actually using notepad and writing ui's by hand?
I've seen an XML UI Builder that simply parses XML in to the .ui style. However, I don't see how that's any easier than typing everything out manually.
It also is ambiguous as to whether IwUI is actually the proper way to go about it anymore.
Some insight would be wonderful.
IwUI is old system replaced with IwNUI. N states for Native.
IwNUI uses native controls for each platform.
I have not used Iw*UI for games it is more simpler to write interface in GL myself.
Basic Screens/Buttons manager is coded in few hours.
It is simpler to write such thing your self then learn that is has something you can't change.
If you develop business apps then Marmalade added PhoneGap support. I believe PhoneGap to be a better way for business cross-platform apps. Especially because most mobile platforms share WebKit engine. What is more important Marmalade Team them selves do not believe in Iw*UI and that is why they are forcing PhoneGap on Marmalade.
In other words do not use Iw*UI if do not know why you actually need it.
There is 3rd party 2D game engine IwGame.
I do not use it but it has its own Interface Builder that defines interface in XML.
unfortunatelly I have a GUI desktop application running on Windows and compiled using Borland C++ compiler. I know it's a very old technology, but that's the life. The company I work for have lots of free licences for QTP. But to be honest I don't like this tool and I want to change it with some open source alternative.
I have an experiance with automation Web services using Selenium, and some java swing applets.
I created my tests based on Java/JUnit. I'm wondering if I can create some tests in Java to test my C++ application - that would be great. If that's not possible if I can use any C/C++ libraries to create my own test framework.
That shouldn't be unit tests. I want to create rather a suite of functional tests.
I want to be able to localize objects like buttons, tables, cells, etc., perform action like clicking, changing focus, etc.
In addition I have access to source. But this is rather an old spaghetti code and I don't have much experiance with C++ apps.
What do you suggest?
We use Sikuli and have been quite happy with it. It works outside of the various automation APIs so it handles non-standard UIs very well.
Open Source Tools
White is a .NET based UI automation tool that can automate win32 applications among others types. As long as the controls used in your app expose their content/behaviour using Windows' standard UIAutomation hooks then you should be able to use it to do what you need.
UIAutomation Verify is another CodePlex project that works with the same APIs to automate UIA compliant UIs.
Visual Studio
Finally if you have access to it then Visual Studio 2010 has scripted UI test functionality built in Premium/Ultimate editions - Coded UI Tests. This supports .NET, native and web applications with record/replay and scripting functionality.
Robert, if your company already has licenses of QTP, I suggest you use that. It's the right tool for the job, and there's no sense reinventing the wheel if your company already owns it. It has its warts - my main objections are to the limited IDE and VBScript scripting language - but it did not become the market-leading tool by accident. There are not a lot of open-source options for testing a Windows C++ GUI app. The main commercial options you should consider are HP QTP, IBM Rational Functional Tester and VS2010 Coded UI Tests.
I agree with Robert that QTP is sometimes not the best tool, especially when you want it to wait for the desktop application to finish its processing. QTP commands are not synchronous, meaning that QTP does not wait until the previous command finishes execution before moving to the next call. We had a lot of issues with this. We are now planning to write a c# dll that can do this for QTP and include that in our QTP programs to make QTP to dynamically wait until the Desktop application finishes its processing. Currently, to my knowledge, no such wait functionality is supported by QTP.
Also, coming back to the object recognition issue, We did have a lot of issue with this and took the alternative route of invoking commands using Mnemonics through keystrokes in QTP. As Simon has already mentioned, this could be due to non-exposure of the GUI elements. We did not have much control in this regard and so we did not investigate further on this.
We have existing WPF application with complicated graphics (rounded edges of the window, speedometer style indicators etc).
Since we rely on .NET our application needs .NET framework. Some users are unwilling to install .NET and we want to provide them with a version that does not need .NET and runs of Windows only (XP,Vista,W7).
Which graphical toolkit we should use to for this scenario?
So far I think Codejock Toolkit Pro is a good adept as we already have GUI in XAML and it could be easier to port GUI to C++.
However, another aspect is download size of the toolkit. We don't want to end up in the situation where toolkit size is so big (e.g. 20-40MB) that's comparable with .NET 4.0 framework size.
I know some of you might think that going back from WPF to C++ is a bad idea, but for our scenario it is absolutely crucial to make user's installation process as easy as possible and .NET framework is a big stumbling block which we are trying to remove.
Sorry I do not have the answer to your question (and I would never ever even dream that I might do such a thing), but I am compelled to tell you as an advice
implementing all the WPF functionality in C++ will take 10-20 times more
installation of a C++ would not be necessarily easier
if you need to make installation easier, use SilverLight
We used QT in one of our projects in past which is Object Oriented and Multi Platform, but QT is mostly like WinForms
As I remember QT keeps UI in xml files too, try to write XSLT from xaml to QT xml. But I don't thik you can do it in complex WPF structures like Triggers or Animation
Have you evaluated QT? Since this relies on XML for UI definition too, it might be easier to port the code to.
You've already gotten a couple of answers recommending Qt -- and I'd agree that's probably your first choice. Note, however, that while Qt does support an XML-based UI description, they seem to be moving away from it toward QML, which is based on JavaScript instead.
Another possibility would be wxWidgets, which also supports an XML-based description of the widgets using XRC.
I've heard of unit testing, and written some tests myself just as tests but never used any testing frameworks. Now I'm writing a wxPython GUI for some in-house data analysis/visualisation libraries. I've read some of the obvious Google results, like http://wiki.wxpython.org/Unit%20Testing%20with%20wxPython and its link http://pywinauto.openqa.org/ but am still uncertain where to start.
Does anyone have experience or good references for someone who sort of knows the theory but has never used any of the frameworks and has no idea how it works with GUIs?
I am on a Windows machine developing a theoretically cross-platform application that uses NumPy, Matplotlib, Newville's MPlot package, and wxPython 2.8.11. Python 2.6 with plans for 3.1. I work for a bunch of scientists, so there is no in-house unit-testing policy.
If you want to unit-test your application, you haven't to focus on GUI testing techniques. It is much better to write the application using MVC, MVP, or other meta-pattern like these. So you get business logic and presentation layer separated.
It is much more important to cover the business layer with tests since this is your code. Presentation layer is tested already by wxWidgets developers. To test the business layer it will be enough just basic tools like standard unittest module and maybe nose.
To make sure the whole application behave correctly, you should add few acceptance tests that will test functionality from end to end. These will deal with GUI, but there will be few such tests comparing to number of unit-tests.
If you will limit yourself with acceptance tests only, you'll get low coverage, fragile and very slow test code base.
To unit test your application without requiring lots of mock objects/stubs, your GUI's event handlers should basically delegate to other method calls, passing in values from the Event object as parameters to the delegated method.
Otherwise you'll be unable to test your application without having to mock wx's objects.
Take a look at the PyPubSub project for a great module to help with MVC.
In one early project of mine I really test wxPython application using GUI layer. So tests really spin live wxApp object, pops up real windows and then starts messing with a real MainLoop(). Very soon I realize it was a wrong way to do testing. My tests was run very slow and unreliable. Much better way is to separate GUI stuff aside and test only the "model" level of your application. Note that you can actually create model for presentation level logic (model that represent some visual part of your application) and test it. But this model should not involve any "real" gui objects (windows, dialogs, widgets).
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
I am working on the issue of testing my GUI and I am not entirely sure of the best approach here. My GUI is built using a traditional MVC framework so I am easily able to test the logic parts of the GUI without bringing up the GUI itself. However, when it comes to testing the functionality of the GUI, I am not really sure if I should worry about individually testing GUI components or if I should mainly just focus on functional testing the system. It is a pretty complex system in which testing the GUI frequently involves sending a message to the server and then observing the response on the GUI. My initial thoughts are that functional testing is the way to go here since I need a whole system running to really test the UI. Comments on this issue would be appreciated.
Other GUI-testing tools I can offer are:
Thoughtworks White,
PyWinAuto,
AutoIt,
AutoHotKey.
One thing to keep in mind when trying to automate GUIs is that the only way you can do that is to build the GUI with automation in mind. Crush devs that think their GUIs should not support testability early on in the project and happily expose all the hooks that can help in automation on demand as your testing needs require that.
You have (at least) 2 issues - the complexity of the environment (the server) and the complexity of the GUI.
There are many tools for automating GUI testing. All of them are more or less fragile and require pretty much constant maintenance in the face of changing layout. There is benefit to be gained from using them, but it's a long term benefit.
The environment, on the other hand, is an area that can be tamed. If your application is architected using the Dependency Injection/Inversion technique (where you 'inject' the server component into the application), then you can use a 'mock' of the relevant server interfaces to enable you to script test cases.
Combining these two techniques will allow you to automate GUI testing.
Depending on where in the spectrum of MVC (that's an overused term) you sit, testing the view could be a mechanical process of ensuring that the correct model methods are called in response to the correct inputs to the view to testing some client side validation to who knows.
A lot of the patterns that have been evolved out of MVC (I'm thinking passive view, supervising controller) are striving to make the view require very little testing because it's really just wiring user inputs to the presenter or model (depending on the exact variant of the pattern you're using).
"testing the GUI frequently involves sending a message to the server and then observing the response on the GUI" This statement worries me.
I'm immediately thinking that the GUI should be tested using a mock or stub of the server to test that the correct interactions are occurring and the GUI responds appropriately.
If you need automated functional tests of the server, I don't see the need to have the GUI involved in those.
Mercury QuickTest Pro, Borland SilkTest, and Ranorex Recorder are some GUI testing tools.
If your application is web-based you can write tests using tools like WatiN or Selenium.
If your application is Windows .NET based, you could try White.
My advice: forget the traditional GUI testing. It's too expensive. Coding the tests takes a lot of time, the tools aren't really stable so you will get unreliable test results. The coupling between the code and the test is very strong and you'll spend a lot of time with the maintenance.
The new trend is to ignore the GUI tests. See the ModelViewPresenter pattern from Fowler as a guideline link text
The clearest way I can say this is:
Don't waste your time writing automated GUI tests.
Especially when your working with an MVC app - in your case, when you send a message to the server, you can make sure the right message number comes back and be done. You can add some additional cases - or another test completely to make sure that the GUI is converting the message id's into the right strings, but you just need to run that test once.
We do incorporate GUI testing in our project, and it has its side effects. The developers however have one critical design principle: Keep the GUI layer as thin as possible!
That means no logic in the GUI classes. Separate this in presentation models responsible for input validation etc.
For testing on a Unix machine we use the Xvfb server as the DISPLAY when running the tests.
Try the hallway usability test. It's cheap and useful: go to the nearest hallway, grab the first person that passes, make them sit at your computer and use your software. Watch over their shoulder, you will see what they try to do, what frustrates them, and so on. Do this a few times and notice the patterns.
What you're looking for is "acceptance testing." How you do it depends on the frameworks you're using, what type of application you are creating and in what language. If you google your particular technology and the above phrase, you should find some tools you can use.
I've found WinTask to be a very good way to do GUI testing. Provided you don't constantly change the way the OS refers to each element of the UI, WinTask addresses the UI elements by name, so even if the layout changes, the UI elements can still be pressed / tweaked / selected.
Don't miss the 'U' in 'GUI'
I mean: if what you're trying to test is all works right and works as it was planned to work, then you may follow Seb Rose's answer.
But please, don't forget a USER interface has to be made thinking about USERS, and not ANY user but the TARGET USER the application was made for. So, after you are sure all works like it have to work, put every single view/screen/form in a test with a team made of users representing every group of different users that may use your application: advanced users, administrators, MS Office users, low computer profile users, high computer profile users... and then, get the critiques of every user, make a mix, re-touch your GUI if it's neccesary and back again to GUI user's test.
For SIMPLE Web based GUI testing try iMacros ( a simple Firefox plug-in , has a cool feature to send the entire test to another person )
Note that SIMPLE was spelled with Initials ...