As part of a new job, I have to devise and implement a complete test strategy for the company's new product. So far, all I really know about it is that it is written in C++, uses an SQL database and has a web API which is used by a browser client written using GWT.
As far as I know, there isn't much of an existing strategy, except for using Python scripts to test the web API.
I need to develop and implement a suitable strategy for unit, system, regression and release testing, preferably a fully automated one.
I'm looking for good references for:
Devising the complete test strategy.
Testing the web API.
Testing the GWT based application.
Unit testing C++ code.
In addition, any suitable tools would be appreciated.
Testing Computer Software is a great soup-to-nuts book on the entire testing process. In addition to the items you mentioned, you'll need to think about other types of testing (performance, security, localization, stress testing, to name a few) and how to manage the test process; test plans, issue tracking, test data, test cases, in addition to the tools.
There's a lot there, and you can't do everything at once. I think a phased approach would be best, where you identify the gaps, weaknesses, and risks in the current process, prioritize them, then set up a high level plan to address them one by one.
Software QA Testing and Test Tool Resources is a good starting place for finding some tools to fit your process. StickyMinds is a nice web site dedicated to software testing, and the folks here at StackOverflow certainly know their stuff, so don't be afraid to ask.
Good luck :)
You can find a ton of excellent information on testing and developing a test strategy in general over on James Bach's blog. Specifically by searching through it for tips on testing strategies.
James is an excellent resource for information about how to do great software testing.
Best of luck.
There's a good conversation here on The Purpose of a Test Strategy.
As testing tools you could use Selenium for web testing and CppUnit for c++ unit testing.
Related
One of the problems our group is having is in regards to sharing relevant information between the Devs, Devs in Test and QAs.
How is this typically done in the industry?
For instance, our devs typically follow TDD with CI/CD pipelines offering the capability to illustrate code coverage. Do devs in test and QAs typically review coverage numbers prior to construction of integration and manual test plans?
How are devs typically communicating edge cases and relevant test procedures to the testers these days?
Finally are there any great resources (docs, podcasts etc) that discuss these types of things?
In my experience, there isn't a whole lot of such information shared between DEV and QA, apart from the requirements, things like API documentation and maybe usage examples. The reason is simple: Developer tests (unit, integration) are usually whilebox tests, while tests performed by QA are usually blackbox tests (i.e. oblivious of the code being tested). What use is a coverage report for blackbox tests?
What i think in this case is , If we use tools like JIRA, then in each story/ticket on which Dev is working he/she should mention all the related doc, details and major technical specifications like which module/API/project_layer he has worked on to solve this particular requirement with atleast few of his unit test links to this ticket will certainly help while testing.
And this thing also helps in accounting of code changes done in the project for the tickets so that we can also guess the major regression areas.
Let me know if u find this helpful.
Thanks
I am looking for a way/solution to implement TDD in Siebel development.
I come from web development world and i used a lot unit testing, integration platform and other XP things.
Today i am working on Siebel CRM development for a new job in business in and i am shocked to see that there is no test automation solution in place.
I am currently searching for some documentations, feedbacks, or all other interesting things regarding TDD in Siebel development environemnt.
I could develop some tools myself but i don't want to reinvent the wheel.
If someone have more informations, interesting articles, interesting way to do this it could be very usefull for me.
Thanks
This is really several questions in one. TDD is a way of working, but I think you are already wel versed in what this entails. It can be applied to any kind of technology. From that stance I feel comfortable saying that it's possible with Siebel as well.
As for the tooling to be used, it very much depends on the version of Siebel you are on (HI/Open UI), the skills of your developers, the way of working at your project (Agile/Waterfall), the test (automation) tooling used and the general level of automation at the project/program.
To me it sounds more like a way-of-working transformation then the implementation of a test approach.
By backend I mean software systems that consume data, process files or communicate using machine interfaces (REST, SOAP, CORBA, etc...). No fancy web or UI testing is necessary. I have in mind Cucumber and Robotframework but I don't know how well suited they are in the task at hand.
There isn't an easy answer to this question.
Sounds like you got your domain right... because if you were trying to test UI/Web components acceptance testing frameworks make not be the right tool for the job.
You have a few options
SpecFlow
Cucumber
StoryTeller
FitNesse
mSpec
a few others.
I'm partial to StoryTeller & mSpec.. but each of these tools have their pros/cons.
Questions you should be asking yourself (and your team are)
Who is going to be writing/maintaining the tests?
Are self documenting acceptance tests provide value for your organization?
Which technology would integrate most easily with your current build process?
I have used Cucumber to test a batch application written in perl and plsql, an informatica transformation, and am currently using it to test a telephony ivr/queueing system. Ruby provided the gems I needed to drive the system and it was very easy for the testers to learn the language/syntax.
If it comes to Robot Framework I have an experience with bad support in Intellij for this. I'm using IntelliBot plugin. In many cases you will know your mistake like wrong syntax, cannot find keyword etc. late in test runtime.
Problems with finding imported libraries, no debug, no simple refactor like rename keyword across a project.
I m just starting using gwt and so far so good, however after reading some sample code I wonder is it necesary to have a high level of test coverage? (I can see that most code is declarative and then add some attributes I can see the sense in checking so me particular attributes are there but not all)
Also i would be interested to know anything about what are the gotchas in TDDing with GWT
I m using eclipse so also if you are really happy with some particualrs add ins for GWT I would be happy to hear about that
Thanks for the input
edit: maybe I m asking a very wide question, but even little pieces of information will help
I come from having nvelocity views with jquery/extJs/prototype/scriptaculous and this is a bit different
When designing GWT applications to be easily testable, it's best to move as much logic out of the view as possible. Use a design pattern which makes GUI testing easier such as Model-View-Presenter (MVP), which is used widely in building desktop applications (The C#/.NET folks have written a lot about this pattern.)
You can use GWTTestCases to test remote communication and code that ultimately executes raw JavaScript (most of the GWT core classes require this, especially widgets). However, these tests are slow to execute, so you should prefer designs which put all that logic in objects that can be tested in plain ol' JUnit TestCases.
For more information about writing GWT applications test-first, I've written an article for Better Software magazine, which is available as a PDF online at my blog.
I think the best reference at the moment would be this Testing Methodologies Using Google Web Toolkit
I think you asked a pretty broad question, which is part of the reason why you didn't get a reply for a while.
Compared to traditional AJAX web development, one could argue a GWT application requires less testing. Because the GWT team has worked so hard to make sure that its widgets work consistently across all web browsers, you don't have to worry about cross-browser compatibility nearly as much for your own application.
That frees you up to focus on your own application. Create a separate test case for each of your own custom widgets and test that they behave as you expect, and then write higher-level tests for each module. Take the extra step to make your tests fully automatable - that way every time you make a change or are about to release, it's easy to run all of your tests.
http://code.google.com/docreader/#p=google-web-toolkit-doc-1-5&s=google-web-toolkit-doc-1-5&t=DevGuideJUnitIntegration
Unit testing and ASP.NET web applications are an ambiguous point in my group. More often than not, good testing practices fall through the cracks and web applications end up going live for several years with no tests.
The cause of this pain point generally revolves around the hassle of writing UI automation mid-development.
How do you or your organization integrate best TDD practices with web application development?
Unit testing will be achievable if you separate your layers appropriately. As Rob Cooper implied, don't put any logic in your WebForm other than logic to manage your presentation. All other stuff logic and persistence layers should be kept in separate classes and then you can test those individually.
To test the GUI some people like selenium. Others complain that is a pain to set up.
I layer out the application and at least unit test from the presenter/controller (whichever is your preference, mvc/mvp) to the data layer. That way I have good test coverage over most of the code that is written.
I have looked at FitNesse, Watin and Selenium as options to automate the UI testing but I haven't got around to using these on any projects yet, so we stick with human testing. FitNesse was the one I was leaning toward but I couldn't introduce this as well as introducing TDD (does that make me bad? I hope not!).
This is a good question, one that I will be subscribing too :)
I am still relatively new to web dev, and I too am looking at a lot of code that is largely untested.
For me, I keep the UI as light as possible (normally only a few lines of code) and test the crap out of everything else. At least I can then have some confidence that everything that makes it to the UI is as correct as it can be.
Is it perfect? Perhaps not, but at least it as still quite highly automated and the core code (where most of the "magic" happens) still has pretty good coverage..
I would generally avoid testing that involves relying on UI elements. I favor integration testing, which tests everything from your database layer up to the view layer (but not the actual layout).
Try to start a test suite before writing a line of actual code in a new project, since it's harder to write tests later.
Choose carefully what you test - don't mindlessly write tests for everything. Sometimes it's a boring task, so don't make it harder. If you write too many tests, you risk abandoning that task under the weight of time-consuming maintenance.
Try to bundle as much functionality as possible into a single test. That way, if something goes wrong, the errors will propagate anyway. For example, if you have a digest-generating class - test the actual output, not every single helper function.
Don't trust yourself. Assume that you will always make mistakes, and so you write tests to make your life easier, not harder.
If you are not feeling good about writing tests, you are probably doing it wrong ;)
A common practice is to move all the code you can out of the codebehind and into an object you can test in isolation. Such code will usually follow the MVP or MVC design patterns. If you search on "Rhino Igloo" you will probably find the link to its Subversion repository. That code is worth a study, as it demonstrate one of the best MVP implementations on Web Forms that I have seen.
Your codebehind will, when following this pattern, do two things:
Transit all user actions to the presenter.
Render data provided by the presenter.
Unit testing the presenter should be trivial.
Update: Rhino Igloo can be found here: https://svn.sourceforge.net/svnroot/rhino-tools/trunk/rhino-igloo/
There have been tries on getting Microsoft's free UI Automation (included in .NET Framework 3.0) to work with web applications (ASP.NET). A german company called Artiso happens to have written a blog entry that explains how to achieve that (link).
However, their blogpost also links an MSDN Webcasts that explains the UI Automation Framework with winforms and after I had a look at this, I noticed you need the AutomationId to get a reference to the respecting controls. However, in web applications, the controls do not have an AutomationId.
I asked Thomas Schissler (Artiso) about this and he explained that this was a major drawback on InternetExplorer. He referenced an older technology of Microsoft (MSAA) and was hoping himself that IE8 will do this better.
However, I was also giving Watin a try and it seems to work pretty well. I even liked Wax, which allows to implement simple testcases via Microsoft Excel worksheets.
Ivonna can unit test your views. I'd still recommend moving most of the code to other parts. However, some code just belongs there, like references to controls or control event handlers.