By backend I mean software systems that consume data, process files or communicate using machine interfaces (REST, SOAP, CORBA, etc...). No fancy web or UI testing is necessary. I have in mind Cucumber and Robotframework but I don't know how well suited they are in the task at hand.
There isn't an easy answer to this question.
Sounds like you got your domain right... because if you were trying to test UI/Web components acceptance testing frameworks make not be the right tool for the job.
You have a few options
SpecFlow
Cucumber
StoryTeller
FitNesse
mSpec
a few others.
I'm partial to StoryTeller & mSpec.. but each of these tools have their pros/cons.
Questions you should be asking yourself (and your team are)
Who is going to be writing/maintaining the tests?
Are self documenting acceptance tests provide value for your organization?
Which technology would integrate most easily with your current build process?
I have used Cucumber to test a batch application written in perl and plsql, an informatica transformation, and am currently using it to test a telephony ivr/queueing system. Ruby provided the gems I needed to drive the system and it was very easy for the testers to learn the language/syntax.
If it comes to Robot Framework I have an experience with bad support in Intellij for this. I'm using IntelliBot plugin. In many cases you will know your mistake like wrong syntax, cannot find keyword etc. late in test runtime.
Problems with finding imported libraries, no debug, no simple refactor like rename keyword across a project.
Related
Do we have any unit testing frameworks in Vapor?
I am looking dependencies/framework which will help to do the Unit testing some of my web services and methods in easyway.
I've tried following solutions.
http://seatoncode.com/unit-testing-vapor/
https://medium.com/#tullio.sebastiani/unit-testing-vapor-toolbox-v-1-0-1-projects-c59a056c3b9b#.qf0kl1pfo
Regarding testing frameworks, I'm quite happy with Apple's XCTest, but if you're looking for something different, I recommend Quick.
If you also want to test the front-end, there are several solutions out there, and choosing one, in my opinion, depends on what front-end framework you pick, if any.
Anyway, I see you found articles on how to configure Vapor for testing. In this post I do something similar, but also I made a template to create new apps already set for testing. You can find it here.
We are working on billing system (java based module)for that we would like to have a testing framework. That testing framework should be able to adoptable for any type of billing scenarios(eg: utility bill payments, water/electricity/or any other type billing) Normally the billing entity have common attributes like customer name/usage/ etc..I would like to pick a suitable testing platform to test our billing module.
It can be opensource/licensed software.
Can anybody suggest such a framework/engine?
If you wanna go for open source tool, then I'd recommend Selenium Webdriver with TestNG Framework. You can get lot of documentation and help on web.
You can go for Cucumber for describing various test scenarios (and their dependencies).
These scenarios will be backed by jUnit for gluing the description to executable code. jUnit will typically be used to write low level tests as well (for use by developers).
Cucumber has the benefit of giving you reports and can serve well for discussion with users and their representatives.
I would suggest following based on your given little description about your need :
1 - For functionality testing and to make all your scenarios automate use Selenium WebDriver
2 -Then if you want to priorities your testes , want to run tests through XML , want to run multiple tests then you can use TestNG Framework
Above are totally open source tool and you will get real benefit of those by scripting. They allow you to do scripting using programming languages like Java , Python , ruby and a little more. You will get all details once you visit my given links in above 2 points.
According to your given requirements I think above 2 tools are enough to make everything automate for testing.
I am beginning a fairly large new project using ColdFusion. This new project will include several developers and as such documentation of code will be key.
Another issue I am hoping to avoid (either with adequate code documentation or some other tool) is the duplication of code. A tool that would be able to "index" the code for searching or diagramming would likely help here.
What are others out there using either specifically for ColdFusion or language agnostic. We will likely be using ColdBox for the underlying framework if that makes a difference.
Thanks for any any all suggestions.
-c
Well, it's impossible to tell you which framework to use without knowing more about your project, but I can list out some tools that will be useful no matter which framework you use.
Language-agnostic tools:
GitHub.com organization+teams
Jenkins continuous integration
Apache ANT build scripts
Apache Maven for project management
Coldfusion-specific tools:
MXunit unit testing framework
MockBox for unit testing (if you use ColdBox: ColdBox-specific tutorial)
ColdDoc documentation generator
Javascript-specific tools:
JSLint or JSHint for JS code cleanup
Jasmine unit testing
Ideally, your Jenkins build server should:
Do a fresh checkout from source control
Run all unit tests and stop the build if they fail
Generate documentation
Generate a production-ready package of your project
At a minimum, I highly recommend using source control, setting up Jenkins with MXunit tests, and scheduling daily automated builds.
We used the Atlassian suite at my last job. Mostly Jira for tracking and Greenhopper for agile but the other tools may help, fisheye, bamboo, and crucible. If you host it yourself I believe they have a one time $10/product price tag that. Depending on your team's size may or may not work. If money is no subject the suite worked really nicely. It also has built in support for svn and maybe more by now.
http://www.atlassian.com/software
Sounds to me what you need is a methodology, not a tool. If you have a clearly defined set of objects/responsibilities. There should be no crossover in scripting, and if you determine a common API for the objects being coded, I would think you'd be fine.
I need to write a unit test suite for a project I am developing in my spare time. Being a CL newbie I was overwhelmed by the amount of choices for a CL implementation, I spent quite some time to choose one. Now I am facing exactly the same thing with unit test frameworks.
A quick glance at http://www.cliki.net/test%20framework shows 20 unit test frameworks! Choice is good but for a novice like me this can be a bit confusing and given the number of frameworks it would be painful to try them all.
I would like to use a framework which:
Is reasonably well maintained
Easy to use but with some degree of flexibility
Offers some sort of integration with Emacs (or it is possible to easily integrate it with Emacs)
It is possible to integrate it with git post-commit hooks
It is possible to integrate it with a continous integration system (such as buildbot)
What are your experiences in this field?
Did you see the link to http://aperiodic.net/phil/archives/Geekery/notes-on-lisp-testing-frameworks.html off the Test framework comparison link on that cliki page you mention? Phil gives his impressions, and what it looks like to use the various test frameworks.
I personally prefer lisp-unit. It's simple to use and has most of the common types of tests.
http://www.cliki.net/lisp-unit
http://repo.or.cz/w/lisp-unit.git/blob_plain/master:/documentation/lisp-unit.html
I don't think it has any integration with post-commit hooks or buildbot built in.
As part of a new job, I have to devise and implement a complete test strategy for the company's new product. So far, all I really know about it is that it is written in C++, uses an SQL database and has a web API which is used by a browser client written using GWT.
As far as I know, there isn't much of an existing strategy, except for using Python scripts to test the web API.
I need to develop and implement a suitable strategy for unit, system, regression and release testing, preferably a fully automated one.
I'm looking for good references for:
Devising the complete test strategy.
Testing the web API.
Testing the GWT based application.
Unit testing C++ code.
In addition, any suitable tools would be appreciated.
Testing Computer Software is a great soup-to-nuts book on the entire testing process. In addition to the items you mentioned, you'll need to think about other types of testing (performance, security, localization, stress testing, to name a few) and how to manage the test process; test plans, issue tracking, test data, test cases, in addition to the tools.
There's a lot there, and you can't do everything at once. I think a phased approach would be best, where you identify the gaps, weaknesses, and risks in the current process, prioritize them, then set up a high level plan to address them one by one.
Software QA Testing and Test Tool Resources is a good starting place for finding some tools to fit your process. StickyMinds is a nice web site dedicated to software testing, and the folks here at StackOverflow certainly know their stuff, so don't be afraid to ask.
Good luck :)
You can find a ton of excellent information on testing and developing a test strategy in general over on James Bach's blog. Specifically by searching through it for tips on testing strategies.
James is an excellent resource for information about how to do great software testing.
Best of luck.
There's a good conversation here on The Purpose of a Test Strategy.
As testing tools you could use Selenium for web testing and CppUnit for c++ unit testing.