I am still trying to totally understand BDD and I am facing some doubts.
From my little experience, I have been using it to automate user acceptance test and I would like to know if it's possible to use it to test a web API, without UI.
In the past I've used BDD using the given-when-then jargon and mapping the steps to UI interactions. I've done this with Specflow in ASP.NET or cucumber/capybara in ruby on rails.
So for example we could have scenarios like this:
Given I am in the home page
When I click login button
Then I should see the login page
The current project I am working at is different. We are implementing an API based in web service which would be consumed by different type of clients. Like an iphone app, android app and an asp based web client. So our main focus is based in the back-end and just that.
In this case, the tests can't be faced from the UI point of view. So our end-to-end tests are based in our service endpoints. We pass some input arguments to a service calls and check the outputs.
Can we do this using BDD? Is this right?
or maybe it would be better to use a different thing like FitNesse?
Hmm.. is using FitNesse doing BDD?
I think you can do what your writing about in BDD. I'm not sure if those 2 links about testing of webservices with SpecFlow will help you but take a look on them if you haven't seen it yet.
http://codedetective.blogspot.co.uk/2012/10/testing-webservices-with-specflow.html
http://www.creamdog.se/blog/2011/02/24/webservices-automated-tests-using-specflow-and-babelfish/
Have a look at Karate, web service testing framework by Intuit. It's recently being open sourced. It has the capability of handling API dealing with HTML, JSON, XML, GraphQL queries and is built on top on cucumber.
Simple intro here : https://medium.com/blueprint-by-intuit/karate-web-services-testing-made-simple-366e8eb5adc0#.qnpy5gagt
Related
I know this question was asked million times but I'm gonna ask it again. I've been researching how to test web services for a while and I can't believe that there's pretty much one tool out there: SOAP UI. The other option that you have is creating web services stubs and writing unit tests per each ws stub.
I can't believe that Microsoft didn't come up with a solid solution for web services testing. So I guess my question: is SOAP UI the only tool for testing multiple web services?
P.S.
I'm also looking for something to integrate with TFS (I know SOAP UI can be integrated with msbuild)
Thanks
There are always options! Here an old article that could help out includes soapUI, Push to test and WebInject.
Looking to shell out dough? You could always use HP Service Test
Which do I use? soapUI of course :) Frankly, it's the most commonly used for a reason.
My alternative to soapUI is HP Service Test.
If I have to build a testing team with testers that are not exposed to web services and SOA in general, then the learning curve and team ramp time associated with HP Service Test is short essentially because of the Canvas based visual drag and drop designer that Service Test boasts.
Perhaps, everything you could do with HP Service Test 11.20 is possible with SOAP UI 4. They both do have striking inherent features that is kind of mutually exclusive between these two.
If your organization already uses HP's Quality Center, QTP and Load Runner, then for easy effortless mangement of all test cases in your app's boundary it is preferred to have HP Service Test because it integrates with all the aforementioned products.
With Service Test, I personally like the Canvas designer in HP Service Test and the ability to call .Net assemblies inside of Service Test is just a boon as I am a .Net developer. And the ToolBox has a list of controls that every tester/dveloper needs. And it is just efforless to built a hi-fi test case with Service Test. It is worth the money.
With soapUI, I like the HTTP Recording (works much like Fiddler) and the Security check options (to name a few, XSLT injection, XSS checks).
soapUI is omnipresent.
TFS integration, I am not there yet.
I've used Selenium to do lots of UI testing from the browser. If you have a web service behind the Java jsp page i.e. in a servlet, you can test it from Selenium.
Can Selenium be used to test a B2B web service i.e. a web service called from a backend that has no browser UI component?
I have used SOAPUI to do this kind of testing in the past but our test department is trying to standardise on Selenium.
You can but I would not recommend it. If the page is returning XML, you won't be able to use the standard Selenium calls to verify what is happening as you won't have access to the DOM. If its returning plain text for JavaScript then you will struggle with verifying the output.
This is a definite case of using the right tool for the job and Selenium is not the right tool for testing web services. I would use soapUI or just use some http library to call the service URL and then verify the results.
If they are looking to standardise they need to standardise tools for their purpose. Selenium for UI, soapUI for webservices,XUnit Framework for unit and integration.
You can, but it's really not the right tool for the job. It's like trying to hammer a nail into a piece of wood using a stapler instead of a hammer.
That said, probably the most appropriate way to create a page with all your input parameters which could do the call for you and echo the results back into a html element. If the service is meant for AJAX calls then this is probably the ideal solution for your service.
The correct approach would be to use a unit testing framework and create a test harness which you can push your parameters into, execute the service call and retrieve the results in a meaningful way for assertion.
I'm looking for a web service testing framework that has a good built in text input / output gui. The point would be for business and other non developer users to be able to test requirements by using a simple gui that they could enter values into and see the results from the web service. I have seen lots of references on Stack Overflow and the web to great developer web service testing tool but a nice, built in, I/O gui is the key feature for me.
By web service testing I mean a tool that sends xml post requests over HTTP and tests the response.
I have been trying out fitnesse but to use it I need to cut out parts of the system and I very much want to do end to end testing. The way fitnesse displays its results is great but the wiki data entry is a bit clunky for business users imo.
We use FitNesse for the most part. The developers maintain a fixture that s the connector between your app and the FitNesse server. The business user can write the test cases in an wiki style. Works fine for us.
Rational Functional Tester was in the discussion too. However, I have never seen it, but would have met our requirements too.
You might want to look at the robot framework. It's a pretty decent testing library which makes it easy to test web services. There is also a library that integrates Selenium into the framework for front-end testing. Tests can be specified in a couple of plain text formats as well as in HTML tables. You also have the option to let users create test data in spreadsheets if you like, then save them in a tab-separated format.
Look at WebServiceStudio http://www.codeplex.com/WebserviceStudio
We use Concordion for business testing. It's very similar to Fitness but the input files are HTML files "executed" by JUnit.
If your web services don't take 500 parameters, a HTML table is a great way to define input values. I agree, there no UI (although a WYSIWYG HTML editor would do it) but the benefit is that your tests can be checked into Subversion (for example), executed regularly (read every night) and the result HTML files be published internally.
I found testing composite application in Netbeans very easy to use ...
As you requested in your question you can define an input XML and a desired output XML, and when you run the test you have a classic JUnit result like this :
alt text http://netbeans.org/images_www/articles/61/soa/ep-understand-trs/testpassed.png
Here you can find a good example:
Understanding the Travel Reservation Service
I use NetBeans 6.5.1 bundled wit GlassFish ESB 2.1 ... I'm not sure that this feature is supported in the newest NetBeans Versions
I am a novice in web services. I am totally new to testing web services.
A new project demands that I test the web services, and the customer is in favor of any open source tool.
What is the approach to testing web services?
Also Please suggest a tool(with minimal scripting) to test web services?
Check out SoapUI - one of the best web service test tools - plus it's free!!
They also have a "Pro" version which costs - you can do more stuff, like load testing etc., but the free version is quite good enough for most of your testing, I'd say!
Given a WSDL (online or stored as file), it'll create stubs for each method, which you can then use to create requests (as XML), fill in the blanks (the parameter values), and then you can send off your request to the web service and see what comes back as a response.
SoapUI also allows you to write scripted tests than can be run over and over again.
Excellent tool - can't praise it enough!
Marc
Additionally you could use Firefox Poster in order to test your web service by passing XML-packets manually.
Check it here:
FF Poster
SoapUI is a great tool to test SOAP webservices. It allows you to test a SOAP client or a SOAP server.
Another very useful tool is Fiddler. Fiddler isn't necessarily aimed at testing webservices (it's a HTTP debugger), but since SOAP webservices run over HTTP, you can use it to testing. Another very important advantage of using Fiddler is the fact that you can test REST webservices also.
You might want to consider robot framework. It is a generic, keyword-driven testing framework. There are libraries for testing REST and SOAP based web services. It can also be used to test web pages (via a selenium library), databases, and a whole lot more.
robotframework has a ton of built-in keywords, and there are additional libraries that do much more. You are also able to develop your own keywords in python, java, .NET languages, or any other language.
We have a monolithic application written in Visual Dataflex, and various complementing applications written in other (.NET) languages. They all share the same database, and need to follow the same business logic. One way to facilitate unified business logic across these is to provide web services as an interface for testing.
Of course, for this to work, we need a good framework for testing web services. Any suggestions? For example, can Cucumber do this "out of the box"?
I'm assuming you're talking about Soap web services. You can use Soap4R to talk to a Soap web service. Wrapping this all up in Cucumber scenarios should work fine.
SoapUI is a pretty nice product for creating webservice tests, and they're easily callable via junit or just via the command line. It's also got some stuff for creating load tests as well.
They've got an opensource version as well as a professional version.