I'm looking for a web service testing framework that has a good built in text input / output gui. The point would be for business and other non developer users to be able to test requirements by using a simple gui that they could enter values into and see the results from the web service. I have seen lots of references on Stack Overflow and the web to great developer web service testing tool but a nice, built in, I/O gui is the key feature for me.
By web service testing I mean a tool that sends xml post requests over HTTP and tests the response.
I have been trying out fitnesse but to use it I need to cut out parts of the system and I very much want to do end to end testing. The way fitnesse displays its results is great but the wiki data entry is a bit clunky for business users imo.
We use FitNesse for the most part. The developers maintain a fixture that s the connector between your app and the FitNesse server. The business user can write the test cases in an wiki style. Works fine for us.
Rational Functional Tester was in the discussion too. However, I have never seen it, but would have met our requirements too.
You might want to look at the robot framework. It's a pretty decent testing library which makes it easy to test web services. There is also a library that integrates Selenium into the framework for front-end testing. Tests can be specified in a couple of plain text formats as well as in HTML tables. You also have the option to let users create test data in spreadsheets if you like, then save them in a tab-separated format.
Look at WebServiceStudio http://www.codeplex.com/WebserviceStudio
We use Concordion for business testing. It's very similar to Fitness but the input files are HTML files "executed" by JUnit.
If your web services don't take 500 parameters, a HTML table is a great way to define input values. I agree, there no UI (although a WYSIWYG HTML editor would do it) but the benefit is that your tests can be checked into Subversion (for example), executed regularly (read every night) and the result HTML files be published internally.
I found testing composite application in Netbeans very easy to use ...
As you requested in your question you can define an input XML and a desired output XML, and when you run the test you have a classic JUnit result like this :
alt text http://netbeans.org/images_www/articles/61/soa/ep-understand-trs/testpassed.png
Here you can find a good example:
Understanding the Travel Reservation Service
I use NetBeans 6.5.1 bundled wit GlassFish ESB 2.1 ... I'm not sure that this feature is supported in the newest NetBeans Versions
Related
I'm a QA and know a little about coding, the developers will convert our SOAP webservice to REST and I want to write tests for this. Basically I want to read our SOAP XML, understand the request/parameters and create a REST Test Case in SOAPUI to be run when they finish the implementation.
Is it possible?
Sorry for my poor english, it's not my native language.
Technically, it's possible.
However, if the developers will be aiming for REST level 3, testing this through hard-coded URLs inside SOAPUP kinda defeats the purpose - ideally you'd be able to navigate between resources through discoverable links.
HAL Browser is one useful tool for navigating such RESTful APIs, however you need a dedicated tool for testing.
Here's an very detailed possible answer for your question.
I am still trying to totally understand BDD and I am facing some doubts.
From my little experience, I have been using it to automate user acceptance test and I would like to know if it's possible to use it to test a web API, without UI.
In the past I've used BDD using the given-when-then jargon and mapping the steps to UI interactions. I've done this with Specflow in ASP.NET or cucumber/capybara in ruby on rails.
So for example we could have scenarios like this:
Given I am in the home page
When I click login button
Then I should see the login page
The current project I am working at is different. We are implementing an API based in web service which would be consumed by different type of clients. Like an iphone app, android app and an asp based web client. So our main focus is based in the back-end and just that.
In this case, the tests can't be faced from the UI point of view. So our end-to-end tests are based in our service endpoints. We pass some input arguments to a service calls and check the outputs.
Can we do this using BDD? Is this right?
or maybe it would be better to use a different thing like FitNesse?
Hmm.. is using FitNesse doing BDD?
I think you can do what your writing about in BDD. I'm not sure if those 2 links about testing of webservices with SpecFlow will help you but take a look on them if you haven't seen it yet.
http://codedetective.blogspot.co.uk/2012/10/testing-webservices-with-specflow.html
http://www.creamdog.se/blog/2011/02/24/webservices-automated-tests-using-specflow-and-babelfish/
Have a look at Karate, web service testing framework by Intuit. It's recently being open sourced. It has the capability of handling API dealing with HTML, JSON, XML, GraphQL queries and is built on top on cucumber.
Simple intro here : https://medium.com/blueprint-by-intuit/karate-web-services-testing-made-simple-366e8eb5adc0#.qnpy5gagt
I know this question was asked million times but I'm gonna ask it again. I've been researching how to test web services for a while and I can't believe that there's pretty much one tool out there: SOAP UI. The other option that you have is creating web services stubs and writing unit tests per each ws stub.
I can't believe that Microsoft didn't come up with a solid solution for web services testing. So I guess my question: is SOAP UI the only tool for testing multiple web services?
P.S.
I'm also looking for something to integrate with TFS (I know SOAP UI can be integrated with msbuild)
Thanks
There are always options! Here an old article that could help out includes soapUI, Push to test and WebInject.
Looking to shell out dough? You could always use HP Service Test
Which do I use? soapUI of course :) Frankly, it's the most commonly used for a reason.
My alternative to soapUI is HP Service Test.
If I have to build a testing team with testers that are not exposed to web services and SOA in general, then the learning curve and team ramp time associated with HP Service Test is short essentially because of the Canvas based visual drag and drop designer that Service Test boasts.
Perhaps, everything you could do with HP Service Test 11.20 is possible with SOAP UI 4. They both do have striking inherent features that is kind of mutually exclusive between these two.
If your organization already uses HP's Quality Center, QTP and Load Runner, then for easy effortless mangement of all test cases in your app's boundary it is preferred to have HP Service Test because it integrates with all the aforementioned products.
With Service Test, I personally like the Canvas designer in HP Service Test and the ability to call .Net assemblies inside of Service Test is just a boon as I am a .Net developer. And the ToolBox has a list of controls that every tester/dveloper needs. And it is just efforless to built a hi-fi test case with Service Test. It is worth the money.
With soapUI, I like the HTTP Recording (works much like Fiddler) and the Security check options (to name a few, XSLT injection, XSS checks).
soapUI is omnipresent.
TFS integration, I am not there yet.
I'm looking for an alternative to the awesome .NET (WCF) REST capabilities.
Why?
I have deep interest in open source technology, but when it comes to webservices I do not have any experience except with .NET webservices.
Besides, I'm currently using a lot of Java and Python, and I am moving away from the Microsoft technology stack.
Please suggest alternatives in any programming language, but explain why it's good or better for some reasons. (this reason may be be tightly related with the choice of language)
What do I want to know?
Ease of use
Installation
Configuration
Generation capabilities
IDE integration
Deployment
Learning curve
Pros and cons
etc.
Spring 3.0 REST:
Spring uses annotation based controllers, which can be used to bind a url to a method in the controller. Annotations are used to differentiate between GET methods and POST methods.
#RequestMapping(value="/hotels/{hotel}/bookings/{booking}",
method=RequestMethod.GET)
public String getBooking(#PathVariable("hotel") long hotelId,
#PathVariable("booking") long bookingId, Model model) {
Hotel hotel = hotelService.getHotel(hotelId);
Booking booking = hotel.getBooking(bookingId);
model.addAttribute("booking", booking);
return "booking";
}
Under the hood, the variable "hotel" in the URI string is converted to a long in the parameter list, as is booking. Spring REST can also marshal JSON objects into custom classes using this same technique. Note that this method is annotated as RequestMethod.GET, which means it's invoked for GET requests but not POST requests.
Spring 3.0 REST makes it easier to create RESTful Web services by eliminating the need to reinvent the wheel or marshal/unmarshal JSON text by hand from/to Java objects.
There is a demo here on the SpringSource Blog titled REST In Spring MVC. The learning curve is low, but getting the demo to work may take some time thanks to dependencies. Once you get setup and have a working demo, the hardest part should be over.
For IDE integration, check out Spring Roo. I've not used it, but I've heard it has some features that integrate with Eclipse IDE to make your life easier.
Restlets:
Restlets were designed solely for REST. As a result, the overhead is a lot lower than Spring 3.0. Restlets are better suited for cases where you don't have a GUI, and where you aren't concerned with MVC. Restlets can easily serve as both a server and a client. It also has an embedded server you can run, which eliminates the need for a container like Jetty or Tomcat.
I've had very little exposure to Python, but from what I've seen of Google App Engine's implementation of the webApp framework, the Router concept feels very similar. Those with a Python background may find the learning curve to be a lot lower:
#Override
public Restlet createInboundRoot() {
Router router = new Router(getContext());
getConnectorService().getClientProtocols().add(Protocol.FILE);
// Serve the files generated by the GWT compilation step.
Directory dir = new Directory(getContext(), LocalReference.createFileReference(new File("war/")));
router.attachDefault(dir);
router.attach("/contacts/123", ContactServerResource.class);
return router;
}
It uses GWT on the client-side; I prefer to take that part out as it reminds me too much of Java Swing. While some people may find that advantageous, my personal preference is to stick with the technologies that feel more like the Web.
Below is a simple example of a REST server using the standalone mode. The server runs on port 8182, and it listens for GET requests. It has a similar annotation-based model as the Spring REST framework, which also helps split up the different HTTP methods and point them at different methods in your classes. This is a very basic "Hello World" REST example:
public class FirstServerResource extends ServerResource {
public static void main(String[] args) throws Exception {
// Create the HTTP server and listen on port 8182
new Server(Protocol.HTTP, 8182, FirstServerResource.class).start();
}
#Get
public String toString() {
return "hello, world";
}
}
Check out the Restlet Web Site for more information and examples of the Restlets framework. Restlets has a slightly less learning curve than Spring because it's targeted to REST; as a result, it doesn't contain all of the extra functionality included with Spring that can sometimes make finding an answer to a problem difficult. Restlets are definitely the way to go if you're looking for something lightweight.
Both of these two frameworks will run in Tomcat, Jetty, as well as on Google App Engine.
If you are using Java and you are familiar with Spring, then you should certainly take a look at Spring MVC 3.x. This version moves away from the ugly XML configuration, and its syntax is very similar to JAX-RS's specs. That said, if you know Spring, then learning Spring MVC 3.0 is going to be minimal. However, if you are having trouble understanding with IoC pattern and what not, then it is going to be a long painful experience. :)
Keep in mind, Spring MVC 3.x is not pure REST, and it will never be in the future at all, based on the Spring MVC developers. Their take was there are already so many good REST implementations and there's no point of making Spring MVC 3.x totally RESTful.
Another option I will certainly recommend to you is Jersey. Jersey is pure REST, in another word, it is an implementation of JAX-RS. Jersey took me 30 minutes to learn. In my opinion, the annotations are so much more powerful and richer than Spring MVC 3.x. The annotations from Spring MVC 3.x seem pretty vanilla to me. Jersey will automatically generate the WADL for you, although it is pretty basic... but having one is better than not having one. You can certainly customize your WADL if you want. (By the way, WADL is REST's version of WSDL, if you don't know what that means). Jersey basically detects your package containing all the Resource classes and generates the WADL based on the configurations you have, pretty neat stuff. The last thing I want to point out is Jersey has a great test framework for you to easily test your Restful web service. In another word, their test framework allows your unit test to easily fire up Grizzly or in-memory server to test your web service. It is certainly one of the best I have ever use thus far. Here's a very easy tutorial for you get your feet wet: http://www.vogella.de/articles/REST/article.html . It is really THAT easy. :)
FYI, I have used both Spring MVC 3.x and Jersey.
ServiceStack is one of the more recent developments. I haven't done much with it yet, but it seems pretty sweet so far.
Ruby and Rails (Ruby on Rails) have great support for RESTful service. In fact Rails supports and encourages design and develop in RESTful manner.
Thanks to ruby's strong DSL feature, writing REST service is very straightforward and easy. Since you have python experience, learning ruby might be easy.
Refer to this guide to have an impression how rest urls (called routes in rails) are defined.
Other Ruby web frameworks such as Sinatra also do a good job on this.
BTW, the best things is that both ruby and rails are open source, and the ruby community is awesome and very active.
There's RESTSharp as a REST/HTTP client (open-source project) and OpenRasta
I welcome you to check out servicestack.net it is designed for simplicity and speed and introduces very low artificial concepts where it is able to maintain a very DRY and succinct API and automatically works out of the box without any configuration or code-gen.
It encourages best practices as it is modelled around Martin Fowlers Gateway and DTO pattern for developing remote services.
The equivalent code for the Spring.NET example above would be
Configuration (in AppHost)
Routes.Add<Booking>("/hotels/{HotelId}/bookings/{BookingId}");
C# Code
public class BookingService : RestServiceBase<Booking>
{
public IHotelService hotelService { get; set; } //auto-injected by IOC
public object OnGet(Booking request)
{
var hotel = hotelService.GetHotel(request.HotelId);
var booking = hotel.GetBooking(request.BookingId);
return booking;
}
}
A similar example to the booking service can be seen by the live Northwind Web Services demo.
That's all the configuration and code (exc DTO) you need to write for that service and it is automatically available via JSON, XML, JSV, CSV, SOAP 1.1/1.2 and HTML endpoints and formats automatically without any extra configuration required.
Checkout the Hello World example for more info on all the endpoints and formats provided as well as the auto generated /metadata and documentation pages.
There is an open source framework entirely developed for RESTful web services which is called Recess
It's not very old, but got good attention from the industry. Alcatel-Lucene already arranged a competition on TopCoder for developing some of their services using this framework.
Check out details at Recess web site
I am a novice in web services. I am totally new to testing web services.
A new project demands that I test the web services, and the customer is in favor of any open source tool.
What is the approach to testing web services?
Also Please suggest a tool(with minimal scripting) to test web services?
Check out SoapUI - one of the best web service test tools - plus it's free!!
They also have a "Pro" version which costs - you can do more stuff, like load testing etc., but the free version is quite good enough for most of your testing, I'd say!
Given a WSDL (online or stored as file), it'll create stubs for each method, which you can then use to create requests (as XML), fill in the blanks (the parameter values), and then you can send off your request to the web service and see what comes back as a response.
SoapUI also allows you to write scripted tests than can be run over and over again.
Excellent tool - can't praise it enough!
Marc
Additionally you could use Firefox Poster in order to test your web service by passing XML-packets manually.
Check it here:
FF Poster
SoapUI is a great tool to test SOAP webservices. It allows you to test a SOAP client or a SOAP server.
Another very useful tool is Fiddler. Fiddler isn't necessarily aimed at testing webservices (it's a HTTP debugger), but since SOAP webservices run over HTTP, you can use it to testing. Another very important advantage of using Fiddler is the fact that you can test REST webservices also.
You might want to consider robot framework. It is a generic, keyword-driven testing framework. There are libraries for testing REST and SOAP based web services. It can also be used to test web pages (via a selenium library), databases, and a whole lot more.
robotframework has a ton of built-in keywords, and there are additional libraries that do much more. You are also able to develop your own keywords in python, java, .NET languages, or any other language.