I'm writing a REST-full webapp and I'm looking for a good testing framework. The app is mostly writen in Erlang, but I would like to test the front-end of the app without relying on the erlang code.
Also integration with a js testframework would be desireable.
Good day, if you have minimal ruby knowledge, then i would recommend you use capybara+rails solution, it also allows testing javascript-rich sites, just use javascript-enabled test driver like selenium, here is short doc about using it with remote sites: http://rubydoc.info/github/jnicklas/capybara/master#Calling_remote_servers
Also check out cucumber http://cukes.info/ which plays well with rails+capybara combo and also has erlang port
Related
I am still trying to totally understand BDD and I am facing some doubts.
From my little experience, I have been using it to automate user acceptance test and I would like to know if it's possible to use it to test a web API, without UI.
In the past I've used BDD using the given-when-then jargon and mapping the steps to UI interactions. I've done this with Specflow in ASP.NET or cucumber/capybara in ruby on rails.
So for example we could have scenarios like this:
Given I am in the home page
When I click login button
Then I should see the login page
The current project I am working at is different. We are implementing an API based in web service which would be consumed by different type of clients. Like an iphone app, android app and an asp based web client. So our main focus is based in the back-end and just that.
In this case, the tests can't be faced from the UI point of view. So our end-to-end tests are based in our service endpoints. We pass some input arguments to a service calls and check the outputs.
Can we do this using BDD? Is this right?
or maybe it would be better to use a different thing like FitNesse?
Hmm.. is using FitNesse doing BDD?
I think you can do what your writing about in BDD. I'm not sure if those 2 links about testing of webservices with SpecFlow will help you but take a look on them if you haven't seen it yet.
http://codedetective.blogspot.co.uk/2012/10/testing-webservices-with-specflow.html
http://www.creamdog.se/blog/2011/02/24/webservices-automated-tests-using-specflow-and-babelfish/
Have a look at Karate, web service testing framework by Intuit. It's recently being open sourced. It has the capability of handling API dealing with HTML, JSON, XML, GraphQL queries and is built on top on cucumber.
Simple intro here : https://medium.com/blueprint-by-intuit/karate-web-services-testing-made-simple-366e8eb5adc0#.qnpy5gagt
I'm looking for an alternative to the awesome .NET (WCF) REST capabilities.
Why?
I have deep interest in open source technology, but when it comes to webservices I do not have any experience except with .NET webservices.
Besides, I'm currently using a lot of Java and Python, and I am moving away from the Microsoft technology stack.
Please suggest alternatives in any programming language, but explain why it's good or better for some reasons. (this reason may be be tightly related with the choice of language)
What do I want to know?
Ease of use
Installation
Configuration
Generation capabilities
IDE integration
Deployment
Learning curve
Pros and cons
etc.
Spring 3.0 REST:
Spring uses annotation based controllers, which can be used to bind a url to a method in the controller. Annotations are used to differentiate between GET methods and POST methods.
#RequestMapping(value="/hotels/{hotel}/bookings/{booking}",
method=RequestMethod.GET)
public String getBooking(#PathVariable("hotel") long hotelId,
#PathVariable("booking") long bookingId, Model model) {
Hotel hotel = hotelService.getHotel(hotelId);
Booking booking = hotel.getBooking(bookingId);
model.addAttribute("booking", booking);
return "booking";
}
Under the hood, the variable "hotel" in the URI string is converted to a long in the parameter list, as is booking. Spring REST can also marshal JSON objects into custom classes using this same technique. Note that this method is annotated as RequestMethod.GET, which means it's invoked for GET requests but not POST requests.
Spring 3.0 REST makes it easier to create RESTful Web services by eliminating the need to reinvent the wheel or marshal/unmarshal JSON text by hand from/to Java objects.
There is a demo here on the SpringSource Blog titled REST In Spring MVC. The learning curve is low, but getting the demo to work may take some time thanks to dependencies. Once you get setup and have a working demo, the hardest part should be over.
For IDE integration, check out Spring Roo. I've not used it, but I've heard it has some features that integrate with Eclipse IDE to make your life easier.
Restlets:
Restlets were designed solely for REST. As a result, the overhead is a lot lower than Spring 3.0. Restlets are better suited for cases where you don't have a GUI, and where you aren't concerned with MVC. Restlets can easily serve as both a server and a client. It also has an embedded server you can run, which eliminates the need for a container like Jetty or Tomcat.
I've had very little exposure to Python, but from what I've seen of Google App Engine's implementation of the webApp framework, the Router concept feels very similar. Those with a Python background may find the learning curve to be a lot lower:
#Override
public Restlet createInboundRoot() {
Router router = new Router(getContext());
getConnectorService().getClientProtocols().add(Protocol.FILE);
// Serve the files generated by the GWT compilation step.
Directory dir = new Directory(getContext(), LocalReference.createFileReference(new File("war/")));
router.attachDefault(dir);
router.attach("/contacts/123", ContactServerResource.class);
return router;
}
It uses GWT on the client-side; I prefer to take that part out as it reminds me too much of Java Swing. While some people may find that advantageous, my personal preference is to stick with the technologies that feel more like the Web.
Below is a simple example of a REST server using the standalone mode. The server runs on port 8182, and it listens for GET requests. It has a similar annotation-based model as the Spring REST framework, which also helps split up the different HTTP methods and point them at different methods in your classes. This is a very basic "Hello World" REST example:
public class FirstServerResource extends ServerResource {
public static void main(String[] args) throws Exception {
// Create the HTTP server and listen on port 8182
new Server(Protocol.HTTP, 8182, FirstServerResource.class).start();
}
#Get
public String toString() {
return "hello, world";
}
}
Check out the Restlet Web Site for more information and examples of the Restlets framework. Restlets has a slightly less learning curve than Spring because it's targeted to REST; as a result, it doesn't contain all of the extra functionality included with Spring that can sometimes make finding an answer to a problem difficult. Restlets are definitely the way to go if you're looking for something lightweight.
Both of these two frameworks will run in Tomcat, Jetty, as well as on Google App Engine.
If you are using Java and you are familiar with Spring, then you should certainly take a look at Spring MVC 3.x. This version moves away from the ugly XML configuration, and its syntax is very similar to JAX-RS's specs. That said, if you know Spring, then learning Spring MVC 3.0 is going to be minimal. However, if you are having trouble understanding with IoC pattern and what not, then it is going to be a long painful experience. :)
Keep in mind, Spring MVC 3.x is not pure REST, and it will never be in the future at all, based on the Spring MVC developers. Their take was there are already so many good REST implementations and there's no point of making Spring MVC 3.x totally RESTful.
Another option I will certainly recommend to you is Jersey. Jersey is pure REST, in another word, it is an implementation of JAX-RS. Jersey took me 30 minutes to learn. In my opinion, the annotations are so much more powerful and richer than Spring MVC 3.x. The annotations from Spring MVC 3.x seem pretty vanilla to me. Jersey will automatically generate the WADL for you, although it is pretty basic... but having one is better than not having one. You can certainly customize your WADL if you want. (By the way, WADL is REST's version of WSDL, if you don't know what that means). Jersey basically detects your package containing all the Resource classes and generates the WADL based on the configurations you have, pretty neat stuff. The last thing I want to point out is Jersey has a great test framework for you to easily test your Restful web service. In another word, their test framework allows your unit test to easily fire up Grizzly or in-memory server to test your web service. It is certainly one of the best I have ever use thus far. Here's a very easy tutorial for you get your feet wet: http://www.vogella.de/articles/REST/article.html . It is really THAT easy. :)
FYI, I have used both Spring MVC 3.x and Jersey.
ServiceStack is one of the more recent developments. I haven't done much with it yet, but it seems pretty sweet so far.
Ruby and Rails (Ruby on Rails) have great support for RESTful service. In fact Rails supports and encourages design and develop in RESTful manner.
Thanks to ruby's strong DSL feature, writing REST service is very straightforward and easy. Since you have python experience, learning ruby might be easy.
Refer to this guide to have an impression how rest urls (called routes in rails) are defined.
Other Ruby web frameworks such as Sinatra also do a good job on this.
BTW, the best things is that both ruby and rails are open source, and the ruby community is awesome and very active.
There's RESTSharp as a REST/HTTP client (open-source project) and OpenRasta
I welcome you to check out servicestack.net it is designed for simplicity and speed and introduces very low artificial concepts where it is able to maintain a very DRY and succinct API and automatically works out of the box without any configuration or code-gen.
It encourages best practices as it is modelled around Martin Fowlers Gateway and DTO pattern for developing remote services.
The equivalent code for the Spring.NET example above would be
Configuration (in AppHost)
Routes.Add<Booking>("/hotels/{HotelId}/bookings/{BookingId}");
C# Code
public class BookingService : RestServiceBase<Booking>
{
public IHotelService hotelService { get; set; } //auto-injected by IOC
public object OnGet(Booking request)
{
var hotel = hotelService.GetHotel(request.HotelId);
var booking = hotel.GetBooking(request.BookingId);
return booking;
}
}
A similar example to the booking service can be seen by the live Northwind Web Services demo.
That's all the configuration and code (exc DTO) you need to write for that service and it is automatically available via JSON, XML, JSV, CSV, SOAP 1.1/1.2 and HTML endpoints and formats automatically without any extra configuration required.
Checkout the Hello World example for more info on all the endpoints and formats provided as well as the auto generated /metadata and documentation pages.
There is an open source framework entirely developed for RESTful web services which is called Recess
It's not very old, but got good attention from the industry. Alcatel-Lucene already arranged a competition on TopCoder for developing some of their services using this framework.
Check out details at Recess web site
I'm looking for a web service testing framework that has a good built in text input / output gui. The point would be for business and other non developer users to be able to test requirements by using a simple gui that they could enter values into and see the results from the web service. I have seen lots of references on Stack Overflow and the web to great developer web service testing tool but a nice, built in, I/O gui is the key feature for me.
By web service testing I mean a tool that sends xml post requests over HTTP and tests the response.
I have been trying out fitnesse but to use it I need to cut out parts of the system and I very much want to do end to end testing. The way fitnesse displays its results is great but the wiki data entry is a bit clunky for business users imo.
We use FitNesse for the most part. The developers maintain a fixture that s the connector between your app and the FitNesse server. The business user can write the test cases in an wiki style. Works fine for us.
Rational Functional Tester was in the discussion too. However, I have never seen it, but would have met our requirements too.
You might want to look at the robot framework. It's a pretty decent testing library which makes it easy to test web services. There is also a library that integrates Selenium into the framework for front-end testing. Tests can be specified in a couple of plain text formats as well as in HTML tables. You also have the option to let users create test data in spreadsheets if you like, then save them in a tab-separated format.
Look at WebServiceStudio http://www.codeplex.com/WebserviceStudio
We use Concordion for business testing. It's very similar to Fitness but the input files are HTML files "executed" by JUnit.
If your web services don't take 500 parameters, a HTML table is a great way to define input values. I agree, there no UI (although a WYSIWYG HTML editor would do it) but the benefit is that your tests can be checked into Subversion (for example), executed regularly (read every night) and the result HTML files be published internally.
I found testing composite application in Netbeans very easy to use ...
As you requested in your question you can define an input XML and a desired output XML, and when you run the test you have a classic JUnit result like this :
alt text http://netbeans.org/images_www/articles/61/soa/ep-understand-trs/testpassed.png
Here you can find a good example:
Understanding the Travel Reservation Service
I use NetBeans 6.5.1 bundled wit GlassFish ESB 2.1 ... I'm not sure that this feature is supported in the newest NetBeans Versions
I am creating an automated testing framework in Perl for regression tests. I would like to post my results from the test machines. I have used django before where the server ran standalone with no installation needed. Which MVC framework in Perl has its own standalone server? Basically, which of the Perl MVC frameworks is closest to django's server capability and DRY (Don't Repeat Yourself) principles?
P.S. I will not rewrite my framework in python
P.S.S I will not use Perl framework and django as MVC
(no python on servers :( )
Catalyst comes with a built-in test server - check Catalyst::Manual::Tutorial for more details. This looks like a good place to start.
Every popular Perl web framework comes with it's own server: Catalyst, CGI::Application, Jifty. Most popular and DRY-following is Catalyst (don't know about DRY in Jifty).
Also there test result aggregators like Smolder.
Catalyst comes with it's own server script/myapp_server.pl. You can also deploy with our without a reverse proxy using the built in server, but in this case it's advisable to use Catalyst::Engine::HTTP::Prefork: CATALYST_ENGINE='HTTP::Prefork' script/myapp_server.pl
I believe Jifty also has a built in dev server, as does Squatting, but I haven't used either of them.
Maybe Catalyst.
I don't know if it has a stand alone server, but it's Perl with MVC,
We have a monolithic application written in Visual Dataflex, and various complementing applications written in other (.NET) languages. They all share the same database, and need to follow the same business logic. One way to facilitate unified business logic across these is to provide web services as an interface for testing.
Of course, for this to work, we need a good framework for testing web services. Any suggestions? For example, can Cucumber do this "out of the box"?
I'm assuming you're talking about Soap web services. You can use Soap4R to talk to a Soap web service. Wrapping this all up in Cucumber scenarios should work fine.
SoapUI is a pretty nice product for creating webservice tests, and they're easily callable via junit or just via the command line. It's also got some stuff for creating load tests as well.
They've got an opensource version as well as a professional version.