I am writing a C++ API which is to be used as a web service. The functions in the API take in images/path_to_images as input parameters, process them, and give a different set of images/paths_to_images as outputs. I was thinking of implementing a REST interface to enable developers to use this API for their projects (independent of whatever language they'd like to work in). But, I understand REST is good only when you have a collection of data that you want to query or manipulate, which is not exactly the case here.
[The collection I have is of different functions that manipulate the supplied data.]
So, is it better for me to implement an RPC interface for this, or can this be done using REST itself?
Like lcfseth, I would also go for REST. REST is indeed resource-based and, in your case, you might consider that there's no resource to deal with. However, that's not exactly true, the image converter in your system is the resource. You POST images to it and it returns new images. So I'd simply create a URL such as:
POST http://example.com/image-converter
You POST images to it and it returns some array with the path to the new images.
Potentially, you could also have:
GET http://example.com/image-converter
which could tell you about the status of the image conversion (assuming it is a time consuming process).
The advantage of doing it like that is that you are re-using HTTP verbs that developers are familiar with, the interface is almost self-documenting (though of course you still need to document the format accepted and returned by the POST call). With RPC, you would have to define new verbs and document them.
REST use common operation GET,POST,DELETE,HEAD,PUT. As you can imagine, this is very data oriented. However there is no restriction on the data type and no restriction on the size of the data (none I'm aware of anyway).
So it's possible to use it in almost every context (including sending binary data). One of the advantages of REST is that web browser understand REST and your user won't need to have a dedicated application to send requests.
RPC presents more possibilities and can also be used. You can define custom operations for example.
Not sure you need that much power given what you intend to do.
Personally I would go with REST.
Here's a link you might wanna read:
http://www.sitepen.com/blog/2008/03/25/rest-and-rpc-relationship/
Compared to RPC, REST's(json style interface) is lightweight, it's easy for API user to use. RPC(soap/xml) seems complex and heavy.
I guess that what you want is HTTP+JSON based API, not the REST API that claimed by the REST author
http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven
Related
I have a set of soap webservices that are tightly coupled to an application within the same architecture but I need it to also be an API for other applications to hook into.
At the moment, the services use a parameter (and method) structure like this
Entity GetEntity(int entityId)
Entity GetEntityByName(string entityName)
.... etc.
In the case of creates I use:
void CreateEntity(Entity entity)
I am wondering though would it be better to do it like this:
EntityResponse GetEntity(EntityRequest requestObj) .....
and in the requestObj I have id, entityName and depending what the user supplies, I can perform either function.
and for the create it would be:
CreateEntityResponse CreateEntity(CreateEntityRequest requestObj).
My thinking is that by doing it like this, the API can change internally...add new parameters etc without immediately breaking any integration that has already been done.
I think there are several design principles that you may want to consider:
1) Database Entity vs Data Transport Object DTO
Looks like those Entities come directly from a database mapping? Just exposing your Entities as API, is basically a fancy SQL Query browser. It's not necessarily wrong but you will achieve better de-coupling if you would expose DTO's in the API.
The DTO's could be more future proof then the Entities and more generic.
2) SOAP vs REST
If you want to achieve a maximum of future proofing you might want to consider REST. With the REST specification you have more options to extend the API later.
For instance if you look at APIs like Facebook they purely pass in parameters and then you receive in return a key-value map of the parameters you passed in. So very generic.
In SOAP you would always end up in defining all of those eventual attributes upfront. You basically need to introduce placeholders et cetera.
There is certainly a reason why SOAP is a good contract protocol and has advantages like code generating tools are more up to date and lots more. But with REST you could be even more flexible while loosing some of the goodies you had in SOAP.
This is also a very good read:
https://www.mulesoft.com/lp/whitepaper/api/secrets-great-api
or generally the RAML design spec from Mule is a very powerful tool when it comes to designing REST APIs.
I have written my own Restful API and am wondering about the best way to deal with large amounts of records returned from the API.
For example, if I use GET method to myapi.co.uk/messages/ this will bring back the XML for all message records, which in some cases could be 1000's. This makes using the API very sluggish.
Can anyone suggest the best way of dealing with this? Is it standard to return results in batches and to specify batch size in the request?
You can change your API to include additional parameters to limit the scope of data returned by your application.
For instance, you could add limit and offset parameters to fetch just a little part. This is how pagination can be done in accordance with REST. A request like this would result in fetching 10 resources from the messages collection, from 21st to 30th. This way you can ask for a specific portion of a huge data set:
myapi.co.uk/messages?limit=10&offset=20
Another way to decrease the payload would be to only ask for certain parts of your resources' representation. Here's how facebook does it:
/joe.smith/friends?fields=id,name,picture
Remember that while using either of these methods, you have to provide a way for the client to discover each of the resources. You can't assume they'll just look at the parameters and start changing them in search of data. That would be a violation of the REST paradigm. Provide them with the necessary hyperlinks to avoid it.
I strongly recommend viewing this presentation on RESTful API design by apigee (the screencast is called "Teach a Dog to REST"). Good practices and neat ideas to approach everyday problems are discussed there.
EDIT: The video has been updated a number of times since I posted this answer, you can check out the 3rd edition from January 2013
There are different ways in general by which one can improve the API performance including for large API sizes. Each of these topics can be explored in depth.
Reduce Size Pagination
Organizing Using Hypermedia
Exactly What a User Need With Schema Filtering
Defining Specific Responses Using The Prefer Header
Using Caching To Make Response
More Efficient More Efficiency Through Compression
Breaking Things Down With Chunked Responses
Switch To Providing More Streaming Responses
Moving Forward With HTTP/2
Source: https://apievangelist.com/2018/04/20/delivering-large-api-responses-as-efficiently-as-possible/
if you are using .net core
you have to try this magic package
Microsoft.AspNetCore.ResponseCompression
then use this line in configureservices in startup file
services.AddResponseCompression();
then in configure function
app.UseResponseCompression();
I've been struggling with understanding a few points I keep reading regarding RESTful services. I'm hoping someone can help clarify.
1a) There seems to be a general aversion to generated code when talking about RESTful services.
1b) The argument that if you use a WADL to generate a client for a RESTful service, when the service changes - so does your client code.
Why I don't get it: Whether you are referencing a WADL and using generated code or you have manually extracted data from a RESTful response and mapped them to your UI (or whatever you're doing with them) if something changes in the underlying service it seems just as likely that the code will break in both cases. For instance, if the data returned changes from FirstName and LastName to FullName, in both instances you will have to update your code to grab the new field and perhaps handle it differently.
2) The argument that RESTful services don't need a WADL because the return types should be well-known MIME types and you should already know how to handle them.
Why I don't get it: Is the expectation that for every "type" of data a service returns there will be a unique MIME type in existence? If this is the case, does that mean the consumer of the RESTful services is expected to read the RFC to determine the structure of the returned data, how to use each field, etc.?
I've done a lot of reading to try to figure this out for myself so I hope someone can provide concrete examples and real-world scenarios.
REST can be very subtle. I've also done lots of reading on it and every once in a while I went back and read Chapter 5 of Fielding's dissertation, each time finding more insight. It was as clear as mud the first time (all though some things made sense) but only got better once I tried to apply the principles and used the building blocks.
So, based on my current understanding let's give it a go:
Why do RESTafarians not like code generation?
The short answer: If you make use of hypermedia (+links) There is no need.
Context: Explicitly defining a contract (WADL) between client and server does not reduce coupling enough: If you change the server the client breaks and you need to regenerate the code. (IMHO even automating it is just a patch to the underlying coupling issue).
REST helps you to decouple on different levels. Hypermedia discoverability is one of the goods ones to start with. See also the related concept HATEOAS
We let the client “discover” what can be done from the resource we are operating on instead of defining a contract before. We load the resource, check for “named links” and then follow those links or fill in forms (or links to forms) to update the resource. The server acts as a guide to the client via the options it proposes based on state. (Think business process / workflow / behavior). If we use a contract we need to know this "out of band" information and update the contract on change.
If we use hypermedia with links there is no need to have “separate contract”. Everything is included within the hypermedia – why design a separate document? Even URI templates are out of band information but if kept simple can work like Amazon S3.
Yes, we still need a common ground to stand on when transferring representations (hypermedia), so we define your own media types or use widely accepted ones such as Atom or Micro-formats. Thus, with the constraints of basic building blocks (link + forms + data - hypermedia) we reduce coupling by keeping out of band information to a minimum.
As first it seems that going for hypermedia does not change the impact of change :) : But, there are subtle differences. For one, if I have a WADL I need to update another doc and deploy/distribute. Using pure hypermedia there is no impact since it's embedded. (Imagine changes rippling through a complex interweave of systems). As per your example having FirstName + LastName and adding FullName does not really impact the clients, but removing First+Last and replacing with FullName does even in hypermedia.
As a side note: The REST uniform interface (verb constraints - GET, PUT, POST, DELETE + other verbs) decouples implementation from services.
Maybe I'm totally wrong but another possibility might be a “psychological kick back” to code generation: WADL makes one think of the WSDL(contract) part in “traditional web services (WSDL+SOAP)” / RPC which goes against REST. In REST state is transferred via hypermedia and not RPC which are method calls to update state on the server.
Disclaimer: I've not completed the referenced article in detail but I does give some great points.
I have worked on API projects for quite a while.
To answer your first question.
Yes, If the services return values change (Ex: First name and Last name becomes Full Name) your code might break. You will no longer get the first name and last name.
You have to understand that WADL is a Agreement. If it has to change, then the client needs to be notified. To avoid breaking the client code, we release a new version of the API.
The version 1.0 will have First Name and last name without breaking your code. We will release 1.1 version which will have the change to Full name.
So the answer in short, WADL is there to stay. As long as you use that version of the API. Your code will not break. If you want to get full name, then you have to move to the new versions. With lot of code generation plugins in the technology market, generating the code should not be a issue.
To answer your next question of why not WADL and how you get to know the mime types.
WADL is for code generation and serves as a contract. With that you can use JAXB or any mapping framework to convert the JSON string to generated bean objects.
If not WADL, you don't need to inspect every element to determine the type. You can easily do this.
var obj =
jQuery.parseJSON('{"name":"John"}');
alert( obj.name === "John" );
Let me know, If you have any questions.
I'm looking to expose a number of services to the web. There will be static web pages with jQuery based JavaScript code that accesses these services, and there will also be all kinds of applications that may access these services. (Or nobody will care; that's also quite possible :-)
Each service would be well defined as a collection of methods that act on some number of input parameters, and return some number of output parameters. Most of it is REST, except for the concept of "identity" -- these services requires some log-in, and logging in does set you up with permissions on which methods on which services you're allowed to use, and what particular entities you're allowed to address using those methods.
Ideally, I want to expose the services using JSONP to make the services easy to consume in a cross-site way -- those static web applications shouldn't all have to be be served from the domain of my application servers.
The set of data types is fairly basic -- varchar (255 chars), text (8191 chars), id (32 chars, C-style identifier constraint), double, long (hard to do in JavaScript), bool, datetime, email (varchar matching a reges) and url (varchar matching a regex) would probably suffice for a very long time.
I want, if possible, to implement these services using an application service technology that can scale on multi-threaded cores -- 24 threads is standard on plain mid-range servers these days. Sticking with Python or Node.js would make me uneasy, because of their lack of threading support. Also, I would like for typing to be static, because I believe static typing prevents a certain class of bugs, which also talks against Python, Node.js, PHP and Ruby.
I also want to serve on Linux. That's a pretty hard requirement :-)
In the static language world, there are really only a few server frameworks or framework approaches. There's HttpListener with Mono. There's Jetty with Java. There's a few others. There are also a few much deeper frameworks, that have more overhead than I'd like -- J2EE, ASP.NET, etc. (In the dynamic world, you have Cake.PHP, Rails, Django, etc etc)
So, in the best of worlds, I'd like the GET URL /foo/bar?arg1=2&arg2=xyzzy to map to an object that I write of type foo, method bar, taking arguments arg1 and arg2 of type int and string (say). I want the server glue that sits between the HTTP GET and the object method to do two things:
1) Permission control (each method could have one or more required permissions)
2) Type checking (each argument should be verified against expected and coerced from the URL string format to the native typ)
Once the method returns one or more values (key/value pairs), the glue should make sure that the return values are also according to spec, convert to proper JSON, and should take care of returning an appropriate error result if an exception was thrown within the code.
I've been looking high and low for this kind of solution, but all solutions I find are pretty far off. JSP and ASP all start out with a HTML assumption -- I'm generally generating JSON, which doesn't integrate as well with the syntax (to put it mildly). JSPX, as well as a number of other technologies (HttpListener, CGI, Python Twisted, ...) all stop at the "do_GET" level -- no dispatch into objects, no permission control, no type checking. All the higher-level frameworks, however, add a lot more on top of that, and often adds complex routing that's not something I need -- and as often as not, still don't do permission checking, but instead leaves that as something you have to write manually in each function implementation.
I think the closest I could find to what I want is Thrift. However, it still doesn't do permission checking, and the "PHP server" support it has seems to be a dumb PHP cli process listening on port 80 instead of integrating with Apache, and it's not set up to support JSONP.
Have I missed something? Is there some (preferrably statically typed and multi-thread-supporting) server technology that will do type checking and permission checking and simple object-method dispatch, without tons of other cruft to get in the way, and can be called from (and respond to) JSONP?
Should I extend Thrift? Adding permission constraints on each method would be a fairly substantial extension, but at least I'd get the other support that Thrift has. (And then I'd have to add JSONP support, and ...)
Is there some (preferrably statically typed and multi-thread-supporting) server technology that will do type checking and permission checking and simple object-method dispatch, without tons of other cruft to get in the way
Scala (static, actors for threading/concurrency & dispatch, no cruft) + Thrift.
I believe the latest thrift has javascript support (not sure about JSONP).
As for authentication, see my answer to this question.
There really doesn't exist any good framework at the right level for this. Existing frameworks are either too low level (think boost::asio), or too high level (think cake, rails, etc). The closest option is probably Erlang/WebMachine. I guess the reason for this is that most web apps end up having a common set of higher-level requirements that then end up migrating into whatever the web framework is, thus lifting it out of the level where I'm looking to go.
We are considering to develop a Flash front-end to a web application written using Django. The Flash front-end will send a simple "id" to the server and in response receive a couple of objects. The application will be open only to authenticated users.
To the extend of my current knowledge (which is basic for Flash) we can either use AMF or take an XML or JSON approach. AMF seems to have an upperhand as there are examples out on the internet showing it can cooperate easily with Django's authentication mechanism (most examples feature pyAMF). On the other hand, implementing a XML/JSON based solution may be easier and hassle free.
Guidance will be much appreciated.
We've used pyAMF + Django on many projects here, and it's a breeze to setup and get running. If you need speed, AMF3 is probably your best bet. It's the smallest/fastest way to transfer data, and serialization is taken care of for you.
On the flip side, setting up json with Django isn't much work either, and it would give you the ability to hook other, non-AMF systems into it without any extra work. You just sacrifice a little speed for that benefit.
If you think you'll ever need other systems working with the backend, or if you think you might switch to an HTML-only, or offer some kind of non-Flash version of your app, I'd go JSON, otherwise, I'd use AMF.
first of all, you should design your app in such a way this doesn't matter. the transport layer should be completely encapsulated, leaving the encoding format transparent to the rest of the app.
personally I prefer JSON to AMF because it's human readable (which makes debugging easier) and there are implementations for every platform/language (so you can reuse the server part with JavaScript for example). And I prefer JSON to XML because it's more compact and semantically less unambiguous as well as closer to common object models. Also it can transport numerical and boolean data in a typesafe manner.
JSON will probaby have the least complications and there's a great google code project that has JSON encoders and decoders here: http://code.google.com/p/as3corelib/