I'm using Django to implement a private rest-like API and I'm unsure of how to handle different versions of the API on the backend.
Meaning, if I have 2 versions of the API what does my code look like? Should I have different apps that handle different version? Should different functions handle different versions? Or should I just use if statements for when one version differs from another?
I plan on stating the version in the Header.
Thanks
You do not need to version REST APIs. With REST, versioning happens at runtime either through what one might call 'must-ignore payload extension rules' or through content negotiation.
'must-ignore payload extension rules' refer to an aspect you build into the design of your messages. 'Must-ignore' means that a piece of software that processes a message of the given format must ignore any unknown syntactical constructs. This is what we all know from HTML and what makes it possible to insert all sorts of fancy tags into an HTML page without the parser choking.
'Must-ignore' allows you to evolve the capabilities of your service by adding stuff to what you send already without considering clients that only understand the older versions.
Content-negotiation refers to the HTTP-built-in mechanism of negotiating the actual representation the server sends to a given client at runtime. The typical scenario is this: Clients send the Accept header in the request to advertise what they are capable of and servers pick the representation to send back based on these capabilities. But there are also variations of this theme (see here for details: http://www.w3.org/Protocols/rfc2616/rfc2616-sec12.html ).
Content negotiation allows for incompatible changes, meaning that I can evolve my service to being able to send incompatible old and new versions and based on the Accept header my service will send the appropriate one.
Bottom line: with both approaches, your API remains as it is. No need to do any versioning at the API level - especially not the often suggested (but totally wrong) inclusion of version identifiers in the URIs (remember, you are doing REST here, not SOAP!)
Related
Background:
A certain government-backed wholesaler of broadband services in Australia took feedback from discussion groups about how best to deliver B2B services to retail ISPs. They settled on EbXML.
Problem:
We're a very small shop (comparatively) that doesn't want to spend a lot of time going forward on integration. We're already familiar with integration of paired (inbound and outbound) SOAP services. In the past we've made use of WSDL-based code generation tooling (mostly with RPC/Literal services) where the WSDL has been descriptive and simple enough for the code generation tools to digest.
If at all possible we'd like to avoid having to hand-integrate the services with our business 'stack'. We know that the 'Interface Schemas' have been updated several times; we'd like to (as much as possible) do code and schema generation such that we can model our relationship with the supplier and the outbound/inbound messages as simple "queues" (tables) in an SQL database -- this will be our point of integration.
Starting with the outbound ("sender") SOAP web-service... it publishes a Document/Literal WSDL description of the service that seems to work correctly with various tools (e.g: wsdl2java, SoapUI) to generate the EBXML 'wrapper' messages. This says nothing about the 'payload' messages themselves which (at least for the MSH we've looked at) need to be multipart/related attachments with type of text/xml.
The 'payload' messages are defined in the provided CPA (something like bindings) and Schema (standard-looking XSD) files. The MSH itself doesn't seem to provide any external validation for the payload messages.
Question:
Is the same kind of code generation (as seen with WSDL-described SOAP web services) tooling available for EbXML CPAs/Schemas? (i.e: tools that can consume the CPA and 'payload' interface schemas and spit out java/c++/whatever, and/or something WSDL-like specific to the 'payload' interface messages and/or example messages).
If so, where do I look?
If not, are there any EbXML-specific problems that would prevent it? (I'd rather not get several weeks into a project to develop tools that are impossible to implement 'correctly' given the information at hand).
The MSH is payload agnostic. The payloads are not defined in the CPA, only the service and action names that are used to send the ebXML payloads are. The service and action are transmitted in the ebXML header, which is the first part of the multipart message. The payloads themselves can be xml, binary or a combination. Each payload is another part.
An MSH is responsible for tasks like:
sending (usually asynchronous) acknowledgements for received messages
resending messages if an acknowledgement has not been received within a certain amount of time
ignoring duplicate messages
assuring the order in which messages are delivered is correct
the actual behaviour is all configurable using the CPA, but a compliant MSH would support all this.
This implies that an MSH has to keep an administration of the messages it has sent and received, which is usually done in a database.
I would be surprised if you could find tooling to generate an MSH from a specific CPA. What you can find is software/components that implement a generic MSH and that can be configured with CPAs.
Assuming you don't want to build your own, look for an existing ebMS adapter. Configure it with your CPA(s). Then generate the payloads however you like and pass them to the ebMS adapter.
Google for "ebMS adapter" or "ebMS support".
Alas, it seems there's no specific tooling around the 'payload' messages for EbXML, spefically because EbXML doesn't regulate those messages.
However, the CPA (through canSend and canRecv) elements acts somewhat like a SOAP WSDL, and the XSDs serve the same purpose as with SOAP, so it's not too far off.
There does exist software for turning types defined in XSDs into messages (merging in user-supplied data) at runtime, but per my question there's no obvious tooling for code generation around CPAs and related XSDs.
Furthermore, actually writing software to do this yourself is made more problematic by the dificulty of searching for the meta-grammar for XML Schema (i.e: that grammar which remains of XML Schema once XML tokenization is factored out). Basically, this was difficult because in the XML world, the word "grammar" has an different meaning which polutes search results.
I was able to write a parser for the XML syntax snippets present at the top of each of the MSDN articals on XML Schema (elements listed down the left), which in turn allowed me to generate an LL1 grammar for XML schema which works on the pre-parsed AST of a given XSD.
From there I built a top-down parser from this meta-grammar which:
Follows <xsd:import>s and <xsd:include>s to resolve namespaces into further XSDs.
Recursively resolves message types in order to produce a 'flattened' type for each CPA message.
Generates a packer/unpacker data-structures for the message types which allow generation of code in various languages, as well as serialisation to and parsing from validated 'payload' XML.
There are still various XML Schema restrictions, keys, and other constraints that my code generators don't know about, but support for these can be added in time.
I'll update this answer with links to grammars (and possibly code -- depends on legals) as time permits. I'll leave the question as non-accepted for a while so that if someone miraculously finds a tool which makes much less work of the code generation, I'll accept an answer based on that.
I'm looking for a possiblity to monitor all messages in a SOA enviroment with an intermediary, who'll be designed to enforce different rule-sets over the message's structure and sequences (e.g., let's say it'll check and ensure that Service A has to be consumed before B).
Obviously the first idea that came to mind is how WS-Adressing might help here, but I'm not sure if it does, as I don't really see any mechanism there to ensure that a message will get delivered via a given intermediary (as it is in WS-Routing, which is an outdated proprietary protocol by Microsoft).
Or maybe there's even a different approach that the monitor wouldn't be part of the route but would be notified on request/responses, which might it then again make somehow harder to actively enforce rules.
I'm looking forward to any suggestions.
You can implement a "service firewall" either by intercepting all the calls in each service as part of your basic servicehost. Alternatively you can use 3rd party solutions and route all your service calls to them (they will do the intercepting and then forward calls to your services).
You can use ESBs to do the routing (and intercepting) or you can use dedicated solutions like IBM's datapower, XML firewall from Layer7 etc.
For all my (technical) services I use messaging and the command processor pattern, which I describe here, without actually calling the pattern name though. I send a message and the framework finds to corresponding class that implements the interface that corresponds to my message. I can create multiple classes that can handle my message, or a single class that handles a multitude of messages. In the article these are classes implementing the IHandleMessages interface.
Either way, as long as I can create multiple classes implementing this interface, and they are all called, I can easily add auditing without adding this logic to my business logic or anything. Just add an additional implementation for every single message, or enhance the framework so it also accepts IHandleMessages implementations. That class can than audit every single message and store all of them centrally.
After doing that, you can find out more information about the messages and the flow. For example, if you put into the header information of your WCF/MSMQ message where it came from and perhaps some unique identifier for that single message, you can track the flow over various components.
NServiceBus also has this functionality for auditing and the team is working on additional tooling for this, called ServiceInsight.
Hope this helps.
I am writing a C++ API which is to be used as a web service. The functions in the API take in images/path_to_images as input parameters, process them, and give a different set of images/paths_to_images as outputs. I was thinking of implementing a REST interface to enable developers to use this API for their projects (independent of whatever language they'd like to work in). But, I understand REST is good only when you have a collection of data that you want to query or manipulate, which is not exactly the case here.
[The collection I have is of different functions that manipulate the supplied data.]
So, is it better for me to implement an RPC interface for this, or can this be done using REST itself?
Like lcfseth, I would also go for REST. REST is indeed resource-based and, in your case, you might consider that there's no resource to deal with. However, that's not exactly true, the image converter in your system is the resource. You POST images to it and it returns new images. So I'd simply create a URL such as:
POST http://example.com/image-converter
You POST images to it and it returns some array with the path to the new images.
Potentially, you could also have:
GET http://example.com/image-converter
which could tell you about the status of the image conversion (assuming it is a time consuming process).
The advantage of doing it like that is that you are re-using HTTP verbs that developers are familiar with, the interface is almost self-documenting (though of course you still need to document the format accepted and returned by the POST call). With RPC, you would have to define new verbs and document them.
REST use common operation GET,POST,DELETE,HEAD,PUT. As you can imagine, this is very data oriented. However there is no restriction on the data type and no restriction on the size of the data (none I'm aware of anyway).
So it's possible to use it in almost every context (including sending binary data). One of the advantages of REST is that web browser understand REST and your user won't need to have a dedicated application to send requests.
RPC presents more possibilities and can also be used. You can define custom operations for example.
Not sure you need that much power given what you intend to do.
Personally I would go with REST.
Here's a link you might wanna read:
http://www.sitepen.com/blog/2008/03/25/rest-and-rpc-relationship/
Compared to RPC, REST's(json style interface) is lightweight, it's easy for API user to use. RPC(soap/xml) seems complex and heavy.
I guess that what you want is HTTP+JSON based API, not the REST API that claimed by the REST author
http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven
I'm looking to expose a number of services to the web. There will be static web pages with jQuery based JavaScript code that accesses these services, and there will also be all kinds of applications that may access these services. (Or nobody will care; that's also quite possible :-)
Each service would be well defined as a collection of methods that act on some number of input parameters, and return some number of output parameters. Most of it is REST, except for the concept of "identity" -- these services requires some log-in, and logging in does set you up with permissions on which methods on which services you're allowed to use, and what particular entities you're allowed to address using those methods.
Ideally, I want to expose the services using JSONP to make the services easy to consume in a cross-site way -- those static web applications shouldn't all have to be be served from the domain of my application servers.
The set of data types is fairly basic -- varchar (255 chars), text (8191 chars), id (32 chars, C-style identifier constraint), double, long (hard to do in JavaScript), bool, datetime, email (varchar matching a reges) and url (varchar matching a regex) would probably suffice for a very long time.
I want, if possible, to implement these services using an application service technology that can scale on multi-threaded cores -- 24 threads is standard on plain mid-range servers these days. Sticking with Python or Node.js would make me uneasy, because of their lack of threading support. Also, I would like for typing to be static, because I believe static typing prevents a certain class of bugs, which also talks against Python, Node.js, PHP and Ruby.
I also want to serve on Linux. That's a pretty hard requirement :-)
In the static language world, there are really only a few server frameworks or framework approaches. There's HttpListener with Mono. There's Jetty with Java. There's a few others. There are also a few much deeper frameworks, that have more overhead than I'd like -- J2EE, ASP.NET, etc. (In the dynamic world, you have Cake.PHP, Rails, Django, etc etc)
So, in the best of worlds, I'd like the GET URL /foo/bar?arg1=2&arg2=xyzzy to map to an object that I write of type foo, method bar, taking arguments arg1 and arg2 of type int and string (say). I want the server glue that sits between the HTTP GET and the object method to do two things:
1) Permission control (each method could have one or more required permissions)
2) Type checking (each argument should be verified against expected and coerced from the URL string format to the native typ)
Once the method returns one or more values (key/value pairs), the glue should make sure that the return values are also according to spec, convert to proper JSON, and should take care of returning an appropriate error result if an exception was thrown within the code.
I've been looking high and low for this kind of solution, but all solutions I find are pretty far off. JSP and ASP all start out with a HTML assumption -- I'm generally generating JSON, which doesn't integrate as well with the syntax (to put it mildly). JSPX, as well as a number of other technologies (HttpListener, CGI, Python Twisted, ...) all stop at the "do_GET" level -- no dispatch into objects, no permission control, no type checking. All the higher-level frameworks, however, add a lot more on top of that, and often adds complex routing that's not something I need -- and as often as not, still don't do permission checking, but instead leaves that as something you have to write manually in each function implementation.
I think the closest I could find to what I want is Thrift. However, it still doesn't do permission checking, and the "PHP server" support it has seems to be a dumb PHP cli process listening on port 80 instead of integrating with Apache, and it's not set up to support JSONP.
Have I missed something? Is there some (preferrably statically typed and multi-thread-supporting) server technology that will do type checking and permission checking and simple object-method dispatch, without tons of other cruft to get in the way, and can be called from (and respond to) JSONP?
Should I extend Thrift? Adding permission constraints on each method would be a fairly substantial extension, but at least I'd get the other support that Thrift has. (And then I'd have to add JSONP support, and ...)
Is there some (preferrably statically typed and multi-thread-supporting) server technology that will do type checking and permission checking and simple object-method dispatch, without tons of other cruft to get in the way
Scala (static, actors for threading/concurrency & dispatch, no cruft) + Thrift.
I believe the latest thrift has javascript support (not sure about JSONP).
As for authentication, see my answer to this question.
There really doesn't exist any good framework at the right level for this. Existing frameworks are either too low level (think boost::asio), or too high level (think cake, rails, etc). The closest option is probably Erlang/WebMachine. I guess the reason for this is that most web apps end up having a common set of higher-level requirements that then end up migrating into whatever the web framework is, thus lifting it out of the level where I'm looking to go.
The situation:
We have a library project that houses much of our code for the various integrations we work on. Many of the integrations consume web service apis, and my supervisor doesn't want 5 gazillion web service references added to the project.
What we generally do, then, is add a reference to a new project and copy the References.vb to the solution and just call the generated code. Not terribly convenient if changes are made to the service, but it works.
Recently, I ran into a problem where we have to use 3 web services for the same integration. 2 of these contain the same class definitions, however, they're in different namespaces because they belong to different services. This became a problem for me because one of the services searches a user based on user ID, and the other pulls back blocks of users. Both return an object, or list of, that is exactly the same semantically. And I need to process the data the same, whether it came from one service or the other.
My solution, was to strip out the duplicated classes in the service and replace them with classes inherited from common base classes. This allowed me to work with both objects as if they were the same, however, it required modifying the generated web service proxy. Therefore this change will need to be made every time I need to regenerate the proxy.
I'm curious what you all might think a better solution to this would be.
You're going to regret playing games with copying Reference.vb and editing generated files.
Switch to WCF and you'll be able to tell it you want to reuse the types, instead of having multiple types that are more or less the same.
BTW, they would be "less" the same if not all of the web references are updated at the same time after a server change.
The other option would be to build an abstraction layer over top of the web service pre-generated proxies, such that when you make to the calls to the abstraction layer you can always use the same objects, as they are squeezed into (and out of) the web service proxies in the abstraction layer. This would also allow for unit testing :)
I think you really should be looking at WCF for 3.5+, but for .NET 2.0 look at something like WSCF (Web Services Contract First), which defines the contracts in XML and generates a set of libraries reusable across services. E.g You define a MyComany.WS.Common namespace and use that namespace in multiple projects. The code generation then builds a shared library of types which get used across all the web-services. We use this extensively in our .NET 2 solutions and it's great. We had to do some additional work around the code generation to get it to fit into our build process, but once that was done we never looked back.
We're migrating to .NET 3.5 over time, so the WSCF will become obsolete
Heres the link to the thinktecture site for WSCF.
wsdl.exe using the /sharetypes switch allows the same types to be used across multiple service definitions, provided the wire signatures are not correct. I was unable to use it in my situation, though, because the various wsdl contracts were carelessly namespaced.