I'm looking to expose a number of services to the web. There will be static web pages with jQuery based JavaScript code that accesses these services, and there will also be all kinds of applications that may access these services. (Or nobody will care; that's also quite possible :-)
Each service would be well defined as a collection of methods that act on some number of input parameters, and return some number of output parameters. Most of it is REST, except for the concept of "identity" -- these services requires some log-in, and logging in does set you up with permissions on which methods on which services you're allowed to use, and what particular entities you're allowed to address using those methods.
Ideally, I want to expose the services using JSONP to make the services easy to consume in a cross-site way -- those static web applications shouldn't all have to be be served from the domain of my application servers.
The set of data types is fairly basic -- varchar (255 chars), text (8191 chars), id (32 chars, C-style identifier constraint), double, long (hard to do in JavaScript), bool, datetime, email (varchar matching a reges) and url (varchar matching a regex) would probably suffice for a very long time.
I want, if possible, to implement these services using an application service technology that can scale on multi-threaded cores -- 24 threads is standard on plain mid-range servers these days. Sticking with Python or Node.js would make me uneasy, because of their lack of threading support. Also, I would like for typing to be static, because I believe static typing prevents a certain class of bugs, which also talks against Python, Node.js, PHP and Ruby.
I also want to serve on Linux. That's a pretty hard requirement :-)
In the static language world, there are really only a few server frameworks or framework approaches. There's HttpListener with Mono. There's Jetty with Java. There's a few others. There are also a few much deeper frameworks, that have more overhead than I'd like -- J2EE, ASP.NET, etc. (In the dynamic world, you have Cake.PHP, Rails, Django, etc etc)
So, in the best of worlds, I'd like the GET URL /foo/bar?arg1=2&arg2=xyzzy to map to an object that I write of type foo, method bar, taking arguments arg1 and arg2 of type int and string (say). I want the server glue that sits between the HTTP GET and the object method to do two things:
1) Permission control (each method could have one or more required permissions)
2) Type checking (each argument should be verified against expected and coerced from the URL string format to the native typ)
Once the method returns one or more values (key/value pairs), the glue should make sure that the return values are also according to spec, convert to proper JSON, and should take care of returning an appropriate error result if an exception was thrown within the code.
I've been looking high and low for this kind of solution, but all solutions I find are pretty far off. JSP and ASP all start out with a HTML assumption -- I'm generally generating JSON, which doesn't integrate as well with the syntax (to put it mildly). JSPX, as well as a number of other technologies (HttpListener, CGI, Python Twisted, ...) all stop at the "do_GET" level -- no dispatch into objects, no permission control, no type checking. All the higher-level frameworks, however, add a lot more on top of that, and often adds complex routing that's not something I need -- and as often as not, still don't do permission checking, but instead leaves that as something you have to write manually in each function implementation.
I think the closest I could find to what I want is Thrift. However, it still doesn't do permission checking, and the "PHP server" support it has seems to be a dumb PHP cli process listening on port 80 instead of integrating with Apache, and it's not set up to support JSONP.
Have I missed something? Is there some (preferrably statically typed and multi-thread-supporting) server technology that will do type checking and permission checking and simple object-method dispatch, without tons of other cruft to get in the way, and can be called from (and respond to) JSONP?
Should I extend Thrift? Adding permission constraints on each method would be a fairly substantial extension, but at least I'd get the other support that Thrift has. (And then I'd have to add JSONP support, and ...)
Is there some (preferrably statically typed and multi-thread-supporting) server technology that will do type checking and permission checking and simple object-method dispatch, without tons of other cruft to get in the way
Scala (static, actors for threading/concurrency & dispatch, no cruft) + Thrift.
I believe the latest thrift has javascript support (not sure about JSONP).
As for authentication, see my answer to this question.
There really doesn't exist any good framework at the right level for this. Existing frameworks are either too low level (think boost::asio), or too high level (think cake, rails, etc). The closest option is probably Erlang/WebMachine. I guess the reason for this is that most web apps end up having a common set of higher-level requirements that then end up migrating into whatever the web framework is, thus lifting it out of the level where I'm looking to go.
Related
I am writing a C++ API which is to be used as a web service. The functions in the API take in images/path_to_images as input parameters, process them, and give a different set of images/paths_to_images as outputs. I was thinking of implementing a REST interface to enable developers to use this API for their projects (independent of whatever language they'd like to work in). But, I understand REST is good only when you have a collection of data that you want to query or manipulate, which is not exactly the case here.
[The collection I have is of different functions that manipulate the supplied data.]
So, is it better for me to implement an RPC interface for this, or can this be done using REST itself?
Like lcfseth, I would also go for REST. REST is indeed resource-based and, in your case, you might consider that there's no resource to deal with. However, that's not exactly true, the image converter in your system is the resource. You POST images to it and it returns new images. So I'd simply create a URL such as:
POST http://example.com/image-converter
You POST images to it and it returns some array with the path to the new images.
Potentially, you could also have:
GET http://example.com/image-converter
which could tell you about the status of the image conversion (assuming it is a time consuming process).
The advantage of doing it like that is that you are re-using HTTP verbs that developers are familiar with, the interface is almost self-documenting (though of course you still need to document the format accepted and returned by the POST call). With RPC, you would have to define new verbs and document them.
REST use common operation GET,POST,DELETE,HEAD,PUT. As you can imagine, this is very data oriented. However there is no restriction on the data type and no restriction on the size of the data (none I'm aware of anyway).
So it's possible to use it in almost every context (including sending binary data). One of the advantages of REST is that web browser understand REST and your user won't need to have a dedicated application to send requests.
RPC presents more possibilities and can also be used. You can define custom operations for example.
Not sure you need that much power given what you intend to do.
Personally I would go with REST.
Here's a link you might wanna read:
http://www.sitepen.com/blog/2008/03/25/rest-and-rpc-relationship/
Compared to RPC, REST's(json style interface) is lightweight, it's easy for API user to use. RPC(soap/xml) seems complex and heavy.
I guess that what you want is HTTP+JSON based API, not the REST API that claimed by the REST author
http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven
How should a Windows 8 Metro application connect to a central database?
I've read about local storage, but I haven't read anything about connecting to a central database.
Obviously, this architectural design decision needs to support the disconnected scenario.
WCF web services seem to make sense.
But even if they do make sense, should we really create separate methods for all read/write operations?
Or are OData WCF services the way to go?
It seems like tablet software architecture should be able to borrow a lot from smartphone software architecture (but I am new to both).
Has Microsoft made any recommendations in its app samples?
It appears that others are asking similar questions on the Microsoft Developer Forums.
Here is what I've found:
According to Tim Heuer:
...You cannot directly have a SQL db embedded in your app or use
something like ADO.NET. This is more of an async/services
infrastructure. So if your data was exposed via services, then of
course you could connect that way. There are some other light-weight
methods you could use for local storage as well using things like the
Windows.Storage namespace (which is similar to Isolated Storage in
.NET).
Morten Nielsen agrees:
You can use HttpClient to download pretty much anything from the web.
Why don't you configure your WCF service to return data as JSON, and
use the DataContractJsonSerializer to deserialize the results?
Also, Tim Heuer cautions:
...Please note that while awesome, the SQLWinRT project on codeplex is a
wrapper to communicate with the classic SQLite engine...which uses
APIs that would not pass store validation currently.
Generic Object Storage Helper for WinRT and WinRTFile Based Database seem to have some promise.
But Daniel Stolt raises some good points:
It's awesome that there is good support for building OData clients and
other REST clients - but this only addresses the online scenario. The
"structured" part of Windows.Storage is a very limited model,
essentially limited to name/value pairs, insufficient for all but the
most basic scenarios. Yes there is local file storage, which is great
of course. But forcing every app developer out there to build her own
DBMS on top of local file storage will simply not cut it, especially
with all of System.Data having been removed from the profile. If local
file storage was sufficient for most device apps, then things like
SQLCE would have no purpose today already. And SQLCE clearly has a
purpose, and has played a very important role for occasionally
connected device apps for a very long time. There is also a tremendous
need for synchronization with a server-side database such as SQL
Azure, mostly to be able to roam data between devices. Yes there is
the roaming storage model in WinRT, but it shares the same limitations
of local storage mentioned above, and on top of this is very limited
in capacity (currently 30KB if memory serves). It is simply
insufficient for all but the simplest roaming data needs. Again,
forcing every app developer to design and implement her own
synchronization solution is very bad. You can do much better to enable
developers.
Many people are disappointed that the System.Data namespace is not supported in WinRT.
Richard Bethell said:
I don't even have words for this. This is astonishing. Leave aside for
the moment they want to force you to abstract to middleware for
database connectivity - I don't agree, but I can quasi understand a
rationale for that. I can even see pathways for developing like that.
But no System.Data.... at all? Do you even understand what you've done
to us?
What System.Data can do, outside of just having providers for Sql,
OleDb and other custom providers like Oracle, is provide a rich
abstraction of XML datasets that allow you to very quickly build a
data oriented Service Oriented Architecture.
For instance, I can easily create a web service using SOAP or WCF that
returns DataSets or DataTables, and then consume those objects easily
and directly. Being able to do this allows very rapid construction of
n-tier architectures, even without direct data connections available.
Without System.Data, and the power of DataViews, DataTables, etc. this
gets a lot harder. Sure you can custom create structs, put data in
there, and serve up structs, and use Linq to do whatever sorting,
filtering, etc. you want to do.... but it ends up being twice the
work, and makes code reuse a lot harder. And it means using our
existing service oriented architecture is impossible (without a big
overhaul.)
The withdrawal of System.Data is as big a thing for developers to deal
with as the loss of the Printer object in VB6 to vb.net 1.0 was. What
is harder to understand in this case is why it is necessary -
re-enabling it in the Metro profile can't possibly be a technical
difficulty of the product, can it?
It is valuable enough that I would seriously consider including Mono's
System.Data classes as part of any app I create (which would obviously
have to be open source.)
I think that this is another of those "it depends" questions...
The first and most obvious issue is that it very much depends on the context in which the application is running as to whether, to take the first case "Obviously...support...disconnected" is actually true - if the app is an internal corporate app then quite possibly not in that case no db == not work.
Secondly you could look (hmm, rash... one assumes you could look, this could be a bad assumption) at database synchronisation between a local SQL database and the remote db and so on and so forth.
Taking a step back... yes - you're absolutely right, look at it as being the same as phone or silverlight (although I don't know if there is yet RIA support) - but the thing is at this point its very hard to be prescriptive because given a general purpose platform one can therefore write applications to suit all sorts of purposes.
Not a hugely helpful answer really - but a start.
Having read #Jim G's answer it seems that I should probably withdrawn mine?
We are considering to develop a Flash front-end to a web application written using Django. The Flash front-end will send a simple "id" to the server and in response receive a couple of objects. The application will be open only to authenticated users.
To the extend of my current knowledge (which is basic for Flash) we can either use AMF or take an XML or JSON approach. AMF seems to have an upperhand as there are examples out on the internet showing it can cooperate easily with Django's authentication mechanism (most examples feature pyAMF). On the other hand, implementing a XML/JSON based solution may be easier and hassle free.
Guidance will be much appreciated.
We've used pyAMF + Django on many projects here, and it's a breeze to setup and get running. If you need speed, AMF3 is probably your best bet. It's the smallest/fastest way to transfer data, and serialization is taken care of for you.
On the flip side, setting up json with Django isn't much work either, and it would give you the ability to hook other, non-AMF systems into it without any extra work. You just sacrifice a little speed for that benefit.
If you think you'll ever need other systems working with the backend, or if you think you might switch to an HTML-only, or offer some kind of non-Flash version of your app, I'd go JSON, otherwise, I'd use AMF.
first of all, you should design your app in such a way this doesn't matter. the transport layer should be completely encapsulated, leaving the encoding format transparent to the rest of the app.
personally I prefer JSON to AMF because it's human readable (which makes debugging easier) and there are implementations for every platform/language (so you can reuse the server part with JavaScript for example). And I prefer JSON to XML because it's more compact and semantically less unambiguous as well as closer to common object models. Also it can transport numerical and boolean data in a typesafe manner.
JSON will probaby have the least complications and there's a great google code project that has JSON encoders and decoders here: http://code.google.com/p/as3corelib/
What simple tools would you recommend to read a text file of addresses, send each record separately to a web service for geocoding, and save the batch of results as a text file?
Looking for no-frills component(s) with usage examples, for minimal code-from-scratch. Language irrelevant as long as dev environment is easy to install.
Requirements
- usable by unsophisticated programmer
- low or no cost
- runs under Windows.
Second thought:
How easy would this be to do inside a browser using JavaScript and a library or two?
I'd go for Java and use a flat file parsing library like jFFP or Flatworm
These libraries are pretty easy to understand and to use (I've worked with both of them in the past) and they both provide code samples.
Spring Integration would be another good option but the learning curve might be too big if you are not familiar with Spring and it might be overkill for your simple workflow.
Actually, in your case, I think I'd choose Flatworm for the parsing. You'll find code samples on its website or in How to read and parse flat files in Java. And you could even use it to write your output file like in Writing flat files in Java with Flatworm).
For the SOAP part, I'd use the JAX-WS Reference Implementation (which is included in the JDK 6 so you won't have to add any library if you are using Java 6) and Netbeans IDE. Netbeans IDE has very good support for developing JAX-WS Web Services Client (or here for later versions of Netbeans) and should really ease the process. Once the various classes generated, calling the web service is a matter of 3 lines of code as shown in the examples of the provided links:
// Call Web Service Operation
com.cdyne.ws.Check service = new com.cdyne.ws.Check();
com.cdyne.ws.CheckSoap port = service.getCheckSoap();
// TODO initialize WS operation arguments here
java.lang.String bodyText = "";
java.lang.String licenseKey = "";
// TODO process result here
com.cdyne.ws.DocumentSummary result = port.checkTextBody(bodyText, licenseKey);
Given the generic nature of the requirements, the relatively simple workflow, but its potential to bring a few twists and turns in the design (for example, the need of using https rather than http for webservices, the need of producing some odd token for authentication, or some fancy marshaling or conversion etc.) it might be best to use a modern script language. A very basic plan could be to use a plain shell script (a bat file), base on curl and a few other command line utilities, but this approach may not be flexible enough to deal with some requirements; instead languages such as Perl, PHP, Python, Ruby would be much preferable.
This would provide a low entry barrier, the ability to test elements of the application interactively before putting them into a formal script, and to leverage extensive libraries to deal with the various requirements that may arise, such as the storage of configuration parameters, parsing detail, output format, webservices, maths associated with geo positions etc. etc.
My inclination would be to use Python, but as said most other modern dynamic languages would do.
I would use XMLunit with Eclipse IDE, + JUnit, and JDK1.6 . A finished program that does this might only be 100 lines of code. It's doable by someone who is a novice programmer...
When the program is done you can compile as an .exe file for future use.
I would choose "Strawberry Perl" as my second choice for programming language. Python is slightly harder to use I think.
I know that some big players have embraced it and are actually exposing some of their services in APP compliant way, already. However, I haven't found many other (smaller) players in this field. Do you know any web application/service that uses APP as its public API protocol? What is your own take on AtomPub? Do you have any practical experiences using it? What are its limitations and drawbacks? Do you prefer AtomPub as your REST style or do you have some other favourite one? And why?
I know, these are many questions, not just one. The thing I'm interested here in is simple, though - how did the APP standard hit the market and particularly how does it seem with its adoption among web developers?
The company that I work for, is developing a lot of RESTful services.
However none of them expose public APIs.(In the sense that all services are internally consumed by our own clients). The reason why we went for REST architectural style was that we wanted our services to be easily consumable and more importantly scale well.
From my own practical experience I have come to the conclusion that HTTP + ATOM syndication format is a good idea, provided you want to keep things flexible(In terms of different content model, attaching and extending meta data associated with payloads, uniform parsing etc). ATOM ensures that everybody interprets the payload in an uniform manner without any scope for ambiguity.
However if one does not have any such complex requirements or does not forsee such requirements then the ATOM format could be a bit of an overhead. (For instance elements like Author,Title etc make sense more in the blogging/RSS world and may not make sense in your particular problem domain).
Also if the goal is to just serialize data structures at one end and reconstruct it at the other end, then most web frameworks(like WCF) have custom formats which are more appealing.
So in my opinion ATOM Pub is good if you need flexiblity in terms of data representation and if the playing field is huge with different kind of client.
However if you have a good knowledge of potential clients and server/client usage patterns then custom formats might be a good idea.
If the client is browser based then formats like JSON are very appealing.
Hope this answers your question.
My own research so far:
Wordpress supports AtomPub as its API protocol since version 2.3
GData is probably the biggest shot in the AtomPub field so far
Habari - new promising blogging system promotes APP as one of its main features
BlogSvc.net - an AtomPub
server, blog engine for .NET
platform, written in C#
Jangle - an open source project
designed to facilitate API access to
Library Systems
There's also mod_atom - an Apache module that stores entries in the filesystem.
Last time I checked (2007 or so) Atompub was fairly complex to implement. While you can whip together something that emits valid Atom feeds during the lunch break, implementing AtomPub was a fairly big undertaking.
That might have changed due to better libraries and tools but still it might be too complex to be implemented by smaller sides just because it's cool.
And the lack of killer AtomPub client applications puts little or no pressure on server operators to offer an AtomPub interface.