Protobuf webservice deserialization in vba - web-services

In my organisation I would like to implement a rest web service with protobuf serialization that seems to me being the best, due to performance constraints. It is a java implementation hosted in a web server.
On client side (that I do not manage and won't develop), we have constraints too: XLS with VBA, not VB.Net (can not change it)
So my questions are:
Does this architecture fit? Specially how to manage the interoperability and the serial/deserial in vb for xls? How to manage the .proto, would you have a sample?
Does it seem possible?
In case it does not work, (or too difficult or time consuming) any idea? (JSON, SOAP) considering this client techno constraint and the fact I would like something very performant/fast?

I am not aware of a non-.NET VB/VBA implementation. You would either have to write one, or choose from the list of existing implementations. Or choose a different serialization technology, but frankly VBA isn't exactly awash with those.
Do you have .NET on the client? If you can use COM interop to call into .NET code then you'll have much more flexibility, including multiple protobuf implementations to choose from.

Related

the difference between owl-s vs wsdl-s in semantic web services?

i am acctually dealing with semantic web services, more specifically WS composition.
to add semantics in web services i find two paths (using owl-s or wsdl-s) ,so what is the limits of each solution?
In my opinion, both OWL-S and WSDL-S are equally obsolete. Both approaches were conceived at a time when heavy-weight service descriptions were thought to be the best way to build a web service architecture. Now, with the possible exception of some in-house corporate development teams, web services API's are generally based on fairly simple HTTP-based approaches, with JSON (usually) or XML (less often) as the payload. They often claim to use REST as a basis for defining the service, though in many cases it's far from clear that the term is being used correctly. By and large, these services are invoked through developers reading API documentation and writing code, rather than some cumbersome semi-automated process involving the processing of a service description language.
I'm not aware of any widely-used technique that can be used to describe the metadata of a modern web service of the form I describe above. There are approaches such as json-service, but I don't know how broadly they are used.
I think the truth is that having a rich service description as a first-class object never proved to be as useful as people expected, or hoped, it was going to be. It is especially true that we never achieved the flexible, dynamic composition of complex services from simpler component services envisaged in service choreography or agent-based web services. I think it's also true to say that describing what service would do upon invocation proved to be a much harder problem than the early research investigations anticipated, and premature attempts at standardisation actually made the problem worse not better.
If you're interested in web service composition, I strongly recommend you read some of the extensive research literature to get a sense of the scale of the effort that tried - and failed - to solve a problem that developers actually want to solve. Then ask yourself what problem for your users you are trying to address. Don't just invent some technology and then try to figure out what it might be useful for - that approach has been tried!
SWS have still long way to go.
WSDL 2.0: RDF mapping W3C specification defines WSDL 2.0 - OWL mapping.
The main problem is not how to describe the services in ontology, but what to do with type system.
Linked Data is based on DL model (OWL2) which is quite different from XML Schema message descriptions. OWL 1/2 partially adopt XML datatypes, but you can't just use OWL2/XML serialization.
One can use OWL/RDF messages directly (passing individuals with correspondent data), but there is no standard way to define this pattern in WSDL.
The XML Schema is another brake here - it doesn't support multiple inheritance, so mapping between OWL / XML is not that easy (and to be fair in XML there no concept of inheritance at all).
IMHO the solution would be to create a new XML Schema language, keeping the type system, but changing a structural specification to something "Linked Data friendly".
Then, provide new OWL serialization format based on say "XML Schema 2.0" specification, where all datatypes and individuals will be directly represented, so may be structurally represented in XML.
Obviously there could be a lot of questions - how to represent OWL IRI in XML Schema QNames?
Kind regards,
D.

With what shall we replace the DCOM communication with?

We currently have a number of C++/MFC applications that communicate with each other via DCOM. Now we will update the applications and also want to replace DCOM with something more modern, something that is easier to work with. But we do not know what. What do you think
Edit
The data exchanged is not something that may be of interest to others. It is only status information between the different parts of the program running on different computers.
there are many C++ messaging libraries, from the old ACE to new ones like Google's Protocol Buffers or Facebook's (now Apache's) Thrift or Cisco's Etch.
Currently I'm hearing good things about ZeroMq which might give you more than you are used to.
DCOM is nothing more than sugar-coating over a messenging system.
Any proper messenging system would do, and would allow you to actually spot where messages are exchanged (which may be important to localize point of failures/performance bottlenecks in waiting).
There are two typical ways to do so, nowadays:
A pure messenging system, for example using Google Protocol Buffers as the exchange format
A webservice (either full webservice in JSON or a REST API)
I've been doing lots of apps in both C++ and Java using REST and I'm pretty satisfied. Far from the complexity of CORBA and SOAP, REST is easy to implement and flexible. I had a bit of a learning curve to ged used to model things as CRUD, but now it seems even more intuitive that way.
Now, for the C++ side I don't use a specific REST library, just cURL and a XML parser (in my case, CPPDOM) because the C++ apps are only clients, and the servers are Java (using the Restlet framework). If you need one, there's another question here at SO that recommends:
Can anyone recommend a good C/C++ RESTful framework
I'd also mention my decision to use XML was arbitrary and I'm seriously considering to replace it with JSON. Unless you have a specific need for XML, JSON is simpler and lightweight. And the beauty of REST is that you could even support both, along with other representations, if you want to.

Network programming: SOAP vs DIY marshalling with XML library?

I know that there are a lot of discussions already on SO about SOAP, bloat, XML, and alternative mechanisms like REST.
Here's the situation. A new team member is really talking up SOAP based upon the difficulty of implementing protocols by hand. He recommends gSOAP (project is all in C++.) He is stating things like WSDL cleaning up lots of messy hand coded C++.
Right now I am handling most networking with XML based text messages, and the expat XML library. So I have some programming effort (not much) associated with modifications to message formats or additions to parameter lists. At the sender end I package up an XML request and send it over a plain old TCP socket. At the receiver I parse the XML using DOM or SAX. Etc. It has worked fine so far. The XML messages are quite compact and average a couple of hundred characters, at most. And I understand every item in those messages.
We want one portion of the product (a server) to be accessible to web sites that are coded using PHP. That is partly driving this idea here, that a SOAP interface will be "easier" for script writers. Everyone on this project believes that SOAP is their salvation.
I see the introduction of a new large library like gSOAP as highly disruptive to the momentum of a mature project.
What I am wondering is if there is a different and more compact way of doing what SOAP gives us. And how to balance claims of gSOAP or other SOAP tools making development life easier against hard reality.
IE, I am being told that WSDL is better, easier, more workmanlike, etc than hand coding C++ using an XML library. That it puts the semantics of the C++ objects right into the declaration of the network messages. The thing is, many of the XML messages that I have defined don't map one for one to a single distinct object at the receiving end.
Or, it is possible that I am worrying about nothing.
But the reality as I scan messages here seems to contradict what I have been told locally.
I'm not buying SOAP.
Don Box's original vision of a Simple Object Access Protocol is anything but simple now. It's become a bloated, design by committee mess.
Throw in all the additional dependencies on bloated libraries and you have a potential mess on your hands.
Tool vendors love SOAP, but I don't see much for anyone else.
I think that you will find that PHP developers are more likely to prefer RESTful interfaces. Here is a 2003 article about it.
http://onlamp.com/pub/a/php/2003/10/30/amazon_rest.html
RESTful interfaces are a growing phenomenon and if you need to attract developers to your platform it will be easier if you catch the wave.
Having said that, is there a good reason why you cannot support multiple interfaces? This is fairly common in web services that do not have a captive audience. You could support your legacy model, a clean RESTful model and a SOAP/WSDL model. Then take stock after 6 months to a year to see which model is the most popular and least effort to support.
When it comes to making the site more accessible to outsiders, REST has more widespread usage. As far as saving your project, it is possible that SOAP would do this because it demands a certain amount of rigor in interface design, however the same could be said of REST. If this is a key criterion, then you should probably abandon the hand-coded XML and go with a high-level interface design that could be implemented as both REST and SOAP.
I know some people believe that SOAP and REST are fundamentally different approaches, but if you take a RESTful approach to the interface design, you shouldn't have great difficulty in creating a SOAP version. Don't try to do it the other way around though.
Here is a classic, hilarious, debunking of SOAP - The "S" stands for Simple". The community I move in is completely converted to REST.
If you look around at RESTful interfaces on the net, you'll notice that SOAP is nearly universally avoided. SOAP is such a complex beast that it effectively locks out languages with no existing SOAP package, since nobody is going to implement it themselves. Raw XML, on the other hand, is pretty universal at this point, and not that difficult to implement in-house if necessary.

WSDL vs REST Pros and Cons

Related:
Why would one use REST instead of Web services?
When deciding whether to implement a web service using SOAP or REST (by which I mean HTTP/XML in a RESTful manner) what should I be aware of and what should I be thinking of? I presume that this isn't a one size fits all thing so how do I choose which to use.
The two protocols have very different uses in the real world.
SOAP(using WSDL) is a heavy-weight XML standard that is centered around document passing. The advantage with this is that your requests and responses can be very well structured, and can even use a DTD. The downside is it is XML, and is very verbose. However, this is good if two parties need to have a strict contract(say for inter-bank communication). SOAP also lets you layer things like WS-Security on your documents. SOAP is generally transport-agnostic, meaning you don't necessarily need to use HTTP.
REST is very lightweight, and relies upon the HTTP standard to do it's work. It is great to get a useful web service up and running quickly. If you don't need a strict
API definition, this is the way to go. Most web services fall into this category. You can version your API so that updates to the API do not break it for people using old versions(as long as they specify a version). REST essentially requires HTTP, and is format-agnostic(meaning you can use XML, JSON, HTML, whatever).
Generally I use REST, because I don't need fancy WS-* features. SOAP is good though if you want computers to understand your webservice using a WSDL. REST specifications are generally human-readable only.
The following links provide useful information about WSDL vs REST including Pros and Cons
A couple of key points are that
1) SOAP was designed for a distributed computing environment where as REST was designed for a point to point environment.
2) WADL can be used to define the interface for REST services.
http://www.ajaxonomy.com/2008/xml/web-services-part-1-soap-vs-rest
http://ajaxonomy.com/2008/xml/web-services-part-2-wsdl-and-wadl
Regarding WSDL (meaning "SOAP") as being "heavy-weight". Heavy matters how? If the toolset is doing all the "heavy lifting" for you, then why does it matter?
I have never yet needed to consume a complicated REST API. When I do, I expect I'll wish for a WSDL, which my tools will gladly convert into a set of proxy classes, so I can just call what appear to be methods. Instead, I suspect that in order to consume a non-trivial REST-based API, it will be necessary to write by hand a substantial amount of "light-weight" code.
Even when that's all done, you still will have translated human-readable documentation into code, with all the attendant risk that the humans read it wrong. Since WSDL is a machine-readable description of the service, it's much harder to "read it wrong".
Just a note: since this post, I have had the opportunity to work with a moderately complicated REST service. I did, indeed, wish for a WSDL or the equivalent, and I did, indeed, have to write a lot of code by hand. In fact, a substantial part of the development time was spent removing the code duplication of all the code that called different service operations "by hand".
This probably really belongs as comments in several of the above posts, but I don't yet have the rep to do that, so here goes.
I think it is interesting that a lot of the pros and cons often cited for SOAP and REST have (IMO) very little to do with the actual values or limits of the two technologies. Probably the most cited pro for REST is that it is "light-weight" or tends to be more "human readable". At one level this is certainly true, REST does have a lower barrier to entry - there is less required structure than SOAP (though I agree with those who have said that good tooling is largely the answer here - too bad much of the SOAP tooling is pretty dreadful).
Beyond that initial entry cost however, I think the REST impression comes from a combination of the form of the request URLs and the complexity of the data exchanged by most REST services. REST tends to encourage simpler, more human readable request URLs and the data tends to be more digestable as well. To what extent however are these inherent to REST and to what extent are they merely accidental. The simpler URL structure is a direct result of the architecture - but it could be equally well applied to SOAP based services. The more digestable data is more likely to be a result of the lack of any defined structure. This means you'd better keep your data formats simple or you are going to be in for a lot of work. So here SOAP's additional structure, which should be a benefit is actually enabling sloppy design and that sloppy design then gets used as a dig against the technology.
So for use in the exchange of structured data between computer systems I'm not sure that REST is inherently better than SOAP (or visa-versa), they are just different. I think the comparison above of REST vs SOAP to dynamic vs. static typing is a good one. Where dyanmic languages tend to run in to trouble is in long term maintenance and upkeep of a system (and by long term I'm not talking a year or 2, I'm talking 5 or 10). It will be interesting to see if REST runs into the same challenges over time. I tend to think it will so if I were building a distributed, information processing system I would gravitate to SOAP as the communication mechanism (also because of the tranmission and application protocol layering and flexibility that it affords as has been mentioned above).
In other places though REST seems more appropriate. AJAX between the client and its server (regardless of payload) is one major example. I don't have much care for the longevity of this type of connection and ease of use and flexibility are at a premimum. Similarly if I needed quick access to some external service and I didn't think I was going to care about the maintainability of the interaction over time (again I'm assuming this is where REST is going to end up costing me more, one way or another), then I might choose REST just so I could get in and out quickly.
Anyway, they are both viable technologies and depending on what tradeoffs you want to make for a given application they can serve you well (or poorly).
REST is not a protocol; It's an architectural style. Or a paradigm if you want. That means that it's a lot looser defined that SOAP is. For basic CRUD, you can lean on standard protocols such as Atompub, but for most services you'll have more commands than just that.
As a consumer, SOAP can be a blessing or a curse, depending on the language support. Since SOAP is very much modelled on a strictly typed system, it works best with statically typed languages. For a dynamic language it can easily become crufty and superfluous. In addition, the client-library support isn't that good outside the world of Java and .NET
To me we should be careful when we use the word web service. We should all the time specify if we are speaking of SOAP web service, REST web service or other kind of web services because we are speaking about different things here and people don't understand anymore if we named all of them web services.
Basically SOAP web services are very well established for years and they follow a strict specification that describe how to communicate with them based on the SOAP specification.
Now REST web services are a bit newer and basically looks like simpler because they are not using any communication protocol. Basically what you send and receive when you use a REST web service is plain XML. People like it because they can parse the xml the way they want without having to deal with a more sophisticated communication protocol like SOAP.
To me REST services are almost like if you would create a servlet instead of a SOAP web service. The servlet get data in and return data out. The format of the data are xml based. We can also imagine to use something else than xml if we want. For instance tags could be used instead of xml and that would be not REST anymore but something else (Could be even lighter in term of weight because xml is not light by nature). Would we call that still a web service? Yes we could but that will not follow any current standard and this is the main issue here if we start to call everything web services but we can do it the way we want then we are loosing on the interoperability side of the things. That means that the format of the data that is exchanged with the web service is not standardized anymore. That requires then that server and client agree on the format of the data whereas with SOAP this is all predefined already and server and client can interoperate without to know each other because they follow the same standard.
What people don't like with SOAP is that they have hard time to understand it and they cannot generate the queries manually. Computers can do that very well however so this is where we need to be clear: are web services queries and response supposed to be used directly by the end users or do we agree that web services are underneath API called by computer systems based on some normalized standards?
SOAP: It can be transported via SMTP also, means we can invoke the service using Email simple text format also
It needs additional framework/engine should be in web service consumer machine to convert SOAP message to respective objects structure in various languages.
REST: Now WSDL2.0 supports to describe REST web service also
We can use when you want to make your service as lightweight, example calling from mobile devices like cell phone, pda etc...
for enterprise systems in which your system is confined within your corporations, its easier and proper to use soap because you are almost in control of clients. it's easier since there a variety of tools which creates classes (proxies) and looks like you are doing your regular OOP which matches your java or .net environment (in which most corporates use).
I would use REST for internet facing applications for exposing interfaces (like twitter api) since clients can be using javascripts or html or others in which typing is not strict. REST being more liberal makes more sense.
Also for internet facing clients (world wide web), its easier to parse json or xml coming out of a rest interface rather than a purely xml coming from a soap interface. it's hard to use proxies on javascript and javascript does not naturally support objects. If you are using REST with javascript, you would just usually parse the json string and you're off. internet facing interfaces are usually very simple (so most of the time its simple parsing) and does not usually demand consistency that is why REST is adequate enough.
For enterprise applications I don't think REST is adequate because transactions, security, strict typing, schemas play a very important in enterprise applications development that is why SOAP is more suited for them.
My conclusion is that SOAP is for Enterprise systems, REST is for the Internet or WWW.
You can use it interchangeably but you may find yourself having a difficult time eventually not using the correct tool for the job.
sorry for my bad english.
In defence of REST it closely follows the principles of HTTP and addressability e.g. read operations use GET, update operations use POST etc. I find this to be a far cleaner approach. The Oreilly book RESTful Web Services explains this far better than I can, if you read it I think you would prefer the REST approach
The toolset on the client side would be one. And the familiarity with SOAP services the other. More and more services are going the RESTful route these days, and testing such services can be done with simple cURL examples.
Although, it's not all that difficult to implement both methods and allow for the widest utilization from clients.
If you need to pick one, I'd suggest REST, it's easier.
The previous answers contain a lot of information, but I think there is a philosophical difference that hasn't been pointed out. SOAP was the answer to "how to we create a modern, object-oriented, platform and protocol independent successor to RPC?". REST developed from the question, "how to we take the insights that made HTTP so successful for the web, and use them for distributed computing?"
SOAP is a about giving you tools to make distributed programming look like ... programming. REST tries to impose a style to simplify distributed interfaces, so that distributed resources can refer to each other like distributed html pages can refer to each other. One way it does that is attempt to (mostly) restrict operations to "CRUD" on resources (create, read, update, delete).
REST is still young -- although it is oriented towards "human readable" services, it doesn't rule out introspection services, etc. or automatic creation of proxies. However, these have not been standardized (as I write). SOAP gives you these things, but (IMHO) gives you "only" these things, whereas the style imposed by REST is already encouraging the spread of web services because of its simplicity. I would myself encourage newbie service providers to choose REST unless there are specific SOAP-provided features they need to use.
In my opinion, then, if you are implementing a "greenfield" API, and don't know that much about possible clients, I would choose REST as the style it encourages tends to help make interfaces comprehensible, and easy to develop to. If you know a lot about client and server, and there are specific SOAP tools that will make life easy for both, then I wouldn't be religious about REST, though.
You can easily transition your WSDL-spewing WCF web components to other uses just by changing your configuration settings. You can go across HTTP and then also named pipes, tcp, custom protocols, etc without having to change your code. I believe WCF components may also be easier to set up for stuff like security, two-way calling, transactions, concurrency, etc.
REST pretty much limits you to HTTP (which is fine in many cases).
I know that this discussion is an old one, but after reading all the answers and commented, I believe that everyone missed the most important point about the difference between the 2 systems: SOAP uses complex types to not only give you the data, but validate it and keep it in the strict type designation it was defined for. A WSDL tells you what the data format is, what the data type is, allows you to add reg-ex pattern-style rules, and defines how many times a piece of data must be, and may be, allowed in a request/response.
Rest on the other-hand has none of these mechanisms.
SOAP is complex and heavy because it allows you to send complex heavy hierarchical data. REST is plain text, with the origin and endpoint sorting out the rules.
SOAP is business independent, because it has all the data rules embedded in the document.
The difference between SOAP and REST is that SOAP is a self-contained business oriented schema. REST is a text document.

Web Services or Custom Protocol?

I have no experience with web services. Historically I've built client-server systems using proprietary communication protocols (even they happen to be XML). I just spent a few hours looking over Axis2 and it sent a shudder down my spine. The learning curve of WS scares me, and seeing all that XML surround so little functionality makes me wonder if it's worth the trouble.
How do you decide whether you need to use Web Services or a custom communication protocol? What are the advantages/disadvantages of each approach and what use-cases are they best suited for?
Please post a clear guideline, not an opinion piece :)
Build RESTful web APIs; then you get a lot of automatic caching and etc benefits that you don't get if you use other methods (SOAP, XML-RPC, etc)
See this post for more details
Another benefit is that if you build a RESTful API for your code to use, you can potentially let your users take advantage of it too - they often have uses for your product that you never dreamed of.
"Web Services" as defined by the W3C means using SOAP over HTTP. SOAP is severe overkill in most cases; it's only really appropriate (IMO) when you're making a public service available to the world, like an API for interacting with your website, for example.
Anything else (especially internal, private communications) rarely need anything more complex than XML-RPC. Only if performance is an issue should you consider a more condensed protocol; XML-RPC is so simple and widely-supported that the ease of development and debugging more than makes up for the performance loss of using bloaty ol' XML.
Remember that there are a number of frameworks out there that make programming web services very trivial stuff. In the VB / C# world .Net makes it a joy. I'm not really sure about specific frameworks for other languages but I am sure most have at least one.
The standardisation and simplicity of implementation and reuse of web services make them very attractive. As previously pointed out- yes, they make communications very verbose. If you are worried about this why not calculate how much data you actually will be trasmitting. chances are, with current network and internet speeds, it will be trivial - even with the XML overhead.
I would always use the custom data formats as a last resort and not a first. What widely used method you use it up to you but it's unlikely you would go wrong with Web Services model.
Maintainability and extensibility are the main benefits. The use of widely used technology your solution will be easier for someone else to understand plus you can use ready to roll libraries as consumers and providers.
I have recently broken my custom protocol habit. I am now using Apache on the server side and libCurl plus libxml2 to load and parse the XML on the client which is written in C++.
The server side can be either PHP or a CGI written in a more serious language. Depends what you want to do.
Webservices have the advantage of being somewhat standard, so it's possible for programs you've never heard of to use a webservice you wrote. Using HTTP can help them communicate over proxies and other network obstacles without any extra work from you. The XML, although rather verbose and ugly, is rather easier to read when debugging than binary data.
When you're transferring stuff over the network, it's unlikely that serialisation/deserialisation to xml will be the limiting factor in performance. It can be a bit of hassle, although a library to do it for you will help a lot.
SOAP and XML -- "all that XML surround so little functionality makes me wonder if it's worth the trouble."
Totally. SOAP is heavy-weight, and -- to a large extent -- a workaround to the need for static binding throughout the Java technology stack.
REST, on the other hand, is much lighter weight. Further, REST with JSON or REST with YAML is very lightweight, and very easy to implement. It builds right on top of the off-the shelf HTTP protocol.
REST requires you to define resources (named via URI's), and transactions based on the canonical CRUD rules (GET, POST, PUT and DELETE). Very simple and canonical.
In my personal (old cranky dude) opinion, web services should only be used as a way to make some of your internal information available to third parties (i.e. other companies, people outside your organization etc.). Of course, that is also the originally intended purpose of XML. :-)
If you have access to a direct connection with the databases containing the information your application needs - that is the way to go. It's faster and simpler - which in application development means "better" and "less buggy".