Different services version could support different features.
For example version 1 of service not support feature x, but version 2 support feature x.
Is there are some standard or practical way to inform which features are supported in current service version?
I think to some way that information could be obtained from commit descriptions. Is there some standard to obtain that information? Or should every service inform about it in own way.
Or should every service inform about it in own way.
Don't do this. This will couple your service with others. Others shouldn't know how to handle your version, but you should know how to handle different versions.
If you use REST and synchronous communication between microservices, you have to look at REST API Versioning or here. This type of versioning allows you to have multiple versions running in the same infrastructure. Then, each microservice should know how to call a specific version of a microservice.
If you have asynchronous communication with an Event Bus, then your Emitter should send backward-compatible events, to not break other microservices.
Related
I am working on an integration project where we want to use JIRA tickets for business follow up operations. The JIRA (externally hosted) is not always available hence I want to use some Guaranteed delivery patterns. So the question, is it possible WSO2 ESB to use existing connectors (JIRA) in the message processor?
Message processors and connectors are independent. This is what you have to do (you are in right track at the moment too).
Put your message to a message store. This can be the in-memory message store (which looses messages upon a server restart) or a persistent message store such as an activemq queue.
Then, configure a message processor to consume messages from this store. There are two types of message processors namely forwarding and sampling processors. Here you need a sampling processor.https://docs.wso2.com/display/ESB490/Message+Processors
These consumed messages can be handed over to a sequence where the sequence can use the jira connector to create the jira.
Problem I see with this approach is, sampling processors do not support guaranteed delivery (but the forwarding processor do). But, AFAIK, we cannot use connectors with forwarding processors because we need to provide an endpoint in the forwarding processors configs.
You will understand the difference and the pros and cons of two types when you go through the docs. As a workaround, I can suggest following.
Create a proxy service which uses jira connector to create the jira
Then use the forwarding processor to send the consumed message to that proxy service.
I think, with above approach, you will be able to achieve guaranteed delivery.
I would like to create a web service in .Net that clients of different types (Web, exe, java) can consume despite of the language they are written with.
In addition, it needs to support callbacks and be able to easily pass through firewalls and NATs (knowing a client internal IP might change, or be removed from NAT).
Thirdly, since it is an enterprise product, I want to avoid being dependent on 3rd parties, especially ones that demand a certain environment or that customer will not want.
What kind of technologies or approaches can I use?
I am looking at web sockets, but there also I see a lot of complexities and I am not sure there aren't a lot of topology and interoperability border cases that may make me unreliable.
Thanks
For simple request-response services, you can use REST (over HTTP). Any client technology can access HTTP at this point (even CLI) and REST is a well-known and well-understood distributed mechanism. The issue involves the callbacks. There are frameworks that handle HTTP callbacks (simple google search will give you good answers), but imo, the solutions that I have seen are clumsy.
Unlike normal HTTP, WebSocket is a persistent connection. And like any other IETF and W3C specification (or any other standard for that matter), there are various implementations with various degrees of reliability, performance, etc. There are probably about 100 implementations of WebSocket clients and servers. Some implementations handle real-world issues like reconnections, network intermediaries, high scalability, mobile capabilities, etc... and some implementations just do not. I would suggest you pick an implementation that provides these enterprise-grade features.
Btw, WebSocket is pretty darn simple
Lets say I have web applicatons/services:
API
Set of Applications
API is used for managing some resources (simple CRUD operations). Now what I need is to subscribe Applications for changes of different API resources. Applications would do some background work on a change.
I came up to idea of callbacks. So that Applications can oauthorise and post to the API a callback config.
I think that this config should look like this:
{
'callback_url': 'http://3rdpartyservice.com/callback',
'resources': ['foo1', 'foo2'],
'ref_data': { 'token': 'abcd1234' }
}
resources is array of the resources that 3rd party service is interested in
ref_data is custom json for 3rd party usage (e.g. for auth)
This way on specified resource change the API would send a request to callback_url. This request would contain resource data, action(create/update/delete) and ref_data.
The intention here is to make this generic enough to allow 3rd party clients configure such callbacks.
So the question are:
Are there any best practices?
What about security potential issues?
Are there any real world examples on the web?
Tx
Sounds very similar as WebHooks or Service Hooks.
Check out the Web Hooks on GitHub, to get a good idea what they are and how they work. See also last alinea Service Hooks, as it explains how github handles these WebHooks. This would be similar for your application. The OAuth explains why and how it is done.
See also Webhooks, REST and the Open Web, from API User Experience.
There is even RestHooks.
The general solution to this requirement is usually called "publish/subscribe". There are dozens of solutions to this - google "publish subscribe REST" for some examples. You can also read "Enterprise Integration Patterns".
They key challenge in this kind of solution is "real-time versus queue".
For instance, if you have an API with a million clients, who are all interested in the same event, you cannot guarantee that in real time you can reach all of those clients within whatever timeframe their application demands. You also have to worry about the network going away, or clients being temporarily down. In this case, you application might define an event queue, and clients look in that queue for events they're interested in. Once you go down that route, you're probably going to use some off-the-shelf software rather than building your own. Apache Camel is a good open source implementation.
In your example, for instance, what happens if you cannot reach 3rdpartyservice.com? Or if http://3rdpartyservice.com/callback throws an error when posting an update to foo1, but not to foo2? Or if http://3rdpartyservice.com/ uses a different flavour of OAuth than you're used to? How do you guarantee http://3rdpartyservice.com/ that it's you who is posting an update, not a hacker?
Your choices really tend to come down to your non-functional requirements, rather than the functional ones - things like uptime, guarantee of notification, guarantee of delivery, etc. are more important than the specifics of how you pass across the parameters, and whether it's "resource-based" or some other protocol.
In developing backend components, I need to decide how these components will interact and communicate with each other. In particular, I need to decide whether it is better to use (RESTful, micro) web services versus a message broker (e.g. RabbitMQ). Are there certain criteria to help decide between using web services for each component versus messaging?
Eranda covered some of this in his answer, but I think three of the key drivers are:
Are you modeling a Request-Response type interaction?
Can your interaction be asynchronous?
How much knowledge does the sender of the information need to have about the recipients?
It is possible to do Request-Response type interactions with an asynchronous messaging infrastructure but it adds significantly to the complexity, so generally Request-Response type interactions (i.e. does the sender need some data returned from the recipient) are more easily modeled as RPC/REST interactions.
If your interaction can be asynchronous then it is possible to implement this using a REST interaction but it may scale better if you use a fire and forget messaging type interaction.
An asynchronous messaging interaction will also be much more appropriate if the provider of the information doesn't care who is consuming the information. An information provider could be publishing information and new consumers of that information could be added to the system later without having to change the provider.
Web server and message broker have their own use cases. Web server used to host web services and the message broker are use to exchange messages between two points. If you need to deploy a web service then you have to use a web server, where you can process that message and send back a response. Now let's think that you need to have publisher/subscriber pattern or/and reliable messaging between any two nodes, between two servers, between client and server, or server and client, that's where the message broker comes into the picture where you can use a message broker in the middle of two nodes to achieve it. Using message broker gives you the reliability but you have to pay it with the performance. So the components you should use depends on your use case though there are multiple options available.
I'm working on a real time application implemented using in a SOA-style (read loosely coupled components connected via some messaging protocol - JMS, MQ or HTTP).
The architect who designed this system opted to use JMS to connect the components. This system is real time so there no need to queue up messages should one component fail (the transaction will simply time out). Further, there is no need for guaranteed delivery or rollback.
In this instance, is there any benefit to using JMS over something like an HTTP web service (speed, resource footprint, etc)?
One thing that I'm thinking is since the JMS approach requires us to set a thread pool size (the number of components listening to a JMS topic/queue), wouldn't a HTTP service be a better fit since this additional configuration is not needed (a new thread is created for each HTTP request making the application scalable to an "unlimited" number of requests until the server runs out of resources).
Am I missing something?
I don't disagree with the points made by S.Lott at all, but here are a couple of points to consider regarding HTTP web services:
Your clients only need to know how to communicate via HTTP - a protocol well supported by just about every modern langauge in one form or another. JMS, though popular, is more specialist than HTTP, and so restricts the languages your interconnected systems can use. Perhaps not an issue for your system at the moment, but will you need to plug in other systems later that might struggle to support JMS connectivity?
Standards like WSDL and SOAP which you could levarage for your services are well supported by many langauges and there are plenty of tools around that will generate code to implement both ends of the pipeline (client and server) for you from a WSDL file, reducing the amount of dev you'll have to do. These standards also make it relatively simple to define and publish the specification of the data you'll be passing between your systems, something you'll presumably have to do by hand using a queueing technology like JMS.
On the downside, as pointed out by S.Lott, JMS gives you functionality that you throw away using the (stateless) HTTP protocol: guaranteed ordering & reliability; monitoring; scalability; etc. Are you sure you don't need these, and won't need these going forward?
Great question, btw.
I think it's really dependent on the situation. Where I work, we support Remoting, JMS, MQ, HTTP, and sFTP. We are implementing a middleware appliance that speaks Remoting, JMS, MQ, and HTTP, and a software middleware component that speaks JMS, MQ, and HTTP.
As sgreeve alluded to above, standards help us become flexible, but proprietary formats allow more functionality.
In a nutshell, I'd say use HTTP for stateless calls (which could end up meeting almost all of your needs), and whatever proprietary formats you need for stateful calls. If you work in a big enterprise, a hardware appliance is usually a great fit as middleware: Lightning fast compression, encryption, transformation, and translation, with very low total cost of ownership.
I don't know enough about your requirements, but you may be overlooking Manageability, Flexibility and Performance.
JMS allows you to monitor and manage the queue. These are features HTTP lacks, and you'd have to build rather than buy from a vendor.
Also, There are queues and topics in JMS, allowing multiple subscribers to a single publisher. Not possible in HTTP.
While you may not need those things in release 1.0, you might want them in the future.
Also, JMS may be able to use other transport mechanisms like named sockets, which reduces the overheads if there isn't all that socket negotiation going on with (almost) every request.
If you go down the HTTP route and you want to support more than one machine or some kind of reliability - you are going to need a load balancer capable of discovering the available web servers and loading requests across them - then failing over to another web server if a particular box/process dies. Clients making HTTP requests are also going to have to deal with servers failing and retrying operations in some loop.
This is one of the main features of a message queue - reliable load balancing with failover and loose coupling among the producers and consumers without them having to include retry logic - so your client or server code doesn't have to worry about this kinda thing. This is totally separate to whether or not you want message persistence or want to use ACID transactions to produce/consume messages (which can be very handy BTW).
If you focus just on the server side using Java - whether Servlets or MessageListener/MDBs they are kinda similar either way really. The difference is the load balancer.
So maybe the question should really be - is a JMS broker easier to setup & work with than setting up your DNS/NAT/IP/HTTP load balancer infrastructure?
I suppose it depends on what you mean by real-time... Neither JMS nor HTTP in my opinion support "real-time" applications well, meaning they cannot offer predictable/deterministic performance nor properly prioritize flows in the presence of contention.
Part of it is that these technologies are built on top of TCP which serializes all traffic into a single FIFO meaning that different traffic flows cannot be easily prioritized. Moreover TCP timers are not easily controlled resulting unpredictable blocking and timeouts... For this reason many streaming applications use UDP instead of TCP as an underlying protocol.
Another problem with JMS is that typical implementations use a broker that centralizes message dispatch. This is not the best architecture to get deterministic performance.
If you are looking for a middleware that can offer you the kind of reliability guarantees and publish-subscribe semantics you get with JMS, but was developed to fit the real-time application domain I recommend you take a look at the OMG Data-Distribution Service (DDS). See dds.omg.org and this article I wrote arguing why DDS is the best middleware to implement a real-time SOA. http://soa.sys-con.com/node/467488