Every description on the Message Bus architecture seems to describe a centralised intermediary for routing messages:
Each application has a single connection to the message bus instead
of multiple dedicated connections to each of the other applications.
http://msdn.microsoft.com/en-us/library/ff647328.aspx
Structure the connecting middleware between these applications as a
Message Bus that enables them to work together using messaging.
http://www.eaipatterns.com/MessageBus.html
Service Bus is messaging infrastructure that sits between applications
allowing them to exchange messages in a loosely coupled way for
improved scale and resiliency.
http://www.windowsazure.com/en-us/home/features/messaging/
The descriptions seem identical to the Message Broker architecture; leading me to believe the following:
The term 'Message Bus' is effectively the same as 'MoM' - a generic term for referring to a messaging infrastructure without explicitly mentioning a topology. I.e. Message Broker is a type of Message Bus.
This seems like a controversial topic, and whilst there's a few similar questions on SO, none have (IMO) definitive answers.
P.s. I am aware that Enterprise Service Bus is entirely different, and is effectively a 'feature rich' MoM.
You are right, this area of terminology is very messy and there are no definitive answer, but this is my view from my professional life:
The first thing is to do is to think about this in different levels of abstractions - architectural concepts vs products.
A message bus (http://www.eaipatterns.com/MessageBus.html) is often referd to a as an architectural princip to decouple applications often together with canonical data formats (http://www.eaipatterns.com/CanonicalDataModel.html), much regardless of technology. Often implemented using MoM products.
A MoM typically refers to a family of middleware products that implements asynchronous message passing. Often built supporting standards such as JMS and AMQP (typically WebSphere MQ, ActiveMQ, Azure messaging, hornetQ, RabbitMQ, etc.etc.).
A message broker is often a MoM, and both terms are used for the same messaging product. There might be confusion here, since the main ESB product of IBM is called WebSphere Message Broker while it rather an ESB used side by side with the MoM product - WebSphere MQ. Usually most MoM call themselves brokers, such as RabbitMQ, ActiveMQ, QPid etc.
Hope this did not cause more confusion :)
The differences are mainly different vendors trying to differentiate themselves in the Market and increase their mind-share. Since vendors typically defined their own terms, there will usually be a slight differences in what each vendor means by the terms they use.
However in general they are the same class or type of system. They organise and manage the transport, queueing and persistence (storage) of messages from one software service to another, beyond that, you need to be looking at vendors specific feature lists for their product.
WebSphere Message Broker belongs to the ESB layer of an SOA and it is also MOM. WebSphere MQ is also MOM. I think a lot of the confusion is around abstract architecture concepts and actual software products. SOA and ESB are abstract concepts. The whole enterprise architecture industry IBM, Microsoft, Oracle, TIBCO etc. are always vying for market share. Software companies often pretend they invented something new and better in order to get attention from their customers. They make a lot of wild promises about their products when in reality they create a whole new set of expensive problems (e.g. XML transformations cause serious performance and security issues). Because it is corporate greed driven and "get to market fast" there is always an abundance of confusion and IT problems and much stress for IT workers.
Related
How do you integrate applications via web services and deal with technical errors like connectivity errors for web service calls which change state?
E.g. when the network connection gets interrupted during a web service call, how does the client know whether the web services has processed its action or not?
Can this issue be solved at the business layer only (e.g. to query a previous call state) or are you aware of some nice frameworks/best practices which can help wrapping transactional guarantees around a web service?
Implementing it all by yourself with some kind of transactional context tracked in the business layer is always an option. You can use some compensation mechanisms to ensure transactions are rolled back if needed, but you'll need to:
have the information on transactions persisted somewhere
use transaction correlation IDs, so you can query when the response has
been lost (having correlation IDs is good idea anyway)
implement the operations needed to read/write/rollback, etc, so it might make your services a bit more complex
Another option I can think of is If you're using SOAP you can go for asynchronous communication and look for some stack implementing WS-Coordination, WS-AtomicTransaction and WS-BusinessActivity specifications, then decide for yourself if it is a good idea in your context or not. For example, I think Axis2 supports these, but of course eventually it depends on technologies and stack you use.
From the article above:
WS-AtomicTransaction defines a coordination type that is most useful
for handling system-generated exceptions, such as an incomplete write
operation or a process terminating abnormally.
Below are the types of 2-Phase Commit that it implements.
Hope this helps!
I am doing some research on SOAP, for a personal project, and I came across a website with a list of pros and cons for using SOAP, and I understood what most of them meant, except for this one under disadvantages:
SOAP is typically limited to pooling, and not event notifications, when leveraging HTTP for transport. What's more, only one client can use the services of one server in typical situations.
From my understanding of pooling, there should be no issue pooling a SOAP Object for re usability. Pooling is simply a way to use the same resources over and over again, like a connection to a database. Also not entirely certain on the context of Event Notifications.
So my two questions here are, what does the above block quoted text actually mean, and is this information correct?
Website: http://searchsoa.techtarget.com/definition/SOAP
SOAP is RPC, and in RPC some local client invokes a method on some remote target and receives a result. That's how it works, so SOAP works that way too. A client invokes a service asking for something and the service just responds.
If you want "events" in this type of communication the most simple approach is to invoke the service more often (i.e. polling). This has the advantage that nothing changes for the server or the client. It's the same RPC call but done more frequently.
These days everyone is connected to the web and everyone is subscribed to all sorts of services. They want to get notified as soon as something happens to the world around them. Pooling becomes inefficient in this sea of users and services because you are wasting resources. You might poll a service a hundred times just to get back one notification. For this reason technology is evolving so that resource use is minimized. And the direction this is moving to is push services.
Now almost everything happens in the browser. Every browser manufacturer rushes to implement the latest technology changes and HTML5 spec. This means actual pages that push notifications to users instead of faking it with Ajax, comet, etc.
SOAP has been around since 1998 and it's not moving as fast as the rest of the web, mainly because SOAP is mostly an enterprise player and because it's a protocol. Because it's a protocol you have to make new technology available to it without breaking that protocol. Things move slower so people have abandoned SOAP in favor of other ways of doing server-client communication.
SOAP is typically limited to pooling, and not event notifications...
That is correct. But be aware that "typically" does not mean "always".
You can have events, but it's harder. It involves using WS-* specifications like WS-Eventing and WS-Addressing. This is a change in the way SOAP clients operate because a client now becomes some sort of a service too because it needs to receive calls too, not just initiate them. If your technology stack implements these specifications then good for you, but if it doesn't, then you have to build it yourself and it's a real pain.
So for these reasons, if you don't have blocking performance or resource usage issues, you "typically" chose doing polling with SOAP and not event notifications.
I would like to create a web service in .Net that clients of different types (Web, exe, java) can consume despite of the language they are written with.
In addition, it needs to support callbacks and be able to easily pass through firewalls and NATs (knowing a client internal IP might change, or be removed from NAT).
Thirdly, since it is an enterprise product, I want to avoid being dependent on 3rd parties, especially ones that demand a certain environment or that customer will not want.
What kind of technologies or approaches can I use?
I am looking at web sockets, but there also I see a lot of complexities and I am not sure there aren't a lot of topology and interoperability border cases that may make me unreliable.
Thanks
For simple request-response services, you can use REST (over HTTP). Any client technology can access HTTP at this point (even CLI) and REST is a well-known and well-understood distributed mechanism. The issue involves the callbacks. There are frameworks that handle HTTP callbacks (simple google search will give you good answers), but imo, the solutions that I have seen are clumsy.
Unlike normal HTTP, WebSocket is a persistent connection. And like any other IETF and W3C specification (or any other standard for that matter), there are various implementations with various degrees of reliability, performance, etc. There are probably about 100 implementations of WebSocket clients and servers. Some implementations handle real-world issues like reconnections, network intermediaries, high scalability, mobile capabilities, etc... and some implementations just do not. I would suggest you pick an implementation that provides these enterprise-grade features.
Btw, WebSocket is pretty darn simple
I have the following situation. Several services provide their functionality mainly via SOAP interfaces. There is one module that wants to consume this functionality for integration into a website. What would be the best practice to do that?
The functionality of the services is subject to change. Therefore, each single function/method should be "reroutable". The web service is probably hosted on a different machine.
Is it reasonable to map all web services to JMS queues (my first idea)? The website module would only talk to JMS then. A router would route all incoming JMS messages to the different web services (or elsewhere).
Or: There could be one dedicated web service, that integrates all functions, to be used exclusively by the web site? The advantage here would be that parameters and return values are typed.
What would you suggest? What could be another, better approach?
If I understood you right, what you're aiming at is providing a homogeneous interface, a coherent API for your webapp module, serving a purpose of a facade for multiple remote interfaces (mainly SOAP ones).
Regarding the JMS approach you've mentioned - it seems reasonable, but:
instead of many queues I'd rather go with a single JMS destination with a Camel content-based router immediately after the queue (or two queues for Request/Reply pattern) That would make things "reroutable" and isolate the web module from service changes.
your requests would be less prone to services-related errors, brief problems in service availability and such, while still retaining an ability to perform Request/Reply style calls (which are common to RPC-ish nature of SOAP).
i'd use one-way style calls wherever they are applicable (for increased reliability and reactivity).
You should not worry about lack of typing, WSDL+SOAP seems to enforce strong typing but that's an illusion driven by auto-generated stubs. You still have to marshal the data back and forth.
Instead of SOAP I'd go with JSON, as it is far cleaner and less redundant than XML (probably faster, but that's usually irrelevant). Jackson is a very efficient JSON library and it's already supported in Camel distributions. JMS ObjectMessage is a big NO (a good article on some of the ObjectMessage pitfalls )
The single-service approach seems to be a good way to separate the web-module from the service layer. It lacks the flexibility and fault-tolerance of the JMS approach but seems tad easier to implement.
If there are many calls that can be concluded in a one-way fashion, I'd say go with JMS and reroute the messages after the queue.
I'm working on a real time application implemented using in a SOA-style (read loosely coupled components connected via some messaging protocol - JMS, MQ or HTTP).
The architect who designed this system opted to use JMS to connect the components. This system is real time so there no need to queue up messages should one component fail (the transaction will simply time out). Further, there is no need for guaranteed delivery or rollback.
In this instance, is there any benefit to using JMS over something like an HTTP web service (speed, resource footprint, etc)?
One thing that I'm thinking is since the JMS approach requires us to set a thread pool size (the number of components listening to a JMS topic/queue), wouldn't a HTTP service be a better fit since this additional configuration is not needed (a new thread is created for each HTTP request making the application scalable to an "unlimited" number of requests until the server runs out of resources).
Am I missing something?
I don't disagree with the points made by S.Lott at all, but here are a couple of points to consider regarding HTTP web services:
Your clients only need to know how to communicate via HTTP - a protocol well supported by just about every modern langauge in one form or another. JMS, though popular, is more specialist than HTTP, and so restricts the languages your interconnected systems can use. Perhaps not an issue for your system at the moment, but will you need to plug in other systems later that might struggle to support JMS connectivity?
Standards like WSDL and SOAP which you could levarage for your services are well supported by many langauges and there are plenty of tools around that will generate code to implement both ends of the pipeline (client and server) for you from a WSDL file, reducing the amount of dev you'll have to do. These standards also make it relatively simple to define and publish the specification of the data you'll be passing between your systems, something you'll presumably have to do by hand using a queueing technology like JMS.
On the downside, as pointed out by S.Lott, JMS gives you functionality that you throw away using the (stateless) HTTP protocol: guaranteed ordering & reliability; monitoring; scalability; etc. Are you sure you don't need these, and won't need these going forward?
Great question, btw.
I think it's really dependent on the situation. Where I work, we support Remoting, JMS, MQ, HTTP, and sFTP. We are implementing a middleware appliance that speaks Remoting, JMS, MQ, and HTTP, and a software middleware component that speaks JMS, MQ, and HTTP.
As sgreeve alluded to above, standards help us become flexible, but proprietary formats allow more functionality.
In a nutshell, I'd say use HTTP for stateless calls (which could end up meeting almost all of your needs), and whatever proprietary formats you need for stateful calls. If you work in a big enterprise, a hardware appliance is usually a great fit as middleware: Lightning fast compression, encryption, transformation, and translation, with very low total cost of ownership.
I don't know enough about your requirements, but you may be overlooking Manageability, Flexibility and Performance.
JMS allows you to monitor and manage the queue. These are features HTTP lacks, and you'd have to build rather than buy from a vendor.
Also, There are queues and topics in JMS, allowing multiple subscribers to a single publisher. Not possible in HTTP.
While you may not need those things in release 1.0, you might want them in the future.
Also, JMS may be able to use other transport mechanisms like named sockets, which reduces the overheads if there isn't all that socket negotiation going on with (almost) every request.
If you go down the HTTP route and you want to support more than one machine or some kind of reliability - you are going to need a load balancer capable of discovering the available web servers and loading requests across them - then failing over to another web server if a particular box/process dies. Clients making HTTP requests are also going to have to deal with servers failing and retrying operations in some loop.
This is one of the main features of a message queue - reliable load balancing with failover and loose coupling among the producers and consumers without them having to include retry logic - so your client or server code doesn't have to worry about this kinda thing. This is totally separate to whether or not you want message persistence or want to use ACID transactions to produce/consume messages (which can be very handy BTW).
If you focus just on the server side using Java - whether Servlets or MessageListener/MDBs they are kinda similar either way really. The difference is the load balancer.
So maybe the question should really be - is a JMS broker easier to setup & work with than setting up your DNS/NAT/IP/HTTP load balancer infrastructure?
I suppose it depends on what you mean by real-time... Neither JMS nor HTTP in my opinion support "real-time" applications well, meaning they cannot offer predictable/deterministic performance nor properly prioritize flows in the presence of contention.
Part of it is that these technologies are built on top of TCP which serializes all traffic into a single FIFO meaning that different traffic flows cannot be easily prioritized. Moreover TCP timers are not easily controlled resulting unpredictable blocking and timeouts... For this reason many streaming applications use UDP instead of TCP as an underlying protocol.
Another problem with JMS is that typical implementations use a broker that centralizes message dispatch. This is not the best architecture to get deterministic performance.
If you are looking for a middleware that can offer you the kind of reliability guarantees and publish-subscribe semantics you get with JMS, but was developed to fit the real-time application domain I recommend you take a look at the OMG Data-Distribution Service (DDS). See dds.omg.org and this article I wrote arguing why DDS is the best middleware to implement a real-time SOA. http://soa.sys-con.com/node/467488