Being a beginner, how should I go about deciding if a particular process has to implemented as ESB or as BPEL ?
What are the various parameters that one should use for deciding if either should be used for implementation?
First of all ESB is just a concept while BPEL is an OASIS standard based on XML and Web Services. A BPEL file is actually XML.
You use an ESB when you need to connect 2 or more applications together, to avoid direct point-to-point integration. This offers various benefits, such as translating messages from one format to another, or introducing other message exchange patterns. An ESB's communication is typically stateless, i.e. a message goes through, gets routed to its destination(s), and it ends there. An ESB is a very broad term, interpreted and misinterpreted by vendors to market their products.
A Business Process Management system implementing BPEL and similar technologies on the other hand are concerned with keeping track of the progress of various activities and their relationship. A BPEL process is very similar to a flow chart. A BPEL process preserves state, keeps track of its progress and flow, and is typically used (although not necessarily) in long-winded transactions which could also involve manual human tasks.
A textbook example of a BPEL process is a loan processing application. A request for a customer loan comes in, and the process first performs some automated checks using web service calls on some systems and if the credit rating is too low, the system informs a manager to evaluate the form manually (via some workflow system). The process then waits for a callback from the human workflow system, uses some correlation method (some ID) to match it with the right BPEL process instance (so that the right customer is serviced), and resumes the process accordingly.
ESB from my experience are always for processes that do not include a wait state. When you are just going through a list of services and will get to point a to point b without any pause states, I would use an ESB. ESBs also can handle higher quantities of message requests.
Any time human interaction is involved(Entering values, Review submission), I lean towards implementing this in a BPM. These tend to have more robust handling of long periods of waiting.
There are several questions you need to ask yourself when making the choice between ESB and BPEL. Among the most important:
- am I dealing with a stateless process (then I choose ESB) or a stateful one (so I choose BPEL)
- do I need to handle a large volume of short messages - in this case I choose ESB
- do I need orchestration of business processes - then I use BPEL
Here you have a good resource for your question:
http://www.ibm.com/developerworks/websphere/library/techarticles/0803_fasbinder2/0803_fasbinder2.html
Related
How do you integrate applications via web services and deal with technical errors like connectivity errors for web service calls which change state?
E.g. when the network connection gets interrupted during a web service call, how does the client know whether the web services has processed its action or not?
Can this issue be solved at the business layer only (e.g. to query a previous call state) or are you aware of some nice frameworks/best practices which can help wrapping transactional guarantees around a web service?
Implementing it all by yourself with some kind of transactional context tracked in the business layer is always an option. You can use some compensation mechanisms to ensure transactions are rolled back if needed, but you'll need to:
have the information on transactions persisted somewhere
use transaction correlation IDs, so you can query when the response has
been lost (having correlation IDs is good idea anyway)
implement the operations needed to read/write/rollback, etc, so it might make your services a bit more complex
Another option I can think of is If you're using SOAP you can go for asynchronous communication and look for some stack implementing WS-Coordination, WS-AtomicTransaction and WS-BusinessActivity specifications, then decide for yourself if it is a good idea in your context or not. For example, I think Axis2 supports these, but of course eventually it depends on technologies and stack you use.
From the article above:
WS-AtomicTransaction defines a coordination type that is most useful
for handling system-generated exceptions, such as an incomplete write
operation or a process terminating abnormally.
Below are the types of 2-Phase Commit that it implements.
Hope this helps!
I am doing some research on SOAP, for a personal project, and I came across a website with a list of pros and cons for using SOAP, and I understood what most of them meant, except for this one under disadvantages:
SOAP is typically limited to pooling, and not event notifications, when leveraging HTTP for transport. What's more, only one client can use the services of one server in typical situations.
From my understanding of pooling, there should be no issue pooling a SOAP Object for re usability. Pooling is simply a way to use the same resources over and over again, like a connection to a database. Also not entirely certain on the context of Event Notifications.
So my two questions here are, what does the above block quoted text actually mean, and is this information correct?
Website: http://searchsoa.techtarget.com/definition/SOAP
SOAP is RPC, and in RPC some local client invokes a method on some remote target and receives a result. That's how it works, so SOAP works that way too. A client invokes a service asking for something and the service just responds.
If you want "events" in this type of communication the most simple approach is to invoke the service more often (i.e. polling). This has the advantage that nothing changes for the server or the client. It's the same RPC call but done more frequently.
These days everyone is connected to the web and everyone is subscribed to all sorts of services. They want to get notified as soon as something happens to the world around them. Pooling becomes inefficient in this sea of users and services because you are wasting resources. You might poll a service a hundred times just to get back one notification. For this reason technology is evolving so that resource use is minimized. And the direction this is moving to is push services.
Now almost everything happens in the browser. Every browser manufacturer rushes to implement the latest technology changes and HTML5 spec. This means actual pages that push notifications to users instead of faking it with Ajax, comet, etc.
SOAP has been around since 1998 and it's not moving as fast as the rest of the web, mainly because SOAP is mostly an enterprise player and because it's a protocol. Because it's a protocol you have to make new technology available to it without breaking that protocol. Things move slower so people have abandoned SOAP in favor of other ways of doing server-client communication.
SOAP is typically limited to pooling, and not event notifications...
That is correct. But be aware that "typically" does not mean "always".
You can have events, but it's harder. It involves using WS-* specifications like WS-Eventing and WS-Addressing. This is a change in the way SOAP clients operate because a client now becomes some sort of a service too because it needs to receive calls too, not just initiate them. If your technology stack implements these specifications then good for you, but if it doesn't, then you have to build it yourself and it's a real pain.
So for these reasons, if you don't have blocking performance or resource usage issues, you "typically" chose doing polling with SOAP and not event notifications.
In developing backend components, I need to decide how these components will interact and communicate with each other. In particular, I need to decide whether it is better to use (RESTful, micro) web services versus a message broker (e.g. RabbitMQ). Are there certain criteria to help decide between using web services for each component versus messaging?
Eranda covered some of this in his answer, but I think three of the key drivers are:
Are you modeling a Request-Response type interaction?
Can your interaction be asynchronous?
How much knowledge does the sender of the information need to have about the recipients?
It is possible to do Request-Response type interactions with an asynchronous messaging infrastructure but it adds significantly to the complexity, so generally Request-Response type interactions (i.e. does the sender need some data returned from the recipient) are more easily modeled as RPC/REST interactions.
If your interaction can be asynchronous then it is possible to implement this using a REST interaction but it may scale better if you use a fire and forget messaging type interaction.
An asynchronous messaging interaction will also be much more appropriate if the provider of the information doesn't care who is consuming the information. An information provider could be publishing information and new consumers of that information could be added to the system later without having to change the provider.
Web server and message broker have their own use cases. Web server used to host web services and the message broker are use to exchange messages between two points. If you need to deploy a web service then you have to use a web server, where you can process that message and send back a response. Now let's think that you need to have publisher/subscriber pattern or/and reliable messaging between any two nodes, between two servers, between client and server, or server and client, that's where the message broker comes into the picture where you can use a message broker in the middle of two nodes to achieve it. Using message broker gives you the reliability but you have to pay it with the performance. So the components you should use depends on your use case though there are multiple options available.
I'm very new at WSO2 stack and wonder when I should use WSO2 ESB proxy service and when – create business process via BPEL?
I think they are doing the same thing – performing a task via services composition and some mediation.
There is a fundamental difference between ESB and BPEL.
The role of ESB is to provide various non-functional properties to the business requests. ESB is thus used for e.g. mediation, transformation, security and virtualization/proxying of the requests. While it can do some simple message-enrichment using sequence diagrams, its primary purpose is to mediate messages between various services/hosts in the system.
On the other hand, BPEL is dedicated to implement business services and handle complex business workloads. Therefore, the role of the BPEL is to provide the functional properties to the business process- e.g. implementing the actual business process logic.
ESB and BPEL thus together deliver the separation-of-concerns which is often emphasized by component and service-oriented architectures.
If you have a well defined long running Business process you need to use WSO2 BPS. You can use WSO2 ESB for short spanning process with a shorter life cycle. WSO2 BPS has many integration points that you can control the Business process with features such as Human Tasks. On the other hand ESB has the capabilities but it may not be convenient and optimized as BPS for the long running well defined business processes.
I've been tasked with creating an intermediate layer which needs to exchange data (over HTTP) between two independent systems (e.g. Receiver <=> Intermediate Layer (IL) <=> Sender). Receiver and Sender both expose a set of API's via Web Services. Everytime a transaction occurs in the Sender system, the IL should know about it (I'm thinking of creating a Windows Service which constantly pings the Sender), massage the data, then deliver it to the Receiver. The IL can temporarily store the data in a SQL database until it is transferred to the Receiver. I have the following questions -
Can WCF (haven't used it a lot) be used to talk to the Sender and Receiver (both expose web services)?
How do I ensure guaranteed delivery?
How do I ensure security of the messages over the Internet?
What are best practices for handling concurrency issues?
What are best practices for error handling?
How do I ensure reliability of the data (data is not tampered along the way)
How do I ensure the receipt of the data back to the Sender?
What are the constraints that I need to be aware of?
I need to implement this on MS platform using a custom .NET solution. I was told not to use any middleware like BizTalk. The receiver is an SDFC instance, if that matters.
Any pointers are greatly appreciated. Thank you.
A Windows Service that orchestras the exchange sounds fine.
Yes WCF can deal with traditional Web Services.
How do I ensure guaranteed delivery?
To ensure delivery you can use TransactionScope to handle the passing of data between the
Receiver <=> Intermediate Layer and Intermediate Layer <=> Sender but I wouldn't try and do them together.
You might want to consider some sort of queuing mechanism to send the data to the receiver; I guess I'm thinking more of a logical queue rather than an actual queuing component. A workflow framework could also be an option.
make sure you have good logging / auditing in place; make sure it's rock solid, has the right information and is easy to read. Assuming you write a service it will execute without supervision so the operational / support aspects are more demanding.
Think about scenarios:
How do you manage failed deliveries?
What happens if the reciever (or sender) is unavailbale for periods of time (and how long is that?); for example: do you need to "escalate" to an operator via email?
How do I ensure security of the messages over the Internet?
HTTPS. Assuming other existing clients make calls to the Web Services how do they ensure security? (I'm thinking encryption).
What are best practices for handling concurrency issues?
Hmm probably a separate question. You should be able to find information on that easily enough. How much data are we taking? what sort of frequency? How many instances of the Windows Service were you thinking of having - if one is enough why would concurrency be an issue?
What are best practices for error handling?
Same as for concurrency, but I can offer some pointers:
Use an established logging framework, I quite like MS EntLibs but there are others (re-using whatever's currently used is probably going to make more sense - if there is anything).
Remember that execution is unattended so ensure information is complete, clear and unambiguous. I'd be tempted to log more and dial it down once a level of comfort is reached.
use a top level handler to ensure nothing get's lost; but don;t be afraid to log deep in the application where you can still get useful context (like the metadata of the data being sent / recieved).
How do I ensure the receipt of the data back to the Sender?
Include it (sending the receipt) as a step that is part of the transaction.
On a different angle - have a look on CodePlex for ESB type libraries, you might find something useful: http://www.codeplex.com/site/search?query=ESB&ac=8
For example ESBasic which seems to be a class library which you could reuse.