I am a bit confused about the relationships between BPMN, BPEL and SOA.
In short words I understand it like this:
BPMN is graphical notation for process witch is using web services. So BPMN combine multiple web services in some order. Then BPEL is representation of this process created using BPMN in XML (executable level). So it is orchestration of multiple web services. The result of BPEL is new service, which I can use in SOA (among with other services)
Do I understand it correctly ? If not please try to explain it as simple as possible. Maybe real life example would help
Thank you very much
Let's start at the high level. SOA is an over-arching concept that can be applied across an enterprise's technology solutions. To that end it is more a philosophy for how you approach your solutions to ensure certain principle's are followed independent of the technology used to implement that solution.
I've used several BPMN engines / suites and have encountered some BPEL solutions within that work. In my experience BPEL seems to lean towards orchestrating system based processes, coordinating complex logic where multiple systems need to interact in order to achieve a business outcome. BPMN can also do this, but many implementations put significantly more focus on the Business Processes that require both coordination of system and of users.
It is important to note that BPMN is silent on implementation portion of the process. While it is intended to capture the "what" of a process, it is silent on how to actually make the items in the diagram execute in the manner indicated. This is indicative of the underlying difference between "Business Process Modeling Notation" and "Business Process Execution Language". The latter at leas implies that execution be included for you to be in compliance while the former is nearly a means to say "This is a notation that can be used to display / understand a business process. Implementation is not part of the specification."
This gets easily blurred as the actual software for BPMN generally gets pulled towards inclusion of implementation, either as an engine for doing the bookkeeping for process execution (e.g. what is currently happening, what is next after that, what happened before) like Camunda, or a full end to end BPM Suite like IBM BPM (now part of their "Digital Process Automation" suite). These include implementation for Process execution, but that is not required by the BPMN standard.
So it is orchestration of multiple web services
Do I understand it correctly?
No. Both BPMN and BPEL are notations, which are used for denoting, representation and execution of processes, not web-services (!!!). And they had no single interconnection in terms of implementation and statement
Then BPEL is representation of this process created using BPMN in XML (executable level)
is not true. Yes, there are tools (very outdated) that allows converting from BPEL to BPMN and vice-versa, but generally it's not that simple and mainly this is manual work. Very good point about the "ease" of converting BPMN2BPEL is given in Wiki.
BPMN and BPMEL is two competing approaches for solving the same problem, like DOCX ad ODF format for documents, like Android and iOS, etc. You can choose either tool, but for some of the tasks some of the tools suit better.
The result of BPEL is new service, which I can use in SOA (among with other services)
generally correct. The result of any BPEL scheme is web-service which represents the process, and you can grab this service as a WSDL-file and push to any WS-tool (SOA is not a tool, it is an architecture).
With BPMN it's a more complicated task, it requires BPMS, engine which will compile/render your service into smth executable like Web-service or Web-app.
BPMN has the focus in the interaction within pieces of business logic, describes as process and subprocess and their relation with an user input.
Usually you will have a process with some tasks, they always will follow a flow described in the process sometimes involving different bifurcation options, or events trigger-catching mechanism.
BPEL : It's main focus it's based on orchestrate (meaning putting all together) different external services, sources. Interacting with some internal logic, it can also use some task for an user to interact but not as a main thing.
Usually a BPMN process will be offered as a WS in a composite, Same as a BPEL. The main difference it's that BPMN will take as priority (user input, business logic representated in steps ), BPEL will take as priority( organizing different sources of data, writing, retrieving info to external systems and so). In a high level you could think of it as A Service BPMN uses multiple BPEL services to interact with external system.
So the complexity of different systems interactions decrease in a BPMN process by passing all of that to BPEL services. Making a more understandable diagram of how the business process work. This as a simple approach, hope it helps. Best!
All of this at a high level
Related
I'm an architect in a large scale financial company and we are in the beginning of implementing a new business oriented infosystem across our different countries.
From the very early on the core idea has been to follow microservice oriented principles as much as possible (and making sure engineers have read Building Microservices book by Sam Newman).
By now I've come to crossroads. Our services are primarily JSON REST services using Swagger for automated documentation, but in order to use these services in our business processes and making sure not to write business logic into services outside the domain of those services, we've been using Camunda as an orchestration tool. And Camunda is fine (though some have considered Corezoid as an alternative), but somewhat clumsy in what is an otherwise an elegant set of services.
Now service orchestration is a concept pretty familiar to most engineers. But it is one that I am not entirely happy with due to still having a central engine that drives everything. It is incredibly expensive to replace later down the road (though still cheaper to replace than a monolith). And even if this central engine is split into multiple engines (which is actually the case today), it does not necessarily make it much better.
In recent years there has been a movement with microservices towards choreographed (close to event-driven) architecture. It is at this point where I am looking for advice from engineers and architects who have faced similar crossroad decision points.
I absolutely love the idea of decoupled architecture and despite feeling good about killing monoliths and having elegant independent services, I still detect a lot of dependencies in business process as a whole in current orchestrated solution in where it should not actually exist.
And it's not like we are avoiding events. We have actually implemented events on our architecture as well in order to decouple many processes with the core principle that if you don't need a synchronized response and just need to notify of something happening to initiate another process an event is put up that may be caught by another process that starts executing. And orchestration is easier to explain and visualize, it is easier to tweak and modify by more technical minded business users. And I think it is easier to test and validate from business perspective. Orchestrated architecture like this also (usually) expects a good service discovery and quality automated documentation and non-functional requirements which are all things I value greatly.
All of those things that are a question to me in choreographed approach since I don't have first-hand experience in running this in large scale - just some local test prototypes.
But I think you see where I am coming from. I'm trying to consider alternatives without having to regret driving the company all the other way in the end.
Perhaps you can share your own experience with a similar situation or share an interesting link or two? Or am I looking for a silver bullet that doesn't exist yet?
Services need to interact - services that don't interact are not part of the same system. The search needs access to the catalog, the cart doesn't get the price info from the page, the account needs the purchase history, the recommender needs purchase history, the cart needs to verify the currently available coupons, the inventory needs to know something was purchased etc.
Service boundaries are set to minimize the needed interactions. It can make sense to cut a service to smaller components but if they share a database (internal structure) they are different aspects of the service.
When services interact it creates a level of coupling - at the least, this coupling is some API (JSON or otherwise) that the service has to "maintain" for so other services can interact with it.
Another coupling type is temporal coupling - which is what you get in request-reply situations (and you can eliminate in event driven systems) However, Orchestration vs. Choreography is not about these differences (even though orchestration is mostly associated with request/reply) - it is about central control and governance vs.flexibility and serendipity.
Orchestration has risks like migrating business logic out of services into the orchestration while choreography runs the risk of chaos. By the way, direct request/reply integration has the worst of both worlds but wins on simplicity when systems are small enough.
Choosing between the two is a balancing act (like most architectural decisions) for instance, Netflix built on choreography for a lot of time but then found they need some control back and introduced an orchestration engine. Nothing is a silver bullet :)
Personally, I like choreography better because of the reduced coupling and flexibility and favor tools like open Zipkin to bring some order into the chaos.
You can see a partial example for an orchestration based arch in slides 10-22 of a presentation I did about microservices
I think I understand where you're coming from, having recently redesigned a system to a "microservices" architecture. I like (and use) the approach by these guys: http://scs-architecture.org/
The main point is, that you try to avoid cross-dependencies between you "services", which basically makes choreography obsolete. The hard part is decomposing your problem domain into chunks which do not need eachother for any of the executed business-cases. They may need different kinds of data that may or may not be "shared", as in present in multiple systems, but they don't need synchronous calls between them for any given business case.
This is quite different from what Netflix is doing for example. Those guys/gals are doing chain-calling through different layers of services, each adding its logic to the "process". This model might fit in some cases, and probably fits in Netflix's case. But it may not be necessary for you.
The ideal Self-Contained System would be completely independent of other Self-Contained Systems, would cover one or more highly cohesive business functions (in full depth from the UI to Persistence!), and would be not calling any other system synchronously. The ideal system would let the client "orchestrate", by just offering links through its Web (HTML) interface.
Think more like Amazon. The "Landing Page" is a different application than the "Search", which is still different from the "Checkout". They are completely different, sometimes even look a bit different! Integrated by links and forms in HTML, not explicitly orchestrated.
This might be what you are looking for.
Some warnings: First instinct of some people is to have "Customer" microservice, or "Product Repository" microservice, and similar. This will not lead to Self-Contained Systems, as you will need synchronous calls to these things, making them essentially "central" components. The key is to split the business domain, so bounded contexts a la Eric Evans.
The project I'm currently involved requires that business logic must be implemented in Web Service that will be consumed by the Presentation Tier Components (i.e. Web Applications).
The company has an Enterprise Service Bus, and up-to-date almost every Web Service developed is exposed through this bus. I asked some colleagues around about when to expose Service through ESB and I got this answers:
If there's an ESB, expose everything through it: There are several benefits like Load-Balancing and location transparency
If the ESB will only act as a Proxy -i.e no message transformation- just don't use it: You'll overload the ESB and lose performance. You'll better do a point-to-point connection.
You should expose a component through ESB if there's a protocol transformation (like exposing a Stored Procedure as a SOAP Service). If this isn't present you better go Point-to-Point.
So I'm curious if there's a general agreement or best-practice of when to expose a Web Service through it or not. Any reading/reference would be a great help.
From my point of view and after 4 years of experience with SOA technologies, using an ESB will always overload the system since you are adding a new layer and making all your communications go through it. Transformation (either messaging or protocol) and routing aren't to hard to accomplish without an ESB and point to point communication will have a bit higher throughput. Same happens also with business process automation, there are ways to get there without the need of an ESB.
In the other hand, the use of an ESB has several benefits in the scope of a corporation but it must be within a vision and strategy. One of the best examples is a company that has been working for a long time with a wide range of tools, each of them for a specific purpose and that made the company be distributed in teams which work in silos, ones isolated from the others. After a long time that makes interaction between teams complex and slow. A well planned SOA strategy will help to integrate all those tools and start replacing them for more meaningful lightweight items.
So, IMHO, Using an ESB just to solve a couple of "issues" in a single project without a corporate strategy isn't a good idea and, eventually, the word SOA will be banned in your company, when the problem isn't SOA by itself by rather the lack of vision and corporate strategy.
The only rule of thumb that I found regarding the use of ESBs is: The requirement of transformation, routing, business process automation (with or without human interaction), etc. in a single project is not a symptom of going SOA (almost every project has to perform transformations, routing and business process automation), but when those needs are the ones for a whole corporation then it's worth to think about it from a business point of view, never a technical one. If there isn't a business perspective, then SOA will fail.
This is a really wide topic and discussion can last for ages, I will suggest you a couple of links for further reading:
Some SOA Case Studies
Top 10 Reasons why SOA fails
What exactly is web service composition?
Composition refers to the way something is build, the new term at the moment is mash-up which basically means utilising a variety of different services in a composite application. So that functionality of disparate application can be used in one application.
I think your referring to service granularity - which means how much functionality a service exposes. a coarse grained service will expose a whole process as a consumable unit whereas a fine grained service will expose a specific unit of logic from a larger process. Obviously, it is up to the service architects to determine what granularity of service works best in the given environment.
This also, in a way has to do with the style of SOAP message you are using whether it is RPC style or document and that a service should be atomic and not hold external state. Meaning it does not need to know any more information other than that in the SOAP message to perform its function.
Hope this gives you a good starting point. The trouble with service-orientation is that it differs depending on who you read, but the main points stay the same!
Jon
Some web services which are provided for clients are abstract and composition of some smaller web services and it's called web service composition.
Sometimes there are more than one web service in order to use as the mentioned small web services, so we choose them based on QoS (Quality of Service) and many researches have been done on this subject.
Web service composition involves integration of two or more web service to achieve more added value of business functionality. A work flow composer is responsible of aggregating different web services to act as a single service according to functional requirements as well as QoS constrains. BPEL is one of the popular composers uses XML language to perform service composition. Fine-grained services perform single business task and provides higher flexibility and reusability. However, coarse-grained service involves performing complex business functionality leading to lower flexibility
Not having dealt much with creating web-services, either from scratch, or by breaking apart an existing application, where does one start? Should a web-service encapsulate an entity, much like a class does, or should the service have more/less to it?
I realize that much of this is based on a case by case analysis of what the needs are, but are there any general guide-lines or best practices or even small nuggets of information that web-service veterans can impart to a relative newbie?
Our web services are built around functional areas. Sometimes this is just for a single entity, sometimes it's more than that.
For example, if you have a CRM, one of your web services might revolve around managing Contacts. Creating, updating, searching for, etc. If you do some type of batch type processing, a web service might exist to create and submit a job.
As far as best practices, bear in mind that web services add to the processing overhead. Mainly in serializing / deserializing the data as it goes across the wire. Because of this the main upside is solely in scalability. Meaning that you trade an increased per transaction processing time for the ability to run the service through multiple machines.
The main parts to pull out into a web service are those areas which are common across multiple applications, or which you intend to expose publicly, or which would benefit from greater load balancing.
Of course, you need to analyze your application to see where any bottlenecks really are. In some cases it doesn't make sense. For example, if you have a single application that isn't sharing its code and/or the bottleneck is primarily database related.
Web Services are exactly what they sound like Services for the Web.
A web service should be built as an API for the service layer of your app.
A service usually encapsulates an entity larger than a single class.
To learn more about service layers and refactoring to add a service layer read about DDD.
Good Luck
The number 1 question is: To what end are you refactoring your application functionality to be consumned as a bunch of web services?
We have a website, where transactions are entered in and put through a workflow. We are going to follow the standard BLL(Business Logic Layer), DTO(Data Transfer Object), DAL(Data Access Layer) etc. for a tiered application. We have the need to separate everything out because some transactions will cross multiple applications with different business logic.
We also have a backend processor. It handles our transactions once the workflow has been completed. It works with various third party systems, some of which are unstable, or the interface to them is unstable, and then reports the status of the transaction. Each website will have its own version of the backend processor.
Now the question, with N-Tier, they suggest a new BLL for each application. With the layout of the application above, it can be argued that the backend processor and website is one application acting in unison, or two applications with different business logic. What would be the ideal way to handle this? Have it act like one system, or two?
One thing that I picked up on while learning MVC over the last couple years is the difference between what I call application logic and domain logic. I don't like the term business logic anymore, because it has too much baggage from all the conflicting theories and practices that have used that term too loosely.
Domain logic is the "traditional" business logic, how things are supposed to act, what they require (validation), etc. Application logic is anything that is specific to a given presentation of your domain, IE when the user clicks this submit button in your web app then they are directed to this web page over here (note that this has nothing to do with how a WinForms app or a background processor would work). Application logic should live in your application. Domain logic should live in your BLL and lower, and be reusable across the different applications that may use your common "business logic".
Kind of a general answer, but I hope that helps.
You might consider partitioning the functionality to reflect the organization of the stakeholders. Usually if you have two distinct organizational groups, then development and administration requirements are easier to manage if the functionality is similarly partioned. And vise versa.
Most of us don't spend that much time writing applications that explore the outer boundaries of hardware and software capabilities.
If you separate your concerns well then I think that you will be able to view them as the same application with a single business logic layer, there is no point writing the same code twice. The trick will be forcing the separation of concerns between the user interface portions of the website and the business logic in your BLL library.
Performance is going to be an issue as well, you have to ensure that your batch processing doesn't block your website from performing tasks that it needs to perform due to your resources. This may be an argument to keep them more separate, however as they're likely sharing a database anyway (or some other file based resource) then that may be an issue regardless.
I would keep a common business logic library programmed to interfaces and fully separated from your other concerns.
The "Ideal" way to do this depends on the project at hand and the various requirements of the system.
My default design is to have it act as one app. But if there are more heavyweight processes taking place, I like to create a batching process where the parameters of the requested job are stored and acted upon by a seperate process.