We have 3 layered application where every call from the service layer goes to the business layer and is persisted by the data layer. Components of each layer can only call the layer below;
However because we have hundreds of entities and we have lots of services related to crud operations so many controversies raised on our team.
Some believe for the sake of maintenance and ease of development it's better to call data access from crud services which just doing crud operation and bypassing business layer.
On the contrary some saying we have to create wrapper for data access of each entity in business layer and call these wrapper from services and never allow services to call data access layer.
In your idea which way we should take? Is it ok for crud services to call data accesses and bypassing business layer?
If there is no business logic to perform, there is no reason to enforce a business layer. The 3-tier architecture is not an arcane protocol, just a best practice that was formed assuming business processing.
In a current application we are often accessing DAOs directly from JSF controllers when there is no business process involved. The idea was given by a java champion who stressed the idea that simplicity is paramount.
If you are worried about future modifications that may require adding business logic. I think of the issue this way: The additional business logic would be added to the business layer anyway, including data access, so there is no difference here.
CRUD code is mostly very simple. So the change in the service would amount to reroute a single call or a couple of calls fron the DAO to an EJB - a simple refactoring. The CRUD code itself would still remain, but will be pushed into the EJB - another simple refactoring.
This is not perfect, but IMO better then the alternative: having an empty indirection layer. This adds complexity that serves no purpose. The business object would do nothing but forward the calls to the DAO.
There are two code smells that I think apply in this case: contrived complexity and feature envy.
I am not saying that DA in the business layer is somehow a code smell. What I mean is that having a business object that does nothing else but proxy the DAO is a smell. It's the same with complexity - an added data structure / architectural layer that serves no own purpose - there seems to be a DAL in your application already.
Another aspect for your consideration would be - how surprising is it for a developer to see a service that uses a DAO directly? Having 5 services where 2 of them access DAO directly is different from having 100 services where only one service accesses the DAOs directly.
In the first case the code simplicity would outweigh the added conceptual complexity (2 concepts for a single thing), in the second case, I would rather stick with the business layer - the surprise (also called the WTF-effect ;) of doing it differently just once would be too big.
In my opinion, having call CRUD services bypassing the business layer will start converting the current service layer into a business layer as you increase the functionality. So your service layer will act as the business layer too, if you are fine with it.
In most cases, you deal with an entity and probably that could involve many data layer crud calls, on one update for example. A business layer is recommended for this purpose. And business layer is the place to execute any business rules, caching or call other business services if required. This will keep the upper layer simple and pass through.
I will not bypass the business layer. Even though it might look like you are only proxying the call for the DAL into the BL, and even though this may be a case of very simple CRUD operation, the BL can encapsulate the validation logic required for the operation. Validation logic can be performed on the BL and proxying to DAL will only be performed if the validation conditions are met.
If you only have CRUD operations, you can use Data Services to publish your data sets as web services. A good example to this is WSO2 Data Services http://wso2.com/products/data-services-server/
Related
When building a microservice oriented application, i wonder what could be the appropriate microservice granularity.
Let's image an application consisting of:
A set of various resources types where each resource map a given business model. (ex: In a todo app resources could be User, TodoList and TodoItem...)
Each of those resources are saved within a NoSQL database that could be replicated.
Each of those resources are exposed through a REST Api
The application manage an internal chat room.
An Api gateway for gathering chat room and REST api interaction.
The application front end: an SPA application connected to the API Gateway
The first (and naive) approach when thinking about how microservices could match the need of this application would be:
One monolith service for managing EVERY resources and business logic:
By managing i mean providing the REST API for all of those resources and handling the persistance of those resources within the database.
One service for each Database replica.
One service providing the internal chat room using websocket or whatever.
One service for Authentification.
One service for the api gateway.
One service serving the static assets for the SPA front end.
An other approach could be to split service 1 into as many service as business models exist in the system. (let's call those services resource services)
I wonder what are the benefit of this second approach.
In fact i see a lot of downsides with this approach:
Need to setup an inter service communication process.
When requesting a service representing resource X that have a relation with resource Y, a lot more work are needed (i.e: interservice request)
More devops work.
More difficulty to share common code between resource services.
Where to put business logic ?
When starting a fresh project this second approach seams to me a bit of an over engineered work.
I feel like starting with the first approach and THEN split the monolith resource service into several specific services depending on the observed needs will minimize the complexity and risks.
What's your opinions regarding that ?
Is there any best practices ?
Thanks a lot !
The first approach is not microservice way, by definition.
And yes, idea is to split - each service for Bounded Context - One for Users, one for Inventory, Todo things etc etc.
The idea of microservices, at very simple, assumes:
You want to pay extra dev-ops work for modularity, and complete/as much as possible removal of dependencies between different bounded contexts (see dev/product/pjm teams).
It's idea lies around ownership, modularity, allowing separate teams develop their own piece of code, without requirement from them to know the rest of the system . As long as there is Umbiqutious Language (common set of conventions/communication protocols/terminology/documentation) they can work in completeley isolated, autotonmous fashion.
Maintaining, managing, testing, and develpoing become much faster - in cost of initial dev-ops and sophisticated architecture engeneering investment.
Sharing code should be minimal, and if required, could be done to represent the Umbiqutious Language (common communication interface/set of conventions). Sharing well-documented code, which acts as integration/infrastructure mini-framework, and have special dev/dev-ops/team attached to it ccould be easy business, as long as it, as i said, well-documented, and threated as separate architecture-related sub-project.
Properly engeneered Microservice architecture could lessen maintenance and development times by huge margin, but it requires quite serious reason to use it (there lot of reasons, and lots of articles on that, I wont start it here) and quite serious engeneering investment at start.
It brings modularity, concept of ownership, de-coupling of different contexts of your app.
My personal advise check if you really need MS architecture. If you can not invest engenerring though and dev-ops effort at start and do not have proper reasons for such system - why bother?
If you do need MS, i would really advise against the first method. You will develop wrong thing's, will miss the true challenges of MS, and could end with huge refactor, which could take more work than engeneering MS system from start properly. It's like to make square to make it fit into round bucket later.
Now answering your question title: granularity. (your question body bit different from your post title).
Attach it to Domain Model / Bounded Context. You can make meaty services at start, in order to avoid complex distributed transactions.
First just answer question if you need them in your design/architecture?
If not, probably you did a good design.
Passing reference ids between models from different microservices should suffice, and if not, try to rethink if more of complex transactions could be avoided.
If your system have unavoidable amount of distributed trasnactions, perhaps look towards using/making some CQRS mini-framework as your "shared code infrastructure component" / communication protocol.
It is the key problem of the microservices or any other SOA approach. It is where the theory meets the reality. In general you should not force the microservices architecture for the sake of it. This should rather naturally come from functional decomposition (top-down) and operational, technological, dev-ops needs (bottom-up). First approach is closer to what you would need to do, however at the first step do not focus so much on the technology aspect. Ask yourself why would you need to implement a separate service for particular business function. Treat it as a micro-application with all its technical resources. Ask yourself if there is reason to implement particular function as a full-stack app.
Some, of the functionalities you have mentioned in scenario 1) are naturally ok, such as 'authentication' service - this is probably good candidate.
For the business functions decomposition into separate service, focus on the 'dependencies' problem, if there are too many dependencies and you see that you have to implement bigger chunk of data mode - naturally this is not a micro service any more.
Try to put litmus test , if you can 'turn off' particular functionality and the system still makes sense - it is the candidate for service or further decomposition
I'm creating a web-application and decided to use micro-services approach. Would you please tell me what is the best approach or at least common to organize access to the database from all web-services (login, comments and etc. web-services). Is it well to create DAO web-service and use only it to to read/write values in the database of the application. Or each web-service should have its own dao layer.
Each microservice should be a full-fledged application with all necessary layers (which doesn't mean there cannot be shared code between microservices, but they have to run in separate processes).
Besides, it is often recommended that each microservice have its own database. See http://microservices.io/patterns/data/database-per-service.html https://www.nginx.com/blog/microservices-at-netflix-architectural-best-practices/ Therefore, I don't really see the point of a web service that would only act as a data access facade.
Microservices are great, but it is not good to start with too many microservices right away. If you have doubt about how to define the boundaries between microservices in your application, start by a monolith (all the time keeping the code clean and a good object-oriented with well designed layers and interfaces). When you get to a more mature state of the application, you will more easily see the right places to split to independently deployable services.
The key is to keep together things that should really be coupled. When we try to decouple everything from everything, we end up creating too many layers of interfaces, and this slows us down.
I think it's not a good approach.
DB operation is critical in any process, so it must be in the DAO layer inside de microservice. Why you don't what to implement inside.
Using a service, you loose control, and if you have to change the process logic you have to change DAO service (Affecting to all the services).
In my opinion it is not good idea.
I think that using Services to expose data from a database is ideal due to the flexibility it provides. Development of a REST service to expose some or all of your data as a service provides flexibility to consume the data directly to the UI via AJAX or by other services which can process the data and generate new information. These consumers do not need to implement a DAO and can be in any language. While a REST Service of your entire database is probably not a Micro-Service, a case could be made for breaking this down as Read only for Students, Professors and Classes for exposing on the School Web site(s), with different services for Create, Update and Delete (CUD) available only to the Registrars office desktop applications.
For example building a Service to exposes a statistical value on data will protect the data from examination by a user/program who only needs a statistical value without the requirement of having the service implement an entire DAO for the components of that statistic. Full function databases like SQL Server or Oracle provide a lot of functionality that application developers can use, including complex queries(using indexes), statistics the application of set operations on data.
Having a database service is a completely valid pattern. In fact, this is one of the key examples of where to start to export aspects of a monolith to a micro service in the Building Microservices book.
How to organize your code around such idea is a different issue. Yes, from the db client programmer's stand point, having the same DAO layer on each DB client makes a lot of sense.
The DAO pattern may be suitable to bind your DB to one programming language that you use. But then you need to ask yourself why you are exposing your database as a web service if all access to it will be mediated by the same DAO infrastructure. Or are you going to create one DAO pattern for each client programming language binding?
If all database clients are going to be written on the same programming language, then are you sure you really need to wrap your DB as a microservice? After all, the DB is usually already a remote service with a well-defined network protocol optimized to transfer data fast and reliably. Why adding HTTP on top of it? What are you expecting to gain from adding such complexity?
Another problem with using the DAO pattern is that the DAO structure does not necessarily follow the evolution of the web service. The web service may evolve in a way that does not make old clients incompatible. You may have different clients using different features of the micro service. In this case you are not sharing the same DAO layer structure on each client.
Make sure you are not using RPC-style programming over web services, which does not make much sense. You will be basically throwing away one of the key advantages of micro services, which is the decoupling between service and client.
I have a question for the microservices community. I'll give an example from the educational field but it applies to every microservices architecture.
Let's say I have student-service and licensing-service with a business requirement that the number of students is limited by a license. So every time a student is created a licensing check has to be made. There are multiple types of licenses so the type of the license would have to be included in the operation.
My question is which approach have you found is better in practice:
Build a composite service that calls the 2 services
Coupling student-service to licensing-service so that when createStudent is called the student-service makes a call to licensing-service and only when that completes will the student be created
Use an event-based architecture
People talk about microservice architectures being more like a graph than a hierarchy and option 1 kinda turns this into a hierarchy where you get increasingly coarse composites. Other downsides is it creates confusion as to what service clients should actually use and there's some duplication going on because the composites API would have to include all of the parameters that are needed to call the downstream services.
It does have a big benefit because it gives you a natural place to do failure handling, choreography and handle consistency.
Option 2 seems like it has disadvantages too:
the API of licensing would have to leak into the student API so that you can specify licensing restrictions.
it puts a lot of burden on the student-service because it has to handle consistency across all of the dependent services
as more services need to react when a student is created I could see the dependency graph quickly getting out of control and the service would have to handle that complexity in addition to the one from its own logic for managing students.
Option 3 While being decoupling heaven, I don't really think would work because this is all triggered from an UI and people aren't really used to "go do something else until this new student shows up" approach.
Thank you
Option 1 and 2 creates tight coupling which should be avoided as much as possible because you would want to have your services to be independent. So the question becomes:
How do we do this with an event-based architecture?
Use events to keep track of licensing information from license service in student service, practically a data duplication. Drawbacks here are: you only have eventual consistency as the data duplication is asynchronous.
Use asynchronous events to trigger event chain which ultimately trigger a student creation. From your question, it looks like you already got the idea, but have an issue dealing with UI. You have two possible options here: wait for the student creation (or failure) event with a small amount of timeout, or (event better), make you system completely reactive (use server-client push mechanism for the UI).
Application licensing and creating students are orthogonal so option 2 doesn't make sense.
Option 1 is more sensible but I would try not to build another service. Instead I would try to "filter" calls to student service through licensing middleware.
This way you could use this middleware for other service calls (e.g. classes service) and changes in API of both licensing and students can be done independently as those things are really independent. It just happens that licensing is using number of students but this could easily change.
I'm not sure how option 3, an event-based approach can help here. It can solve other problems though.
IMHO, I would go with option 2. A couple of things to consider. If you are buying complete into SOA and furthermore microservices, you can't flinch everytime a service needs to contact another service. Get comfortable with that.... remember that's the point. What I really like about option 2 is that a successful student-service response is not sent until the license-service request succeeds. Treat the license-service as any other external service, where you might wrap the license-service in a client object that can be published by the license-service JAR.
the API of licensing would have to leak into the student API so that you can specify licensing restrictions.
Yes the license-service API will be used. You can call it leakage (someone has to use it) or encapsulation so that the client requesting the student-service need not worry about licensing.
it puts a lot of burden on the student-service because it has to handle consistency across all of the dependent services
Some service has to take on this burden. But I would manage it organically. We are talking about 1 service needing another one. If this grows and becomes concretely troublesome then a refactoring can be done. If the number of services that student-service requires grows, I think it can be elegantly refactored and maybe the student-service becomes the composite service and groups of independently used services maybe be consolidated into new services if required. But if the list of dependency services that student-service uses is only used by student-service, then I do not know if its worth grouping them off into their own service. I think instead of burden and leakage you can look at it as encapsulation and ownership.... where student-service is the owner of that burden so it need not leak to other clients/services.
as more services need to react when a student is created I could see the dependency graph quickly getting out of control and the service would have to handle that complexity in addition to the one from its own logic for managing students.
The alternative would be various composite services. Like my response for the previous bullet point, this can be tackled elegantly if it surfaces as a real problem.
If forced each of your options can be turned into viable solution. I am making an opinionated case for option 2.
I recommend option 3. You have to choose between availability and consistency - and availability is most often desired in microservices architecture.
Your 'Student' aggregate should have a 'LicenseStatus' attribute. When a student is created, its license status is set to 'Unverfied', and publishes an event 'StudentCreated'. The LicenseService should then react to this event and attempt to reserve a license for this student. It would then publish a 'Reserved' or 'Rejected' event accordingly. The student service would update the student's status by subscribing to these events.
When the UI calls your API gateway to create a student, the gateway would simply call the Student service for creation and return a 202 Accepted or 200 OK response without having to wait for the student to be properly licensed. The UI can notify the user when the student is licensed through asynchronous communication (e.g. via long-polling or web sockets).
In case the license service is down or slow, only licensing would be affected. The student service would still be available and would continue to handle requests successfully. Once the license service is healthy again, the service bus will push any pending 'StudentCreated' events from the queue (Eventual consistency).
This approach also encourages expansion. A new microservice added in the future can subscribe to these events without having to make any changes to the student or license microservices (Decoupling).
With option 1 or option 2, you do not get any of these benefits and many of your microservices would stop working due to one unhealthy microservice.
I know the question has been asked a while ago, but I think I have something to say that might be of value here.
First of all, your approach will depend on the overall size of your final product. I tend to go with a rule of thumb: if I would have too many dependencies between individual micro-services, I tend to use something that would simplify and possibly remove these dependencies. I don't want to end up with a spider-web of services! A good thing to look at here are Message queues, like RabbitMQ for example.
However, if I have just a few services that talk to each other, I will just make them call each other directly, as any alternative solutions whilst simplifying the architecture, add some computing and infrastructure overhead.
Whatever approach you will decide to go with, design your services in a Hexagonal architecture in mind! This will save you trouble when you decide to migrate from one solution to another. What I tend to do is design my DAOs as "adapters", so a DAO that calls Service A will either call it directly or via message queue, independent of the business logic. When I need to change it, I can just change this DAO for another one, without having to touch any of the business logic (at the end of the day business logic doesn't care how it gets the data). Hexagonal architecture fits really well with micro-service, TDD and black-box testing.
I've built a RESTful service for the Data Access Layer (DAL) of my architecture:
POST http://example.com/data/User
GET|PUT|DELETE http://example.com/data/User/{UserId}
However, for the Business Logic Layer (BLL), a second non-RESTful service is used:
POST http://example.com/accountapi/register
POST http://example.com/accountapi/login
This BLL service, instead of making calls to the DAL service, it talks directly to the database.
How would you improve this architecture?
Should the BLL service call the DAL service ?
Should I drop the DAL service and only expose the BLL service ?
Should I, somehow, inject business logic on my RESTful DAL service ? If yes, how ?
To answer the main question. No, not really. To answer the secondary questions. None of the above.
REST based architectures do not fit nicely into the standard 3 tier model. The simplistic view of the three tiered model looks like this:
Presentation Layer <-> Business
Logic Layer <-> Data Layer
Consider for a moment breaking the presentation layer into two parts,
Rendering Layer <-> User Interface
Content <-> BLL <-> DAL
If you think about a regular Web application, the browser takes HTML, CSS and Javascript content and renders them visually in a browser. It is the User Interface Content layer that the REST constraints apply to. This is most obvious if you think about the hypermedia constraint. REST interfaces are mean to be navigated just like user interfaces are. REST interfaces return representations of resources.
REST interfaces should return user interface content that is independent of how the user interface will be displayed.
REST Client <-> REST Interface <-> BLL <-> DAL
In my opinion REST clients come in two forms, either very thin media type rendering engines (e.g. Web browsers) or screen scrapers (spiders, mashups). I use the term "screen scraper" loosely because if you choose your media-types wisely it should be trivial for the client to scrape the data out of your user interface content.
Any attempt to expose Business Logic Layers as REST interfaces usually has a few effects. Developers end up asking how to do transactions in REST. They end up creating huge amounts of coupling between the client and the BLL interface because of the need to expose semantically rich representations. They forget all about the hypermedia constraint, because much of that linking information is not available in the business logic layer. And they start to complain about the performance overhead of HTTP and text based content types.
(1) Avoid having your (non-REST) web service business logic layer make further (RESTful) HTTP requests onto the data access layer. Doing so of course would be less efficient than making direct method calls. But much more importantly, it would require you to deploy the BLL web services and the DAL web services onto separate web server instances (or separate clusters). Otherwise you can have a case where all your HTTP worker threads are busy trying to serve up BLL responses, and each is blocked waiting fruitlessly for a free HTTP worker thread to serve it a DAL response. This can lead to stalls (if you do timeout/retry processing) or outright deadlocks (if you don't).
(2 and 3) If your web service clients need both business logic and data access, provide those as a unified set of services. Internally they both depend on the same data access layer method calls: it's just that the data oriented web service implementations make just one data access layer call while the business logic oriented web service implementations may make many DAL calls. You do most definitely want to structure the BLL and DAL layers separately beneath the web service layer though.
I like to think of the web services as just the part of the presentation layer oriented towards "users" who happen to be other programs.
If this is the same application, then you should probably have the BLL layer call the DAL layer directly in code instead of using a service call again. That would keep it more performant while keeping the fundamental purposes of the code separate (high cohesion).
Probably so. Your services should generally be course-grained components that perform a business function. Saving a user in the database isn't a business function, it's a specific implementation. The Register function abstracts that notion into a business function. The BLL layer can then enforce things like password strength, password encryption, uniqueness of usernames, etc. that aren't directly related to data access.
Probably not. See #2.
We are developing a web application. We want to possibly reuse the work we do here for a different application that will use the same database, and use the same business rules for reading and writing to said database.
Which design would be more correct
Having the UI call web services, which would use business objects containing the business logic, which would talk to the data access layer.
Have the UI use business objects containing the business logic, which would call web services, which would then talk to the data access layer.
Have the UI user business objects containing the business logic, which would talk to the data access layer.
Don't mix logical design with physical design. Logical design operates over layers and physical design - tiers. Web Service is not a layer. It is simply a tier.
In logical design there is standard approach: UI layer-> BL layer -> DAL
In physical design all layers can reside within one client-side application connecting local database, or can be distributed over the remote tiers. But for distributed applications usually is added one more layer: Application layer, which hides from BL layer communication over the wire.
I would say the 3rd one. I tend to think of web services as another presentation layer.
Think of it this way: you have a web UI, which calls your business layer code to do things like create a new user (User.Add), find all products that match a given description (Products.FindByDescription), etc.
You can now re-use that same business layer code to build a set of public-facing web services for 3rd parties to make use of. There can be a method which adds a user - that calls your internal User.Add() method, another one to find products, etc..
What you get is a parallel set of presentations/interfaces to the same underlying data and business logic.
Behind the scenes (totally out of the scope of web services or UI layers), the business layer calls a data access layer that takes care of physically querying the database. If you were to change to a different DBMS, you should ideally (and in theory) be able to rebuild the data layer for the new database and have everything simply work.
Your business layer contains the rules like a username has to be 4 to 15 characters long; users are only allowed to search for and load products that are at a store they have access to; etc.
If you decide to change a business rule - like a user is allowed to search for products in any store in their state - then you change it in once place, and don't have to touch the web service or UI to make it work.
From your description, you haven't provided a reason why you would need the use of a web service layer. Assuming your database is reachable by your UI system, i.e. within the same network behind your firewall, a basic business-object layer that your website UI code (server-side, I'm assuming) will employ meets your requirements.
Bring in a web service tier when the distance between your UI system and your data layer starts to cross boundaries that a Data access layer or Business logic layer would begin to encounter difficulties.
In terms of the design being "correct" or not, it's not really possible to give a 100% answer to the correctness of a design without the full context. What are the requirements (functional and non-functional)? What design goals do you want to fulfill? How important is each goal?
The only goal your question mentions is that you want to reuse the business logic with another application. When I want to reuse the business logic of an application in a standard way I choose web services. So based solely on your one requirement I would say that option 1 ( UI->Web Service->Business Layer->Data Layer ) is a good choice.
Logically, web-services belong in the UI layer. Think of "User" being not only a human but another system and it becomes clear. Maintaining strict separation of concerns between these logical layers will allow you to easily implement and maintain your application.
Check Out: http://www.icemanind.com/layergen.aspx
The way it should go is, you have your UI layer on top, your data layer on the bottom and your business layer in between the two. Each layer can only communicate with the layer below it. So the UI talks to the business layer only...the business layer talks to the data layer only. Your UI should never talk with the data layer and your data layer should never interact with your UI.
Unless you have a reason to use a web service, then I wouldn't.
Do you hear anything about Service layer ? I think you can use a service layer for your transactions and operations and using a facade layer helps you to isolate and manage accessing from UI to data access layer directly or indirectly after visiting the Business layer . it depends on your requirements.