is it possible to achieve transactionality using web services that are implemented in different technologies?
For example: let's imagine a case where we want to offer an integrated service of 2 different organizations, each of which already have their different systems implemented using different technologies and located in different countries.
Organization A has a Java server exposing Rest services that allow consumers to insert data in a table, then Organization B has a .Net server exposing Rest services that also allow consumers to insert data in a table.
Then I want to create a new server to integrate both these services in one, allowing consumers to insert in both organization's databases. So from this new server I have to invoke those 2 rest services in a transactional way (meaning that both organizations will insert or none will insert if there is a failure, it will rollback).
Is that possible to achieve even tho server 1 and server 2 are implemented with different technologies? What if there were n servers all implemented in different technologies and all exposing Rest services?
It is not possible to have real transactions in case of microservice architecture. You have to implement compensations which roll back changes in case of failures. See Saga pattern for one of the approaches.
Look at the Cadence Workflow that makes implementing Sagas trivial. Here is an example.
Related
I'd like to use the microservices architectural pattern for a new system, but I'm having trouble figuring out how to share and merge data between the services when the services are isolated from each other. In particular, I'm thinking of returning consolidated data to populate a web app UI over HTTP.
For context, I'm intending to deploy each service to its own isolated environment (Heroku) where I won't be able to communicate internally between services (e.g. via //localhost:PORT. I plan to use RabbitMQ for inter-service communication, and Postgres for the database.
The decoupling of services makes sense for CREATE operations:
Authenticated user with UserId submits 'Join group' webform on the frontend
A new GroupJoinRequest including the UserId is added to the RabbitMQ queue
The Groups service picks up the event and processes it, referencing the user's UserId
However, READ operations are much harder if I want to merge data across tables/schemas. Let's say I want to get details for all the users in a certain group. In a monolithic design, I'd just do a SQL JOIN across the Users and the Groups tables, but that loses the isolation benefits of microservices.
My options seem to be as follows:
Database per service, public API per service
To view all the Users in a Group, a site visitor gets a list of UserIDs associated with a group from the Groups service, then queries the Users service separately to get their names.
Pros:
very clear separation of concerns
each service is entirely responsible for its own data
Cons:
requires multiple HTTP requests
a lot of postprocessing has to be done client-side
multiple SQL queries can't be optimized
Database-per-service, services share data over HTTP, single public API
A public API server handles request endpoints. Application logic in the API server makes requests to each service over a HTTP channel that is only accessible to other services in the system.
Pros:
good separation of concerns
each service is responsible for an API contract but can do whatever it wants with schema and data store, so long as API responses don't change
Cons:
non-performant
HTTP seems a weird transport mechanism to be using for internal comms
ends up exposing multiple services to the public internet (even if they're notionally locked down), so security threats grow from greater attack surface
Database-per-service, services share data through message broker
Given I've already got RabbitMQ running, I could just use it to queue requests for data and then to send the data itself. So for example:
client requests all Users in a Group
the public API service sends a GetUsersInGroup event with a RequestID
the Groups service picks this up, and adds the UserIDs to the queue
The `Users service picks this up, and adds the User data onto the queue
the API service listens for events with the RequestID, waits for the responses, merges the data into the correct format, and sends back to the client
Pros:
Using existing infrastructure
good decoupling
inter-service requests remain internal (no public APIs)
Cons:
Multiple SQL queries
Lots of data processing at the application layer
harder to reason about
Seems strange to pass large quantities around data via event system
Latency?
Services share a database, separated by schema, other services read from VIEWs
Services are isolated into database schemas. Schemas can only be written to by their respective services. Services expose a SQL VIEW layer on their schemas that can be queried by other services.
The VIEW functions as an API contract; even if the underlying schema or service application logic changes, the VIEW exposes the same data, so that
Pros:
Presumably much more performant (single SQL query can get all relevant data)
Foreign key management much easier
Less infrastructure to maintain
Easier to run reports that span multiple services
Cons:
tighter coupling between services
breaks the idea of fundamentally atomic services that don't know about each other
adds a monolithic component (database) that may be hard to scale (in contrast to atomic services which can scale databases independently as required)
Locks all services into using the same system of record (Postgres might not be the best database for all services)
I'm leaning towards the last option, but would appreciate any thoughts on other approaches.
To evaluate the pros and cons I think you should focus on what microservices architecture is aiming to achieve. In my opinion Microservices is architectural style aiming to build loosely couple applications. It is not designed to build high performance application so scarification of performance and data redundancy are something we are ready accept when we decided to build applications in a microservices way.
I don't think you services should share database. Tighter coupling scarify the main objective of the microservices architecture. My suggestion is to create a consolidated data service which pick up the data changes events from all the other services and update the database behind it. You might want to design the database behind the consolidated data service in a way that is optimised for query (like a data warehouse) because that's all this service will be used for. You might want to consider using a NoSQL database to support your consolidated data service.
I'm creating a web-application and decided to use micro-services approach. Would you please tell me what is the best approach or at least common to organize access to the database from all web-services (login, comments and etc. web-services). Is it well to create DAO web-service and use only it to to read/write values in the database of the application. Or each web-service should have its own dao layer.
Each microservice should be a full-fledged application with all necessary layers (which doesn't mean there cannot be shared code between microservices, but they have to run in separate processes).
Besides, it is often recommended that each microservice have its own database. See http://microservices.io/patterns/data/database-per-service.html https://www.nginx.com/blog/microservices-at-netflix-architectural-best-practices/ Therefore, I don't really see the point of a web service that would only act as a data access facade.
Microservices are great, but it is not good to start with too many microservices right away. If you have doubt about how to define the boundaries between microservices in your application, start by a monolith (all the time keeping the code clean and a good object-oriented with well designed layers and interfaces). When you get to a more mature state of the application, you will more easily see the right places to split to independently deployable services.
The key is to keep together things that should really be coupled. When we try to decouple everything from everything, we end up creating too many layers of interfaces, and this slows us down.
I think it's not a good approach.
DB operation is critical in any process, so it must be in the DAO layer inside de microservice. Why you don't what to implement inside.
Using a service, you loose control, and if you have to change the process logic you have to change DAO service (Affecting to all the services).
In my opinion it is not good idea.
I think that using Services to expose data from a database is ideal due to the flexibility it provides. Development of a REST service to expose some or all of your data as a service provides flexibility to consume the data directly to the UI via AJAX or by other services which can process the data and generate new information. These consumers do not need to implement a DAO and can be in any language. While a REST Service of your entire database is probably not a Micro-Service, a case could be made for breaking this down as Read only for Students, Professors and Classes for exposing on the School Web site(s), with different services for Create, Update and Delete (CUD) available only to the Registrars office desktop applications.
For example building a Service to exposes a statistical value on data will protect the data from examination by a user/program who only needs a statistical value without the requirement of having the service implement an entire DAO for the components of that statistic. Full function databases like SQL Server or Oracle provide a lot of functionality that application developers can use, including complex queries(using indexes), statistics the application of set operations on data.
Having a database service is a completely valid pattern. In fact, this is one of the key examples of where to start to export aspects of a monolith to a micro service in the Building Microservices book.
How to organize your code around such idea is a different issue. Yes, from the db client programmer's stand point, having the same DAO layer on each DB client makes a lot of sense.
The DAO pattern may be suitable to bind your DB to one programming language that you use. But then you need to ask yourself why you are exposing your database as a web service if all access to it will be mediated by the same DAO infrastructure. Or are you going to create one DAO pattern for each client programming language binding?
If all database clients are going to be written on the same programming language, then are you sure you really need to wrap your DB as a microservice? After all, the DB is usually already a remote service with a well-defined network protocol optimized to transfer data fast and reliably. Why adding HTTP on top of it? What are you expecting to gain from adding such complexity?
Another problem with using the DAO pattern is that the DAO structure does not necessarily follow the evolution of the web service. The web service may evolve in a way that does not make old clients incompatible. You may have different clients using different features of the micro service. In this case you are not sharing the same DAO layer structure on each client.
Make sure you are not using RPC-style programming over web services, which does not make much sense. You will be basically throwing away one of the key advantages of micro services, which is the decoupling between service and client.
This post http://www.theserverside.net/tt/articles/showarticle.tss?id=Top5WSMistakes
encourages me to create the web service for business logic layer but many people use it in the data access layer.
I want to create a project where i want to access the same data repository from a desktop application, website and a cell phone. What would you recommend me?
Is there any case it may be a good idea to implement web services to both layers?
The question is too open ended so the answer is: it depends.
What needs do your applications have for the data? Is it just data access or some business logic involved? If it is just accessing of data, do you really want the client to have direct control over it? How similar are the three applications? Do they share functionality or just data?
As I see it there are two main paths you can chose:
1 - expose a web service for the business, with the data hidden behind the web service. This is a good setup if the three clients (I'll call the desktop app, web app and cell phone "clients" since that is what they are) share functionality (i.e. they are different views for the same business model). This avoids duplicating similar business logic in all the clients;
2 - expose the data directly with a web service. This is a good setup if the three clients have nothing in common but just use the same data for different purposes. But in this case, with the three sets of business logic, where are you going to put the logic? In the clients? How will that work for the desktop application (considering you install this desktop app 300 times or so)? You again need some service and the clients to be thin clients not thick ones.
If you take 1) and 2) into consideration you will see that usually it is better to have a service layer in front of your data.
Going back to the "it depends", analyze your special needs first and only then choose the solution that is best suited for your situation.
How about a point 3? make your data access layer into a library (.jar, .dll or whatever technology you are using) and make that available to the (1? 2? 3?) business web services that serve your clients?
Could someone please direct me to some good documentation or feedback here on what are best practices for implementing web services in an application that handles different concerns? For example, should I create different services, one that handles security, (AuthService), one that handles data-entry for customer service reps, (CRUDService), BillingService and so on or should I just encapsulate all these "services" into one, e.g. ApplicationService? Basically, I am asking if it is bad design to create multiple services (files) within one application. Can some of you note on your experiences or what you've experienced?
Also, let's say three of the listed services from above connect to the same database, but are actually hitting totally different concerns, e.g. one is for all transactions like CRUD, and the other one is for purely reporting purposes. Should I create two services here, one CRUDService and the other for ReportingService? Is it bad to create two different database connections via these 2 services? Or how can I share the same database connection with different services?
I think there is a tendency among publicly available services to just dump everything into one service. Which, may not be a bad idea for a publicly available API. It just makes it easier for developers. However, for any project i work on, i try to break things down into logical groups. This way your client doesn't need to be inheriting functionality it may not need. Updating services would also be a slightly easier task because you're only affecting a certain subset of your web service framework and not everything. So if your service contract breaks and your clients no longer support it, they may still be able to use other parts of your system, but not that particular one. Where as if you break a contract on your aggregated service, everything fails. Finally, if you have to implement something like a fail-over support, you have more flexibility to choose which service requires more fail-over nodes, allowing you to better manage your resources allocation.
If you want best practices take a look to the SOA Design Pattern Catalog
What exactly is web service composition?
Composition refers to the way something is build, the new term at the moment is mash-up which basically means utilising a variety of different services in a composite application. So that functionality of disparate application can be used in one application.
I think your referring to service granularity - which means how much functionality a service exposes. a coarse grained service will expose a whole process as a consumable unit whereas a fine grained service will expose a specific unit of logic from a larger process. Obviously, it is up to the service architects to determine what granularity of service works best in the given environment.
This also, in a way has to do with the style of SOAP message you are using whether it is RPC style or document and that a service should be atomic and not hold external state. Meaning it does not need to know any more information other than that in the SOAP message to perform its function.
Hope this gives you a good starting point. The trouble with service-orientation is that it differs depending on who you read, but the main points stay the same!
Jon
Some web services which are provided for clients are abstract and composition of some smaller web services and it's called web service composition.
Sometimes there are more than one web service in order to use as the mentioned small web services, so we choose them based on QoS (Quality of Service) and many researches have been done on this subject.
Web service composition involves integration of two or more web service to achieve more added value of business functionality. A work flow composer is responsible of aggregating different web services to act as a single service according to functional requirements as well as QoS constrains. BPEL is one of the popular composers uses XML language to perform service composition. Fine-grained services perform single business task and provides higher flexibility and reusability. However, coarse-grained service involves performing complex business functionality leading to lower flexibility