I have two systems, one (let's call it S1) that exposes a RESTful API (let's call it WS1) and another one (let's call it S2) that exposes a SOAP API (let's call it WS2).
I'm trying to figure out a way to get data from S1 and add it into S2.
WS1 exposes methods for both adding/getting data (into/from S1) while WS2 only has methods for adding data (into S2).
Can these two web service communicate directly between each other or should there be some kind of mechanism in between. My guess is that "somebody should be managing their discussion".
Where are the clients?
You have a REST Server and a SOAP Server, both of their goals is to wait for requests from their respective clients -- it doesn't matter if the operations they implement read and/or write their data sets, a client still needs to initiate communication.
Bridge the gap.
Because of this you'll need a bridging client to request to read something from the REST Server and request to write something into the SOAP Server. The rest of the infrastructure for the bridge is up to you.
You can write a light script that pulls RESTful data and pushes SOAP messages for a handful of particular RESTful resources or you can write a general purpose REST2SOAP bridge which can map a RESTful resource to a SOAP message endpoint based on a conversion convention.
Direct vs. Message Queues.
Writing an abstract bridge client will allow you to run it by directly calling the REST service, receiving data, processing it, directly calling the SOAP service and sending it the data. If this is a low-load situation that's fine.
If we have a high load of data to process doing things synchronously will not be feasible, so we introduce message queues.
The producer:
reads data from the REST service;
(perhaps) processes it into a local form (which the bridge client understands such as arrays and objects);
then serializes it (serialize, json_encode, etc.);
puts it on a message queue.
The consumer(s)
listen to the message queue;
when it receives a new message it deserializes it;
processes it;
sends it to the SOAP service.
The overall advantage of the message queues is the fact that you can start up as many producers or consumers as you need depending on which of these services runs slower.
Related
I have a dynamic peer to peer network where nodes communicates with grpc. Each node has its own server and client. One grpc method is defined for login of a new Node. I use a sinchronous message to communicate the login to all others, where I create a new channel with each other servers, send a single message and wait a response.
rpc enter(LogIn) returns (Response);
If I have one node in my network (node 1) and then two or more nodes enter at the same time, for example node 2 and node 3, they'll call both the grpc method "enter" on node1's server. With this type of method, which is the behavior of node1's server? It's able to manage both this request? so with a method like this, grpc queues messages that arrive concurrently or will it handle only one request?
thanks
gRPC supports concurrent execution of multiple RPCs. When an RPC (or RPC event) arrives, it is queued on the serverBuilder.executor() specified when building the server. If you don't specify, a default executor is used that uses as many threads as necessary. Nothing in gRPC will behave differently if the RPCs are the same RPC method or different RPC methods.
In restful web service I read something like below ,
"The constraint to the client-server interaction is that communication must be stateless. The server should not be relied upon to maintain application state by storing session objects."
so does it mean in SOAP web services the server saves the session with them ? I have used soap user interface tool for testing the soap services in which i will be sending the request XML with all the parameters and will be getting the response, In which way restful web services differed in terms of statelessness from Soap ?
No it does not mean SOAP is stateful in all cases. Typically you design all services as stateless. Having stateful services either in SOAP or REST introduces complexity which creates all sort of problems.
However some people implement both SOAP/REST services with state and they normally aren't very successful. Maintaining state requires both the server and client to be aware of and track the current state.
So in short the principle remains never store state in a service. When calling an operation it should either return a success or failed code, it should not remember that the last operation for this client did not succeed.
Also while REST says services should be stateless you can implement it in a stateful method. Never ever go stateful it creates a lot of confusion for service consumers i.e. the client. Think about it this way if an operation is stateless you can call it and get repeatable results however if it is stateful an operation might change the results based on the state.
Also stateful operations limits scalability as it has to remember more.
In developing backend components, I need to decide how these components will interact and communicate with each other. In particular, I need to decide whether it is better to use (RESTful, micro) web services versus a message broker (e.g. RabbitMQ). Are there certain criteria to help decide between using web services for each component versus messaging?
Eranda covered some of this in his answer, but I think three of the key drivers are:
Are you modeling a Request-Response type interaction?
Can your interaction be asynchronous?
How much knowledge does the sender of the information need to have about the recipients?
It is possible to do Request-Response type interactions with an asynchronous messaging infrastructure but it adds significantly to the complexity, so generally Request-Response type interactions (i.e. does the sender need some data returned from the recipient) are more easily modeled as RPC/REST interactions.
If your interaction can be asynchronous then it is possible to implement this using a REST interaction but it may scale better if you use a fire and forget messaging type interaction.
An asynchronous messaging interaction will also be much more appropriate if the provider of the information doesn't care who is consuming the information. An information provider could be publishing information and new consumers of that information could be added to the system later without having to change the provider.
Web server and message broker have their own use cases. Web server used to host web services and the message broker are use to exchange messages between two points. If you need to deploy a web service then you have to use a web server, where you can process that message and send back a response. Now let's think that you need to have publisher/subscriber pattern or/and reliable messaging between any two nodes, between two servers, between client and server, or server and client, that's where the message broker comes into the picture where you can use a message broker in the middle of two nodes to achieve it. Using message broker gives you the reliability but you have to pay it with the performance. So the components you should use depends on your use case though there are multiple options available.
Hi to all the experts out there :)
This is my first question here.
Problem Description :
I have to write a Market Data Feed Handler. This is going to be a Windows Service, will be using two Sockets.
Socket A : For communication between Subscribing applications and Feed Handler (Feed Handler will be accepting the connection request and the Item Request).
Socket B : For communication between Feed Handler and External Market Data provider, like Reuters/Bloomberg.
In both the cases Request/Response will be using the same port.
Note : The volume of Data coming from the external system is low (External system will only send the information which has been subscribed for, at this point of time).
However later on we may want to scale it, some providers throw all the data, and Feed Handler has to filter out locally, based on the subscription.
My questions :
What threading model i should use?
Which I/O strategy i should use?
Keeping in mind both the cases, should i create separate Request/Response thread?
EDIT 1: After reading few tutorials on Winsock, i'm planning to use Event Objects for asynchronous behavior.
The point of concern here is that, a single thread should listen for incoming client connections (Accept them) and also Connect to other server, in turn send/recv on two different ports.
Thread A
1) Listening for incoming connections. (Continuous)
2) Receiving Subscribe/Unsubscribe request from connected clients. (Rarely)
3) Connect to the external server (Onetime only).
4) Forward the request coming from client to the external server. (Rarely)
5) Receive data from external server. (Continuous)
6) send this data back to the connected clients. (Continuous)
My question is can a single thread act as both Client and Server, using asynchronous I/O models?
Thanks in advance.
Deepak
The easiest threading model seems to be single threaded synchronous. If you need to implement a filter for a provider, implement that as a socket-in/socket-out separate process.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
Under what conditions would one favor apps talking via a message queue instead of via web services (I just mean XML or JSON or YAML or whatever over HTTP here, not any particular type)?
I have to talk between two apps on a local network. One will be a web app and have to request commands on another app (running on different hardware). The requests are things like creating users, moving files around, and creating directories. Under what conditions would I prefer XML Web Services (or straight TCP or something) to using a Message queue?
The web app is Ruby on Rails, but I think the question is broader than that.
When you use a web service you have a client and a server:
If the server fails the client must take responsibility to handle the error.
When the server is working again the client is responsible of resending it.
If the server gives a response to the call and the client fails the operation is lost.
You don't have contention, that is: if million of clients call a web service on one server in a second, most probably your server will go down.
You can expect an immediate response from the server, but you can handle asynchronous calls too.
When you use a message queue like RabbitMQ, Beanstalkd, ActiveMQ, IBM MQ Series, Tuxedo you expect different and more fault tolerant results:
If the server fails, the queue persist the message (optionally, even if the machine shutdown).
When the server is working again, it receives the pending message.
If the server gives a response to the call and the client fails, if the client didn't acknowledge the response the message is persisted.
You have contention, you can decide how many requests are handled by the server (call it worker instead).
You don't expect an immediate synchronous response, but you can implement/simulate synchronous calls.
Message Queues has a lot more features but this is some rule of thumb to decide if you want to handle error conditions yourself or leave them to the message queue.
There's been a fair amount of recent research in considering how REST HTTP calls could replace the message queue concept.
If you introduce the concept of a process and a task as a resource, the need for middle messaging layer starts to evaporate.
Ex:
POST /task/name
- Returns a 202 accepted status immediately
- Returns a resource url for the created task: /task/name/X
- Returns a resource url for the started process: /process/Y
GET /process/Y
- Returns status of ongoing process
A task can have multiple steps for initialization, and a process can return status when polled or POST to a callback URL when complete.
This is dead simple, and becomes quite powerful when you realize that you can now subscribe to an rss/atom feed of all running processes and tasks without any middle layer. Any queuing system is going to require some sort of web front end anyway, and this concept has it built in without another layer of custom code.
Your resources exist until you delete them, which means you can view historical information long after the process and task complete.
You have built in service discovery, even for a task that has multiple steps, without any extra complicated protocols.
GET /task/name
- returns form with required fields
POST (URL provided form's "action" attribute)
Your service discovery is an HTML form - a universal and human readable format.
The entire flow can be used programmatically or by a human, using universally accepted tools. It's a client driven, and therefore RESTful. Every tool created for the web can drive your business processes. You still have alternate message channels by POSTing asynchronously to a separate array of log servers.
After you consider it for a while, you sit back and start to realize that REST may just eliminate the need for a messaging queue and an ESB altogether.
http://www.infoq.com/presentations/BPM-with-REST
Message queues are ideal for requests which may take a long time to process. Requests are queued and can be processed offline without blocking the client. If the client needs to be notified of completion, you can provide a way for the client to periodically check the status of the request.
Message queues also allow you to scale better across time. It improves your ability to handle bursts of heavy activity, because the actual processing can be distributed across time.
Note that message queues and web services are orthogonal concepts, i.e. they are not mutually exclusive. E.g. you can have a XML based web service which acts as an interface to a message queue. I think the distinction your looking for is Message Queues versus Request/Response, the latter is when the request is processed synchronously.
Message queues are asynchronous and can retry a number of times if delivery fails. Use a message queue if the requester doesn't need to wait for a response.
The phrase "web services" make me think of synchronous calls to a distributed component over HTTP. Use web services if the requester needs a response back.
I think in general, you'd want a web service for a blocking task (this tasks needs to be completed before we execute more code), and a message queue for a non-blocking task (could take quite a while, but we don't need to wait for it).