I have several Go and C++ services, that act as servers and clients, i.e. those are long-living services, that from time-to-time talk to other services.
I cannot find anywhere in gRPC docs whether the client channels and stubs should be long-lasting reusable objects or they should be created on every call. Or, maybe, I should create some kind of a channel/stub pool, where a pair of a channel and a stub exists, for example, for each thread and gets updated, refreshed on some interval or condition.
According to official docs, you are encouraged to reuse stubs and channels when possible.
https://grpc.io/docs/guides/performance/
Related
Is there a common way to establish a network connection from a CloudFoundry-Service to a CloudFoundry App which the service is bound to.
In typical fashion apps receive their bind credentials and establish network connections to provisioned service for example databases.
It would be very handy to establish a connection from a service to an app, so the service could scrape endpoints that are provided by the app.
Any thoughts on this, why is it / or isn't it possible, why could it be a bad idea.
Normally, you have your service and the application receives credentials from the service through the service binding (i.e. VCAP_SERVICES).
You want to reverse this arrangement, which is fine, but the service will need to have some way to know how to reach the applications. The way to do this would be through routes bound to your application.
I have seen something like this done before, this is roughly the process. I'm sure you can adapt it to your requirements.
Create a service broker. The broker is responsible for managing service instances and service credentials. The broker is notified when an instance is created and when a binding occurs. Your broker will need to handle these requests.
The broker, in addition to its normal responsibilities, is going to need to maintain state indicating which applications have instances & bindings. In addition, the broker is going to need to use the org/space/app guids it's provided through the service broker API and talk to the CloudFoundry API to fetch the routes for the applications that are bound to it. You don't usually get these through the service broker API, but since you want to talk to the applications from the service, you need this information. It gives the service a way to communicate with the application.
Your broker may also provide the service in question (i.e. talking to applications), or it can delegate to some other process/container/VM to provide the service. If your service does the latter, then you need a way to a.) create the process/container/VM and b.) pass along the information it requires to talk to your application.
Obviously, you need to code the logic that will take the routes for applications that have created instances and bindings and communicate with them.
There can be some limitations with using the routes. First, not all routes are public. For internal routes, it would be kind of complicated to allow the broker/service to talk to the app. The broker/service would need to be an application on CF and you would need to specifically allow that communication (would require more API calls). Second, some apps just don't have routes. Perhaps this won't happen in your case, but it's worth considering. Lastly, not all routes are HTTP, some can be TCP as well. Your broker/service would need to handle both of those.
A variation on the above process, instead of using routes or talking to the API, you could have your broker/service provide some mechanism through the credentials to the application such that it registers itself with the broker/service. Thus when your applications start, they'll read the service info, register with the service and then go about their business. In this way, the application would have some additional flexibility about what information it provides when it registers with the broker/service. The downside is that the app has to do some work to be compatible.
The gRPC C++ API for creating channels returns a shared_ptr. The generated function NewStub returns a unique_ptr. However I've seen reports of people having issues trying to create multiple instances of a stub type, sharing a channel. Their solution was to share the stub.
It is not clear from the documentation or API if a client is meant to create multiple stub instances sharing the channel or share a single stub. Please clarify the conceptual relationship between stubs, channels, and unique client connections.
Diving a little deeper:
A server can serve more than one service and a client endpoint can use a single channel to connect corresponding stub types to each of those services. For that purpose, it is clear the different stub types share the single channel to talk to the server endpoint. Does gRPC expect only one client per channel for a given service or can I have multiple clients on the client endpoint talking to a single service? If allowed, how do I achieve multiple clients for a given service on a client endpoint? How does the server distinguish these as independent clients?
As an aside, This SO post indicates both Channels and Stubs are thread-safe. (The post is specifically for Java, but I'm assuming it carries over to C++).
I think first we need to clarify the definitions for channel and stub, according to the official documents :
channel :
A gRPC channel provides a connection to a gRPC server on a specified host and port...
Conclude : A channel represents a single TCP connection.
stub :
On the client side, the client has a local object known as stub (for some languages, the preferred term is client) that implements the same methods as the service.
Conclude : A stub represents a single client.
From reading many other materials, I'm sure that a single channel(tcp connection) can be multiplexed, and now we get two options to achieve it:
one channel <--> one stub
one stub <--> multiple streams
one channel <--> multiple stubs
one stub <--> multiple streams
the only difference is that whether to share a channel among the stubs or not. My answer is: Yes, you can, for the reasons:
your reports of people having issues example is written in ruby, but I can also find python examples standing on the opposite side. So the behavior may depends on language implementations.
the synchronous cpp client examples use one stub object to issue a rpc, and it doesn't have any stream objects associated with, then the only way for multiplexing purpose is just share one single channel object among the stubs.
Combining the above two reasons, I conclude that: sharing a channel among the stubs is valid in C++.
Now, back to you questions:
Does gRPC expect only one client per channel for a given service or can I have multiple clients on the client endpoint talking to a single service?
Sure, you can have multiple clients talking to a single service.
If allowed, how do I achieve multiple clients for a given service on a client endpoint?
You can just have multiple stubs generated either from one single channel or from multiple channels, former is a better choice for connection overhead considerations.
How does the server distinguish these as independent clients?
This is due to http2, it can be multiplexed and has its own rules to achieve this, what grpc need to do is just follow thoes rules.
Hope this helps!
In developing backend components, I need to decide how these components will interact and communicate with each other. In particular, I need to decide whether it is better to use (RESTful, micro) web services versus a message broker (e.g. RabbitMQ). Are there certain criteria to help decide between using web services for each component versus messaging?
Eranda covered some of this in his answer, but I think three of the key drivers are:
Are you modeling a Request-Response type interaction?
Can your interaction be asynchronous?
How much knowledge does the sender of the information need to have about the recipients?
It is possible to do Request-Response type interactions with an asynchronous messaging infrastructure but it adds significantly to the complexity, so generally Request-Response type interactions (i.e. does the sender need some data returned from the recipient) are more easily modeled as RPC/REST interactions.
If your interaction can be asynchronous then it is possible to implement this using a REST interaction but it may scale better if you use a fire and forget messaging type interaction.
An asynchronous messaging interaction will also be much more appropriate if the provider of the information doesn't care who is consuming the information. An information provider could be publishing information and new consumers of that information could be added to the system later without having to change the provider.
Web server and message broker have their own use cases. Web server used to host web services and the message broker are use to exchange messages between two points. If you need to deploy a web service then you have to use a web server, where you can process that message and send back a response. Now let's think that you need to have publisher/subscriber pattern or/and reliable messaging between any two nodes, between two servers, between client and server, or server and client, that's where the message broker comes into the picture where you can use a message broker in the middle of two nodes to achieve it. Using message broker gives you the reliability but you have to pay it with the performance. So the components you should use depends on your use case though there are multiple options available.
Following is a point mentioned in a presentation slide related to SOA, and it confuses me with the concepts of service orchestration and service choreography. To enable service choreography, shouldn't a web service be able to call another web service?
SOA builds applications out of software services. Services comprise intrinsically
unassociated, loosely coupled units of functionality that have no calls to
each other embedded in them.
In theory, a service can do anything it needs to do to accomplish its job. So there doesn't seem to be a good reason to forbid using a second service to do your work. Why reinvent the wheel?
In practice, the issue is more complicated. If you start calling other services on your own web server, then you'll eventually starve it of resources. At best, "real" clients will have to wait a bit longer for their answers while your web service server plays with itself.
Another issue is recursive loops: Service A calls B calls C calls A calls B ... you get the idea. A small change in one service can introduce such a loop without anyone noticing and it can sit there for a long time until it suddenly kills your server.
That is why you should build micro services in a hierarchy inside the server (i.e. below the web service layer - this is not exposed to clients). Those micro services can use each other in a top-down manner (to avoid the loops). Unit tests then make sure they behave properly.
Lastly, such reuse is very slow. Each HTTP request takes a lot of resources to create, send, parse and process. Calling an internal method directly can be 10 - 10000 times faster.
These are the main reasons why the services exposed by a single server shouldn't reuse each other via the "public client API".
Note: There are web services which build new services by using existing ones. IFTTT - "If This Then That" is one such beast.
You could adopt every concept according to your needs. In my current project we have a separate module that is responsible for the Orchestration. This is required since in real life usage, scenarios can be very complicated. So in order to be close to the actual management of your system, you need to have such one.
Another advantage of this approach is that the Separation_of_concerns is kept. Also aligns the business request with the applications, data, and infrastructure that you have. It defines policies and service levels through automated workflows, provisioning etc.
Orchestration is critical in the delivery of Cloud services too. As they are networked to allow sharing of data-processing tasks, centralized data storage, and online access to services or resources.
What would be a more standard way of notifying a web service consumer of a data change?
Having the consumer periodically calling the web service to pull change notification.
Consumer setting up a call back web service that can be invoked to forward notification about the change.
Other?
Both of these are options. There is also something called "comet" which is like setting up a stream between between the consumer and producer - messages can then be passed back and forth between the two. Wikipedia is probably the best place to start investigating to see if it will work for you project: http://en.wikipedia.org/wiki/Comet_(programming)
Depends on the scenario. If you're working in a closed environment with only a few consumers of your service, you could switch to a COMET style service which allows a callback from the service to the client. More about that here:
Wikipedia - COMET
From what I've read, that method doesn't scale well in larger environments so I'd be careful.
The more traditional method is your first option of polling the service for changes. As long as your service performs well and you have the appropriate hardware to serve up the requests, it's probably your best bet for a public facing web service.
In case you weren't aware of it, and in case it helps: WCF can work with a Duplex contract that in effect creates a callback service contract on the client. It's fairly transparent.