Is this a valid way to multi thread RPC handlers in gRPC async server in C++? - c++

Question

I'm trying to work on the basic async gRPC C++ code as mentioned in https://github.com/grpc/grpc/tree/master/examples/cpp/helloworld
Aim is to experiment multi threading of RPC handlers. Can someone let me know that whether the below logic is correct?
Few questions,
Is the completion queue thread-safe?
I see that there is a possibility of one instance of CallData being accessed in multiple threads (returned as part of tag). Is CallData thread-safe here or do we need to have a mutex for the same?
Please note that this is an incomplete program.
class ServerImpl final {
public:
~ServerImpl() {
server_->Shutdown();
// Always shutdown the completion queue after the server.
cq_->Shutdown();
}
// There is no shutdown handling in this code.
void Run() {
std::string server_address("0.0.0.0:50051");
ServerBuilder builder;
// Listen on the given address without any authentication mechanism.
builder.AddListeningPort(server_address, grpc::InsecureServerCredentials());
// Register "service_" as the instance through which we'll communicate with
// clients. In this case it corresponds to an *asynchronous* service.
builder.RegisterService(&service_);
// Get hold of the completion queue used for the asynchronous communication
// with the gRPC runtime.
cq_ = builder.AddCompletionQueue();
// Finally assemble the server.
server_ = builder.BuildAndStart();
std::cout << "Server listening on " << server_address << std::endl;
// Proceed to the server's main loop.
std::thread thread1(HandleRpcsHelper, 1, this);
sleep(1);
std::thread thread2(HandleRpcsHelper, 2, this);
//HandleRpcs();
thread1.join();
thread2.join();
}
private:
// Class encompasing the state and logic needed to serve a request.
class CallData {
public:
// Take in the "service" instance (in this case representing an asynchronous
// server) and the completion queue "cq" used for asynchronous communication
// with the gRPC runtime.
CallData(Greeter::AsyncService* service, ServerCompletionQueue* cq)
: service_(service), cq_(cq), responder_(&ctx_), status_(CREATE) {
// Invoke the serving logic right away.
Proceed();
}
void Proceed() {
if (status_ == CREATE) {
std::cout << "CREATE " << this << std::endl;
// Make this instance progress to the PROCESS state.
status_ = PROCESS;
// As part of the initial CREATE state, we *request* that the system
// start processing SayHello requests. In this request, "this" acts are
// the tag uniquely identifying the request (so that different CallData
// instances can serve different requests concurrently), in this case
// the memory address of this CallData instance.
service_->RequestSayHello(&ctx_, &request_, &responder_, cq_, cq_,
this);
} else if (status_ == PROCESS) {
// Spawn a new CallData instance to serve new clients while we process
// the one for this CallData. The instance will deallocate itself as
// part of its FINISH state.
std::cout << "PROCESS " << this << std::endl;
new CallData(service_, cq_);
// The actual processing.
std::string prefix("Hello ");
reply_.set_message(prefix + request_.name());
// And we are done! Let the gRPC runtime know we've finished, using the
// memory address of this instance as the uniquely identifying tag for
// the event.
status_ = FINISH;
std::cout << "SETTING TO FINISH " << this << std::endl;
responder_.Finish(reply_, Status::OK, this);
} else {
std::cout << "FINISH " << this <<std::endl;
GPR_ASSERT(status_ == FINISH);
// Once in the FINISH state, deallocate ourselves (CallData).
delete this;
}
}
private:
// The means of communication with the gRPC runtime for an asynchronous
// server.
Greeter::AsyncService* service_;
// The producer-consumer queue where for asynchronous server notifications.
ServerCompletionQueue* cq_;
// Context for the rpc, allowing to tweak aspects of it such as the use
// of compression, authentication, as well as to send metadata back to the
// client.
ServerContext ctx_;
// What we get from the client.
HelloRequest request_;
// What we send back to the client.
HelloReply reply_;
// The means to get back to the client.
ServerAsyncResponseWriter<HelloReply> responder_;
// Let's implement a tiny state machine with the following states.
enum CallStatus { CREATE, PROCESS, FINISH };
CallStatus status_; // The current serving state.
};
static void HandleRpcsHelper(int id, ServerImpl *server)
{
std::cout<< "ID:" << id << std::endl;
server->HandleRpcs(id);
}
// This can be run in multiple threads if needed.
void HandleRpcs(int id) {
// Spawn a new CallData instance to serve new clients.
new CallData(&service_, cq_.get());
void* tag; // uniquely identifies a request.
bool ok;
while (true) {
// Block waiting to read the next event from the completion queue. The
// event is uniquely identified by its tag, which in this case is the
// memory address of a CallData instance.
// The return value of Next should always be checked. This return value
// tells us whether there is any kind of event or cq_ is shutting down.
std::cout << "waiting... " << id << std::endl;
GPR_ASSERT(cq_->Next(&tag, &ok));
GPR_ASSERT(ok);
std::cout << "wakeup... " << id <<std::endl;
static_cast<CallData*>(tag)->Proceed();
}
}
std::unique_ptr<ServerCompletionQueue> cq_;
Greeter::AsyncService service_;
std::unique_ptr<Server> server_;
};
int main(int argc, char** argv) {
ServerImpl server;
server.Run();
return 0;
}

Answer 1

First of all, you may want to read gRPC Performance Best Practices for this topic.
Is the completion queue thread-safe?
Yes
I see that there is a possibility of one instance of CallData being accessed in multiple threads (returned as part of tag). Is CallData thread-safe here or do we need to have a mutex for the same?
No because your code creates a new CallData instance for every RPC calls so it shouldn't be accessed by multiple completion queue. If you don't have instances among multiple CallData instances (i.e. static member variables), you don't need to worry about synchronization.