I am using gRPC sync api with C++.
Here is how on server side I am checking if the client has stopped the stream.
grpc::Status AuthServer::ConnectServiceImpl::HearthBeat(grpc::ServerContext *context,
grpc::ServerReaderWriter<Pulse, Pulse> *stream) {
Pulse note;
if(ctx_.IsCancelled()){
std::cout << "DISCONNECT" << std::endl;
}
while (stream->Read(¬e)) {
Pulse reply;
reply.set_rate(note.rate()+1);
std::cout << "RECEIVED: " << note.rate() << std::endl;
stream->Write(reply);
}
return grpc::Status::OK;
}
This is bidi stream which is stopped forcefully on client side with killing the client app and still the "DISCONNECT" message does not appear.
Why is that, am I using IsCancelled() not correctly?
I think I already answered this in GRPC/C++ - How to detect client disconnected in Async Server.
Your code appears to be checking IsCancelled() on ctx_. I'm not sure what that object is, but the context you want to be checking is the one passed into the request handler method as context.
Related
I am writing a plugin that interfaces with a desktop application through a ZeroMQ REQ/REP request-reply communication archetype. I can currently receive a request, but the application seemingly crashes if a reply is not sent quick enough.
I receive the request on a spawned thread and put it in a queue. This queue is processed in another thread, in which the processing function is invoked by the application periodically.
The message is correctly being received and processed, but the response cannot be sent until the next iteration of the function, as I cannot get the data from the application until then.
When this function is conditioned to send the response on the next iteration, the application will crash. However, if I send fake data as the response soon after receiving the request, in the first iteration, the application will not crash.
Constructing the socket
zmq::socket_t socket(m_context, ZMQ_REP);
socket.bind("tcp://*:" + std::to_string(port));
Receiving the message in the spawned thread
void ZMQReceiverV2::receiveRequests() {
nInfo(*m_logger) << "Preparing to receive requests";
while (m_isReceiving) {
zmq::message_t zmq_msg;
bool ok = m_respSocket.recv(&zmq_msg, ZMQ_NOBLOCK);
if (ok) {
// msg_str will be a binary string
std::string msg_str;
msg_str.assign(static_cast<char *>(zmq_msg.data()), zmq_msg.size());
nInfo(*m_logger) << "Received the message: " << msg_str;
std::pair<std::string, std::string> pair("", msg_str);
// adding to message queue
m_mutex.lock();
m_messages.push(pair);
m_mutex.unlock();
}
std::this_thread::sleep_for(std::chrono::milliseconds(100));
}
nInfo(*m_logger) << "Done receiving requests";
}
Processing function on seperate thread
void ZMQReceiverV2::exportFrameAvailable()
// checking messages
// if the queue is not empty
m_mutex.lock();
if (!m_messages.empty()) {
nInfo(*m_logger) << "Reading message in queue";
smart_target::SMARTTargetCreateRequest id_msg;
std::pair<std::string, std::string> pair = m_messages.front();
std::string topic = pair.first;
std::string msg_str = pair.second;
processMsg(msg_str);
// removing just read message
m_messages.pop();
//m_respSocket.send(zmq::message_t()); wont crash if I reply here in this invocation
}
m_mutex.unlock();
// sending back the ID that has just been made, for it to be mapped
if (timeToSendReply()) {
sendReply(); // will crash, if I wait for this to be exectued on next invocation
}
}
My research shows that there is no time limit for the response to be sent, so this, seeming to be, timing issue, is strange.
Is there something that I am missing that will let me send the response on the second iteration of the processing function?
Revision 1:
I have edited my code, so that the responding socket only ever exists on one thread. Since I need to get information from the processing function to send, I created another queue, which is checked in the revised the function running on its own thread.
void ZMQReceiverV2::receiveRequests() {
zmq::socket_t socket = setupBindSocket(ZMQ_REP, 5557, "responder");
nInfo(*m_logger) << "Preparing to receive requests";
while (m_isReceiving) {
zmq::message_t zmq_msg;
bool ok = socket.recv(&zmq_msg, ZMQ_NOBLOCK);
if (ok) {
// does not crash if I call send helper here
// msg_str will be a binary string
std::string msg_str;
msg_str.assign(static_cast<char *>(zmq_msg.data()), zmq_msg.size());
NLogger::nInfo(*m_logger) << "Received the message: " << msg_str;
std::pair<std::string, std::string> pair("", msg_str);
// adding to message queue
m_mutex.lock();
m_messages.push(pair);
m_mutex.unlock();
}
std::this_thread::sleep_for(std::chrono::milliseconds(100));
if (!sendQueue.empty()) {
sendEntityCreationMessage(socket, sendQueue.front());
sendQueue.pop();
}
}
nInfo(*m_logger) << "Done receiving requests";
socket.close();
}
The function sendEntityCreationMessage() is a helper function that ultimately calls socket.send().
void ZMQReceiverV2::sendEntityCreationMessage(zmq::socket_t &socket, NUniqueID id) {
socket.send(zmq::message_t());
}
This code seems to be following the thread safety guidelines for sockets. Any suggestions?
Q : "Is there something that I am missing"
Yes,the ZeroMQ evangelisation, called a Zen-of-Zero, since ever promotes never try to share a Socket-instance, never try to block and never expect the world to act as one wishes.
This said, avoid touching the same Socket-instance from any non-local thread, except the one that has instantiated and owns the socket.
Last, but not least, the REQ/REP-Scalable Formal Communication Pattern Archetype is prone to fall into a deadlock, as a mandatory two-step dance must be obeyed - where one must keep the alternating sequence of calling .send()-.recv()-.send()-.recv()-.send()-...-methods, otherwise the principally distributed-system tandem of Finite State Automata (FSA) will unsalvageably end up in a mutual self-deadlock state of the dFSA.
In case one is planning to professionally build on ZeroMQ, the best next step is to re-read the fabulous Pieter HINTJENS' book "Code Connected: Volume 1". A piece of a hard read, yet definitely worth one's time, sweat, tears & efforts put in.
I use the grpc cpp example "helloworold" code to test limit handle thread. But I can't find any way to do it.
grpc version: 1.15
linux: ubuntu 16.04
I set the builder like this:
builder.SetSyncServerOption(ServerBuilder::SyncServerOption::MIN_POLLERS, 1);
builder.SetSyncServerOption(ServerBuilder::SyncServerOption::MAX_POLLERS, 1);
builder.SetSyncServerOption(ServerBuilder::SyncServerOption::NUM_CQS, 1);
set the handle like this:
class GreeterServiceImpl final : public Greeter::Service {
Status SayHello(ServerContext* context, const HelloRequest* request,
HelloReply* reply) override {
std::string prefix("Hello ");
std::cout << "start " << std::this_thread::get_id() << std::endl;
reply->set_message(prefix + request->name());
//**** sleep 5s, keep this thread block ****
std::this_thread::sleep_for(std::chrono::seconds(5));
std::cout << "end " << std::this_thread::get_id() << std::endl;
return Status::OK;
}
};
I use the example client and call SayHello in 100 threads, and server log show the thread is created by 100 times.
In this test, is my test way wrong? or somethings miss setup??
You can use SetMaxThread in this way:
grpc::ResourceQuota rq;
rq.SetMaxThreads(n);
builder.SetResourceQuota(rq);
It seems that a thread is needed for every completion queue. So if n=4 when you have 1 completion queue, 3 threads are remained for processing requests.
What you are using is the sync API, which will initiate a thread per call. You can look at the async API to reduce the number of threads.
Current, I'm using a hackish way – a global variable – to make RPC handlers able to detect that the Server has been (about to be) called Shutdown().
bool g_ServerIsNotDead = true; // Hack!
Status StreamServiceImpl::GetCurrentTemperature(ServerContext *context_,
const UpdateInterval *request_,
ServerWriter<Temperature> *stream_)
{
auto currentTemp = 100.0f;
while(g_ServerIsNotDead) // Hack!!!
{
qDebug() << QThread::currentThreadId() << currentTemp << "farenheit.";
Temperature message;
message.set_temperature(currentTemp);
stream_->Write(message);
QThread::sleep(2);
currentTemp += 1.0f;
}
return Status::OK;
}
void insideSomeFunction() {
// Testing shutdown 5 seconds later
QTimer::singleShot(std::chrono::seconds(5), this, [=]() {
qDebug() << "Shuting down!";
g_ServerIsNotDead = false; // Hack!!
this->server->Shutdown(); // This method actually blocks until all RPC handlers have exited, believe it or not!
emit shutdown();
qDebug() << "All dead.";
});
}
Ref: https://github.com/C0D1UM/grpc-qt-example/blob/master/rpc_server/hellostream_server.cpp
It would be really nice if I could somehow check that Server has been Shutdown() from grpc::ServerContext, but I didn't see any relevant methods to achieve this.
Even better if someone could propose a way to take out the while loop completely (?). I'm using Qt so everything is event-driven.
So, I think it's worth making clear what Shutdown does. There's a detailed comment about this but basically, server Shutdown doesn't fail, cancel, or kill your existing in-progress calls (unless you use the deadline argument and the gRPC C++ async API).
Rather, it stops listening for new connections, stops accepting new calls, fails requested-but-not-yet-accepted calls. If you want to fail or terminate your calls at shutdown, you can do it at application-level code as you've done above.
I would just recommend that instead of using a global variable, you should use a member function of your StreamServiceImpl class so that you can support multiple services running in the same process if you choose.
We can use ServerContext::IsCancelled as a breaking/termination criteria in streaming APIs. I changed GetCurrentTemperature(...) as follows (just replaced g_ServerIsNotDead with !context_->IsCancelled()) and it worked:
Status StreamServiceImpl::GetCurrentTemperature(ServerContext *context_,
const UpdateInterval *request_,
ServerWriter<Temperature> *stream_) {
auto currentTemp = 100.0f;
while(!context_->IsCancelled) {
qDebug() << QThread::currentThreadId() << currentTemp << "farenheit.";
Temperature message;
message.set_temperature(currentTemp);
stream_->Write(message);
QThread::sleep(2);
currentTemp += 1.0f;
}
return Status::OK;
}
I'm sending a HTTP GET request to a remote server. Using various parameters I define the content I'm interested in. In particular I make sure that output=xml is in the query since it makes the server return a reply as XML.
I have the following connections between my class HttpRetriever and the respective QNetworkReply and QNetworkAccessManager (for the QNetworkRequest see slotStartRequest()):
connect(this->reply, SIGNAL(error(QNetworkReply::NetworkError)), this,
SLOT(slotErrorRequest(QNetworkReply::NetworkError)));
connect(this->reply, &QNetworkReply::finished, this, &HttpRetriever::slotFinishRequest);
connect(this->manager, &QNetworkAccessManager::finished, this->reply, &QNetworkReply::deleteLater);
connect(this->reply, &QIODevice::readyRead, this, &HttpRetriever::slotReadyReadRequest);
The slots that are of interest here have the following declaration:
slotFinishRequest():
void HttpRetriever::slotFinishRequest()
{
LOG(INFO) << "Finishing HTTP GET request from URL \"" << this->url.toString() << "\"";
this->reply = Q_NULLPTR;
// Reset validity of reply from a previous request
this->validReply = false;
// Skip validation if it's disabled
if (!this->validateReply)
{
LOG(WARNING) << "Validation disabled. In order to enable it see the \"validate\" and \"validationMode\" in \"settings.ini\"";
this->validReply = true;
}
else
{
// Validate reply
this->validReply = validateReply();
}
if (!this->validReply)
{
return;
}
processReply(); // Parsing
this->processingRequest = false;
}
slotReadyReadRequest():
void HttpRetriever::slotReadyReadRequest()
{
LOG(INFO) << "Received data from \"" << this->url.toString() << "\"";
this->bufferReply = this->reply->readAll();
}
Inside the slotFinishRequest() I call the processReply():
void HttpRetriever::processReply()
{
LOG(INFO) << "Processing reply for request \"" << this->url.toString() << "\"";
LOG(DEBUG) << QString(this->bufferReply);
// Process the XML from the reply and extract necessary data
QXmlStreamReader reader;
reader.addData(this->bufferReply);
// Read the XML reply and extract required data
// TODO
while (!reader.atEnd())
{
LOG(DEBUG) << "Reading XML element";
reader.readNextStartElement();
QXmlStreamAttributes attributes = reader.attributes();
foreach (QXmlStreamAttribute attrib, attributes)
{
LOG(DEBUG) << attrib.name();
}
}
if (reader.hasError())
{
LOG(ERROR) << "Encountered error while parsing XML data:" << reader.errorString();
}
LOG(INFO) << "Sending data to data fusion handler";
// TODO
}
I trigger the HTTP get request through the following slot:
void HttpRetriever::slotStartRequest(quint32 id)
{
if (this->processingRequest)
{
this->reply->abort();
}
this->processingRequest = false;
// The first request also instantiates the manager. If the slot is called after the instance of HafasHttpRetriever
// is moved to a new thread this will ensure proper parenting
if (!this->manager)
{
this->manager = new QNetworkAccessManager(this);
}
quint32 requestId = generateRequestId(stopId);
if (!this->url.hasQuery())
{
LOG(WARNING) << "Attempting to start request without parameters";
}
// Part of the filters applied to the request to reduce the data received (for more see slotSetRequestParams())
QUrlQuery query(this->url.query());
query.addQueryItem("input", QString::number(requestId));
// TODO Add more filters; see documentation
this->url.setQuery(query);
LOG(INFO) << "Requesting data from \"" << this->url.toString() << "\" with request ID:" << requestId;
QNetworkRequest request(this->url);
this->reply = this->manager->get(request);
// Establish connections from/to the reply and the network access manager
connect(this->reply, SIGNAL(error(QNetworkReply::NetworkError)), this,
SLOT(slotErrorRequest(QNetworkReply::NetworkError)));
connect(this->reply, &QNetworkReply::finished, this, &HttpRetriever::slotFinishRequest);
connect(this->manager, &QNetworkAccessManager::finished, this->reply, &QNetworkReply::deleteLater);
connect(this->reply, &QIODevice::readyRead, this, &HttpRetriever::slotReadyReadRequest);
}
As you can see so far I have laid down the foundation for the network communication between my class and the server and I am yet to start working on the parsing of the XML reply and extracting the information I need from it.
The problem is that I am getting (very, very often) either
Encountered error while parsing XML data: Start tag expected.
or
Encountered error while parsing XML data: Premature end of document
in my processReply() function. This happens every time I get a large reply (a couple of hundreds up to a couple of thousands of lines). It never happens when I get a small one (30-40 lines give or take).
So the issue is obviously somewhere in the amount of data I am receiving, the way it is put together by the QNetworkAccessManager (or whichever Qt component in all this buffers the received chunks of data) and/or perhaps the way I have setup the instances of the network-related components in my class. I also have to make an important note here namely that in my browser (latest Firefox with the HttpRequester add-on) I am always receiving the complete XML no matter how large it is. So this seems to be a problem exclusive to my application and has nothing to do with the network settings on my system.
Since #Marco didn't write the answer...
The problem was that I was rewriting my buffer all the time by assigning the result from QNetworkReply::readAll(). As suggested using QByteArray::append() solves the problem.
In order to prevent a possible issue from this solution namely that you keep appending with each and every next reply you get, QByteArray::clear() needs to be called at some point for example when the finished() signal is emitted. Of course one needs to first process its contents before flushing it down the drain.
Maybe this is stupid question, actually it's appeal, or Qt is just to complicated for me.
Here's the thing:
I'm used to java when writing client-server application, and it's very simple. I would like to do same things in C++ (I'm very familiar with C++ itself), and I choose to learn Qt. I tried to write some applications in qt, but with partial success.
First thing that bothers me is signals and slots. I know how to use them in GUI programming but it confuses me with networking. And there's problem with blocking. When I call BufferedReader's readLine() method in java it blocks until it receives line from socket connection. In Qt I must make sure that there is line available every time, and handle it when there isn't one.
And when I connect QSocket's error signal to some of my custom slots, the signal is emitted when server sends last line and closes the connection, and in client's slot/function that reads I never read that last line. That are some problems I faced so far.
Slots and checking if there is data available makes me confused when I had to implements even the simplest protocols.
Important part:
I tried to find good example on the internet, but problem is that all examples are to complicated an big. Is there anyone how can show me how to write simple client-server application. Server accepts only one client. Client sends textual line containing command. If command is "ADD" or "SUB", server sends "SUP" indicating that command is supported. Otherwise it sends "UNS" and closes the connection. If client receives "SUP" it sends to more lines containing numbers to be subtracted or added. Server responds with result and closes connection.
I know that C++ requires more coding, but in Java this would take only 5 minutes, so it shouldn't take to long to write it in C++ either.
I'm sure this example would be very valuable to anyone who wants to learn networking in Qt.
edit:
This is my try to make the application (described above):
here is the server part:
#ifndef TASK_H
#define TASK_H
#include <QObject>
#include <QTcpServer>
class Task : public QObject
{
Q_OBJECT
public:
Task(QObject *parent = 0) : QObject(parent) {}
public slots:
void run();
void on_newConnection();
void on_error(QAbstractSocket::SocketError);
signals:
void finished();
private:
QTcpServer server;
};
#endif // TASK_H
void Task::run()
{
connect(&server,SIGNAL(newConnection()),this,SLOT(on_newConnection()));
connect(&server,SIGNAL(acceptError(QAbstractSocket::SocketError)),this,SLOT(on_error(QAbstractSocket::SocketError)));
if(server.listen(QHostAddress::LocalHost, 9000)){
qDebug() << "listening";
}else{
qDebug() << "cannot listen";
qDebug() << server.errorString();
}
}
void Task::on_newConnection(){
std::cout << "handeling new connection...\n";
QTcpSocket* socket = server.nextPendingConnection();
QTextStream tstream(socket);
while(!socket->canReadLine()){
socket->waitForReadyRead((-1));
}
QString operation = tstream.readLine();
qDebug() << "dbg:" << operation;
if(operation != "ADD" && operation != "SUB"){
tstream << "UNS\n";
tstream.flush();
socket->disconnect();
return;
}
tstream << "SUP\n";
tstream.flush();
double op1,op2;
while(!socket->canReadLine()){
socket->waitForReadyRead((-1));
}
op1 = socket->readLine().trimmed().toDouble();
qDebug() << "op1:" << op1;
while(!socket->canReadLine()){
socket->waitForReadyRead(-1);
}
op2 = socket->readLine().trimmed().toDouble();
qDebug() << "op2:" << op2;
double r;
if(operation == "ADD"){
r = op1 + op2;
}else{
r = op1 - op2;
}
tstream << r << "\n";
tstream.flush();
qDebug() << "result is: " << r;
socket->disconnect();
}
void Task::on_error(QAbstractSocket::SocketError ){
qDebug() << "server error";
server.close();
}
This is client side (header is similar to server's so I wont post it):
void Task::run()
{
QTcpSocket socket;
std::string temp;
socket.connectToHost(QHostAddress::LocalHost,9000);
if(socket.waitForConnected(-1))
qDebug() << "connected";
else {
qDebug() << "cannot connect";
return;
}
QTextStream tstream(&socket);
QString op;
std::cout << "operation: ";
std::cin >> temp;
op = temp.c_str();
tstream << op << "\n";
tstream.flush();
qDebug() << "dbg:" << op << "\n";
while(!socket.canReadLine()){
socket.waitForReadyRead(-1);
}
QString response = tstream.readLine();
qDebug() << "dbg:" << response;
if(response == "SUP"){
std::cout << "operand 1: ";
std::cin >> temp;
op = temp.c_str();
tstream << op + "\n";
std::cout << "operand 2: ";
std::cin >> temp;
op = temp.c_str();
tstream << op + "\n";
tstream.flush();
while(!socket.canReadLine()){
socket.waitForReadyRead(-1);
}
QString result = tstream.readLine();
std::cout << qPrintable("result is: " + result);
}else if(response == "UNS"){
std::cout << "unsupported operatoion.";
}else{
std::cout << "unknown error.";
}
emit finished();
}
What I could do better?
What are some good practices in similar situations?
When using blocking (not signal/slot mechanism), what is the best way to handle event when other side closes the connection?
Can someone rewrite this to make it look more professional (I just what to see how it supposed to look like, because I think that my solution is far from perfect) ?
Can someone rewrite this using signals and slots?
Thanks you.
Sorry for my English, and probably stupidity :)
Networking with Qt is not that difficult.
Communication between two points is handled by a single class; in the case of TCP/IP, that would be the QTcpSocket class. Both the client and server will communicate with a QTcpSocket object.
The only difference with the server is that you start with a QTcpServer object and call listen() to await a connection...
QTcpServer* m_pTcpServer = new QTcpServer
//create the address that the server will listen on
QHostAddress addr(QHostAddress::LocalHost); // assuming local host (127.0.0.1)
// start listening
bool bListening = m_pServer->listen(addr, _PORT); //_PORT defined as whatever port you want to use
When the server receives a connection from a client QTcpSocket, it will notify you with a newConnection signal, so assuming you've made a connection to a socket in your own class to receive that signal, we can get the server QTcpSocket object to communicate with the client...
QTcpSocket* pServerSocket = m_pServer->nextPendingConnection();
The server will receive a QTcpSocket object for each connection made. The server socket can now be used to send data to a client socket, using the a write method...
pServerSocket->write("Hello!");
When a socket (either client or server) receives data, it emits the readyRead signal. So, assuming you have made a connection to the readyRead signal for the socket, a slot function can retrieve the data...
QString msg = pSocket->readAll();
The other code you'll need is to handle the connect, disconnect and error signals, which you should connect relevant slots for receiving these notifications.
Ensure you only send data when you know the connection has been made. Normally, I would have the server receive a connection and send a 'hello' message back to the client. Once the client receives the message, it knows it can send to the server.
When either side disconnects, the remaining side will receive the disconnect signal and can act appropriately.
As for the client, it will just have one QTcpSocket object and after calling connectToHost, you will either receive a connected signal if the connection was succesfully made, or the error signal.
Finally, you can use QLocalServer and QLocalSocket in the same way, if you're just trying to communicate between processes on the same machine.