I got a problem with the ZeroMQ Majordomo worker API, which fails on an assertion, using this simple worker, client.
The broker I am using is all from the example section from ZeroMQ site. What's the m_reply_to used for and when is it set?
mdwrkapi.hpp:123: zmsg* mdwrk::recv(zmsg*&): Assertion `m_reply_to.size()!=0' failed.
Here is the worker code.
mdwrk session ("tcp://localhost:5555", "GenericData", verbose);
zmsg *reply = 0;
while (1) {
zmsg *request = session.recv (reply);
if (request == 0) {
break; // Worker was interrupted
}
reply = request; // Echo is complex… :-)
}
And here is the client part:
mdcli session ("tcp://localhost:5555", verbose);
int count = 1;
while(1) {
zmsg * request = new zmsg("Hello world");
zmsg * reply = session.send ("GenericData", request);
if (reply) {
delete reply;
} else {
continue; // Interrupt or failure
puts("Interupt or failure");
}
sleep(1);
puts("sleeping");
}
What's the m_reply_to used for?
As taken from the Majordomo source code, m_reply_to is declared as:
/* =====================================================================
mdwrkapi.hpp
Majordomo Protocol Worker API
Implements the MDP/Worker spec at http://rfc.zeromq.org/spec:7.
---------------------------------------------------------------------
Copyright (c) 1991-2011 iMatix Corporation <www.imatix.com>
...
*/
...
private:
...
// Return address, if any
std::string m_reply_to; // <<------------------------- RETURN ADDRESS
and serves for storing a return address like here, in recv():
// We should pop and save as many addresses as there are
// up to a null part, but for now, just save one...
m_reply_to = msg->unwrap ();
When it is set?
As taken from the source code, it may happen inside a recv():
// ---------------------------------------------------------------------
// Send reply, if any, to broker and wait for next request.
zmsg *
recv (zmsg *&reply_p)
{
// Format and send the reply if we were provided one
zmsg *reply = reply_p;
assert (reply || !m_expect_reply);
if (reply) {
assert (m_reply_to.size()!=0);
...
Related
I'm trying to setup a gRPC Async server which received a stream from the client (ServerAsyncReader usage). I'm a bit confusing with the process. I'm stuck on an assertion error.
Here it's my GreeterAsyncServer.
The assertion failed is at line GPR_ASSERT(ok); : ok is false when client sends colors (string)
void GreeterAsyncServer::HandleRpcs() {
// Spawn a new CallData instance to serve new clients.
new CallDataSendColors(&service_, cq_.get(), this);
void* tag; // uniquely identifies a request.
bool ok;
while (true) {
// Block waiting to read the next event from the completion queue. The
// event is uniquely identified by its tag, which in this case is the
// memory address of a CallData instance.
// The return value of Next should always be checked. This return value
// tells us whether there is any kind of event or cq_ is shutting down.
GPR_ASSERT(cq_->Next(&tag, &ok));
GPR_ASSERT(ok);
static_cast<CallData*>(tag)->Proceed();
}
}
void GreeterAsyncServer::Run(std::string server_address) {
ServerBuilder builder;
// Listen on the given address without any authentication mechanism.
builder.AddListeningPort(server_address, grpc::InsecureServerCredentials());
// Register "service" as the instance through which we'll communicate with
// clients. In this case it corresponds to an *asynchronous* service.
builder.RegisterService(&service_);
// Get hold of the completion queue used for the asynchronous communication
// with the gRPC runtime.
cq_ = builder.AddCompletionQueue();
// Finally assemble the server.
server_ = builder.BuildAndStart();
std::cout << "Server listening on " << server_address << std::endl;
// Proceed to the server's main loop.
HandleRpcs();
}
Here it's my .proto file
// .proto file
// The greeting service definition.
service Greeter {
// Sends several colors to the server and receives the replies
rpc SendColors (stream ColorRequest) returns (HelloReply) {}
}
// The response message containing the greetings
message HelloReply {
string message = 1;
}
// The request message containing the color
message ColorRequest {
string color = 1;
}
Here it's my CallData class:
#pragma once
#include "CallDataT.h"
using grpc::ServerAsyncReader;
using helloworld::HelloReply;
using helloworld::ColorRequest;
class CallDataSendColors {
protected:
enum CallStatus { CREATE, PROCESS, FINISH };
CallStatus status_;
Greeter::AsyncService* service_;
ServerCompletionQueue* completionQueue_;
ServerContext serverContext_;
// What we get from the client.
ColorRequest request_;
// What we send back to the client.
HelloReply reply_;
// The means to get back to the client.
ServerAsyncReader<HelloReply, ColorRequest> reader_;
virtual void AddNextToCompletionQueue() override {
new CallDataSendColors(service_, completionQueue_, observer_);
}
virtual void WaitForRequest() override {
service_->RequestSendColors(&serverContext_, &reader_, completionQueue_, completionQueue_, this);
}
virtual void HandleRequest() override {
reader_.Read(&request_, this);
}
virtual void Proceed() override {
if (status_ == CREATE) {
status_ = PROCESS;
WaitForRequest();
}
else if (status_ == PROCESS) {
AddNextToCompletionQueue();
HandleRequest();
status_ = FINISH;
reply_.set_message(std::string("Color ") + std::string("received"));
reader_.Finish(reply_, grpc::Status::OK, this);
}
else {
// We're done! Self-destruct!
if (status_ != FINISH) {
// Log some error message
}
delete this;
}
}
public:
CallDataSendColors(Greeter::AsyncService* service, ServerCompletionQueue* completionQueue, IObserver* observer = nullptr) :
status_(CREATE),
service_(service),
completionQueue_(completionQueue),
reader_(&serverContext_) {
Proceed();
}
};
As you can see in the HandleRequest() method, the reader_.Read(&request_, this) will be called only one time. I don't know how to call it as long as there are messages commin in.
I will appreciate if someone can help me.
Thank you in advance.
I've found the solution. The method GreeterAsyncServer::HandleRpcs() must be edited link this:
while (true) {
bool gotEvent = cq_->Next(&tag, &ok);
if (false == gotEvent || true == isShuttingDown_) {
break;
}
//GPR_ASSERT(ok);
if (true == ok) {
static_cast<CallData*>(tag)->Proceed();
}
}
I am using a very simple proto where the Message contains only 1 string field. Like so:
service LongLivedConnection {
// Starts a grpc connection
rpc Connect(Connection) returns (stream Message) {}
}
message Connection{
string userId = 1;
}
message Message{
string serverMessage = 1;
}
The use case is that the client should connect to the server, and the server will use this grpc for push messages.
Now, for the client code, assuming that I am already in a worker thread, how do I properly set it up so that I can continuously receive messages that come from server at random times?
void StartConnection(const std::string& user) {
Connection request;
request.set_userId(user);
Message message;
ClientContext context;
stub_->Connect(&context, request, &reply);
// What should I do from now on?
// notify(serverMessage);
}
void notify(std::string message) {
// generate message events and pass to main event loop
}
I figured out how to used the api. Looks like it is pretty flexible, but still a little bit weird given that I typically just expect the async api to receive some kind of lambda callback.
The code below is blocking, you'll have to run this in a different thread so it doesn't block your application.
I believe you can have multiple thread accessing the CompletionQueue, but in my case I just had one single thread handling this grpc connection.
GrpcConnection.h file:
public:
void StartGrpcConnection();
private:
std::shared_ptr<grpc::Channel> m_channel;
std::unique_ptr<grpc::ClientReader<push_notifications::Message>> m_reader;
std::unique_ptr<push_notifications::PushNotificationService::Stub> m_stub;
GrpcConnection.cpp files:
...
void GrpcConnectionService::StartGrpcConnection()
{
m_channel = grpc::CreateChannel("localhost:50051",grpc::InsecureChannelCredentials());
LongLiveConnection::Connect request;
request.set_user_id(12345);
m_stub = LongLiveConnection::LongLiveConnectionService::NewStub(m_channel);
grpc::ClientContext context;
grpc::CompletionQueue cq;
std::unique_ptr<grpc::ClientAsyncReader<LongLiveConnection::Message>> reader =
m_stub->PrepareAsyncConnect(&context, request, &cq);
void* got_tag;
bool ok = false;
LongLiveConnection::Message reply;
reader->StartCall((void*)1);
cq.Next(&got_tag, &ok);
if (ok && got_tag == (void*)1)
{
// startCall() is successful if ok is true, and got_tag is void*1
// start the first read message with a different hardcoded tag
reader->Read(&reply, (void*)2);
while (true)
{
ok = false;
cq.Next(&got_tag, &ok);
if (got_tag == (void*)2)
{
// this is the message from server
std::string body = reply.server_message();
// do whatever you want with body, in my case i push it to my applications' event stream to be processed by other components
// lastly, initialize another read
reader->Read(&reply, (void*)2);
}
else if (got_tag == (void*)3)
{
// if you do something else, such as listening to GRPC channel state change, in your call, you can pass a different hardcoded tag, then, in here, you will be notified when the result is received from that call.
}
}
}
}
There is zero documentation how to do an async bidirectional stream with grpc. I've made guesses by piecing together the regular async examples with what I found in peope's github.
With the frankestein code I have, I cannot figure out how to tell when a client sent me a message. Here is the procedure I have running on its own thread.
void GrpcStreamingServerImpl::listeningThreadProc()
{
try
{
// I think we make a call to the RPC method and wait for others to stream to it?
::grpc::ServerContext context;
void * ourOneAndOnlyTag = reinterpret_cast<void *>(1); ///< Identifies the call we are going to make. I assume we can only handle one client
::grpc::ServerAsyncReaderWriter<mycompanynamespace::OutputMessage,
mycompanynamespace::InputMessage>
stream(&context);
m_service.RequestMessageStream(&context, &stream, m_completionQueue.get(), m_completionQueue.get(), ourOneAndOnlyTag);
// Now I'm going to loop and get events from the completion queue
bool keepGoing = false;
do
{
void* tag = nullptr;
bool ok = false;
const std::chrono::time_point<std::chrono::system_clock> deadline(std::chrono::system_clock::now() +
std::chrono::seconds(1));
grpc::CompletionQueue::NextStatus nextStatus = m_completionQueue->AsyncNext(&tag, &ok, deadline);
switch(nextStatus)
{
case grpc::CompletionQueue::NextStatus::TIMEOUT:
{
keepGoing = true;
break;
}
case grpc::CompletionQueue::NextStatus::GOT_EVENT:
{
keepGoing = true;
if(ok)
{
// This seems to get called if a client connects
// It does not get called if we didn't call 'RequestMessageStream' before the loop started
// TODO - How do we tell when the client send us a messages?
// TODO - How do we know if they are just connecting?
// TODO - How do we get the message client sent?
// The tag corresponds to the request we made
if(tag == reinterpret_cast<void *>(1))
{
// SNIP successful writing of a message
stream.Write(*(outputMessage.get()), reinterpret_cast<void*>(2));
}
else if(tag == reinterpret_cast<void *>(2))
{
// This is telling us the message we sent was completed
}
else
{
// TODO - I dunno what else it can be
}
}
break;
}
case grpc::CompletionQueue::NextStatus::SHUTDOWN:
{
keepGoing = false;
break;
}
}
} while(keepGoing);
// Completion queue was shutdown
}
catch(std::exception& e)
{
QString errorMessage(
QString("An std::exception was caught in the listening thread. Exception message: %1").arg(e.what()));
m_backPointer->onImplError(errorMessage);
}
catch(...)
{
QString errorMessage("An exception of unknown type, was caught in the listening thread.");
m_backPointer->onImplError(errorMessage);
}
}
Setup looked like this
// Start up the grpc service
grpc::ServerBuilder builder;
builder.RegisterService(&m_service);
builder.AddListeningPort(endpoint.toStdString(), grpc::InsecureServerCredentials());
m_completionQueue = builder.AddCompletionQueue();
m_server = builder.BuildAndStart();
// Start the listening thread
m_listeningThread = QThread::create(&GrpcStreamingServerImpl::listeningThreadProc, this);
I'm trying to create a multithread server application using mongoose web server library.
I have main thread serving connections and sending requests to processors that are working in their own threads. Then processors place results into queue and queue observer must send results back to clients.
Sources are looking that way:
Here I prepare the data for processors and place it to queue.
typedef std::pair<struct mg_connection*, const char*> TransferData;
int server_app::event_handler(struct mg_connection *conn, enum mg_event ev)
{
Request req;
if (ev == MG_AUTH)
return MG_TRUE; // Authorize all requests
else if (ev == MG_REQUEST)
{
req = parse_request(conn);
task_queue->push(TransferData(conn,req.second));
mg_printf(conn, "%s", ""); // (1)
return MG_MORE; // (2)
}
else
return MG_FALSE; // Rest of the events are not processed
}
And here I'm trying to send the result back. This function is working in it's own thread.
void server_app::check_results()
{
while(true)
{
TransferData res;
if(!res_queue->pop(res))
{
boost::this_thread::sleep_for(boost::chrono::milliseconds(100));
continue;
}
mg_printf_data(res.first, "%s", res.second); // (3)
}
}
The problem is a client doesn't receive anything from the server.
If I run check_result function manualy in the event_handler after placing a task into the queue and then pass computed result back to event_handler, I'm able to send it to client using mg_printf_data (with returning MG_TRUE). Any other way - I'm not.
What exactly should I change in this sources to make it works?
Ok... It looks like I've solved it myself.
I'd been looking into mongoose.c code and an hour later I found the piece of code below:
static void write_terminating_chunk(struct connection *conn) {
mg_write(&conn->mg_conn, "0\r\n\r\n", 5);
}
static int call_request_handler(struct connection *conn) {
int result;
conn->mg_conn.content = conn->ns_conn->recv_iobuf.buf;
if ((result = call_user(conn, MG_REQUEST)) == MG_TRUE) {
if (conn->ns_conn->flags & MG_HEADERS_SENT) {
write_terminating_chunk(conn);
}
close_local_endpoint(conn);
}
return result;
}
So I've tried to do mg_write(&conn->mg_conn, "0\r\n\r\n", 5); after line (3) and now it's working.
I wrote two programs, one as server and another as client. The server is written in standard C++ using WinSock2.h. It is a simple echo server which means the server responds what it receives back to the client. I used a new thread for every client's connection, and in each thread:
Socket* s = (Socket*) a;
while (1) {
std::string r = s->ReceiveLine()
if (r.empty()) {
break;
}
s->SendLine(r);
}
delete s;
return 0;
Socket is a class from here. The server runs properly and I've tested it using telnet, it works well.
Then I wrote the client using C++.NET (or C++/CLI), TcpClient is used to send and receive message from the server. The code is like:
String^ request = "test";
TcpClient ^ client = gcnew TcpClient(server, port);
array<Byte> ^ data = Encoding::ASCII->GetBytes(request);
NetworkStream ^ stream = client->GetStream();
stream->Write(data, 0, data->Length);
data = gcnew array<Byte>(256);
String ^ response = String::Empty;
int bytes = stream->Read(data, 0, data->Length);
response = Encoding::ASCII->GetString(data, 0, bytes);
client->Close();
When I run the client and tries to show the response message onto my form, the program halted at the line int bytes = stream->Read(data, 0, data->Length); and cannot fetch the response. The server is running and there's nothing to do with the network as they are all running on the same computer.
I guess the reason is that the data server responds with is less than data->Length, so the Read method is waiting for more data. Is that right? How should I solve this problem?
Edit
I think I've solved the problem... There are another two methods in the Socket class, ReceiveBytes and SendBytes, and these two methods will not delete the unused space in the bytes array. So the length of data back from the server will match that from the client, thus the Read method will not wait for more data to come.
std::string Socket::ReceiveLine() {
std::string ret;
while (1) {
char r;
switch(recv(s_, &r, 1, 0)) {
case 0: // not connected anymore;
// ... but last line sent
// might not end in \n,
// so return ret anyway.
return ret;
case -1:
return "";
// if (errno == EAGAIN) {
// return ret;
// } else {
// // not connected anymore
// return "";
// }
}
ret += r;
if (r == '\n') return ret;
}
}
i guess receiveline function of the server is waiting for an enter key '\n'.
just try with "test\n" string.
String^ request = "test\n";
// other codes....