ZMQ messages not being received - c++

Please forgive me if I'm missing something simple, this is my first time doing anything with messaging and I inherited this codebase from someone else.
I am trying to send a message from a windows machine with an IP of 10.10.10.200 to an Ubuntu machine with an IP of 10.10.10.15.
I got the following result when running TCPView from the Windows machine, which makes me suspect that the problem lies in the Ubuntu machine. If I'm reading that right, then my app on the windows machine has created a connection on port 5556 which is what it is supposed to do. In case I'm wrong, I'll include the windows code too.
my_app.exe 5436 TCP MY_COMPUTER 5556 MY_COMPUTER 0 LISTENING
Windows app code:
void
NetworkManager::initializePublisher()
{
globalContext = zmq_ctx_new();
globalPublisher = zmq_socket(globalContext, ZMQ_PUB);
string protocol = "tcp://*:";
string portNumber = PUBLISHING_PORT; //5556
string address = protocol + portNumber;
char *address_ptr = new char[address.size() + 1];
strncpy_s(address_ptr, address.size() + 1, address.c_str(), address.size());
int bind_res = zmq_bind(globalPublisher, address_ptr);
if (bind_res != 0)
{
cerr << "FATAL: couldn't bind to port[" << portNumber << "] and protocol [" << protocol << "]" << endl;
}
cout << " Connection: " << address << endl;
}
void
NetworkManager::publishMessage(MESSAGE msgToSend)
{
// Get the size of the message to be sent
int sizeOfMessageToSend = MSG_MAX_SIZE;//sizeof(msgToSend);
// Copy IDVS message to buffer
char buffToSend[MSG_MAX_SIZE] = "";
// Pack the message id
size_t indexOfId = MSG_ID_SIZE + 1;
size_t indexOfName = MSG_NAME_SIZE + 1;
size_t indexOfdata = MSG_DATABUFFER_SIZE + 1;
memcpy(buffToSend, msgToSend.get_msg_id(), indexOfId - 1);
// Pack the message name
memcpy(buffToSend + indexOfId, msgToSend.get_msg_name(), indexOfName - 1);
// Pack the data buffer
memcpy(buffToSend + indexOfId + indexOfName, msgToSend.get_msg_data(), indexOfdata - 1);
// Send message
int sizeOfSentMessage = zmq_send(globalPublisher, buffToSend, MSG_MAX_SIZE, ZMQ_DONTWAIT);
getSubscriptionConnectionError();
// If message size doesn't match, we have an issue, otherwise, we are good
if (sizeOfSentMessage != sizeOfMessageToSend)
{
int errorCode = zmq_errno();
cerr << "FATAL: couldn't not send message." << endl;
cerr << "ERROR: " << errorCode << endl;
}
}
I can include more of this side's code if you think it's needed, but the error is popping up on the Ubuntu side, so I'm going to focus there.
The problem is when I call zmq_recv it returns -1 and when I check zmq_errno I get EAGAIN (Non-blocking mode was requested and no messages are available at the moment.) I also checked with netstat and I didn't see anything on port 5556
First is the function to connect to the publisher, then the function to get data, followed by main.
Ubuntu side code:
void
*connectoToPublisher()
{
void *context = zmq_ctx_new();
void *subscriber = zmq_socket(context, ZMQ_SUB);
string protocol = "tcp://";
string ipAddress = PUB_IP; //10.10.10.15
string portNumber = PUB_PORT; // 5556
string address = protocol + ipAddress + ":" + portNumber;
cout << "Address: " << address << endl;
char *address_ptr = new char[address.size() + 1];
strcpy(address_ptr, address.c_str());
// ------ Connect to Publisher ------
bool isConnectionEstablished = false;
int connectionStatus;
while (isConnectionEstablished == false)
{
connectionStatus = zmq_connect(subscriber, address_ptr);
switch (connectionStatus)
{
case 0: //we are good.
cout << "Connection Established!" << endl;
isConnectionEstablished = true;
break;
case -1:
isConnectionEstablished = false;
cout << "Connection Failed!" << endl;
getSubscriptionConnectionError();
cout << "Trying again in 5 seconds..." << endl;
break;
default:
cout << "Hit default connecting to publisher!" << endl;
break;
}
if (isConnectionEstablished == true)
{
break;
}
sleep(5); // Try again
}
// by the time we get here we should have connected to the pub
return subscriber;
}
static void *
getData(void *subscriber)
{
const char *filter = ""; // Get all messages
int subFilterResult = zmq_setsockopt(subscriber, ZMQ_SUBSCRIBE, filter, strlen(filter));
// ------ Get in main loop ------
while (1)
{
//get messages from publisher
char bufferReceived[MSG_MAX_SIZE] = "";
size_t expected_messageSize = sizeof(bufferReceived);
int actual_messageSize = zmq_recv(subscriber, bufferReceived, MSG_MAX_SIZE, ZMQ_DONTWAIT);
if (expected_messageSize == actual_messageSize)
{
MESSAGE msg = getMessage(bufferReceived); //Uses memcpy to copy id, name, and data strutct data from buffer into struct of MESSAGE
if (strcmp(msg.get_msg_id(), "IDXY_00000") == 0)
{
DATA = getData(msg); //Uses memcpy to copy data from buffer into struct of DATA
}
} else
{
// Something went wrong
getReceivedError(); //This just calls zmq_errno and cout the error
}
usleep(1);
}
}
int main (int argc, char*argv[])
{
//Doing some stuff...
void *subscriber_socket = connectoToHeadTrackerPublisher();
// Initialize Mux Lock
pthread_mutex_init(&receiverMutex, NULL);
// Initializing some variables...
// Launch Thread to get updates from windows machine
pthread_t publisherThread;
pthread_create(&publisherThread,
NULL, getData, subscriber_socket);
// UI stuff
zmq_close(subscriber_socket);
return 0;
}
If you cannot provide a solution, then I will accept identifying the problem as a solution. My main issue is that I don't have the knowledge or experience with messaging or networking to correctly identify the issue. Typically if I know what is wrong, I can fix it.

Ok, this has nothing to do with signalling / messaging framework
Your Ubuntu code instructs the ZeroMQ Context()-instance engine to create a new SUB-socket instance and next the code insist this socket to try to _connect() ( to setup a tcp:// transport-class connection towards the peering counterparty ) to "opposite" access-point, sitting on an address of the Ubuntu localhost:port# that was setup as 10.10.10.15:5556, while the intended PUB-side archetype access-point actually lives not on this Ubuntu machine, but on another, Windows host, IP:port# of which is 10.10.10.200:5556
This seems to be the root-cause of the problem, so change it accordingly to match the physical layout and you may get the toys work.

Related

recv() char array size

I'm working on implementing a C++ client server chat program to learn more / practice socket programming. I'm using winsockV2.
Briefly,
the client program connects to a server, who stores the client socket in a vector
client program sends messages for the server to distribute to other clients in the vector.
The problem I think I'm running into is that the clients and server are receiving the message and storing it in a char message[256] and if the message is shorter than 256, strange chars are displayed when I std::cout << message; which I'm being told is uninitialized memory. Here's an example of the output:
k:message from client to other client╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠╠(■o
Is there some way of creating a character array of the size of the received message? i.e
char recvMessage[4096];
int s = recv(socket, recvMessage, sizeof(recvMessage),0);
char recvOutput[strlen(recvMessage)] = recvMessage;
std::cout << recvOutput << std::endl;
Otherwise what is your solution for recv'ing messages which you do not know the length of?
If I'm being a complete idiot, please be kind, I came from PHP. classes are below:
SVR.CPP
See receiveMessages() and distributeMessages() functions
#include "stdafx.h"
#include "svr.h"
svr::svr()
{
//WSA Business I don't understand
WORD wVersionRequested;
WSADATA wsaData;
int err;
/* Use the MAKEWORD(lowbyte, highbyte) macro declared in Windef.h */
wVersionRequested = MAKEWORD(2, 2);
err = WSAStartup(wVersionRequested, &wsaData);
if (err != 0)
{
/* Tell the user that we could not find a usable */
/* Winsock DLL. */
printf("WSAStartup failed with error: %d\n", err);
}
//End of WSA Business
//get addressSize
addressSize = sizeof(address);
//set address data members
address.sin_family = AF_INET;
address.sin_port = htons(444);
address.sin_addr.s_addr = INADDR_ANY;
//init sListen
sListen = socket(AF_INET, SOCK_STREAM, 0);
bind(sListen, (sockaddr*)&address, addressSize);
}
svr::~svr()
{
}
void svr::start()
{
std::thread newConnThread(&svr::newConnection, this);
newConnThread.join();
}
void svr::receiveMessages(int clientIndex)
{
std::cout << "\tsvr::recv thread started for client index:" << clientIndex << std::endl;
//create char arr
char recvMessage[256];
//forever
while (true)
{
//receive message and input it to recvMessage char arr.
recv(clients[clientIndex], recvMessage, sizeof(recvMessage), 0);
//if message is not null, send out to other clients
if (recvMessage != NULL)
{
std::cout << "\t\tINFO:Received message of length: " << std::strlen(recvMessage) << " size: " << sizeof(recvMessage) << " : " << recvMessage << std::endl;
distributeMessages(recvMessage, clientIndex);
}
}
}
//distributes messages to all clients in vector. called by receiveMessages function, normally in rMessages thread.
void svr::distributeMessages(std::string message, int clientIndex)
{
for (unsigned int i = 0; i < clients.size(); i++)
{
if (clientIndex != i)
{
send(clients[i], message.c_str(), message.length(), 0);
}
else
{
//would have sent to self, not useful.
}
}
}
//accepts new connections and adds sockets to vector.
void svr::newConnection()
{
//mark for accept, unsure of somaxconn value;
listen(sListen, SOMAXCONN);
std::cout << "\tSERVER: awaiting new connections..." << std::endl;
while (true)
{
//accept connection and push on to vector.
clients.push_back(accept(sListen, (sockaddr*)&address, &addressSize));
//responds to new clients.
const char *message = "Hi, you've successfully connected!";
int clientIndex = clients.size() - 1;
int sent = send(clients[clientIndex], message, 33, 0);
//start new receiveMessage thread
std::thread newClient(&svr::receiveMessages, this, clientIndex);
//detach here, let newConn thread operate without depending on receiveMessages
newClient.detach();
}
std::cout << "\tSERVER: no longer listening for new connections" << std::endl;
}
CLI.CPP
See cSend() and cRecv() functions
#include "stdafx.h"
#include "cli.h"
cli::cli(char *ip)
{
//WSA
{
WORD wVersionRequested;
WSADATA wsaData;
int err;
// Use the MAKEWORD(lowbyte,highbyte) macro declared in windef.h
wVersionRequested = MAKEWORD(2, 2);
err = WSAStartup(wVersionRequested, &wsaData);
if (err != 0)
{
std::cout << "WSAStartup failed with the error: " << err;
}
}
//get addressSize
addressSize = sizeof(address);
//set address struct data members
address.sin_family = AF_INET;
address.sin_port = htons(444);
//if ip empty, prompt user;
if (ip == NULL)
{
std::string ipInput;
std::cout << "\n\tConnect to which IP: ";
std::cin >> ipInput;
address.sin_addr.s_addr = inet_addr(ipInput.c_str());
}
else
{
address.sin_addr.s_addr = inet_addr(ip);
}
sock = socket(AF_INET, SOCK_STREAM, 0);
std::cout << "\n\tYour username: ";
std::cin >> uname;
}
cli::~cli()
{
}
void cli::start()
{
try
{
//hold string
char message[33];
std::cout << "\n\tcli::start() called";
int conRet;
//connects to server socket & receives a message, stores in it message variable
conRet = connect(sock, (sockaddr*)&address, (int)addressSize);
recv(sock, message, sizeof(message), 0);
std::cout << "\n\tSERVER: " << message;
//starts threads, pass this for object scope.
std::thread sendThread(&cli::cSend, this);
std::thread recvThread(&cli::cRecv, this);
//this function (start) will return/end when send and recv threads end.
sendThread.join();
recvThread.join();
}
catch (std::exception e)
{
std::cerr << e.what() << std::endl;
}
}
void cli::cSend()
{
std::cout << "\n\tcli::send thread started";
//char arr for sending str;
std::string getLine;
while (true)
{
std::cout << "\n\t" << uname << ":" << std::flush;
//set to "" because i suspected the value remains in the string after a loop.
std::string message = "";
//get input, put it in message
std::getline(std::cin, message);
//get full message
std::string fullMessage = uname + ":" + message;
//get constant int, size of fullMessage
const int charArrSize = fullMessage.length();
std::cout << "\t\tINFO: Sending character array of length: " << charArrSize << " size: " << sizeof(fullMessage.c_str()) << " : " << fullMessage.c_str() << std::endl;
//sends it
send(sock, fullMessage.c_str(), charArrSize, 0);
}
}
void cli::cRecv()
{
std::cout << "\n\tcli::recv thread started";
//initialize arr to 0, will hopefully help avoid the weird chars in the cout
char recvMessage[256]{ '\0' };
while (true)
{
recv(sock, recvMessage, sizeof(recvMessage), 0);
std::cout << "\t\tINFO:Received message of length: " << std::strlen(recvMessage) << " size: " << sizeof(recvMessage) << " : " << recvMessage << std::endl;
std::cout << recvMessage << std::endl;
}
}
what is your solution for recv'ing messages which you do not know the
length of?
recv() tells you the length of the message it received. You don't have to wonder what it is. That's recv()'s return value.
int s = recv(socket, recvMessage, sizeof(recvMessage),0);
See -- there you go. It's right here in front of you. It's s. Of course if there was an error s would be negative and you need to check for that. But, ignoring that little detail, your worries are over: s is the length of your message you just received.
char recvOutput[strlen(recvMessage)] = recvMessage;
That's not going to work. What is strlen() doing here? strlen() computes the size of the string, expecting the string to be an old-fashioned, C-style character string that's terminated by a \0 byte. recv() does not terminate anything it receives with a \0 byte. Instead, it returns the actual character count.
And, besides, this won't work anyway. You can't initialize an array this way.
Your obvious intent here, apparently, is to expect to receive a text string as message. Well, since your language of choice is C++, and you tagged your question as such, the logical conclusion is that you should be using what C++ gives you to deal with text strings: the std::string class:
std::string recvOutput{recvMessage, recvMessage+s};
There you go. Mission accomplished. Since you already known the length of the received message in s, as we've determined before (and after double-checking that s is not negative), you can simply use std::string's existing constructor that initializes the new string given an iterator, or a pointer, to the start and the end of string.
When dealing with low-level operating system interfaces, like sockets, you have no choice but to use primitive data types, like plain char arrays and buffers, because that's the only thing that the operating system understands. But, with the rich set of templates and classes offered by the C++ library, your code should switch to using C++ classes and templates at the first opportunity, in order to be able to use all those resources. As such, as soon as you've determined how big is the text string recv() just came up with, just stuff it into a std::string before figuring out what to do with it.

Handeling SSL Client not reading all data

I am trying to accomplish, that my ssl server does not break down, when a client does not collect all data. (fixed with one minor bug)
when the data is too long.
Basically what I'm trying to do is write in a non-blocking way. For that I found two different approaches:
First approach
using this code
int flags = fcntl(ret.fdsock, F_GETFL, 0);
fcntl(ret.fdsock, F_SETFL, flags | O_NONBLOCK);
and creating the ssl connection with it
Second approach:
Doing this directly after creating the SSL Object using SSL_new(ctx)
BIO *sock = BIO_new_socket(ret.fdsock, BIO_NOCLOSE);
BIO_set_nbio(sock, 1);
SSL_set_bio(client, sock, sock);
Both of which have their downsides, but neither of which helps solving the problem.
The first approach seems to read in a unblocking way just fine, but when I write more data, than the client reads, my server crashes.
The second approach does not seem to do anything, so my guess is, that I did something wrong or did not understand what a BIO actually does.
For more Information here is how the server writes to the client:
int SSLConnection::send(char* msg, const int size){
int rest_size = size;
int bytes_sent = 0;
char* begin = msg;
std::cout << "expected bytes to send: " << size << std::endl;
while(rest_size > 0) {
int tmp_bytes_sent = SSL_write(connection, begin, rest_size);
std::cout << "any error : " << ERR_get_error()<< std::endl;
std::cout << "tmp_bytes_sent: " << tmp_bytes_sent << std::endl;
if (tmp_bytes_sent < 0){
std::cout << tmp_bytes_sent << std::endl;
std::cout << "ssl error : " << SSL_get_error(this->connection, tmp_bytes_sent)<< std::endl;
} else {
bytes_sent += tmp_bytes_sent;
rest_size -= tmp_bytes_sent;
begin = msg+bytes_sent;
}
}
return bytes_sent;
}
Output:
expected bytes to send: 78888890
Betätigen Sie die <RETURN> Taste, um das Fenster zu schließen...
(means: hit <return> to close window)
EDIT: After people said, that I need to cache errors appropriate, here is my new code:
Setup:
connection = SSL_new(ctx);
if (connection){
BIO * sbio = BIO_new_socket(ret.fdsock, BIO_NOCLOSE);
if (sbio) {
BIO_set_nbio(sbio, false);
SSL_set_bio(connection, sbio, sbio);
SSL_set_accept_state(connection);
} else {
std::cout << "Bio is null" << std::endl;
}
} else {
std::cout << "client is null" << std::endl;
}
Sending:
int SSLConnection::send(char* msg, const int size){
if(connection == NULL) {
std::cout << "ERR: Connection is NULL" << std::endl;
return -1;
}
int rest_size = size;
int bytes_sent = 0;
char* begin = msg;
std::cout << "expected bytes to send: " << size << std::endl;
while(rest_size > 0) {
int tmp_bytes_sent = SSL_write(connection, begin, rest_size);
std::cout << "any error : " << ERR_get_error()<< std::endl;
std::cout << "tmp_bytes_sent: " << tmp_bytes_sent << std::endl;
if (tmp_bytes_sent < 0){
std::cout << tmp_bytes_sent << std::endl;
std::cout << "ssl error : " << SSL_get_error(this->connection, tmp_bytes_sent)<< std::endl;
break;
} else if (tmp_bytes_sent == 0){
std::cout << "tmp_bytes are 0" << std::endl;
break;
} else {
bytes_sent += tmp_bytes_sent;
rest_size -= tmp_bytes_sent;
begin = msg+bytes_sent;
}
}
return bytes_sent;
}
Using a client, that fetches 60 bytes, here is the output:
Output writing 1,000,000 Bytes:
expected bytes to send: 1000000
any error : 0
tmp_bytes_sent: 16384
any error : 0
tmp_bytes_sent: 16384
Betätigen Sie die <RETURN> Taste, um das Fenster zu schließen...
(translates to: hit <RETURN> to close window)
Output writing 1,000 bytes:
expected bytes to send: 1000
any error : 0
tmp_bytes_sent: 1000
connection closed <- expected output
First, a warning: non-blocking I/O over SSL is a rather baroque API, and it's difficult to use correctly. In particular, the SSL layer sometimes needs to read internal data before it can write user data (or vice versa), and the caller's code is expected to be able to handle that based on the error-codes feedback it gets from the SSL calls it makes. It can be made to work correctly, but it's not easy or obvious -- you are de facto required to implement a state machine in your code that echoes the state machine inside the SSL library.
Below is a simplified version of the logic that is required (it's extracted from the Write() method in this file which is part of this library, in case you want to see a complete, working implementation)
enum {
SSL_STATE_READ_WANTS_READABLE_SOCKET = 0x01,
SSL_STATE_READ_WANTS_WRITEABLE_SOCKET = 0x02,
SSL_STATE_WRITE_WANTS_READABLE_SOCKET = 0x04,
SSL_STATE_WRITE_WANTS_WRITEABLE_SOCKET = 0x08
};
// a bit-chord of SSL_STATE_* bits to keep track of what
// the SSL layer needs us to do next before it can make more progress
uint32_t _sslState = 0;
// Note that this method returns the number of bytes sent, or -1
// if there was a fatal error. So if this method returns 0 that just
// means that this function was not able to send any bytes at this time.
int32_t SSLSocketDataIO :: Write(const void *buffer, uint32 size)
{
int32_t bytes = SSL_write(_ssl, buffer, size);
if (bytes > 0)
{
// SSL was able to send some bytes, so clear the relevant SSL-state-flags
_sslState &= ~(SSL_STATE_WRITE_WANTS_READABLE_SOCKET | SSL_STATE_WRITE_WANTS_WRITEABLE_SOCKET);
}
else if (bytes == 0)
{
return -1; // the SSL connection was closed, so return failure
}
else
{
// The SSL layer's internal needs aren't being met, so we now have to
// ask it what its problem is, then give it what it wants. :P
int err = SSL_get_error(_ssl, bytes);
if (err == SSL_ERROR_WANT_READ)
{
// SSL can't write anything more until the socket becomes readable,
// so we need to go back to our event loop, wait until the
// socket select()'s as readable, and then call SSL_Write() again.
_sslState |= SSL_STATE_WRITE_WANTS_READABLE_SOCKET;
_sslState &= ~SSL_STATE_WRITE_WANTS_WRITEABLE_SOCKET;
bytes = 0; // Tell the caller we weren't able to send anything yet
}
else if (err == SSL_ERROR_WANT_WRITE)
{
// SSL can't write anything more until the socket becomes writable,
// so we need to go back to our event loop, wait until the
// socket select()'s as writeable, and then call SSL_Write() again.
_sslState &= ~SSL_STATE_WRITE_WANTS_READABLE_SOCKET;
_sslState |= SSL_STATE_WRITE_WANTS_WRITEABLE_SOCKET;
bytes = 0; // Tell the caller we weren't able to send anything yet
}
else
{
// SSL had some other problem I don't know how to deal with,
// so just print some debug output and then return failure.
fprintf(stderr,"SSL_write() ERROR!");
ERR_print_errors_fp(stderr);
}
}
return bytes; // Returns the number of bytes we actually sent
}
I think your problem is
rest_size -= bytes_sent;
You should do rest_size -= tmp_bytes_sent;
Also
if (tmp_bytes_sent < 0){
std::cout << tmp_bytes_sent << std::endl;
//its an error condition
return bytes_sent;
}
I dont know whether this will fix the issue, but the code you pasted has the above mentioned issues
When I write more data, than the client reads, my server crashes.
No it doesn't, unless you've violently miscoded something else that you haven't posted here. It either loops forever or it gets an error: probably ECONNRESET, which means the client has behaved as you described, and you've detected it, so you should close the connection and forget about him. Instead of which, you are just looping forever, trying to send the data to a broken connection, which can never happen.
And when you get an error, there's not much use in just printing a -1. You should print the error, with perror() or errno or strerror().
Speaking of looping forever, don't loop like this. SSL_write() can return 0, which you aren't handling at all: this will cause an infinite loop. See also David Schwartz's comments below.
NB you should definitely use the second approach. OpenSSL needs to know that the socket is in non-blocking mode.
Both of which have their downsides
Such as?
And as noted in the other answer,
rest_size -= bytes_sent;
should be
rest_size -= tmp_bytes_sent;

"Connection was broken" error with UDT (UDP-based data transfer protocol)

I am programming a real-time game in which I need reliable UDP, so I've chosen to work with UDT (UDP-based data transfer protocol - http://sourceforge.net/projects/udt/).
The clients (on browsers) send real-time messages to my server via CGI scripts. The problem is that there are some messages that are being lost, and I don't know why because the server says that it sent all the messages successfully to the corresponding clients, but sometimes the client doesn't receive the message.
In my debug file, I've found that when a message is not received by the client, its script says:
error in recv();
recv: Connection was broken.
I would like to get some help on how the server shall know if the client got its message; should I send a NACK or something from the client side? I thought that UDT should do that for me. Can someone clarify this situation?
The relevant sections of the communication parts of my code are bellow, with some comments:
server's relevant code:
//...
void send_msg_in(player cur, char* xml){
/*this function stores the current message, xml, in a queue if xml!=NULL, and sends the 1st message of the queue to the client*/
/*this function is called when the player connects with the entering xml=NULL to get the 1st message of the queue,
or with xml!=NULL when a new message arrives: in this case the message is stored in the queue, and then the message will be sent in the appropriate time, i.e. the messages are ordered.*/
char* msg_ptr=NULL;
if (xml!=NULL){ //add the message to a queue (FIFO), the cur.xml_msgs
msg_ptr=(char*) calloc(strlen(xml)+1, sizeof(char));
strcpy(msg_ptr, xml);
(*(cur.xml_msgs)).push(msg_ptr);
} //get the 1st message of the queue
if (!(*(cur.xml_msgs)).empty()){
xml=(*(cur.xml_msgs)).front();
}
if (cur.get_udt_socket_in()!=NULL){
UDTSOCKET cur_udt = *(cur.get_udt_socket_in());
// cout << "send_msg_in(), cur_udt: " << cur_udt << endl;
//send the "xml", i.e. the 1st message of the queue...
if (UDT::ERROR == UDT::send(cur_udt, xml, strlen(xml)+1, 0)){
UDT::close(cur_udt);
cur.set_udt_socket_in(NULL);
}
else{ //if no error this else is reached
cout << "TO client:\n" << xml << "\n"; /*if there is no error,
i.e. on success, the server prints the message that was sent.*/
// / \
// /_!_\
/*the problem is that
the messages that are lost don't appear on the client side,
but they appear here on the server! */
if (((string) xml).find("<ack.>")==string::npos){
UDT::close(cur_udt);
cur.set_udt_socket_in(NULL); //close the socket
}
(*(cur.xml_msgs)).pop();
}
}
}
//...
client's relevant code:
//...
#define MSGBUFSIZE 1024
char msgbuf[MSGBUFSIZE];
UDTSOCKET client;
ofstream myfile;
//...
main(int argc, char *argv[]){
//...
// connect to the server, implict bind
if (UDT::ERROR == UDT::connect(client, (sockaddr*)&serv_addr, sizeof(serv_addr))){
cout << "error in connect();" << endl;
return 0;
}
myfile.open("./log.txt", ios::app);
send(xml);
char* cur_xml;
do{
cur_xml = receive(); //wait for an ACK or a new message...
myfile << cur_xml << endl << endl; // / \
/* /_!_\ the lost messages don't appear on the website
neither on this log file.*/
} while (((string) cur_xml).find("<ack.>")!=string::npos);
cout << cur_xml << endl;
myfile.close();
UDT::close(client);
return 0;
}
char* receive(){
if (UDT::ERROR == UDT::recv(client, msgbuf, MSGBUFSIZE, 0)){
// / \
/* /_!_\ when a message is not well received
this code is usually reached, and an error is printed.*/
cout << "error in recv();" << endl;
myfile << "error in recv();" << endl;
myfile << "recv: " << UDT::getlasterror().getErrorMessage() << endl << endl;
return 0;
}
return msgbuf;
}
void* send(string xml){
if (UDT::ERROR == UDT::send(client, xml.c_str(), strlen(xml.c_str())+1, 0)){
cout << "error in send();" << endl;
myfile << "error in send();" << endl;
myfile << "send: " << UDT::getlasterror().getErrorMessage() << endl << endl;
return 0;
}
}
Thank you for any help!
PS. I tried to increase the linger time on close(), after finding the link http://udt.sourceforge.net/udt4/doc/opt.htm, adding the following to the server's code:
struct linger l;
l.l_onoff = 1;
l.l_linger = ...; //a huge value in seconds...
UDT::setsockopt(*udt_socket_ptr, 0, UDT_LINGER, &l, sizeof(l));
but the problem is still the same...
PPS. the other parts of the communication in the server side are: (note: it seams for me that they are not so relevant)
main(int argc, char *argv[]){
char msgbuf[MSGBUFSIZE];
UDTSOCKET serv = UDT::socket(AF_INET, SOCK_STREAM, 0);
sockaddr_in my_addr;
my_addr.sin_family = AF_INET;
my_addr.sin_port = htons(PORT);
my_addr.sin_addr.s_addr = INADDR_ANY;
memset(&(my_addr.sin_zero), '\0', sizeof(my_addr.sin_zero));
if (UDT::ERROR == UDT::bind(serv, (sockaddr*)&my_addr, sizeof(my_addr))){
cout << "error in bind();";
return 0;
}
UDT::listen(serv, 1);
int namelen;
sockaddr_in their_addr;
while (true){
UDTSOCKET recver = UDT::accept(serv, (sockaddr*)&their_addr, &namelen);
if (UDT::ERROR == UDT::recv(recver, msgbuf, MSGBUFSIZE, 0)){
//this recv() function is called only once for each aqccept(), because the clients call CGI scripts via a browser, they need to call a new CGI script with a new UDT socket for each request (this in in agreement to the clients' code presented before).
cout << "error in recv();" << endl;
}
char* player_xml = (char*) &msgbuf;
cur_result = process_request((char*) &msgbuf, &recver, verbose); //ACK
}
}
struct result process_request(char* xml, UDTSOCKET* udt_socket_ptr, bool verbose){
//parse the XML...
//...
player* cur_ptr = get_player(me); //searches in a vector of player, according to the string "me" of the XML parsing.
UDTSOCKET* udt_ptr = (UDTSOCKET*) calloc(1, sizeof(UDTSOCKET));
memcpy(udt_ptr, udt_socket_ptr, sizeof(UDTSOCKET));
if (cur_ptr==NULL){
//register the player:
player* this_player = (player*) calloc(1, sizeof(player));
//...
}
}
else if (strcmp(request_type.c_str(), "info_waitformsg")==0){
if (udt_ptr!=NULL){
cur_ptr->set_udt_socket_in(udt_ptr);
if (!(*(cur_ptr->xml_msgs)).empty()){
send_msg_in(*cur_ptr, NULL, true);
}
}
}
else{ //messages that get instant response from the server.
if (udt_ptr!=NULL){
cur_ptr->set_udt_socket_out(udt_ptr);
}
if (strcmp(request_type.c_str(), "info_chat")==0){
info_chat cur_info;
to_object(&cur_info, me, request_type, msg_ptr); //convert the XML string values to a struct
process_chat_msg(cur_info, xml);
}
/* else if (...){ //other types of messages...
}*/
}
}
void process_chat_msg(info_chat cur_info, char* xml_in){
player* player_ptr=get_player(cur_info.me);
if (player_ptr){
int i=search_in_matches(matches, cur_info.match_ID);
if (i>=0){
match* cur_match=matches[i];
vector<player*> players_in = cur_match->followers;
int n=players_in.size();
for (int i=0; i<n; i++){
if (players_in[i]!=msg_owner){
send_msg_in(*(players_in[i]), xml, flag);
}
}
}
}
}
Looking at the UDT source code at http://sourceforge.net/p/udt/git/ci/master/tree/udt4/src/core.cpp, the error message "Connection was broken" is produced when either of the Boolean flags m_bBroken or m_bClosing is true and there is no data in the receive buffer.
Those flags are set in just a few cases:
In sections of code marked "should not happen; attack or bug" (unlikely)
In deliberate close or shutdown actions (don't see this happening in your code)
In expiration of a timer that checks for peer activity (the likely culprit)
In that source file at line 2593 it says:
// Connection is broken.
// UDT does not signal any information about this instead of to stop quietly.
// Application will detect this when it calls any UDT methods next time.
//
m_bClosing = true;
m_bBroken = true;
// ...[code omitted]...
// app can call any UDT API to learn the connection_broken error
Looking at the send() call, I don't see anywhere that it waits for an ACK or NAK from the peer before returning, so I don't think a successful return from send() on the server side is indicative of successful receipt of the message by the client.
You didn't show the code on the server side that binds to the socket and listens for responses from the client; if the problem is there then the server might be happily sending messages and never listening to the client that is trying to respond.
UDP is not a guaranteed-transmission protocol. A host will send a message, but if the recipient does not receive it, or if it is not received properly, no error will be raised. Therefore, it is commonly used in applications that require speed over perfect delivery, such as games. TCP does guarantee delivery, because it requires that a connection be set up first, and each message is acknowledged by the client.
I would encourage you to think about whether you actually need guaranteed receipt of that data, and, if you do, consider using TCP.

Multi-threaded client not receiving messages from server

I have a multiclient-server program. The client console will need to wait for user to input a message to send onto the server, and the client also needs to always be available for receiving messages from the server (this happens intermittently). In order to do this, I have tried to set up two threads on the client-side, one for receiving, and one for sending.
Initially, the client program (which I downloaded from here) was designed to send a message to the server and receive a message back immediately. This is the initial code (I have not included the functions here because I know they work, and the function names are self-explanatory):
int main()
{
//initialize the winsock library
myTcpSocket::initialize();
//get client's information
string clientName = clientInfo.getHostName();
string clientIPAddress = clientInfo.getHostIPAddress();
//get server's IP address and name
string serverIPAddress = "";
readServerConfig(serverIPAddress);
myHostInfo serverInfo(serverIPAddress,ADDRESS);
string serverName = serverInfo.getHostName();
//create the socket for client
myTcpSocket myClient(PORTNUM);
// connect to the server.
myClient.connectToServer(serverIPAddress, ADDRESS);
int recvBytes = 0;
while (1)
{
// send message to server
char messageToServer[MAX_MSG_LEN+1];
memset(messageToServer, 0, sizeof(messageToServer));
cout << "[SEND] ";
cin.getline(messageToServer,MAX_MSG_LEN);
winLog << "[SEND] " << messageToServer << endl;
myClient.sendMessage(string(messageToServer));
if ( !string(messageToServer).compare("Quit") || !string(messageToServer).compare("quit") )
break;
//receive message from server
string messageFromServer = "";
recvBytes = myClient.receiveMessage(messageFromServer);
if ( recvBytes == -99 ) break;
cout << "[RECV:" << serverName << "]: " << messageFromServer << endl;
winLog << "[RECV:" << serverName << "]: " << messageFromServer << endl;
}
return 1;
}
This code works, but since the server does not always have a message to send back to the client upon receiving a message, this holds up processes on the client side, which is why I am using a multi-threaded structure as below (functions not included for the above reason), separating the receiving and sending functions into different threads:
DWORD WINAPI sendHandleThread(LPVOID threadInfo) //sending thread
{
//this structure contains all the data this callback will work on
myThreadArgument* sendArgument = (myThreadArgument*)threadInfo;
//create the socket for client
myTcpSocket myClient(PORTNUM); //PORTNUM = 1200
// connect to the server.
string serverIPAddress = "";
readServerConfig(serverIPAddress);
myClient.connectToServer(serverIPAddress, ADDRESS);
myHostInfo serverInfo(serverIPAddress,ADDRESS);
string serverName = serverInfo.getHostName();
int recvBytes = 0;
while (1)
{
// send message to server
char messageToServer[MAX_MSG_LEN+1];
memset(messageToServer, 0, sizeof(messageToServer));
cout << "[SEND] ";
cin.getline(messageToServer,MAX_MSG_LEN);
winLog << "[SEND] " << messageToServer << endl;
myClient.sendMessage(string(messageToServer));
if ( !string(messageToServer).compare("Quit") || !string(messageToServer).compare("quit") )
break;
}
return 1;
}
DWORD WINAPI recHandleThread(LPVOID threadInfo) //receiving thread
{
//this structure contains all the data this callback will work on
myThreadArgument* recArgument = (myThreadArgument*)threadInfo;
//create the socket for client
myTcpSocket myClient(PORTNUM); //PORTNUM = 1200
// connect to the server.
string serverIPAddress = "";
readServerConfig(serverIPAddress);
myClient.connectToServer(serverIPAddress, ADDRESS);
myHostInfo serverInfo(serverIPAddress,ADDRESS);
string serverName = serverInfo.getHostName();
int recvBytes = 0;
while (1)
{
//receive message from server
string messageFromServer = "";
recvBytes = myClient.receiveMessage(messageFromServer);
if ( recvBytes == -99 ) break;
cout << "[RECV:" << serverName << "]: " << messageFromServer << endl;
winLog << "[RECV:" << serverName << "]: " << messageFromServer << endl;
}
return 1;
}
int main()
{
//semaphore
mySemaphore coutSemaphore(string(""), 1);
//initialize the winsock library
myTcpSocket::initialize();
//get client's information (assume neither the name nor the address is given)
string clientName = clientInfo.getHostName();
string clientIPAddress = clientInfo.getHostIPAddress();
//get server's IP address and name
string serverIPAddress = "";
readServerConfig(serverIPAddress);
myHostInfo serverInfo(serverIPAddress,ADDRESS);
string serverName = serverInfo.getHostName();
//create sending thread
myTcpSocket send;
string sendName = "sendName";
myThreadArgument* sendArgument = new myThreadArgument(&send, &coutSemaphore, sendName);
myThread* sendThread = new myThread(sendHandleThread, (void*)sendArgument);
sendThread->execute();
//create receiving thread
myTcpSocket rec;
string recName = "recName";
myThreadArgument* recArgument = new myThreadArgument(&rec, &coutSemaphore, recName);
myThread* recThread = new myThread(recHandleThread, (void*)recArgument);
recThread->execute();
while (1)
{
//dummy process
Sleep(30000);
cout << "--" << endl;
}
return 1;
}
I have tested that I have set up the threads properly - but only my send thread works i.e. when the user inputs a message, it will be sent to and received by the server. However, I cannot seem to receive the message from my server.
I'm new to this - is there anything that I am doing wrongly? Sorry if the question isn't clear enough - I don't know what else is needed.
The relevant snippet of my server code is here for reference (I don't know if more is required, since the server portion seems to work the way I need it to) - at the moment I have set it up for testing so that when it receives a message from a client, it will 'reply' immediately (so that I can test the receiving thread on my client). Subsequently it will not reply immediately, but only when it has messages to pass on to the client.
DWORD WINAPI clientHandleThread(LPVOID threadInfo)
{
// this structure will contain all the data this callback will work on
myThreadArgument* clientArgument = (myThreadArgument*)threadInfo;
// get the client connection: receiving messages from client and
// sending messages to the client will all be done by using
// this client connection
myTcpSocket* clientConnection = clientArgument->getClientConnect();
string clientName = clientArgument->getHostName();
// the server is communicating with this client here
while(1)
{
string messageFromClient = "";
// receive from the client
int numBytes = clientConnection->recieveMessage(messageFromClient);
if ( numBytes == -99 ) break;
cout << "[RECV fr " << clientName << "]: " << messageFromClient << endl;
// if the client wants to disconnect
if ( messageFromClient.compare("quit") == 0 || messageFromClient.compare("Quit") == 0 )
{
break;
}
else // send to the client
{
clientConnection->sendMessage(string("testing")); //test reply
}
}
// if we reach here, this session with the client is done,
// so we set the event on this thread to inform the main
// control that this session is finished
clientArgument->getExitEvent()->setEvent();
return 1;
}
In your multi-threaded program, you are creating two clients, one in sendHandleThread and the other in recHandleThread, and therewith two connections. Of course, the server sends its reply message over the connection where the client's message came in, i. e. to the sendHandleThread, which is not prepared to receive it, while the recHandleThread receives nothing over its connection, because nothing was sent there.
To rectify that, you must create only one client connection and let both threads use the same.

TCP/Ip network communication in c++

I am trying to write a threaded function that sends system information via Tcp/ip over the local network to another computer. I have been using sockets to achieve this and this has worked out quite allright thus far. But I am now at a point where this usually works but around 30% of the time I get error messages telling me that the socket can not be opened. I use the activeSocket library for the sockets.
#include "tbb/tick_count.h"
#include "ActiveSocket.h"
using namespace std;
CActiveSocket socket;
extern int hardwareStatus;
int establishTCP() {
char time[11];
int communicationFailed = 0;
memset(&time, 0, 11);
socket.Initialize();
socket.SetConnectTimeout(0, 20);
socket.SetSendTimeout(0, 20);
return communicationFailed;
}
int monitor() {
cout << "Monitor: init continious monitoring" << endl;
int communicationFailed;
tbb::tick_count monitorCounter = tbb::tick_count::now();
while (!closeProgram) {
tbb::tick_count currentTick = tbb::tick_count::now();
tbb::tick_count::interval_t interval;
interval = currentTick - monitorCounter;
if (interval.seconds() > 2) {
monitorCounter = tbb::tick_count::now();
communicationFailed = 1;
char buffer[256];
sprintf(buffer, "%d;", hardwareStatus);
establishTCP();
char *charip = new char[monitoringIP.size() + 1];
charip[monitoringIP.size()] = 0;
memcpy(charip, monitoringIP.c_str(), monitoringIP.size());
const uint8* realip = (const uint8 *) charip;
int monitorCount = 0;
cout << "Monitor: " << buffer << endl;
while (communicationFailed == 1 && monitorCount < 2) {
monitorCount++;
if (socket.Open(realip, 2417)) {
if (socket.Send((const uint8 *) buffer, strlen(buffer))) {
cout << "Monitor: Succeeded sending data" << endl;
communicationFailed = 0;
socket.Close();
} else {
socket.Close();
communicationFailed = 1;
cout << "Monitor: FAILED TO SEND DATA" << endl;
}
} else {
socket.Close();
communicationFailed = 1;
cout << "Monitor: FAILED TO OPEN SOCKET FOR DATA" << endl;
}
}
if (monitorCount == 2) cout << "Monitor: UNABLE TO SEND DATA" << endl;
}
}
return communicationFailed;
}
I think I am doing something wrong with these functions and that the problem is not on the other side of the line where this data is received. Can anyone see any obvious mistakes in this code that could cause the failure? I keep getting my own cout message "Monitor: FAILED TO OPEN SOCKET FOR DATA"
EDIT: With telnet everything works fine, 100% of the time
You can use netstat to check that the server is listening on the port and connections are being established. Snoop is another good application in your Armour for finding out what is going wrong. Another possibility is to use telnet to see if the client can connect to that IP address and port. As to the code I will take a look at it later to see if something has gone awry.
socket is a global variable. It might be re-used concurrently between two threads or sequentially inside one thread. In fact, the while(~closeProgram) loop indicates that you intend to use it sequentially.
Some documentation for CActiveSocket::Open reads: "Connection-based protocol sockets (CSocket::SocketTypeTcp) may successfully call Open() only once..."
Perhaps your program fails when you call .Open() twice on the same object.
I eventually found out the problem with my code. As the connection was unstable and working for 70% of the time it seemed to be a timeout issue. I removed the two timeout settings
socket.SetConnectTimeout(0, 20);
socket.SetSendTimeout(0, 20);
Now it works perfectly fine, thanks for the troubleshooting tips though!