Writing Data to Poll Invalid Socket Causes Uncatchable Exception - c++

I am working on a game server that uses sockets and implemented a polling function that sends the message "[POLL]" over all player sockets in a lobby every second to notify the player clients that their connection is still alive.
If I disconnect on the client-side the socket is still polled with no errors, however, if I create a new connection with the same client (Gets a new FD and is added to the map as a second player), the whole server crashes without any exceptions/warnings/messages when it attempts to write to the previous socket FD. My call to Write on the socket is wrapped in a try/catch that doesn't catch any exceptions and, when debugging using gdb, I am not given any error messaging.
This is the Socket Write function:
int Socket::Write(ByteArray const& buffer)
{
if (!open)
{
return -1;
}
// Convert buffer to raw char array
char* raw = new char[buffer.v.size()];
for (int i=0; i < buffer.v.size(); i++)
{
raw[i] = buffer.v[i];
}
// Perform the write operation
int returnValue = write(GetFD(), raw, buffer.v.size()); // <- Crashes program
if (returnValue <= 0)
{
open = false;
}
return returnValue;
}
And this is the Poll function (Players are stored in a map of uint -> Socket*):
/*
Polls all connected players to tell them
to keep their connections alive.
*/
void Lobby::Poll()
{
playerMtx.lock();
for (auto it = players.begin(); it != players.end(); it++)
{
try
{
if (it->second != nullptr && it->second->IsOpen())
{
it->second->Write("[POLL]");
}
}
catch (...)
{
std::cout << "Failed to write to " << it->first << std::endl;
}
}
playerMtx.unlock();
}
I would expect to see the "Failed to write to " message but instead the entire server program exits with no messaging. What could be happening here?

I was unable to find a reason for the program crashing in the call to write but I was able to find a workaround.
I perform a poll operation on the file descriptor prior to calling write and I query the POLLNVAL event. If I receive a nonzero value, the FD is now invalid.
// Check if FD is valid
struct pollfd pollFd;
pollFd.fd = GetFD();
pollFd.events = POLLNVAL;
if (poll(&pollFd, 1, 0) > 0)
{
open = false;
return -1;
}

Related

Socket Handle Read Is never read In Ns3

I stayed many days to find where the error is and I'm stuck.Can anyone help me please.
MY application is a clustering program in NS3 that was written from one else and free to anyone.
The program is run and no errors and print messages but always the number of neighbors in every Cluster Head CH is zero, this mean that the hellow messages are not reach every node and every node consider itself as cluster head because it dosn't see any neighbor node!. every node (vehicle) has two sockets one for send data m_socket and one for listening m_socketlistening, the code is:
if (!m_socket)
{
// TypeId::LookupByName ("ns3::UdpSocketFactory
TypeId m_tid = TypeId::LookupByName("ns3::UdpSocketFactory");
//m_socket = Socket::CreateSocket(GetNode() , TypeId::LookupByName("ns3::UdpSocketFactory"));
m_socket = Socket::CreateSocket(GetNode(), m_tid);
// i added the down line
// InetSocketAddress remote = InetSocketAddress(Ipv4Address::GetBroadcast(),80);
if (Inet6SocketAddress::IsMatchingType(m_peer))
{
m_socket->Bind6();
}
else if (InetSocketAddress::IsMatchingType(m_peer)
|| PacketSocketAddress::IsMatchingType(m_peer))
{
m_socket->Bind();
}
m_socket->SetAllowBroadcast(true);
m_socket->ShutdownRecv();
m_socket->SetConnectCallback(
MakeCallback(&V2vControlClient::ConnectionSucceeded, this),
MakeCallback(&V2vControlClient::ConnectionFailed, this));
//m_socket->Connect(Ipv4Address::GetBroadcast());
m_socket->Connect(m_peer);
}
now this is a part pf the listening socket creation
if (!m_socketListening)
{
NS_LOG_UNCOND("\n ...creating socket muhsen...");
m_socketListening = Socket::CreateSocket(GetNode(), m_tidListening);
m_socketListening->Bind(m_peerListening);
m_socketListening->Listen();
m_socketListening->ShutdownSend();
if (addressUtils::IsMulticast(m_peerListening))
{
Ptr<UdpSocket> udpSocket = DynamicCast<UdpSocket>(m_socketListening);
if (udpSocket)
{
// equivalent to setsockopt (MCAST_JOIN_GROUP)
udpSocket->MulticastJoinGroup(0, m_peerListening);
}
else
{
NS_FATAL_ERROR("Error: joining multicast on a non-UDP socket");
}
}
}
m_socketListening->SetRecvCallback(MakeCallback(&V2vControlClient::HandleRead, this));
m_socketListening->SetAcceptCallback(
MakeNullCallback<bool, Ptr<Socket>, const Address &>(),
MakeCallback(&V2vControlClient::HandleAccept, this));
m_socketListening->SetCloseCallbacks(
MakeCallback(&V2vControlClient::HandlePeerClose, this),
MakeCallback(&V2vControlClient::HandlePeerError, this));
void V2vControlClient::HandleRead (Ptr<Socket> socket)
{
NS_LOG_UNCOND("\n this message is never executed..");
NS_LOG_FUNCTION (this << socket);
Ptr<Packet> packet;
Address from;
while ((packet = socket->RecvFrom(from)))
{
if (packet->GetSize() == 0)
{ //EOF
break;
}
When i run the application the first statement after the HandleRead Function which is
NS_LOG_UNCOND("\n this message is never executed..");
is never printed when run the program, this means that the handle read is never executed.
Any help is very appreciate!

Reason for losing messeges over NNG sockets in raw mode

Some context to my problem:
I need to establish an inter-process communication using C++ and sockets and I picked NNG library for that along with nngpp c++ wrapper. I need to use push/pull protocol so no contexts handling is available to me. I wrote some code based on raw example from nngpp demo. The difference here is that, by using push/pull protocol I split this into two separate programs. One for sending and one for receiving.
Problem descripion:
I need to receive let's say a thousand or more messages per second. For now, all messages are captured only when I send about 50/s. That is way too slow and I do believe it can be done faster. The faster I send, the more I lose. At the moment, when sending 1000msg/s I lose about 150 msgs.
Some words about the code
The code may be in C++17 standard. It is written in object-oriented manner so in the end I want to have a class with "receive" method that would simply give me the received messages. For now, I just print the results on screen. Below, I supply some parts of the project with descriptions:
NOTE msgItem is a struct like that:
struct msgItem {
nng::aio aio;
nng::msg msg;
nng::socket_view itemSock;
explicit msgItem(nng::socket_view sock) : itemSock(sock) {}
};
And it is taken from example mentioned above.
Callback function that is executed when message is received by one of the aio's (callback is passed in constructor of aio object). It aims at checking whether everything was ok with transmission, retrieving my Payload (just string for now) and passing it to queue while a flag is set. Then I want to print those messages from the queue using separate thread.
void ReceiverBase<Payload>::aioCallback(void *arg) try {
msgItem *msgItem = (struct msgItem *)arg;
Payload retMsg{};
auto result = msgItem->aio.result();
if (result != nng::error::success) {
throw nng::exception(result);
}
//Here we extract the message
auto msg = msgItem->aio.release_msg();
auto const *data = static_cast<typename Payload::value_type *>(msg.body().data());
auto const count = msg.body().size()/sizeof(typename Payload::value_type);
std::copy(data, data + count, std::back_inserter(retMsg));
{
std::lock_guard<std::mutex> lk(m_msgMx);
newMessageFlag = true;
m_messageQueue.push(std::move(retMsg));
}
msgItem->itemSock.recv(msgItem->aio);
} catch (const nng::exception &e) {
fprintf(stderr, "server_cb: %s: %s\n", e.who(), e.what());
} catch (...) {
fprintf(stderr, "server_cb: unknown exception\n");
}
Separate thread for listening to the flag change and printing. While loop at the end is for continuous work of the program. I use msgCounter to count successful message receival.
void ReceiverBase<Payload>::start() {
auto listenerLambda = [](){
std::string temp;
while (true) {
std::lock_guard<std::mutex> lg(m_msgMx);
if(newMessageFlag) {
temp = std::move(m_messageQueue.front());
m_messageQueue.pop();
++msgCounter;
std::cout << msgCounter << "\n";
newMessageFlag = false;
}}};
std::thread listenerThread (listenerLambda);
while (true) {
std::this_thread::sleep_for(std::chrono::microseconds(1));
}
}
This is my sender application. I tweak the frequency of msg sending by changing the value in std::chrono::miliseconds(val).
int main (int argc, char *argv[])
{
std::string connection_address{"ipc:///tmp/async_demo1"};
std::string longMsg{" here normally I have some long test text"};
std::cout << "Trying connecting sender:";
StringSender sender(connection_address);
sender.setupConnection();
for (int i=0; i<1000; ++i) {
std::this_thread::sleep_for(std::chrono::milliseconds(3));
sender.send(longMsg);
}
}
And this is receiver:
int main (int argc, char *argv[])
{
std::string connection_address{"ipc:///tmp/async_demo1"};
std::cout << "Trying connecting receiver:";
StringReceiver receiver(connection_address);
receiver.setupConnection();
std::cout<< "Connection set up. \n";
receiver.start();
return 0;
}
Nothing speciall in those two applications as You see. the setup method from StringReciver is something like that:
bool ReceiverBase<Payload>::setupConnection() {
m_connected = false;
try {
for (size_t i = 0; i < m_parallel; ++i) {
m_msgItems.at(i) = std::make_unique<msgItem>(m_sock);
m_msgItems.at(i)->aio =
nng::aio(ReceiverBase::aioCallback, m_msgItems.at(i).get());
}
m_sock.listen(m_adress.c_str());
m_connected = true;
for (size_t i = 0; i < m_parallel; ++i) {
m_msgItems.at(i)->itemSock.recv(m_msgItems.at(i)->aio);
}
} catch (const nng::exception &e) {
printf("%s: %s\n", e.who(), e.what());
}
return m_connected;
}
Do You have any suggestions why the performance is so low? Do I use lock_guards properly here? What I want them to do is basically lock the flag and queue so only one side has access to it.
NOTE: Adding more listeners thread does not affect the performance either way.
NOTE2: newMessageFlag is atomic

Sockets - keeping a socket open after data transfer

I have written simple server/client programs, in which the client sends some hardcoded data in small chunks to the server program, which is waiting for the data so that it can print it to the terminal. In the client, I'm calling send() in a loop while there is more data to send, and on the server, I'm doing the same with read(), that is, while the number of bytes returned is > 0, I continue to read.
This example works perfectly if I specifically call close() on the client's socket after I've finished sending, but if I don't, the server won't actually exit the read() loop until I close the client and break the connection. On the server side, I'm using:
while((bytesRead = read(socket, buffer, BUFFER_SIZE)) > 0)
Shouldn't bytesRead be 0 when all the data has been received? And if so, why will it not exit this loop until I close the socket? In my final application, it will be beneficial to keep the socket open between requests, but all of the sample code and information I can find calls close() immediately after sending data, which is not what I want.
What am I missing?
When the other end of the socket is connected to some other network system halfway around the world, the only way that the receiving socket knows "when all the data has been received" is precisely when the other side of the socket is closed. That's what tells the other side of the socket that "all the data has been received".
All that a socket knows about is that it's connected to some other socket endpoint. That's it. End of story. The socket has no special knowledge of the inner workings of the program that has the other side of the socket connection. Nor should it know. That happens to be the responsibility of the program that has the socket open, and not the socket itself.
If your program, on the receiving side, has knowledge -- by the virtue of knowing what data it is expected to receive -- that it has now received everything that it needs to receive, then it can close its end of the socket, and move on to the next task at hand.
You will have to incorporate in your program's logic, a way to determine, in some form or fashion, that all the data has been transmitted. The exact nature of that is going to be up to you to define. Perhaps, before sending all the data on the socket, your sending program will send in advance, on the same socket, the number of bytes that will be in the data to follow. Then, your receiving program reads the number of bytes first, followed by the data itself, and then knows that it has received everything, and can move on.
That's one simplistic approach. The exact details is up to you. Alternatively, you can also implement a timeout: set a timer and if any data is not received in some prescribed period of time, assume that there is no more.
You can set a flag on the recv call to prevent blocking.
One way to detect this easily is to wrap the recv call:
enum class read_result
{
// note: numerically in increasing order of severity
ok,
would_block,
end_of_file,
error,
};
template<std::size_t BufferLength>
read_result read(int socket_fd, char (&buffer)[BufferLength], int& bytes_read)
{
auto result = recv(socket_fd, buffer, BufferLength, MSG_DONTWAIT);
if (result > 0)
{
return read_result::ok;
}
else if (result == 0)
{
return read_result::end_of_file;
}
else {
auto err = errno;
if (err == EAGAIN or err == EWOULDBLOCK) {
return read_result::would_block;
}
else {
return read_result ::error;
}
}
}
One use case might be:
#include <unistd.h>
#include <sys/socket.h>
#include <cstdlib>
#include <cerrno>
#include <iostream>
enum class read_result
{
// note: numerically in increasing order of severity
ok,
would_block,
end_of_file,
error,
};
template<std::size_t BufferLength>
read_result read(int socket_fd, char (&buffer)[BufferLength], int& bytes_read)
{
auto result = recv(socket_fd, buffer, BufferLength, MSG_DONTWAIT);
if (result > 0)
{
return read_result::ok;
}
else if (result == 0)
{
return read_result::end_of_file;
}
else {
auto err = errno;
if (err == EAGAIN or err == EWOULDBLOCK) {
return read_result::would_block;
}
else {
return read_result ::error;
}
}
}
struct keep_reading
{
keep_reading& operator=(read_result result)
{
result_ = result;
}
const operator bool() const {
return result_ < read_result::end_of_file;
}
auto get_result() const -> read_result { return result_; }
private:
read_result result_ = read_result::ok;
};
int main()
{
int socket; // = open my socket and wait for it to be connected etc
char buffer [1024];
int bytes_read = 0;
keep_reading should_keep_reading;
while(keep_reading = read(socket, buffer, bytes_read))
{
if (should_keep_reading.get_result() != read_result::would_block) {
// read things here
}
else {
// idle processing here
}
}
std::cout << "reason for stopping: " << should_keep_reading.get_result() << std::endl;
}

boost::asio write: Broken pipe

I have a TCP server that handles new connections, when there's a new connection two threads will be created (std::thread, detached).
void Gateway::startServer(boost::asio::io_service& io_service, unsigned short port) {
tcp::acceptor TCPAcceptor(io_service, tcp::endpoint(tcp::v4(), port));
bool UARTToWiFiGatewayStarted = false;
for (;;) { std::cout << "\nstartServer()\n";
auto socket(std::shared_ptr<tcp::socket>(new tcp::socket(io_service)));
/*!
* Accept a new connected WiFi client.
*/
TCPAcceptor.accept(*socket);
socket->set_option( tcp::no_delay( true ) );
// This will set the boolean `Gateway::communicationSessionStatus` variable to true.
Gateway::enableCommunicationSession();
// start one thread
std::thread(WiFiToUARTWorkerSession, socket, this->SpecialUARTPort, this->SpecialUARTPortBaud).detach();
// start the second thread
std::thread(UARTToWifiWorkerSession, socket, this->UARTport, this->UARTbaud).detach();
}
}
The first of two worker functions look like this (here I'm reading using the shared socket):
void Gateway::WiFiToUARTWorkerSession(std::shared_ptr<tcp::socket> socket, std::string SpecialUARTPort, unsigned int baud) {
std::cout << "\nEntered: WiFiToUARTWorkerSession(...)\n";
std::shared_ptr<FastUARTIOHandler> uart(new FastUARTIOHandler(SpecialUARTPort, baud));
try {
while(true == Gateway::communicationSessionStatus) { std::cout << "WiFi->UART\n";
unsigned char WiFiDataBuffer[max_incoming_wifi_data_length];
boost::system::error_code error;
/*!
* Read the TCP data.
*/
size_t length = socket->read_some(boost::asio::buffer(WiFiDataBuffer), error);
/*!
* Handle possible read errors.
*/
if (error == boost::asio::error::eof) {
// this will set the shared boolean variable from "true" to "false", causing the while loop (from the both functions and threads) to stop.
Gateway::disableCommunicationSession();
break; // Connection closed cleanly by peer.
}
else if (error) {
Gateway::disableCommunicationSession();
throw boost::system::system_error(error); // Some other error.
}
uart->write(WiFiDataBuffer, length);
}
}
catch (std::exception &exception) {
std::cerr << "[APP::exception] Exception in thread: " << exception.what() << std::endl;
}
std::cout << "\nExiting: WiFiToUARTWorkerSession(...)\n";
}
And the second one (here I'm writing using the thread-shared socket):
void Gateway::UARTToWifiWorkerSession(std::shared_ptr<tcp::socket> socket, std::string UARTport, unsigned int baud) {
std::cout << "\nEntered: UARTToWifiWorkerSession(...)\n";
/*!
* Buffer used for storing the UART-incoming data.
*/
unsigned char UARTDataBuffer[max_incoming_uart_data_length];
std::vector<unsigned char> outputBuffer;
std::shared_ptr<FastUARTIOHandler> uartHandler(new FastUARTIOHandler(UARTport, baud));
while(true == Gateway::communicationSessionStatus) { std::cout << "UART->WiFi\n";
/*!
* Read the UART-available data.
*/
auto bytesReceived = uartHandler->read(UARTDataBuffer, max_incoming_uart_data_length);
/*!
* If there was some data, send it over TCP.
*/
if(bytesReceived > 0) {
boost::asio::write((*socket), boost::asio::buffer(UARTDataBuffer, bytesReceived));
std::cout << "\nSending data to app...\n";
}
}
std::cout << "\nExited: UARTToWifiWorkerSession(...)\n";
}
For stopping this two threads I do the following thing: from the WiFiToUARTWorkerSession(...) function, if the read(...) fails (there's an error like boost::asio::error::eof, or any other error) I set the Gateway::communicationSessionStatus boolean switch (which is shared (global) by the both functions) to false, this way the functions should return, and the threads should be killed gracefully.
When I'm connecting for the first time, this works well, but when I'm disconnecting from the server, the execution flow from the WiFiToUARTWorkerSession(...) goes through else if (error) condition, it sets the while condition variable to false, and then it throws boost::system::system_error(error) (which actually means Connection reset by peer).
Then when I'm trying to connect again, I got the following exception and the program terminates:
terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::system::system_error> >'
what(): write: Broken pipe
What could be the problem?
EDIT: From what I found about this error, it seems that I write(...) after the client disconnects, but how could this be possible?
EDIT2: I have debugged the code even more and it seems that one thread (on which runs the UARTToWifiWorkerSession(...) function) won't actually exit (because there's a blocking read(...) function call at where the execution flow stops). This way that one thread will hang until there's some data received by the read(...) function, and when I'm reconnecting there will be created another two threads, this causing some data racing problems.
Can someone confirm me that this could be the problem?
The actual problem was that the function UARTToWifiWorkerSession(...) didn't actually exit (because of a blocking read(...) function, this causing two threads (the hanging one, and one of the latest two created ones) to write(...) (without any concurrency control) using the same socket.
The solution was to set a read(...) timeout, so I can return from the function (and thus destroy the thread) without pending from some input.

C++ How to exit out of a while loop recvfrom()

I'm trying to create a UDP broadcast program to check for local game servers, but I'm having some trouble with the receiving end. Since the amount of servers alive is unknown at all times, you must have a loop that only exits when you stop it. So in this bit of code here:
while(1) // start a while loop
{
if(recvfrom(sd,buff,BUFFSZ,0,(struct sockaddr *)&peer,&psz) < 0) // recvfrom() function call
{
cout << red << "Fatal: Failed to receive data" << white << endl;
return;
}
else
{
cout << green << "Found Server :: " << white;
cout << yellow << inet_ntoa(peer.sin_addr), htons(peer.sin_port);
cout << endl;
}
}
I wish to run this recvfrom() function until I press Ctrl + C. I've tried setting up handlers and such (from related questions), but they're all either too complicated for me, or it's a simple function that just exits the program as a demonstration. Here's my problem:
The program hangs on recvfrom until it receives a connection (my guess), so, there's never a chance for it to specifically wait for input. How can I set up an event that will work into this nicely?
Thanks!
In the CTRL-C handler, set a flag, and use that flag as condition in the while loop.
Oh, and if you're not on a POSIX systems where system-calls can be interrupted by signals, you might want to make the socket non-blocking and use e.g. select (with a small timeout) to poll for data.
Windows have a couple of problems with a scheme like this. The major problem is that functions calls can not be interrupted by the CTRL-C handler. Instead you have to poll if there is anything to receive in the loop, while also checking the "exit loop" flag.
It could be done something like this:
bool ExitRecvLoop = false;
BOOL CtrlHandler(DWORD type)
{
if (type == CTRL_C_EVENT)
{
ExitRecvLoop = true;
return TRUE;
}
return FALSE; // Call next handler
}
// ...
SetConsoleCtrlHandler((PHANDLER_ROUTINE) CtrlHandler, TRUE);
while (!ExitRecvLoop)
{
fd_set rs;
FD_ZERO(&rs);
FD_SET(sd, &rs);
timeval timeout = { 0, 1000 }; // One millisecond
if (select(sd + 1, &rs, NULL, NULL, &timeout) < 0)
{
// Handle error
}
else
{
if (FD_ISSET(sd, &rs))
{
// Data to receive, call `recvfrom`
}
}
}
You might have to make the socket non-blocking for this to work (see the ioctlsocket function for how to).
Thread off your recvFrom() loop so that your main thread can wait for user input. When user requests stop, close the fd from the main thread and the recvFrom() will return immediately with an error, so allowing your recvFrom() thread to exit.