server and multiple clients using pthreads and select() - c++

consider the next piece of code -
int get_ready_connection(int s) {
/* socket of connection */
int caller;
if ((caller = accept(s,NULL,NULL)) < SUCCESS)
{
server_log->write_to_log(sys_call_error(SERVER, "accept"));
return FAILURE;
}
return caller;
}
int establish_connection(sockaddr_in& connection_info)
{
// Create socket
if ((server_sock = socket(AF_INET, SOCK_STREAM, 0)) < SUCCESS)
{
server_log->write_to_log(sys_call_error(SERVER, "socket"));
return FAILURE;
}
// Bind `sock` with given addresses
if (bind(server_sock, (struct sockaddr *) &connection_info, \
sizeof(struct sockaddr_in)) < SUCCESS)
{
close(server_sock);
server_log->write_to_log(sys_call_error(SERVER, "bind"));
return FAILURE;
}
// Max # of queued connects
if (listen(server_sock, MAX_PENDING_CONNECTIONS) < SUCCESS)
{
server_log->write_to_log(sys_call_error(SERVER, "listen"));
return FAILURE;
}
// Create a set of file descriptors and empty it.
fd_set set;
bool is_inside;
int ret_val;
while(true)
{
FD_ZERO(&set);
FD_SET(STDIN_FILENO, &set);
FD_SET(server_sock, &set);
struct timeval tv = {2, 0};
ret_val = select(server_sock + 1, &set, NULL, NULL, &tv); // TODO ret_val
is_inside = FD_ISSET(STDIN_FILENO, &set);
if(is_inside)
{
// get user input
string user_input;
getline(cin, user_input);
if ((strcasecmp(user_input.c_str(), EXIT_TEXT) == 0))
{
return SUCCESS;
}
}
is_inside = FD_ISSET(server_sock, &set);
if(is_inside)
{
// get the first connection request
int current_connection = get_ready_connection(server_sock);
if (current_connection == FAILURE) {
free_allocated_memory();
exit_write_close(server_log, sys_call_error(SERVER, "accept"),
ERROR);
}
// if exit was not typed by the server's stdin, process the request
pthread_t thread;
// create thread
if (pthread_create(&thread, NULL, command_thread_func, &current_connection) != 0)
{
free_allocated_memory();
exit_write_close(server_log, sys_call_error(SERVER, "pthread_create"), ERROR);
}
}
}
}
All im trying to do, is to "listen" to STDIN for the user to type 'EXIT' in server's shell, and to wait for clients to pass commands from their shells (every time a command is recieved by the server from the user, the server parses it, and the server creates a thread that handles execution of the command)
To do it simultaniously, i used select().
When i work with a single thread, everything's perfect. But the problem is when im connecting another client i get a seg fault. i suspect that the problem is right here. any suggestions?

Hard to know if this is your exact problem, but this is definitely a problem:
You can't call pthread_create and provide a pointer to a stack variable (&current_connection) as your thread function's argument. For one thing, it's subject to immediate destruction as soon as the parent exits that scope.
Secondly, it will be overwritten on the next call to get_ready_connection.

Related

Let recv() only the previous accepted socket

I am using this server application:
I'd like to add some conditions to FD_ISSET() before recv():
if (`client's socket` was the previous `accepted socket`) {
canRecv = TRUE;
} else {
canRecv = FALSE;
}
This is my idea of ​​program functionality:
recv only from the previous accepted socket
Wait for the communication to end
FD_CLR()
I don't know how to:
loop through each fd from select()
let only one recv()
return the others to the queue of select()
I use simple example from IBM Knowledge Center:
https://www.ibm.com/support/knowledgecenter/ssw_ibm_i_72/rzab6/xnonblock.htm
You could create a std::vector<int> sockets; to keep your sockets. Checking if it's the latest you added will then be done by just checking if(current_socket == sockets[sockets.size()-1]) ...
Here's an example with a helper class to keep a list of your sockets and function for waiting on activity.
#include <cerrno>
#include <cstring>
#include <utility>
#include <vector>
constexpr unsigned other_socket = 0b00;
constexpr unsigned server_socket = 0b01;
constexpr unsigned latest_addition = 0b10;
class SocketList {
public:
explicit SocketList(int server) : readfds{} { add(server); }
void add(int s) {
sockets.push_back(s);
FD_SET(s, &readfds);
if(s > max_fd) max_fd = s;
}
// return the ready sockets and a state for each
std::vector<std::pair<int, unsigned>> wait() {
int ready_sockets;
do {
ready_sockets = select(max_fd + 1, &readfds, nullptr, nullptr, nullptr);
} while(ready_sockets == -1 && errno == EINTR); // retry if interrupted
// throw if an error occured
if(ready_sockets == -1) throw std::runtime_error(std::strerror(errno));
std::vector<std::pair<int, unsigned>> result;
// loop through each fd used in the select()
for(int s : sockets) {
if(FD_ISSET(s, &readfds)) {
auto x = other_socket;
if(s == sockets[0]) x |= server_socket;
if(s == sockets[sockets.size() - 1]) x |= latest_addition;
result.emplace_back(s, x);
}
}
return result;
}
private:
int max_fd = 0;
fd_set readfds;
std::vector<int> sockets;
};
It can be used like this:
int server = socket(...);
SocketList ss(server);
// all sockets in result are ready
auto result = ss.wait();
for(auto [sock, state] : result) {
if(state & server_socket) {
// do server things on sock
} else if(state & latest_addition) {
// do stuff if sock was the latest addition
} else {
// do this if sock is not the server socket or the latest addition
}
}
recv only from the previous accepted socket
Wait for the communication to end
FD_CLR()
For that you really don't need select. Just recv directly on the previously accepted socket. This is usually not a good behavior of a server that is supposed to server many clients simultaneously since a bad client could connect without sending anything, and that would stop the server from responding to any new clients - until the bad client decides to disconnect (if that ever happens).
I don't know how to:
1. loop through each fd from select()
That is shown in the code above.
let only one recv()
When you have the result vector in the example above, you can loop through them and only keep the part dealing with latest_addition:
if(state & latest_addition) {
// do stuff if sock was the latest addition
}
return the others to the queue of select()
The state of the other ready sockets in result will remain unchanged if you don't read from them, so they are returned automatically. This also means that the next select will return immediately if you don't read from all fds that are ready, so the program will spin really fast until there's some action on the latest added socket again, effectively making this a polling program and the select is sort of useless.

how to wakeup select() within timeout from another thread

According to the "man select" information:
"On success, select() and pselect() return the number of file descrip‐
tors contained in the three returned descriptor sets which may be zero
if the timeout expires before anything interesting happens. On error,
-1 is returned, and errno is set appropriately; the sets and timeout become
undefined, so do not rely on their contents after an error."
Select will wakup because of:
1)read/write availability
2)select error
3)descriptoris closed.
However, how can we wake up the select() from another thread if there is no data available and the select is still within timeout?
[update]
Pseudo Code
// Thread blocks on Select
void *SocketReadThread(void *param){
...
while(!(ReadThread*)param->ExitThread()) {
struct timeval timeout;
timeout.tv_sec = 60; //one minute
timeout.tv_usec = 0;
fd_set rds;
FD_ZERO(&rds);
FD_SET(sockfd, &rds)'
//actually, the first parameter of select() is
//ignored on windows, though on linux this parameter
//should be (maximum socket value + 1)
int ret = select(sockfd + 1, &rds, NULL, NULL, &timeout );
//handle the result
//might break from here
}
return NULL;
}
//main Thread
int main(){
//create the SocketReadThread
ReaderThread* rthread = new ReaderThread;
pthread_create(&pthreadid, NULL, SocketReaderThread,
NULL, (void*)rthread);
// do lots of things here
............................
//now main thread wants to exit SocketReaderThread
//it sets the internal state of ReadThread as true
rthread->SetExitFlag(true);
//but how to wake up select ??????????????????
//if SocketReaderThread currently blocks on select
}
[UPDATE]
1) #trojanfoe provides a method to achieve this, his method writes socket data (maybe dirty data or exit message data) to wakeup select. I am going to have a test and update the result there.
2) Another thing to mention, closing a socket doesn't guarantee to wake up select function call, please see this post.
[UPDATE2]
After doing many tests, here are some facts about waking up select:
1) If the socket watched by select is closed by another application, then select() calling
will wakeup immediately. Hereafter, reading from or writing to the socket will get return value of 0 with an errno = 0
2) If the socket watched by select is closed by another thread of the same application,
then select() won't wake up until timeout if there is no data to read or write. After select timeouts, making read/write operation results in an error with errno = EBADF
(because the socket has been closed by another thread during timeout period)
I use an event object based on pipe():
IoEvent.h:
#pragma once
class IoEvent {
protected:
int m_pipe[2];
bool m_ownsFDs;
public:
IoEvent(); // Creates a user event
IoEvent(int fd); // Create a file event
IoEvent(const IoEvent &other);
virtual ~IoEvent();
/**
* Set the event to signalled state.
*/
void set();
/**
* Reset the event from signalled state.
*/
void reset();
inline int fd() const {
return m_pipe[0];
}
};
IoEvent.cpp:
#include "IoEvent.h"
#include <string.h>
#include <unistd.h>
#include <errno.h>
#include <fcntl.h>
#include <poll.h>
using namespace std;
IoEvent::IoEvent() :
m_ownsFDs(true) {
if (pipe(m_pipe) < 0)
throw MyException("Failed to create pipe: %s (%d)", strerror(errno), errno);
if (fcntl(m_pipe[0], F_SETFL, O_NONBLOCK) < 0)
throw MyException("Failed to set pipe non-blocking mode: %s (%d)", strerror(errno), errno);
}
IoEvent::IoEvent(int fd) :
m_ownsFDs(false) {
m_pipe[0] = fd;
m_pipe[1] = -1;
}
IoEvent::IoEvent(const IoEvent &other) {
m_pipe[0] = other.m_pipe[0];
m_pipe[1] = other.m_pipe[1];
m_ownsFDs = false;
}
IoEvent::~IoEvent() {
if (m_pipe[0] >= 0) {
if (m_ownsFDs)
close(m_pipe[0]);
m_pipe[0] = -1;
}
if (m_pipe[1] >= 0) {
if (m_ownsFDs)
close(m_pipe[1]);
m_pipe[1] = -1;
}
}
void IoEvent::set() {
if (m_ownsFDs)
write(m_pipe[1], "x", 1);
}
void IoEvent::reset() {
if (m_ownsFDs) {
uint8_t buf;
while (read(m_pipe[0], &buf, 1) == 1)
;
}
}
You could ditch the m_ownsFDs member; I'm not even sure I use that any more.

C++ select stops accepting connections

I'm trying to make a select-server in order to receive connection from several clients (all clients will connect to the same port).
The server accepts the first 2 clients, but unless one of them disconnects, it will not accept a new one.
I'm starting to listen the the server port like this:
listen(m_socketId, SOMAXCONN);
and using the select command like this:
int selected = select(m_maxSocketId + 1, &m_socketReadSet, NULL, NULL, 0);
I've added some code.
bool TcpServer::Start(char* ipAddress, int port)
{
m_active = true;
FD_ZERO(&m_socketMasterSet);
bool listening = m_socket->Listen(ipAddress, port);
// Start listening.
m_maxSocketId = m_socket->GetId();
FD_SET(m_maxSocketId, &m_socketMasterSet);
if (listening == true)
{
StartThread(&InvokeListening);
StartReceiving();
return true;
}
else
{
return false;
}
}
void TcpServer::Listen()
{
while (m_active == true)
{
m_socketReadSet = m_socketMasterSet;
int selected = select(m_maxSocketId + 1, &m_socketReadSet, NULL, NULL, 0);
if (selected <= 0)
continue;
bool accepted = Accept();
if (accepted == false)
{
ReceiveFromSockets();
}
}
}
bool TcpServer::Accept()
{
int listenerId = m_socket->GetId();
if (FD_ISSET(listenerId, &m_socketReadSet) == true)
{
struct sockaddr_in remoteAddr;
int addrSize = sizeof(remoteAddr);
unsigned int newSockId = accept(listenerId, (struct sockaddr *)&remoteAddr, &addrSize);
if (newSockId == -1) // Invalid socket...
{
return false;
}
if (newSockId > m_maxSocketId)
{
m_maxSocketId = newSockId;
}
m_clientUniqueId++;
// Remembering the new socket, so we'll be able to check its state
// the next time.
FD_SET(newSockId, &m_socketMasterSet);
CommEndPoint remote(remoteAddr);
CommEndPoint local = m_socket->GetLocalPoint();
ClientId* client = new ClientId(m_clientUniqueId, newSockId, local, remote);
m_clients.Add(client);
StoreNewlyAcceptedClient(client);
char acceptedMsg = CommInternalServerMsg::ConnectionAccepted;
Server::Send(CommMessageType::Internal, client, &acceptedMsg, sizeof(acceptedMsg));
return true;
}
return false;
}
I hope it's enough :)
what's wrong with it?
The by far most common error with select() is not re-initializing the fd sets on every iteration. The second, third, and forth arguments are updated by the call, so you have to populate them again every time.
Post more code, so people can actually help you.
Edit 0:
fd_set on Windows is a mess :)
It's not allowed to copy construct fd_set objects:
m_socketReadSet = m_socketMasterSet;
This combined with Nikolai's correct statement that select changes the set passed in probably accounts for your error.
poll (On Windows, WSAPoll) is a much friendlier API.
Windows also provides WSAEventSelect and (Msg)WaitForMultipleObjects(Ex), which doesn't have a direct equivalent on Unix, but allows you to wait on sockets, files, thread synchronization events, timers, and UI messages at the same time.

select() behaviour for writeability?

I have a fd_set "write_set" which contains sockets that I want to use in a send(...) call. When I call select(maxsockfd+1, NULL, &write_set, NULL, &tv) there it always returns 0 (timeout) although I haven't sent anything over the sockets in the write_set yet and it should be possible to send data.
Why is this? Shouldn't select return instantly when it's possible to send data over the sockets in write_set?
Thanks!
Edit: My code..
// _read_set and _write_set are the master sets
fd_set read_set = _read_set;
fd_set write_set = _write_set;
// added this for testing, the socket is a member of RemoteChannelConnector.
std::list<RemoteChannelConnector*>::iterator iter;
for (iter = _acceptingConnectorList->begin(); iter != _acceptingConnectorList->end(); iter++) {
if(FD_ISSET((*iter)->getSocket(), &write_set)) {
char* buf = "a";
int ret;
if ((ret = send((*iter)->getSocket(), buf, 1, NULL)) == -1) {
std::cout << "error." << std::endl;
} else {
std::cout << "success." << std::endl;
}
}
}
struct timeval tv;
tv.tv_sec = 10;
tv.tv_usec = 0;
int status;
if ((status = select(_maxsockfd, &read_set, &write_set, NULL, &tv)) == -1) {
// Terminate process on error.
exit(1);
} else if (status == 0) {
// Terminate process on timeout.
exit(1);
} else {
// call send/receive
}
When I run it with the code for testing if my socket is actually in the write_set and if it is possible to send data over the socket, I get a "success"...
I don't believe that you're allowed to copy-construct fd_set objects. The only guaranteed way is to completely rebuild the set using FD_SET before each call to select. Also, you're writing to the list of sockets to be selected on, before ever calling select. That doesn't make sense.
Can you use poll instead? It's a much friendlier API.
Your code is very confused. First, you don't seem to be setting any of the bits in the fd_set. Secondly, you test the bits before you even call select.
Here is how the flow generally works...
Use FD_ZERO to zero out your set.
Go through, and for each file descriptor you're interested in the writeable state of, use FD_SET to set it.
Call select, passing it the address of the fd_set you've been calling the FD_SET function on for the write set and observe the return value.
If the return value is > 0, then go through the write set and use FD_ISSET to figure out which ones are still set. Those are the ones that are writeable.
Your code does not at all appear to be following this pattern. Also, the important task of setting up the master set isn't being shown.

Socket class Error

ListenSocket.h
// class does not contain WSASTARTUP () and WSACLEANUP ()
#ifndef LISTENTHREAD_H
#define LISTENTHREAD_H
#include "includes.h"
#include "LOGMSGs.h"
// 1, CListenSocket: class is used to create the listener thread local variable.
// This class can be reused. When you call Close () is closed, re-calling Open () the new listening port. But the system did not use the feature.
class CListenSocket
{
public:
// main method:
// BIND each object only to a port.
CListenSocket(u_short nPort, int nSndSize = 0);
// to release SOCKET
~CListenSocket(){};
// Create server listening SOCKET, specific options see the code. Fails to return false.
bool Open(); // call can be repeated
// error return INVALID_SOCKET
SOCKET Accept(u_long & nClientIP);
// repeated calls. Usually not, can be used to take the initiative to close the SOCKET.
// close the re-call after Open () re-use the object.
void Close(); // call can be repeated
bool IsOpen() { return m_bState; }
bool Rebuild();
public:
SOCKET Socket() { return m_sockListen; }
protected:
// main member variables:
const u_short m_nPort;
const int m_nSndBuf;
SOCKET m_sockListen;
// network status is normal sign.
// When the value is false that the object is not available. May not have Open (), may also be a network error.
bool m_bState;
time_t m_tCloseTime; // SOCKET last closed the time delay for the automatic re-SOCKET
};
#endif // LISTENTHREAD_H
ListenSocket.cpp
#include "ListenSocket.h"
long s_nSocketCount = 0;
int REBUILDLISTENDELAYSEC;
CListenSocket::CListenSocket(u_short nPort, int nSndBuf /*= 0*/) // 0: Default
: m_nPort(nPort), m_nSndBuf(nSndBuf)
{
m_sockListen = INVALID_SOCKET;
m_bState = false;
// m_nPort = nPort;
m_tCloseTime = 0;
}
// Error returned INVALID_SOCKET
SOCKET CListenSocket::Accept(u_long & nClientIP)
{
/*
// Reconstruction SOCKET
if(!m_bState)
{
if(clock() < m_tCloseTime + REBUILDLISTENDELAYSEC*CLOCKS_PER_SEC)
return INVALID_SOCKET;
else
{
LOGMSG("Anti-crash system start listening SOCKET [%d] re under construction...", m_nPort);
if(Open())
{
LOGMSG("... listen SOCKET reconstruction success.");
PrintText("Listen SOCKET [%d] failed to rebuild SOCKET success. Server continues to run in the ...", m_nPort);
}
else
{
Error("... listen SOCKET reconstruction has failed. Server will not accept new connections");
PrintText("Listen SOCKET [%d] error, [%d] seconds after the re-SOCKET. Server continues to run in the ...", m_nPort, REBUILDLISTENDELAYSEC); // nDelaySec);
}
m_tCloseTime = clock();
}
}
//*/
if(!m_bState)
{
Error("ACCEPT inner exception a1");
return INVALID_SOCKET;
}
// ACCEPT
struct sockaddr_in addr;
memset(&addr, 0, sizeof(addr));
int len = sizeof(addr);
SOCKET newsock = accept(m_sockListen, (sockaddr*)&addr, (int*)&len); // receive to the other side of the map, you can use
#ifdef PROFILE_X
// Analysis Accept speed (cycle speed)
const int nTimes2 = ACCEPTPROFILESEC; // Statistics once every 30 seconds the speed ACCEPT
static clock_t tNextTime2 = clock() + nTimes2 * CLOCKS_PER_SEC; //? Only one monitor thread, no sharing violation
static long nCount2 = 0; //? Only one monitor thread, no sharing violation
if(clock() >= tNextTime2)
{
PrintText("Each [%d] seconds to execute a [%d] times Accept ()", nTimes2, InterlockedExchange(&nCount2, 0));
tNextTime2 = clock() + nTimes2 * CLOCKS_PER_SEC;
}
else
{
InterlockedIncrement(&nCount2);
}
#endif // PROFILE
if(newsock == INVALID_SOCKET)
{
// Network Error
int err = WSAGetLastError();
if(err != WSAEWOULDBLOCK)
{
PrintText("Listen SOCKET %d failed, %s seconds after the re-SOCKET.", m_nPort, REBUILDLISTENDELAYSEC);
Error("Listen SOCKET [%d] failed [%d], [%s] seconds after the re-SOCKET.", m_nPort, err, REBUILDLISTENDELAYSEC);
Close();
}
else
Error("ACCEPT inner exception a2");
return INVALID_SOCKET;
}
else
{
nClientIP = addr.sin_addr.S_un.S_addr;
InterlockedIncrement(&s_nSocketCount);
}
// Check whether the SOCKET closed
fd_set readmask;
FD_ZERO(&readmask);
FD_SET(newsock, &readmask);
struct timeval timeout = {0, 0};
/*
char nTemp;
if(select(FD_SETSIZE, &readmask, (fd_set *) 0, (fd_set *) 0, &timeout)
&& recv(newsock, &nTemp, 1, MSG_PEEK) == 0)
{
#ifdef ALPHA_X
LOGMSG("ACCEPT a new SOCKET is invalid .");
#endif
closesocket(newsock);
InterlockedDecrement(&s_nSocketCount);
return INVALID_SOCKET;
}
//else*/
//*
fd_set exceptmask;
FD_ZERO(&exceptmask);
FD_SET(newsock, &exceptmask);
int ret = select(FD_SETSIZE, &readmask, (fd_set *) 0, (fd_set *) &exceptmask, &timeout);
if(ret < 0)
{
Error("ACCEPT a new SOCKET is invalid . can't read"); // Not trigger
closesocket(newsock);
InterlockedDecrement(&s_nSocketCount);
return INVALID_SOCKET;
}
else if(ret > 0)
{
if(FD_ISSET(newsock, &exceptmask))
{
LOGMSG("ACCEPT a new SOCKET is invalid.except"); // Not trigger
closesocket(newsock);
InterlockedDecrement(&s_nSocketCount);
return INVALID_SOCKET;
}
else if(FD_ISSET(newsock, &readmask))
{
char nTemp;
if(recv(newsock, &nTemp, 1, MSG_PEEK) == 0)
{
#ifdef ALPHA_X
LOGMSG("ACCEPT a new SOCKET is invalid. recv==0"); // Not trigger
#endif
closesocket(newsock);
InterlockedDecrement(&s_nSocketCount);
return INVALID_SOCKET;
}
}
}
//*/
#ifdef PROFILE_X
// analysis Accept speed (received valid SOCKET)
const int nTimes = ACCEPTPROFILESEC; // Statistics once every 10 seconds the speed ACCEPT
static clock_t tNextTime = clock() + nTimes * CLOCKS_PER_SEC; //? Only one monitor thread, no sharing violation
static long nCount = 0; //? Only one monitor thread, no sharing violation
if(clock() >= tNextTime)
{
LOGPROFILE("Port [%d] for every [%d] seconds, the successful implementation of the [%d] times Accept()",
m_nPort, nTimes, InterlockedExchange(&nCount, 0));
tNextTime = clock() + nTimes * CLOCKS_PER_SEC;
}
else
{
InterlockedIncrement(&nCount);
}
#endif // PROFILE
return newsock;
}
Main.cpp
#include "includes.h"
#include "IniFile.h"
#include "LOGMSGs.h"
#include "ListenSocket.h"
CListenSocket Sock(9985);
int main()
{
Sock.Open();
if(!Sock.Open())
{
Sock.Rebuild();
}
if(Sock.IsOpen())
PrintText("okey");
Sock.Socket();
u_long ip;
Sock.Accept(ip);
}
but i always got this error : ACCEPT inner exception a2 while it should work anyclue why?
CListenSocket Sock(9985);
int main()
{
Sock.Open();
if(!Sock.Open())
/* I think you meant 'IsOpen()' */
{
Sock.Rebuild();
}
if(Sock.IsOpen())
PrintText("okey");
Sock.Socket();
u_long ip;
Sock.Accept(ip);
}
Incidentally, this code sure reads funny. It feels like a generic toolkit programmed without a specific goal in mind. Maybe I'm missing it, but I have to think you'd have better results if you just wrote the network code that actually needed, and then abstract out the common bits into some helper routines later. There's no point in trying too hard to make the be-all and end-all network helper library, but there is a huge point in making tools that collapse common cases.
Feel free to ignore that last paragraph if you know what you're doing :) but if you're just starting out, I'd like to suggest writing a few smaller clients and servers, and then try writing your abstraction layer.