how to deal with multiple clients in c++ socket problem? - c++

Question

I need some help with a socket program with multiple clients and one server. To simplify, I create
3 socket clients
1 socket server
For each client, it opens a new connection for sending a new message and closes the connection after a response is received.
For the server, it does not need to deal with connections concurrently, it can deal with the message one by one
here is my code (runnable), compile it with /usr/bin/g++ mycode.cpp -g -lpthread -lrt -Wall -o mycode
#include <iostream>
#include <arpa/inet.h>
#include <string.h>
#include <sys/socket.h>
#include <unistd.h>
#include <unordered_map>
#include <thread>
using namespace std;
void Warning(string msg) { std::cout<< msg << std::endl; }
namespace mySocket {
class Memcached {
public:
// start a server
static void controller(int port=7111) { std::thread (server, port).detach(); }
// open a new connection to send a message:
// 1. open a connection
// 2. send the message
// 3. read the message
// 4. close the connection
std::string sendMessage(string msg, string host, int port=7111) {
int sock = 0, client_fd;
struct sockaddr_in serv_addr;
char buffer[1024] = { 0 };
if ((sock = socket(AF_INET, SOCK_STREAM, 0)) < 0) {
std::cout << "Socket creation error, msg: " << msg << ", host: " << host << ", port: " << port << std::endl;
exit(1);
}
serv_addr.sin_family = AF_INET;
serv_addr.sin_port = htons(port);
if (inet_pton(AF_INET, host.c_str(), &serv_addr.sin_addr) <= 0) {
std::cout << "\nInvalid address/ Address not supported, kmsgey: " << msg << ", host: " << host << ", port: " << port << std::endl;
exit(1);
}
while ((client_fd = connect(sock, (struct sockaddr*)&serv_addr, sizeof(serv_addr))) < 0) { sleep(10*1000); }
std::cout << "client sends a message:"<<msg<<", msg size:"<<msg.size()<<std::endl;
send(sock, msg.c_str(), msg.size(), 0);
read(sock, buffer, 1024);
close(client_fd);
return std::string(buffer, strlen(buffer));
}
private:
// start a server
// 1. open a file descriptor
// 2. listen the fd with queue size 10
// 3. accept one connection at a time
// 4. deal with message in the connection
// 5. accept the next connection
// 6. repeat step 3
static void server(int port) {
int server_fd, new_socket;
struct sockaddr_in address;
int opt = 1;
int addrlen = sizeof(address);
char buffer[1024] = { 0 };
unordered_map<string,string> data;
if ((server_fd = socket(AF_INET, SOCK_STREAM, 0)) == 0) {
Warning("socket failed"); exit(1);
}
if (setsockopt(server_fd, SOL_SOCKET, SO_REUSEADDR | SO_REUSEPORT, &opt, sizeof(opt))) {
Warning("setsockopt failed"); exit(1);
}
address.sin_family = AF_INET;
address.sin_addr.s_addr = INADDR_ANY;
address.sin_port = htons(port);
if (bind(server_fd, (struct sockaddr*)&address, sizeof(address)) < 0) {
Warning("bind failed"); exit(1);
}
// the queue size is 10 > 3
if (listen(server_fd, 10) < 0) {
Warning("listen failed"); exit(1);
}
while(1)
{
if ((new_socket = accept(server_fd, (struct sockaddr*)&address, (socklen_t*)&addrlen)) < 0) {
std::cout << "accept failed"; exit(1);
}
memset(&buffer, 0, sizeof(buffer)); //clear the buffer
read(new_socket, buffer, 1024);
std::string msg = std::string(buffer, strlen(buffer));
if (msg.size()==0) {
std::cout<<"I can't believe it"<<std::endl;
}
std::cout<<"received msg from the client:"<<msg<<",msg size:"<<msg.size()<<std::endl;
std::string results="response from the server:["+msg+"]";
send(new_socket, results.c_str(), results.length(), 0);
//usleep(10*1000);
}
if (close(new_socket)<0){
std::cout <<"close error"<<std::endl;
}
shutdown(server_fd, SHUT_RDWR);
}
} ;
}
void operation(int client_id) {
auto obj = new mySocket::Memcached();
for (int i=0; i<10;i++){
int id=client_id*100+i;
std::cout<<obj->sendMessage(std::to_string(id), "127.0.0.1", 7111)<<std::endl<<std::endl;
}
}
int main(int argc, char const* argv[]) {
// start a socket server
mySocket::Memcached::controller();
// start 3 socket clients
std::thread t1(operation, 1);
std::thread t2(operation, 2);
std::thread t3(operation, 3);
t1.join();
t2.join();
t3.join();
}
In the code above, the client always sends a message with a length of 3. However, the server can receive messages with a length of 0 which causes further errors.
I'm struggling with this for several days and can't figure out why it happens. I noticed
if I add a short sleep inside the server while loop, the problem is solved. (uncomment usleep(10*1000); in the code).
or if I only use one client, the problem is also solved.
Any thought helps.

Answer 1

You are using TCP sockets. You may want to use some application-level protocol like HTTP, websockets instead, that will be much easier, because you will not need to worry about how message is sent/received and in which sequence. If you have to stick with TCP sockets, you firstly have to understand few things:
There's two types of TCP sockets you can use: non-blocking and blocking IO (input/output). You are currently using blocking IO. That IO will be sometimes blocked and you won't be able to do anything with sockets. In blocking IO, it can be work arounded by using one socket per thread on server-side. It's not efficient, but it's relatively easy comparing to Non-blocking IO. Non-blocking IO doesn't wait for anything. While in blocking IO you wait for data, in non-blocking IO you create something like events, callbacks, that are used when there's some data. You probably have to read about these types of IO.
In your server function, would be better, if you listen for incoming connections in one thread, and when there's incoming connection, move this connection into another thread and function, that will handle other things. This may solve your problem related to multiple clients at the same time.
In function operation, instead of allocating memory using raw pointer, use static allocation or smart pointers to avoid memory leaks. If you don't want to, then at least, do delete obj; in the end of function.
And the last one thing. You can use some TCP socket wrapper like sockpp to make things a lot easier. You will have anything TCP sockets have, but in C++ style and a little bit easier to understand and maintain. If you can't use application-level protocol, I strongly suggest you to use some wrapper at least.
Update
As was stated by commenters, there are more things you need to know:
TCP sockets are streams. This means that if you send your message with length of 1024 bytes, it can be divided into several TCP data packets and you can't know if it will be divided or not, how much packets other side will receive etc. You have to read in a while loop using recv() and wait for data. There's some tricks which can help you to properly receive data:
You can send length of your message first, so other side will know how much bytes it needs to receive.
You can place some terminating symbol or sequence of terminating symbols in the end of your message and read until these will be received. This can be a little risky, because there's chance that you would not receive these symbols at all and will be reading next.
You have to join client threads only when you know, that server is already started and listening for incoming connections. You can use some variable as a flag for these purposes, but make note, that you have to pay a lot of attention, when reading/writing variable from two or more different threads. For these purposes, you can use mutexes, which are some mechanism that will allow you safely access one variable from several threads.