Two programs blocking each other while communicating in local loop - c++

I've programmed a simple c++ client which should communicate with another program using a listener thread (a downloaded EchoServer in Java) over the local loop on Ubuntu 14.04. However, there's a problem in the following situation:
Client connects to server
Server sends greeting message
Client receives message and sends a new message to server
Client waits for an answer; the main thread sleeps and the listener thread waits for an answer using recv()
In the next step, the server should receive the message sent by the server, but it doesn't. Instead it first receives the message once the client terminates.
I think that the problem is that the client blocks resources and thus not allowing the server to receive the message, but I'm not sure. Unfortunately I don't have the option to test this on two separate machines.
Code snippets:
// main method
int main(void) {
Client client("127.0.0.1", 13050);
std::cout << client.open() << std::endl;
client.attachListener(foo);
usleep(1000 * 1000 * 2);
std::cout << client.send("hello") << std::endl;
usleep(1000 * 1000 * 5);
}
// send method
int Client::send(const char* msg) {
return write(sockfd, msg, strlen(msg));
}
// listener function
void* Client::listen() {
char buffer[256];
unsigned int receive_size = 0;
while(true) {
receive_size = 0;
while((receive_size = recv(sockfd, &buffer, 256, 0)) > 0) {
buffer[receive_size] = '\0';
msgHandler(buffer);
bzero(&buffer, 256);
}
if(receive_size == 0) {
msgHandler("Server disconnected");
} else if(receive_size == 1) {
msgHandler("Connection failure!");
}
}
return NULL;
}
Output:
1
Welcome to the Java EchoServer. Type 'bye' to close.
6

The EchoServer implementations typically want to see a newline on the message you send before they'll echo it back. Instead of client.send("hello") try client.send("hello\n").
Also, though this isn't really necessary for an application you're just experimenting with, you might want to turn off the Nagle algorithm on your client socket so that small messages get sent immediately. Add code like this just after the point where you call connect with client socket:
int flag = 1;
int res = setsockopt(sockfd, IPPROTO_TCP, TCP_NODELAY, &flag, sizeof flag);
if (res < 0) // handle setsockopt failure here...
This code requires these header files:
#include <sys/socket.h>
#include <netinet/in.h>
#include <netinet/tcp.h>

Related

Issue with sending packet via ZMQ

I have the following C++ code that sends a zmq packet of 10 bytes to a ZMQ Pull Source in GNU Radio Companion.
#include <iostream>
#include <zmq.h>
using namespace std;
int main()
{
void *send_context, *responder;
unsigned short gnu_trigger[10];
string trig_addr = "tcp://127.0.0.1:4500";
int rc;
/* Bind ZMQ address for transmitting data */
send_context = zmq_ctx_new();
responder = zmq_socket(send_context, ZMQ_PUSH);
rc = zmq_bind(responder, trig_addr.c_str());
while(1)
{
cout << "Sending trigger to gnuradio\n";
memset(gnu_trigger, 0x0, sizeof(unsigned short) * 10);
gnu_trigger[0] = 1;
rc = zmq_send(responder, &gnu_trigger, sizeof(gnu_trigger), 0);
cout << "Sent trigger to gnuradio\n";
sleep(1);
}
zmq_close(responder);
zmq_ctx_destroy(send_context);
return 0;
}
If I remove/comment the cout statements before and after sending the ZMQ packets, the packet is not transmitted. However, if the cout statements are enabled, the packet is sent successfully.
Why does this happen?
Zmq is asynchronous so messages aren't sent when you call zmq_send and are sent at some point in the future. As your program ends immediately after calling zmq_send zmq doesn't have time to send the messages.
Adding the print statements slows your program down enough to allow zmq time to send the messages.
You should call zmq_close and zmq_term before exiting your program.

how to deal with multiple clients in c++ socket problem?

I need some help with a socket program with multiple clients and one server. To simplify, I create
3 socket clients
1 socket server
For each client, it opens a new connection for sending a new message and closes the connection after a response is received.
For the server, it does not need to deal with connections concurrently, it can deal with the message one by one
here is my code (runnable), compile it with /usr/bin/g++ mycode.cpp -g -lpthread -lrt -Wall -o mycode
#include <iostream>
#include <arpa/inet.h>
#include <string.h>
#include <sys/socket.h>
#include <unistd.h>
#include <unordered_map>
#include <thread>
using namespace std;
void Warning(string msg) { std::cout<< msg << std::endl; }
namespace mySocket {
class Memcached {
public:
// start a server
static void controller(int port=7111) { std::thread (server, port).detach(); }
// open a new connection to send a message:
// 1. open a connection
// 2. send the message
// 3. read the message
// 4. close the connection
std::string sendMessage(string msg, string host, int port=7111) {
int sock = 0, client_fd;
struct sockaddr_in serv_addr;
char buffer[1024] = { 0 };
if ((sock = socket(AF_INET, SOCK_STREAM, 0)) < 0) {
std::cout << "Socket creation error, msg: " << msg << ", host: " << host << ", port: " << port << std::endl;
exit(1);
}
serv_addr.sin_family = AF_INET;
serv_addr.sin_port = htons(port);
if (inet_pton(AF_INET, host.c_str(), &serv_addr.sin_addr) <= 0) {
std::cout << "\nInvalid address/ Address not supported, kmsgey: " << msg << ", host: " << host << ", port: " << port << std::endl;
exit(1);
}
while ((client_fd = connect(sock, (struct sockaddr*)&serv_addr, sizeof(serv_addr))) < 0) { sleep(10*1000); }
std::cout << "client sends a message:"<<msg<<", msg size:"<<msg.size()<<std::endl;
send(sock, msg.c_str(), msg.size(), 0);
read(sock, buffer, 1024);
close(client_fd);
return std::string(buffer, strlen(buffer));
}
private:
// start a server
// 1. open a file descriptor
// 2. listen the fd with queue size 10
// 3. accept one connection at a time
// 4. deal with message in the connection
// 5. accept the next connection
// 6. repeat step 3
static void server(int port) {
int server_fd, new_socket;
struct sockaddr_in address;
int opt = 1;
int addrlen = sizeof(address);
char buffer[1024] = { 0 };
unordered_map<string,string> data;
if ((server_fd = socket(AF_INET, SOCK_STREAM, 0)) == 0) {
Warning("socket failed"); exit(1);
}
if (setsockopt(server_fd, SOL_SOCKET, SO_REUSEADDR | SO_REUSEPORT, &opt, sizeof(opt))) {
Warning("setsockopt failed"); exit(1);
}
address.sin_family = AF_INET;
address.sin_addr.s_addr = INADDR_ANY;
address.sin_port = htons(port);
if (bind(server_fd, (struct sockaddr*)&address, sizeof(address)) < 0) {
Warning("bind failed"); exit(1);
}
// the queue size is 10 > 3
if (listen(server_fd, 10) < 0) {
Warning("listen failed"); exit(1);
}
while(1)
{
if ((new_socket = accept(server_fd, (struct sockaddr*)&address, (socklen_t*)&addrlen)) < 0) {
std::cout << "accept failed"; exit(1);
}
memset(&buffer, 0, sizeof(buffer)); //clear the buffer
read(new_socket, buffer, 1024);
std::string msg = std::string(buffer, strlen(buffer));
if (msg.size()==0) {
std::cout<<"I can't believe it"<<std::endl;
}
std::cout<<"received msg from the client:"<<msg<<",msg size:"<<msg.size()<<std::endl;
std::string results="response from the server:["+msg+"]";
send(new_socket, results.c_str(), results.length(), 0);
//usleep(10*1000);
}
if (close(new_socket)<0){
std::cout <<"close error"<<std::endl;
}
shutdown(server_fd, SHUT_RDWR);
}
} ;
}
void operation(int client_id) {
auto obj = new mySocket::Memcached();
for (int i=0; i<10;i++){
int id=client_id*100+i;
std::cout<<obj->sendMessage(std::to_string(id), "127.0.0.1", 7111)<<std::endl<<std::endl;
}
}
int main(int argc, char const* argv[]) {
// start a socket server
mySocket::Memcached::controller();
// start 3 socket clients
std::thread t1(operation, 1);
std::thread t2(operation, 2);
std::thread t3(operation, 3);
t1.join();
t2.join();
t3.join();
}
In the code above, the client always sends a message with a length of 3. However, the server can receive messages with a length of 0 which causes further errors.
I'm struggling with this for several days and can't figure out why it happens. I noticed
if I add a short sleep inside the server while loop, the problem is solved. (uncomment usleep(10*1000); in the code).
or if I only use one client, the problem is also solved.
Any thought helps.
You are using TCP sockets. You may want to use some application-level protocol like HTTP, websockets instead, that will be much easier, because you will not need to worry about how message is sent/received and in which sequence. If you have to stick with TCP sockets, you firstly have to understand few things:
There's two types of TCP sockets you can use: non-blocking and blocking IO (input/output). You are currently using blocking IO. That IO will be sometimes blocked and you won't be able to do anything with sockets. In blocking IO, it can be work arounded by using one socket per thread on server-side. It's not efficient, but it's relatively easy comparing to Non-blocking IO. Non-blocking IO doesn't wait for anything. While in blocking IO you wait for data, in non-blocking IO you create something like events, callbacks, that are used when there's some data. You probably have to read about these types of IO.
In your server function, would be better, if you listen for incoming connections in one thread, and when there's incoming connection, move this connection into another thread and function, that will handle other things. This may solve your problem related to multiple clients at the same time.
In function operation, instead of allocating memory using raw pointer, use static allocation or smart pointers to avoid memory leaks. If you don't want to, then at least, do delete obj; in the end of function.
And the last one thing. You can use some TCP socket wrapper like sockpp to make things a lot easier. You will have anything TCP sockets have, but in C++ style and a little bit easier to understand and maintain. If you can't use application-level protocol, I strongly suggest you to use some wrapper at least.
Update
As was stated by commenters, there are more things you need to know:
TCP sockets are streams. This means that if you send your message with length of 1024 bytes, it can be divided into several TCP data packets and you can't know if it will be divided or not, how much packets other side will receive etc. You have to read in a while loop using recv() and wait for data. There's some tricks which can help you to properly receive data:
You can send length of your message first, so other side will know how much bytes it needs to receive.
You can place some terminating symbol or sequence of terminating symbols in the end of your message and read until these will be received. This can be a little risky, because there's chance that you would not receive these symbols at all and will be reading next.
You have to join client threads only when you know, that server is already started and listening for incoming connections. You can use some variable as a flag for these purposes, but make note, that you have to pay a lot of attention, when reading/writing variable from two or more different threads. For these purposes, you can use mutexes, which are some mechanism that will allow you safely access one variable from several threads.

How do I handle socket requests in separate threads, thread seems to be blocking main process

I am trying to create a simple HTTP server in C++. I want each request to be handled simultaneously in separate threads, but when I create a thread and put a simple sleep(10) at the start to cause some delay, another request to the server cannot be made until the first thread finishes. What am I doing wrong?
Here is my code so far:
#include <unistd.h>
#include <stdio.h>
#include <sys/socket.h>
#include <stdlib.h>
#include <netinet/in.h>
#include <string>
#include <ostream>
#include <iostream>
#include <thread>
using namespace std;
void send_response (int socket)
{
sleep (10);
char buffer[2048] = {0};
int request = recv (socket, buffer, 2048, 0);
if (request == -1) {
perror ("error");
}
string message = "Hello from server";
string length = to_string (message.length ());
string hello = "HTTP/1.1 200 OK\nContent-Type: text/plain\nContent-Length: " + length + "\n\n" + message;
send (socket, hello.c_str (), hello.length (), 0);
close (socket);
}
int main(int argc, char const *argv[])
{
struct sockaddr_in address;
int opt = 1;
socklen_t addrlen = sizeof (address);
int server_fd = socket (AF_INET, SOCK_STREAM, 0);
if (server_fd == 0) {
perror ("socket failed");
exit (EXIT_FAILURE);
}
if (setsockopt (server_fd, SOL_SOCKET, SO_REUSEADDR | SO_REUSEPORT, &opt, sizeof (opt))) {
perror ("setsockopt");
exit (EXIT_FAILURE);
}
address.sin_family = AF_INET;
address.sin_addr.s_addr = INADDR_ANY;
address.sin_port = htons (8080);
if (bind (server_fd, (struct sockaddr *)(&address), sizeof (address)) < 0) {
perror ("bind failed");
exit (EXIT_FAILURE);
}
if (listen (server_fd, 3) < 0) {
perror ("listen");
exit (EXIT_FAILURE);
}
while (true) {
int new_socket = accept (server_fd, (struct sockaddr *)(&address), &addrlen);
if (new_socket < 0) {
perror ("accept");
exit (EXIT_FAILURE);
}
thread response (&send_response, new_socket);
response.detach ();
cout << "request handled" << endl;
}
return 0;
}
I don't need to wait for the thread to finish or get any data back from the thread, I just want it to start, do some work, and send something back to the client. I expect each thread will be doing a decent amount of work, that's why I used sleep.
Edit: It seems the loop is ready and waiting for a second connection and will successfully accept a new connection and create a new thread to handle it, but only if the second connection is from a different IP address. So if I visit the server at 127.0.0.1:8080 and from a different local IP address like 192.168.1.91, both connections will go through with two threads running simultaneously. However, if I simply open two tabs, both at 127.0.0.1:8080, only one connection will be accepted at a time, the other will stall. It seems I need the code to handle multiple connections from the same IP address, but a search for that kind of problem on Google hasn't given me any good results.
Ok, my bad, everything is working as expected. Using wget to send multiple requests from the same IP causes multiple threads to be created and for the requests to be processed simultaneously. In Chrome, for some reason opening multiple tabs from the same IP causes them to be sent in serial, but in Firefox they are sent in parallel and arrive and are processed at the same time. That is weird, but I'll chock it up to an oddity of Chrome.

ZMQ_HEARTBEAT_TTL does not discard outgoing queue even if ZMQ_LINGER is set

I have a server which uses a ZMQ_ROUTER to communicate with ZMQ_DEALER clients. I set the ZMQ_HEARTBEAT_IVL and ZMQ_HEARTBEAT_TTL options on the client socket to make the client and server ping pong each other. Beside, because of the ZMQ_HEARTBEAT_TTL option, the server will timeout the connection if it does not receive any pings from the client in a time period, according to zmq man page:
The ZMQ_HEARTBEAT_TTL option shall set the timeout on the remote peer for ZMTP heartbeats. If
this option is greater than 0, the remote side shall time out the connection if it does not
receive any more traffic within the TTL period. This option does not have any effect if
ZMQ_HEARTBEAT_IVL is not set or is 0. Internally, this value is rounded down to the nearest
decisecond, any value less than 100 will have no effect.
Therefore, what I expect the server to behave is that, when it does not receive any traffic from a client in a time period, it will close the connection to that client and discard all the messages in the outgoing queue after the linger time expires. I create a toy example to check if my hypothesis is correct and it turns out that it is not. The chain of events is as followed:
The server sends a bunch of data to the client.
The client receives and processes the data, which is slow.
All send commands return successfully.
While the client is still receiving the data, I unplug the internet cable.
After a few seconds (set by the ZMQ_HEARTBEAT_TTL option), the server starts sending FIN signals to the client, which are not being ACKed back.
The outgoing messages are not discarded (I check the memory consumption) even after a while. They are discarded only if I call zmq_close on the router socket.
So my question is, is this suppose to be how one should use the ZMQ heartbeat mechanism? If it is not then is there any solution for what I want to achieve? I figure that I can do heartbeat myself instead of using ZMQ's built in. However, even if I do, it seems that ZMQ does not provide a way to close a connection between a ZMQ_ROUTER and a ZMQ_DEALER, although that another version of ZMQ_ROUTER - ZMQ_STREAM provides a way to do this by sending an identity frame followed by an empty frame.
The toy example is below, any help would be thankful.
Server's side:
#include <zmq.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
int main(int argc, char **argv)
{
void *context = zmq_ctx_new();
void *router = zmq_socket(context, ZMQ_ROUTER);
int router_mandatory = 1;
zmq_setsockopt(router, ZMQ_ROUTER_MANDATORY, &router_mandatory, sizeof(router_mandatory));
int hwm = 0;
zmq_setsockopt(router, ZMQ_SNDHWM, &hwm, sizeof(hwm));
int linger = 3000;
zmq_setsockopt(router, ZMQ_LINGER, &linger, sizeof(linger));
char bind_addr[1024];
sprintf(bind_addr, "tcp://%s:%s", argv[1], argv[2]);
if (zmq_bind(router, bind_addr) == -1) {
perror("ERROR");
exit(1);
}
// Receive client identity (only 1)
zmq_msg_t identity;
zmq_msg_init(&identity);
zmq_msg_recv(&identity, router, 0);
zmq_msg_t dump;
zmq_msg_init(&dump);
zmq_msg_recv(&dump, router, 0);
printf("%s\n", (char *) zmq_msg_data(&dump)); // hello
zmq_msg_close(&dump);
char buff[1 << 16];
for (int i = 0; i < 50000; ++i) {
if (zmq_send(router, zmq_msg_data(&identity),
zmq_msg_size(&identity),
ZMQ_SNDMORE) == -1) {
perror("ERROR");
exit(1);
}
if (zmq_send(router, buff, 1 << 16, 0) == -1) {
perror("ERROR");
exit(1);
}
}
printf("OK IM DONE SENDING\n");
// All send commands have returned successfully
// While the client is still receiving data, I unplug the intenet cable on the client machine
// After a while, the server starts sending FIN signals
printf("SLEEP before closing\n"); // At this point, the messages are not discarded (memory usage is high).
getchar();
zmq_close(router);
zmq_ctx_destroy(context);
}
Client's side:
#include <zmq.h>
#include <stdlib.h>
#include <string.h>
int main(int argc, char **argv)
{
void *context = zmq_ctx_new();
void *dealer = zmq_socket(context, ZMQ_DEALER);
int heartbeat_ivl = 3000;
int heartbeat_timeout = 6000;
zmq_setsockopt(dealer, ZMQ_HEARTBEAT_IVL, &heartbeat_ivl, sizeof(heartbeat_ivl));
zmq_setsockopt(dealer, ZMQ_HEARTBEAT_TIMEOUT, &heartbeat_timeout, sizeof(heartbeat_timeout));
zmq_setsockopt(dealer, ZMQ_HEARTBEAT_TTL, &heartbeat_timeout, sizeof(heartbeat_timeout));
int hwm = 0;
zmq_setsockopt(dealer, ZMQ_RCVHWM, &hwm, sizeof(hwm));
char connect_addr[1024];
sprintf(connect_addr, "tcp://%s:%s", argv[1], argv[2]);
zmq_connect(dealer, connect_addr);
zmq_send(dealer, "hello", 6, 0);
size_t size = 0;
int i = 0;
while (size < (1ll << 16) * 50000) {
zmq_msg_t msg;
zmq_msg_init(&msg);
if (zmq_msg_recv(&msg, dealer, 0) == -1) {
perror("ERROR");
exit(1);
}
size += zmq_msg_size(&msg);
printf("i = %d, size = %ld, total = %ld\n", i, zmq_msg_size(&msg), size); // This causes the cliet to be slow
// Somewhere in this loop I unplug the internet cable.
// The client starts sending FIN signals as well as trying to reconnect. The recv command hangs forever.
zmq_msg_close(&msg);
++i;
}
zmq_close(dealer);
zmq_ctx_destroy(context);
}
PS: I know that setting the highwater mark to unlimited is bad practice, however I figure that the problem will be the same even if the highwater mark is low so let's ignore it for now.

C++ Creating multiple socket clients

I'm trying to build a simulation for multiple socket clients.
My server has the following code to listen to multiple clients
My socket are from a very simple class drive from CAsyncSocket and my environment is windows MFC.
m_server.Create(....); // with the correct values
if (m_server.Listen()==FALSE)
and later on the OnSocketAccept() function
if (m_server.Accept(tempSock))
{
CSocketThread* pThread = (CSocketThread*)AfxBeginThread(RUNTIME_CLASS(CSocketThread), THREAD_PRIORITY_NORMAL, 0, CREATE_SUSPENDED);
...
My simulation apps has the following code:
for (int i = 0; i < numOfClients; i++)
{
m_sConnected[i].Create();
int rVal = m_sConnected[i].Connect(csIPAddress.GetString(), m_port);
That doesn't work.
In WireShark I can see that my (numOfClients = 10 for example) 10 clients are connected with different client source port.
But each new socket of m_sConnected[i] is becoming NULL after the second connection to all sockets including m_sConnected[0].
Closing the sockets or destroy the simulation app create socket close at the server side for all open threads for the listen sockets.
What is the problem?
Can I use the same process/thread for all my socket clients?
10x
UrAv.
your problem is that you are not using the CSocketThread object the right way.
as mentiend in microsoft documention
after the accept function you need to do the following :
CSockThread* pSockThread = (CSockThread*)AfxBeginThread( RUNTIME_CLASS(CSockThread), THREAD_PRIORITY_NORMAL, 0, CREATE_SUSPENDED);
if (NULL != pSockThread) {
// Detach the newly accepted socket and save
//the SOCKET handle in our new thread object.
//After detaching it, it should no longer be
//used in the context of this thread.
pSockThread->m_hConnected = sConnected.Detach();
pSockThread->ResumeThread();
} }
when you attach your socket to the thread then it will run.
link to microsoft doc:
https://msdn.microsoft.com/en-us/library/wxzt95kb.aspx
your solution has worked for me. I have used multiple threads to stress test the server in c++ under linux. Pasting my code, it will be helpful to somebody...Experts can improve my code, if they find any flaws in my handling of code. I know, I am doing something wrong but no other go to test the server as no one has provided the solution for this till now. I am able to test the server for 100000 clients using this code. - Kranti.
include //for threading , link with lpthread
void *connection_handler(void *);
#define PORT 9998
#define SERVER_IP "127.0.0.1"
#define MAXSZ 100
#define MAXSOCK 70000
int main()
{
int sockfd[MAXSOCK];//to create socket
int socket_desc , new_socket[MAXSOCK], *new_sock;
struct sockaddr_in serverAddress;//client will connect on this
int n;
char msg1[MAXSZ];
char msg2[MAXSZ];
int NoOfClients = MAXSOCK;
memset(msg2,0,100);
void *ret;
for(int i=0;i<NoOfClients;i++){
//create socket
sockfd[i]=socket(AF_INET,SOCK_STREAM,0);
//initialize the socket addresses
memset(&serverAddress,0,sizeof(serverAddress));
serverAddress.sin_family=AF_INET;
serverAddress.sin_addr.s_addr=inet_addr(SERVER_IP);
serverAddress.sin_port=htons(PORT);
//client connect to server on port
new_socket[i] = connect(sockfd[i],(struct sockaddr *)&serverAddress,sizeof(serverAddress));
printf("new socket connected= %d",new_socket[i]);
pthread_t sniffer_thread[MAXSOCK];
new_sock = malloc(sizeof(int));
*new_sock = new_socket[i];
int p=-1;
if( p = pthread_create( &sniffer_thread[i] , NULL , connection_handler , (void*) new_sock) < 0)
{
perror("could not create thread");
return 1;
}
}
return 0;
}
/*
* This will handle connection for each client
* */
void *connection_handler(void *socket_desc)
{
//Get the socket descriptor
int sock = *(int*)socket_desc;
int read_size;
char *message , client_message[50];
printf("we are in connection handler");
//Send some messages to the server
message = "Greetings! I am your connection handler\n";
int wlen = write(sock , message , strlen(message));
printf("write length is %d", wlen);
//Free the socket pointer
//close(sock);
free(sock);
return 0;
}