IPC First send() and recv() calls in C++ TCP takes longer - c++

Question

I am using IPC (Inter-process communication) to send data between two C++ programs. For the IPC I use the basic TCP/IP Winsock2 code.
Basically the idea is that I send 3D data frames between the two applications, one processes the data using GPU/CUDA and the other displays it using a 3de party library (PCL). My applications are very rich in data (ie 30fps, 4Mb per frame), but that should not be a problem for IPC as far as I know.
So as each frame is finished processing on the GPU, I convert the frames data (X,Y,Z coords of type float; R,G,B of type uint8_t each) to bytes and send them one after the other.
Doing this I have noticed something odd. In my code I have 9 send() commands one after the other.
1ste : Sending a single char that acts as the name of the frame.
2de : A single int with the amount of 3D points incoming.
3-5th : RGB values
6-8th : XYZ values
9th : End of comms check to close the viewer application.
This whole process takes ~30ms to complete. What I found odd was where most of the time went. After timing each event I got the following:
1ste : 20ms (1 byte data)
2de : <1ms (4 bytes data)
3-5 : 2ms (921600 bytes of data)
6-8 : 3ms (3686400 bytes of data)
9th : 1ms (1 byte data)
Why is it that for the first send command it takes such a long time to complete, even when its only 1 byte of data, and then the rest of the data completes in record time. In between each time this loop runs, there is about a 20ms time delay waiting for the GPU code to finish. Does the TCP connection go into a sleep state, and if so can I disable it somehow.
TCP Socket code:
SOCKET Create_Server_Socket(PCSTR IP, PCSTR port)
{
struct addrinfo *result = NULL, *ptr = NULL, hints;
int iResult;
ZeroMemory(&hints, sizeof(hints));
hints.ai_family = AF_INET;
hints.ai_socktype = SOCK_STREAM;
hints.ai_protocol = IPPROTO_TCP;
hints.ai_flags = AI_PASSIVE;
// Resolve the local address and port to be used by the server
iResult = getaddrinfo(NULL, port, &hints, &result);
if (iResult != 0) {
printf("getaddrinfo failed: %d\n", iResult);
WSACleanup();
return 1;
}
SOCKET ListenSocket = INVALID_SOCKET;
ListenSocket = socket(result->ai_family, result->ai_socktype, result->ai_protocol);
if (ListenSocket == INVALID_SOCKET) {
printf("Error at socket(): %ld\n", WSAGetLastError());
freeaddrinfo(result);
WSACleanup();
return 1;
}
iResult = bind(ListenSocket, result->ai_addr, (int)result->ai_addrlen);
if (iResult == SOCKET_ERROR) {
printf("bind failed with error: %d\n", WSAGetLastError());
freeaddrinfo(result);
closesocket(ListenSocket);
WSACleanup();
return 1;
}
freeaddrinfo(result);
if (listen(ListenSocket, SOMAXCONN) == SOCKET_ERROR) {
printf("Listen failed with error: %ld\n", WSAGetLastError());
closesocket(ListenSocket);
WSACleanup();
return 1;
}
SOCKET ClientSocket;
ClientSocket = INVALID_SOCKET;
// Accept a client socket
ClientSocket = accept(ListenSocket, NULL, NULL);
if (ClientSocket == INVALID_SOCKET) {
printf("accept failed: %d\n", WSAGetLastError());
closesocket(ListenSocket);
WSACleanup();
return 1;
}
return (ClientSocket);
}
Code in question:
iResult = send(ConnectSocket, (char*)&name, sizeof(char), 0); //Takes >20ms to complete
iResult = send(ConnectSocket, (char*)&points, 4, 0);
iResult = send(ConnectSocket, (char*)Red_t, sizeof(uint8_t) * depth_width *depth_height, 0);
iResult = send(ConnectSocket, (char*)Green_t, sizeof(uint8_t) * depth_width *depth_height, 0);
iResult = send(ConnectSocket, (char*)Blue_t, sizeof(uint8_t) * depth_width *depth_height, 0);
iResult = send(ConnectSocket, (char*)z_t, sizeof(uint16_t) * depth_width *depth_height, 0);
iResult = send(ConnectSocket, (char*)x_t, sizeof(float) * depth_width *depth_height, 0)
iResult = send(ConnectSocket, (char*)y_t, sizeof(float) * depth_width *depth_height, 0);
iResult = send(ConnectSocket, "a", sizeof(char), 0);
I am working on Windows 10 (ie the Winsock2 lib).
Thank you in advance!

Answer 1

You are probably suffering the penalty of "Nagle's Algorithm"
See https://en.wikipedia.org/wiki/Nagle%27s_algorithm
Long story short, there is a delay built-in to TCP/IP with the purpose of collecting enough data to be worth sending a packet before the first packet is sent. There is a TCP_NODELAY option that you can use when opening the socket to disable this, if it is a problem for you.
That having been said, if performance is of crucial importance, you might be better off using shared memory for inter-process communication rather than sockets.