I have sender and receiver. When I try in same computer, receiver receives image correctly but when execute sender in another computer, receiver receives distorted images. Others are correctly received (integer, char array ...)
receiver:
char sockData[record.imageSize];
bytes_received = recv(new_sd, sockData, record.imageSize, 0);
cv::Mat img = cv::Mat::zeros(height,width,CV_8UC1);
memcpy(img.data, sockData, record.imageSize);
std::vector<int> compression_params;
compression_params.push_back(CV_IMWRITE_PNG_COMPRESSION);
compression_params.push_back(3);
customerFacePath << id << ".png";
cv::imwrite(customerFacePath.str().c_str(),img,compression_params);
sender:
rec.imageSize = av.mImg.total()*av.mImg.elemSize();
bytes_sent = send(socketfd, av.mImg.data, rec.imageSize, 0);
Here the distorted image,
Only a small part is correct at top. How can i fix this?
In TCP, data is not required to be sent and received in the same structure (as a block). A data sent using TCP can be split to or combined from multiple packets on receive. This modification can happen anywhere between your computer and destination computer (i.e. in router).
It is also worth noting that recv does not wait until all the buffer is filled. It fetches as much as it can and it returns the size that it received. As a result the following execution may happen:
On the sending part:
int sent = send(socket, bufPtr, 100, 0);
std::cout << "Bytes sent: " << sent << std::endl;
Outputs:
Bytes sent: 15
And on the receiving part:
int received = recv(socket, bufPtr, 100, 0);
std::cout << "Bytes received: " << received << std::endl;
Outputs:
Bytes received: 5
TCP on Wikipedia
You have to make sure you are receiving all the data, using some kind of packet structure. i.e.
2-byte packetId | 4-byte packetLength | variable-byte packetData
Usually sending/receiving on the same machine does not modify the packets so you code works. On the other hand, sending to a remote endpoint, protocol features like Nagle's algortihm come to play to modify your data.
Do not disable nagle to solve the issue as it will still not be guaranteed that packets will not be modified. That is how TCP works. It is a streaming protocol.
You can use UDP, packets are sent and received as blocks without modification in that protocol. But UDP is missing the verification that the data is delivered correctly let alone checking if it is delivered at all.
UDP on Wikipedia
Related
Good afternoon all,
I have been making a UDP server for gathering metrics on my Windows server (SNMP isn't accurate on Windows as it doesn't have 64bit counters). The server runs on the Windows server and the client is running on a Linux monitoring box.
I have set it up running as a service and it is running great except for, every once and a while, the UDP packet is not received from the Linux machine. I am using the following bit of code to receive UDP packets:
bytes_received = recvfrom(serverSocket, serverBuf, serverBufLen, 0, (SOCKADDR*)&SenderAddr, &SenderAddrSize);
The socket is set to timeout every 15 seconds (So any service control requests like stop can be executed). What I am thinking is happening is either:
The UDP packet is arriving in between the 15 second timeout and when it starts listening again.
The packet is arriving a fraction of a second after another UDP packet has arrived (for a different metric) and it has gone onto starting up a process to send a packet back, and thus it isn't at the recvfrom yet.
(I am basing both of those off my assumption that it is only waiting for a packet when it is at recvfrom).
I could possibly move over to TCP to solve this issue, but since the information is time sensitive, I would prefer to stay with UDP for it's speed.
Is there anyway to queue up incoming packets and have them be processed or would I be best to look at TCP instead?
I ended up coming up with the idea of transmitting the UDP packet if the first one doesn't get a response after 2 seconds. Works a treat so far.
Edit:
It is asking for code:
std::string returnMsg;
returnMsg = "CRITICAL - No packet recieved back.";
int i = 0;
while(returnMsg == "CRITICAL - No packet recieved back.") {
if(i == 5) {
std::cout << "CRITICAL - No packet recieved back." << "\n";
return 2;
}
//std::cout << "Try " << i << "\n";
// Now lets send the message
send_message(args[2],message.c_str());
// Now lets wait for response
returnMsg = recieve_message();
i++;
}
The recieve_message function returns "CRITICAL - No packet recieved back" when the timeout occurs.
I'm trying to write a UDP client App, which receives some control packets(length 52-104 bytes) from a server fragmented to datagrams of size 1-4 bytes each (Why this is not a big packet and is fragmented instead? That's a mystery to me...).
I created a thread, and in this thread I used a typical recvfrom example from MS. The received data from the small buffer I append to string to recreate the packet (If the packet is too big, the string would be cleared).
My problem is the latency:
The inbound packets are changed, but the data from the buffer and the string hasn't changed during the minute or more. I tried to use a circular buffer instead of a string, but it has no effect on the latency.
So, what am I doing wrong and how do I receive a fragmented UDP packet in a proper way?
I don't have the original sender code, so i'm attaching a part of my sender emulator. As you can see, the original data string (mSendString) is fragmented to some four-bytes packets and sent to the net. When the data string has changed on sender side, the data on receiver side hasn't changed in aceptable time, it changed a few minutes later.
UdpClient mSendClient = new UdpClient();
string mSendString = "head,data,data,data,data,data,data,data,chksumm\n";//Control string
public static void SendCallback(IAsyncResult ar)
{
UdpClient u = (UdpClient)ar.AsyncState;
mMsgSent = true;
}
public void Send()
{
while (!mThreadStop)
{
if (!mSendStop)
{
for (int i = 0; i < mSendString.Length; i+=4)
{
Byte[] sendBytes = new Byte[4];
Encoding.ASCII.GetBytes(mSendString,i,4,sendBytes,0);
mSendClient.BeginSend(sendBytes, 1, mEndPoint, new AsyncCallback(SendCallback), mSendClient);
}
}
Thread.Sleep(100);
}
}
I was wrong when I asked this question in some points:
First,the wrong terms - the string was chopped/sliced/divided into
four bytes packets, not fragmented.
Second, I was thought, that too
much small UDP packets are the cause of latency in my app, but when I
ran my UDP receive code separately from other app code, I found this
UDP receive code is working without latency.
Seems like there are threading problems, not UDP sockets.
I have been trying to create a program that sends multiple packets via sendto to different IP addresses, but after exactly 1238 callings to sendto I'm getting the error: "SendTo: Invalid argument" (printed by perror).
Edit: After an hour the number of callings to sendto is exactly 1231 and remains like that every run. After I added a code that prints something on the screen, it was back to 1238 callings every run until error, deleted that code, it became 1241 and about an hour later it's 1231.
If I take down the IP addresses (making the aliases offline), it sends those packets correctly without an error but it get stuck for a moment after about every 500 sendto callings,
This error only happens when those IP addresses are not in the same server, when they are in the same server (aliases) the sendto works correctly.
Also, the error doesn't appear when sending to the same IP multiple times instead of multiple times to different IP addresses.
I have tried different fixes that I found when searching in Google. I have tried playing with the configurations in sysctl.conf file, raised the send buffer, somaxconn, backlog, and other things.. When I raised the send buffer, I have also raised the buffer in the application itself.
Here is the sample code I have written:
http://pastebin.com/FCn0ALzn
And the code that gives the error:
for (size_t i = 0; i < ips.size(); i++)
{
cout << i << ") Sending message to: " << ips[i] << endl;
server.sin_addr.s_addr = inet_addr(ips[i].c_str());
n = sendto(sock, buffer, strlen(buffer), 0, (const struct sockaddr *)&server, length);
if (n < 0)
{
perror("Sendto");
return;
}
}
I have managed to fix this issue by clearing IP addresses from the ARP cache. Every 500 callings to sendto, the program sleeps for few milliseconds and then clears the IP addresses that were processed from the ARP cache using the shell command: arp -d [ip] like this:
// Clear ARP cache
void clearIpArp(char* ip)
{
char arp[100] = {0};
sprintf(arp, "arp -d %s", ip);
system(arp);
}
I am developing a client-server application (TCP) in Linux using C++. This application is in charge of testing the network performance.
The connection between client and server is established only once, and then data are transmitted/received using write()/read() with an own-defined protocol.
When data exceeds 40Kb I receive just a part of the data only once. (i.e. I receive about 48KB)
Please find down the relevant part of the code:
while (1) {
servMtx.lock();
...
serv_bytes = (byte *) malloc(size_bytes);
n = read(newsockfd, serv_bytes,size_bytes);
if (n != (int)size_bytes ) {
std::cerr << "No enough data available for msg. Received just: " << n << std::endl;
continue;
}
receivedBytes += n + size_header_bytes + sizeof(ssize_t);
....
}
I increased the kernel buffer size to become 1MB using:
int buffsize = 1024*1024;
setsockopt(newsockfd, SOL_SOCKET, SO_RCVBUF, &buffsize, sizeof(buffsize));
and modified sysctl variables too:
sysctl -w net.core.rmem_max=8388608;
sysctl -w net.core.wmem_max=8388608;
as mentioned on this How to recive more than 65000 bytes in C++ socket using recv() but nothing was changed. Also, I tried to change the package size to no avail.
You should read or recv in several chunks (in general; if you are unlucky, the "several" becomes "one"). So you need to manage your buffering and keep (and use) the count of received bytes.
So at some point, you'll code
int nbrecv = recv(s, buffer + off, bufsize, 0);
if (nbrec>0) { off += nbrecv; bufsize -= nbrecv; }
and you probably should do that in your event loop (often around poll(2)...). And it does happen that nbrec is a lot less than bufsize and you should be handling that common case.
TCP does not guarantee that you'll get all the bytes in the same recv! It could depend on external factors (routing, network hardware, ...); it is a stream-oriented protocol, not a message-packet one. If your application wants messages it should buffer the input and chunk that input into messages according to the content. Look at HTTP or SMTP: their message have a well defined boundary given by header information (Content-Length: in HTTP) or by ending convention (line with a single . in SMTP).
Please read carefully read(2), recv(2), socket(7), tcp(7), some sockets tutorial, Advanced Linux Programming.
I am trying to send 2 request one by one at same time. My code is following (this is example code):
QTcpSocket client;
...
client->write(block);
client->write(block);
Problem is following. Server receives only first request. There is no second request. I sniffed using wireshark and see that there is no second request in tcp packets.
What must i do to send many requests via QTcpSocket one by one?
UPD: I inserted qDebug() << this->bytesAvailable() << "bytes"; to server in readyRead() and qDebug() << this->bytesToWrite(); after each client->write(block); in client. Also, I added this to client:
connect(this, SIGNAL(bytesWritten(qint64)), this, SLOT(bytesWritten(qint64)));
void Connection::bytesWritten(qint64 count)
{
qDebug() << count << "bytes written";
}
I send ORDER_STATUS_GET_LIST constant in first request and ORDER_GET_LIST in second. I added data output in server. I received first command.
There is output listing:
Client:
Sending ORDER_STATUS_GET_LIST
11 bytes to write
Sending ORDER_GET_LIST
68 bytes to write
68 bytes written
Server:
68 bytes
ORDER_STATUS_GET_LIST received
According to the documentation, you need to flush() the socket IF you don't return to the event loop between multiple writes.
The proper solution would be to buffer your blocks into, e.g., a QByteArray and send the buffer at once.
QTcpSocket client;
QByteArray buffer;
...
buffer << block;
buffer << block;
client->write(buffer);
I found solution myself. I think that somebody needs this too.
Solution is simple:
QTcpSocket client;
...
client->write(block);
client->flush();
client->write(block);
client->flush();
We need to flush qt socket buffer to net before fill it with another data.