I have a function for receiving messages of variable length through TCP. The send-function creates a buffer, puts the length of message in first four bytes, fills the rest with the message, and sends by parts. But the receive-function was receiving 4 bytes less. And suddenly, when I put one printf, everything is working as it should.
bool TCP_Server::recvMsg(SOCKET client_sock, std::unique_ptr<char[]>& buf_ptr, int* buf_len)
{
int msg_len;
int rcvd = 0, tmp;////
/* get msg len */
if((tmp = recv(client_sock, (char*)&msg_len, sizeof(msg_len), 0)) == -1)
{
handle_error("recv");
return false;
}
*buf_len = msg_len;
printf("msg_len = %d\n", msg_len); //
printf("tmp getting msg_len = %d\n", tmp);//
rcvd += tmp;//
buf_ptr.reset((char*)malloc(msg_len));
if(buf_ptr.get() == nullptr) // not enough memory
{
handle_error("malloc");
return false;
}
/* get msg of specified len */
/* get by biggest available pieces */
int i = 1;
while(int(msg_len - 1440 * i) > 0)
{
char* cur_ptr = buf_ptr.get() + 1440 * (i - 1);
if((tmp=recv(client_sock, cur_ptr, 1440, 0)) == -1)
{
handle_error("recv");
return false;
}
printf("1440 = %d\n", tmp); // doesn't work if I comment this line
rcvd += tmp;
i++;
}
int rest = msg_len - 1440 * (i - 1);
/* get the rest */
if((tmp = recv(client_sock, buf_ptr.get() + msg_len - rest, rest, 0)) == -1)
{
handle_error("(recv)reading with msg_len");
return false;
}
rcvd += tmp;//
printf("rcvd = %d\n", rcvd);//
return true;
}
In sum, if I comment printf("1440 = %d\n", tmp);, the function is receiving 4 bytes less.
I'm compiling with x86 Debug.
Here's the dissimilar lines in asm(/FA flag): http://text-share.com/view/50743a5e
But I don't see anything suspicious
printf writes to the console, which is a fairly slow operation, relatively speaking. The extra delay it produces might easily change how much data has arrived in the buffer when you call recv.
As Tulon comments, reads from TCP streams can be any length. TCP doesn't preserve message boundaries, so they don't necessarily match the send sizes on the other end. And if less data has been sent across the network than you asked to read, you'll get what is available.
Solution: stop thinking of 1440 byte chunks. Get rid of i and simply compare rcvd to msg_len.
Related
I am new in Socket Programming. I am trying to create a client application. The server is a camera which communicates using TCP. The camera is sending continuous data. Using Wireshark, I can see that the camera is sending continuous packets of different sizes, but not more than 1514 bytes. But my recv function is always returning 2000 which is the size of my buffer.
unsigned char buf[2000];
int bytesIn = recv(sock, (char*)buf, sizeof(buf) , 0);
if (bytesIn > 0)
{
std::cout << bytesIn << std::endl;
}
The first packet I receive is of size 9 bytes, which recv returns correct, but after that it always returns 2000.
Can anyone please tell me the solution so that I can get the correct size of the actual data payload?
EDIT
int bytesIn = recv(sock, (char*)buf, sizeof(buf) , 0);
if (bytesIn > 0)
{
while (bytes != 1514)
{
if (count == 221184)
{
break;
}
buffer[count++] = buf[bytes++];
}
std::cout << count;
}
EDIT:
Here is my Wireshark capture:
My Code to handle packets
int bytesIn = recv(sock, (char*)&buf, sizeof(buf) , 0);
if (bytesIn > 0)
{
if (flag1 == true)
{
while ((bytes != 1460 && (buf[bytes] != 0)) && _fillFlag)
{
buffer[fill++] = buf[bytes++];
if (fill == 221184)
{
flag1 = false;
_fillFlag = false;
fill = 0;
queue.Enqueue(buffer, sizeof(buffer));
break;
}
}
}
if ((strncmp(buf, _string2, 10) == 0))
{
flag1 = true;
}
}
For each frame camera is sending 221184 bytes and after each frame it sends a packet of data 9 bytes which I used to compare this 9 bytes are constant.
This 221184 bytes send by camera doesn't have 0 so I use this condition in while loop. This code is working and showing the frame but after few frame it shows fully black frame. I think the mistake is in receiving the packet.
Size of per frame is : 221184 (fixed)
Size of per recv is : 0 ~ 1514
My implementation here :
DWORD MakeFrame(int socket)
{
INT nFrameSize = 221184;
INT nSizeToRecv = 221184;
INT nRecvSize = 2000;
INT nReceived = 0;
INT nTotalReceived = 0;
BYTE byCamera[2000] = { 0 }; // byCamera size = nRecvSize
BYTE byFrame[221184] = { 0 }; // byFrame size = nFrameSize
while(0 != nSizeToRecv)
{
nRecvSize = min(2000, nSizeToRecv);
nReceived = recv(socket, (char*)byCamera, nRecvSize, 0);
memcpy_s(byFrame + nTotalReceived, nFrameSize, byCamera, nReceived);
nSizeToRecv -= nReceived;
nTotalReceived += nReceived;
}
// byFrame is ready to use.
// ...
// ...
return WSAGetLastError();
}
The first packet I receive is of size 9 bytes which it print correct after that it always print 2000. So can anyone please tell me the solution that I only get the size of actual data payload.
TCP is no packet-oriented, but a stream-oriented transport protocol. There is no notion of packets in TCP (apart maybe from a MTU). If you want to work in packets, you have to either use UDP (which is in fact packet-oriented, but by default not reliable concerning order, discarding and alike) or you have to implement your packet logic in TCP, i.e. reading from a stream and partition the data into logical packets once received.
I am trying to read complete messages from my GPS via serial port.
The message I am looking for starts with:
0xB5 0x62 0x02 0x13
So I read from the serial port like so
while (running !=0)
{
int n = read (fd, input_buffer, sizeof input_buffer);
for (int i=0; i<BUFFER_SIZE; i++)
{
if (input_buffer[i]==0xB5 && input_buffer[i+1]== 0x62 && input_buffer[i+2]== 0x02 && input_buffer[i+3]== 0x13 && i<(BUFFER_SIZE-1) )
{
// process the message.
}
}
The problem I am having is that I need to get a complete message. Half of a message could be in the buffer one iteration. And the other half could come into the message the next iteration.
Somebody suggested that free the buffer up from the complete message. And then I move the rest of data in the buffer to the beginning of the buffer.
How do I do that or any other way that make sure I get every complete selected message that comes in?
edit//
I want a particular class and ID. But I can also read in the length
To minimize the overhead of making many read() syscalls of small byte counts, use an intermediate buffer in your code.
The read()s should be in blocking mode to avoid a return code of zero bytes.
#define BLEN 1024
unsigned char rbuf[BLEN];
unsigned char *rp = &rbuf[BLEN];
int bufcnt = 0;
static unsigned char getbyte(void)
{
if ((rp - rbuf) >= bufcnt) {
/* buffer needs refill */
bufcnt = read(fd, rbuf, BLEN);
if (bufcnt <= 0) {
/* report error, then abort */
}
rp = rbuf;
}
return *rp++;
}
For proper termios initialization code for the serial terminal, see this answer. You should increase the VMIN parameter to something closer to the BLEN value.
Now you can conveniently access the received data a byte at a time with minimal performance penalty.
#define MLEN 1024 /* choose appropriate value for message protocol */
unsigned char mesg[MLEN];
while (1) {
while (getbyte() != 0xB5)
/* hunt for 1st sync */ ;
retry_sync:
if ((sync = getbyte()) != 0x62) {
if (sync == 0xB5)
goto retry_sync;
else
continue; /* restart sync hunt */
}
class = getbyte();
id = getbyte();
length = getbyte();
length += getbyte() << 8;
if (length > MLEN) {
/* report error, then restart sync hunt */
continue;
}
for (i = 0; i < length; i++) {
mesg[i] = getbyte();
/* accumulate checksum */
}
chka = getbyte();
chkb = getbyte();
if ( /* valid checksum */ )
break; /* verified message */
/* report error, and restart sync hunt */
}
/* process the message */
switch (class) {
case 0x02:
if (id == 0x13) {
...
...
You can break the read into three parts. Find the start of a message. Then get the LENGTH. Then read the rest of the message.
// Should probably clear these in case data left over from a previous read
input_buffer[0] = input_buffer[1] = 0;
// First make sure first char is 0xB5
do {
n = read(fd, input_buffer, 1);
} while (0xB5 != input_buffer[0]);
// Check for 2nd sync char
n = read(fd, &input_buffer[1], 1);
if (input_buffer[1] != 0x62) {
// Error
return;
}
// Read up to LENGTH
n = read(fd, &input_buffer[2], 4);
// Parse length
//int length = *((int *)&input_buffer[4]);
// Since I don't know what size an int is on your system, this way is better
int length = input_buffer[4] | (input_buffer[5] << 8);
// Read rest of message
n = read(fd, &input_buffer[6], length);
// input_buffer should now have a complete message
You should add error checking...
I need to download a html page in chunks. I had build a GET reuest whick can download a certain range of data. But i am unsuccessful in doing this in a repetitive manner.
Basically I have to reciver first 0-99 bytes then 100-199 and so on...
Also I would be grateful to know how toh know the exact size of receiving file beforehand using c or c++ code.
Following is my code.
i have exempted connectig to sockets etc. as it have been done successfully.
int c=0,s=0;
while(1)
{
get = build_get_query(host, page,s);
c+=1;
fprintf(stderr, "Query is:\n<<START>>\n%s<<END>>\n", get);
//Send the query to the server
int sent = 0;
cout<<"sending "<<c<<endl;
while(sent < strlen(get))
{
tmpres = send(sock, get+sent, strlen(get)-sent, 0);
if(tmpres == -1)
{
perror("Can't send query");
exit(1);
}
sent += tmpres;
}
//now it is time to receive the page
memset(buf, 0, sizeof(buf));
int htmlstart = 0;
char * htmlcontent;
cout<< "reciving "<<c<<endl;
while((tmpres = recv(sock, buf, BUFSIZ, 0)) > 0)
{
if(htmlstart == 0)
{
/* Under certain conditions this will not work.
* If the \r\n\r\n part is splitted into two messages
* it will fail to detect the beginning of HTML content
*/
htmlcontent = strstr(buf, "\r\n\r\n");
if(htmlcontent != NULL)
{
htmlstart = 1;
htmlcontent += 4;
}
}
else
{
htmlcontent = buf;
}
if(htmlstart)
{
fprintf(stdout, htmlcontent);
}
memset(buf, 0, tmpres);
}
if(tmpres < 0)
{
perror("Error receiving data");
}
s+=100;
if(c==5)
break;
}
char *build_get_query(char *host, char *page,int i)
{
char *query;
char *getpage = page;
int j=i+99;
char tpl[100] = "GET /%s HTTP/1.1\r\nHost: %s\r\nRange: bytes=%d-%d\r\nUser- Agent: %s\r\n\r\n";
if(getpage[0] == '/')
{
getpage = getpage + 1;
fprintf(stderr,"Removing leading \"/\", converting %s to %s\n", page, getpage);
}
query = (char *)malloc(strlen(host)+strlen(getpage)+8+strlen(USERAGENT)+strlen(tpl)-5);
sprintf(query, tpl, getpage, host, i , j, USERAGENT);
return query;
}
Also I would be grateful to know how toh know the exact size of receiving file beforehand using c or c++ code.
If the server supports a range request to the specific resource (which is not guaranteed) then the answer will look like this:
HTTP/1.1 206 partial content
Content-Range: bytes 100-199/12345
This means that the response will contain the bytes 100..199 and that the total size of the content is 12345 bytes.
There are lots of questions here which deal with parsing HTTP headers so I will not go into the detail on how to specifically use C/C++ to extract these data from the header.
Please note also that you are doing a HTTP/1.1 request and thus must deal with possible chunked responses and implicit keep alive. I really recommend to use existing HTTP libraries instead of doing it all by hand and doing it wrong. If you really want to implement it all by your own please study the specification of HTTP.
I'm currently working on a project using sockets via WinSock and have come across a peculiar problem. I'll attach the code before I start explaining.
#include "Connection.h"
Connection::Connection(SOCKET sock, int socketType)
: m_sock(sock), m_recvCount(0), m_sendCount(0), m_socketType(socketType)
{
printf("Succesfully created connection\n");
}
Connection::~Connection(void)
{
printf("Closing socket %d", m_sock);
closesocket(m_sock);
}
void Connection::ProcessMessage(const NetMessage *message){
printf("Got network message: type %d, data %s\n", message->type, message->data);
}
bool Connection::ReadSocket(){
// Call this when the socket is ready to read.
// Returns true if the socket should be closed.
// used to store count between the sockets
int count = 0;
if(m_socketType == SOCK_STREAM){
// attempt to read a TCP socket message
// Receive as much data from the client as will fit in the buffer.
count = recv(m_sock, &m_recvBuf[m_recvCount], sizeof(m_recvBuf) - m_recvCount, 0);
}
else if(m_socketType == SOCK_DGRAM){
// attempt to read UDP socket message
// temporarily stores details of the address which sent the message
// since UDP doesn't worry about whether it's connected to the
// sender or not
sockaddr_in fromAddr;
int fromAddrSize = sizeof(fromAddr);
count = recvfrom(m_sock, &m_recvBuf[m_recvCount], sizeof(m_recvBuf) - m_recvCount, 0, (sockaddr*) &fromAddr, &fromAddrSize);
}
else{
printf("Unknown socket type %d\n", m_socketType);
return true;
}
if (count <= 0)
{
printf("Tried to receive on socket %d and got %d bytes\n", m_sock, count);
printf("Client connection closed or broken\n");
return true;
}
// if we get to this point we have essentially received a complete message
// and must process it
printf("Received %d bytes from the client (total %d)\n", count, m_recvCount);
m_recvCount += count;
// Have we received a complete message?
// if so, process it
if (m_recvCount == sizeof NetMessage)
{
ProcessMessage((const NetMessage *) m_recvBuf);
m_recvCount = 0;
}
return false;
}
bool Connection::WriteSocket(){
// Sends the data in the send buffer through the socket
int count;
if(m_socketType == SOCK_STREAM){
// attempt to read TCP socket message
count = send(m_sock, m_sendBuf, m_sendCount, 0);
}
else if(m_socketType == SOCK_DGRAM){
// attempt to read UDP socket message
count = sendto(m_sock, m_sendBuf, m_sendCount, 0, 0, 0);
}
else{
// unhandled type of socket, kill server
printf("Unknown socket type %d", m_socketType);
return true;
}
if (count <= 0)
{
// we have received an error from the socket
printf("Client connection closed or broken\n");
return true;
}
m_sendCount -= count;
printf("Sent %d bytes to the client (%d left)\n", count, m_sendCount);
printf("Data: %s", m_sendBuf);
// Remove the sent data from the start of the buffer.
memmove(m_sendBuf, &m_sendBuf[count], m_sendCount);
return false;
}
bool Connection::WantWrite(){
if(m_sendCount > 0){
return true;
}
return false;
}
bool Connection::WantRead(){
return true;
}
bool Connection::SetMessage(const NetMessage *message){
// store contents of the message in the send buffer
// to allow us to send later
if (m_sendCount + sizeof(NetMessage) > sizeof(m_sendBuf))
{
return true;
}
memcpy(&m_sendBuf, message, sizeof(message));
m_sendCount += sizeof(NetMessage);
return false;
}
and the protocol
/* Definitions for the network protocol that the client and server use to communicate */
#ifndef PROTOCOL_H
#define PROTOCOL_H
// Message types.
enum MessageType
{
MT_UNKNOWN = 0,
MT_WELCOME = 1,
MT_KEYPRESS = 2,
MT_CHATMESSAGE = 3
};
// The message structure.
// This is a "plain old data" type, so we can send it over the network.
// (In a real program, we would want this structure to be packed.)
struct NetMessage
{
MessageType type;
char* data;
NetMessage()
: type(MT_UNKNOWN)
{
}
};
#endif
Essentially the protocol holds the definition of the messages that the client and server throw around to each other. The problem I am having is that, in connection.cpp line 132 (memcpy), the message becomes garbled in sendBuf.
http://imgur.com/MekQfgm,9ShRtHi
The image above shows exactly what is happening. As said in protocol.h the struct is a POD so when I do memcpy it should transfer the number of bytes as is held in the struct (so for example the message type should be 1 byte, followed by 7 or 8 bytes of data, in the example).
Can anyone shed some light on this? It's driving me crazy.
The line you wrote will copy 4 bytes (sizeof(pointer)) on 32bit systems:
memcpy(&m_sendBuf, message, sizeof(message));
what you probably meant is:
memcpy(&m_sendBuf, message, sizeof(NetMessage));
Edit:
In addition, as a commenter remarked, your data type is NOT a POD. It holds a pointer. You transfer that pointer. At the target system, it will point to the same place in RAM, but there will not be anything there. You need to actually make your datatype a POD by using an array or you need to find a way to transfer the data pointed to. You can achieve this by transfering the type, a length and a number of characters. That means that your receiver can NOT rely on messages being of fixed size.
I am having a problem with a Serial Port communication between Arduino Nano and C++, even though the problem is in C++ side. Basically I want to send integers (or long,...) from the Arduino to a C++ program to be processed.
First I did a test sending information from the Arduino to the computer using Matlab. The Arduino code is pretty simple:
int i = 0;
void setup() {
// start serial port at 9600 bps:
Serial.begin(9600);
establishContact();
}
void loop() {
Serial.println(i);
i=i+1;
delay(10);
}
void establishContact() {
while (Serial.available() <= 0) {
Serial.println('A', BYTE);
delay(10);
}
}
The Matlab side is also simple:
clc;
clear all;
numSec=2;
t=[];
v=[];
s1 = serial('COM3'); % define serial port
s1.BaudRate=9600; % define baud rate
set(s1, 'terminator', 'LF'); % define the terminator for println
fopen(s1);
try % use try catch to ensure fclose
% signal the arduino to start collection
w=fscanf(s1,'%s'); % must define the input % d or %s, etc.
if (w=='A')
display(['Collecting data']);
fprintf(s1,'%s\n','A'); % establishContact just wants
% something in the buffer
end
i=0;
t0=tic;
while (toc(t0)<=numSec)
i=i+1;
t(i)=toc(t0);
t(i)=t(i)-t(1);
v(i)=fscanf(s1,'%d');
end
fclose(s1);
plot(t,v,'*r')
catch me
fclose(s1);
end
My goal is, with C++, do the same that is done in Matlab using fscanf(s1, '%d').
Here is the current code that I am using (C++ code):
void main()
{
HANDLE hSerial;
hSerial = CreateFile(TEXT("COM3"),
GENERIC_READ | GENERIC_WRITE,
0,
NULL,
OPEN_EXISTING,
FILE_ATTRIBUTE_NORMAL,//FILE_FLAG_OVERLAPPED,
NULL);
if ( hSerial == INVALID_HANDLE_VALUE)
{
printf("Error initializing handler");
}
else
{
// Set the parameters of the handler to the serial port.
DCB dcb = {0};
dcb.DCBlength = sizeof(dcb);
if ( !GetCommState(hSerial, &dcb) )
{
printf("Error setting parameters");
}
FillMemory(&dcb, sizeof(dcb), 0);
dcb.BaudRate = CBR_9600;
dcb.ByteSize = 8;
dcb.StopBits = ONESTOPBIT;
dcb.Parity = NOPARITY;
if ( !SetCommState(hSerial, &dcb) )
{
// error setting serial port state.
}
// Tell the program not to wait for data to show up
COMMTIMEOUTS timeouts = {0};
timeouts.ReadIntervalTimeout = 0;//20;
timeouts.ReadTotalTimeoutConstant = 0;//20;
timeouts.ReadTotalTimeoutMultiplier = 0;//50;
timeouts.WriteTotalTimeoutConstant = 0;//100;
timeouts.WriteTotalTimeoutMultiplier = 0;//100;
if ( !SetCommTimeouts(hSerial, &timeouts) )
{
printf("Error setting the timeouts");
}
char szBuff[5] = "";
DWORD dwBytesRead = 0;
int i = 0;
char test[] = "B\n";
int maxSamples = 10;
DWORD dwCommStatus;
WriteFile(hSerial, test, 2, &dwBytesRead, NULL);
SetCommMask(hSerial,EV_RXCHAR);
while (i < maxSamples)
{
WaitCommEvent (hSerial, &dwCommStatus, 0);
if (dwCommStatus & EV_RXCHAR)
{
memset(szBuff,0,sizeof(szBuff));
ReadFile(hSerial, LPVOID(szBuff), 4, &dwBytesRead, NULL);
cout<<szBuff;
printf(" - %d - \n", atoi(szBuff));
}
i++;
}
scanf("%d", &i);
CloseHandle(hSerial);
}
}
The goal of my code would be something like num = ReadSerialCOM(hSerial, "%d");
My current C++ code reads the information from the buffer, but there is not an accepted end of line, which implies that my numbers (integers) are received cut.
Eg:
I send 8889 from the Arduino, which places it in the COM port. And the command ReadFile saves '88' into szBuff. At the next iteration '89\n' is saved into sZBuff. Basically I want to avoid to post-process sZBuff to concat '88' and '89\n'.
Anyone?
Thanks!
If I understand your question correctly, one way to avoid having to 'post-process' is to move the pointer passed to ReadFile to the end of the available data, so the ReadFile call is appending to the buffer, instead of overwriting.
Essentially, you would have two pointers. One to the buffer, the other to the end of the data in the buffer. So when your program starts, both pointers will be the same. Now, you read the first 2 bytes. You increment the end-of-data pointer by 2. You do another read, but instead of szBuff, you pass a pointer to the end of the previously read data. You read the next three bytes and you have the complete entry in szBuff.
If you need to wait until some delimiter to mark the end of an entry is received, you could just search the received data for it. If it's not there, you keep reading until you find it. If it is there, you can just return.
// Fill the buffer with 0
char szBuff[256] = {0};
// We have no data in the buffer, so the end of data points to the beginning
// of the buffer.
char* szEndOfData = szBuff;
while (i < maxSamples)
{
WaitCommEvent (hSerial, &dwCommStatus, 0);
if (dwCommStatus & EV_RXCHAR)
{
// Append up to 4 bytes from the serial port to the buffer
ReadFile(hSerial, LPVOID(szEndOfData), 4, &dwBytesRead, NULL);
// Increment the end of data pointer, so it points to the end of the
// data available in the buffer.
szEndOfData += dwBytesRead;
cout<<szBuff;
printf(" - %d - \n", atoi(szBuff));
}
i++;
}
// Output, assuming what you mentioned happens:
// - 88 -
// - 8889 -
If this approach is acceptable to you, it will require a bit more work. For example, you would have to ensure you don't overflow your buffer. When you remove data from the buffer, you'll have to move all of the data after the removed segment to the beginning, and fix the end of data pointer. Alternatively, you could use a circular buffer.
As Hans Passant and dauphic pointed, it doesn't seem to be a general solution for my question. I am writing, though, the code that I was trying to avoid, just in case somebody finds it useful or face the same problem that I had:
int i = 0;
DWORD dwBytesRead = 0;
DWORD dwCommStatus = 0;
char szBuff[2] = "";
int maxRead = 20;
int sizeNum = 1;
int *num = (int*)malloc(maxRead*sizeof(int));
char *currNum;
char *pastNum;
// Write something into the Serial Port to start receive
// information from the Arduino
WriteFile(hSerial, (LPCVOID)"A\0", 1, &dwBytesRead, NULL);
SetCommMask(hSerial, EV_RXCHAR);
// Start reading from the Serial Port
while ( i < maxRead )
{
WaitCommEvent (hSerial, &dwCommStatus, 0);
if (dwCommStatus & EV_RXCHAR) // if a char is received in the serial port
{
ReadFile(hSerial, LPVOID(szBuff), 1, &dwBytesRead, NULL);
if ( szBuff[0] > 47 && szBuff[0] < 58 )
{
sizeNum++;
if (sizeNum ==2)
{
currNum = (char*)malloc(sizeNum*sizeof(char));
strcpy(currNum, szBuff);
} else
{
if (pastNum != NULL)
free(pastNum);
pastNum = currNum;
currNum = (char*)malloc(sizeNum*sizeof(char));
strcpy(currNum, pastNum);
strcpy(currNum+(sizeNum-2)*sizeof(char), szBuff);
}
cout << szBuff<<endl;
} else if (szBuff[0] == '\n' && sizeNum > 1) // end of number
{
num[i] = atoi(currNum);
i++;
sizeNum = 1;
if (currNum!=NULL)
free(currNum);
}
}
}