I am trying to send out Mat image by TCP. Firstly the Mat has been transferred into uchar and then into char format. The whole image in char format will be send out buffer by buffer whose size is 1024 byte. The following is my code.
Mat decodeImg = imdecode(Mat(bufferFrame), 1);
uchar *transferImg = decodeImg.data;
char* charImg = (char*) transferImg;
int length = strlen(charImg);
int offset = 0;
while (true)
{
bzero(bufferSend, BUFFER_SIZE);
if (offset + BUFFER_SIZE <= length)
{
for (int i = 0; i < BUFFER_SIZE; i++)
{
bufferSend[i] = charImg[i + offset];
}
// memcpy(charImg+offset, bufferSend,BUFFER_SIZE);
if (send(sockfd, bufferSend, sizeof(bufferSend), 0) < 0)
{
printf("Send FIle Failed,total length is%d,failed offset is%d\n",
length,
offset);
break;
}
}
else
{
for (int i = 0; i < length - offset; i++)
{
bufferSend[i] = charImg[i + offset];
}
if (send(sockfd, bufferSend, sizeof(bufferSend), 0) < 0)
{
printf("Send FIle Failed,total length is%d,failed offset is%d\n",
length,
offset);
break;
}
break;
}
offset += BUFFER_SIZE;
}
The output of the code shows : send file failed, total length is 251035, failed offset is 182272.
I am really appreciated on your help. Thank you in advance!
Pulling out the crystal ball here. This might be OP's problem, but if it isn't, this is certainly a problem that needs to be addressed.
Mat decodeImg = imdecode(Mat(bufferFrame), 1);
uchar *transferImg = decodeImg.data;
Get data. Not a bad idea if that's what you need to send.
char* charImg = (char*) transferImg;
Take the array of bytes from above and treat it as an array of characters.
int length = strlen(charImg);
And Boom. Matrix data is not ascii formated data, a string, so it should not be treated like a string.
strlen counts data until it reaches a null character, a character with the numerical value 0, which does not exist in the normal alpha numeric world and thus can be used as a canary value to signal the end of a string. The count is the number of characters before the first null character in the string.
In this case we don't have a string. We have a blob of binary numbers, any one of which could bee 0. There could be a null value anywhere. Could be right at the beginning. Could be a hundred bytes in. There might not be a null value in the until long after all of the valid image data has been read.
Anyway, strlen will almost certainly return the wrong value. Too few bytes and the receiver doesn't get all of the image data and I have no idea what it does. That code's not available to us. It probably gets upset and discards the result. Maybe it crashes. There's no way to know. If there is too much information, we also don't know what happens. Maybe it processes the file happily and ignores the extra crap that's sent. Maybe it crashes.
But what if it closes the TCP/IP connection when it has enough bytes? That leaves the sender trying to write a handful of unsent and unwanted bytes into a closed socket. send will fail and set the error code to socket closed.
Solution:
Get the right size of the data.
What I'm reading from the openCV documentation is inside a Mat is Mat::elemSize which will give you the size of each item in the matrix and Mat::size which returns a Size object containing the rows and columns. Multiply rows * columns * elemSize and you should have the number of bytes to send.
EDIT
This looks to be a better way to get the size.
Related
I know how my packet looks like. It has 6 header fields (1 byte each, each header has 8 fields) and then it has the payload (data).
I would like to build a raw packet in C or C++ (it should look the same I think).
Here's what I think I should do:
unsigned char packet[11];
packet[0] = (0x81); // first header with 8 fields
packet[1] = (0x8c); // second header with 8 fields
packet[2] = (0xfe);
packet[3] = (0x84);
packet[4] = (0x1d);
packet[5] = (0x79);
packet[6] = (0x96); // payload, the 'h' letter, masked
packet[7] = (0xe1); // 'e'
packet[8] = (0x71); // 'l'
packet[9] = (0x15); // 'l'
packet[10] = (0x91);// 'o'
Where, for instance, 0x81 is the first byte (I simply converted every field (bit) of my first header to hex).
And then, simply, I want to send it to server: send(sockfd, packet, sizeof(packet), 0) to send it.
Receiving and printing the response:
unsigned char buffer[1024];
if ((recv(sockfd, buffer, len, 0)) == 0)
{
if (errno != 0)
{
exit(1);
}
}
int i;
for(i = 0; i<len; i++)
printf("%x ", buffer[i]);
Am I right?
Other than mishandling the return value from recv, your code looks okay.
if ((recv(sockfd, buffer, len, 0)) == 0)
{
if (errno != 0)
{
exit(1);
}
}
A zero return indicates normal close of the connection. There's no reason to check errno if it returns zero.
A return value of -1 indicates an error. In that case, it does make sense to check errno.
A value greater than zero indicates that number of bytes have been received. Be aware that it is perfectly normal for recv to return fewer bytes than you asked it for. If you want to receive exactly some number of bytes, you must call recv in a loop.
TCP is a byte-stream protocol and has no idea where your "packets" (really, messages) begin and end.
Your code will not appear to be error-prone!
But a good practice would be:
const std::uint32_t BUFFER_SIZE = 11;
std::vector<std::uint8_t> buffer;
buffer.reserve(BUFFER_SIZE)
buffer = {0x81,0x8c.....};
send( sockfd,
reinterpret_cast <const char*> ( buffer.data() ),
static_cast <int> ( buffer.size() ),
0
);
Doing so, your code gets more optimized, and avoids possible leaks, using the std vectors.
May also benefit from taking a look at ZeroMQ, as an example of a ready-made, high-performance asynchronous messaging library, aimed at use in distributed or concurrent applications.
I'm trying to read by ReadFile but always get that its read 4 bytes, doesn't mutter how long was the string.
UART* uart = (UART*)lpParam;
char TempChar; //Temporary character used for reading
char SerialBuffer[256];//Buffer for storing Rxed Data
DWORD NoBytesRead;
int i = 0;
do
{
NoBytesRead = 0;
ReadFile(uart->connHandle, //Handle of the Serial port
&SerialBuffer, //Temporary character
sizeof(256),//Size of TempChar
&NoBytesRead, //Number of bytes read
NULL);
//SerialBuffer[i] = TempChar;// Store Tempchar into buffer
i++;
if (NoBytesRead > 0)
{
char* strMsg = (char*)malloc(sizeof(256 * sizeof(char)));
SerialBuffer[NoBytesRead] = '\0';
TRACE("read %d- %s\n", NoBytesRead,SerialBuffer);
strcpy_s(strMsg, 256,SerialBuffer);
ControllerPublishMsg(uart->controller, SerialBuffer);
}
SerialBuffer[0] = '\0';
In case i send string "hh" to connection I'm get output "read 4- hh".
The string is 2 bytes long, but NoBytesRead = 4.
thanks.
sizeof(256) defaults to sizeof(int) this is four bytes. replace sizeof(256) by 256. Also replace sizeof(256 * sizeof(char)) by (256 * sizeof(char)).
Think about the statement
sizeof(256)
that you pass as a buffer size.
That expression evaluates to the same thing as
sizeof(int)
which probably evaluates to 4 on your platform. You'd need to hand over the literal value 256 or better sizeof SerialBuffer to ReadFile.
And you got the same error in your malloc arguments.
Why you are receiving 4 characters when you (think you) are sending only 2 is impossible to see without the code on the sender side. In case ReadFile returns 4, it most probably received 4 characters. Due to the messed up buffer size argument, it will however not be able to receive more than 4 characters.
You are misusing sizeof.
When calling ReadFile(), you are using sizeof(256) as the number of bytes to read. A numeric literal is an int by default, so you are really using sizeof(int), which is 4 bytes on your compiler. Get rid of the sizeof and just use 256 by itself:
ReadFile(uart->connHandle, //Handle of the Serial port
&SerialBuffer, //Temporary character
256,//Size of TempChar
&NoBytesRead, //Number of bytes read
NULL);
Or better, get rid of the 256 and use sizeof(SerialBuffer) instead, since it is a static array with a fixed size known at compile time:
ReadFile(uart->connHandle, //Handle of the Serial port
&SerialBuffer, //Temporary character
sizeof(SerialBuffer),//Size of TempChar
&NoBytesRead, //Number of bytes read
NULL);
You are making a similar mistake when calling malloc(). sizeof(char) is always 1, so you are really calling sizeof(256) again. So again, you can get rid of sizeof and just use 256 by itself:
char* strMsg = (char*) malloc(256 * sizeof(char));
// or just: char* strMsg = (char*) malloc(256);
Although, you are not actually using strMsg for anything (and you are leaking it), so you should just get rid of it completely.
Try something more like this:
UART* uart = (UART*)lpParam;
char SerialBuffer[257];//Buffer for storing Rxed Data
DWORD NoBytesRead;
do
{
NoBytesRead = 0;
ReadFile(uart->connHandle, //Handle of the Serial port
SerialBuffer, //Temporary buffer
sizeof(SerialBuffer)-1,//Size of buffer minus null-terminator
&NoBytesRead, //Number of bytes read
NULL);
if (NoBytesRead > 0)
{
SerialBuffer[NoBytesRead] = '\0';
TRACE("read %u- %s\n", NoBytesRead, SerialBuffer);
ControllerPublishMsg(uart->controller, SerialBuffer);
}
I am trying to implement a simple file transfer. Below here is two methods that i have been testing:
Method one: sending and receiving without splitting the file.
I hard coded the file size for easier testing.
sender:
send(sock,buffer,107,NULL); //sends a file with 107 size
receiver:
char * buffer = new char[107];
recv(sock_CONNECTION,buffer,107,0);
std::ofstream outfile (collector,std::ofstream::binary);
outfile.write (buffer,107);
The output is as expected, the file isn't corrupted because the .txt file that i sent contains the same content as the original.
Method two: sending and receiving by splitting the contents on receiver's side. 5 bytes each loop.
sender:
send(sock,buffer,107,NULL);
Receiver:
char * buffer = new char[107]; //total file buffer
char * ptr = new char[5]; //buffer
int var = 5;
int sizecpy = size; //orig size
while(size > var ){ //collect bytes
recv(sock_CONNECTION,ptr,5,0);
strcat(buffer,ptr); //concatenate
size= size-var; //decrease
std::cout<<"Transferring.."<<std::endl;
}
std::cout<<"did it reach here?"<<std::endl;
char*last = new char[size];
recv(sock_CONNECTION,last,2,0); //last two bytes
strcat(buffer,last);
std::ofstream outfile (collector,std::ofstream::binary);
outfile.write (buffer,107);
Output: The text file contains invalid characters especially at the beginning and the end.
Questions: How can i make method 2 work? The sizes are the same but they yield different results. the similarity of the original file and the new file on method 2 is about 98~99% while it's 100% on method one. What's the best method for transferring files?
What's the best method for transferring files?
Usually I'm not answering questions like What's the best method. But in this case it's obvious:
You sent the file size and a checksum in network byte order, when starting a transfer
Sent more header data (e.g filename) optionally
The client reads the file size and the checksum, and decodes it to host byte order
You sent the file's data in reasonably sized chunks (5 bytes isn't a reasonable size), chunks should match tcp/ip frames maximum available payload size
You receive chunk by chunk at the client side until the previously sent file size is matched
You calculate the checksum for the received data at the client side, and check if it matches the one that was received beforhand
Note: You don't need to combine all chunks in memory at the client side, but just append them to a file at a storage medium. Also the checksum (CRC) usually can be calculated from running through data chunks.
Disagree with Galik. Better not to use strcat, strncat, or anything but the intended output buffer.
TCP is knda fun. You never really know how much data you are going to get, but you will get it or an error.
This will read up to MAX bytes at a time. #define MAX to whatever you want.
std::unique_ptr<char[]> buffer (new char[size]);
int loc = 0; // where in buffer to write the next batch of data
int bytesread; //how much data was read? recv will return -1 on error
while(size > MAX)
{ //collect bytes
bytesread = recv(sock_CONNECTION,&buffer[loc],MAX,0);
if (bytesread < 0)
{
//handle error.
}
loc += bytesread;
size= size-bytesread; //decrease
std::cout<<"Transferring.."<<std::endl;
}
bytesread = recv(sock_CONNECTION,&buffer[loc],size,0);
if (bytesread < 0)
{
//handle error
}
std::ofstream outfile (collector,std::ofstream::binary);
outfile.write (buffer.get(),size);
Even more fun, write into the output buffer so you don't have to store the whole file. In this case MAX should be a bigger number.
std::ofstream outfile (collector,std::ofstream::binary);
char buffer[MAX];
int bytesread; //how much data was read? recv will return -1 on error
while(size)
{ //collect bytes
bytesread = recv(sock_CONNECTION,buffer,MAX>size?size:MAX,0);
// MAX>size?size:MAX is like a compact if-else: if (MAX>size){size}else{MAX}
if (bytesread < 0)
{
//handle error.
}
outfile.write (buffer,bytesread);
size -= bytesread; //decrease
std::cout<<"Transferring.."<<std::endl;
}
The initial problems I see are with std::strcat. You can't use it on an uninitialized buffer. Also you are not copying a null terminated c-string. You are copying a sized buffer. Better to use std::strncat for that:
char * buffer = new char[107]; //total file buffer
char * ptr = new char[5]; //buffer
int var = 5;
int sizecpy = size; //orig size
// initialize buffer
*buffer = '\0'; // add null terminator
while(size > var ){ //collect bytes
recv(sock_CONNECTION,ptr,5,0);
strncat(buffer, ptr, 5); // strncat only 5 chars
size= size-var; //decrease
std::cout<<"Transferring.."<<std::endl;
}
beyond that you should really as error checking so the sockets library can tell you if anything went wrong with the communication.
I have an arduino board that is connected to a sensor. From Arduino IDE serial monitor, I see the readings are mostly 160, 150, etc. Arduino has a 10 bit ADC, so I assume the readings range from 0 to 1024.
I want to fetch that readings to my computer so that I can do further processing. It must be done this way up to this point. Now, I wrote a c++ program to read serial port buffer with Windows APIs (DCB). The transfer speed of the serial ports are set to 115200 on both the Arduino IDE and the c++ program.
I will describe my problem first: Since I want to send the readings to my computer, I expect the data looks like the following:
124
154
342
232
...
But now it looks like
321
43
5
2
123
...
As shown, the data are concatenated. I knew it because I tried to display them with [], and the data are truly messed up.
The section of the code that is doing the serial port reading on the computer is as here:
// Read
int n = 10;
char szBuff[10 + 1] = {0};
DWORD dwBytesRead = 0;
int i;
for (i = 0; i < 200; i++){
{
if(!ReadFile(hSerial, szBuff, n, &dwBytesRead, NULL)){
//error occurred. Report to user.
printf("Cannot read.\n");
}
else{
printf("%s\n" , szBuff);
}
}
}
The Arduino code that's doing the serial port sending is:
char buffer [10] = { 0 };
int analogIn = 0;
int A0_val = 0;
void setup() {
Serial.begin(115200);
}
void loop() {
A0_val = analogRead(analogIn);
sprintf(buffer, "%d", A0_val);
Serial.println(buffer);
}
I suspect that the messing up of the data is caused by different size of the buffer used to transmit and receive data in the serial port. What is the good suggestion for the size of the buffer and even better method to guarantee the successful transmission of valid data?
Thanks very much!
Your reciever code cannot assume a single read from the serial port will yield a complete line (i.e. the 2 or 3 digits followed by a '\n' that the arduino continuously sends).
It is up to the receiver to synthetize complete lines of text on reception, and only then try to use them as meaningful numbers.
Since the serial interface is extremely slow compared with your average PC computing power, there is little point in reading more than one character at a time: literally millions of CPU cycles will be spent waiting for the next character, so you really don't need to react fast to the arduino input.
Since in that particular case it will not hinder performances in the slightest, I find it more convenient to read one character at a time. That will save you the hassle of moving bits of strings around. At least it makes writing an educational example easier.
// return the next value received from the arduino as an integer
int read_arduino (HANDLE hserial)
{
char buffer[4]; // any value longer than 3 digits must come
// from a faulty transmission
// the 4th caracter is used for a terminating '\0'
size_t buf_index = 0; // storage position of received characters
for (;;)
{
char c; // read one byte at a time
if (!ReadFile(
hSerial,
&c, // 1 byte buffer
1, // of length 1
NULL, // we will read exactly one byte or die trying,
// so length checking is pointless
NULL)){
/*
* This error means something is wrong with serial port config,
* and I assume your port configuration is hard-coded,
* so the code won't work unless you modify and recompile it.
* No point in keeping the progam running, then.
*/
fprintf (stderr, "Dang! Messed up the serial port config AGAIN!");
exit(-1);
}
else // our read succeded. That's a start.
{
if (c == '\n') // we're done receiving a complete value
{
int result; // the decoded value we might return
// check for buffer overflow
if (buf_index == sizeof (buffer))
{
// warn the user and discard the input
fprintf (stderr,
"Too many characters received, input flushed\n");
}
else // valid number of characters received
{
// add a string terminator to the buffer
buffer[buf_index] = '\0';
// convert to integer
result = atoi (buffer);
if (result == 0)
{
/*
* assuming 0 is not a legit value returned by the arduino, this means the
* string contained something else than digits. It could happen in case
* of electricval problems on the line, typically if you plug/unplug the cable
* while the arduino is sending (or Mr Fluffy is busy gnawing at it).
*/
fprintf (stderr, "Wrong value received: '%s'\n", buffer);
}
else // valid value decoded
{
// at last, return the coveted value
return res; // <-- this is the only exit point
}
}
// reset buffer index to prepare receiving the next line
buf_index = 0;
}
else // character other than '\n' received
{
// store it as long as our buffer does not overflow
if (buf_index < sizeof (buffer))
{
buffer[buf_index++] = c;
/*
* if, for some reason, we receive more than the expected max number of
* characters, the input will be discarded until the next '\n' allow us
* to re-synchronize.
*/
}
}
}
}
}
CAVEAT: this is just code off the top of my head. I might have left a few typos here and there, so don't expect it to run or even compile out of the box.
A couple of basic problems here. First, it is unlikely that the PC can reliably keep up with 115,200 baud data if you only read 10 bytes at a time with ReadFile. Try a slower baud rate and/or change the buffer size and number of bytes per read to something that will get around 20 milliseconds of data, or more.
Second, after you read some data put a nul at the end of it
szBuf[dwBytesRead] = 0;
before you pass it to printf or any other C string code.
Is there a way for me to take a memory address and advance it a certain amount that is stored in a variable? And what would that variable type have to be?
For example, in the following code I'd like to first look at data + 0, and then for each step after that look at data + sent. If I'm looking at this correctly, sent is stored as bytes, and data is a memory address.
bool sendAll(int socket, const void *data, ssize_t size) {
ssize_t sent = 0;
ssize_t just_sent;
while (sent < size) {
just_sent = send(socket, data + sent, size - sent, 0);
if (just_sent < 0) {
return false;
}
sent += just_sent;
}
return true;
}
That's what char* will do. Pointer math, when the pointer has type T*, always works on increments of sizeof (T). And sizeof (char) == 1 by definition.
So try:
just_sent = send(socket, sent + (const char*)data, size - sent, 0);
Just complementing #Ben's answer because I think that "Just replace the one line calling send with the version in my answer." isn't a really good answer...
#Rick, what it is important is that you know how many bytes were transferred. No matter if the array is an array of integers, if you want to start from the sent-th byte you can cast data to (char*) so the compiler do the correct pointer arithmetic on (char*)data + sent adding sent bytes to data. Thats because sizeof(char) == 1 byte. If instead, you cast data to (int*), instead of shifting sent bytes, you would shift sent * sizeof(int) bytes.