Is there a way for me to take a memory address and advance it a certain amount that is stored in a variable? And what would that variable type have to be?
For example, in the following code I'd like to first look at data + 0, and then for each step after that look at data + sent. If I'm looking at this correctly, sent is stored as bytes, and data is a memory address.
bool sendAll(int socket, const void *data, ssize_t size) {
ssize_t sent = 0;
ssize_t just_sent;
while (sent < size) {
just_sent = send(socket, data + sent, size - sent, 0);
if (just_sent < 0) {
return false;
}
sent += just_sent;
}
return true;
}
That's what char* will do. Pointer math, when the pointer has type T*, always works on increments of sizeof (T). And sizeof (char) == 1 by definition.
So try:
just_sent = send(socket, sent + (const char*)data, size - sent, 0);
Just complementing #Ben's answer because I think that "Just replace the one line calling send with the version in my answer." isn't a really good answer...
#Rick, what it is important is that you know how many bytes were transferred. No matter if the array is an array of integers, if you want to start from the sent-th byte you can cast data to (char*) so the compiler do the correct pointer arithmetic on (char*)data + sent adding sent bytes to data. Thats because sizeof(char) == 1 byte. If instead, you cast data to (int*), instead of shifting sent bytes, you would shift sent * sizeof(int) bytes.
Related
I want to send a string from a server to a client. To account for the case that the string is bigger than the buffer that the client reads into, I send a byte count to the client before the actual string. Then on the client side, I split the read operation into multiple recv() calls if needed.
Server:
// Send byte count to client so it knows how much data to expect
strcpy(buffer, std::to_string(string_size).c_str());
int result{ send(socket, buffer, buffer_size, 0) };
// Send actual data
unsigned int bytes_to_send{ sizeof(s) };
while (bytes_to_send > 0) {
send_bytes = std::min(buffer_size, bytes_to_send);
strcpy(buffer, s.substr(0, send_bytes).c_str());
result = send(socket, buffer, send_bytes, 0);
if (s.size() > send_bytes) {
s = s.substr(send_bytes, s.size() - send_bytes);
}
bytes_to_send -= send_bytes;
return_int += send_bytes;
(In the actual code, I check result to see whether send() succeeded)
Client:
result = recv(socket, buffer, buffer_size, 0);
count = std::stoi(buffer);
while(count > 0){
read_bytes = std::min(count, buffer_size);
result = recv(socket, buffer, read_bytes, 0);
if(result < 1 ) {
// throw exception
}else{
return_string += buffer;
}
count -= read_bytes;
What I expected:
Client blocks on first recv() call until server sends byte count
Client proceeds until second recv() call, then blocks until server sends string
What actually happens:
Client blocks on first recv() call until server sends byte count
Client does not block on second recv() call. recv() return value is equal to read_bytes, but does not fill the buffer with anything
Thanks in advance for any help with this
There are quite a few problems with your code.
strcpy(buffer, std::to_string(string_size).c_str()); is a buffer overflow waiting to happen.
send(socket, buffer, buffer_size, 0) - you did not set buffer_size to the number of bytes copied into buffer by strcpy(). It looks like you are sending the entire buffer without regard to how many bytes it is actually holding.
sizeof(s) does not give you the number of characters in a std::string. It gives you the byte size of the std::string class itself, which is not the same thing. You need to use the string's size() or length() method to get the proper number of characters.
TCP is a byte stream, there is no 1-to-1 relationship between send() and recv(). Both can report fewer bytes were processed than requested, but you are not accounting for that properly.
You are completely ignoring the return value of send(), assuming that all requested bytes have been sent in full on each call, when in actuality fewer bytes may have been sent, or none at all. On the 1st call, you are not accounting for the possibility that the entire size string may not be sent in 1 call and may need to be looped. Inside of the loop, you are completely ignoring the return value of send() for purposes of looping.
Likewise, you are ignoring the return value of recv() on the 1st call, assuming the entire string size is received in 1 call, without regard to how many characters are actually in the string size. Inside the loop, you are paying attention to the return value of recv() for purposes of looping, except that you are not accounting for the possibility that recv() returns 0 on a graceful disconnect. You are not breaking your loop if that happens.
It is good that you want to send the string's size before sending its content. However, you are sending the size as another variable-length string, which really defeats the purpose of sending a size at all. You should send the size using a fixed-length integer in binary form instead.
Try something more like this instead:
void send_raw(int socket, const void *buffer, size_t buffer_size)
{
const char *ptr = (const char*) buffer;
while (buffer_size > 0) {
int sent = send(socket, ptr, buffer_size, 0);
if (sent < 0) {
// throw exception
}
ptr += sent;
buffer_size -= sent;
}
}
void send_u32(int socket, uint32_t value)
{
value = htonl(value);
send_raw(socket, &value, sizeof(value));
}
...
// Send byte count to client so it knows how much data to expect
send_u32(socket, s.size());
// Send actual data
send_raw(socket, s.c_str(), s.size());
void recv_raw(int socket, void *buffer, size_t buffer_size)
{
char *ptr = (char*) buffer;
while (buffer_size > 0) {
int recvd = recv(socket, ptr, buffer_size, 0);
if (recvd <= 0) {
// throw exception
}
ptr += recvd;
buffer_size -= recvd;
}
}
uint32_t recv_u32(int socket)
{
uint32_t value;
recv_raw(socket, &value, sizeof(value));
return ntohl(value);
}
...
size_t count = recv_u32(socket);
return_string.resize(count);
recv_raw(socket, &return_string[0], count);
However, if you really want to stick with sending the string size as another string, you need to delimit it, such as sending its null terminator, eg:
// Send byte count to client so it knows how much data to expect
std::string size_buffer = std::to_string(s.size());
send_raw(socket, size_buffer.c_str(), size_buffer.size()+1);
// Send actual data
send_raw(socket, s.c_str(), s.size());
std::string size_buffer;
char ch;
do {
recv_raw(socket, &ch, 1);
if (ch == '\0') break;
size_buffer += ch;
}
while (true);
size_t count = std::stoi(size_buffer);
return_string.resize(count);
recv_raw(socket, &return_string[0], count);
You can add MSG_WAITALL to the recv() flags and do only one read.
I know how my packet looks like. It has 6 header fields (1 byte each, each header has 8 fields) and then it has the payload (data).
I would like to build a raw packet in C or C++ (it should look the same I think).
Here's what I think I should do:
unsigned char packet[11];
packet[0] = (0x81); // first header with 8 fields
packet[1] = (0x8c); // second header with 8 fields
packet[2] = (0xfe);
packet[3] = (0x84);
packet[4] = (0x1d);
packet[5] = (0x79);
packet[6] = (0x96); // payload, the 'h' letter, masked
packet[7] = (0xe1); // 'e'
packet[8] = (0x71); // 'l'
packet[9] = (0x15); // 'l'
packet[10] = (0x91);// 'o'
Where, for instance, 0x81 is the first byte (I simply converted every field (bit) of my first header to hex).
And then, simply, I want to send it to server: send(sockfd, packet, sizeof(packet), 0) to send it.
Receiving and printing the response:
unsigned char buffer[1024];
if ((recv(sockfd, buffer, len, 0)) == 0)
{
if (errno != 0)
{
exit(1);
}
}
int i;
for(i = 0; i<len; i++)
printf("%x ", buffer[i]);
Am I right?
Other than mishandling the return value from recv, your code looks okay.
if ((recv(sockfd, buffer, len, 0)) == 0)
{
if (errno != 0)
{
exit(1);
}
}
A zero return indicates normal close of the connection. There's no reason to check errno if it returns zero.
A return value of -1 indicates an error. In that case, it does make sense to check errno.
A value greater than zero indicates that number of bytes have been received. Be aware that it is perfectly normal for recv to return fewer bytes than you asked it for. If you want to receive exactly some number of bytes, you must call recv in a loop.
TCP is a byte-stream protocol and has no idea where your "packets" (really, messages) begin and end.
Your code will not appear to be error-prone!
But a good practice would be:
const std::uint32_t BUFFER_SIZE = 11;
std::vector<std::uint8_t> buffer;
buffer.reserve(BUFFER_SIZE)
buffer = {0x81,0x8c.....};
send( sockfd,
reinterpret_cast <const char*> ( buffer.data() ),
static_cast <int> ( buffer.size() ),
0
);
Doing so, your code gets more optimized, and avoids possible leaks, using the std vectors.
May also benefit from taking a look at ZeroMQ, as an example of a ready-made, high-performance asynchronous messaging library, aimed at use in distributed or concurrent applications.
I am trying to send out Mat image by TCP. Firstly the Mat has been transferred into uchar and then into char format. The whole image in char format will be send out buffer by buffer whose size is 1024 byte. The following is my code.
Mat decodeImg = imdecode(Mat(bufferFrame), 1);
uchar *transferImg = decodeImg.data;
char* charImg = (char*) transferImg;
int length = strlen(charImg);
int offset = 0;
while (true)
{
bzero(bufferSend, BUFFER_SIZE);
if (offset + BUFFER_SIZE <= length)
{
for (int i = 0; i < BUFFER_SIZE; i++)
{
bufferSend[i] = charImg[i + offset];
}
// memcpy(charImg+offset, bufferSend,BUFFER_SIZE);
if (send(sockfd, bufferSend, sizeof(bufferSend), 0) < 0)
{
printf("Send FIle Failed,total length is%d,failed offset is%d\n",
length,
offset);
break;
}
}
else
{
for (int i = 0; i < length - offset; i++)
{
bufferSend[i] = charImg[i + offset];
}
if (send(sockfd, bufferSend, sizeof(bufferSend), 0) < 0)
{
printf("Send FIle Failed,total length is%d,failed offset is%d\n",
length,
offset);
break;
}
break;
}
offset += BUFFER_SIZE;
}
The output of the code shows : send file failed, total length is 251035, failed offset is 182272.
I am really appreciated on your help. Thank you in advance!
Pulling out the crystal ball here. This might be OP's problem, but if it isn't, this is certainly a problem that needs to be addressed.
Mat decodeImg = imdecode(Mat(bufferFrame), 1);
uchar *transferImg = decodeImg.data;
Get data. Not a bad idea if that's what you need to send.
char* charImg = (char*) transferImg;
Take the array of bytes from above and treat it as an array of characters.
int length = strlen(charImg);
And Boom. Matrix data is not ascii formated data, a string, so it should not be treated like a string.
strlen counts data until it reaches a null character, a character with the numerical value 0, which does not exist in the normal alpha numeric world and thus can be used as a canary value to signal the end of a string. The count is the number of characters before the first null character in the string.
In this case we don't have a string. We have a blob of binary numbers, any one of which could bee 0. There could be a null value anywhere. Could be right at the beginning. Could be a hundred bytes in. There might not be a null value in the until long after all of the valid image data has been read.
Anyway, strlen will almost certainly return the wrong value. Too few bytes and the receiver doesn't get all of the image data and I have no idea what it does. That code's not available to us. It probably gets upset and discards the result. Maybe it crashes. There's no way to know. If there is too much information, we also don't know what happens. Maybe it processes the file happily and ignores the extra crap that's sent. Maybe it crashes.
But what if it closes the TCP/IP connection when it has enough bytes? That leaves the sender trying to write a handful of unsent and unwanted bytes into a closed socket. send will fail and set the error code to socket closed.
Solution:
Get the right size of the data.
What I'm reading from the openCV documentation is inside a Mat is Mat::elemSize which will give you the size of each item in the matrix and Mat::size which returns a Size object containing the rows and columns. Multiply rows * columns * elemSize and you should have the number of bytes to send.
EDIT
This looks to be a better way to get the size.
First question: I am confused between Buffers in TCP. I am trying to explain my proble, i read this documentation TCP Buffer, author said a lot about TCP Buffer, thats fine and a really good explanation for a beginner. What i need to know is this TCP Buffer is same buffer with the one we use in our basic client server program (Char *buffer[Some_Size]) or its some different buffer hold by TCP internally ?
My second question is that i am sending a string data with prefix length (This is data From me) from client over socket to server, when i print my data at console along with my string it prints some garbage value also like this "This is data From me zzzzzz 1/2 1/2....." ?. However i fixed it by right shifting char *recvbuf = new char[nlength>>3]; nlength to 3 bits but why i need to do it in this way ?
My third question is in relevance with first question if there is nothing like TCP Buffer and its only about the Char *buffer[some_size] then whats the difference my program will notice using such static memory allocation buffer and by using dynamic memory allocation buffer using char *recvbuf = new char[nlength];. In short which is best and why ?
Client Code
int bytesSent;
int bytesRecv = SOCKET_ERROR;
char sendbuf[200] = "This is data From me";
int nBytes = 200, nLeft, idx;
nLeft = nBytes;
idx = 0;
uint32_t varSize = strlen (sendbuf);
bytesSent = send(ConnectSocket,(char*)&varSize, 4, 0);
assert (bytesSent == sizeof (uint32_t));
std::cout<<"length information is in:"<<bytesSent<<"bytes"<<std::endl;
// code to make sure all data has been sent
while (nLeft > 0)
{
bytesSent = send(ConnectSocket, &sendbuf[idx], nLeft, 0);
if (bytesSent == SOCKET_ERROR)
{
std::cerr<<"send() error: " << WSAGetLastError() <<std::endl;
break;
}
nLeft -= bytesSent;
idx += bytesSent;
}
std::cout<<"Client: Bytes sent:"<< bytesSent;
Server code:
int bytesSent;
char sendbuf[200] = "This string is a test data from server";
int bytesRecv;
int idx = 0;
uint32_t nlength;
int length_received = recv(m_socket,(char*)&nlength, 4, 0);//Data length info
char *recvbuf = new char[nlength];//dynamic memory allocation based on data length info
//code to make sure all data has been received
while (nlength > 0)
{
bytesRecv = recv(m_socket, &recvbuf[idx], nlength, 0);
if (bytesRecv == SOCKET_ERROR)
{
std::cerr<<"recv() error: " << WSAGetLastError() <<std::endl;
break;
}
idx += bytesRecv;
nlength -= bytesRecv;
}
cout<<"Server: Received complete data is:"<< recvbuf<<std::endl;
cout<<"Server: Received bytes are"<<bytesRecv<<std::endl;
WSACleanup();
system("pause");
delete[] recvbuf;
return 0;
}
You send 200 bytes from the client, unconditionally, but in the server you only receive the actual length of the string, and that length does not include the string terminator.
So first of all you don't receive all data that was sent (which means you will fill up the system buffers), and then you don't terminate the string properly (which leads to "garbage" output when trying to print the string).
To fix this, in the client only send the actual length of the string (the value of varSize), and in the receiving server allocate one more character for the terminator, which you of course needs to add.
First question: I am confused between Buffers in TCP. I am trying to
explain my proble, i read this documentation TCP Buffer, author said a
lot about TCP Buffer, thats fine and a really good explanation for a
beginner. What i need to know is this TCP Buffer is same buffer with
the one we use in our basic client server program (Char
*buffer[Some_Size]) or its some different buffer hold by TCP internally ?
When you call send(), the TCP stack will copy some of the bytes out of your char array into an in-kernel buffer, and send() will return the number of bytes that it copied. The TCP stack will then handle the transmission of those in-kernel bytes to its destination across the network as quickly as it can. It's important to note that send()'s return value is not guaranteed to be the same as the number of bytes you specified in the length argument you passed to it; it could be less. It's also important to note that sends()'s return value does not imply that that many bytes have arrived at the receiving program; rather it only indicates the number of bytes that the kernel has accepted from you and will try to deliver.
Likewise, recv() merely copies some bytes from an in-kernel buffer to the array you specify, and then drops them from the in-kernel buffer. Again, the number of bytes copied may be less than the number you asked for, and generally will be different from the number of bytes passed by the sender on any particular call of send(). (E.g if the sender called send() and his send() returned 1000, that might result in you calling recv() twice and having recv() return 500 each time, or recv() might return 250 four times, or (1, 990, 9), or any other combination you can think of that eventually adds up to 1000)
My second question is that i am sending a string data with prefix
length (This is data From me) from client over socket to server, when
i print my data at console along with my string it prints some garbage
value also like this "This is data From me zzzzzz 1/2 1/2....." ?.
However i fixed it by right shifting char *recvbuf = new
char[nlength>>3]; nlength to 3 bits but why i need to it in this way ?
Like Joachim said, this happens because C strings depend on the presence of a NUL-terminator byte (i.e. a zero byte) to indicate their end. You are receiving strlen(sendbuf) bytes, and the value returned by strlen() does not include the NUL byte. When the receiver's string-printing routine tries to print the string, it keeps printing until if finds a NUL byte (by chance) somewhere later on in memory; in the meantime, you get to see all the random bytes that are in memory before that point. To fix the problem, either increase your sent-bytes counter to (strlen(sendbuf)+1), so that the NUL terminator byte gets received as well, or alternatively have your receiver manually place the NUL byte at the end of the string after it has received all of the bytes of the string. Either way is acceptable (the latter way might be slightly preferable as that way the receiver isn't depending on the sender to do the right thing).
Note that if your sender is going to always send 200 bytes rather than just the number of bytes in the string, then your receiver will need to always receive 200 bytes if it wants to receive more than one block; otherwise when it tries to receive the next block it will first get all the extra bytes (after the string) before it gets the next block's send-length field.
My third question is in relevance with first question if there is
nothing like TCP Buffer and its only about the Char *buffer[some_size]
then whats the difference my program will notice using such static
memory allocation buffer and by using dynamic memory allocation buffer
using char *recvbuf = new char[nlength];. In short which is best and
why ?
In terms of performance, it makes no difference at all. send() and receive() don't care a bit whether the pointers you pass to them point at the heap or the stack.
In terms of design, there are some tradeoffs: if you use new, there is a chance that you can leak memory if you don't always call delete[] when you're done with the buffer. (This can particularly happen when exceptions are thrown, or when error paths are taken). Placing the buffer on the stack, on the other hand, is guaranteed not to leak memory, but the amount of space available on the stack is finite so a really huge array could cause your program to run out of stack space and crash. In this case, a single 200-byte array on the stack is no problem, so that's what I would use.
I have a trouble, my server application sends packet 8 bytes length - AABBCC1122334455 but my application receives this packet in two parts AABBCC1122 and 334455, via "recv" function, how can i fix that?
Thanks!
To sum up a liitle bit:
TCP connection doesn't operate with packets or messages on the application level, you're dealing with stream of bytes. From this point of view it's similar to writing and reading from a file.
Both send and recv can send and receive less data than provided in the argument. You have to deal with it correctly (usually by applying proper loop around the call).
As you're dealing with streams, you have to find the way to convert it to meaningful data in your application. In other words, you have to design serialisation protocol.
From what you've already mentioned, you most probably want to send some kind of messages (well, it's usually what people do). The key thing is to discover the boundaries of messages properly. If your messages are of fixed size, you simply grab the same amount of data from the stream and translate it to your message; otherwise, you need a different approach:
If you can come up with a character which cannot exist in your message, it could be your delimiter. You can then read the stream until you reach the character and it'll be your message. If you transfer ASCII characters (strings) you can use zero as a separator.
If you transfer binary data (raw integers etc.), all characters can appear in your message, so nothing can act as a delimiter. Probably the most common approach in this case is to use fixed-size prefix containing size of your message. Size of this extra field depends on the max size of your message (you will be probably safe with 4 bytes, but if you know what is the maximum size, you can use lower values). Then your packet would look like SSSS|PPPPPPPPP... (stream of bytes), where S is the additional size field and P is your payload (the real message in your application, number of P bytes is determined by value of S). You know every packet starts with 4 special bytes (S bytes), so you can read them as an 32-bit integer. Once you know the size of the encapsulated message, you read all the P bytes. After you're done with one packet, you're ready to read another one from the socket.
Good news though, you can come up with something completely different. All you need to know is how to deserialise your message from a stream of bytes and how send/recv behave. Good luck!
EDIT:
Example of function receiving arbitrary number of bytes into array:
bool recv_full(int sock, char *buffer, size_t size)
{
size_t received = 0;
while (received < size)
{
ssize_t r = recv(sock, buffer + received, size - received, 0);
if (r <= 0) break;
received += r;
}
return received == size;
}
And example of receiving packet with 2-byte prefix defining size of payload (size of payload is then limited to 65kB):
uint16_t msgSize = 0;
char msg[0xffff];
if (recv_full(sock, reinterpret_cast<char *>(&msgSize), sizeof(msgSize)) &&
recv_full(sock, msg, msgSize))
{
// Got the message in msg array
}
else
{
// Something bad happened to the connection
}
That's just how recv() works on most platforms. You have to check the number of bytes you receive and continue calling it in a loop until you get the number that you need.
You "fix" that by reading from TCP socket in a loop until you get enough bytes to make sense to your application.
my server application sends packet 8 bytes length
Not really. Your server sends 8 individual bytes, not a packet 8 bytes long. TCP data is sent over a byte stream, not a packet stream. TCP neither respects nor maintains any "packet" boundary that you might have in mind.
If you know that your data is provided in quanta of N bytes, then call recv in a loop:
std::vector<char> read_packet(int N) {
std::vector buffer(N);
int total = 0, count;
while ( total < N && (count = recv(sock_fd, &buffer[N], N-total, 0)) > 0 )
total += count;
return buffer;
}
std::vector<char> packet = read_packet(8);
If your packet is variable length, try sending it before the data itself:
int read_int() {
std::vector<char> buffer = read_packet(sizeof (int));
int result;
memcpy((void*)&result, (void*)&buffer[0], sizeof(int));
return result;
}
int length = read_int();
std::vector<char> data = read_buffer(length);