It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.
Closed 10 years ago.
I'm having some strange things happen with my program, and I'm not sure what I should be doing. This is a pseudocode version of what my code looks like so far:
Server:
//Set up Server sockets
int maximum;
// Collect the maximum
cout << "\nEnter a Maximum:";
cin >> maximum;
cout << "\n";
int *array = new int[maximum + 1];
memset(array, 0, sizeof(array));
while(array[0] < anInt){
//receive the array from the client
if(recv(newsockfd, array, maximum, 0) < 0){
perror("ERROR receiving from socket");
}
mathFunction(array); //A function that alters the contents of array
array[0]++;
//If array[0] isn't too big
if(array[0] < anInt){
// Send the array to the client
if(send(newsockfd, array, maximum, 0) < 0){
perror("ERROR sending to socket");
}
}
}
Client:
//Set up Client sockets
//The maximum was already sent over earlier
int *array = new int[maximum + 1];
while(array[0] < anInt){
//receive the array from the server
if(recv(sockfd, array, maximum, 0) < 0){
perror("ERROR receiving from socket");
}
mathFunction(array); //A function that alters the contents of array
array[0]++;
if(send(sockfd, array, maximum, 0) < 0){
perror("ERROR sending to socket");
}
}
My problem is that I keep getting a "Connection reset by peer" error, which leads to a Segmentation Fault, crashing my program. Also, when playing around with the 3rd argument of the send/recv functions (currently set as maximum), my program acts differently. It will actually work perfectly if the user enters a maximum of 100, but anything more than that screws it up.
I know this is a long shot, but can anyone see something that I'm doing wrong?
First of all code that posted by you has a logical error:
Server first receive data from the client, do something with it and then send its result back to the client.
In the other side client also receive data from server, do something with it and then send it back to the server.
And that's obviously a race condition, no one send data to other side to begin communication.
Beside that logical error you have some C++ errors:
1) memset(array, 0, sizeof(array)) only 0 initialize sizeof(int*) bytes from your array not entire array, since sizeof(array) is always sizeof(int*) if you want to 0 initialize entire array (and I think you want it) you should call:
memset(array, 0, (maximum + 1) * sizeof(int));
or even better:
std::fill( array, array + maximum + 1, 0 );
And in C++ it is much better to use classes like std::vector instead of raw pointers:
std::vector<int> array( maximum + 1 ); // automatically initialize to 0
2) Your array type is int* and send/recv count its input by byte, so if you want to send/recv entire array you must have something like:
send(sockfd, (char*)array, maximum * sizeof(int), 0);
3) You should check return value of send/recv, specially recv since it may recv less data in each call, for example you send 8K data and recv only receive first 1K and rest of it remain in the network buffer, so you should call it repeatedly until you read your buffer completely.
One thing that seems obviously incorrect is:
mathFunction(array);
doesn't tell mathFunction() how many elements are in the array. In fact, you throw away this information when you call recv() by not storing it anywhere (all your code does is check to see if it's less than zero, but doesn't use it if it is positive). When calling recv(), your code must be prepared to receive any number of bytes from 1 through maximum. If you don't get all the bytes you ask for, then you need to call recv() again to get more.
Related
I am trying to send data from a vector over a TCP socket.
I'm working with a vector that I fill with values from 0 to 4999, and then send it to the socket.
Client side, I'm receiving the data into a vector, then I copy its data to another vector until I received all the data from the server.
The issue I'm facing is that when I receive my data, sometimes I will get all of it, and sometimes I will only receive the correct data from 0 to 1625 and then I get trash data until the end (please see the image below). I even received for example from 0 to 2600 correct data, then from 2601 to 3500 it's trash and finally from 3501 to 4999 it's correct again.
(left column is line number and right column is the data).
This is the server side :
vector<double> values2;
for(int i=0; i<5000; i++)
values2.push_back(i);
skt.sendmsg(&values2[0], values2.size()*sizeof(double));
The function sendmsg :
void Socket::sendmsg(const void *buf, size_t len){
int bytes=-1;
bytes = send(m_csock, buf, len, MSG_CONFIRM);
cout << "Bytes sent: " << bytes << endl;
}
Client side :
vector<double> final;
vector<double> msgrcvd(4096);
do{
bytes += recv(sock, &msgrcvd[0], msgrcvd.size()*sizeof(double), 0);
cout << "Bytes received: " << bytes << endl;
//Get rid of the trailing zeros
while(!msgrcvd.empty() && msgrcvd[msgrcvd.size() - 1] == 0){
msgrcvd.pop_back();
}
//Insert buffer content into final vector
final.insert(final.end(), msgrcvd.begin(), msgrcvd.end());
}while(bytes < sizeof(double)*5000);
//Write the received data in a txt file
for(int i=0; i<final.size(); i++)
myfile << final[i] << endl;
myfile.close();
The outputs of the bytes are correct, the server outputs 40 000 when sending the data and the client also outputs 40 000 when receiving the data.
Removing the trailing zeros and then inserting the content of the buffer into a new vector is not very efficient, but I don't think it's the issue. If you have any clues on how to make it more efficient, it would be great!
I don't really know if the issue is when I send the data or when I receive it, and also I don't really get why sometimes (rarely), I get all the data.
recv receives bytes, and doesn't necessarily wait for all the data that was sent. So you can be receiving part of a double.
Your code works if you receive complete double values, but will fail when you receive part of a value. You should receive your data in a char buffer, then unpack it into doubles. (Possibly converting endianness if the server and client are different.)
#include <cstring> // For memcpy
std::array<char, 1024> msgbuf;
double d;
char data[sizeof(double)];
int carryover = 0;
do {
int b = recv(sock, &msgbuf[carryover], msgbuf.size() * sizeof(msgbuf[0]) - carryover, 0);
bytes += b;
b += carryover;
const char *mp = &msgbuf[0];
while (b >= sizeof(double)) {
char *bp = data;
for (int i = 0; i < sizeof(double); ++i) {
*bp++ = *mp++;
}
std::memcpy(&d, data, sizeof(double));
final.push_back(d);
b -= sizeof(double);
}
carryover = b % sizeof(double);
// Take care of the extra bytes. Copy them down to the start of the buffer
for (int j = 0; j < carryover; ++j) {
msgbuf[j] = *mp++;
}
} while (bytes < sizeof(double) * 5000);
This uses type punning from What's a proper way of type-punning a float to an int and vice-versa? to convert the received binary data to a double, and assumes the endianness of the client and server are the same.
Incidentally, how does the receiver know how many values it is receiving? You have a mix of hard coded values (5000) and dynamic values (.size()) in your server code.
Note: code not compiled or tested
TL/DR:
Never-ever send raw data via a network socket and expect them properly received/unpacked on other side.
Detailed answer:
Network is built on top of various protocols, and this is for a reason. Once you send something, there is no warranty you counterparty is on the same OS and same software version. There is no standard how primitive types should be coded on byte level. There is no restriction how much intermittent nodes could be involved into the data delivery, and each of your send() may traverse via different routes. So, you have to formalize the way you send the data, then other party can be sure what is proper way to retrieve them from the socket.
Simplest solution: use a header before your data. So, you plan to send 5000 doubles? Then send a DWORD first, which contains 40000 inside (5k elements, 8 bytes each -> 40k) and push all your 5k doubles right after that. Then, your counterparty should read 4 bytes from the socket first, interpret it as DWORD and understand how much bytes should come then.
Next step: you may want to send not only doubles, but ints and strings as well. That way, you have to expand your header so it can indicate
Total size of further data (so called payload size)
Kind of the data (array of doubles, string, single int etc)
Advanced solution:
Take a look on ready-to-go solutions:
ProtoBuf https://developers.google.com/protocol-buffers/docs/cpptutorial
Boost.Serialization https://www.boost.org/doc/libs/1_67_0/libs/serialization/doc/index.html
Apache Thrift https://thrift.apache.org
YAS https://github.com/niXman/yas
Happy coding!
Hie, everyone! I have a simple TCP server and client on winsock2 lib c++. The server simply send string messages. The client simply receives them. Everything is fine here. But when I use the zlib library to compress the string, the data is corrupting and I can't properly receive them on the client to unzip. Can someone help me?
Server:
{
std::lock_guard<std::mutex> lock(mtx);
std::cout << "Client connected\n";
int k = rand() % strings.size();
msg = strings[k];
msg_size = msg.size();
msgl_size = msg_size + msg_size*0.1 + 12;
msgl = new unsigned char[msgl_size + 1]{0};
if (Z_OK != compress((Bytef*)msgl,
&msgl_size,
reinterpret_cast<const unsigned char*>(msg.c_str()),
msg.size()))
{
std::cout << "Compression error! " << std::endl;
exit(2);
}
}
std::thread * thread = new std::thread([&newConnection, msgl, msgl_size, msg_size, msg]() {
std::lock_guard<std::mutex> lock(mtx);
send(newConnection, (char*)&msgl_size, sizeof(unsigned long), NULL);
send(newConnection, (char*)&msg_size, sizeof(unsigned long), NULL);
int res;
do {
res = send(newConnection, (char*)(msgl), sizeof(msgl_size), NULL);
}
while (msgl_size != res);
});
Client:
std::lock_guard<std::mutex> lock(mtxx);
unsigned long msgl_size, msg_size;
recv(Connection, (char*)&msg_size, sizeof(unsigned long), NULL);
recv(Connection, (char*)&msgl_size, sizeof(unsigned long), NULL);
unsigned char * msgl = new unsigned char[msgl_size + 1]{0};
int res;
do {
res = recv(Connection, reinterpret_cast<char*>(msgl), msgl_size, NULL);
}
while (msgl_size != res);
char * msg = new char[msg_size + 1];
if (Z_OK == uncompress(reinterpret_cast<unsigned char*>(msg),
&msg_size,
reinterpret_cast<unsigned char*>(msgl),
msgl_size))
{
msg[msg_size] = '\0';
std::cout << msg << std::endl;
std::cout << "Compress ratio: " << msgl_size / (float)msg_size << std::endl;
}
delete[] msgl;
Client side:
recv only returns whatever data is immediately available or blocks until data becomes available, this is unlikely to happen with a large file or a slow network. Quite likely recv will block until the first network packet arrives and depending on the underlying network that could be anywhere from a few hundred bytes to tens of thousands. Maybe the message fits in that and maybe not.
Setting recv's flags parameter to MSG_WAITALL is useful for shorter messages because you will either get exactly the number of bytes what you asked for or an error. Because of the possibility for error you always have to test the return value.
To repeat: Always check the return value.
recv's return value is either negative for socket failure, 0 for socket shutdown, or the number of bytes read. For more, consult the winsock documentation for recv.
So...
recv(Connection, (char*)&msg_size, sizeof(unsigned long), NULL);
and
recv(Connection, (char*)&msgl_size, sizeof(unsigned long), NULL);
do not check the return value. The socket could have failed or the call to recv could have returned less than what was requested and the remainder of the program will be operating on garbage.
These are a decent place to use MSG_WAITALL, but it's possible that the socket is fine and you were interrupted by a signal. Not sure if this can happen on Windows, but it can on Linux. Beware.
if (recv(Connection, (char*)&msg_size, sizeof(unsigned long), MSG_WAITALL) != sizeof(unsigned long) &&
recv(Connection, (char*)&msgl_size, sizeof(unsigned long), NULL) != sizeof(unsigned long)(
{
// log error
// exit function, loop, or whatever.
}
Next,
do {
res = recv(Connection, reinterpret_cast<char*>(msgl), msgl_size, NULL);
} while (msgl_size != res);
will loop until one recv returns exactly the right amount in a single call. Unlikely, but if it does, it must happen on the first read because the code writes over the previous read every time.
Say only 1/2 of the message is read from the socket on the first try. Since this isn't the full message, the loop enters and tries to read again, overwriting the first half of the message with the second half and perhaps enough bytes from the subsequent message to satisfy the requested number of bytes. This amalgam of two messages will not decrypt.
For a payload of potentially great size, loop until the program has it all.
char * bufp = reinterpret_cast<char*>(msgl);
int msg_remaining = msgl_size;
while (msg_remaining )
{
res = recv(Connection, bufp, msg_remaining, NULL);
if (res <= 0)
{
// log error
// exit function, loop, or whatever.
}
msg_remaining -= res; // reduce message remaining
bufp += res; // move next insert point in msgl
}
There may be problems with the decompression. I don't know enough about that to be able to answer. I suggest removing it and sending easily-debuggable plaintext until you have all of the network issues worked out.
Server side:
Like recv, send sends what it can. You may have to loop sending to make sure you didn't overfill the socket with a message too large for the socket to eat in one shot. And again like recv, ssend can fail. Always check the return value to see what really happened. Check the documentation for send for more information.
It looks to me like you have the right basic idea: send the size of data to expect, followed by the data itself. On the receiving side, read the size first, then read the specified amount of data.
Unfortunately, you've made a mistake or two when it came to the details of implementing that intent. The first big one is when you send the data:
do {
res = send(newConnection, (char*)(msgl), sizeof(msgl_size), NULL);
}
while (msgl_size != res);
This has a couple of problems. First of all, it uses sizeof(msg1_size), so it's only trying to send the size of an unsigned long (at least I'm guessing that msg1_size is an unsigned long).
What I'm pretty sure you intended here was to send the entire buffer instead:
unsigned long sent = 0;
unsigned long remaining = msg1_size;
do {
res = send(newConnection, (char*)(msgl + sent), remaining, NULL);
sent += res;
remaining -= res;
} while (msgl_size != sent);
With this, we start sending from the beginning of the buffer. If send returns after sending only part of that (as it's allowed to), we record how much was sent. Then on the next iteration, we re-start sending from the point where it left off. Meanwhile, we keep track of how much remains to be sent, and only attempt to send that much on each subsequent iteration.
At least at first glance, it looks like your receive loop probably needs roughly the same kind of repair, keeping track of the total received rather than trying to wait for a single transfer of the entire amount.
Oh, and of course for real code you also want to check for res being 0 or negative. As it stands right now, this doesn't even attempt to detect or properly react to most network errors.
As a test, I'm writing a series of byte arrays to a tcp socket from an Android application, and reading them in a C++ application.
Java
InetAddress address = InetAddress.getByName("192.168.0.2");
Socket socket = new Socket(address, 1300);
DataOutputStream out = new DataOutputStream(socket.getOutputStream())
...
if(count == 0) {
out.write(first, 0, first.length);
} else if(count == 1) {
out.write(second, 0, second.length);
}
C++
do {
iResult = recv(ClientSocket, recvbuf, 3, 0);
for (int i = 0; i < 3; i++) {
std::cout << (int)(signed char)recvbuf[i] << std::endl;
}
} while (iResult > 0);
As it stands, on the first receipt, recv[2] = -52, which I assume to be a junk value, as the output stream has not yet written the second byte array by the time I've received the first segment.
However, when I pause after the the ListenSocket has accepted the connection:
ClientSocket = accept(ListenSocket, NULL, NULL);
std::cin.ignore();
...giving the sender time to do both writes to the stream, recv[2] = 3, which is the first value of the second written byte array.
If I ultimately want to send and receive a constant stream of discrete arrays, how can I determine after I've received the last value of one array, whether the next value in the buffer is the first value of the next array or whether it's a junk value?
I've considered that udp is more suitable for sending a series of discrete data sets, but I need the reliability of tcp. I imagine that tcp is used in this way regularly, but it's not clear to me how to mitigate this issue.
EDIT:
In the actual application for which I'm writing this test, I do implement length prefixing. I don't think that's relevant though; even if I know I'm at the end of a data set, I need to know whether the next value on the buffer is junk or the beginning of the next set.
for (int i = 0; i < 3; i++)
The problem is here. It should be:
for (int i = 0; i < iResult; i++)
You're printing out data that you may not have received. This is the explanation of the 'junk value'.
You can't assume that recv() fills the buffer.
You must also check iResult for both -1 and zero before this loop, and take the appropriate actions, which are different in each case.
As you point out, TCP is stream-based, so there's no built-in way to say "here's a specific chunk of data". What you want to do is add your own "message framing". A simple way to do that is called "length prefixing". Where you first send the size of the data packet, and then the packet itself. Then the receiver will know when they've gotten all the data.
Sending side
send length of packet (as a known size -- say a 32-bit int)
send packet data
Receiving side
read length of packet
read that many bytes of data
process fully-received packet
Check out this article for more information: http://blog.stephencleary.com/2009/04/message-framing.html
So this is the first time I'm actually asking a question in here, although I have been using this site for ages!
My problem is a bit tricky. I'm trying to develop a client server application for sending large files, using UDP with my own error checking and flow control. Now, I've developed a fully-functioning server and client. Client requests for a specific file, server starts sending. The file is read in parts into a buffer to avoid having to read small bits of the file every time a packet is send, thus saving processing time. Packets consist of 1400 bytes of actual data + a header of 28 bytes (sequence numbers, ack numbers, checksum etc..).
So I had the basics down, a simple stop-and-wait protocol. Send packet and receive ack, before sending next packet.
To be able to implement a smarter flow control algorithm, for starters with just some windowing, I have to run the sending-part and receiving-ack part in two different threads. Now here's where I got into problems. This is my first time working with threads, so please bear with me.
My problem is that the file written from the packets on the client side is corrupt. Well, when testing with a small jpg file, the file is only corrupt 50% of the times, when testing with a MP4 file, it's always corrupt! So I guess maybe the thread somehow rearranges the order in which the packets are send? I use sequence numbers, so the problem must occur before assigning the sequence number to the packets...
I know for sure that the part where I split up the file is correct, and also where I reassemble it on the client side, since I have tested this before trying to implement the threading. It should also be noted that I copied the exact sending-part of the code into the sending-thread, and this also worked perfectly before putting it into a thread.. This is also why I'm just posting the threading-part of my code, since this is clearly what is creating the problem (and since the entire code of the project would take up a loooot of space)
My sending thread code:
pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
pthread_cond_t condition_var = PTHREAD_COND_INITIALIZER;
static void *send_thread(void *){
if (file.is_open()) {
while(!file.reachedEnd()){
pthread_mutex_lock(& mutex);
if(seq <= upperwindow) {
int blocksize = file.getNextBlocksize();
senddata = new unsigned char[blocksize + 28];
Packet to_send;
to_send.data = new char[blocksize];
to_send.sequenceNumber = seq;
to_send.ackNumber = 0;
to_send.type = 55; // DATA
file.readBlock(*to_send.data);
createPacket(senddata, to_send, blocksize + 28);
if (server.sendToClient(reinterpret_cast<char*>(senddata), blocksize + 28) == -1)
perror("sending failed");
incrementSequenceNumber(seq);
/* free memory */
delete [] to_send.data;
delete [] senddata;
}
pthread_mutex_unlock(& mutex);
}
pthread_exit(NULL);
} else {
perror("file opening failed!");
pthread_exit(NULL);
}
}
My receiving ack thread code:
static void *wait_for_ack_thread(void *){
while(!file.reachedEnd()){
Packet ack;
if (server.receiveFromClient(reinterpret_cast<char*>(receivedata), 28) == -1) {
perror("error receiving ack");
} else {
getPacket(receivedata, ack, 28);
pthread_mutex_lock(& mutex);
incrementSequenceNumber(upperwindow);
pthread_mutex_unlock(& mutex)
}
}
pthread_exit(NULL);
}
All comments are very much appreciated! :)
EDIT:
Added code of the readBlock function:
void readBlock(char & in){
memcpy(& in, buffer + block_position, blocksize);
block_position = block_position + blocksize;
if(block_position == buffersize){
buf_position ++;
if(buf_position == buf_reads){
buffersize = filesize % buffersize;
}
fillBuffer();
block_position = 0;
}
if(blocksize < MAX_DATA_SIZE){
reached_end = true;
return;
}
if((buffersize - block_position) < MAX_DATA_SIZE){
blocksize = buffersize % blocksize;
}
}
Create an array that represents the status of the communication.
0 means unsent, or sent and receiver reported error. 1 means sending. 2 means sent, and ack gotten.
Allocate this array, and guard access to it with a mutex.
The sending thread keeps two pointers into the array -- "has been sent up to" and "should sent next". These are owned by the sending thread.
The ack thread simply gets ack packets, locks the array, and does the transition on the state.
The sending thread locks the array, checks if it can advance the "has been sent up to" pointer (or if it should resend old stuff). If it notices an error, it reduces the "should be sent next" pointer to point at it.
It then sees if it should send stuff next. If it should, it marks the node as "being sent", unlocks the array, and sends it.
If the sending thread did no work, and found nothing to do, it goes to sleep on a timeout, and possibly a "kick awake" by the ack thread.
Now, note that the client can get the packets sent by this in the wrong order, unless you limit it to having 1 packet in transit.
The connection status array does not have to be a literal array, but it is easier if you start with that and optimize later.
On the receiving end, you have to pay attention to the sequence number, as the packets can get there out of sequence. To test this, write a server that sends the packets in the wrong order on purpose, and ensure that the client manages to stitch it together properly.
I am writing a server client program that needs to communicate with each other. I have records stored in memory and I want to send all those records to the client. Right now I first send the number or records to the clients then I am trying to use a while loop to send each record but I am only receiving the first record on the client side.
The client code is first:
enter code here
while(i < j){
recv(sockfd,infobuf,BUFFERSIZE-1,0);//records from mem
printf("\nrecords revieved %s",infobuf);
memset(infobuf,'\0',BUFFERSIZE);
i++;
}
The server code is below:
//send number of rec to client
printf("count as j %s", recordList);
send(reply_sock1_fd,recordList,strlen(recordList),0);
memset(recordList,'\0',BUFFERSIZE);
printf("\n count %d:", count);
int i = 0;
//look and send records to client
while(i < count){
printf("\ninside loopi (%d)", i);
strcat(strcat(strcat(strcat(strcpy(recordList,bufRec[i].first), " "),bufRec[i].last), " "),bufRec[i].userid);
printf("recordlist %s", recordList);
send(reply_sock1_fd,recordList,BUFFERSIZE,0);
memset(recordList,'\0',BUFFERSIZE);
i++;
}
Not quite sure what your problem is but you are not checking the return code from the recv() call. This should either be a positive number indicating the number of bytes actually received , a negative number indicating an error, or 0 indicating a dropped connection.
Note, it is quite common to get less data than you asked for have a look at the tutorial here for how to handle this situation.