I'm sending some data trough the socket, but I need to set the timeout.
I'm using something like:
fd_set rfds;
struct timeval tv;
int retval;
/* Watch stdin (fd 0) to see when it has input. */
FD_ZERO(&rfds);
FD_SET(sockDesc, &rfds);
/* Wait up to five seconds. */
tv.tv_sec = 5;
tv.tv_usec = 0;
retval = select(sockDesc + 1, &rfds, NULL, NULL, &tv);
if(rtn = ::recv(sockDesc, (raw_type*) buffer, bufferLen, 0)) < 0){
throw SocketException("error", true);
}
return rtn;
So.
Depending the type of data I return I need to implement timeout or not.
If I just send text data I don't need, if I send one file I need...
To explain more or less I'm sending some data through a socket and processing on the other side.
So, if I send one tar:
while(readtar){
senddata
get processed data
}
but sometimes the data sent is just the header, so when the other side process data it doesn't needs to return data and the socket stops on read.
To illustrate:
cat file.tar | myprogram -c "tar -zvt"
So it don't return nothing until receive enough data to return the name of file.
If i just send one file and returns the "cat" i dont have this problem
echo "asjdoiajdlaijdkasdjlkas" | myprogram -c "cat"
or
cat HUGEFILE.tar | myprogram -c "cat" | tar -zvt
In this case it does the same thing, but is not on the server side... so it dont work for me.
Now.... If I just use the recv without the select when I return the data using the cat it works... no problems with that.
BUT if I implement the select the data comes messed up.
WITHOUT SELECT
send "command line text temp test"
recv "command line text temp test"
WITH
send "command line text temp test"
recv "commmand lin/ˆ
1k5d99ck"
it's just to illustrate what is happening
Client loop:
while(size = read(fileno(stdin), thebuffer, 10000)){
sock->send(thebuffer, size); // if this data is not enough the other side never sends the data back
sock->recv //receive data
}
On the other side I do
if(pid == 0){
//stuffs closes, dup2
execlp("bash", "bash", "-c", "run.c_str(), NULL); // if i use one tar -zvt i need a bunch of data to generate the return
}
else
while(size = read(fout[0], buffer, 10000) > 0)){
sock->send(buffer, size);
}
So.. if the data sent is not enough to generate on
If I could check if read have anything or on the stdin side if the execlp send a terminator I could solve the problem
It sounds like you are expecting your TCP recv() to return data to you in chunks of a particular size... however, TCP recv() does not work that way. TCP is stream-based, so the number of bytes returned by recv() may vary anywhere between 1 (or 0 if you are using non-blocking I/O) and the size of the buffer you passed in. It's then up to your receiving code to loop as necessary to re-concatenate the received data again.
Also, it looks like you are trying to print out un-terminated ASCII strings -- that would explain the garbage characters at the end of your second recv() example. (i.e. if you want to print out the received data bytes as a string, be sure to place a NUL/0 byte after the last received byte; recv() won't do that for you)
Related
I have this function that takes a SOCKET and a file name to the output file, then it tries to receive and output the data to a file, when compiling and running the code it works fine but the program does not exit and the output file (file.txt) contains some unreadable garbage text
the rest of the code work fine and tested, I am sure the problem is in this file but can't tell what and where
int recv_file(SOCKET soc, const char * fname)
{
FILE * ptr;
char buffer[MAX];
unsigned long long int x = 0;
if (fopen_s(&ptr, fname , "wb") != 0)
{
printf("Failed to receive file \n");
return 1;
}
do
{
c = recv(soc, buffer, MAX, 0);
printf("recv count : %d\n",c);
// here was -> fwrite(buffer, 1, sizeof(buffer), ptr);
fwrite(buffer, 1, c, ptr);
}
while ( c > 0);
printf("Total received : %lld\n", x);
fclose(ptr);
return 0;
}
as an output it prints only one line recv count : 65
I tried to use closesocket(soc);WSACleanup(); but this does not solve the problem
for the server, I tested with netcat and python socketserver and had same behavior for both
also the last printf right after the do-while does not get printed to the screen
Data from network can come in chuncks
recv returns negative value on errors that what break handles (stops loop).
recv returns number of bytes received.
fwrite should write number of bytes equal to those which has been received, so it should be:
fwrite(buffer, 1, c, ptr);
Passing there sizeof(buff) will record trash in a file (content form previous read or memory trash).
Now question is what sender does when there is no data. If nothing recv will block waiting for new data. If sender closes tcp connection then respective error should be reported by recv and loop should be ended.
If you are fetching data form some HTTP server then most probably server assumes that client will close connection. Server just waits for next request.
Note also that depending on SOCKET configuration recv can be blocking or not. So depending on that recv may return 0 on success when in fact there will be new data in future.
I am trying to write reverse proxy with nonblocking socket and epoll. That seems ok at first, but when I tried to open a big jpg file, I got stuck.
When I try to write into client sometimes It may not writable and how can I handle proper way.
Additional Notes:
this->getFd() = ProxyFd
this->clientHandler->getFd = clientFd
I am using EPOLLET flag both proxy and client
if( (flag & EPOLLIN) ){
char buffer[1025] = {'\0'};
int readSize;
while( (readSize = read(this->getFd(),buffer,1024)) > 0){
this->headerParse(buffer);
this->readSize += readSize;
int check = 0;
do{
check = write(this->clientHandler->getFd(),buffer,readSize);
}while(check < 0);
}
if(this->headerEnd == 1 && this->readSize >= this->headerLenght ){
close(this->clientHandler->getFd());
close(this->getFd());
delete this->clientHandler;
delete this;
}
}
Thanks for taking time to read.
Assuming your headerParse() method doesn't change buffer in a size-extending way (you'd need to update readsize, at least, not to mention the buffer full scenario), it seems like your write() path is broken.
if the socket you're writing to is also in nonblocking mode, it's perfectly legal for write() to return -1 (and set errno to EGAIN or EWOULDBLOCK or whatever your platform has) before you wrote all data.
In that case, you must store the remaining data (the remainder of buffer minus what was written if one or more calls to write() succeeded), program epoll to notify the clientHandler->getFd() descriptor for writeability, if not already, and when you get subsequent "write ready" event, you write the data you stored. On this case, the write() can again be unable to flush all your data, so you must cycle until all data is sent.
I'm writing a C++ program. I need to receive a file and I'm using recv() function over a TCP socket to do that.
download_file() {
while (left_bytes != 0 && !connection_closed) {
if (left_bytes >= buffer_max_size)
bytes_to_download = buffer_max_size;
else
bytes_to_download = left_bytes;
if (request.conn->read_data(buffer, bytes_to_download))
{
left_bytes -= buffer->get_size();
temporary_file.write_data(buffer);
} else connection_closed = true;
}
}
read_data() {
while (bytes_received < size && alive_) {
bytes_read = recv(sock_, read_buffer, size, 0);
if (bytes_read == SOCKET_ERROR) {
delete[] local_buffer;
throw SocketException(WSAGetLastError());
}
// the connection is closed
if (bytes_read == 0) alive_ = false;
else {
bytes_received += bytes_read;
buffer->add(local_buffer, bytes_read);
}
}
}
The problem is that the recv never returns. It receives the whole file except for few KB and it freeze on the recv(). The buffer size is 1460.
I receive the file only if I print something to the console with cout every time the recv is called. Only in this case I receive the whole file.
Otherwise if I set as socket option the WAITALL and the client closes the connection after the file is sent, I receive the whole file.
Here's the code for the Client side that sends the file:
TransmitFile(file_request->connection_->get_handle_socket(), file_handler.get_file_handle(), file_request->file_size_, 65535, nullptr, nullptr, TF_USE_SYSTEM_THREAD)
EDIT
Here's how I send and read the file size between the Client and Server.
std::stringstream stream_;
stream_.str(std::string());
// append the file size
const __int64 file_size = htonll(GetFileSize(file_handle_, nullptr););
stream_ << ' ' << file_size << ' ';
Then I use the send to send this string
Here's how I read the file size
// Within stream_ there is all the content of the received packet
std::string message;
std::getline(stream_, message, ' ');
this->request_body_.file_size_ = ntohll(strtoll(message.c_str(), nullptr, 0));
EDIT
I cleaned up the code and I found out that read_data() is obviously called once and I was updating the buffer variable wrongly. Hence I was tracking the size of the content within the buffer in a wrong way which make me call the recv() once more.
First thing: recv() will block if there are no bytes left to read but the connection is still open. So whatever you might say about what your code is doing, that must be what is happening here.
That could be for any of the following reasons:
the sender lied about the size of the file, or did not send the promised number of bytes
the file size was not interpreted correctly at the receiving end for whatever reason
the logic that 'counts down' the number of bytes left in the receiver is somehow flawed
Trouble is, looking at the code samples you have posted, it's hard to say which because the code is a bit muddled and, in my eyes, more complicated than it needs to be. I'm going to recommend you sort that out.
Sending the size of the file.
Don't mess about sending this as a string. Send it instead in binary, using (say) htonll() at the sending end and ntohll() at the receiving end. Then, the receiver knows to read exactly 8 bytes to figure out what's coming next. It's hard to get that wrong.
Sending the file itself.
TransmitFile() looks to be a good choice here. Stick with it.
Receiving the file and counting down how many bytes are left.
Take a closer look at that code and consider rewriting it. It's a bit of a mess.
What to do if it still doesn't work.
Check with WireShark that the expected data is being sent and then walk through the code in the receiver in the debugger. There is absolutely no excuse for not doing this unless you don't have a debugger for some reason, in which case please say so and somebody will try to help you. The fact that logging to cout fixes your problems is a red-herring. That just changes the timing and then it just happens to work right.
That's all. Best of luck.
I'm writing a UDP client that sends a string to a server, when the server sends back several packets, the behavior of the program is not as my expectation. I want to process any incoming packet by process() one by one until the entry buffer gets empty, but I think there is a problem related to blocking behavior of recv.
#include <cstdlib>
#include <cstdio>
#include <cstring>
#include <winsock.h>
using namespace std;
void process(const char *in, int size)
{
fprintf(stdout, "%s\n", in);
}
int main()
{
char quack_addr[] = "127.0.0.1";
unsigned short quack_port = 9091;
WSAData data;
WSAStartup(MAKEWORD(2, 2), &data);
sockaddr_in qserver;
qserver.sin_family = AF_INET;
qserver.sin_addr.s_addr = inet_addr(quack_addr);
qserver.sin_port = htons(quack_port);
SOCKET client = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP);
if (client <= 0)
{
fprintf(stderr, "Error - Can not create socket.\n");
exit(1);
}
while (true)
{
const int MAX = 1024;
char sbuf[MAX];
char rbuf[MAX];
fprintf(stdout, ": ");
fgets(sbuf, MAX, stdin);
int slen = strlen(sbuf);
int r = sendto(client,sbuf,slen,0,(sockaddr*)&qserver,sizeof(qserver));
// Current code:
// int rlen = recv(client, rbuf, MAX, 0);
// if (rlen > 0)
// {
// rbuf[rlen] = 0;
// process(rbuf, rlen);
// }
// Question starts here:
//
// While there is data in queue do:
// {
// (break if there is no data)
// int rlen = recv(client, rbuf, MAX, 0);
// rbuf[rlen] = 0;
// process(rbuf, rlen);
// }
}
return 0;
}
How can I check if the buffer is empty or not, before calling recv(...) ?
The problem occurs in this scenario:
User is typing a command in the client program (cmd1).
Simultaneously, the server sends 3 packets to client (pkt1, pkt2, pkt3).
After pressing Enter in the client side, I expect to receive those 3 packets and probably the result corresponding to cmd1, and process() all of them one by one.
But after pressing Enter in stage 3, I receive pkt1! and after sending another command to the server I will receive pkt2 and so on ...!
I know my code is not enough to handle this issue, so, my question is how to handle it?
Note: I'm using netcat -L -p 9091 -u as UDP server
I think the problems (unsatisfying behavior you do not describe) come from a different source. Let me just list some ideas and comments c./ what was said before:
(1) recvfrom() blocks too. However, you want to use it. Your communication currently sends and receives from loopback, which is fine for your toy program (but: see below). When receiving UDP data, with recv() you don't know who sent it, as the socket was never connect()ed. Use recvfrom() to prepare yourself for some minimal error checking in a more serious program
(2) as select() suspends the program to i/o availibity, it would only put any issue with your socket blocking to a different level. But this is not the problem
(3) to check is the receive buffer is empty, use flag MSG_PEEK in recvfrom() in an appropriate position. It's usually only used to deal with scarce memory, but it should do the job.
(4) reason 1 why I believe you see the issues you don't describe in more detail:
UDP datagrams preserve message boundaries. This means that recvfrom() will read in an entire chunk of data making up any message sent. However, if the buffer you read this into is smaller than the data read, any surplus will be silently discarded. So make sure you have a big buffer (65k something ideally).
(5) reason 2:
You receive any data sent to the loopback. If you're currently also connected to some net (sat, the Internet), what you catch might actually be from a different source than you expect. So at least in a resting phase, disconnect.
Blocking shouldn't be an issue. Your basic logic, when coded cleanly, is:
Recvfrom() (block/wait until ready)
Process
Peek if buffer empty
Exit if yes
Loop back to receive more if not,
and you seem to want to do this currently. As you don't multi-thread, optimize fie perfiormance, or similar, you shouldn't care about blocking. If you find your receive buffer too small, increase its size using
Setsockopt() for optName SO_RCVBUF
Use select() (with a suitable timeout) to check for incoming data prior to calling recv().
Something like the following (nonportable code)
#include <winsock2.h>
...
/* somewhere after your sendto, or your first recv */
fd_set recv_set;
timeval tv = {1, 0}; /* one second */
FD_ZERO(&recv_set);
FD_SET(client, &recv_set);
while (select(0, &recv_set, NULL, NULL, &tv) > 0)
{
/* recv... */
FD_SET(client, &recv_set); /* actually redundant, since it is already set */
}
iPhone sometimes bugs out and doesn't let me post comments. Thanks, Steve. This is just continuing the conversation.
I assume this means 'uncomment to 'question starts here'. Partial answer, as this still depends on my 2nd comment; this is more or less than what to expect. Assuming the three messages to be sent by the server are already queued up, after you hit enter for the first time, your packet is sent (never blocked as sendto() doesn't block for UDP), received by the server and (I assume, see above, echoed back and added to the FIFO receive buffer in which you alread have three messages queued up. You then have a recv() in your program which receives the first queued up message, printing it out. Your current logic goes back to top of loop, expect another input and waits for it (so this isn't blocked on a socket level, but as your program requests input, eg simply 'enter'), then comes to the second originally sent message (by the server) and processes that one. One more loop, and all three are done. Hitting enter again, and assuming the server echoes what you sent, you should start receiving your typed in messages (which might be empty if you only hit enter). The loop will currently not exit except you kill it.
I am working on network programming using epoll and I have this code...
int read = read(socket, buf, bufsize);
I have a huge buffer size and I assumed it will receive everything clients sent.
However, I started facing problems like packet segmentation.
One example is that if a client sent 500 bytes but it somehow got into two 250 bytes packets then there is no way to handle this situation.
I looked up online and found this code
int handle_read(client *cli, struct epoll_event *ev) {
size_t len = 4096;
char *p;
ssize_t received;
cli->state = 1;
if (cli->buffer != NULL) {
//free(cli->buffer);
//printf("Buff not null %s\n", cli->buffer);
}
//allocate space for data
cli->buffer = (char*)malloc( (size_t)(sizeof(char) * 4096) );
p = cli->buffer;
do { //read until loop conditions
received = recv(ev->data.fd, p, len, 0);
if (received < 0 && errno != EAGAIN && errno != EWOULDBLOCK) {
//if error, remove from epoll and close socket
printf("Handle error!!!\nClient disconnected!\n");
epoll_ctl(epollfd, EPOLL_CTL_DEL, ev->data.fd, ev);
close(ev->data.fd);
}
p = &cli->buffer[received];
} while (received >= len && errno != EAGAIN && errno != EWOULDBLOCK);
return received;
}
Do you guys think it handles all the exceptions might happen while receiving? Also could you please provide me tutorials or examples that handles socket exceptions? Sample codes online don't cover details.. Thanks in advance
recv can return any of three things, and your code needs to handle each one correctly:
1) Positive number. This means it read some bytes.
2) Negative number. This means an "error" occurred.
3) Zero. This means the other end of the connection performed a successful shutdown() (or close()) on the socket. (In general, a return of 0 from read() or recv() means EOF.)
The "error" case further breaks down into "EAGAIN or EWOULDBLOCK" and "everything else". The first two just means it is a non-blocking socket and there was no data to give you at this time. You probably want to go back and call poll() (or select() or epoll()) again to avoid busy waiting...
"Everything else" means a real error. You need to handle those too; see the POSIX spec for recv() for a complete list.
Given all this, I would say your sample code is bad for several reasons. It does not handle 0 (closed connection) properly. It does not handle any errors. It does a busy-loop when the recv() returns EAGAIN/EWOULDBLOCK.
Oh, and it uses sizeof(char), which is a sure sign it was written by somebody who is not familiar with the C or C++ programming languages.
You can't know "How many datas client sent" in normaly. you should use scalable data format(that have data length in the header) or separator for data tokens. For example, you may add \xff between data and next data. Or, you should use fixed data format.