I am trying to write reverse proxy with nonblocking socket and epoll. That seems ok at first, but when I tried to open a big jpg file, I got stuck.
When I try to write into client sometimes It may not writable and how can I handle proper way.
Additional Notes:
this->getFd() = ProxyFd
this->clientHandler->getFd = clientFd
I am using EPOLLET flag both proxy and client
if( (flag & EPOLLIN) ){
char buffer[1025] = {'\0'};
int readSize;
while( (readSize = read(this->getFd(),buffer,1024)) > 0){
this->headerParse(buffer);
this->readSize += readSize;
int check = 0;
do{
check = write(this->clientHandler->getFd(),buffer,readSize);
}while(check < 0);
}
if(this->headerEnd == 1 && this->readSize >= this->headerLenght ){
close(this->clientHandler->getFd());
close(this->getFd());
delete this->clientHandler;
delete this;
}
}
Thanks for taking time to read.
Assuming your headerParse() method doesn't change buffer in a size-extending way (you'd need to update readsize, at least, not to mention the buffer full scenario), it seems like your write() path is broken.
if the socket you're writing to is also in nonblocking mode, it's perfectly legal for write() to return -1 (and set errno to EGAIN or EWOULDBLOCK or whatever your platform has) before you wrote all data.
In that case, you must store the remaining data (the remainder of buffer minus what was written if one or more calls to write() succeeded), program epoll to notify the clientHandler->getFd() descriptor for writeability, if not already, and when you get subsequent "write ready" event, you write the data you stored. On this case, the write() can again be unable to flush all your data, so you must cycle until all data is sent.
Related
I'm writing a C++ program. I need to receive a file and I'm using recv() function over a TCP socket to do that.
download_file() {
while (left_bytes != 0 && !connection_closed) {
if (left_bytes >= buffer_max_size)
bytes_to_download = buffer_max_size;
else
bytes_to_download = left_bytes;
if (request.conn->read_data(buffer, bytes_to_download))
{
left_bytes -= buffer->get_size();
temporary_file.write_data(buffer);
} else connection_closed = true;
}
}
read_data() {
while (bytes_received < size && alive_) {
bytes_read = recv(sock_, read_buffer, size, 0);
if (bytes_read == SOCKET_ERROR) {
delete[] local_buffer;
throw SocketException(WSAGetLastError());
}
// the connection is closed
if (bytes_read == 0) alive_ = false;
else {
bytes_received += bytes_read;
buffer->add(local_buffer, bytes_read);
}
}
}
The problem is that the recv never returns. It receives the whole file except for few KB and it freeze on the recv(). The buffer size is 1460.
I receive the file only if I print something to the console with cout every time the recv is called. Only in this case I receive the whole file.
Otherwise if I set as socket option the WAITALL and the client closes the connection after the file is sent, I receive the whole file.
Here's the code for the Client side that sends the file:
TransmitFile(file_request->connection_->get_handle_socket(), file_handler.get_file_handle(), file_request->file_size_, 65535, nullptr, nullptr, TF_USE_SYSTEM_THREAD)
EDIT
Here's how I send and read the file size between the Client and Server.
std::stringstream stream_;
stream_.str(std::string());
// append the file size
const __int64 file_size = htonll(GetFileSize(file_handle_, nullptr););
stream_ << ' ' << file_size << ' ';
Then I use the send to send this string
Here's how I read the file size
// Within stream_ there is all the content of the received packet
std::string message;
std::getline(stream_, message, ' ');
this->request_body_.file_size_ = ntohll(strtoll(message.c_str(), nullptr, 0));
EDIT
I cleaned up the code and I found out that read_data() is obviously called once and I was updating the buffer variable wrongly. Hence I was tracking the size of the content within the buffer in a wrong way which make me call the recv() once more.
First thing: recv() will block if there are no bytes left to read but the connection is still open. So whatever you might say about what your code is doing, that must be what is happening here.
That could be for any of the following reasons:
the sender lied about the size of the file, or did not send the promised number of bytes
the file size was not interpreted correctly at the receiving end for whatever reason
the logic that 'counts down' the number of bytes left in the receiver is somehow flawed
Trouble is, looking at the code samples you have posted, it's hard to say which because the code is a bit muddled and, in my eyes, more complicated than it needs to be. I'm going to recommend you sort that out.
Sending the size of the file.
Don't mess about sending this as a string. Send it instead in binary, using (say) htonll() at the sending end and ntohll() at the receiving end. Then, the receiver knows to read exactly 8 bytes to figure out what's coming next. It's hard to get that wrong.
Sending the file itself.
TransmitFile() looks to be a good choice here. Stick with it.
Receiving the file and counting down how many bytes are left.
Take a closer look at that code and consider rewriting it. It's a bit of a mess.
What to do if it still doesn't work.
Check with WireShark that the expected data is being sent and then walk through the code in the receiver in the debugger. There is absolutely no excuse for not doing this unless you don't have a debugger for some reason, in which case please say so and somebody will try to help you. The fact that logging to cout fixes your problems is a red-herring. That just changes the timing and then it just happens to work right.
That's all. Best of luck.
This is more of a request for confirmation than a question, so I'll keep it brief. (I am away from my PC and so can't simply implement this solution to test).
I'm writing a program to send an image file taken via webcam (along with meta data) from a raspberryPi to my PC.
I've worked out that the image is roughly around 130kb, the packet header is 12b and the associated meta data another 24b. Though I may increase the image size in future, once I have a working prototype.
At the moment I am not able to retrieve this whole packet successfully as, after sending it to the PC I only ever get approx 64kb recv'd in the buffer.
I have assumed that this is because for whatever reason the default buffer size for a socket declared like:
SOCKET sock = socket(PF_INET, SOCK_STREAM, 0);
is 64kb (please could someone clarify this if you're 'in the know')
So - to fix this problem I intend to increase the socket size to 1024kb via the setsockopt(x..) command.
Please could someone confirm that my diagnosis of the problem, and proposed solution are correct?
I ask this question as I am away form my PC right now and am unable to try it until I get back home.
This most likely has nothing to do with the socket buffers, but with the fact that recv() and send() do not have to receive and send all the data you want. Check the return value of those function calls, it indicates how many bytes have actually been sent and received.
The best way to deal with "short" reads/writes is to put them in a loop, like so:
char *buf; // pointer to your data
size_t len; // length of your data
int fd; // the socket filedescriptor
size_t offset = 0;
ssize_t result;
while (offset < len) {
result = send(fd, buf + offset, len - offset, 0);
if (result < 0) {
// Deal with errors here
}
offset += result;
}
Use a similar construction for receiving data. Note that one possible error condition is that the function call was interrupted (errno = EAGAIN or EWOULDBLOCK), in that case you should retry the send command, in all other cases you should exit the loop.
I am working on network programming using epoll and I have this code...
int read = read(socket, buf, bufsize);
I have a huge buffer size and I assumed it will receive everything clients sent.
However, I started facing problems like packet segmentation.
One example is that if a client sent 500 bytes but it somehow got into two 250 bytes packets then there is no way to handle this situation.
I looked up online and found this code
int handle_read(client *cli, struct epoll_event *ev) {
size_t len = 4096;
char *p;
ssize_t received;
cli->state = 1;
if (cli->buffer != NULL) {
//free(cli->buffer);
//printf("Buff not null %s\n", cli->buffer);
}
//allocate space for data
cli->buffer = (char*)malloc( (size_t)(sizeof(char) * 4096) );
p = cli->buffer;
do { //read until loop conditions
received = recv(ev->data.fd, p, len, 0);
if (received < 0 && errno != EAGAIN && errno != EWOULDBLOCK) {
//if error, remove from epoll and close socket
printf("Handle error!!!\nClient disconnected!\n");
epoll_ctl(epollfd, EPOLL_CTL_DEL, ev->data.fd, ev);
close(ev->data.fd);
}
p = &cli->buffer[received];
} while (received >= len && errno != EAGAIN && errno != EWOULDBLOCK);
return received;
}
Do you guys think it handles all the exceptions might happen while receiving? Also could you please provide me tutorials or examples that handles socket exceptions? Sample codes online don't cover details.. Thanks in advance
recv can return any of three things, and your code needs to handle each one correctly:
1) Positive number. This means it read some bytes.
2) Negative number. This means an "error" occurred.
3) Zero. This means the other end of the connection performed a successful shutdown() (or close()) on the socket. (In general, a return of 0 from read() or recv() means EOF.)
The "error" case further breaks down into "EAGAIN or EWOULDBLOCK" and "everything else". The first two just means it is a non-blocking socket and there was no data to give you at this time. You probably want to go back and call poll() (or select() or epoll()) again to avoid busy waiting...
"Everything else" means a real error. You need to handle those too; see the POSIX spec for recv() for a complete list.
Given all this, I would say your sample code is bad for several reasons. It does not handle 0 (closed connection) properly. It does not handle any errors. It does a busy-loop when the recv() returns EAGAIN/EWOULDBLOCK.
Oh, and it uses sizeof(char), which is a sure sign it was written by somebody who is not familiar with the C or C++ programming languages.
You can't know "How many datas client sent" in normaly. you should use scalable data format(that have data length in the header) or separator for data tokens. For example, you may add \xff between data and next data. Or, you should use fixed data format.
I am using read function to read data from a socket, but when the data is more than 4k, read function just read part of the data, for example, less than 4k. Here is the key code:
mSockFD = socket(AF_INET, SOCK_STREAM, 0);
if (connect(mSockFD, (const sockaddr*)(&mSockAdd), sizeof(mSockAdd)) < 0)
{
cerr << "Error connecting in Crawl" << endl;
perror("");
return false;
}
n = write(mSockFD, httpReq.c_str(), httpReq.length());
bzero(mBuffer, BUFSIZE);
n = read(mSockFD, mBuffer, BUFSIZE);
Note than BUFSIZE is much larger than 4k.
When data is just a few hundred bytes, read function works as expected.
This is by design and to be expected.
The short answer to your question is you should continue calling "read" until you get all the data you expect. That is:
int total_bytes = 0;
int expected = BUFSIZE;
int bytes_read;
char *buffer = malloc(BUFSIZE+1); // +1 for null at the end
while (total_bytes < expected)
{
int bytes_read = read(mSockFD, buffer+total_bytes, BUFSIZE-total_bytes);
if (bytes_read <= 0)
break;
total_bytes += bytes_read;
}
buffer[total_bytes] = 0; // null terminate - good for debugging as a string
From my experience, one of the biggest misconceptions (resulting in bugs) that you'll receive as much data as you ask for. I've seen shipping code in real products written with the expectation that sockets work this way (and no one certain as to why it doesn't work reliably).
When the other side sends N bytes, you might get lucky and receive it all at once. But you should plan for receiving N bytes spread out across multiple recv calls. With the exception of a real network error, you'll eventually get all N bytes. Segmentation, fragmentation, TCP window size, MTU, and the socket layer's data chunking scheme are the reasons for all of this. When partial data is received, the TCP layer doesn't know about how much more is yet to come. It just passes what it has up to the app. It's up to the app to decide if it got enough.
Likewise, "send" calls can get conglomerated into the same packet together.
There may be ioctls and such that will make a socket block until all the expected data is received. But I don't know of any off hand.
Also, don't use read and write for sockets. Use recv and send.
Read this book. It will change your life with regards to sockets and TCP:
I'm facing a particular issue that regards serial communication under win32.
I'm communicating with a device can only accept frames when it is not already communicating. So I must find a valid frame and then inmediatelly send my request.
I developed a class named Serial that handles basic operations on serial port (open, close, read, write) and then a Thread calls inside a loop read and write functions.
Thread loop
//Device is an object of class Serial
while( device->isOpen() && !terminate )
{
unsigned int readed = 0;
unsigned long error = ERROR_SUCCESS;
unsigned char* data = device->read( &readed, &error );
if( error==ERROR_SUCCESS )
{
//If data received, deliver to upper level
if( readed>0 )
{
QByteArray output( (const char*)data, (signed int)readed );
emit dataArrived( output, readed );
}
}
else
{
//unrelated stuff
}
//Here I manage the writting issue
//Only when nothing is received, and Upper layer wants to send a frame
//(Upper layer only will mark as something to send when it detects a valid frame)
if( readed==0 )
{
out_lock.lock();
//If something to send...
if( something_to_send > 0 )
{
if( device->write( output_buffer, output_size, &error ) )
{ //things...
}
}
}
}
The Thread basically keeps reading, and when nothing is received, sees if somebody has signaled to send a frame (this means that a valid frame is just received).
When this happens, it writes the frame through serial port.
Here comes my problem.
Inside the Serial::read() function:
I use the overlapped way of reading:
::ClearCommError( handle, &dwErrors, &stat);
if( stat.cbInQue )
{
//If there's something to read, read it, please note the bytes to read parameter, here 1.
bool ok = ::ReadFile( handle, buffer_in, 1, &bytes_read, &ov_reader );
if( !ok )
{
DWORD _error = ::GetLastError();
if( _error == ERROR_IO_PENDING )
{
DWORD result = ::WaitForMultipleObjects( 2, waiters, FALSE,INFINITE );
switch( result )
{ //Eventshutdown
case WAIT_OBJECT_0: /*code omitted*/break;
case WAIT_OBJECT_0+1: ok = ::GetOverlappedResult( handle, &ov_reader, &bytes_read, true );
//check ok value omitted
break;
}
}
}
}
if( bytes_read>0 )
{
*size = bytes_read;
}
Here starts my problem.
When device sends me small frames (around 30 bytes) everything works fine, but when larger frames are sent, the code is not able to find any free time between frames causing the thread to never be able send any frame because readed is never 0.
If I increase the number of bytes to read inside the read() function, lose the ability to detect when the device "listens":
bool ok = ::ReadFile(handle, buffer_in, 50, &bytes_read, &ov_reader );
This happens because my app can receive the end of a frame together with the start of the next one. This behaviour is very common.
In the other hand, if I change the INFINITE argument by a valid timeout in the WaitForMultipleObjects function, I lose data.
So my question basically is... what I'm doing wrong? Why when reading 1 byte each time I don't find any free time to send my own frames?
Thank you
I'm not sure if this will help or not, but since you already have a good idea of how many bytes are in the serial device's input queue (stat.cbInQue) maybe it would help to read in that many bytes instead of just 1 byte or an arbitrary number of bytes (like 50):
bool ok = ::ReadFile( handle, buffer_in, stat.cbInQue, &bytes_read, &ov_reader );
Of course, you'd need to make sure that the buffer_in had the capacity for that number of bytes, so there might be some other logic you'd have to add to make sure there's no buffer overruns.
Also, because the serial driver and ReadFile() APIs depend heavily on buffering for handling received characters, you might be able to get more precise indications of when characters have been received (and not received) using the
WaitCommEvent() and SetCommMask() APIs.
How big are the "larger frames?" When you call ReadFile one byte at a time, it will obviously take a long time to work through the whole frame, probably longer than it takes the send the frame itself due to call-overhead.
Some alternatives:
Does the device send frames whenever it feels like it? If you have the opportunity to design both ends of the protocol, can you switch to a command/response style of communication?
Can you, from the start of the packet, predict the number of characters in the rest of the packet? If so, you could build a state machine into your read function. You could poll one byte at a time, then when you detect the start of the packet read most of the rest of the packet on one call, then switch back to a byte at a time.
Can you use DSR/CTS to control the timing?
In general, it's really hard to read whole packets from within a serial-port read function. The usual procedure is the read a bunch of characters and pass them up to a higher level for protocol parsing. It sounds like you have to have tighter timing control than that method allows though. Good luck...