Too many data on socket using epoll edge trigger [closed] - c++

Question

Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 4 years ago.
Improve this question
The scenario is like this: one process is using epoll on several sockets, all sockets are set non-blocking and edge triggered; then EPOLLIN event occurs on one socket, then we start to read data on its fd, but the problem is that there are too many data coming in, and in the while loop reading data, the return value of recv is always larger than 0. So the application is stuck there, reading data and cannot move on.
Any idea how should I deal with this?
constexpr int max_events = 10;
constexpr int buf_len = 8192;
....
epoll_event events[max_events];
char buf[buf_len];
int n;
auto fd_num = epoll_wait(...);
for(auto i = 0; i < fd_num; i++) {
if(events[i].events & EPOLLIN) {
for(;;) {
n = ::read(events[i].data.fd, buf, sizeof(buf));
if (errno == EAGAIN)
break;
if (n <= 0)
{
on_disconnect_(events[i].data.fd);
break;
}
else
{
on_data_(events[i].data.fd, buf, n);
}
}
}
}

Answer 1

When using edge triggered mode the data must be read in one recv call, otherwise it risks starving other sockets. This issue has been written about in numerous blogs, e.g. Epoll is fundamentally broken.
Make sure that your user-space receive buffer is at least the same size as the kernel receive socket buffer. This way you read the entire kernel buffer in one recv call.
Also, you can process ready sockets in a round-robin fashion, so that the control flow does not get stuck in recv loop for one socket. That works best with the user-space receive buffer being of the same size as the kernel one. E.g.:
auto n = epoll_wait(...);
for(int dry = 0; dry < n;) {
for(auto i = 0; i < n; i++) {
if(events[i].events & EPOLLIN) {
// Do only one read call for each ready socket
// before moving to the next ready socket.
auto r = recv(...);
if(-1 == r) {
if(EAGAIN == errno) {
events[i].events ^= EPOLLIN;
++dry;
}
else
; // Handle error.
}
else if(!r){
// Process client disconnect.
}
else {
// Process data received so far.
}
}
}
}
This version can be further improved to avoid scanning the entire events array on each iteration.
In you original post do {} while(n > 0); is incorrect and it leads to an endless loop. I assume it is a typo.