I am trying to read a filedescriptor with read() but my program freezes there and after a while is Killed. Is there a timeout for reading filedescriptors or a good way to check why this happens?
My program uses gadgetfs to read/write from/to USB in user space.
Thanks!
This is the code. It's a modified version of the usb.c samle (http://www.linux-usb.org/gadget/usb.c)
struct pollfd ep0_poll;
ep0_poll.fd = gadgetFS_FD;
ep0_poll.events = POLLIN | POLLOUT | POLLHUP;
while(){
int nPoll = poll(&ep0_poll, 1, 250);
if(nPoll == 0)
continue;
if (nPoll < 0) {
printf("poll failed");
}
usb_gadgetfs_event aoEvent [5];
ssize_t nRead = read (ep0_poll.fd, &aoEvent, sizeof aoEvent);
if (nRead < 0) {
if (errno == EAGAIN) {
sleep (1);
continue;
}
printf("ep0 read after poll");
}
...
}
Related
I am trying to write a my own small linux shell , and i want to write the function cp , the function format is like the following :
cp <old-file-path> <new-file-path>
It copies the first file into the second file (overwriting it) , and if the second file doesn't exist it will create a new one.
If the files didn't open or any system call did not succeed it will print an error message.
However, sometimes I want to copy large files so I want to run this cp command in the background (using fork without waiting for it to finish).
My problem is: how can I use fork and not wait for the process to finish?
Currently, the child process becomes a zombie process.
Here is my code :
// num_args contains the number of arguments sent to cp
class CopyCommand : public BuiltInCommand {
public:
CopyCommand(const char* cmd_line) : BuiltInCommand(cmd_line){}
virtual ~CopyCommand() {}
void execute() override{
if(this->num_args < 1){ // if no arguments were send to cp
perror("invalid arguments");
return;
}
char* buff;
int fd1 = open(args[1], O_RDONLY);
if(fd1 == -1){
perror("open failed");
return;
}
if(this->num_args==2){ // copy file1 into file2 (overrite file 1)
int fd2 = open(args[2], O_TRUNC);
if (fd2 == -1) { // if we couldn't open the file then create a new one (not sure if we supposed to this ?)
fd2 = open(args[2], O_CREAT, 0666);
if (fd2 == -1) {
perror("open failed");
return;
}
}
pid_t PID = fork();
if(PID == -1){
perror("fork failed");
return;
}
else if(PID == 0){
// i need to use fork here :( before i start to write
int read_res = read(fd1, &buff, 1); /// read from the file fd1 into fd2
while (read_res != -1) {
if (!read_res) {
break;
}
if (write(fd2, buff, 1) == -1) {
perror("write failed");
}
read_res = read(fd1, buff, 1);
}
if (read_res == -1) {
perror("read failed");
}
}
}
else if(this->num_args==1){ // create file2 and copy file1 into file2
// don't know how to do this yet
// i need to use fork here :(
}
}
};
For starters, I rewrote your code a bit.
In particular, note that the child branch (PID == 0) exits when it is done.
The parent closes the passed-down file descriptors after forking and in case of error.
if (this->num_args == 2) {
int fd1 = open(args[1], O_RDONLY);
if (fd1 == -1) {
perror("open failed");
return;
}
int fd2 = open(args[2], O_TRUNC);
if (fd2 == -1) {
fd2 = open(args[2], O_CREAT, 0666);
if (fd2 == -1) {
perror("open failed");
close(fd1);
return;
}
}
pid_t PID = fork();
if (PID == -1) {
perror("fork failed");
} else if (PID == 0) {
char buff[1024];
int read_res = read(fd1, &buff, 1024); /// read from the file fd1 into fd2
while (read_res != -1) {
if (!read_res) {
break;
}
if (write(fd2, buff, read_res) == -1) {
perror("write failed");
}
read_res = read(fd1, buff, 1024);
}
if (read_res == -1) {
perror("read failed");
}
exit(0);
} else {
printf("Copy running in background (pid: %d)\n", PID);
}
close(fd1);
close(fd2);
return
}
When the child process calls exit, the process will stick around in "Zombie" state. This state allows the parent process (you) to call wait or waitpid to retrieve the exit code.
As a secondary effect of the process ending, the kernel will send a SIGCHLD to your process, to let you know you can actually call wait without blocking. In your situation, you do not care about the exit code, so you can set up a "don't care" signal handler at the start of your program and let the kernel clean up the process:
signal(SIGCHLD, SIG_IGN);
This is documented in signal(2):
If a process explicitly specifies SIG_IGN as the action for the signal
SIGCHLD, the system will not create zombie processes when children of the
calling process exit. As a consequence, the system will discard the exit
status from the child processes.
I have a non-blocking socket server which supports all connecting clients. It's using multi-threading and it's cross-compilable using GCC.
It works perfect (as I want) in Linux, but when I try to use it in Windows, when I send a 70MB of file through it, it loses around 20MB from the file.
All sockets are non-blocking, so for recv/send socket calls, I don't have check/stop. It's in a loop and it sends what it receive, it sort of acts as a Echo server, but it loses data in Windows. I'm using Winsock 2.2 in WSAStartup.
What is wrong? How can I have wait/flush send calls, but never block recv calls? (if this is the issue)
Code pieces:
How I make it non-blocking:
iMode = 1;
ioctlsocket(sock1,FIONBIO, &iMode);
ioctlsocket(sock2,FIONBIO, &iMode);
How I send/receive between two sockets:
for (;;)
{
memset(buffer, 0, 8192);
int count = recv(sock1, buffer, sizeof(buffer), 0);
receiveResult = WSAGetLastError();
if (receiveResult == WSAEWOULDBLOCK && count <= 0)
continue;
if (count <= 0)
{
closesocket(sock1);
closesocket(sock2);
return;
}
if (count > 0)
{
int retval = send(sock2, buffer, count, 0);
}
}
int count = recv(sock1, buffer, sizeof(buffer), 0);
receiveResult = WSAGetLastError();
if (receiveResult == WSAEWOULDBLOCK && count <= 0)
When calling recv() or send(), WSAGetLastError() will return a meaningful value only if -1 (SOCKET_ERROR) is returned, but you are also checking it when 0 is returned instead. They do not set an error code for WSAGetLastError() when returning >= 0. You need to separate those conditions.
Also, just because you have read X number of bytes does not guarantee that you will be able to send X number of bytes at one time, so you need to check send() for WSAEWOULDBLOCK until you have no more data to send.
Try something more like this:
bool keepLooping = true;
do
{
int count = recv(sock1, buffer, sizeof(buffer), 0);
if (count > 0)
{
// data received...
char *p = buffer;
do
{
int retval = send(sock2, p, count, 0);
if (retval > 0)
{
p += retval;
count -= retval;
}
else if (retval == 0)
{
// peer disconnected...
keepLooping = false;
}
else if (WSAGetLastError() != WSAEWOULDBLOCK)
{
// a real error occurred...
keepLooping = false;
}
else
{
// peer is not ready to receive...
// optionally use select() to wait here until it is...
}
}
while ((count > 0) && (keepLooping));
}
else if (count == 0)
{
// peer disconnected...
keepLooping = false;
}
else if (WSAGetLastError() != WSAEWOULDBLOCK)
{
// a real error occurred...
keepLooping = false;
}
else
{
// no data is available for reading...
// optionally use select() to wait here until it is...
}
}
while (keepLooping);
closesocket(sock1);
closesocket(sock2);
return;
I'm trying to learn how to use epoll() for tcp server application, 'cause i'm expecting many connections.
i tried checking samples and tutorials, they always recommend using/setting sockets that are added in epoll() to be NON-BLOCKING sockets. why?
For level-triggered epoll, nonblocking sockets can help to minimize epoll_wait() calls, its an optimization issue.
For edge-triggered epoll, you MUST use nonblocking sockets AND call read() or write() until they return EWOULDBLOCK. If you don't, you can miss kernel notifications.
You can find a detailed answer here: https://eklitzke.org/blocking-io-nonblocking-io-and-epoll
It's a good question and not duplicated. Recently I also find a tutorial using nonblocking socket in select (select is level-triggered only), which causes me to think.
The question is:
Why using nonblocking IO or set fd to nonblicking, in level-triggered epoll, select or other similar interfaces?
There are in fact very solid reasons for this case.
Cite from the book The Linux Programming Interface :
63.1.2 Employing Nonblocking I/O with Alternative I/O Models
Nonblocking I/O (the O_NONBLOCK flag) is often used in conjunction
with the I/O models described in this chapter. Some examples of why
this can be useful are the following:
As explained in the previous section, nonblocking I/O is usually employed in conjunction with I/O models that provide edge-triggered
notification of I/O events.
If multiple processes (or threads) are performing I/O on the same open file descriptions, then, from a particular process’s point of
view, a descriptor’s readiness may change between the time the
descriptor was notified as being ready and the time of the subsequent
I/O call. Consequently, a blocking I/O call could block, thus
preventing the process from monitoring other file descriptors. (This
can occur for all of the I/O models that we describe in this chapter,
regardless of whether they employ level-triggered or edge-triggered
notification.)
Even after a level-triggered API such as select() or poll() informs us that a file descriptor for a stream socket is ready for writing, if
we write a large enough block of data in a single write() or send(),
then the call will nevertheless block.
In rare cases, level-triggered APIs such as select() and poll() can return spurious readiness notifications—they can falsely inform us
that a file descriptor is ready. This could be caused by a kernel bug
or be expected behavior in an uncommon scenario.
First, let's check case #2: "If multiple processes (or threads) are performing I/O on the same open file descriptions...".
Read this code from libevent introduction, http://www.wangafu.net/~nickm/libevent-book/01_intro.html .
/* For sockaddr_in */
#include <netinet/in.h>
/* For socket functions */
#include <sys/socket.h>
/* For fcntl */
#include <fcntl.h>
/* for select */
#include <sys/select.h>
#include <assert.h>
#include <unistd.h>
#include <string.h>
#include <stdlib.h>
#include <stdio.h>
#include <errno.h>
#define MAX_LINE 16384
char
rot13_char(char c)
{
/* We don't want to use isalpha here; setting the locale would change
* which characters are considered alphabetical. */
if ((c >= 'a' && c <= 'm') || (c >= 'A' && c <= 'M'))
return c + 13;
else if ((c >= 'n' && c <= 'z') || (c >= 'N' && c <= 'Z'))
return c - 13;
else
return c;
}
struct fd_state {
char buffer[MAX_LINE];
size_t buffer_used;
int writing;
size_t n_written;
size_t write_upto;
};
struct fd_state *
alloc_fd_state(void)
{
struct fd_state *state = malloc(sizeof(struct fd_state));
if (!state)
return NULL;
state->buffer_used = state->n_written = state->writing =
state->write_upto = 0;
return state;
}
void
free_fd_state(struct fd_state *state)
{
free(state);
}
void
make_nonblocking(int fd)
{
fcntl(fd, F_SETFL, O_NONBLOCK);
}
int
do_read(int fd, struct fd_state *state)
{
char buf[1024];
int i;
ssize_t result;
while (1) {
result = recv(fd, buf, sizeof(buf), 0);
if (result <= 0)
break;
for (i=0; i < result; ++i) {
if (state->buffer_used < sizeof(state->buffer))
state->buffer[state->buffer_used++] = rot13_char(buf[i]);
if (buf[i] == '\n') {
state->writing = 1;
state->write_upto = state->buffer_used;
}
}
}
if (result == 0) {
return 1;
} else if (result < 0) {
if (errno == EAGAIN)
return 0;
return -1;
}
return 0;
}
int
do_write(int fd, struct fd_state *state)
{
while (state->n_written < state->write_upto) {
ssize_t result = send(fd, state->buffer + state->n_written,
state->write_upto - state->n_written, 0);
if (result < 0) {
if (errno == EAGAIN)
return 0;
return -1;
}
assert(result != 0);
state->n_written += result;
}
if (state->n_written == state->buffer_used)
state->n_written = state->write_upto = state->buffer_used = 0;
state->writing = 0;
return 0;
}
void
run(void)
{
int listener;
struct fd_state *state[FD_SETSIZE];
struct sockaddr_in sin;
int i, maxfd;
fd_set readset, writeset, exset;
sin.sin_family = AF_INET;
sin.sin_addr.s_addr = 0;
sin.sin_port = htons(40713);
for (i = 0; i < FD_SETSIZE; ++i)
state[i] = NULL;
listener = socket(AF_INET, SOCK_STREAM, 0);
make_nonblocking(listener);
#ifndef WIN32
{
int one = 1;
setsockopt(listener, SOL_SOCKET, SO_REUSEADDR, &one, sizeof(one));
}
#endif
if (bind(listener, (struct sockaddr*)&sin, sizeof(sin)) < 0) {
perror("bind");
return;
}
if (listen(listener, 16)<0) {
perror("listen");
return;
}
FD_ZERO(&readset);
FD_ZERO(&writeset);
FD_ZERO(&exset);
while (1) {
maxfd = listener;
FD_ZERO(&readset);
FD_ZERO(&writeset);
FD_ZERO(&exset);
FD_SET(listener, &readset);
for (i=0; i < FD_SETSIZE; ++i) {
if (state[i]) {
if (i > maxfd)
maxfd = i;
FD_SET(i, &readset);
if (state[i]->writing) {
FD_SET(i, &writeset);
}
}
}
if (select(maxfd+1, &readset, &writeset, &exset, NULL) < 0) {
perror("select");
return;
}
if (FD_ISSET(listener, &readset)) {
struct sockaddr_storage ss;
socklen_t slen = sizeof(ss);
int fd = accept(listener, (struct sockaddr*)&ss, &slen);
if (fd < 0) {
perror("accept");
} else if (fd > FD_SETSIZE) {
close(fd);
} else {
make_nonblocking(fd);
state[fd] = alloc_fd_state();
assert(state[fd]);/*XXX*/
}
}
for (i=0; i < maxfd+1; ++i) {
int r = 0;
if (i == listener)
continue;
if (FD_ISSET(i, &readset)) {
r = do_read(i, state[i]);
}
if (r == 0 && FD_ISSET(i, &writeset)) {
r = do_write(i, state[i]);
}
if (r) {
free_fd_state(state[i]);
state[i] = NULL;
close(i);
}
}
}
}
int
main(int c, char **v)
{
setvbuf(stdout, NULL, _IONBF, 0);
run();
return 0;
}
This is not an example of multiple processes (or threads) performing I/O on the same open file descriptions, but it demostrates the same idea.
In the do_read function, it uses recv in side a while(1) to read as many bytes as possible, but 1024 bytes for each recv. I guess this is a typical pattern.
So you need nonblocking here, otherwise recv will eventually block when there's no data in network input.
For #3, if you write too much data in a blocking socket and there's no enough buffer. send will block until all data are sent. And it could block for long enough time if there's no enough space in the send buffer. More details check https://stackoverflow.com/a/74172742/5983841 .
I'm learning some examples about C++ socket. One of the code here has an error : "expect token while got fclose" at the line above the last line
The code seems fine with me, so I can't figure out what is wrong here.
Any ideas are appreciated.
void RecvFile(int sock, const char* filename)
{
int rval;
char buf[0x1000];
FILE *file = fopen(filename, "wb");
if (!file)
{
printf("Can't open file for writing");
return;
}
do
{
rval = recv(sock, buf, sizeof(buf), 0);
if (rval < 0)
{
// if the socket is non-blocking, then check
// the socket error for WSAEWOULDBLOCK/EAGAIN
// (depending on platform) and if true then
// use select() to wait for a small period of
// time to see if the socket becomes readable
// again before failing the transfer...
printf("Can't read from socket");
fclose(file);
return;
}
if (rval == 0)
break;
int off = 0;
do
{
int written = fwrite(&buf[off], 1, rval - off, file);
if (written < 1)
{
printf("Can't write to file");
fclose(file);
return;
}
off += written;
}
while (off < rval);
}
fclose(file);
}
You have a do with no corresponding while:
do
{
// ...
do
{
// ...
}
while (off < rval);
}
// No while here
fclose(file);
It appears that it should just be while (true), which you might as well just stick at the top, instead of doing a do while. Execution will break from the loop if recv returns 0 or less, which indicate an orderly shutdown and an error respectively. So change it to:
while (true)
{
// ...
do
{
// ...
}
while (off < rval);
}
fclose(file);
You have a do statement without a corresponding while:
do // <== THERE IS NO CORRESPONDING while FOR THIS do
{
rval = recv(sock, buf, sizeof(buf), 0);
if (rval < 0)
{
// ...
}
// ...
do
{
// ...
}
while (off < rval); // <== This is OK: the "do" has a matching "while"
}
// Nothing here! Should have a "while (condition)"
If you just want to repeat your loop indefinitely, then you should use while (true) - either replacing the do keyword (preferably), or adding it where the missing while should go (as indicated by the above comments).
You started a do without actually supplying a while();
do
{
rval = recv(sock, buf, sizeof(buf), 0);
if (rval < 0)
{
// if the socket is non-blocking, then check
// the socket error for WSAEWOULDBLOCK/EAGAIN
// (depending on platform) and if true then
// use select() to wait for a small period of
// time to see if the socket becomes readable
// again before failing the transfer...
printf("Can't read from socket");
fclose(file);
return;
}
if (rval == 0)
break;
int off = 0;
do
{
int written = fwrite(&buf[off], 1, rval - off, file);
if (written < 1)
{
printf("Can't write to file");
fclose(file);
return;
}
off += written;
}
while (off < rval);
} //while() Needs to go here
fclose(file);
In my program I'm redirecting output of child process to pipe and in parent getting this result and doing something (this is not important). But my program doesn't work when I'm using tail -f test.txt command and doesn't recieve any data during tail is running, and getting this data only after tail is finished (or killed).
At first I have thought that problem was that tail -f doesn't flushing and that's why no data I can recieve, but when I have tried to redirect output of tail -f to some file the data were in this file even when tail were not finished.
//the code of creating child and redirecting data (important part)
//only core is here so please don't tell me that maybe pipe() or fork() is failed
pid_t pid;
int outpipe[2]; //pipe for reading from stdout
int errpipe[2]; //pipe for reading from stderr
// Createing pipes for childs stdout and stderr streams
pipe(outpipe);
pipe(errpipe);
pid = fork();
if(pid == 0)
{
// This is the child process. Closing read end of pipes and duplicating stdout and stderr streams
close(outpipe[0]);
dup2(outpipe[1], STDOUT_FILENO);
close(errpipe[0]);
dup2(errpipe[1], STDERR_FILENO);
if(execvp(argv[0], (char * const *)argv) == -1)
{
fprintf(stderr, "Failed to execute command %s: %s", argv[0], strerror(errno));
_exit(EXIT_FAILURE);
}
_exit(EXIT_SUCCESS);
}
else if (pid != -1)
{
// This is the parent process, Closing write end of pipes and opening fds as FILE
close(outpipe[1]);
*child_stdout_stream=fdopen(outpipe[0], "rt");
close(errpipe[1]);
*child_stderr_stream=fdopen(errpipe[0], "rt");
*child_pid=pid;
}
Then I'm reading from child_stderr_stream and child_stdout_stream which were passed as parameters to function the part above is from what.
For reading I'm using select() to not block program until reading from one of the streams.
Adding part of select and read
int select_and_read(FILE **files, bool *is_eof, char *chars, int *mask, int nfiles, int timeout, pid_t child_pid)
{
int max_fd_plus_1 = 0;
fd_set rfds;
struct timeval tv;
FD_ZERO(&rfds);
for(int i = 0; i < nfiles; ++i)
{
if(is_eof[i]==false)
{
FD_SET(fileno(files[i]), &rfds);
max_fd_plus_1 = (max_fd_plus_1 > fileno(files[i])) ? max_fd_plus_1 : fileno(files[i]);
}
}
++max_fd_plus_1;
tv.tv_sec = timeout / 1000;
tv.tv_usec = (timeout % 1000) * 1000;
int retval = select(max_fd_plus_1, &rfds, NULL, NULL, &tv);
if(retval > 0)
{
*mask = 0;
for(int i = 0; i < nfiles; ++i)
{
if(is_eof[i]==false)
{
if(FD_ISSET(fileno(files[i]), &rfds))
{
*mask |= 1 << i;
chars[i] = fgetc(files[i]);
}
}
}
}
else
{
kill(child_pid, SIGKILL);
}
return retval;
}
This strange problem have been solved very strangely. I have just set buffers of files to 0 this way:
else if (pid != -1)
{
// This is the parent process, Closing write end of pipes and opening fds as FILE
close(outpipe[1]);
*child_stdout_stream=fdopen(outpipe[0], "rt");
setbuf(*child_stdout_stream, NULL);
close(errpipe[1]);
*child_stderr_stream=fdopen(errpipe[0], "rt");
setbuf(*child_stderr_stream, NULL);
*child_pid=pid;
}
This is very strange, that this helps, but in any case my program is now working well.