I wrote a UNIX daemon (targeting Debian, but it shouldn't matter) and I wanted to provide some way of creating a ".pid" file, (a file which contains the process identifier of the daemon).
I searched for a way of opening a file only if it doesn't exist, but couldn't find one.
Basically, I could do something like:
if (fileexists())
{
//fail...
}
else
{
//create it with fopen() or similar
}
But as it stands, this code does not perform the task in a atomic fashion, and doing so would be dangerous, because another process might create the file during my test, and the file creation.
Do you guys have any idea on how to do that?
Thank you.
P.S: Bonus point for a solution which only involves std::streams.
man 2 open:
O_EXCL Ensure that this call creates the file: if this flag is specified in conjunction with O_CREAT, and pathname already exists, then open()
will fail. The behavior of O_EXCL is undefined if O_CREAT is not specified.
so, you could call fd = open(name, O_CREAT | O_EXCL, 0644); /* Open() is atomic. (for a reason) */
UPDATE: and you should of course OR one of the O_RDONLY, O_WRONLY, or O_RDWR flags into the flags argument.
I learned about proper daemonizing here (back in the day):
http://www.enderunix.org/docs/eng/daemon.php
It is a good read. I have since improved the locking code to eliminate race conditions on platforms that allow advisory file locking with specific regions specified.
Here is a relevant snippet from a project that I was involved in:
static int zfsfuse_do_locking(int in_child)
{
/* Ignores errors since the directory might already exist */
mkdir(LOCKDIR, 0700);
if (!in_child)
{
ASSERT(lock_fd == -1);
/*
* before the fork, we create the file, truncating it, and locking the
* first byte
*/
lock_fd = creat(LOCKFILE, S_IRUSR | S_IWUSR);
if(lock_fd == -1)
return -1;
/*
* only if we /could/ lock all of the file,
* we shall lock just the first byte; this way
* we can let the daemon child process lock the
* remainder of the file after forking
*/
if (0==lockf(lock_fd, F_TEST, 0))
return lockf(lock_fd, F_TLOCK, 1);
else
return -1;
} else
{
ASSERT(lock_fd != -1);
/*
* after the fork, we instead try to lock only the region /after/ the
* first byte; the file /must/ already exist. Only in this way can we
* prevent races with locking before or after the daemonization
*/
lock_fd = open(LOCKFILE, O_WRONLY);
if(lock_fd == -1)
return -1;
ASSERT(-1 == lockf(lock_fd, F_TEST, 0)); /* assert that parent still has the lock on the first byte */
if (-1 == lseek(lock_fd, 1, SEEK_SET))
{
perror("lseek");
return -1;
}
return lockf(lock_fd, F_TLOCK, 0);
}
}
void do_daemon(const char *pidfile)
{
chdir("/");
if (pidfile) {
struct stat dummy;
if (0 == stat(pidfile, &dummy)) {
cmn_err(CE_WARN, "%s already exists; aborting.", pidfile);
exit(1);
}
}
/*
* info gleaned from the web, notably
* http://www.enderunix.org/docs/eng/daemon.php
*
* and
*
* http://sourceware.org/git/?p=glibc.git;a=blob;f=misc/daemon.c;h=7597ce9996d5fde1c4ba622e7881cf6e821a12b4;hb=HEAD
*/
{
int forkres, devnull;
if(getppid()==1)
return; /* already a daemon */
forkres=fork();
if (forkres<0)
{ /* fork error */
cmn_err(CE_WARN, "Cannot fork (%s)", strerror(errno));
exit(1);
}
if (forkres>0)
{
int i;
/* parent */
for (i=getdtablesize();i>=0;--i)
if ((lock_fd!=i) && (ioctl_fd!=i)) /* except for the lockfile and the comm socket */
close(i); /* close all descriptors */
/* allow for airtight lockfile semantics... */
struct timeval tv;
tv.tv_sec = 0;
tv.tv_usec = 200000; /* 0.2 seconds */
select(0, NULL, NULL, NULL, &tv);
VERIFY(0 == close(lock_fd));
lock_fd == -1;
exit(0);
}
/* child (daemon) continues */
setsid(); /* obtain a new process group */
VERIFY(0 == chdir("/")); /* change working directory */
umask(027); /* set newly created file permissions */
devnull=open("/dev/null",O_RDWR); /* handle standard I/O */
ASSERT(-1 != devnull);
dup2(devnull, 0); /* stdin */
dup2(devnull, 1); /* stdout */
dup2(devnull, 2); /* stderr */
if (devnull>2)
close(devnull);
/*
* contrary to recommendation, do _not_ ignore SIGCHLD:
* it will break exec-ing subprocesses, e.g. for kstat mount and
* (presumably) nfs sharing!
*
* this will lead to really bad performance too
*/
signal(SIGTSTP,SIG_IGN); /* ignore tty signals */
signal(SIGTTOU,SIG_IGN);
signal(SIGTTIN,SIG_IGN);
}
if (0 != zfsfuse_do_locking(1))
{
cmn_err(CE_WARN, "Unexpected locking conflict (%s: %s)", strerror(errno), LOCKFILE);
exit(1);
}
if (pidfile) {
FILE *f = fopen(pidfile, "w");
if (!f) {
cmn_err(CE_WARN, "Error opening %s.", pidfile);
exit(1);
}
if (fprintf(f, "%d\n", getpid()) < 0) {
unlink(pidfile);
exit(1);
}
if (fclose(f) != 0) {
unlink(pidfile);
exit(1);
}
}
}
See also http://gitweb.zfs-fuse.net/?p=sehe;a=blob;f=src/zfs-fuse/util.c;h=7c9816cc895db4f65b94592eebf96d05cd2c369a;hb=refs/heads/maint
The only way I can think of is to use system level locks. See this: C++ how to check if file is in use - multi-threaded multi-process system
One way to approach this problem is to open the file for appending. If the function succeeds and the position is at 0 then you can be fairly certain this is a new file. Could still be an empty file but that scenario may not be important.
FILE* pFile = fopen(theFilePath, "a+");
if (pFile && gfetpos(pFile) == 0) {
// Either file didn't previously exist or it did and was empty
} else if (pFile) {
fclose(pFile);
}
It would appear that there's no way to do it strictly using streams.
You can, instead, use open (as mentioned above by wildplasser) and if that succeeds, proceed to open the same file as a stream. Of course, if all you're writing to the file is a PID, it is unclear why you wouldn't just write it using C-style write().
O_EXCL only excludes other processes that are attempting to open the same file using O_EXCL. This, of course, means that you never have a perfect guarantee, but if the file name/location is somewhere nobody else is likely to be opening (other than folks you know are using O_EXCL) you should be OK.
Related
I have a following scenario: Someone creates a pseudo terminal via opening /dev/ptmx. New terminal is created and named for example /dev/pts/2. Then, in my program I open /dev/pts/2 for reading. But I also open other devices for reading and use select() function to wait for any incoming data. The select have also some timeout specified for performing other stuff when no data arrives for too long. After successful select i read data using read() function and then print it on the screen.
I encountered an issue if the pseudo terminal is closed by the one who created it. In this case select function ends immediately indicating success as well as read ends indicating "no data" by returning zero. The issue imho is that neither select nor read returns error in such case. How should I handle this to detect that the terminal is no longer existing?
Status processData()
{
fd_set readFileDescriptorSet; // defined somewhere else
int maxFileDescriptor; // defined somewhere else
struct timeval timeout; // defined somewhere else
int ret = select(maxFileDescriptor + 1, &readFileDescriptorSet, nullptr, nullptr, &timeout);
if (!ret) // timeout
return Status::success();
if (ret < 0) // error from select()
return Status::error("select error");
ssize_t rd;
char buff[10];
do {
rd = read(interfaces.serialPort.getFileDescriptor(), buff, sizeof(buff) - 1);
if (rd > 0) { // some data has been read
buff[rd] = '\0';
std::cout << buff;
}
} while (rd > 0);
if (rd < 0) // error from read()
return Status::error("read error");
return Status::success();
}
While the way I open the pseudo terminal is following:
Status internalOpen(std::string fileName)
{
close();
fileDescriptor = ::open(fileName.c_str(), O_RDWR | O_NOCTTY | O_NONBLOCK);
if (fileDescriptor == -1)
return Status::error("Terminal::internalOpen::open('" + fileName + "')");
struct termios attributes;
if (tcgetattr(fileDescriptor, &attributes))
return Status::error("Terminal::internalOpen::tcgetattr()");
setAttributes(attributes);
if (tcsetattr(fileDescriptor, TCSANOW, &attributes))
return Status::error("Terminal::internalOpen::tcsetattr()");
return Status::success();
}
void setAttributes(struct termios &attributes)
{
cfmakeraw(&attributes);
cfsetspeed(&attributes, Config::baudRate);
attributes.c_iflag &= ~(IXOFF | IXANY);
attributes.c_oflag &= ~(ONLCR);
attributes.c_lflag &= ~(ECHOE);
attributes.c_cflag &= ~(CSTOPB | CRTSCTS);
attributes.c_cflag |= CREAD | CLOCAL;
attributes.c_cc[VMIN] = 0;
attributes.c_cc[VTIME] = 0;
}
After select() returns indicating that there's something to be read, the shown code loops repeatedly trying to read() from the non-blocking file descriptor until it is 0:
do {
rd = read( ...
} while (rd > 0);
That's certainly reasonable. Except that the closed connection results in the very first read() returning 0, which the shown logic cannot discriminate.
All that's really needed here is to keep track of whether anything has been read, prior read() returning 0. But if read() returned 0 right off the bat, your goose is cooked.
Additionally, there a few other improvements will make things more robust.
After select() returns, actually check if the file descriptor's bit remains set in the readFileDescriptorSet. The shown logic simply assumes that it is, by checking for all other possibilities. Still, this is somewhat fragile. It's easy to forget this assumption if something tangentially related gets modified (i.e., another fle descriptor gets thrown into the mix).
Use poll() instead of select(), and explicitly check for POLLHUP|POLLRDHUP in revents. The file descriptor closure condition is more explicitly called out, in the poll() interface.
I'm doing some practice on process management in Linux and how to use system calls and communication between child and parent processes. I need to implement a pipe to get the string provided by child process, which is the directory list as string and pass it to the parent process to count the number of lines in that string and find the number of files in that directory by doing that. The problem i faced is here:
error: initializer fails to determine size of ‘dirFileList’
char dirFileList[] = read(tunnel[0],buf,MAX_BUF)
Also my code is down below:
#define die(e) do { fprintf(stderr, "%s\n", e); exit(EXIT_FAILURE); } while (0);
#define MAX_BUF 2024
int main()
{
const char *path = (char *)"/"; /* Root path */
const char *childCommand = (char *)"ls |"; /* Command to be executed by the child process */
const char *parentCommand = (char *)"wc -l"; /* Command to be executed by the parent process */
int i = 0; /* A simple loop counter :) */
int counter = 0; /* Counts the number of lines in the string provided in the child process */
int dirFileNum; /* Keeps the list of files in the directory */
int tunnel[2]; /* Defining an array of integer to let the child process store a number and parent process to pick that number */
pid_t pID = fork();
char buf[MAX_BUF]; /* Fork from the main process */
if (pipe(tunnel) == -1) /* Pipe from the parent to the child */
die("pipe died.");
if(pID == -1) /* Check if the fork result is valid */
{
die("fork died.");
}
else if(pID == 0) /* Check if we are in the child process */
{
dup2 (tunnel[1], STDOUT_FILENO); /* Redirect standard output */
close(tunnel[0]);
close(tunnel[1]);
execl(childCommand, path); /* Execute the child command */
die("execl died.");
}
else /* When we are still in the main process */
{
close(tunnel[1]);
char dirFileList[] = read(tunnel[0],buf,MAX_BUF); /* Read the list of directories provided by the child process */
for(i;i<strlen(dirFileList);i++) /* Find the number of lines in the list provided by the child process */
if(dirFileList[i] == '\n')
counter++;
printf("Root contains %d files.", counter); /* Print the result */
wait(NULL); /* Wait until the job is done by the child process */
}
return 0;
}
If you'd shown us the whole error message, we'd see it's referring to this line:
char dirFileList[] = read(tunnel[0],buf,MAX_BUF);
You can't declare an indeterminate array like that. And if you read the man page of read(2), you'll see that the return value is
On success, the number of bytes read ...
On error, -1 ...
So you want something like
int bytes_read = read(...);
if (bytes_read < 0) {
perror("read");
exit(1);
}
Some additional review (which you didn't ask for, but may be instructive):
Don't cast string literals to char*, especially when you're then assigning to const char* variables.
Instead of just printing a fixed message on error, you can be more informative after a call that has set errno if you use perror() - see my sample above.
die() could be implemented as a function, which will make it easier to debug and to use correctly than a macro.
The function fopen("file-name",a); will return a pointer to the end of the file. If the file exist it is opened, otherwise a new file is created.
Is it possible to use the append mode and open the file only if it already exist? (and return a NULL pointer otherwise).
Thanks in advance
To avoid race conditions, opening and checking for existence should be done in one system call. In POSIX this can be done with open as it will not create the file if the flag O_CREAT is not provided.
int fd;
FILE *fp = NULL;
fd = open ("file-name", O_APPEND);
if (fd >= 0) {
/* successfully opened the file, now get a FILE datastructure */
fp = fdopen (fd, "a")
}
open may fail for other reasons too. If you do not want to ignore all of them, you will have to check errno.
int fd;
FILE *fp = NULL;
do {
fd = open ("file-name", O_APPEND);
/* retry if open was interrupted by a signal */
} while (fd < 0 && errno == EINTR);
if (fd >= 0) {
/* successfully opened the file, now get a FILE datastructure */
fp = fdopen (fd, "a")
} else if (errno != ENOENT) { /* ignore if the file does not exist */
perror ("open file-name"); /* report any other error */
exit (EXIT_FAILURE)
}
First check if the file already exists. A simple code to do that might be like this:
int exists(const char *fname)
{
FILE *file;
if ((file = fopen(fname, "r")))
{
fclose(file);
return 1;
}
return 0;
}
It will return 0 if file doesn't exist...
and use it like this:
if(exists("somefile")){file=fopen("somefile","a");}
Using Linux and C++, I would like a function that does the following:
string f(string s)
{
string r = system("foo < s");
return r;
}
Obviously the above doesn't work, but you get the idea. I have a string s that I would like to pass as the standard input of a child process execution of application "foo", and then I would like to record its standard output to string r and then return it.
What combination of Linux syscalls or POSIX functions should I use?
I'm using Linux 3.0 and do not need the solution to work with older systems.
The code provided by eerpini does not work as written. Note, for example, that the pipe ends that are closed in the parent are used afterwards. Look at
close(wpipefd[1]);
and the subsequent write to that closed descriptor. This is just transposition, but it shows this code has never been used. Below is a version that I have tested. Unfortunately, I changed the code style, so this was not accepted as an edit of eerpini's code.
The only structural change is that I only redirect the I/O in the child (note the dup2 calls are only in the child path.) This is very important, because otherwise the parent's I/O gets messed up. Thanks to eerpini for the initial answer, which I used in developing this one.
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#define PIPE_READ 0
#define PIPE_WRITE 1
int createChild(const char* szCommand, char* const aArguments[], char* const aEnvironment[], const char* szMessage) {
int aStdinPipe[2];
int aStdoutPipe[2];
int nChild;
char nChar;
int nResult;
if (pipe(aStdinPipe) < 0) {
perror("allocating pipe for child input redirect");
return -1;
}
if (pipe(aStdoutPipe) < 0) {
close(aStdinPipe[PIPE_READ]);
close(aStdinPipe[PIPE_WRITE]);
perror("allocating pipe for child output redirect");
return -1;
}
nChild = fork();
if (0 == nChild) {
// child continues here
// redirect stdin
if (dup2(aStdinPipe[PIPE_READ], STDIN_FILENO) == -1) {
exit(errno);
}
// redirect stdout
if (dup2(aStdoutPipe[PIPE_WRITE], STDOUT_FILENO) == -1) {
exit(errno);
}
// redirect stderr
if (dup2(aStdoutPipe[PIPE_WRITE], STDERR_FILENO) == -1) {
exit(errno);
}
// all these are for use by parent only
close(aStdinPipe[PIPE_READ]);
close(aStdinPipe[PIPE_WRITE]);
close(aStdoutPipe[PIPE_READ]);
close(aStdoutPipe[PIPE_WRITE]);
// run child process image
// replace this with any exec* function find easier to use ("man exec")
nResult = execve(szCommand, aArguments, aEnvironment);
// if we get here at all, an error occurred, but we are in the child
// process, so just exit
exit(nResult);
} else if (nChild > 0) {
// parent continues here
// close unused file descriptors, these are for child only
close(aStdinPipe[PIPE_READ]);
close(aStdoutPipe[PIPE_WRITE]);
// Include error check here
if (NULL != szMessage) {
write(aStdinPipe[PIPE_WRITE], szMessage, strlen(szMessage));
}
// Just a char by char read here, you can change it accordingly
while (read(aStdoutPipe[PIPE_READ], &nChar, 1) == 1) {
write(STDOUT_FILENO, &nChar, 1);
}
// done with these in this example program, you would normally keep these
// open of course as long as you want to talk to the child
close(aStdinPipe[PIPE_WRITE]);
close(aStdoutPipe[PIPE_READ]);
} else {
// failed to create child
close(aStdinPipe[PIPE_READ]);
close(aStdinPipe[PIPE_WRITE]);
close(aStdoutPipe[PIPE_READ]);
close(aStdoutPipe[PIPE_WRITE]);
}
return nChild;
}
Since you want bidirectional access to the process, you would have to do what popen does behind the scenes explicitly with pipes. I am not sure if any of this will change in C++, but here is a pure C example :
void piped(char *str){
int wpipefd[2];
int rpipefd[2];
int defout, defin;
defout = dup(stdout);
defin = dup (stdin);
if(pipe(wpipefd) < 0){
perror("Pipe");
exit(EXIT_FAILURE);
}
if(pipe(rpipefd) < 0){
perror("Pipe");
exit(EXIT_FAILURE);
}
if(dup2(wpipefd[0], 0) == -1){
perror("dup2");
exit(EXIT_FAILURE);
}
if(dup2(rpipefd[1], 1) == -1){
perror("dup2");
exit(EXIT_FAILURE);
}
if(fork() == 0){
close(defout);
close(defin);
close(wpipefd[0]);
close(wpipefd[1]);
close(rpipefd[0]);
close(rpipefd[1]);
//Call exec here. Use the exec* family of functions according to your need
}
else{
if(dup2(defin, 0) == -1){
perror("dup2");
exit(EXIT_FAILURE);
}
if(dup2(defout, 1) == -1){
perror("dup2");
exit(EXIT_FAILURE);
}
close(defout);
close(defin);
close(wpipefd[1]);
close(rpipefd[0]);
//Include error check here
write(wpipefd[1], str, strlen(str));
//Just a char by char read here, you can change it accordingly
while(read(rpipefd[0], &ch, 1) != -1){
write(stdout, &ch, 1);
}
}
}
Effectively you do this :
Create pipes and redirect the stdout and stdin to the ends of the two pipes (note that in linux, pipe() creates unidirectional pipes, so you need to use two pipes for your purpose).
Exec will now start a new process which has the ends of the pipes for stdin and stdout.
Close the unused descriptors, write the string to the pipe and then start reading whatever the process might dump to the other pipe.
dup() is used to create a duplicate entry in the file descriptor table. While dup2() changes what the descriptor points to.
Note : As mentioned by Ammo# in his solution, what I provided above is more or less a template, it will not run if you just tried to execute the code since clearly there is a exec* (family of functions) missing, so the child will terminate almost immediately after the fork().
Ammo's code has some error handling bugs. The child process is returning after dup failure instead of exiting. Perhaps the child dups can be replaced with:
if (dup2(aStdinPipe[PIPE_READ], STDIN_FILENO) == -1 ||
dup2(aStdoutPipe[PIPE_WRITE], STDOUT_FILENO) == -1 ||
dup2(aStdoutPipe[PIPE_WRITE], STDERR_FILENO) == -1
)
{
exit(errno);
}
// all these are for use by parent only
close(aStdinPipe[PIPE_READ]);
close(aStdinPipe[PIPE_WRITE]);
close(aStdoutPipe[PIPE_READ]);
close(aStdoutPipe[PIPE_WRITE]);
I'm using the following code to write data through a named pipe from one application to another. The thread where the writing is taken place should never be exited. But if r_write() returns less than it should, the thread/program stops for some reason. How can I make the thread continue once write has returned less than it should?
ssize_t r_write(int fd, char *buf, size_t size)
{
char *bufp;
size_t bytestowrite;
ssize_t byteswritten;
size_t totalbytes;
for (bufp = buf, bytestowrite = size, totalbytes = 0;
bytestowrite > 0;
bufp += byteswritten, bytestowrite -= byteswritten) {
byteswritten = write(fd, bufp, bytestowrite);
if ((byteswritten) == -1 && (errno != EINTR))
return -1;
if (byteswritten == -1)
byteswritten = 0;
totalbytes += byteswritten;
}
return totalbytes;
}
void* sendData(void *thread_arg)
{
int fd, ret_val, count, numread;
string word;
char bufpipe[5];
ret_val = mkfifo(pipe, 0777); //make the sprout pipe
if (( ret_val == -1) && (errno != EEXIST))
{
perror("Error creating named pipe");
exit(1);
}
while(1)
{
if(!sproutFeed.empty())
{
string s;
s.clear();
s = sproutFeed.front();
int sizeOfData = s.length();
snprintf(bufpipe, 5, "%04d", sizeOfData);
char stringToSend[strlen(bufpipe) + sizeOfData +1];
bzero(stringToSend, sizeof(stringToSend));
strncpy(stringToSend,bufpipe, strlen(bufpipe));
strncat(stringToSend,s.c_str(),strlen(s.c_str()));
strncat(stringToSend, "\0", strlen("\0"));
int fullSize = strlen(stringToSend);
cout << "sending string" << stringToSend << endl;
fd = open(pipe,O_WRONLY);
int numWrite = r_write(fd, stringToSend, strlen(stringToSend) );
if(numWrite != fullSize)
{
bzero(bufpipe, strlen(bufpipe));
bzero(stringToSend, strlen(stringToSend));
cout << "NOT FULL SIZE WRITE " << endl; //program ends here??
}
else
{
sproutFeed.pop();
bzero(bufpipe, strlen(bufpipe));
bzero(stringToSend, strlen(stringToSend));
}
}
else
{
sleep(1);
}
}
}
If the write() returns a positive (non-zero, non-negative) value for the number of bytes written, it was successful, but there wasn't room for all the data. Try again, writing the remainder of the data from the buffer (and repeat as necessary). Don't forget, a FIFO has a limited capacity - and writers will be held up if necessary.
If the write() returns a negative value, the write failed. The chances are that you won't be able to recover, but check errno for the reason why.
I think the only circumstance where write() can return zero is if you have the file descriptor open with O_NONBLOCK and the attempt to write would block. You might need to scrutinize the manual page for write() to check for any other possibilities.
What your thread does then depends on why it experienced a short write, and what you want to do about it.
The write to the FIFO failed. Investigate the value of errno to find out why. Look in errno.h on your system to decipher the value of errno. If the program is ending upon trying to write to the console, the reason may be related.
Also, your loop doesn't appear to be closing the file descriptor for the FIFO (close(fd)).
Finally, you mention multithreading. The standard library stream cout on your system may not (and probably isn't) thread-safe. In that case, writing to the console concurrently from multiple threads will cause unpredictable errors.
You need to make the file descriptor non-blocking. You can do it like this:
fcntl(fd, F_SETFL, fcntl(fd, F_GETFL) | O_NONBLOCK);
Explanation
This is how fcntl works (not a complete description - look at man fcntl for that). First of all, the includes:
#include <unistd.h>
#include <fcntl.h>
reading the file descriptor's flags
Use F_GETFL to get the file descriptor's flags. From man fcntl:
F_GETFL
Read the file descriptor's flags.
RETURN VALUE
For a successful call, the return value depends on the operation:
F_GETFL Value of flags.
and this is how it's used:
int fd_flags = fcntl(fd, F_GETFL);
writing the file descriptor's flags
Use F_SETFL to set the O_NONBLOCK flag. Again, quoting from man fcntl:
F_SETFL
Set the file status flags part of the descriptor's flags to the
value specified by arg. Remaining bits (access mode, file cre?
ation flags) in arg are ignored. On Linux this command can
only change the O_APPEND, O_NONBLOCK, O_ASYNC, and O_DIRECT
flags.
and this is how it's used:
fcntl(fd, F_SETFL, fd_flags | O_NONBLOCK);