Fork, pipe and file operations - c++

I have got a pipe to enable communication between 2 processes in forked program. It was created with pipe() call - http://linux.die.net/man/2/pipe
. Everything goes right until I want to perform some file operations.
This code works:
pipe.writeBuffer(message.c_str(), message.length());
ofstream file;
file.open(name.c_str(), ios::app);
file << "stringData"; // put some data to file (many times)
But this one not:
ofstream file;
file.open(name.c_str(), ios::app);
pipe.writeBuffer(message.c_str(), message.length());
file << "stringData"; // put some data to file (many times)
In the second example there is no effect of "file << someStream" - I get the empty file.
What is wrong with that? Is it a problem with file descriptor? Pipe uses fd[0] - input and fd[1] - output. Maybe fstream uses also the same output file handler?
Here is the "working" sample:
http://pastebin.com/gJ4PbHvy
#include <sys/types.h>
#include <cstdlib>
#include <unistd.h>
#include <iostream>
#include <fstream>
#define maxSize 64
using namespace std;
class Pipe
{
public:
Pipe()
{
pipe(fdesc);
}
~Pipe() {}
void writeBuffer(const char* message, size_t length)
{
close(fdesc[0]);
write(fdesc[1], message, length);
}
void readBuffer()
{
char buffer[maxSize];
close(fdesc[1]);
size_t result = read(fdesc[0], &buffer, sizeof(buffer));
cout << buffer << endl;
}
private:
int fdesc[2];
};
class Writer
{
public:
Writer(Pipe &obj)
{
pipe = obj;
}
~Writer()
{}
void entry()
{
std::string name = "./myFile";
ofstream file;
file.open(name.c_str(), ios::app);
std::string message = "hello world";
pipe.writeBuffer(message.c_str(), message.length()+1);
if (file.is_open())
{
file << "Hello World!" << endl;
file.close();
}
else
{
perror("file.is_open()");
}
sleep(1);
}
private:
Pipe pipe;
};
class Reader
{
public:
Reader(Pipe &obj)
{
pipe = obj;
}
~Reader()
{}
void entry()
{
pipe.readBuffer();
sleep(1);
}
private:
Pipe pipe;
};
int main(int argc, char *argv[])
{
Pipe pipe;
Reader reader(pipe);
Writer writer(pipe);
pid_t pid = fork();
if (pid == -1)
{
perror("fork");
exit(EXIT_FAILURE);
}
if (pid == 0)
{
// child process
while(1)
reader.entry();
}
else
{
// parent process
while(1)
writer.entry();
}
}

With the posted program, the described issue of getting an empty file isn't reproducible, since on each run it does write one line Hello World! into myFile, but this still shows the error, because you intend to write one line every second. The reason is the close(fdesc[0]) in writeBuffer(): While it is correct to close the read end of the pipe once in the writer process, it is incorrect to do it every time writeBuffer() is called, as that file descriptor can (and in the case at hand, does) have been reused after the first close() for another file (here the file ofstream file), which is after it closed instead of the (already closed) pipe, so that nothing can be written into the file. Fix: Arrange your program to close the pipe end only once, e. g. by changing
close(fdesc[0]);
to
if (0 <= fdesc[0]) close(fdesc[0]), fdesc[0] = -1;

Related

C++ wrong usage of close and dup2?

I wrote the following code:
void execute() {
std::cout << "smash pid is " << getpid() << std::endl;
}
int main()
{
int pid=fork();
if (pid==0)
{
int fd=open("my_file.txt", O_WRONLY | O_CREAT, 0666); // 3=my_file
dup2(fd,1); // replace output stream
close(fd); //close duplicate access to my_file
execute();
close (1); // close last access to my file
}
else if (pid>0)
{
std::cout << "Hello!" << std::endl;
}
return 0;
}
my question is am I doing things correctly? and can the main process still have access to print to terminal as usual?
I tried to add notes of what I am doing, please let me know if something isn't clear.
Version 2:
int main()
{
int pid=fork();
if (pid==0)
{
close (1);
int fd=open("my_file.txt", O_WRONLY | O_CREAT, 0666); // 3=my_file
execute();
close (1); // close last access to my file
}
else if (pid>0)
{
std::cout << "Hello!" << std::endl;
}
return 0;
}
my question is am I doing things correctly?
Your child will have its output directed into the opened file and the parent will write "Hello!" to the stdout that was provided to the program at startup. It looks like that's what you want, so, yes.
I'd use pid_t instead of int for the process id though, but that may be different on different platforms.
can the main process still have access to print to terminal as usual?
Yes. The dup2 in the child process does not affect the parent process in any way.
One note. Use fileno(stdout) instead of 1:
dup2(fd, fileno(stdout));
// and
close(fileno(stdout)); // not needed really
The int fileno(FILE*) function returns the internal file descriptor from the standard FILE* that is stdout. It's just a way of making it clearer to readers of the code.
You could however redirect stdout to a file using the standard C++ function std::freopen instead.
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <cstdio> // std::freopen
#include <iostream>
void execute() {
std::cout << "smash pid is " << getpid() << std::endl;
system("echo the redirect is inherited by the grand children too");
}
int main() {
pid_t pid = fork();
if(pid < 0) { // check for errors
std::perror("fork failed");
return 1;
}
if(pid == 0) { // child
// using std::freopen
if(std::freopen("my_file.txt", "w", stdout) == nullptr) {
std::perror("freopen failed");
return 1;
}
execute();
return 0;
}
// parent
std::cout << "Parent says hello!\n";
// wait for child
int wstatus;
if(waitpid(pid, &wstatus, 0) == pid) {
std::cout << "child exited with status " << wstatus << '\n';
} else {
std::perror("waitpid failed");
}
}

How to avoid duplicate input on stdin when using 2-process pipeline with stdin and stdout redirect

I am intending to set up a pipeline between two processes: parent and child. The parent forks the child and uses execve to replace its image with that of a specified process.
The parent reads from stdin via std::getline(std::cin, input_line).
The child writes to the stdout via std::cout << output_line.
I am looking to setup a pipe and redirect the output of the child to the input of the parent.
The problem is that the parent receives each input (where each input is a number output by the child on stdout) twice. I would like to fix this issue but I don't understand why it is happening.
Code is compiled with g++ 7.4.0 and C++11 standard version.
Child is compiled to a binary called 'p1'.
Parent code:
#include <sys/types.h>
#include <sys/wait.h>
#include <unistd.h>
#include <iostream>
char *
const p1argv[] = {
(char * )
"./p1",
nullptr
};
char *
const p1envp[] = {
(char * ) nullptr
};
int main(int argc, char ** argv) {
pid_t p1id;
int p1fd[2];
pipe(p1fd);
if (p1id = fork() == 0) {
close(p1fd[0]);
dup2(p1fd[1], STDOUT_FILENO);
execve(argv[0], p1argv, p1envp);
perror("Error: failed to execve ./p1.");
} else {
dup2(p1fd[0], STDIN_FILENO);
close(p1fd[1]);
std::string line;
while (std::getline(std::cin, line)) {
std::cout << "d(" << line << ")" << std::endl;
}
int status;
waitpid(p1id, & status, 0);
close(p1fd[0]);
}
}
Child code:
#include <iostream>
#include <thread>
int main(int argc, char** argv) {
long it = 0;
while(true) {
it += 1;
std::cout << std::to_string(it) << std::endl;
std::this_thread::sleep_for(std::chrono::seconds(2));
}
}
The actual output for the sample code is:
d(d(1))
d(d(2))
...
The expected output is:
d(1)
d(2)
...
The problem is that this line:
execve(argv[0], p1argv, p1envp);
Is re-executing the main parent program, because that is what the content of argv[0] is at this point. I think you want to find some way to specify "./p1" there.

Creating (or passing) a string from a pipe

Alright, I'm working on a "simple" project of using forks and piping to search a file for the number of occurrences of a given word. The program is to fork the process in a manner where the parent sends a word at a time to the child for the child to then search through a file and sum up the occurrences of the passed word. My issue is that I'm not familiar with C++ and therefore am having a hard time figuring out how to get the string out of the pipe. The program is currently passing the words through the pipe, but they are only coming out as a long line of characters at run time. Can anyone provide some examples or tips for retrieving as a string and not individual characters? Here is my code as of now:
int main (int argc, char * argv[]) {
fstream fileWell ("well.txt");
fstream fileWords ("words.txt");
int pipefd[2];
int counter = 0;
pid_t cpid;
char buf;
const char * sentWord;
string toCheck;
string toSend;
pipe(pipefd);
cpid = fork();
if (cpid == -1) {
cout << "ERROR" << endl;
exit(1);
}
if (cpid == 0) {
close(pipefd[1]);
while (read(pipefd[0], &buf, 1) > 0)
write(1, &buf, 1);
cout << endl;
write(1, "\n", 1);
}
else {
while (getline(fileWords, toSend)) {
close(pipefd[0]);
sentWord = toSend.c_str();
write(pipefd[1], sentWord, strlen(sentWord));
}
close(pipefd[0]);
toSend = "-1";
sentWord = toSend.c_str();
write(pipefd[1], sentWord, 3);
close(pipefd[1]);
wait(NULL);
exit(EXIT_SUCCESS);
}
return 0;
}
I feel like I know what to do once I've gotten the string, but I can't really move forward without that part. Thank you for any suggestions or help.
Pulling out the unused data and concentrating on just the purpose of your functional code, I'm fairly certain the following was what you were at least trying to accomplish. Some things that were addressed.
Input file stream not opened unless on parent process only.
Fixed multiple closures on pipe handles.
Use std:string members for data pointers and length calculation
Ensure the string terminator was sent as part of the data package.
Treat a terminator on the child as a signal to finish the string.
What you do with the words you receive is up to you. I adapted this to simply send them to standard out before resetting. I feel you likely have alternate plans, but that is somewhat unrelated to the question.
See below. Adapt as needed:
#include <cstdlib>
#include <iostream>
#include <fstream>
#include <string>
#include <unistd.h>
using namespace std;
#define READ_FD 0
#define WRITE_FD 1
int main (int argc, char * argv[])
{
int fd[2];
pid_t cpid;
pipe(fd);
if ((cpid = fork()) == -1)
{
cout << "ERROR" << endl;
exit(1);
}
// child process
if (cpid == 0)
{
// don't need the write-side of this
close(fd[WRITE_FD]);
std::string s;
char ch;
while (read(fd[READ_FD], &ch, 1) > 0)
{
if (ch != 0)
s.push_back(ch);
else
{
std::cout << s << '\n';
s.clear();
}
}
// finished with read-side
close(fd[READ_FD]);
}
// parent process
else
{
// don't need the read-side of this
close(fd[READ_FD]);
fstream fileWords ("words.txt");
string toSend;
while (fileWords >> toSend)
{
// send word including terminator
write(fd[WRITE_FD], toSend.c_str(), toSend.length()+1);
}
// finished with write-side
close(fd[WRITE_FD]);
wait(NULL);
}
return EXIT_SUCCESS;
}
Test
The word file I sent through this was Billy Shakespeare's monologue from As You Like It, Act II, Scene VII. The beginning and ending of the output appear below:
All
the
worlds
a
stage
And
all
the
men
and
women
merely
players
....
oblivion
Sans
teeth
sans
eyes
sans
taste
sans
everything
Alternative: A Custom Stream Buffer
An alternative (and perhaps what you're really looking for) is to adapt a stream buffer that can be married to a std::istream to use on the client side similarly to regular stream io. The simplest example I can muster, a one-char-buffer streambuf, appears below:
// adapt a single char streambuf for an input stream
class ifd_streambuf : public std::streambuf
{
protected:
int d_fd;
char d_buffer[1];
public:
ifd_streambuf(int fd) : d_fd(fd)
{
setg(d_buffer, d_buffer + 1, d_buffer + 1);
};
private:
int underflow()
{
if (read(d_fd, d_buffer, 1) <= 0)
return EOF;
setg(d_buffer, d_buffer, d_buffer + 1);
return *gptr();
}
};
Utilizing this, the client process code segment from the previous source listing can be adapted to simply following:
ifd_streambuf fdbuf(fd[READ_FD]);
std::istream inf(&fdbuf);
std::string s;
while (inf >> s)
std::cout << s << '\n';
which is considerably more the style of C++ you're likely accustom to. The server side would need to change as well, appending any whitespace as the word-separator:
while (fileWords >> toSend)
{
write(fd[WRITE_FD], toSend.c_str(), toSend.length());
write(fd[WRITE_FD], " ", 1);
}
It would take extra work to adapt this streambuf to buffer more than a single character (some extra housekeeping stuff), but I leave that for you to discover.

How to tail a log file in C++? [duplicate]

I want to create a small code in C++ with the same functionality as "tail-f": watch for new lines in a text file and show them in the standard output.
The idea is to have a thread that monitors the file
Is there an easy way to do it without opening and closing the file each time?
Have a look at inotify on Linux or kqueue on Mac OS.
Inotify is Linux kernel subsystem that allows you to subscribe for events on files and it will report to your application when the even happened on your file.
Just keep reading the file. If the read fails, do nothing. There's no need to repeatedly open and close it. However, you will find it much more efficient to use operating system specific features to monitor the file, should your OS provide them.
Same as in https://stackoverflow.com/a/7514051/44729 except that the code below uses getline instead of getc and doesn't skip new lines
#include <iostream>
#include <string>
#include <fstream>
#include <sstream>
using namespace std;
static int last_position=0;
// read file untill new line
// save position
int find_new_text(ifstream &infile) {
infile.seekg(0,ios::end);
int filesize = infile.tellg();
// check if the new file started
if(filesize < last_position){
last_position=0;
}
// read file from last position untill new line is found
for(int n=last_position;n<filesize;n++) {
infile.seekg( last_position,ios::beg);
char test[256];
infile.getline(test, 256);
last_position = infile.tellg();
cout << "Char: " << test <<"Last position " << last_position<< endl;
// end of file
if(filesize == last_position){
return filesize;
}
}
return 0;
}
int main() {
for(;;) {
std::ifstream infile("filename");
int current_position = find_new_text(infile);
sleep(1);
}
}
I read this in one of Perl manuals, but it is easily translated into standard C, which, in turn, can be translated to istreams.
seek FILEHANDLE,POSITION,WHENCE
Sets FILEHANDLE's position, just like the "fseek" call of
"stdio".
<...>
A WHENCE of 1 ("SEEK_CUR") is useful for not moving the file
position:
seek(TEST,0,1);
This is also useful for applications emulating "tail -f". Once
you hit EOF on your read, and then sleep for a while, you might
have to stick in a seek() to reset things. The "seek" doesn't
change the current position, but it does clear the end-of-file
condition on the handle, so that the next "<FILE>" makes Perl
try again to read something. We hope.
As far as I remember, fseek is called iostream::seekg. So you should basically do the same: seek to the end of the file, then sleep and seek again with ios_base::cur flag to update end-of-file and read some more data.
Instead of sleeping, you may use inotify, as suggested in the other answer, to sleep (block while reading from an emulated file, actually) exactly until the file is updated/closed. But that's Linux-specific, and is not standard C++.
I needed to implement this too, I just wrote a quick hack in standard C++. The hack searches for the last 0x0A (linefeed character) in a file and outputs all data following that linefeed when the last linefeed value becomes a larger value. The code is here:
#include <iostream>
#include <string>
#include <fstream>
using namespace std;
int find_last_linefeed(ifstream &infile) {
infile.seekg(0,ios::end);
int filesize = infile.tellg();
for(int n=1;n<filesize;n++) {
infile.seekg(filesize-n-1,ios::beg);
char c;
infile.get(c);
if(c == 0x0A) return infile.tellg();
}
}
int main() {
int last_position=-1;
for(;;) {
ifstream infile("testfile");
int position = find_last_linefeed(infile);
if(position > last_position) {
infile.seekg(position,ios::beg);
string in;
infile >> in;
cout << in << endl;
}
last_position=position;
sleep(1);
}
}
#include <iostream>
#include <fstream>
#include <string>
#include <list>
#include <sys/stat.h>
#include <stdlib.h>
#define debug 0
class MyTail
{
private:
std::list<std::string> mLastNLine;
const int mNoOfLines;
std::ifstream mIn;
public:
explicit MyTail(int pNoOfLines):mNoOfLines(pNoOfLines) {}
const int getNoOfLines() {return mNoOfLines; }
void getLastNLines();
void printLastNLines();
void tailF(const char* filename);
};
void MyTail::getLastNLines()
{
if (debug) std::cout << "In: getLastNLines()" << std::endl;
mIn.seekg(-1,std::ios::end);
int pos=mIn.tellg();
int count = 1;
//Get file pointer to point to bottom up mNoOfLines.
for(int i=0;i<pos;i++)
{
if (mIn.get() == '\n')
if (count++ > mNoOfLines)
break;
mIn.seekg(-2,std::ios::cur);
}
//Start copying bottom mNoOfLines string into list to avoid I/O calls to print lines
std::string line;
while(getline(mIn,line)) {
int data_Size = mLastNLine.size();
if(data_Size >= mNoOfLines) {
mLastNLine.pop_front();
}
mLastNLine.push_back(line);
}
if (debug) std::cout << "Out: getLastNLines()" << std::endl;
}
void MyTail::printLastNLines()
{
for (std::list<std::string>::iterator i = mLastNLine.begin(); i != mLastNLine.end(); ++i)
std::cout << *i << std::endl;
}
void MyTail::tailF(const char* filename)
{
if (debug) std::cout << "In: TailF()" << std::endl;
int date = 0;
while (true) {
struct stat st;
stat (filename, &st);
int newdate = st.st_mtime;
if (newdate != date){
system("#cls||clear");
std::cout << "Print last " << getNoOfLines() << " Lines: \n";
mIn.open(filename);
date = newdate;
getLastNLines();
mIn.close();
printLastNLines();
}
}
if (debug) std::cout << "Out: TailF()" << std::endl;
}
int main(int argc, char **argv)
{
if(argc==1) {
std::cout << "No Extra Command Line Argument Passed Other Than Program Name\n";
return 0;
}
if(argc>=2) {
MyTail t1(10);
t1.tailF(argv[1]);
}
return 0;
}

Implement "tail -f" in C++

I want to create a small code in C++ with the same functionality as "tail-f": watch for new lines in a text file and show them in the standard output.
The idea is to have a thread that monitors the file
Is there an easy way to do it without opening and closing the file each time?
Have a look at inotify on Linux or kqueue on Mac OS.
Inotify is Linux kernel subsystem that allows you to subscribe for events on files and it will report to your application when the even happened on your file.
Just keep reading the file. If the read fails, do nothing. There's no need to repeatedly open and close it. However, you will find it much more efficient to use operating system specific features to monitor the file, should your OS provide them.
Same as in https://stackoverflow.com/a/7514051/44729 except that the code below uses getline instead of getc and doesn't skip new lines
#include <iostream>
#include <string>
#include <fstream>
#include <sstream>
using namespace std;
static int last_position=0;
// read file untill new line
// save position
int find_new_text(ifstream &infile) {
infile.seekg(0,ios::end);
int filesize = infile.tellg();
// check if the new file started
if(filesize < last_position){
last_position=0;
}
// read file from last position untill new line is found
for(int n=last_position;n<filesize;n++) {
infile.seekg( last_position,ios::beg);
char test[256];
infile.getline(test, 256);
last_position = infile.tellg();
cout << "Char: " << test <<"Last position " << last_position<< endl;
// end of file
if(filesize == last_position){
return filesize;
}
}
return 0;
}
int main() {
for(;;) {
std::ifstream infile("filename");
int current_position = find_new_text(infile);
sleep(1);
}
}
I read this in one of Perl manuals, but it is easily translated into standard C, which, in turn, can be translated to istreams.
seek FILEHANDLE,POSITION,WHENCE
Sets FILEHANDLE's position, just like the "fseek" call of
"stdio".
<...>
A WHENCE of 1 ("SEEK_CUR") is useful for not moving the file
position:
seek(TEST,0,1);
This is also useful for applications emulating "tail -f". Once
you hit EOF on your read, and then sleep for a while, you might
have to stick in a seek() to reset things. The "seek" doesn't
change the current position, but it does clear the end-of-file
condition on the handle, so that the next "<FILE>" makes Perl
try again to read something. We hope.
As far as I remember, fseek is called iostream::seekg. So you should basically do the same: seek to the end of the file, then sleep and seek again with ios_base::cur flag to update end-of-file and read some more data.
Instead of sleeping, you may use inotify, as suggested in the other answer, to sleep (block while reading from an emulated file, actually) exactly until the file is updated/closed. But that's Linux-specific, and is not standard C++.
I needed to implement this too, I just wrote a quick hack in standard C++. The hack searches for the last 0x0A (linefeed character) in a file and outputs all data following that linefeed when the last linefeed value becomes a larger value. The code is here:
#include <iostream>
#include <string>
#include <fstream>
using namespace std;
int find_last_linefeed(ifstream &infile) {
infile.seekg(0,ios::end);
int filesize = infile.tellg();
for(int n=1;n<filesize;n++) {
infile.seekg(filesize-n-1,ios::beg);
char c;
infile.get(c);
if(c == 0x0A) return infile.tellg();
}
}
int main() {
int last_position=-1;
for(;;) {
ifstream infile("testfile");
int position = find_last_linefeed(infile);
if(position > last_position) {
infile.seekg(position,ios::beg);
string in;
infile >> in;
cout << in << endl;
}
last_position=position;
sleep(1);
}
}
#include <iostream>
#include <fstream>
#include <string>
#include <list>
#include <sys/stat.h>
#include <stdlib.h>
#define debug 0
class MyTail
{
private:
std::list<std::string> mLastNLine;
const int mNoOfLines;
std::ifstream mIn;
public:
explicit MyTail(int pNoOfLines):mNoOfLines(pNoOfLines) {}
const int getNoOfLines() {return mNoOfLines; }
void getLastNLines();
void printLastNLines();
void tailF(const char* filename);
};
void MyTail::getLastNLines()
{
if (debug) std::cout << "In: getLastNLines()" << std::endl;
mIn.seekg(-1,std::ios::end);
int pos=mIn.tellg();
int count = 1;
//Get file pointer to point to bottom up mNoOfLines.
for(int i=0;i<pos;i++)
{
if (mIn.get() == '\n')
if (count++ > mNoOfLines)
break;
mIn.seekg(-2,std::ios::cur);
}
//Start copying bottom mNoOfLines string into list to avoid I/O calls to print lines
std::string line;
while(getline(mIn,line)) {
int data_Size = mLastNLine.size();
if(data_Size >= mNoOfLines) {
mLastNLine.pop_front();
}
mLastNLine.push_back(line);
}
if (debug) std::cout << "Out: getLastNLines()" << std::endl;
}
void MyTail::printLastNLines()
{
for (std::list<std::string>::iterator i = mLastNLine.begin(); i != mLastNLine.end(); ++i)
std::cout << *i << std::endl;
}
void MyTail::tailF(const char* filename)
{
if (debug) std::cout << "In: TailF()" << std::endl;
int date = 0;
while (true) {
struct stat st;
stat (filename, &st);
int newdate = st.st_mtime;
if (newdate != date){
system("#cls||clear");
std::cout << "Print last " << getNoOfLines() << " Lines: \n";
mIn.open(filename);
date = newdate;
getLastNLines();
mIn.close();
printLastNLines();
}
}
if (debug) std::cout << "Out: TailF()" << std::endl;
}
int main(int argc, char **argv)
{
if(argc==1) {
std::cout << "No Extra Command Line Argument Passed Other Than Program Name\n";
return 0;
}
if(argc>=2) {
MyTail t1(10);
t1.tailF(argv[1]);
}
return 0;
}