Alright, I'm working on a "simple" project of using forks and piping to search a file for the number of occurrences of a given word. The program is to fork the process in a manner where the parent sends a word at a time to the child for the child to then search through a file and sum up the occurrences of the passed word. My issue is that I'm not familiar with C++ and therefore am having a hard time figuring out how to get the string out of the pipe. The program is currently passing the words through the pipe, but they are only coming out as a long line of characters at run time. Can anyone provide some examples or tips for retrieving as a string and not individual characters? Here is my code as of now:
int main (int argc, char * argv[]) {
fstream fileWell ("well.txt");
fstream fileWords ("words.txt");
int pipefd[2];
int counter = 0;
pid_t cpid;
char buf;
const char * sentWord;
string toCheck;
string toSend;
pipe(pipefd);
cpid = fork();
if (cpid == -1) {
cout << "ERROR" << endl;
exit(1);
}
if (cpid == 0) {
close(pipefd[1]);
while (read(pipefd[0], &buf, 1) > 0)
write(1, &buf, 1);
cout << endl;
write(1, "\n", 1);
}
else {
while (getline(fileWords, toSend)) {
close(pipefd[0]);
sentWord = toSend.c_str();
write(pipefd[1], sentWord, strlen(sentWord));
}
close(pipefd[0]);
toSend = "-1";
sentWord = toSend.c_str();
write(pipefd[1], sentWord, 3);
close(pipefd[1]);
wait(NULL);
exit(EXIT_SUCCESS);
}
return 0;
}
I feel like I know what to do once I've gotten the string, but I can't really move forward without that part. Thank you for any suggestions or help.
Pulling out the unused data and concentrating on just the purpose of your functional code, I'm fairly certain the following was what you were at least trying to accomplish. Some things that were addressed.
Input file stream not opened unless on parent process only.
Fixed multiple closures on pipe handles.
Use std:string members for data pointers and length calculation
Ensure the string terminator was sent as part of the data package.
Treat a terminator on the child as a signal to finish the string.
What you do with the words you receive is up to you. I adapted this to simply send them to standard out before resetting. I feel you likely have alternate plans, but that is somewhat unrelated to the question.
See below. Adapt as needed:
#include <cstdlib>
#include <iostream>
#include <fstream>
#include <string>
#include <unistd.h>
using namespace std;
#define READ_FD 0
#define WRITE_FD 1
int main (int argc, char * argv[])
{
int fd[2];
pid_t cpid;
pipe(fd);
if ((cpid = fork()) == -1)
{
cout << "ERROR" << endl;
exit(1);
}
// child process
if (cpid == 0)
{
// don't need the write-side of this
close(fd[WRITE_FD]);
std::string s;
char ch;
while (read(fd[READ_FD], &ch, 1) > 0)
{
if (ch != 0)
s.push_back(ch);
else
{
std::cout << s << '\n';
s.clear();
}
}
// finished with read-side
close(fd[READ_FD]);
}
// parent process
else
{
// don't need the read-side of this
close(fd[READ_FD]);
fstream fileWords ("words.txt");
string toSend;
while (fileWords >> toSend)
{
// send word including terminator
write(fd[WRITE_FD], toSend.c_str(), toSend.length()+1);
}
// finished with write-side
close(fd[WRITE_FD]);
wait(NULL);
}
return EXIT_SUCCESS;
}
Test
The word file I sent through this was Billy Shakespeare's monologue from As You Like It, Act II, Scene VII. The beginning and ending of the output appear below:
All
the
worlds
a
stage
And
all
the
men
and
women
merely
players
....
oblivion
Sans
teeth
sans
eyes
sans
taste
sans
everything
Alternative: A Custom Stream Buffer
An alternative (and perhaps what you're really looking for) is to adapt a stream buffer that can be married to a std::istream to use on the client side similarly to regular stream io. The simplest example I can muster, a one-char-buffer streambuf, appears below:
// adapt a single char streambuf for an input stream
class ifd_streambuf : public std::streambuf
{
protected:
int d_fd;
char d_buffer[1];
public:
ifd_streambuf(int fd) : d_fd(fd)
{
setg(d_buffer, d_buffer + 1, d_buffer + 1);
};
private:
int underflow()
{
if (read(d_fd, d_buffer, 1) <= 0)
return EOF;
setg(d_buffer, d_buffer, d_buffer + 1);
return *gptr();
}
};
Utilizing this, the client process code segment from the previous source listing can be adapted to simply following:
ifd_streambuf fdbuf(fd[READ_FD]);
std::istream inf(&fdbuf);
std::string s;
while (inf >> s)
std::cout << s << '\n';
which is considerably more the style of C++ you're likely accustom to. The server side would need to change as well, appending any whitespace as the word-separator:
while (fileWords >> toSend)
{
write(fd[WRITE_FD], toSend.c_str(), toSend.length());
write(fd[WRITE_FD], " ", 1);
}
It would take extra work to adapt this streambuf to buffer more than a single character (some extra housekeeping stuff), but I leave that for you to discover.
Related
I'm trying to invert a string in C++ using fork(), such that each process prints at most one character. My thinking is that after printing each character, I fork into a new process, end the parent process, and continue. Here is my code:
#include <string>
#include <iostream>
#include <unistd.h>
/*
Recursively print one character at a time,
each in a separate process.
*/
void print_char(std::string str, int index, pid_t pid)
{
/*
If this is the same process,
or the beginning of the string has been reached, quit.
*/
if (pid != 0 || index <= -1)
return;
std::cout << str[index];
if (index == 0)
{
std::cout << std::endl;
return;
}
print_char(str, index-1, fork());
}
int main(int argc, char** argv)
{
std::string str(argv[1]);
print_char(str, str.length()-1, 0);
}
However, when testing the code with the argument "hey", it prints "yeheyy". My understanding of fork() is that it creates a duplicate process with a copy of the memory space, and whenever I mentally "walk through" the code it seems like it should work, but I cannot figure out where my logic is failing.
It seems, your code is OK, but you have trouble with cout.
try change only the output line
std::cout << str[index];
with
std::cout << str[index] << std::flush;
tried it and worked for me.
I have created a pseudo terminal in C++ using the following code:
int main(int, char const *[])
{
int master, slave;
char name[1024];
char mode[] = "0777"; //I know this isn't good, it is for testing at the moment
int access;
int e = openpty(&master, &slave, &name[0], 0, 0);
if(0 > e) {
std::printf("Error: %s\n", strerror(errno));
return -1;
}
if( 0 != unlockpt(slave) )
{
perror("Slave Error");
}
access = strtol(mode, 0, 8);
if( 0 > chmod(name, access) )
{
perror("Permission Error");
}
//std::cout << "Master: " << master << std::endl;
std::printf("Slave PTY: %s\n", name);
int r;
prompt = "login: ";
while(true)
{
std::cout << prompt << std::flush;
r = read(master, &name[0], sizeof(name)-1);
checkInput(name);
name[r] = '\0';
std::printf("%s", &name[0]);
std::printf("\n");
}
close(slave);
close(master);
return 0;
}
It works pretty well in the sense that from another terminal, I can do:
printf 'username' > /dev/pts/x
and it will appear and be processed as it should.
My question is: when I try to use screen, nothing appears on the screen terminal. Then when I type, it comes through to my slave 1 character at a time.
Does anyone know why this is? Or how I can fix it.
I can provide more detail if required.
Thank you :)
Because you're not flushing the buffer after you use printf.
As pauls answer already suggest you need to flush the buffer.
To do so you can use the tcflush function.
The first argument is the int of the file descriptor and the second can be one of the following:
TCIFLUSH Flushes input data that has been received by the system but
not read by an application.
TCOFLUSH Flushes output data that has been written by an application
but not sent to the terminal.
TCIOFLUSH Flushes both input and output data.
For more information see: https://www.ibm.com/docs/en/zos/2.3.0?topic=functions-tcflush-flush-input-output-terminal
I have got a pipe to enable communication between 2 processes in forked program. It was created with pipe() call - http://linux.die.net/man/2/pipe
. Everything goes right until I want to perform some file operations.
This code works:
pipe.writeBuffer(message.c_str(), message.length());
ofstream file;
file.open(name.c_str(), ios::app);
file << "stringData"; // put some data to file (many times)
But this one not:
ofstream file;
file.open(name.c_str(), ios::app);
pipe.writeBuffer(message.c_str(), message.length());
file << "stringData"; // put some data to file (many times)
In the second example there is no effect of "file << someStream" - I get the empty file.
What is wrong with that? Is it a problem with file descriptor? Pipe uses fd[0] - input and fd[1] - output. Maybe fstream uses also the same output file handler?
Here is the "working" sample:
http://pastebin.com/gJ4PbHvy
#include <sys/types.h>
#include <cstdlib>
#include <unistd.h>
#include <iostream>
#include <fstream>
#define maxSize 64
using namespace std;
class Pipe
{
public:
Pipe()
{
pipe(fdesc);
}
~Pipe() {}
void writeBuffer(const char* message, size_t length)
{
close(fdesc[0]);
write(fdesc[1], message, length);
}
void readBuffer()
{
char buffer[maxSize];
close(fdesc[1]);
size_t result = read(fdesc[0], &buffer, sizeof(buffer));
cout << buffer << endl;
}
private:
int fdesc[2];
};
class Writer
{
public:
Writer(Pipe &obj)
{
pipe = obj;
}
~Writer()
{}
void entry()
{
std::string name = "./myFile";
ofstream file;
file.open(name.c_str(), ios::app);
std::string message = "hello world";
pipe.writeBuffer(message.c_str(), message.length()+1);
if (file.is_open())
{
file << "Hello World!" << endl;
file.close();
}
else
{
perror("file.is_open()");
}
sleep(1);
}
private:
Pipe pipe;
};
class Reader
{
public:
Reader(Pipe &obj)
{
pipe = obj;
}
~Reader()
{}
void entry()
{
pipe.readBuffer();
sleep(1);
}
private:
Pipe pipe;
};
int main(int argc, char *argv[])
{
Pipe pipe;
Reader reader(pipe);
Writer writer(pipe);
pid_t pid = fork();
if (pid == -1)
{
perror("fork");
exit(EXIT_FAILURE);
}
if (pid == 0)
{
// child process
while(1)
reader.entry();
}
else
{
// parent process
while(1)
writer.entry();
}
}
With the posted program, the described issue of getting an empty file isn't reproducible, since on each run it does write one line Hello World! into myFile, but this still shows the error, because you intend to write one line every second. The reason is the close(fdesc[0]) in writeBuffer(): While it is correct to close the read end of the pipe once in the writer process, it is incorrect to do it every time writeBuffer() is called, as that file descriptor can (and in the case at hand, does) have been reused after the first close() for another file (here the file ofstream file), which is after it closed instead of the (already closed) pipe, so that nothing can be written into the file. Fix: Arrange your program to close the pipe end only once, e. g. by changing
close(fdesc[0]);
to
if (0 <= fdesc[0]) close(fdesc[0]), fdesc[0] = -1;
This question follows from my attempt to implement the instructions in:
Linux Pipes as Input and Output
How to send a simple string between two programs using pipes?
http://tldp.org/LDP/lpg/node11.html
My question is along the lines of the question in: Linux Pipes as Input and Output, but more specific.
Essentially, I am trying to replace:
/directory/program < input.txt > output.txt
using pipes in C++ in order to avoid using the hard drive. Here's my code:
//LET THE PLUMBING BEGIN
int fd_p2c[2], fd_pFc[2], bytes_read;
// "p2c" = pipe_to_child, "pFc" = pipe_from_child (see above link)
pid_t childpid;
char readbuffer[80];
string program_name;// <---- includes program name + full path
string gulp_command;// <---- includes my line-by-line stdin for program execution
string receive_output = "";
pipe(fd_p2c);//create pipe-to-child
pipe(fd_pFc);//create pipe-from-child
childpid = fork();//create fork
if (childpid < 0)
{
cout << "Fork failed" << endl;
exit(-1);
}
else if (childpid == 0)
{
dup2(0,fd_p2c[0]);//close stdout & make read end of p2c into stdout
close(fd_p2c[0]);//close read end of p2c
close(fd_p2c[1]);//close write end of p2c
dup2(1,fd_pFc[1]);//close stdin & make read end of pFc into stdin
close(fd_pFc[1]);//close write end of pFc
close(fd_pFc[0]);//close read end of pFc
//Execute the required program
execl(program_name.c_str(),program_name.c_str(),(char *) 0);
exit(0);
}
else
{
close(fd_p2c[0]);//close read end of p2c
close(fd_pFc[1]);//close write end of pFc
//"Loop" - send all data to child on write end of p2c
write(fd_p2c[1], gulp_command.c_str(), (strlen(gulp_command.c_str())));
close(fd_p2c[1]);//close write end of p2c
//Loop - receive all data to child on read end of pFc
while (1)
{
bytes_read = read(fd_pFc[0], readbuffer, sizeof(readbuffer));
if (bytes_read <= 0)//if nothing read from buffer...
break;//...break loop
receive_output += readbuffer;//append data to string
}
close(fd_pFc[0]);//close read end of pFc
}
I am absolutely sure that the above strings are initialized properly. However, two things happen that don't make sense to me:
(1) The program I am executing reports that the "input file is empty." Since I am not calling the program with "<" it should not be expecting an input file. Instead, it should be expecting keyboard input. Furthermore, it should be reading the text contained in "gulp_command."
(2) The program's report (provided via standard output) appears in the terminal. This is odd because the purpose of this piping is to transfer stdout to my string "receive_output." But since it is appearing on screen, that indicates to me that the information is not being passed correctly through the pipe to the variable. If I implement the following at the end of the if statement,
cout << receive_output << endl;
I get nothing, as though the string is empty. I appreciate any help you can give me!
EDIT: Clarification
My program currently communicates with another program using text files. My program writes a text file (e.g. input.txt), which is read by the external program. That program then produces output.txt, which is read by my program. So it's something like this:
my code -> input.txt -> program -> output.txt -> my code
Therefore, my code currently uses,
system("program < input.txt > output.txt");
I want to replace this process using pipes. I want to pass my input as standard input to the program, and have my code read the standard output from that program into a string.
Your primary problem is that you have the arguments to dup2() reversed. You need to use:
dup2(fd_p2c[0], 0); // Duplicate read end of pipe to standard input
dup2(fd_pFc[1], 1); // Duplicate write end of pipe to standard output
I got suckered into misreading what you wrote as OK until I put error checking on the set-up code and got unexpected values from the dup2() calls, which told me what the trouble was. When something goes wrong, insert the error checks you skimped on before.
You also did not ensure null termination of the data read from the child; this code does.
Working code (with diagnostics), using cat as the simplest possible 'other command':
#include <unistd.h>
#include <string>
#include <iostream>
using namespace std;
int main()
{
int fd_p2c[2], fd_c2p[2], bytes_read;
pid_t childpid;
char readbuffer[80];
string program_name = "/bin/cat";
string gulp_command = "this is the command data sent to the child cat (kitten?)";
string receive_output = "";
if (pipe(fd_p2c) != 0 || pipe(fd_c2p) != 0)
{
cerr << "Failed to pipe\n";
exit(1);
}
childpid = fork();
if (childpid < 0)
{
cout << "Fork failed" << endl;
exit(-1);
}
else if (childpid == 0)
{
if (dup2(fd_p2c[0], 0) != 0 ||
close(fd_p2c[0]) != 0 ||
close(fd_p2c[1]) != 0)
{
cerr << "Child: failed to set up standard input\n";
exit(1);
}
if (dup2(fd_c2p[1], 1) != 1 ||
close(fd_c2p[1]) != 0 ||
close(fd_c2p[0]) != 0)
{
cerr << "Child: failed to set up standard output\n";
exit(1);
}
execl(program_name.c_str(), program_name.c_str(), (char *) 0);
cerr << "Failed to execute " << program_name << endl;
exit(1);
}
else
{
close(fd_p2c[0]);
close(fd_c2p[1]);
cout << "Writing to child: <<" << gulp_command << ">>" << endl;
int nbytes = gulp_command.length();
if (write(fd_p2c[1], gulp_command.c_str(), nbytes) != nbytes)
{
cerr << "Parent: short write to child\n";
exit(1);
}
close(fd_p2c[1]);
while (1)
{
bytes_read = read(fd_c2p[0], readbuffer, sizeof(readbuffer)-1);
if (bytes_read <= 0)
break;
readbuffer[bytes_read] = '\0';
receive_output += readbuffer;
}
close(fd_c2p[0]);
cout << "From child: <<" << receive_output << ">>" << endl;
}
return 0;
}
Sample output:
Writing to child: <<this is the command data sent to the child cat (kitten?)>>
From child: <<this is the command data sent to the child cat (kitten?)>>
Note that you will need to be careful to ensure you don't get deadlocked with your code. If you have a strictly synchronous protocol (so the parent writes a message and reads a response in lock-step), you should be fine, but if the parent is trying to write a message that's too big to fit in the pipe to the child while the child is trying to write a message that's too big to fit in the pipe back to the parent, then each will be blocked writing while waiting for the other to read.
It sounds like you're looking for coprocesses. You can program them in C/C++ but since they are already available in the (bash) shell, easier to use the shell, right?
First start the external program with the coproc builtin:
coproc external_program
The coproc starts the program in the background and stores the file descriptors to communicate with it in an array shell variable. Now you just need to start your program connecting it to those file descriptors:
your_program <&${COPROC[0]} >&${COPROC[1]}
#include <stdio.h>
#include <unistd.h>
#include <sys/stat.h>
#include <sys/wait.h>
#include <fcntl.h>
#include <string.h>
#include <iostream>
using namespace std;
int main() {
int i, status, len;
char str[10];
mknod("pipe", S_IFIFO | S_IRUSR | S_IWUSR, 0); //create named pipe
pid_t pid = fork(); // create new process
/* Process A */
if (pid == 0) {
int myPipe = open("pipe", O_WRONLY); // returns a file descriptor for the pipe
cout << "\nThis is process A having PID= " << getpid(); //Get pid of process A
cout << "\nEnter the string: ";
cin >> str;
len = strlen(str);
write(myPipe, str, len); //Process A write to the named pipe
cout << "Process A sent " << str;
close(myPipe); //closes the file descriptor fields.
}
/* Process B */
else {
int myPipe = open("pipe", O_RDONLY); //Open the pipe and returns file descriptor
char buffer[21];
int pid_child;
pid_child = wait(&status); //wait until any one child process terminates
int length = read(myPipe, buffer, 20); //reads up to size bytes from pipe with descriptor fields, store results
// in buffer;
cout<< "\n\nThis is process B having PID= " << getpid();//Get pid of process B
buffer[length] = '\0';
cout << "\nProcess B received " << buffer;
i = 0;
//Reverse the string
for (length = length - 1; length >= 0; length--)
str[i++] = buffer[length];
str[i] = '\0';
cout << "\nRevers of string is " << str;
close(myPipe);
}
unlink("pipe");
return 0;
}
I want to create a small code in C++ with the same functionality as "tail-f": watch for new lines in a text file and show them in the standard output.
The idea is to have a thread that monitors the file
Is there an easy way to do it without opening and closing the file each time?
Have a look at inotify on Linux or kqueue on Mac OS.
Inotify is Linux kernel subsystem that allows you to subscribe for events on files and it will report to your application when the even happened on your file.
Just keep reading the file. If the read fails, do nothing. There's no need to repeatedly open and close it. However, you will find it much more efficient to use operating system specific features to monitor the file, should your OS provide them.
Same as in https://stackoverflow.com/a/7514051/44729 except that the code below uses getline instead of getc and doesn't skip new lines
#include <iostream>
#include <string>
#include <fstream>
#include <sstream>
using namespace std;
static int last_position=0;
// read file untill new line
// save position
int find_new_text(ifstream &infile) {
infile.seekg(0,ios::end);
int filesize = infile.tellg();
// check if the new file started
if(filesize < last_position){
last_position=0;
}
// read file from last position untill new line is found
for(int n=last_position;n<filesize;n++) {
infile.seekg( last_position,ios::beg);
char test[256];
infile.getline(test, 256);
last_position = infile.tellg();
cout << "Char: " << test <<"Last position " << last_position<< endl;
// end of file
if(filesize == last_position){
return filesize;
}
}
return 0;
}
int main() {
for(;;) {
std::ifstream infile("filename");
int current_position = find_new_text(infile);
sleep(1);
}
}
I read this in one of Perl manuals, but it is easily translated into standard C, which, in turn, can be translated to istreams.
seek FILEHANDLE,POSITION,WHENCE
Sets FILEHANDLE's position, just like the "fseek" call of
"stdio".
<...>
A WHENCE of 1 ("SEEK_CUR") is useful for not moving the file
position:
seek(TEST,0,1);
This is also useful for applications emulating "tail -f". Once
you hit EOF on your read, and then sleep for a while, you might
have to stick in a seek() to reset things. The "seek" doesn't
change the current position, but it does clear the end-of-file
condition on the handle, so that the next "<FILE>" makes Perl
try again to read something. We hope.
As far as I remember, fseek is called iostream::seekg. So you should basically do the same: seek to the end of the file, then sleep and seek again with ios_base::cur flag to update end-of-file and read some more data.
Instead of sleeping, you may use inotify, as suggested in the other answer, to sleep (block while reading from an emulated file, actually) exactly until the file is updated/closed. But that's Linux-specific, and is not standard C++.
I needed to implement this too, I just wrote a quick hack in standard C++. The hack searches for the last 0x0A (linefeed character) in a file and outputs all data following that linefeed when the last linefeed value becomes a larger value. The code is here:
#include <iostream>
#include <string>
#include <fstream>
using namespace std;
int find_last_linefeed(ifstream &infile) {
infile.seekg(0,ios::end);
int filesize = infile.tellg();
for(int n=1;n<filesize;n++) {
infile.seekg(filesize-n-1,ios::beg);
char c;
infile.get(c);
if(c == 0x0A) return infile.tellg();
}
}
int main() {
int last_position=-1;
for(;;) {
ifstream infile("testfile");
int position = find_last_linefeed(infile);
if(position > last_position) {
infile.seekg(position,ios::beg);
string in;
infile >> in;
cout << in << endl;
}
last_position=position;
sleep(1);
}
}
#include <iostream>
#include <fstream>
#include <string>
#include <list>
#include <sys/stat.h>
#include <stdlib.h>
#define debug 0
class MyTail
{
private:
std::list<std::string> mLastNLine;
const int mNoOfLines;
std::ifstream mIn;
public:
explicit MyTail(int pNoOfLines):mNoOfLines(pNoOfLines) {}
const int getNoOfLines() {return mNoOfLines; }
void getLastNLines();
void printLastNLines();
void tailF(const char* filename);
};
void MyTail::getLastNLines()
{
if (debug) std::cout << "In: getLastNLines()" << std::endl;
mIn.seekg(-1,std::ios::end);
int pos=mIn.tellg();
int count = 1;
//Get file pointer to point to bottom up mNoOfLines.
for(int i=0;i<pos;i++)
{
if (mIn.get() == '\n')
if (count++ > mNoOfLines)
break;
mIn.seekg(-2,std::ios::cur);
}
//Start copying bottom mNoOfLines string into list to avoid I/O calls to print lines
std::string line;
while(getline(mIn,line)) {
int data_Size = mLastNLine.size();
if(data_Size >= mNoOfLines) {
mLastNLine.pop_front();
}
mLastNLine.push_back(line);
}
if (debug) std::cout << "Out: getLastNLines()" << std::endl;
}
void MyTail::printLastNLines()
{
for (std::list<std::string>::iterator i = mLastNLine.begin(); i != mLastNLine.end(); ++i)
std::cout << *i << std::endl;
}
void MyTail::tailF(const char* filename)
{
if (debug) std::cout << "In: TailF()" << std::endl;
int date = 0;
while (true) {
struct stat st;
stat (filename, &st);
int newdate = st.st_mtime;
if (newdate != date){
system("#cls||clear");
std::cout << "Print last " << getNoOfLines() << " Lines: \n";
mIn.open(filename);
date = newdate;
getLastNLines();
mIn.close();
printLastNLines();
}
}
if (debug) std::cout << "Out: TailF()" << std::endl;
}
int main(int argc, char **argv)
{
if(argc==1) {
std::cout << "No Extra Command Line Argument Passed Other Than Program Name\n";
return 0;
}
if(argc>=2) {
MyTail t1(10);
t1.tailF(argv[1]);
}
return 0;
}