Non-blocking way to check if there is data on a iofstream - c++

I need to have a way to check if there is data to read on a file (fifo) in a non-blocking way.
I have tried using peek; but it is blocking, I have tried to get and then unget a character in order to check the file without altering the contents; but once again get is blocking...
The only non-blocking solution I have found is to use std::getline(file, line_str) and check if the string is empty; however this does not suit my needs as it alters the data on the file. (The data is a serialized object I will read once I detect there is something to read).
Note: I need this to be non-blocking: I have multiple file streams and need to check all of them regularly to see if there is an object to read/deserialize.
Here is a simple example of what I am trying to achieve:
Sender.cpp:
#include <fstream>
#include <iostream>
#include <string>
extern "C"{
#include <sys/stat.h> // S_IRUSR, S_IWUSR, mkfifo
}
#include <cerrno> // errno
int main(int, char** argv) {
std::string pipe = "foobar";
if(mkfifo(pipe.c_str(), S_IRUSR | S_IWUSR) < 0){
if (errno != EEXIST){
std::cerr << errno;
}
}
std::ofstream file{pipe.c_str()};
file.write("boop", 4); // Simulated object serialization
}
Reader.cpp:
#include <fstream>
#include <iostream>
#include <string>
extern "C"{
#include <sys/stat.h> // S_IRUSR, S_IWUSR, mkfifo
}
#include <cerrno> // errno
int main(int, char** argv) {
std::string pipe = "foobar";
if(mkfifo(pipe.c_str(), S_IRUSR | S_IWUSR) < 0){
if (errno != EEXIST){
std::cerr << errno;
}
}
std::ifstream file{pipe.c_str()};
// ...
/* Do check for data and read/deserialize if any data */
// This is in some sort of loop that goes over the different
// filestreams and checks to see if they have data to treat
}
Any help is really appreciated...
EDIT:
Following Zoso's answer I tried using the file size to determine if the file had been changed; however attempeting to get the size of a fifo named pipe is not possible : filesystem error: cannot get file size: Operation not supported [myFilePath]

I'm not sure if this would work for your particular use case but you could use the filesystem APIs. A simple example is
#include <iostream>
#include <fstream>
#include <filesystem>
namespace fs = std::filesystem;
int main()
{
while (true) {
auto path = fs::current_path().append("test");
std::cout <<"Press enter to know file size of "<<path.c_str() <<'\n';
char c= getchar();
try {
std::cout<<"Size of "<<path.c_str()<<"is "<<fs::file_size(path)<<'\n';
} catch(fs::filesystem_error& e) {
std::cout << e.what() << '\n';
}
}
}
As and when the file gets data, that can be kept track of based on the increasing size and the data to be processed can be tracked as and when that data is consumed.

Related

cin.tellg returns -1 when receiving input from a pipe

I have a case where I need pipe the output of a child process to an ifstream.
I am trying both creating an ifstream from a file descriptor using the method here: How to construct a c++ fstream from a POSIX file descriptor?
and I am also trying to just use a pipe from the child stderr to my own stdin and using cin as my stream.
In both cases I am getting -1 when I call tellg.
Here is my code with the pipe from child stderr to parent stdin:
#include <iostream>
#include <unistd.h>
#include <sys/wait.h>
using namespace std;
int
main()
{
int mypipe[2];
pipe(mypipe);
dup2(mypipe[0], STDIN_FILENO);
dup2(mypipe[1], STDERR_FILENO);
__pid_t pid = fork();
if(pid == 0)
{
// this is a process that outputs stuff into std::err
char* argv[] = {"copy-feats", nullptr};
int ret = execvp(argv[0], argv);
exit(ret);
}
int status;
waitpid(pid, &status, WNOHANG);
cin.clear(); // attempting to clear the error state. Not working.
long size = cin.tellg();
cout << size << endl;
}
so as I said the output of tellg is -1.
thought if I try to use getline(cin, some_string) I'll actually be able to see the output of the child program.
I tried creating a stream from a pipe, but it still gives me -1.
Here's the code I used:
#include <iostream>
#include <fstream>
#include <ext/stdio_filebuf.h>
#include <unistd.h>
#include <stdio.h>
#include <sys/wait.h>
#define READ_FD 0
#define WRITE_FD 1
using namespace std;
using FilebufType = __gnu_cxx::stdio_filebuf<std::ifstream::char_type>;
int
main()
{
int mypipe[2];
pipe(mypipe);
__pid_t pid = fork();
if(pid == 0)
{
dup2(mypipe[WRITE_FD], STDERR_FILENO);
char* argv[] = {"copy-feats", nullptr};
int ret = execvp(argv[0], argv);
exit(ret);
}
int status;
waitpid(pid, &status, WNOHANG);
FilebufType filebuf(mypipe[READ_FD], std::ios::in);
istream is(&filebuf);
is.clear();
auto size = is.tellg();
cout << size << endl;
}
Thanks in advance.
In practice tellg() returns an actual file position only when the input stream is a real file. That is, when the input stream is a std::ifstream, and then only when the underlying file is a plain file.
Pipes, and other non-plain files don't have a concept of a file position.
On Linux, which you are using, tellg() is typically implemented (indirectly, but that's not relevant here) by using lseek(2), and lseek(2)'s documentation explicitly specifies that it returns an ESPIPE error if the file descriptor is a pipe. And an error return, eventually, translates to tellg() returning -1.

C++ piping issue

I am trying to to fork my c++ program and direct the parents output into the childs input, I am using pipe() and fork(). In the directory of the program there is a file called input.txt. Unfortunately the only output I get is "wc: stdin: read: Bad file descriptor". Does anyone know why this is? If so what am I doing wrong? Thanks
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <errno.h>
#include <string.h>
#include <iostream>
#include<sys/types.h>
#include<sys/stat.h>
#include<fcntl.h>
#include<stdio.h>
int main(int argc, char *argv[]){
int pipes[2],pid,stdIn,stdOut;
stdIn = dup(0);
stdOut = dup(1);
pipe(pipes);
pid = fork();
if(pid == 0){
dup2(pipes[1],0);
close(pipes[1]);
execlp("wc","wc",NULL);
}
else{
dup2(pipes[0],1);
close(pipes[0]);
std::cout<<"input.txt"<<std::endl;
dup2(stdOut,0);
std::cout<<"parent done\n";
wait(NULL);
}
std::cout<<"after"<<std::endl;
return 0;
}
There are several things that should be fixed in your program:
Use STDIN_FILENO and STDOUT_FILENO instead of 0 and 1. This values may change on different platforms and you have also made a mistake which could probably be avoided if you've used names instead of value, e.g. dup2(stdOut,0); duplicated stdin and you need to duplicate stdout here.
You should close write end of the pipe in both child and parent.
By making wc read from stdin, you are then passing "input.txt" string to it - it will return stats for that string, not for the file. You could either fix it be opening a file descriptor for that file or using exec* with cat.
None of your calls the functions like pipe() or execlp() checks for failure. You should do it like that:
if (pipe(pipes) == -1) {
perror("pipe");
exit(1);
}
You don't need stdIn variable.
You will find fixed code below (it does not implement what I've described in the (5) though):
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <errno.h>
#include <string.h>
#include <iostream>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdio.h>
int main(int argc, char *argv[]) {
int pipes[2], pid, stdOut;
stdOut = dup(STDOUT_FILENO);
pipe(pipes);
pid = fork();
if (pid == 0) {
dup2(pipes[0], STDIN_FILENO);
/* You need to close write end of the pipe here */
close(pipes[1]);
execlp("wc", "wc", NULL);
} else {
std::cout << "Parent setup" << std::endl;
dup2(pipes[1], STDOUT_FILENO);
/* You need to close write end of the pipe here as well */
close(pipes[1]);
/* This will only send the string "input.txt" through the pipe, to the
* wc command */
std::cout << "input.txt" << std::endl;
dup2(stdOut, STDOUT_FILENO);
std::cout << "Parent done" << std::endl;
wait(NULL);
}
std::cout << "Program finished" << std::endl;
return 0;
}
EDIT: As suggested in the comment to the other answer, you could simple use xargs wc to read stdint as file argument:
execlp("xargs", "xargs","wc",NULL);
You have the pipe backwards, you have connected the write end of the pipe to the standard input of wc. You will need to close the write end of the pipe in both processes before wc will detect an end of file condition and terminate normally.
You also incorrectly restore the original standard output to the standard input of the parent.
Furthermore wc will by default not interpret standard input as a list filenames and will therefore not read input.txt.

file descriptors, difference between printf and std::cout

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
//#include <iostream>
#include <string.h>
#include <fcntl.h>
#include <signal.h>
int main(){
int stdout = dup(1);
char p[] = "test.txt";
close(1);
int output = open(p, O_WRONLY | O_CREAT | O_TRUNC, 0777);
//std::cout << "lala" << endl;
printf("lala\n");
close(output);
dup(stdout);
close(stdout);
printf("lolo\n");
// std::cout << "lolo" << endl;
return 0;
}
I think that printf and std::cout have to output the same thing, I want "lala" on the file, and "lolo" on the terminal screen, why this version (with prinf) print everything on screen and the version with "std::cout" print the things as I like.
This has to do with the internal buffering of the stdio library. "lala\n" is too small to be flushed to the stream straight away, so it is kept in printf internal buffer until it is flushed later.
If you add an fflush then you get the desired result:
int main(){
int stdout_fd = dup(1);
char p[] = "test.txt";
close(1);
int output = open(p, O_WRONLY | O_CREAT | O_TRUNC, 0777);
//std::cout << "lala" << endl;
printf("lala\n");
fflush(stdout); // flush the internal buffer
close(output); // fclose would flush too
dup(stdout_fd);
close(stdout_fd);
printf("lolo\n");
// std::cout << "lolo" << endl;
return 0;
}
Also I have renamed your local variable stdout because that is a globally defined one by stdio. Also stdin and stderr are FILE * pointing to the stdio control streams for those.
The correct way of redirecting streams is http://www.cplusplus.com/reference/cstdio/freopen/

Returning a vector of strings

I'm a bit confused with all the namespaces for vector and how to properly return a vector of strings in my class. Here is the code:
main.cpp
#include <fstream>
#include <iostream>
#include <stdlib.h>
#include <vector>
#include <string>
#include "lab1.h"
using namespace std;
readwords wordsinfile;
words wordslist;
int main ( int argc, char *argv[] )
{
if ( argc != 2 ) {
// Looks like we have no arguments and need do something about it
// Lets tell the user
cout << "Usage: " << argv[0] <<" <filename>\n";
exit(1);
} else {
// Yeah we have arguements so lets make sure the file exists and it is readable
ifstream ourfile(argv[1]);
if (!ourfile.is_open()) {
// Then we have a problem opening the file
// Lets tell the user and exit
cout << "Error: " << argv[0] << " could not open the file. Exiting\n";
exit (1);
}
// Do we have a ASCII file?
if (isasciifile(ourfile)) {
cout << "Error: " << argv[0] << " only can handle ASCII or non empty files. Exiting\n";
exit(1);
}
// Let ensure we are at the start of the file
ourfile.seekg (0, ios::beg);
// Now lets close it up
ourfile.close();
}
// Ok looks like we have past our tests
// Time to go to work on the file
ifstream ourfile2(argv[1]);
wordsinfile.getwords(ourfile2);
lab1.h
#ifndef LAB1_H
#define LAB1_H
bool isasciifile(std::istream& file);
class readwords {
public:
int countwords(std::istream& file);
std::vector<std::string> getwords(std::istream& file);
};
class words {
public:
void countall( void );
void print( void );
};
#endif
lab1.cpp
#include <fstream>
#include <iostream>
#include <map>
#include "lab1.h"
#include <vector>
using std::vector;
#include <string>
using namespace std;
vector<string> readwords::getwords(std::istream& file) {
char c;
string aword;
vector<string> sv;
int i = 0;
while(file.good()) {
c = file.get();
if (isalnum(c)) {
if(isupper(c)) {
c = (tolower(c));
}
if(isspace(c)) { continue; }
aword.insert(aword.end(),c);
} else {
if (aword != "") {sv.push_back(aword);}
aword = "";
i++;
continue;
}
}
return sv;
}
Here is the error from compiling.
g++ -g -o lab1 -Wall -pedantic main.cpp lab1.cpp
In file included from lab1.cpp:4:0:
lab1.h:9:4: error: ‘vector’ in namespace ‘std’ does not name a type
lab1.cpp:48:54: error: no ‘std::vector<std::basic_string<char> > readwords::getwords(std::istream&)’ member function declared in class ‘readwords’
make: *** [lab1] Error 1
Why do I get this error and how do I fix it. Thank you for any help you can provide.
Ryan
You have to #include <vector> in the header file as well. Actually, including it in the header is enough, as all files including that header will implicitly also include <vector>.
The thing is your include order is:
#include "lab1.h"
#include <vector>
and since you use std::vector in the header (before including it) you get the error. Reversing the include order would fix the compilation error, but doesn't solve the underlying error - that lab1 uses symbols that weren't defined yet. The proper fix is to include <vector>.
The compiler looks at code in the order it's written. That also applies to #include directives: the contents of the file are treated as if they had been written in the file that #include's them. As #LuchianGrigore has mentioned, the best solution is to add
#include <vector>
to "lab1.h". But you could hide the problem by moving the #include <vector> in "lab1.cpp" so that it comes before the #include "lab1.h". That would make the error go away, because the compiler would have already read` before it started to read "lab1.h". That's not what you should do, but it's the kind of thing that can happen accidentally and hide the actual problem.

output list of files from popen to ifstream.open

Basically I need to open and read a list of files I get from another command.
For each line of output of popen
open a file usen ifstream.open
it compiles and if I put the file name directly it works fine, but it doesn't do anything when using popen output. I've seen questions like this but none of this particular way of giving filenames.
here's the code:
#include <iostream>
#include <sqlite3.h>
#include <stdio.h>
#include <fstream>
using namespace std;
int main () {
ifstream singlefile;
FILE *filelist;
char filename[512];
string progline;
if(!(filelist = popen("find `pwd` -name \"*.js\"", "r"))){
return 1;
}
while( fgets(filename, sizeof(filename), filelist)!=NULL)
{
cout << filename;
singlefile.open(filename, ifstream::in);
while ( singlefile.good() )
{
getline (singlefile,progline);
cout << progline << endl;
}
singlefile.close();
}
pclose(filelist);
return 0;
}
next step would be not open each file inside the loop but to store the file list and then open each file.
Thanks
fgets keeps the trailing newline, resulting in a filename of a non-existing file. Also the stream state is only updated after reading. If I replace the while body with the following code, it works for me:
cout << filename;
size_t len = strlen(filename);
// chop off trailing newline
if (len > 1 && filename[len - 1] == '\n') filename[len - 1] = 0;
singlefile.open(filename, ifstream::in);
while ( getline(singlefile, progline) )
{
cout << progline << endl;
}
singlefile.close();
If you actually want to iterate through a list of files, I'd use Boost.Filesystem, which has a nice C++ interface, works for all filenames (even for those with newlines), and is platform-independent.
If this actually is only an example and your actual command is not find, there is still some room for simplification. Here is a suggestion that uses Boost.Iostreams to get rid of most of the C function calls (it would be great to have a device source reading from a process's standard output, but Boost.Iostreams lacks that):
#include <cstdio>
#include <iostream>
#include <fstream>
#include <stdexcept>
#include <string>
#include <stdio.h>
#include <boost/noncopyable.hpp>
#include <boost/iostreams/stream.hpp>
#include <boost/iostreams/device/file_descriptor.hpp>
using namespace std;
namespace io = boost::iostreams;
class Popen: private boost::noncopyable {
public:
explicit Popen(const char* command):
m_stream(popen(command, "r")) {
if (!m_stream) throw runtime_error("popen failed");
}
~Popen() {
pclose(m_stream);
}
FILE* stream() const {
return m_stream;
}
private:
FILE* m_stream;
};
int main() {
Popen pipe_wrapper("find `pwd` -name \"*.cpp\"");
io::file_descriptor_source pipe_device(fileno(pipe_wrapper.stream()), io::never_close_handle);
io::stream<io::file_descriptor_source> pipe_stream(pipe_device, 0x1000, 0x1000);
string filename;
while (getline(pipe_stream, filename)) {
cout << filename << endl;
ifstream file_stream(filename.c_str(), ifstream::in);
string progline;
while (getline(file_stream, progline)) {
cout << progline << endl;
}
}
}