I want to use boost-process to read the stdout from a command:
std::string command = "cat /tmp/my_file";
namespace bp = boost::process;
bp::ipstream is;
bp::child c(command, bp::std_out > is);
std::string line;
int no_lines = 0;
while (c.running() && std::getline(is, line) && !line.empty()) {
++no_lines;
}
c.wait();
std::cout << "line count: " << no_lines << "\n";
This is almost identical to the boost-process tutorial.
For testing, the "command" is simply dumping a text file containing 10000 lines.
The problem is my code doesn't read in all the output from the command (for the test case, it only reads in about 9700 lines).
What am I doing wrong?
It seems like the child process is terminating before all of the stdout has been read.
Related
Let me start with saying that I'm around 3 days old in C++.
Ok to the main question, I have a file that spans multiple lines, and I'm trying to print one specific line repeatedly, which is subject to change arbitrarily by some other process.
Example file :
line0
line1
somevar: someval
line3
line4
I'm trying to print the middle line (one that starts with somevar). My first naive attempt was the following where I open the file, loop through the contents and print the exact line, then move to the beginning of the file.
#include <iostream>
#include <fstream>
#include <string>
int main (int argc, char *argv[])
{
std::string file = "input.txt";
std::ifstream io {file};
if (!io){
std::cerr << "Error opening file" <<std::endl;
return EXIT_FAILURE;
}
std::string line;
std::size_t pos;
while (getline (io, line))
{
pos = line.find_first_of(' ');
if (line.substr (0, pos) == "somevar:")
{
// someval is expected to be an integer
std::cout << std::stoi( line.substr (pos) ) ) << std::endl;
io.seekg (0, std::ios::beg);
}
}
io.close();
return EXIT_SUCCESS;
}
Result : Whenever the file's updated, the program exits.
I came to think the fact that the IO I'm performing is actually buffered, therefore updating the file shouldn't reflect in our existing buffer just like that (this isn't shell scripting). So now I thought let's open and close the file on each iteration, which should effectively refresh the buffer every time, I know not the best solution, but I wanted to test the theory. Here's the new source :
#include <iostream>
#include <fstream>
#include <string>
int main (int argc, char *argv[])
{
std::string proc_file = "input.txt";
std::ifstream io;
if (!io){
std::cerr << "Error opening file" <<std::endl;
return EXIT_FAILURE;
}
std::string line;
std::size_t pos;
while (io.open(proc_file, std::ios::in), io)
{
io.sync();
getline (io, line);
pos = line.find_first_of(' ');
// The line starting with "somevar:" is always going to be there.
if (line.substr (0, pos) == "somevar:")
{
std::cout << std::stoi( line.substr (pos) ) ) << std::endl;
io.close();
}
}
io.close();
return EXIT_SUCCESS;
}
Result : Same as before.
What would be the ideal way of achieving what I'm trying to? Also, why's the program exiting whenever the file in question is being updated? Thanks (:
EDIT: The file I'm trying to read is "/proc/" + std::to_string( getpid() ) + "/io", and the line is the bytes read one (starts with read_bytes:).
As discovered in the comments, you are not reading a "real" file on disk, but rather /proc/PID/io which is a virtual file whose contents may only be determined when it is opened, thanks to VFS. Your statement that it can "change arbitrarily by some other process" is misleading, the file never changes, it simply has different content each time it is opened.
So now we know that no amount of seeking will help. We simply need to open the file afresh each time we want to read it. That can be done fairly simply:
char content[1000]; // choose a suitable value
const char key[] = "read_bytes:";
while (true)
{
std::ifstream io(io_filename);
if (!io.read(content, sizeof(content)))
break;
auto it = std::search(content, std::end(content), key, key + strlen(key));
std::cout << atoi(it + strlen(key)) << std::endl;
}
You should do something more careful than atoi() which won't stop at the end of the array, but I assume your real application will do something else there so I elided handling that.
The file I'm trying to read is some /proc/1234/io
That is the most important information.
Files in proc(5) are small pseudo-files (a bit like pipe(7)-s) which can only be read in sequence.
That pseudo file is not updated, but entirely regenerated (by the Linux kernel whose source code you can study) at every open(2)
So you just read all the file quickly in memory, and process that content in memory once you have read it.
See this answer to a very related question.... Adapt it to C++
I am having an issue in C++ in which a program that I am writing reads a pre-existing text file called Builders.txt and then writes the same information into a new text file called output.txt, however, when I go to run the program the only line visible in the output.txt file is the last line, which is Tom:90:3.
Text file
Reliable Rover:70:1.
Sloppy Simon:20:4.
Technical Tom:90:3.
The expect output from the output.txt file would be the file shown above
Body Program
int main() {
readFile();
writeFile();
getch();
return 0;
}
Implementation file
std::ifstream filereadBuilders("Builders.txt");
std::string input;
void readFile() { //fumction that reads each file.
std::vector < std::string > readBuilders;
//loop to read builders.txt
while (filereadBuilders >> input) {
readBuilders.push_back(input);
}
std::cout << endl;
}
void writeFile() {
std::ofstream file("output.txt");
std::vector < std::string > RAT;
RAT.push_back(input);
for (std::string outputFile: RAT) {
file << outputFile << std::endl;
}
}
I am also having an issue where when it reads/writes, it skips white space. As seen in the expected output file the last line is Technical Tom:90:3., however, my program only returns Tom:90:3., as if its ignoring all white space.
When you use
std::vector<std::string> RAT;
RAT.push_back(input);
you are adding only the last line that was read from the file to RAT. It makes sense that only that line is written to the output file.
Suggestion for future coding projects -- avoid global variables as much as possible.
In your case, make sure that readFile returns the vector of lines to the calling function. Then, pass that data to writeFile. While at it, pass the input file to readFile and pass the output file to writeFile as well.
void readFile(std::string const& inputFile,
std::vector<std::string>& lines)
{
std::ifstream file(inputFile);
std::string line;
// Use getline. This makes sure that even whitespaces are read in.
while(std::getline(file, line))
{
lines.push_back(line);
}
std::cout << endl;
}
void writeFile(std::string const& outputFile,
std::vector<std::string> const& lines)
{
std::ofstream file(outputFile);
for(std::string const& line: lines)
{
file << line << std::endl;
}
}
and call them as
std::vector<std::string> lines;
std::string inputFile = "Builders.txt";
std::string outputFile = "output.txt";
readFile(inputFile, lines);
writeFile(outputFile, lines);
I'm trying to spawn a shell and read the stdout and stderr, and let the user run commands by writing to stdin. I can't seem to read from stdout or stderr, and writing to stdin also apparently doesn't do anything.
If I don't try redirecting any of stdin, stdout, or stderr, it spawns a normal shell like I'd expect. However, redirecting them just makes everything vanish. The >> just hangs mysteriously. I pull from the child's stdout and just put it right back into the actual stdout, so I'd expect at lease something like my username to show up, but like I said, it just blocks because there must be nothing in the buffer.
boost::process::child shell;
boost::process::opstream instream;
boost::process::ipstream errstream;
boost::process::ipstream outstream;
void console_out_read()
{
char ch;
do
{
outstream >> ch;
std::cout << ch;
}
while(true);
}
void console_err_read()
{
char ch;
do
{
errstream >> ch;
std::cerr << ch;
}
while(true);
}
void console_in_write()
{
int ch;
do
{
std::cin >> ch;
instream << ch;
}
while(true);
}
int main()
{
std::error_code ec;
shell = boost::process::child("bash",
boost::process::std_out > outstream,
boost::process::std_err > errstream,
boost::process::std_in < instream,
ec);
boost::thread cro(console_out_read);
boost::thread cre(console_err_read);
boost::thread cwi(console_in_write);
shell.wait();
return 0;
}
There are several issues with the code you posted.
First, in console_in_write you declared ch to be an int instead of a char. That means that std::cin will only accept things that look like numbers for it. This leads to std::cin getting into an error state if you enter anything but a number.
Second, even after changing ch to be a char, std::cin >> ch will never read a newline character because the formatted input operations discard whitespace. The easiest solution to this problem would be to use std::getline to read line-by-line instead of char-by-char and just write a newline after each line you read.
Third, boost's pipe streams are fully-buffered. That means that you need to flush the stream after you write a newline so that the data gets send to your child process. Otherwise you'll type a command and nothing will happen because the data is just sitting in the stream's buffer.
Fourth, std::threads must be joined or detached before they go out of scope. Since you don't currently do so with your I/O threads, as soon as your subprocess exits the parent will crash.
Putting these all together, I would expect a minimal example to look something like this:
boost::process::child shell;
boost::process::opstream instream;
boost::process::ipstream errstream;
boost::process::ipstream outstream;
void console_out_read()
{
std::string line;
while (std::getline(outstream, line))
{
std::cout << line << '\n';
}
}
void console_err_read()
{
std::string line;
while (std::getline(errstream, line))
{
std::cout << line << '\n';
}
}
void console_in_write()
{
std::string line;
while (std::getline(std::cin, line))
{
instream << line << std::endl;
if (line == "exit") return;
}
}
int main()
{
std::error_code ec;
shell = boost::process::child("bash",
boost::process::std_out > outstream,
boost::process::std_err > errstream,
boost::process::std_in < instream,
ec);
std::thread cro(console_out_read);
std::thread cre(console_err_read);
std::thread cwi(console_in_write);
shell.wait();
cro.join();
cre.join();
cwi.join();
}
Currently I have code that uses cin to get the name of the file to be used as input when the program is executed. I would like to have it so that when i run the program i can add the file redirection and the filename and run it as such: ./a.out < file.txt. How can i feed my file into my code using redirection.
this is an example of how i am currently taking input:
int main(){
std::string filename;
std::cin >> filename;
std::ifstream f(filename.c_str());
}
Do it like this
#include <iostream>
#include <fstream>
#include <string>
int main()
{
std::string line;
std::cout << "received " << std::endl;
while(std::getline(std::cin, line))
{
std::cout << line << std::endl;
}
std::cout << "on stdin" << std::endl;
}
You do not need to open the file yourself as the file content is passed to you on stdin as a result of using < on the command line.
This code has a drawback that if you do not input anything on stdin it freezes. Detecting that stdin is empty can only be achieved in a non portable manner (see here).
It would be better to accept your filename as a normal command line argument and open it inside your program.
My program is a common shell that I am trying to write in C++. In addition to taking commands from the command line it has to be able to read commands in a file - the file name being passed as an optional arg, not by redirection.
If the arg is present I open the passed filename otherwise I open "/dev/stdin". I'm not thrilled about opening the dev file and it isn't my main question but if someone has a better method I would love to hear it.
Eventually I have to read commands given to the shell but first I must present the prompt if it I am reading from stdin or skip the prompt if the input is coming from a file. My question is: is there a better way to determine in getCommand if the input stream is stdin than declaring a global or passing a bool or similar hacks?
It occurs to me that if I can somehow use std::cin rather than opening the /dev file I can pass the stream as an istream. Would that make it easier to distinguish between the two? E.G. if (source == cin) ?
Thanks for any and all suggestions.
bool getCommand(ifstream source, std::string command)
{
if (source == stdin)
//print prompt to stdout
// do the rest of stuff
return true;
}
int main(int argc, char *argv[])
{
std::ifstream input;
std::string command;
if (argc == 2)
{
input.open(argv[1], std::ifstream::in);
if (! input)
{
perror("input command file stream open");
exit(EXIT_FAILURE);
}
}
else
{
input.open("/dev/stdin", std::ifstream::in);
if (! input)
{
perror("input stdin stream open");
exit(EXIT_FAILURE);
}
}
//.......
if (getCommand(input, command))
//.......
}
If you use std::istream& as the type of your variable then you can use std::cin instead of opening /dev/stdin:
std::ifstream fileinput;
if (argc == 2)
{
fileinput.open(argv[1], std::ifstream::in);
if (! fileinput)
{
perror("input command file stream open");
exit(EXIT_FAILURE);
}
}
std::istream &input = (argc == 2) ? fileinput : std::cin;
The remainder of your question should be about identifying a tty (Detect if stdin is a terminal or pipe?). But failing that, once you've made the above change you could compare with the address of std::cin:
if (&input == &std::cin) {
std::cout << "prompt:";
}
Obviously that won't identify the case where the command line argument is /dev/stdin, so it's a bit of a bodge. But I'd probably argue that's actually an improvement given the funny requirement, because someone can at least write yourprogram /dev/stdin < input.txt, or sort input.txt | yourprogram /dev/stdin in order to work around your restriction and avoid the prompt :-)
First off, if you want to read either from std::cin or from a std::ifstream, I would implement the work in terms of an std::istream and certainly avoid opening /dev/stdin:
std::istream in(std::cin.rdbuf());
std::ifstream fin;
if (argc == 2) {
fin.open(av[1]);
// check that the file is actually open or produce an error
in.rdbuf(fin.rdbuf());
}
else {
// determine if you need to create a prompt
}
if (getCommand(in, command)) {
...
}
Now, to determine if you actually need to write a prompt it isn't sufficient to determine if you are reading from standard input but you also need to determine if the standard input is connected to something with a screen and a keyboard. There is no portable way to do so, however. On UNIXes there are functions like isatty() which can be used to determine if file descriptor 0 is connected to a tty. I would use this information to set up an iword() which can then be used to check if the stream need a prompt:
static int const needs_prompt = std::ios_base::xalloc();
...
in.iword(needs_prompt) = isatty(0);
...
if (in.iword(needs_prompt)) {
std::cout << "pompt>";
}
If you want to get fancy you can create a custom std::streambuf which is hooked up with an std::ostream which is tie()ed to the input stream stream and writes a prompt upon sync(): Every time a read operation on an input stream is done the tie()ed std::ostream is flushed. However, this requires that you only read once for each prompt being produced, e.g. using std::getline(in, line). Obviously, if you don't need a prompt, you don't tie() an std::ostream. Below is a simple program demonstrating the auto-prompt approach (without doing any of the other business):
#include <iostream>
#include <streambuf>
struct prompt
: std::streambuf
{
std::string d_prompt;
std::ostream& d_out;
prompt(std::string const& p, std::ostream& out)
: d_prompt(p)
, d_out(out)
{
}
int sync() {
this->d_out << this->d_prompt << std::flush;
return 0;
}
};
int main()
{
prompt p("prompt>", std::cout);
std::ostream helper(&p);
std::cin.tie(&helper);
for (std::string line; std::getline(std::cin, line); ) {
std::cout << "read '" << line << "'\n";
}
}