Writing popen() output to a file - c++

I've been trying to call another program from c++, and save the stout of that program to a text file. popen() seems to be the appropriate function, but saving it to a text file isn't working.
ofstream delaunayfile;
delaunayfile.open ("triangulation/delaunayedpoints.txt");
FILE *fp;
fp = popen("qdelaunay < triangulation/rawpoints.txt", "r");
delaunayfile << fp;
delaunayfile.close();
Any help? Thanks in advance!

You cannot write a FILE* directly into a stream. It will write a memory address instead of the actual file contents, therefore it will not give you the desired result.
The ideal solution would be to read from an ifstream and write to your ofstream, but there's no way to construct an ifstream from a FILE*.
However, we can extend the streambuf class, make it work over a FILE*, and then pass it to an istream instead. A quick search revealed someone already implemented that, and properly named popen_streambuf. See this specific answer.
Your code then would look like this:
std::ofstream output("triangulation/delaunayedpoints.txt");
popen_streambuf popen_buf;
if (popen_buf.open("qdelaunay < triangulation/rawpoints.txt", "r") == NULL) {
std::cerr << "Failed to popen." << std::endl;
return;
}
char buffer[256];
std::istream input(&popen_buf);
while (input.read(buffer, 256)) {
output << buffer;
}
output.close();
As pointed by Simon Richter in comments, there's an operator<< that accepts streambuf and writes data to ostream until EOF is reached. This way, the code would be simplified to:
std::ofstream output("triangulation/delaunayedpoints.txt");
popen_streambuf popen_buf;
if (popen_buf.open("qdelaunay < triangulation/rawpoints.txt", "r") == NULL) {
std::cerr << "Failed to popen." << std::endl;
return;
}
output << &popen_buf;
output.close();

There are two ways to do this: The simple way
int rc = system("qdelaunay < triangulation/rawpoints.txt >triangulation/delaunayedpoints.txt");
and the slightly more elaborate way, using fork(), dup2() and execve(), the latter working without a shell interpreter installed on the system. Given that this looks like you are doing computation work, I suspect this is not an embedded system, so you can assume a working shell.

popen opens a pipe but I am not aware you can just stream it into delaunayfile that way.
Of course it would be very nice if you could just do that and it would read from the pipe until it was complete.
The normal way to check for data on the pipe is to use select(). I found a useful link http://codenewbie.com/forums/threads/2908-Using-std-fstream-in-a-pipe that integrates pipes with fstream though and it may help you achieve what you want.
In this instance though as all you want to do is write the output to a file, why not redirect the output of the process to it rather than to a pipe? The purpose of a pipe is Inter-Process communication but your process does not appear to be using the data it receives from the other process for any practical purpose.

Related

I can't get the ofstream function to work

Hello and sorry if the answer is clear to those out there. I am still fairly new to programming and ask for some guidance.
This function should write just one of the three string parameters it takes in to the txt file I have already generated. When I run the program the function seems to work fine and the cout statement shows the info is in the string and does get passes successfully. The issue is after running the program I go to check the txt file and find it is still blank.
I am using C++17 on visual studio professional 2015.
void AddNewMagicItem(const std::string & ItemKey,
const std::string & ItemDescription,
const std::string &filename)
{
const char* ItemKeyName = ItemKey.c_str();
const char* ItemDescriptionBody = ItemDescription.c_str();
const char* FileToAddItemTo = filename.c_str();
std::ofstream AddingItem(FileToAddItemTo);
std::ifstream FileCheck(FileToAddItemTo);
AddingItem.open(FileToAddItemTo, std::ios::out | std::ios::app);
if (_access(FileToAddItemTo, 0) == 0)
{
if (FileCheck.is_open())
{
AddingItem << ItemKey;
std::cout << ItemKey << std::endl;
}
}
AddingItem.close(); // not sure these are necessary
FileCheck.close(); //not sure these are necessary
}
This should print out a message onto a .txt file when you pass a string into the ItemKey parameter.
Thank you very much for your help and again please forgive me as I am also new to stackoverflow and might have made some mistakes in formatting this question or not being clear enough.
ADD ON: Thank you everyone who has answered this question and for all your help. I appreciate the help and would like to personally thank you all for your help, comments, and input on this topic. May your code compile every time and may your code reviews always be commented.
As mentioned by previous commenters/answerers, your code can be simplified by letting the destructor of the ofstream object close the file for you, and by refraining from using the c_str() conversion function.
This code seems to do what you wanted, on GCC v8 at least:
#include <string>
#include <fstream>
#include <iostream>
void AddNewMagicItem(const std::string& ItemKey,
const std::string& ItemDescription,
const std::string& fileName)
{
std::ofstream AddingItem{fileName, std::ios::app};
if (AddingItem) { // if file successfully opened
AddingItem << ItemKey;
std::cout << ItemKey << std::endl;
}
else {
std::cerr << "Could not open file " << fileName << std::endl;
}
// implicit close of AddingItem file handle here
}
int main(int argc, char* argv[])
{
std::string outputFileName{"foobar.txt"};
std::string desc{"Description"};
// use implicit conversion of "key*" C strings to std::string objects:
AddNewMagicItem("key1", desc, outputFileName);
AddNewMagicItem("key2", desc, outputFileName);
AddNewMagicItem("key3", desc, outputFileName);
return 0;
}
Main Problem
std::ofstream AddingItem(FileToAddItemTo);
opened the file. Opening it again with
AddingItem.open(FileToAddItemTo, std::ios::out | std::ios::app);
caused the stream to fail.
Solution
Move the open modes into the constructor (std::ofstream AddingItem(FileToAddItemTo, std::ios::app);) and remove the manual open.
Note that only the app open mode is needed. ofstream implies the out mode is already set.
Note: If the user does not have access to the file, the file cannot be opened. There is no need to test for this separately. I find testing for an open file followed by a call to perror or a similar target-specific call to provide details on the cause of the failure to be a useful feature.
Note that there are several different states the stream could be in and is_open is sort of off to the side. You want to check all of them to make sure an IO transaction succeeded. In this case the file is open, so if is_open is all you check, you miss the failbit. A common related bug when reading is only testing for EOF and winding up in a loop of failed reads that will never reach the end of the file (or reading past the end of the file by checking too soon).
AddingItem << ItemKey;
becomes
if (!(AddingItem << ItemKey))
{
//handle failure
}
Sometimes you will need better granularity to determine exactly what happened in order to properly handle the error. Check the state bits and possibly perror and target-specific
diagnostics as above.
Side Problem
Opening a file for simultaneous read and write with multiple fstreams is not recommended. The different streams will provide different buffered views of the same file resulting in instability.
Attempting to read and write the same file through a single ostream can be done, but it is exceptionally difficult to get right. The standard rule of thumb is read the file into memory and close the file, edit the memory, and the open the file, write the memory, close the file. Keep the in-memory copy of the file if possible so that you do not have to reread the file.
If you need to be certain a file was written correctly, write the file and then read it back, parse it, and verify that the information is correct. While verifying, do not allow the file to be written again. Don't try to multithread this.
Details
Here's a little example to show what went wrong and where.
#include <iostream>
#include <fstream>
int main()
{
std::ofstream AddingItem("test");
if (AddingItem.is_open()) // test file is open
{
std::cout << "open";
}
if (AddingItem) // test stream is writable
{
std::cout << " and writable\n";
}
else
{
std::cout << " and NOT writable\n";
}
AddingItem.open("test", std::ios::app);
if (AddingItem.is_open())
{
std::cout << "open";
}
if (AddingItem)
{
std::cout << " and writable\n";
}
else
{
std::cout << " and NOT writable\n";
}
}
Assuming the working directory is valid and the user has permissions to write to test, we will see that the program output is
open and writable
open and NOT writable
This shows that
std::ofstream AddingItem("test");
opened the file and that
AddingItem.open("test", std::ios::app);
left the file open, but put the stream in a non-writable error state to force you to deal with the potential logic error of trying to have two files open in the same stream at the same time. Basically it's saying, "I'm sorry Dave, I'm afraid I can't do that." without Undefined Behaviour or the full Hal 9000 bloodbath.
Unfortunately to get this message, you have to look at the correct error bits. In this case I looked at all of them with if (AddingItem).
As a complement of the already given question comments:
If you want to write data into a file, I do not understand why you have used a std::ifstream. Only std::ofstream is needed.
You can write data into a file this way:
const std::string file_path("../tmp_test/file_test.txt"); // path to the file
std::string content_to_write("Something\n"); // content to be written in the file
std::ofstream file_s(file_path, std::ios::app); // construct and open the ostream in appending mode
if(file_s) // if the stream is successfully open
{
file_s << content_to_write; // write data
file_s.close(); // close the file (or you can also let the file_s destructor do it for you at the end of the block)
}
else
std::cout << "Fail to open: " << file_path << std::endl; // write an error message
As you said being quite new to programming, I have explicitly commented each line to make it more understandable.
I hope it helps.
EDIT:
For more explanation, you tried to open the file 3 times (twice in writing mode and once in reading mode). This is the cause of your problems. You only need to open the file once in writing mode.
Morever, checking that the input stream is open will not tell you if the output stream is open too. Keep in mind that you open a file stream. If you want to check if it is properly open, you have to check it over the related object, not over another one.

Blocking read from std::ifstream

I am reading from a pipe (Linux) or a pipe-like device object (Windows) using std::ifstream::read. However, when there is no more data, read reads 0 bytes and sets EOF. Is there a way to make a blocking read from an ifstream, such that it only returns when there is some more data?
I'd rather not busy wait for the EOF flag to clear.
If it is not possible with the C++ standard library, what is the closest other option? Can I do it in plain C, or do I have to resort to operating system specific APIs?
Unfortunately, std is very poor on any non-algorithmic functionality, like IO. You always have to rely on 3rd-party solutions. Fortunately, there is Boost and, if you do not mind, I will suggest to use it to reduce OS specific code.
namespace bs = boost::iostreams;
int fd; // Create, for example, Posix file descriptor and specify necessary flags for it.
bs::file_descriptor_source fds(fd);
bs::stream<bs::file_descriptor_source> stream(fds);
// Work with the stream as it is std stream
In this small example I use Boost IO Streams and specifically file_descriptor_source that works as an underlying stream device and hides Windows or Posix specific pipe inside. The pipe you open yourself, so you can configure the pipe as you want.
well there seems no way to do a blocking read. clearing the error bit will not help. Only a re-open of the fifo like in this example:
int main(int argc, char **argv)
{
int rc=0;
enum FATAL {ERR_ARGV,ERR_OPEN_FILE};
try
{
if( argv[1] == NULL) throw ERR_ARGV;
std::ifstream fifo;
while(1)
{
fifo.open(argv[1],std::ifstream::in);
if( !fifo.is_open() ) throw ERR_OPEN_FILE;
std::string line;
while(std::getline(fifo,line))
{
std::cout << line << "\n"; fflush(stdout);
}
fifo.close();
}
// never should come here
}
catch(FATAL e)
{
rc=e;
switch(e)
{
case ERR_ARGV:
std::cerr << "ERROR: argument 1 should be a fifo file name\n";
break;
case ERR_OPEN_FILE:
std::cerr << "ERROR: unabel to open file " << argv[1] << "\n";
break;
}
}
return(rc);
}
I have tested this code and it works to do an endless read from a fifo.

Standard output hangs after adding fprintf() statement with custom standard error

I have a C++ class Archive with a member function extractData(). This function calls realExtractData(), which is implemented in a separate C library.
I want to pass the extractData() function a pair of FILE * instances that are usually stdout and stderr, but I want to provide the option of custom file pointers, as well:
class Archive {
public:
...
int extractData(string id, FILE *customOut, FILE *customErr);
...
};
int
Archive::extractData(string id, FILE *customOut, FILE *customErr)
{
if (realExtractData(id.c_str(), customOut) != EXIT_SUCCESS) {
fprintf(stderr, "something went wrong...\n");
return EXIT_FAILURE;
}
return EXIT_SUCCESS;
}
If I call the above as listed, there is no delay in outputting data to standard output. All extracted data get sent to standard output (stdout) almost immediately:
FILE *outFp = stdout;
FILE *errFp = stderr;
Archive *archive = new Archive(inFilename);
if (archive->extractData(id, outFp, errFp) != EXIT_SUCCESS) {
fprintf(errFp, "[error] - could not extract %s\n", archive->getInFnCStr());
return EXIT_FAILURE;
}
If I change extractData() so that its fprintf() call uses customErr:
int
Archive::extractData(string id, FILE *customOut, FILE *customErr)
{
if (realExtractData(id.c_str(), customOut) != EXIT_SUCCESS) {
fprintf(customErr, "something went wrong...\n"); /* <-- changed this line */
return EXIT_FAILURE;
}
return EXIT_SUCCESS;
}
...then when I run the binary, the binary seems to hang in processing input and printing to standard output.
If I change fprintf() back to using stderr and not customErr, things once again work properly, i.e., data are flushed to standard output (my customOut) immediately.
Is this a buffering issue? Is there a way to fix this?
"stderr and not customErr"
Standard error is un-buffered which means it prints out almost immediately. other output streams are buffered unless you're using low-level OS calls, which means they will take longer to print unless you do a buffer flush with something like an endl, ::flush, or whatever else.
If you want to go for the low-level OS calls and you're working with unix, check this out:
http://www.annrich.com/cs590/notes/cs590_lecture_2.pdf
I haven't read the whole thing, but on scanning it it looks as if it has similar info to the good Stevens Advanced Programming in Unix book which defitely talks through this.

c++: how to print new line without carriage return [duplicate]

This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
How to make cout behave as in binary mode?
I am using C++ to print binary data to the stdout and then I redirect it to a file.
(I do not want to write directly to a file).
However, when I try to print the value '0A' I am getting '0D 0A'. (this is what I see when I copy the output to HexEdit).
From what I can figure out, '0A' is the ascii for 'new line' so cout automatically adds the '0D' which is 'carriage return'.
What I want is to print 0A alone. how can I do this?
my code:
unsigned char *buf = new unsigned char[width+1];
for (int x = 0; x < width; x++)
{
buf[x] = (unsigned char) src[x];
}
buf[width] = '\0'
cout<<buf;
I've seen this: Print new line to a text file without carriage return (CR) in windows but it does not seem to help me.
Not sure you really can use cout and the << operator without a massive hack. This is because when using << cout treats all data as text data and will perform appropriate transformations as necessary -- including adding a CR character when printing a newline on a Windows platform.
You will need to use the write method of cout, instead of using the insertion operator <<. The write method will not format your data, but is more awkward to use.
http://www.cplusplus.com/reference/iostream/ostream/write/
You can't control the end of line sequence with any of the standard output streams or files. The only way to deal with this is to freopen() the standard output stdout with an open mode of wb or wb+` to control the underlying formatting, e.g.:
FILE* out = freopen("stdout.txt", "wb", stdout);
std::cout << "hello, world\n";
If you are only using C++ streams you can also replace std::cout's stream buffer with the stream buffer of an appropriately opened std::ofstream, e.g.:
std::ofstream file("stdout.txt", std::ios_base::out);
std::streambuf* coutbuf = std::cout.rdbuf(file.rdbuf());
std::cout << "hello, world\n";
...
std::cout.rdbuf(coutbuf);
You need to restore the original stream buffer to avoid problems with uses of std::cout during destruction: at the very least the stream gets flushed at some point after main() is exited. If you can accept a memory leak, you can use a dynamically allocated std::filebuf directly.
None of the solutions above actually writes the output to the standard output, they all write to a file. I don't know how to reopen the standard output on a Windows system (on a UNIX system you'd either create a stream buffer using file descriptor 1 but on UNIX there is no need anyway because the \n doesn't get replaced in the first place). In particular, reopening the stream to fit in with output redirection is probably not possible.

Can't write to file in a loop containing sleep()

The essence of my problem is that I can't write to a file in a loop with sleep(). If I have the following code:
ofstream file
file.open("file.name");
for(;;) {
file << "HELLO\n";
}
This code works perfectly and prints HELLO repeatedly into "file.name". However, I want to do something like this (I'm recording data from a real-time application):
for(;;) {
file << "HELLO\n";
sleep(1);
}
This doesn't seem to print anything into my file. Any ideas?
You need to flush the output. The output stream is buffering your data into memory but not writing it out to disk. You should either use std::endl (which prints a newline and flushes) instead of the string literal '\n', or explicitly flush the stream with std::flush:
for(;;) {
file << "HELLO" << endl;
}
// or
for(;;) {
file << "HELLO\n" << flush;
}
The magic word you are looking for is "flush".
c++ std::ofstream flush() but not close()
before the sleep, flush the file so that it isn't pending in a buffer waiting for there to be enough of a change to bother writing out.
It's probably just a buffering issue. Because you are now writing much slower, the output buffer wont fill up so fast so you may not 'see' the written data. Try adding a flush() before the sleep.
file.flush()