First of all, this question is based on my last question here: Reading Console Buffer / Output C++
I have a compiled executable binary file. It has some outputs, what I would like to redirect it to an other program, that handles the lines. I successfully found where the output is sent, and I modified it to STDOUT. The problem is that, when I use it like:
./jampDed.exe | stdout.exe
then the output is not synced. I got the content after every 1000-2000 bytes.
stdout.cpp
#include <iostream>
int main() {
std::string s;
while (std::getline(std::cin, s, '\n')) {
std::cout << s << std::endl;
}
return 0;
}
I also created a picture about assembly modification, where Kernel32.WriteFile function was used by default.
So the question is that, how can I make it synced? How to get every line as soon as it happens on the dedicated server?
Somewhere in the executable where it establishes stdout is an option bit for unbuffered output. Just set (or clear) that bit. Then every call to write is transferred without delay. This adds significant execution time and i/o system effort to that program but is probably okay for this.
The program which processes that output (as input) should buffer full lines because the program is unlikely to do full line output itself.
Why don't you try:
std::cout << s << std::endl << std::flush;
^^^^^^^^^^
Related
I want to read in one file line-by-line and output each line I read to a new file. In this code, cin has been redirected to refer to the input file, and cout has been redirected to refer to the output file.
The loop successfully writes every line in the file, but then it gets stuck on the final getline call. As a result, "Done" is not written to the file and the program does not terminate.
#include <string>
#include <iostream>
using namespace std;
int main() {
string line;
while(getline(cin, line)) {
cout << line << endl;
}
cout << "Done";
return 0;
}
Strangely, if I forcibly terminate the program, it seems to suddenly execute as desired, with "Done" being written.
Can someone point me in the right direction? Is there a flaw in the code, or is this some external configuration issue?
Notes: The input file in question ends with a newline character. Also, I do not want to use any includes besides these two.
The code should terminate on end of file (EOF) or any sort of file error. (The getline being called is:
http://en.cppreference.com/w/cpp/string/basic_string/getline
It returns the cin istream and then invokes its boolean conversion operator:
http://www.cplusplus.com/reference/ios/ios/operator_bool/
that checks if badbit or failbit is set on the stream. The failbit state should be set when a read is attempted with the stream already at EOF, or if there is an error.)
Per the comments above, it seems like this does work when the code is run from the shell directly. My guess is Eclipse is doing something complicated where it either intentionally sends the file into the program and then switches to an interactive input mode, or has a bug in which it doesn't close its end of a pipe or pty/tty it is using to send input to the program. (I.e. Eclipse is not binding stdin directly to the file itself in running the program.)
If one wanted to debug it further, one could look at the process state using tools like lsof. (Assuming a UNIXy system.) Might also be worth raising the issue in an Eclipse forum. The IDE is not my area of expertise.
I have seen cout.rdbuf() in for example here. This implies, the stream cout have a stream buffer associated with it, which is non empty before we flush it.
But, how can I ever peek into cout's stream buffer for cout before it's flushed?
Ie
cout << "I want to read this before this get flushed";
cout.UnknownFunction(); //this would save the buffer into a string variable
cout << flush;
But in current form of the code, everything will be flushed onto the screen after the first line..
So, what kind of construct allows me to peek inside the cout buffer?
PS. im running VC++ 2010 on windows7
I think that this doesn't flush after the first line, but I'm absolutely NOT sure.
I experienced that endl flushes, but the others don't, it's possible that too much character automatically flushes, but I don't know.
I created (accidentally) a program like this (in short):
cout << "x";
while (true) {}
The program ran this, and the output would be debug, but it haven't written anything for me, so I thought the program doesn't get there...
Following link is closely related to this topic
C++ buffered stream IO
(But I'm still not sure how/when to get cout.rdbuf() onto a string.)
I want to get the output from a shell command (my_program). Here is a example taken from this site:
#include <iostream>
#include <stdio.h>
using namespace std;
int main() {
FILE *in;
char buff[512];
if(!(in = popen("my_program", "r"))){
return 1;
}
while(fgets(buff, sizeof(buff), in)!=NULL){
cout << buff;
}
pclose(in);
return 0;
}
The problem:
I want to run my program permanently and get the output while the program is still running, NOT only get the output from the program once.
Any idea?
Your question is somewhat unclear but I suspect that the problem is buffer and the answer flushing: The way I understand is the program my_program produced outputs occassionally and the program you show should kick into action when my_program prints something. Further I suspect that this program doesn't produce a lot of output in total, e.g. less than 4k.
The problem is that most implementations buffer standard output differently depending of whether this is directed to a "console" or something else: when writing to the console output tends to be line-buffer while other destinations buffer substantially bigger buffers. However, the buffer can be explicitly flushed, reducing the performance of the output substantially. Of course, in the scenario above you'd be more interest in timely outputs rather than output performance.
A simple approach to flush an output buffer in C++ us to use out << std::flush. If you quickly want to verify that this the problem you can set up the stream to flush after every output operation using out << std::unitbuf. There are a number of other approaches to automatically flush but these are a bit more involved.
I have the following code:
ofstream mOutFile.open(logPath, ios_base::app);
string lBuilder;
lBuilder.append("========================================================\n");
lBuilder.append("Date: ");
lBuilder.append(asctime(timeinfo));
lBuilder.append("\n");
lBuilder.append("Log Message:\n");
lBuilder.append(toLog);
lBuilder.append("\n");
lBuilder.append("========================================================\n\n");
int lSize = lBuilder.size();
char* lBuffer = new char[lSize];
int index = 0;
for each (char c in lBuilder)
lBuffer[index++] = c;
mOutFile.write(lBuffer, lSize);
mOutFile.flush();
Unfortunately, until I close the app (I assume that closing the ofstream would work as well) the output does not get written to the text file. I know I could probably close and reopen the stream and everything will "just work" but that seems like a silly and incorrect solution. What am I doing wrong here?
I have also tried the following variations based on other questions I have found here, but these solutions did not work:
mOutputFile << flush;
mOutputFile << endl;
Thanks in advance for any assistance on this.
edit Everything in this code is working visual c++, it builds and works fine except the file is not written to until the stream is closed, even if I force a flush. Also, I switched from using the << operator to the char * and .write () to see if anything behaved differently.
std::ofstream file(logPath, ios_base::app);
file << "========================================================\n"
<< "Date: " << asctime(timeinfo)
<< "\nLog Message:\n" << toLog
<< "\n========================================================\n\n"
<< std::flush;
//if you want to force it write to the file it will also flush when the the file object is destroyed
//file will close itself
This is not only easier to read but it will probably also be faster than your method + it is a more standard appraoch
I ended up just "making it work" by closing and reopening the stream after the write operation.
mOutputFile << "all of my text" << endl;
mOutputFile.close();
mOutputFile.open(mLogPath);
EDIT After trying out forcing the flush on a few other systems, it looks like something just isn't performing correctly on my development machine. Not good news but at least the above solution seems to work when programmatically flushing the ofstream fails. I am not sure of the implications of the above code though, so if anyone wants to chime in if there are implications of closing and reopening the stream like this.
You can perform the following steps to validate some assumptions:
1.) After flush(), the changes to the file should be visible to your application. Open the file as std::fstream instead of std::ofstream. After flushing, reset the file pointer to the beginning and read the contents of the file. Your newly written record should be there. If not, you probably have a memory corruption somewhere in your code.
2.) Open the same file in an std::ifstream after your call to flush(). Then read the contents of the file. Your newly written record should be there. If not, then there's probably another process interfering with your file.
If both works, then you may want to read up on "file locking" and "inter-process syncronization". The OS can (theoretically) take as much time as it wants to make file changes visible to other processes.
I have the following code, running on Suse 10.1 / G++ 4.1.0, and it doesn't write to the file:
#include <fstream>
#include <iostream>
int main(){
std::ofstream file("file.out");
file << "Hello world";
}
The file is correctly created and opened, but is empty.
If I change the code to:
#include <fstream>
#include <iostream>
int main(){
std::ofstream file("file.out");
file << "Hello world\n";
}
(add a \n to the text), it works.
I also tried flushing the ofstream, but it didn't work.
Any suggestions?
If you check your file doing a cat , it may be your shell that is wrongly configured and does not print the line if there is no end of line.
std::endl adds a \n and flush.
Don't know if this is what you tried but you should do:
file << "Hello World" << std::flush;
Update; I'm leaving this answer here because of the useful comments
Based on feedback, I'll modify my advice: you shouldn't have to explicitly call std::flush (or file.close() for that matter), because the destructor does it for you.
Additionally, calling flush explicitly forces an I/O operation that may not be the most optimized way. Deferring to the underlying iostreams and operating system would be better.
Obviously the OP's issue was not related to calling or not calling std::flush, and was probably due to attempting to read the file before the file stream destructor was called.
The destructor should flush and close the file.
I am pretty sure, the error is an another place, either
1) You do not check at the right point in time. At which point do you compare the content of the file, "after" the exits, or do you set a breakpoint before the program exits and then you check the files content?
2) Somehow the program crashes before it exits?
Does
file << "Hello world" << std::endl;
work?
endl inserts a newline and flushes the buffer. Is that what you were referring to when you said that you'd already tried flushing it?
You are working on Linux, which is a POSIX-compliant system. The POSIX standard defines what a line is:
A sequence of zero or more non-newline characters plus a
terminating newline character.
So without the newline character, the file contains 0 lines and is therefore empty.