I have written a class to log messages in my application. The class has the options to either log to console, log to file or both.
A thread monitors a queue of strings, if the queue is empty. When I process a string to log to file:
if ( (blnOpen = clsDebugService::msFile.isOpen()) != true ) {
clsDebugService::msFile.open(QIODevice::WriteOnly | QIODevice::Text);
blnOpen = clsDebugService::msFile.isOpen();
}
if ( blnOpen == true ) {
QTextStream tsText(&clsDebugService::msFile);
tsText << strLine.toLatin1().data() << "\r\n";
}
This works perfectly and I can see that the information logged in the file is always complete, however when I do the following to output to the console:
if ( clsDebugService::msblnToConsole == true ) {
std::cout << strLine.toLatin1().data() << "\n";
std::cout << std::flush;
}
Sometimes, actually quite frequently the message displayed in the console isn't complete and is missing some characters from the end of the string. Originally I had the std::flush at the end of the same statement that sends the data to the console, this did the same, so I changed it to the above hoping that would fix it, it didn't.
So, it looks that std::flush doesn't wait for previous statement to complete?
Can you try something like std::cout.setf(std::ios::unitbuf);. and see if that helps. (Not like I recommend that from performance point of view).
(This may obsolete need of std::flush as well).
Related
I use two different txt files at work for a task. One of them gets updated any time I need to call someone back (callback.txt), and one that gets updated with all of the callbacks that didn't get finished by the end of the day (recap.txt). I have written a basic python program, as well as a PowerShell script, that updates recap.txt at the end of the day by adding the newest entries to the top of the recap.txt from the day before.
Now, I am trying to write it in C++ just to get some experience with the language, but I can't seem to find why I am running into this issue. Everything works as intended, except that "recapNEW.txt" has 0 and 1 written and the end of the file on separate lines each. For example:
callback.txt:
----------
here's something
here is something else
yadda yadda
recapOLD.txt
----------
old news
still need it though
so we keep it around
recapNEW.txt (blank)
----------
And, after running the program:
recapNEW:
----------
here's something
here is something else
yadda yadda
old news
still need it though
so we keep it around
0
1
This is what I have written:
#include<fstream>
#include<iostream>
#include<string>
void update () {
std::ifstream callback("callback.txt");
std::ifstream recapBAK("recapBAK.txt");
std::ofstream recapNEW("recapNEW.txt");
std::string callLine;
std::string bakLine;
if (!callback || !recapNEW || !recapBAK) {
std::cerr << "Could not open needed files. Exiting.";
std::exit(1);
}
while (std::getline ( callback,callLine )) {
recapNEW << callLine << std::endl ;
}
while (std::getline ( recapBAK,bakLine )) {
recapNEW << bakLine << std::endl ;
}
recapNEW << callback << std::endl << recapNEW << std::endl ;
callback.close();
recapBAK.close();
recapNEW.close();
}
I split the recap.txt file into two separate files ("recapBAK.txt" & "recapNEW.txt") to keep a copy of the previous day's recap and am updating it in a separate function (posted below), but I have narrowed it down that it is the above function that is writing the extra numbers to the end of the file.
void backup () {
std::ofstream backup("recapBAK.txt");
std::ifstream today("recapNEW.txt");
std::string todayLine;
while (std::getline (today,todayLine)) {
backup << todayLine << std::endl;
}
backup.close();
today.close();
}
What is confusing me the most is that I don't see what I am doing differently in the first function that is causing the integers to be tagged on to the end of the file, because no additional integers are added to the updated version of recapUSED.
Apologies if this is not clear enough, first time I've hit a wall hard enough that I posted to a forum lol. I appreciate the help, I'm self-teaching and love having input to work with to see where I can/should improve.
I think the problem is that in:
recapNEW << callback << std::endl << recapNEW << std::endl ;
... you are attempting to output the contents of the streams callback and recapNEW to recapNEW, but it doesn't work that way. Rather, the Boolean "good" status of the file streams is output instead. Prepending to a stream isn't that simple.
Actually, it looks like you have what you need without that line. Have you tried just removing it?
When developing code, I have many console logging (std::clog) and some console output (std::cout). But now, I wanted to do online submission of my source code and I want to disable all the console logging (clog) but keep the console output (cout)
I can surely comment all of my //std::clog, but is there a better way to disable all logging inside my source file,?
You can redirect clog, create your own ofstream and use rdbuf function.
std::ofstream nullstream;
std::clog.rdbuf(nullstream.rdbuf());
Copied from Andreas Papadopoulos' answer to a slightly different question -- be sure to upvote him there!
Sure, you can (example here):
int main() {
std::clog << "First message" << std::endl;
std::clog.setstate(std::ios_base::failbit);
std::clog << "Second message" << std::endl;
std::clog.clear();
std::clog << "Last message" << std::endl;
return 0;
}
Outputs:
First message
Last message
This is because putting the stream in fail state will make it silently discard any output, until the failbit is cleared.
const boost::filesystem::path fileName( "/tmp/hello.log" );
if ( boost::filesystem::exists( fileName ) )
{
// do sth
}
else
{
std::cout << "file doesn't exist: " << std::endl;
}
Here is the issue I have:
Before I can process the log file(i.e. /tmp/hello.log), I need to check whether or not the file is completed. If the file is not complete, I will check the file later.
To run simulation, I choose the following methods:
Case I:
The log file first is incomplete(i.e. without END as the last line)
$echo "END" >> /tmp/hello.log
My application runs as expected. In other word, my application will try again if the file is incomplete and later successfully process the completed log file.
Case II:
The log file first is incomplete(i.e. without END as the last line)
I use vi to manually insert one line in the end while the application at the same time keeps checking the following lines:
const boost::filesystem::path fileName( "/tmp/hello.log" );
if ( boost::filesystem::exists( fileName ) )
{
// do sth
}
else
{
std::cout << "file doesn't exist: " << std::endl;
}
After I append the last line to the file, my application will report error and say "file doesn't exist". But in fact, the log file is there.
Why in Case II, the boost function will return false while in Case I the function return true.
why boost::filesystem::exists return false when a file does exist?
It won't. The file does not exist.
You say that this works unless you perform the editing in vi. Bear in mind that vi is not a simple command-line tool, but a powerful text editor. It may very well be using a temporary file (say, /tmp/hello.log~) for modifications. Until you save changes, those changes won't be found at /tmp/hello.log. You should study vi's documentation for more information on how it works.
It's possible that your program does not have sufficient privileges to stat the file, or that the request to stat it has failed for other reasons.
From boost's documentation, note that the exists() function's return value depends on whether or not the stat operation returned an error.
bool exists(file_status s) noexcept
Returns: status_known(s) && s.type() != file_not_found
bool status_known(file_status s) noexcept
Returns: s.type() != status_error
http://www.boost.org/doc/libs/1_60_0/libs/filesystem/doc/reference.html#exists
I want to be able to read and write to a program from C++. It seems like pstream can do the job, but I find the documentation difficult to understand and have not yet find an example.
I have setup the following minimum working example. This opens python, which in turn (1) prints hello (2) ask input, and (3) prints hello2:
#include <iostream>
#include <cstdio>
#include "pstream.h"
using namespace std;
int main(){
std::cout << "start";
redi::pstream proc(R"(python -c "if 1:
print 'hello'
raw_input()
print 'hello2'
")");
std::string line;
//std::cout.flush();
while (std::getline(proc.out(), line)){
std::cout << " " << "stdout: " << line << '\n';
}
std::cout << "end";
return 0;
}
If I run this with the "ask input" part commented out (i.e. #raw_input()), I get as output:
start stdout: hello
stdout: hello2
end
But if I leave the "ask input" part in (i.e. uncommented raw_input()), all I get is blank, not even start, but rather what seems like a program waiting for input.
My question is, how can one interact with this pstream, how can one establish a little read-write-read-write session? Why does the program not even show start or the first hello?
EDIT:
I don't seem to be making much progress. I don't think I really grasp what is going on. Here are some further attempts with commentary.
1) It seems like I can successfully feed raw_inputI prove this by writing to the child's stderr:
int main(){
cout << "start" <<endl;
redi::pstream proc(R"(python -c "if 1:
import sys
print 'hello'
sys.stdout.flush()
a = raw_input()
sys.stdin.flush()
sys.stderr.write('hello2 '+ a)
sys.stderr.flush()
")");
string line;
getline(proc.out(), line);
cout << line << endl;
proc.write("foo",3).flush();
cout << "end" << endl;
return 0;
}
output:
start
hello
end
hello2 foo
But it locks if I try to read from the stdout again
int main(){
...
a = raw_input()
sys.stdin.flush()
print 'hello2', a
sys.stdout.flush()
")");
...
proc.write("foo",3).flush();
std::getline(proc.out(), line);
cout << line << endl;
...
}
output
start
hello
2) I can't get the readsome approach to work at all
int main(){
cout << "start" <<endl;
redi::pstream proc(R"(python -c "if 1:
import sys
print 'hello'
sys.stdout.flush()
a = raw_input()
sys.stdin.flush()
")");
std::streamsize n;
char buf[1024];
while ((n = proc.out().readsome(buf, sizeof(buf))) > 0)
std::cout.write(buf, n).flush();
proc.write("foo",3).flush();
cout << "end" << endl;
return 0;
}
output
start
end
Traceback (most recent call last):
File "<string>", line 5, in <module>
IOError: [Errno 32] Broken pipe
The output contains a Python error, it seems like the C++ program finished while the Python pipe were still open.
Question: Can anyone provide a working example of how this sequential communication should be coded?
But if I leave the "ask input" part in (i.e. uncommented raw_input()), all I get is blank, not even start, but rather what seems like a program waiting for input.
The Python process is waiting for input, from its stdin, which is connected to a pipe in your C++ program. If you don't write to the pstream then the Python process will never receive anything.
The reason you don't see "start" is that Python thinks it's not connected to a terminal, so it doesn't bother flushing every write to stdout. Try import sys and then sys.stdout.flush() after printing in the Python program. If you need it to be interactive then you need to flush regularly, or set stdout to non-buffered mode (I don't know how to do that in Python).
You should also be aware that just using getline in a loop will block waiting for more input, and if the Python process is also blocking waiting for input you have a deadlock. See the usage example on the pstreams home page showing how to use readsome() for non-blocking reads. That will allow you to read as much as is available, process it, then send a response back to the child process, so that it produces more output.
EDIT:
I don't think I really grasp what is going on.
Your problems are not really problems with pstreams or python, you're just not thinking through the interactions between two communicating processes and what each is waiting for.
Get a pen and paper and draw state diagrams or some kind of chart that shows where the two processes have got to, and what they are waiting for.
1) It seems like I can successfully feed raw_input
Yes, but you're doing it wrong. raw_input() reads a line, you aren't writing a line, you're writing three characters, "foo". That's not a line.
That means the python process keeps trying to read from its stdin. The parent C++ process writes the three characters then exits, running the pstream destructor which closes the pipes. Closing the pipes causes the Python process so get EOF, so it stops reading (after only getting three characters not a whole line). The Python process then prints to stderr, which is connected to your terminal, because you didn't tell the pstream to attach a pipe to the child's stderr, and so you see that output.
But it locks if I try to read from the stdout again
Because now the parent C++ process doesn't exit, so doesn't close the pipes, so the child Python process doesn't read EOF and keeps waiting for more input. The parent C++ process is also waiting for input, but that will never come.
If you want to send a line to be read by raw_input() then write a newline!
This works fine, because it sends a newline, which causes the Python process to get past the raw_input() line:
cout << "start" <<endl;
redi::pstream proc(R"(python -c "if 1:
import sys
print 'hello'
sys.stdout.flush()
a = raw_input()
print 'hello2', a
sys.stdout.flush()
")");
string line;
getline(proc, line);
cout << line << endl;
proc << "foo" << endl; // write to child FOLLOWED BY NEWLINE!
std::getline(proc, line); // read child's response
cout << line << endl;
cout << "end" << endl;
N.B. you don't need to use proc.out() because you haven't attached a pipe to the process' stderr, so it always reads from proc.out(). You would only need to use that when reading from both stdout and stderr, where you would use proc.out() and proc.err() to distinguish them.
2) I can't get the readsome approach to work at all
Again, you have the same problem that you're only writing three characters and so the Python processes waits forever. The C++ process is trying to read as well, so it also waits forever. Deadlock.
If you fix that by sending a newline (as shown above) you have another problem: the C++ program will run so fast that it will get to the while loop that calls readsome before the Python process has even started. It will find nothing to read in the pipe, and so the first readsome call returns 0 and you exit the loop. Then the C++ program gets to the second while loop, and the child python process still hasn't started printing anything yet, so that loop also reads nothing and exits. Then the whole C++ program exits, and finally the Python child is ready to run and tries to print "hello" but by then it's parent is gone and it can't write to the pipe.
You need readsome to keep trying if there's nothing to read the first time you call it_, so it waits long enough for the first data to be readable.
For your simple program you don't really need readsome because the Python process only writes a single line at a time, so you can just read it with getline. But if it might write more than one line you need to be able to keep reading until there's no more data coming, which readsome can do (it reads only if there's data available). But you also need some way to tell whether more data is still going to come (maybe the child is busy doing some calculations before it sends more data) or if it's really finished. There's no general way to know that, it depends on what the child process is doing. Maybe you need the child to send some sentinel value, like "---END OF RESPONSE---", which the parent can look for to know when to stop trying to read more.
For the purposes of your simple example, let's just assume that if readsome gets more than 4 bytes it received the whole response:
cout << "start" <<endl;
redi::pstream proc(R"(python -c "if 1:
import sys
print 'hello'
sys.stdout.flush()
a = raw_input()
sys.stdin.flush()
print 'hello2', a
sys.stdout.flush()
")");
string reply;
streamsize n;
char buf[1024];
while ((n = proc.readsome(buf, sizeof(buf))) != -1)
{
if (n > 0)
reply.append(buf, n);
else
{
// Didn't read anything. Is that a problem?
// Need to try to process the content of 'reply' and see if
// it's what we're expecting, or if it seems to be incomplete.
//
// Let's assume that if we've already read more than 4 characters
// it's a complete response and there's no more to come:
if (reply.length() > 3)
break;
}
}
cout << reply << std::flush;
proc << "foo" << std::endl;
while (getline(proc, reply)) // maybe use readsome again here
cout << reply << std::endl;
cout << "end" << endl;
This loops while readsome() != -1, so it keeps retrying if it reads nothing and only stops the loop if there's an error. In the loop body it decides what to do if nothing was read. You'll need to insert your own logic in here that makes sense for whatever you're trying to do, but basically if readsome() hasn't read anything yet, then you should loop and retry. That makes the C++ program wait long enough for the Python program to print something.
You'd probably want to split out the while loop into a separate function that reads a whole reply into a std::string and returns it, so that you can re-use that function every time you want to read a response. If the child sends some sentinel value that function would be easy to write, as it would just stop every time it receives the sentinel string.
This question was inspired by that one.
I've understood how to execute utils from C or C++ code. But how can we get result from some command, that doesn't just printing the result, but enters some interactive mode and works until we press Ctrl+zor something like this? The example of such a command is top.
Usually you don't. You launch the command with options that makes them non interactive.
Technically you could get information from an interactive terminal interface, but you will have a hard time doing it, because in order for the interface to be human like, terminal capabilities are often used (termcaps, ncurses..) which basically works by outputting special characters, so you 'll have to dodge these characters by knowing what is expected when, so except if the interface is quite simple and static (actually even in this case) it's gonna be a pain.
Some applications (such as "dialog") can work interactively and write their final result to a different output stream. In the case of dialog, that is done using stderr (by default). If you have control over the application, you can provide for doing something like that, for passing back information to the calling application.
You simply can access stdin, stdout and stderr. Before you fork create the pipes as needed and fork and after that call execv or any other variant.
An example can be found here:
https://jineshkj.wordpress.com/2006/12/22/how-to-capture-stdin-stdout-and-stderr-of-child-program/
There is also a common used library to capture the output of a child program and react with some new actions on found items. This library was first written for tcl but can also be used for c and c++.
http://expect.sourceforge.net/
With some glue code around the expect lib your source can look like this:
int main()
{
Tcl_Interp *interp = Tcl_CreateInterp();
Expect_Init(interp);
// read from file
int lfd = open( "test.txt", O_RDONLY );
int fd = exp_spawnfd(lfd);
// or read from application
int fd ? exp_spawn("top","top", (char*)0)));
bool cont= true;
Expections set1 =
{
{ exp_glob, "YYY", []( Expection& exp)->void { cout << "found: YYY" << endl; }},
{ exp_regexp, "XXX ([0-9]+)", []( Expection& exp)->void { cout << "found: XXX" << exp.GetResult(1) << endl;}},
{ exp_glob, "END", [&cont]( Expection& exp)->void { cout << "EOF" << endl; cont = false; }}
};
int cnt = 0;
do
{
cout << "Run " << cnt << endl;
cnt++;
Expect ( fd, set1, 2 );
} while ( cont );
cout << "Finish" << endl;
return 0;
}