I'm using the google logger (glog) in a C++ project, where source code contains printout of informational items and progress:
std::cout << some_useful_stuff << "\n";
And I can use GLOG to log informational items:
LOG(INFO) << some_useful_stuff << "\n"; \\ gets echoed to stdout
I can't change all the std::cout statements throughout the codebase (as I'm using some subrepos from third parties etc, and unable to add my own logging statements to them), this prevents me from using an if-else construct (which would be ugly anyway, wrapping every single output statement in logic).
So how could I replicate stdout outputs to GLog?
std::cout << some_useful_stuff << "\n"; \\ message also goes to LOG(INFO)
Possibly useful: I actually don't need this to happen at the same time; it's useful to have a choice like {CONSOLE, LOGS, BOTH} but I could put up with the simpler choice between {CONSOLE, LOGS}.
Also - experienced programmer but C++ newbie so if this is mega simple; apologies - and thanks in advance!
Not sure about glog, but usually we use dup2 system call to redirect output of one file handler to another.
For e.g. in your case to make all std::cout write to the log file used by glog just do
dup2(glog_file_handler,stdout);
This would work if you can some how get filehandle of the log file that glog is writing to. Refer http://man7.org/linux/man-pages/man2/dup.2.html for more details.
Hope this helps.
P.S. This won't work if your program has multiple threads/procesess writing to same glog file and if glog does synchronized writes to log file
Related
The situation is: I have an external application so I don't have the source code and i can't change it. While running, the application writes logs to the stderr. The task is to write a program that check the output of it and separate some part of the output to other file. My solution is to start the app like
./externalApp 2>&1 | myApp
the myApp is a c++ app with the following source:
using namespace std;
int main ()
{
string str;
ofstream A;
A.open("A.log");
ofstream B;
B.open("B.log");
A << "test start" << endl;
int i = 0;
while (getline(cin,str))
{
if(str.find("asdasd") != string::npos)
{
A << str << endl;
}
else
{
B << str << endl;
}
++i;
}
A << "test end: " << i << " lines" << endl;
A.close();
B.close();
return 0;
}
The externalApp can crash or be terminated. A that moment the myApp gets terminated too and it is don't write the last lines and don't close the files. The file can be 60Gb or larger so saving it and processing it after not a variant.
Correction: My problem is that when the externalApp crash it terminate myApp. That mean any code after while block will never run. So the question is: Is there a way to run myApp even after the externalApp closed?
How can I do this task correctly? I interesed in any other idea to do this task.
There's nothing wrong with the shown code, and nothing in your question offers any evidence of anything being wrong with the shown code. No evidence was shown that your logging application actually received "the last lines" to be written from that external application. Most likely that external application simply failed to write them to standard output or error, before crashing.
The most likely explanation is that your external application checks if its standard output or error is connected to an interactive terminal; if so each line of its log message is followed by an explicit buffer flush. When the external application's standard output is a pipe, no such flushing takes place, so the log messages get buffered up, and are flushed only when the application's internal output buffer is full. This is a fairly common behavior. But because of that, when the external application crashes its last logged lines are lost forever. Because your logger never received them. Your logger can't do anything about log lines it never read.
In your situation, the only available option is to set up and connect a pseudo-tty device to the external application's standard output and error, making it think that's connected to an interactive terminal, while its output is actually captured by your application.
You can't do this from the shell. You need to write some code to set this up. You can start by reading the pty(7) manual page which explains the procedure to follow, at which point you will end up with file descriptors that you can take, and attach to your external application.
If you want your program to cleanly deal with the external program crashing you will probably need to handle SIGPIPE. The default behaviour of this signal is to terminate the process.
So the problem was not that when the first element of the pipe ended it terminate the second. The real problem was that the two app with pipes launched from bash script and when the bash script ended it terminated all of it child process. I solved it using
signal(SIGHUP,SIG_IGN);
that way my app executed to the end.
Thank you for all the answer at least I learned lot about the signals and pipes.
I have g++ version 4.8.4 compiler with Xubuntu 14.04. In the midst of my OpenCV code (written in Eclipse CDT), I wrote the following three lines consecutively:
/* Some codes here*/
cerr << "No match found. # of false positives: " << falsePositives << endl;
cout << "Press a key to continue..." << endl;
waitKey(0);
and here is the result:
Press a key to continue...
No match found. # of false positives: 1
/*there is a blank line*/
Why the order of those two lines changed at the execution time? There is no parallel code at all in the previous lines but they seems to work like parallel (at the same time).
I know that cerr is not buffered whereas cout is buffered (which means, afaik, cerr is slower compared to cout); however, no matter what, shouldn't the order of execution be changed? And where does that blank line comes from? (probably from one of those endls but which one?)
Can someone explain what is going on those two lines?
Thank you so much.
EDIT:
I don't use ubuntu, I use Xubuntu 14.04. Sorry for that mistake, my mind was too messy but I think it does not effect the result.
I use Eclipse's provided console to display them. I tried to append std:: prefix to all cout, cerr, endl. The result is same.
The interesting point is that when I just wrote a new file including:
#include <iostream>
#include <cstdlib>
int main()
{
std::cerr << "No match found. # of false positives: " << 2 << std::endl;
std::cout << "Press a key to continue..." << std::endl;
return 0;
}
I got the expected output(first cerr and then cout) by using xfce4-terminal and g++ compiler.
The problem occurs when using Eclipse CDT. I also want to remind all of you that I work on OpenCV.
Chris Dodd's 4th suggestion:
"your code is actually something other than what you've posted above, and the difference, while seemingly unimportant, is actually crucial."
Of course my code does contain other than what I typed but there are lots of, I mean lots of computation etc. before those lines. However, there might be relating parts before which I could not realise. Also, I didn't redirected stdout and/or stderr to different devices/files/pipes before those lines at all.
EDIT 2:
When I execute the program in the Debug mode of Eclipse CDT, following assembly lines,
After cerr line the followings are executed (as well as other assembly codes of course):
callq 0x403500 <_ZStlsISt11char_traitsIcEERSt13basic_ostreamIcT_ES5_PKc#plt>
callq 0x403470 <_ZNSolsEi#plt>
callq 0x403770 <_ZNSolsEPFRSoS_E#plt>
After cout line the followings are executed (as well as other assembly codes of course):
callq 0x403500 <_ZStlsISt11char_traitsIcEERSt13basic_ostreamIcT_ES5_PKc#plt>
callq 0x403770 <_ZNSolsEPFRSoS_E#plt>
callq 0x403670 <_ZN2cv7waitKeyEi#plt>
and they all give the same error message:
No source available for "std::basic_ostream >& std::operator<< >(std::basic_ostream >&, char const*)#plt at 0x403500"
and the process continues with other lines of assembly code without termination.
PS: When I commented out everthing but those two lines, it works as expected. Thus, my conclusion is that, there may be relevant code part before those lines but I couldn't figure them out.
std::cerr and std::cout are different streams and they are not synchronized.
So you really can't assume anything about how output to both gets shown. In this case, the output happens to be shown before the error.
You can rely on the order within either stream.
Additionally, std::cout is buffered and std::cerr is not, and that often causes this kind of problem, but because you are using std::endl (which flushes the stream) this doesn't really apply in your case.
The order of those two lines was not changed. However, whatever code produced the output you saw failed to preserve the order in which the output was sent to the two streams. It's possible that it just waited and then read both streams to produce the final output. It's hard to be sure without knowing what your environment looks like.
Well, std::endl flushes the stream to the underlying device, which mean that the output cannot legally be what you describe -- the first endl is sequenced before the output to cout, so the cerr stream must be flushed and the output appear on the terminal before the second line is executed.
Which means there are a number of possibilities
your computer is broken or compiler is buggy (unlikely)
you've defined endl to be something other than std::endl, and it doesn't flush (possible if you're trying to be obfuscated)
you've redirected stdout and/or stderr to different devices/files/pipes and are later combining them, and that later combination step is reordering things.
your code is actually something other than what you've posted above, and the difference, while seemingly unimportant, is actually crucial.
So if you want a real answer to this question, you need to post and MVCE demonstrating what you are actually doing.
Eclipse CDT is inserting itself into the cerr and cout streams during the creation of your process. It may echo the cerr stream to one of its windows and then write to the intended console. A tee of sorts.
Since it is polling these streams, it is not be possible to synchronize them. This could explain the behavior.
Using cout needs a little bit more time to output the statement which isn't good for me. But when using cerr the output is faster. Why?
Just trying to help :
- cout -> Regular output (console output)
- cerr -> Error output (console error)
cout is buffered, cerr is not, so cout should be faster in most cases. (Although if you really care about speed, the C output functions such as printf tend to be a lot faster than cout/cerr).
cout and cerr are ostream objects. You can call rdbuf() on them to redirect their output independently wherever you want, from within the application. You can open a network socket, wrap it in a stream buffer and redirect there, if you want.
By default, cout is tied to the application's standard output. By default, the standard output is the screen. You can direct the OS to redirect stdout elsewhere. Or it might do it by itself - the nohup utility in Linux, for example, does. Services in Windows also have their standard streams redirected, I think.
And, cerr are tied to the application's standard error. By default the standard error is the screen. You can again redirect stderr elsewhere.
Another issue here is that clog, by default, is buffered like cout, whereas cerr is unit-buffered, meaning it automatically calls flush() after every complete output operation. This is very useful, since it means that the output is not lost in the buffer if the application crashes directly afterwards.
If you run a program like this:
yourprog > yourfile
What you write to cout will go to yourfile. What you write to cerr will go to your screen. That's usually a good thing. I probably don't want your error messages mixed in with your program output. (Especially if some of your error messages are just warnings or diagnostic stuff).
It's also possible to redirect cout to 1 file, and cerr to another. That's a handy paradigm: I run your program, redirect output to a file, error messages to a different file. If your program returns 0 from main, then I know it's OK to process the output file. If it returns an error code, I know NOT to process the output file. The error file will tell me what went wrong.
reference :
- http://www.tutorialspoint.com/cplusplus/cpp_basic_input_output.htm
- http://cboard.cprogramming.com/cplusplus-programming/91613-cout-cerr-clog.html
I'm working on a BTS C++ code, i faced a command that i don't know its functionality, i wish anyone here could help me
LOG(INFO) << *cmsrq;
Here what is the function of LOG. it's not a logarithmic function.
From the context, the line of code:
LOG(INFO) << *cmsrq;
writes an entry to a log.
Logs are typically used to record the activities of a computer system. One purpose of keeping such logs is troubleshooting malfunctions.
In the code that you show, the function (or macro) LOG() returns an output stream that is used to log messages associated with the given logging level (INFO probably stands for "informational messages").
That's very probably a MACRO that gives you back an object which logs (to console or file) what you pass it through the << operator.
Much like qDebug().
The value "INFO" you see in there indicates that you want to output the *cmsrq value to the information log level.
I can imagine some macro definition like that:
#define LOG( X ) Logging::logger( X )
Where logger() is a static function returning a reference of the logging engine class, initialized with the correct log level.
I've been wondering, what is the point of clog? As near as I can tell, clog is the same as cerr but with buffering so it is more efficient. Usually stderr is the same as stdout, so clog is the same as cout. This seems pretty lame to me, so I figure I must be misunderstanding it. If I have log messages going out to the same place I have error messages going out to (perhaps something in /var/log/messages), then I probably am not writing too much out (so there isn't much lost by using non-buffered cerr). In my experience, I want my log messages up to date (not buffered) so I can help find a crash (so I don't want to be using the buffered clog). Apparently I should always be using cerr.
I'd like to be able to redirect clog inside my program. It would be useful to redirect cerr so that when I call a library routine I can control where cerr and clog go to. Can some compilers support this? I just checked DJGPP and stdout is defined as the address of a FILE struct, so it is illegal to do something like "stdout = freopen(...)".
Is it possible to redirect clog, cerr, cout, stdin, stdout, and/or stderr?
Is the only difference between clog and cerr the buffering?
How should I implement (or find) a more robust logging facility (links please)?
Is it possible to redirect clog, cerr, cout, stdin, stdout, and/or stderr?
Yes. You want the rdbuf function.
ofstream ofs("logfile");
cout.rdbuf(ofs.rdbuf());
cout << "Goes to file." << endl;
Is the only difference between clog and cerr the buffering?
As far as I know, yes.
If you're in a posix shell environment (I'm really thinking of bash), you can redirect any
file descriptor to any other file descriptor, so to redirect, you can just:
$ myprogram 2>&5
to redirect stderr to the file represented by fd=5.
Edit: on second thought, I like #Konrad Rudolph's answer about redirection better. rdbuf() is a more coherent and portable way to do it.
As for logging, well...I start with the Boost library for all things C++ that isn't in the std library. Behold: Boost Logging v2
Edit: Boost Logging is not part of the Boost Libraries; it has been reviewed, but not accepted.
Edit: 2 years later, back in May 2010, Boost did accept a logging library, now called Boost.Log.
Of course, there are alternatives:
Log4Cpp (a log4j-style API for C++)
Log4Cxx (Apache-sponsored log4j-style API)
Pantheios (defunct? last time I tried I couldn't get it to build on a recent compiler)
Google's GLog (hat-tip #SuperElectric)
There's also the Windows Event logger.
And a couple of articles that may be of use:
Logging in C++ (Dr. Dobbs)
Logging and Tracing Simplified (Sun)
Since there are several answers here about redirection, I will add this nice gem I stumbled across recently about redirection:
#include <fstream>
#include <iostream>
class redirecter
{
public:
redirecter(std::ostream & dst, std::ostream & src)
: src(src), sbuf(src.rdbuf(dst.rdbuf())) {}
~redirecter() { src.rdbuf(sbuf); }
private:
std::ostream & src;
std::streambuf * const sbuf;
};
void hello_world()
{
std::cout << "Hello, world!\n";
}
int main()
{
std::ofstream log("hello-world.log");
redirecter redirect(log, std::cout);
hello_world();
return 0;
}
It's basically a redirection class that allows you to redirect any two streams, and restore it when you're finished.
Redirections
Konrad Rudolph answer is good in regard to how to redirect the std::clog (std::wclog).
Other answers tell you about various possibilities such as using a command line redirect such as 2>output.log. With Unix you can also create a file and add another output to your commands with something like 3>output.log. In your program you then have to use fd number 3 to print the logs. You can continue to print to stdout and stderr normally. The Visual Studio IDE has a similar feature with their CDebug command, which sends its output to the IDE output window.
stderr is the same as stdout?
This is generally true, but under Unix you can setup the stderr to /dev/console which means that it goes to another tty (a.k.a. terminal). It's rarely used these days. I had it that way on IRIX. I would open a separate X-Window and see errors in it.
Also many people send error messages to /dev/null. On the command line you write:
command ...args... 2>/dev/null
syslog
One thing not mentioned, under Unix, you also have syslog().
The newest versions under Linux (and probably Mac OS/X) does a lot more than it used to. Especially, it can use the identity and some other parameters to redirect the logs to a specific file (i.e. mail.log). The syslog mechanism can be used between computers, so logs from computer A can be sent to computer B. And of course you can filter logs in various ways, especially by severity.
The syslog() is also very simple to use:
syslog(LOG_ERR, "message #%d", count++);
It offers 8 levels (or severity), a format a la printf(), and a list of arguments for the format.
Programmatically, you may tweak a few things if you first call the openlog() function. You must call it before your first call to syslog().
As mentioned by unixman83, you may want to use a macro instead. That way you can include some parameters to your messages without having to repeat them over and over again. Maybe something like this (see Variadic Macro):
// (not tested... requires msg to be a string literal)
#define LOG(lvl, msg, ...) \
syslog(lvl, msg " (in " __FILE__ ":%d)", __VA_ARGS__, __LINE__)
You may also find __func__ useful.
The redirection, filtering, etc. is done by creating configuration files. Here is an example from my snapwebsites project:
mail.err /var/log/mail/mail.err
mail.* /var/log/mail/mail.log
& stop
I install the file under /etc/rsyslog.d/ and run:
invoke-rc.d rsyslog restart
so the syslog server handles that change and saves any mail related logs to those folders.
Note: I also have to create the /var/log/mail folder and the files inside the folder to make sure it all works right (because otherwise the mail daemon may not have enough permissions.)
snaplogger (a little plug)
I've used log4cplus, which, since version 1.2.x, is quite good. I have three cons about it, though:
it requires me to completely clear everything if I want to call fork(); somehow it does not survive a fork(); call properly... (at least in the version I had it used a thread)
the configuration files (.properties) are not easy to manage in my environment where I like the administrators to make changes without modifying the original
it uses C++03 and we are now in 2019... I'd like to have at least C++11
Because of that, and especially because of point (1), I wrote my own version called snaplogger. This is not exactly a standalone project, though. I use many other projects from the snapcpp environment (it's much easier to just get snapcpp and run the bin/build-snap script or just get the binaries from launchpad.)
The advantage of using a logger such as snaplogger or log4cplus is that you generally can define any number of destinations and many other parameters (such as the severity level as offered by syslog()). The log4cplus is capable of sending its output to many different places: files, syslog, MS-Windows log system, console, a server, etc. Check out the appenders in those two projects to have an idea of the list of possibilities. The interesting factor here is that any log can be sent to all the destinations. This is useful to have a file named all.log where all your services send their logs. This allows to understand certain bugs which would not be as easy with separate log files when running many services in parallel.
Here is a simple example in a snaplogger configuration file:
[all]
type=file
lock=true
filename=/var/log/snapwebsites/all.log
[file]
lock=false
filename=/var/log/snapwebsites/firewall.log
Notice that for the all.log file I require a lock so multiple writers do not mangle the logs between each others. It's not necessary for the [file] section because I only have one process (no threads) for that one.
Both offer you a way to add your own appenders. So for example if you have a Qt application with an output window, you could write an appender to send the output of the SNAP_LOG_ERROR() calls to that window.
snaplogger also offers you a way to extend the variable support in messages (also called the format.) For example, I can insert the date using the ${date} variable. Then I can tweak it with a parameter. To only output the year, I use ${date:year}. This variable parameter support is also extensible.
snaplogger can filter the output by severity (like syslog), by a regex, and by component. We have a normal and a secure component, the default is normal. I want logs sent to the secure component to be written to secure files. This means in a sub-directory which is way more protected than the normal logs that most admins can review. When I run my HTTP services, some times I send information such as the last 3 digits of a credit card. I prefer to have those in a secure log. It could also be password related errors. Anything I deem to be a security risk in a log, really. Again, components are extensible so you can have your own.
One little point about the redirecter class. It needs to be destroyed properly, and only once. The destructor will ensure this will happen if the function it is declared in actually returns, and the object itself is never copied.
To ensure it can't be copied, provide private copy and assignment operators:
class redirecter
{
public:
redirecter(std::ostream & src, std::ostream & dst)
: src_(src), sbuf(src.rdbuf(dst.rdbuf())) {}
~redirecter() { src.rdbuf(sbuf); }
private:
std::ostream & src_;
std::streambuf * const sbuf_;
// Prevent copying.
redirecter( const redirecter& );
redirecter& operator=( const redirecter& );
};
I'm using this technique by redirecting std::clog to a log file in my main(). To ensure that main() actually returns, I place the guts of main() in a try/catch block. Then elsewhere in my program, where I might call exit(), I throw an exception instead. This returns control to main() which can then execute a return statement.
Basic Logger
#define myerr(e) {CriticalSectionLocker crit; std::cerr << e << std::endl;}
Used as myerr("ERR: " << message); or myerr("WARN: " << message << code << etc);
Is very effective.
Then do:
./programname.exe 2> ./stderr.log
perl parsestderr.pl stderr.log
or just parse stderr.log by hand
I admit this is not for extremely performance critical code. But who writes that anyway.