Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I want to understand the basics of a logging library.
What exactly is the purpose of a logging library? I understand that a log is basically information about your application process during execution. One way to do it is by writing information in a file directly.
What is the purpose of designing a dedicated library such as glog for logging purposes?
Is my understanding of logging correct, or do I need to change it?
Can someone give a practical example to exhibit the importance of using a logging library?
What features should one look at while choosing a logging library?
How can logging be effectively employed during implementations?
Logging information during the execution of your application can help you understand what led to a bug or crash, giving you more context than you get from simply a report of a crash, a call stack or even a minidump. This is particularly important when you are getting bug or crash reports from people who are not developers and are not running under a debugger, either end users / customers or non-developers on your team.
My background is in games and logging can be particularly valuable with games for a few reasons. One is that many issues can relate to the specifics of the hardware on a system so logging information like what kind of GPU the user has, which graphics driver version they are running, etc. can be essential to debugging problems that only show up on a specific configuration. Another is that games have a simulation aspect where the state of the game is evolving over time in response to user input combined with simulation of things like physics, AI and the game rules. Understanding what was going on in the run up to a crash or bug helps figure out how to reproduce it and can give valuable clues to the root cause of the issue.
A logging library adds functionality that is useful for logging and goes beyond what is available from a simple printf. This includes things like:
The ability to control the amount of logging based on factors like debug vs. release builds and runtime settings like a -verbose flag.
The concept of 'channels' that can be independently enabled, disabled or set to a particular verbosity. For example, to debug a graphics issue you may want the 'graphics' channel set to maximum verbosity while muting the 'network' and 'audio' channels.
A flexible back end ranging from logging to a local file on disk to logging to a remote database over a network.
Thread safety so that logging behaves itself when potentially logging simultaneously from multiple different threads.
Automatic tagging of log entries with a timestamp and any other relevant information (channel, verbosity level, etc.).
As for how to make use of a logging library, that is somewhat dependent on your application, but here's some general suggestions:
Make good use of channels and verbosity levels if your logging library provides them (and it should). This will help you manage what can become a very large volume of log messages as your application grows.
If you encounter an unexpected but non-fatal condition and handle it, log some information about it in case it leads to unforeseen problems later on.
On application startup, log any information that might come in useful for reproducing rare errors later if you receive a bug or crash report from a customer. Err on the side of too much information, you never know what might be useful in advance. This might include things like CPU type, GPU model and driver version, available memory, OS version, available hard drive space, etc.
Log key state transitions so you can track what state your application was in and how it got there when you are debugging an issue.
A lot of programs use some sort of logging, and there is little point to re-inventing the wheel every time, even if the code is relatively simple.
Other libraries can use the logging library too, so instead of having to configure the log files for each library you include in a project, you can just configure the one logging library. This also means that any bugs that might appear in the logging code can be fixed by just replacing the one library instead of having to replace multiple libraries.
Finally, it makes code easier to read for other developers because they don't have to figure out how you implemented your custom logging.
Related
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 8 years ago.
Improve this question
I wrote own code, using C++ Win32 API and boost library. The code compiles to EXE application for windows. I can guarantee that it is malware free, but virustotal.com reports that 9 of 56 antivirus softwares will recognize the exe file as having malware.
I see no reason why this should happen. I noticed some time ago that compiling things with LCC-Win32 compiler raised some malware flags while compiling the same code with Visual C++ produced a clean EXE file, however now even Visual C++ produces EXE file which flags as malware at virustotal.
I can say that my build computer is not infected, since if I compile just Hello World code, or some other bigger application, then virustotal doesn't report any malware at all for the newly compiled exe, which it would if some crazy stuff happened behind my back.
Is there any way to get rid of the incorrect malware flags reported by virustotal? I mean, without changing my code (since I know it's clean, I wrote it). Can I report somewhere at virustotal that their virus test is broken? Do I have to contact all the antivirus companies whose antivirus triggers the flag individually, one by one, by email, asking them to fix their antivirus software? Is there any place where one can report a false positive?
I've worked in a leading antivirus company and couple of others. So trying to answer your questions.
Is there any way to get rid of the incorrect malware flags reported by
virustotal?
Antiviruses basically check executable for few suspecious symptoms. For example malicious packer used, entry point obfuscation, suspicious instruction set used, header info compromised etc. They essentially do it around executable's initial set of bytes. If any of these being incorporated in your executable behind your sense, antivirus will trigger it as malicious. If you want to "get rid of malicious flags" you have to narrow down on what causing them (It could be anything like: A function call, a module, specific post processing etc.) and then simply try to remove that root cause from your application.
At least if you can tell what type of malware is being reported by each antivirus for your executable, it would be helpful mitigating the problem.
I mean, without changing my code (since I know it's clean, I wrote
it).
Without changing the code if you thinking of editing directly executable binary to remove "those few flags", it's not simply that straight forward. (As your might have realized by reading what I written above on how antiviruses analyzes the file before they trigger it as malicious).
Also, you cannot claim that it's clean "because you wrote it". Because you actually code portion of it. May be there is third party library/component you are using unknowingly, which you are not aware it is causing to trigger the whole executable as malicious. (And moreover, if your system has been infected, your newly developed executable can get infected immediately after you build it. This happens behind your sense.)
Can I report somewhere at virustotal that their virus test is broken?
In your case, this is called "false positive". This is what virustotal says on their faq about false positives:
VirusTotal is detecting a legitimate software I have developed,
please remove the detections
VirusTotal acts simply as an information aggregator, presenting
antivirus results, file characterization tool outputs, URL scanning
engine results, etc. VirusTotal is not responsible for false positives
generated by any of the resources it uses, false positive issues
should be addressed directly with the company or individual behind the
product under consideration.
We can, however, help you in combatting false positives. VirusTotal
has built an early warning system regarding false positives whereby
developers can upload their software to a private store, such software
gets scanned on a daily basis with the latest antivirus signatures.
Whenever there is a change in the detections of any of your files, you
are immediately notified in order to mitigate the false positive as
soon as possible.
Do I have to contact all the antivirus companies whose antivirus
triggers the flag individually, one by one, by email, asking them to
fix their antivirus software? Is there any place where one can report a false positive?
Yes. As mentioned by virustotal in above faq, false positive issues
should be addressed directly with the company or individual behind the
product under consideration.
i want to log a lot of events in a dynamically search-algorithm (e.g. information about convergence to global optimum). This logging should have a switch to turn it off/on. Now there are a lot of possibilities to achieve that:
implement a log-version and a non-log-version of the algorithm -> redundancy
use macros -> ugly and not that safe
use a c++ logging library (or: use 1% of the functional range of a logging library).
I decided to use Pantheios, especially because of the performance claims made (don't want to lose much performance because of this logging, which is only needed in development). I have to admit, that i don't need much of the functionality of this library, but i thought i would be a much nicer/safer way to use it. Maybe there would be a better suited alternative i don't know of (i need compile-time decisions only for logging -> don't know if there are logging-libraries designed for that purpose).
After compiling, installing and finally linking (no success with manually linking, but with the scons building tool; using gcc -> explicit linking), i wanted to try a little example.
Let's reduce that to something like the following:
#include "pantheios/pantheios.hpp"
#include "pantheios/frontends/stock.h"
const PAN_CHAR_T PANTHEIOS_FE_PROCESS_IDENTITY[] = "pantheios_test"; // specify process identity
int main()
{
pantheios::log_ALERT("alert-event");
pantheios::log_DEBUG("debug-event");
pantheios::log_INFORMATIONAL("just information");
return 1;
}
Linking is done with the following files:
LIBS=['pantheios.1.core.gcc44', 'pantheios.1.be.fprintf.gcc44', 'pantheios.1.bec.fprintf.gcc44',
'pantheios.1.fe.all.gcc44', 'pantheios.1.util.gcc44']
The simple question is now: how to switch the console-output off/on? How to choose the severity-level which is given to the back-end?
Looking into the documentation lead me to a lot of functions which take a severity level, but these functions are automatically called once for initialization. I know that dynamic changes of severity-levels are possible. I don't know exactly, where i change these settings. They should be in the front-end component i think. Is linking to "fe_all" already some kind of decisions which level is logged ad which level isn't?
It should be relatively easy because in my case, i only need a compile-time decision about logging on/off.
Thanks for your help.
Sascha
Is linking to "fe_all" already some kind of decisions which level is logged ad which level isn't?
Short answer: yes.
fe.all enables all severity levels: "all" meesages are logged.
If you want better control, try fe.simple. By default it switches on everything in debug builds and everything except DEBUG and INFO in release. You can change the threshold at anytime - in debug or release - by calling the function pantheios_fe_simple_setSeverityThresdhold() (or something similar) in the fe.simple header (which is pantheios/frontends/fe.simple.h, iirc).
Be aware that Pantheios is described as a logging API library. I think the intention is that it provides an API, initialization control and output over an existing "rich" logging library. That it provides "stock" front-ends and back-ends is a convenience to users for (i) getting up to speed quickly, and (ii) simple requirements (for which your program may qualify).
btw, I think the Pantheios forums on the project site are pretty well supported, so you may want to post questions there as well as here.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
I'm looking for a C++ logging framework with the following features:
logs have a severity (info, warning, error, critical, etc)
logs are tagged with a module name
framework has a UI (or CLI) to configure for which modules we will actually log to file, and the minimum severity required for a log to be written to file.
has a viewer which lets me search per module, severity, module name, error name, etc
Not sure about the configuration from a UI or CLI. I've used both of these logging frameworks at one point or other.
https://sourceforge.net/projects/log4cplus/
https://logging.apache.org/log4cxx/index.html
It wouldn't be too hard to drive your logging based on a configuration file that could be editable by hand or through a quick and dirty GUI or CLI app. Might be a bit harder to adjust these dynamically but not too bad.
Update:
It looks like the proposed Boost.Log is now in Boost 1.54 which is at a stable release. If you are already using Boost than I would take a look at it.
No viewer but you could try pantheios. I have been using it for almost a year now and am quite happy with it.
I strongly suggest Pantheios, as it's the only one that's completely type-safe, and is also very efficient. It imposes a little work on the user, in selecting the right "front-end" and "back-end", but once you've got it working, you can just fix and forget.
It doesn't provide sophisticated logging facilities - e.g. rolling files - but that's by design, because it's intended to be used in combination with other logging libraries that have more functionality (but poorer performance / type-safety).
If you care about performance, I suggest you check out Pantheios. In particular, it's got very high performance, and it can be used in combination with other logging libraries -- it acts as an efficient and type-safe layer between the logging library (such as log4cxx) and your application code.
You could use wxWidgets and use it's excellent class for logging. It's rather easy and straightforward. For instance, you can create a dialog which gathers all your logs (e.g. wxLogError, wxLogMessage, wxLogDebug, etc.).
Pantheios is a good candidate in term of perormance but my personal preference is P7 library.
My internal tests (CPU i7-4870HQ, SSD) shows that P7 is faster than Pantheios.
Pantheios writes 1.8M logs lines per second (time & text message)
P7 writes 2.4M logs lines per second (time, thread, CPU core, function, file, line and text message)
I'm thinking about adding code to my application that would gather diagnostic information for later examination. Is there any C++ library created for such purpose? What I'm trying to do is similar to profiling, but it's not the same, because gathered data will be used more for debugging than profiling.
EDIT:
Platform: Linux
Diagnostic information to gather: information resulting from application logic, various asserts and statistics.
You might also want to check out libcwd:
Libcwd is a thread-safe, full-featured debugging support library for C++
developers. It includes ostream-based debug output with custom debug
channels and devices, powerful memory allocation debugging support, as well
as run-time support for printing source file:line number information
and demangled type names.
List of features
Tutorial
Quick Reference
Reference Manual
Also, another interesting logging library is pantheios:
Pantheios is an Open Source C/C++ Logging API library, offering an
optimal combination of 100% type-safety, efficiency, genericity
and extensibility. It is simple to use and extend, highly-portable (platform
and compiler-independent) and, best of all, it upholds the C tradition of you
only pay for what you use.
I tend to use logging for this purpose. Log4cxx works like a charm.
If debugging is what you're doing, perhaps use a debugger. GDB scripts are pretty easy to write up and use. Maintaining them in parallel to your code might be challenging.
Edit - Appending Annecdote:
The software I maintain includes a home-grown instrumentation system. Macros are used to queue log messages and configuration options control what classes of messages are logged and the level of detail to be logged. A thread processes the logging queue, flushing messages to file and rotating files as they become too large (which they commonly do). The system provides a lot of detail, but often all too often it provides huge files our support engineers must wade through for hours to find anything useful.
Now, I've only used GDB to diagnose bugs a few times, but for those issues it had a few nice advantages over the logging system. GDB scripting allowed me to gather new instrumentation data without adding new instrumentation lines and deploying a new build of my software to the client. GDB can generate messages from third-party libraries (needed to debug into openssl at one point). GDB adds no run-time impact to the software when not in use. GDB does a pretty good job of printing the contents of objects; the code-level logging system requires new macros to be written when new objects need to have their states logged.
One of the drawbacks was that the gdb scripts I generated had no explicit relationship to the source code; the source file and the gdb script were developed independently. Ideally, changes to the source file should impact and update the gdb script. One thought is to put specially-formatted comments in code and have a scripting language make a pass on the source files to generate the debugger script file for the source file. Finally, have the makefile execute this script during the build cycle.
It's a fun exercise to think about the potential of using GDB for this purpose, but I must admit that there are probably better code-level solutions out there.
If you execute your application in Linux, you can use "ulimit" to generate a core when your application crash (or assert(false), or kill -6 ), later, you can debug with gdb (gdb -c core_file binary_file) and analyze the stack.
Salu2.
PD. for profiling, use gprof
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
We are producing a portable code (win+macOs) and we are looking at how to make the code more rubust as it crashes every so often... (overflows or bad initializations usually) :-(
I was reading that Google Chrome uses a process for every tab so if something goes wrong then the program does not crash compleatelly, only that tab. I think that is quite neat, so i might give it a go!
So i was wondering if someone has some tips, help, reading list, comment, or something that can help me build more rubust c++ code (portable is always better).
In the same topic i was also wondering if there is a portable library for processes (like boost)?
Well many Thanks.
I've developed on numerous multi-platform C++ apps (the largest being 1.5M lines of code and running on 7 platforms -- AIX, HP-UX PA-RISC, HP-UX Itanium, Solaris, Linux, Windows, OS X). You actually have two entirely different issues in your post.
Instability. Your code is not stable. Fix it.
Use unit tests to find logic problems before they kill you.
Use debuggers to find out what's causing the crashes if it's not obvious.
Use boost and similar libraries. In particular, the pointer types will help you avoid memory leaks.
Cross-platform coding.
Again, use libraries that are designed for this when possible. Particularly for any GUI bits.
Use standards (e.g. ANSI vs gcc/MSVC, POSIX threads vs Unix-specific thread models, etc) as much as possible, even if it requires a bit more work. Minimizing your platform specific code means less overall work, and fewer APIs to learn.
Isolate, isolate, isolate. Avoid in-line #ifdefs for different platforms as much as possible. Instead, stick platform specific code into its own header/source/class and use your build system and #includes to get the right code. This helps keep the code clean and readable.
Use the C99 integer types if at all possible instead of "long", "int", "short", etc -- otherwise it will bite you when you move from a 32-bit platform to a 64-bit one and longs suddenly change from 4 bytes to 8 bytes. And if that's ever written to the network/disk/etc then you'll run into incompatibility between platforms.
Personally, I'd stabilize the code first (without adding any more features) and then deal with the cross-platform issues, but that's up to you. Note that Visual Studio has an excellent debugger (the code base mentioned above was ported to Windows just for that reason).
The Chrome answer is more about failure mitigation and not about code quality. Doing what Chrome is doing is admitting defeat.
Better QA that is more than just programmer testing their own work.
Unit testing
Regression testing
Read up on best practices that other
companies use.
To be blunt, if your software is crashing often due to overflows and bad initializations, then you have a very basic programming quality problem that isn't going to be easily fixed. That sounds a hash and mean, that isn't my intent. My point is that the problem with the bad code has to be your primary concern (which I'm sure it is). Things like Chrome or liberal use to exception handling to catch program flaw are only distracting you from the real problem.
You don't mention what the target project is; having a process per-tab does not necessarily mean more "robust" code at all. You should aim to write solid code with tests regardless of portability - just read about writing good C++ code :)
As for the portability section, make sure you are testing on both platforms from day one and ensure that no new code is written until platform-specific problems are solved.
You really, really don't want to do what Chrome is doing, it requires a process manager which is probably WAY overkill for what you want.
You should investigate using smart pointers from Boost or another tool that will provide reference counting or garbage collection for C++.
Alternatively, if you are frequently crashing you might want to perhaps consider writing non-performance critical parts of your application in a scripting language that has C++ bindings.
Scott Meyers' Effective C++ and More Effective C++ are very good, and fun to read.
Steve McConnell's Code Complete is a favorite of many, including Jeff Atwood.
The Boost libraries are probably an excellent choice. One project where I work uses them. I've only used WIN32 threading myself.
I agree with Torlack.
Bad initialization or overflows are signs of poor quality code.
Google did it that way because sometimes, there was no way to control the code that was executed in a page (because of faulty plugins, etc.). So if you're using low quality plug ins (it happens), perhaps the Google solution will be good for you.
But a program without plugins that crashes often is just badly written, or very very complex, or very old (and missing a lot of maintenance time). You must stop the development, and investigate each and every crash. On Windows, compile the modules with PDBs (program databases), and each time it crashes, attach a debugger to it.
You must add internal tests, too. Avoid the pattern:
doSomethingBad(T * t)
{
if(t == NULL) return ;
// do the processing.
}
This is very bad design because the error is there, and you just avoid it, this time. But the next function without this guard will crash. Better to crash sooner to be nearer from the error.
Instead, on Windows (there must be a similar API on MacOS)
doSomethingBad(T * t)
{
if(t == NULL) ::DebugBreak() ; // it will call the debugger
// do the processing.
}
(don't use this code directly... Put it in a define to avoid delivering it to a client...)
You can choose the error API that suits you (exceptions, DebugBreak, assert, etc.), but use it to stop the moment the code knows something's wrong.
Avoid the C API whenever possible. Use C++ idioms (RAII, etc.) and libraries.
Etc..
P.S.: If you use exceptions (which is a good choice), don't hide them inside a catch. You'll only make your problem worse because the error is there, but the program will try to continue and will probably crash sometimes after, and corrupt anything it touches in the mean time.
You can always add exception handling to your program to catch these kinds of faults and ignore them (though the details are platform specific) ... but that is very much a two edged sword. Instead consider having the program catch the exceptions and create dump files for analysis.
If your program has behaved in an unexpected way, what do you know about your internal state? Maybe the routine/thread that crashed has corrupted some key data structure? Maybe if you catch the error and try to continue the user will save whatever they are working on and commit the corruption to disk?
Beside writing more stable code, here's one idea that answers your question.
Whether you are using processes or threads. You can write a small / simple watchdog program. Then your other programs register with that watchdog. If any process dies, or a thread dies, it can be restarted by the watchdog. Of course you'll want to put in some test to make sure you don't keep restarting the same buggy thread. ie: restart it 5 times, then after the 5th, shutdown the whole program and log to file / syslog.
Build your app with debug symbols, then either add an exception handler or configure Dr Watson to generate crash dumps (run drwtsn32.exe /i to install it as the debugger, without the /i to pop the config dialog). When your app crashes, you can inspect where it went wrong in windbg or visual studio by seeing a callstack and variables.
google for symbol server for more info.
Obviously you can use exception handling to make it more robust and use smart pointers, but fixing the bugs is best.
I would recommend that you compile up a linux version and run it under Valgrind.
Valgrind will track memory leaks, uninitialized memory reads and many other code problems. I highly recommend it.
After over 15 years of Windows development I recently wrote my first cross-platform C++ app (Windows/Linux). Here's how:
STL
Boost. In particular the filesystem and thread libraries.
A browser based UI. The app 'does' HTTP, with the UI consisting of XHTML/CSS/JavaScript (Ajax style). These resources are embedded in the server code and served to the browser when required.
Copious unit testing. Not quite TDD, but close. This actually changed the way I develop.
I used NetBeans C++ for the Linux build and had a full Linux port in no time at all.
Build it with the idea that the only way to quit is for the program to crash and that it can crash at any time. When you build it that way, crashing will never/almost never lose any data. I read an article about it a year or two ago. Sadly, I don't have a link to it.
Combine that with some sort of crash dump and have it email you it so you can fix the problem.