I have a multi-threaded sever application that I'm writing in C++ and I need to implement a good and fairly efficient logging system. By efficient I mean that whatever amount of logging is configured, the application shouldn't ever come to a grinding halt. So preferably there is some thread that is dedicated to writing it's log files.
I want to log each request that the server component handles in it's own file, having a rotation system that removes files older then some threshold. A request is handled by 2 threads, one that does some conversion work and the a worker-thread that is part of thread pool (BOOST threadpool) that does all the other actions (database gets, calculations, etc). So the logging need be threadsafe and I have to be able to configure it for levels and let each Logger class instance (my own logger that implements some library) accept a new file name. So that each new Logger instance is created for a specific request.
My ultimate question is: Which logging library allows me to have a new Log file for each request and allows me to configure log levels? (IE: error, warning, critical, etc)
Or should I implement everything myself? (no logging is not an option)
I have looked at Boost::Logging v2 and since the main logger object, that holds all state (levels, files) is global, I cannot change the files for each request.
I have looked at templog.org and this I can't even get to compile. No matter what I include or which references I set, it can never find the templog namespace or any of its classes.
Have a look at Apache log4cxx. It a great logging library !
Related
What is the best practice of logging DLL activity and internal parameter values.
I wrote a DLL that is used by a few applications I wrote. Lately, some users were complaining about problems that I suspect are related to this DLL but I cannot reproduce on my dev machine.
I'd like send them an equivalent DLL file that will log its activity, such as function calls, function return values and some internal parameters.
I don't think there should be a problem for a DLL to create a log file and write to it. Is there a common practice related to this issue. Is there a standard place to write the data to?
I am currently facing a similar issue of how to provide logging from a DLL. The way I went about it is to provide callback function setters and a callback type header file as part of the API. That way, the caller of the dll can create their own type of logging (boost, log4cpp, etc) and the dll will call the callback function in the exe that the user can do whatever they want with the data. In my case, I forward the log msgs to a boost logger that is instantiated in the exe. It's a little hacky looking but seems to work pretty well since the exe controls when and how to log everything and the user can format the data any way they choose.
You could provide an interface on your DLL which allows the user to provide a log file path/handle as the logging sink.
If you auto-generate the log file you'll probably want the file name to contain the process name and the process id (to disambiguate multiple simultaneous runs).
on unix machines the usual place is /var/log
on windows the standard way is to log to the event log (a separate subsystem of the OS).
I would suggest using the active user's TEMP directory if you want to log to file...
...or you could be clever and get the DLL to log to your own machines via TCP.
I am writing an application on Linux which requires me to catch system events like:
System reboot
User 'xyz' logs in
'xyz' application crashes etc.
and need to execute some functionality based on that. For e.g.:
Run backup script
Run recovery program etc.
Can anyone please tell me how to catch system events in C/Linux ?
P.S: I am not talking about 'file system' events here :P
There is no concept of "system event". You need to specify which events you need to handle and implement appropriate mechanism for handling each:
System startup: The init process calls scripts from /etc/init.d during startup. The exact infrastructure differs slightly between distributions, but Linux Standards Base System Initialization should generally work on all.
User login/logout: The LSB also defines interface to the Pluggable Authentication Modules library. You can implement a shared library that will be called during login (and also other actions that require authentication and authorization). Depending on what you want to do there may already be a module that will work for you, so try looking for it first. In either case I don't think there is distribution-independent way of installing it and even on given distribution you have to consider that administrator might have made custom modification, so the installation will need manual intervention by the administrator.
Application crashes: You would have to instrument it.
I think you should consider reading systems logs - everything you ask about is logged to the syslog (for standard configuration). If your system uses syslog-ng, then you could even configure it to write directly to your program, see http://www.syslog.org/syslog-ng/v2/#id2536904 for details. But even with any other syslog daemon, you can always read file (or files) from /var/log just like tail -f does, end react on particular messages.
I'm not sure about catching application crashes - there's a kernel option to log every SIGSEGV in user processes, but AFAIK it is available only on ARM architecture - last resort would be to instrument your application (as Jan Hudec pointed out) to log something to syslog.
My app will need to periodically access a trusted time source, so can not rely on system time since this one can be modified easily by user or batery failure etc. My first idea is to statically link to libntp (from ntp.org) and use its functions, is this a good idea?
Libntp looks a bit complex framework, is there some simpler, client implementation (preferably ANSI C since the app needs to be for different platforms... though can be also Cpp if can be compiled with gcc / MS VS)?
Is there some other alternative to ntp?
Thanks!!
Edit: Just to add some more info... it is important that the trusted-time-server values can not be modified (lets say, attacker modifies the trusted-time-server response and app accepts fake time). I started looking at ntp and see that it takes care of that issue. The question is now should i use ntp sources from ntp.org as a starting point or there are some simple client-only implementatios? Ideally, some pointer to which module / source files from ntp.org sources should I use for client-only implementation and which header file shows the API I need to use, like for example a call getTrustedTime()... etc.
If you can rely on there being a network connection, your application could ask a remote server for the time, perhaps also over a signed or encrypted connection.
If you are using Boost you could use this
I am having a problem with using Boost Logging library, that if I add a formatter or a destination to a logger, using my own Log class, I cannot change that destination or formatter.
Does anybody know how to change the destination or formatter on a boost log object?
The scenario I have is that I want a different destination (file name) for each request my server component handles, so I need to have flexible way to change them. Also the fact that I will be logging from different thread simultanuously, and each Log should really have it's own destination's, easily added - removed.
The fact that with the macro's the logging objects are really app global, does not really aid this.
Can anybody give me some guidance on how I can create a flexible way to add/remove destinations to a Logger from boost::logging?
Ok, here's what I would try. It might work for you. It looks as if the logging library is tailored for global loggers, while you are wanting to use thread-local loggers. I'd look up how to create a logger on demand (i.e. directly), for example by analysing BOOST_DECLARE_LOG. Then you can declare a std::map<int, Logger> that you use to map thread-id to specific logger. Probably you can create your own wrapper class that handles this transparently for client code. Then you just log using your own logging layer and create thread-specific loggers when needed. If you need to remove them after your request is finished you can add a method to do this.
Hope this helps!
I've encountered the following problem pattern frequently over the years:
I'm writing complex code for a package comprised of a standalone application and also a library version of the core that people can use from inside other apps.
Both our own app and presumably ones that users create with the core library are likely to be run both in batch mode (off-line, scripted, remote, and/or from command line), as well as interactively.
The library/app takes complex and large runtime input and there may be a variety of error-like outputs including severe error messages, input syntax warnings, status messages, and run statistics. Note that these are all incidental outputs, not the primary purpose of the application which would be displayed or saved elsewhere and using different methods.
Some of these (probably only the very severe ones) might require a dialog box if run interactively; but it needs to log without stalling for user input if run in batch mode; and if run as a library the client program obviously wants to intercept and/or examine the errors as they occur.
It all needs to be cross-platform: Linux, Windows, OSX. And we want the solution to not be weird on any platform. For example, output to stderr is fine for Linux, but won't work on Windows when linked to a GUI app.
Client programs of the library may create multiple instances of the main class, and it would be nice if the client app could distinguish a separate error stream with each instance.
Let's assume everybody agrees it's good enough for the library methods to log errors via a simple call (error code and/or severity, then printf-like arguments giving an error message). The contentious part is how this is recorded or retrieved by the client app.
I've done this many times over the years, and am never fully satisfied with the solution. Furthermore, it's the kind of subproblem that's actually not very important to users (they want to see the error log if something goes wrong, but they don't really care about our technique for implementing it), but the topic gets the programmers fired up and they invariably waste inordinate time on this detail and are never quite happy.
Anybody have any wisdom for how to integrate this functionality into a C++ API, or is there an accepted paradigm or a good open source solution (not GPL, please, I'd like a solution I can use in commercial closed apps as well as OSS projects)?
We use Apache's Log4cxx for logging which isn't perfect, but provides a lot of infrastructure and a consistent approach across projects. I believe it is cross-platform, though we only use it on Windows.
It provides for run time configuration via an ini file which allows you to control how the log file is output, and you could write your own appenders if you want specific behaviour (e.g. an error dialog under the UI).
If clients of your library also adopt it then it would integrate their logging output into the same log file(s).
Differentiation between instances of the main class could be supported using the nested diagnostic context (NDC) feature.
Log4Cxx should work for you. You need to implement a provider that allows the library user to catch the log output in callbacks. The library would export a function to install the callbacks. That function should, behind the scenes, reconfigure log4cxxx to get rid of all appenders and set up the "custom" appender.
Of course, the library user has an option to not install the callbacks and use log4cxx as is.