Related
I'm a member of a university team designing a cubesat (nanosatellite).
Another guy on the same subsystem was tasked to implement a logging library that we can use with the error stream.
The core changes happen in two files, Logger.hpp and Logger.cpp, respectively.
He #defines different "log levels", each level corresponding to the severity of an error:
#if defined LOGLEVEL_TRACE
#define LOGLEVEL Logger::trace
#elif defined LOGLEVEL_DEBUG
#define LOGLEVEL Logger::debug
#elif defined LOGLEVEL_INFO
[...]
#else
#define LOGLEVEL Logger::disabled
#endif
Levels are inside of an enum:
enum LogLevel {
trace = 32, // Very detailed information, useful for tracking the individual steps of an operation
debug = 64, // General debugging information
info = 96, // Noteworthy or periodical events
[...]
};
Additionally, he introduces the concept of "global level".
That is, only errors with a level as severe as the global level's one or higher will be logged.
To set the "global level", you need to set one of the constants mentioned above, such as LOGLEVEL_TRACE.
More on that below.
Last but not least, he creates a custom stream and uses some macro magic to make logging easy, just by using the << operator:
template <class T>
Logger::LogEntry& operator<<(Logger::LogEntry& entry, const T value) {
etl::to_string(value, entry.message, entry.format, true);
return entry;
}
This question is about the following piece of code; he introduces a fancy macro:
#define LOG(level)
if (Logger::isLogged(level)) \
if (Logger::LogEntry entry(level); true) \
entry
isLogged is just a helper constexpred function that compares each level with the "global" one:
static constexpr bool isLogged(LogLevelType level) {
return static_cast<LogLevelType>(LOGLEVEL) <= level;
}
I have never seen using macros like this, and before I go on with my question, here's his explanation:
Implementation details
This macro uses a trick to pass an object where the << operator can be used, and which is logged when the statement
is complete.
It uses an if statement, initializing a variable within its condition. According to the C++98 standard (1998), Clause 3.3.2.4,
"Names declared in the [..] condition of the if statement are local to the if [...]
statement (including the controlled statement) [...]".
This results in the Logger::LogEntry::~LogEntry() to be called as soon as the statement is complete.
The bottom if statement serves this purpose, and is always evaluated to true to ensure execution.
Additionally, the top `if` checks the sufficiency of the log level.
It should be optimized away at compile-time on invisible log entries, meaning that there is no performance overhead for unused calls to LOG.
This macro seems cool, but makes me somewhat uneasy and my knowledge isn't sufficient to be able to form a proper opinion.
So here goes:
Why would anyone choose to go with implementing a design as this?
What are the pitfalls to look out for with this approach, if any?
(bonus) If this approach isn't considered good practice, what could be done instead?
What surprised (and alerted) me the most is that while the idea behind this doesn't seem too complicated, I couldn't find a similar example anywhere on the internet.
I've come to learn that constexpr is my friend and that
macros can be dangerous
the preprocessor shouldn't be trusted
This is why a design built around a macro scares me, but I don't know whether this concern is valid, or whether it stems from my lack of understanding.
Lastly, I feel that I didn't phrase (and/or title) the question nearly as good as one could.
So feel free to modify it :)
One issue here is that the macro parameter is used twice. If some function is called or some other expression with side effects is used within the LOG() argument, that expression (which need not be a constant expression) could be evaluated twice. Maybe not a big deal, since there's little reason in this case to use anything other than a direct LogLevel enumerator in LOG().
One more dangerous pitfall: consider code like
if (!test_valid(obj))
LOG(Logger::info) << "Unexpected invalid input: " << obj;
else
result = compute(obj);
Expanding the macro turns this into
if (!test_valid(obj))
if (Logger::isLogged(Logger::info))
if (Logger::LogEntry entry(Logger::info); true)
entry << "Unexpected invalid input: " << obj;
else
result = compute(obj);
The compute function can never be called, no matter what the global log level is!
If your team does like this syntax, here's a way to get safer behavior. The if (declaration; expression) syntax implies at least C++17, so I assume other C++17 features. First, we'll need the LogLevel enumerators to be objects with different types so that the LOG expressions using them can have different behaviors.
namespace Logger {
template <unsigned int Value>
class pseudo_unscoped_enum
{
public:
constexpr operator unsigned int() const noexcept
{ return m_value; }
};
inline namespace LogLevel {
inline constexpr pseudo_unscoped_enum<32> trace;
inline constexpr pseudo_unscoped_enum<64> debug;
inline constexpr pseudo_unscoped_enum<96> info;
}
}
Next, define a dummy logger object that supports operator<< but does nothing.
namespace Logger {
struct dummy_logger {};
template <typename T>
dummy_logger& operator<<(dummy_logger& dummy, T&&)
{ return dummy; }
}
LOGLEVEL can keep its same macro definition. Finally, a couple of overloaded function templates replace the LOG macro (possibly in the global namespace):
#include <type_traits>
template <unsigned int Level,
std::enable_if_t<(Level >= LOGLEVEL), std::nullptr_t> = nullptr>
LogEntry LOG(pseudo_unscoped_enum<Level>) { return LogEntry(Level); }
template <unsigned int Level,
std::enable_if_t<(Level < LOGLEVEL), std::nullptr_t> = nullptr>
dummy_logger LOG(pseudo_unscoped_enum<Level>) { return {}; }
According to the description of if statement in cppreference.com, if you use an init-statement inside the if condition, like this:
if constexpr(optional) ( init-statement(optional) condition )
statement-true
else
statement-false
Then this will be equivalent to:
{
init_statement
if constexpr(optional) ( condition )
statement-true
else
statement-false
}
So, this means that in your case, the entry variable will go out of scope as soon as the scope of the whole if statement if finished. At this point, the destructor of the entry object is called and you will log some information about the instructions of the current scope. In addition, for using if constexpr statements, you should update your macro like this:
#define LOG(level)
if constexpr (Logger::isLogged(level)) \
...
Why would anyone choose to go with implementing a design as this?
So, using if constexpr statements allows your to check a condition at compile time and if the condition if false, do not compile the statement-true. If you are using logging statements a lot in the code and you do not want to make your binary bigger when there is no logging necessary, you can go on with this approach.
What are the pitfalls to look out for with this approach, if any?
I see no specific pitfalls with this design. It is just complex to understand. This is one of those cases that you cannot replace macros with something else, e.g. template functions.
I am trying to do a simple logging library only for my own. I know there exists several once, but I have not found any header-only, small and very "c++" like logging library which fits into my application.
Currently I have the following syntax:
logger << debug << "A debug message" << end; //debug and end is my custom manipulators
I have implemented all necessary operator<< and it works great, specially when it have backward compatibility with std::ostream. But I wonder, just for efficiency if it is a why to stop evaluate anything if some message should not be logged (after debug in the example)? Making everything after the severity manipulator "disappear"?
Just now do I have the following code in short:
template <typename Type>
Logger & Logger::operator<<(const Type & message)
{
if(this->get_message_level() <= this->get_severity())
{
BOOST_FOREACH(std::ostream* stream, this->_sinks)
{
*stream << message;
}
}
return *this;
}
Logger & Logger::operator<< (Logger & (*pf)(Logger&))
{
return pf(*this);
}
Logger & debug(Logger& logger)
{
logger.lock();
logger.set_severity(7);
//...
return logger;
}
Logger & end(Logger& logger)
{
logger << std::endl;
logger.unlock();
return logger;
}
Thanks in advance.
You could do something as simple as
extern "C" bool want_log;
#define LOG(Arg) do { if (want_log) \
cout << __FILE__ << ":" << __LINE__ ": " << Arg << endl; } while(0)
and use it as LOG("x=" << x)
It can be a bit tricky, depending on what compromizes you're willing to
accept in the syntax. If you really want to support everything that
outputting to an ostream does, then the best you can do (as far as I
know) is a wrapper class, along the lines of:
class Logger
{
std::ostream* myDest;
public:
// Appropriate constructors...
template<typename T>
Logger& operator<<( T const& obj )
{
if ( myDest != NULL )
(*myDest) << obj;
return *this;
}
};
If logging is turned off (Logger::myDest == NULL), none of the
conversion code will execute, but you'll still evaluate each of the
arguments. In my experience, this is not usually an issue, since most
of the arguments are either string literals or a simple variable, but
it's not a total 0 cost. It also has the potential disadvantage that
the propagated type is not std::ostream& (although I've never found
this to be a problem in practice).
A somewhat tricker solution would be to use a macro along the lines of:
#define logger loggerInstance != NULL && (*loggerInstance)
This will still allow most of the actual uses with the same syntax
(because the && operator has very low precedence), but could fail in
cases where someone has tried to be too clever, and embedded the logging
output in a more complicated expression. In addition to not doing the
conversions, the arguments are not even evaluated if logging is turned
off.
Finally, if you accept a different syntax, you can write something like:
#ifndef NDEBUG
#define LOG(argList) logger << argList
#else
#define LOG(argList)
#endif
This requires the user to write LOG("x = " << x), instead of log <<
"x = " << x, and requires recompiling if you want to turn logging on,
but it is the only solution I know which has absolute 0 cost if logging
is turned off.
In my experience, most applications can support the first solution; in a
very few cases, you might want to use the second; and I've never seen an
application where performance required the third.
Note that even with the first, you'll probably want to use a macro to
get the logger instance, in order to automatically insert __FILE__ and
__LINE__, and that in the second, you'll probably still want to use a
wrapper class, in order to ensure a flush in the destructor; if the
application is multithreaded, you'll want the wrapper in all cases, in
order to make the entire sequence of output atomic.
You could check for the severity in the Logger & Logger::operator<< (Logger & (*pf)(Logger&)) operator and just return an "empty" logger that doesn't print anything in that case:
class EmptyLogger : public Logger {
template <typename Type>
Logger & Logger::operator<<(const Type & message) { return *this; }
};
EmptyLogger empty; // Can be global/static, it won't cause any race conditions as it does nothing
Logger & Logger::operator<< (Logger & (*pf)(Logger&))
{
if (logger.get_severity() < 5) // Replace with a proper condition
return empty;
return pf(*this); // What does pf() do? Aren't you returning reference to a temporary object?
}
Take a look at this article in dr dobbs about logging in c++:
http://drdobbs.com/cpp/201804215?pgno=2
This page addresses your particular concern but I'd recommend reading the whole article.
I implemented a logging system based on this article and was really pleased with it.
Please note that this is NOT a duplicate of How write a unit test for verifying compiling error? as I'm not concerned about testing the correctness of external libraries or the compiler itself.
It is typical in C++, particularly when dealing with templates, to employ techniques that prevent some particular piece of code from being compiled. As these can get convoluted, what is the best way to ensure that particular pieces of code do indeed generate compiler errors?
As the test shouldn't even get compiled, you can't rely on things such as boost-test, so I guess it should be integrated in the build system? How are these issues usually approached?
Do it in the similar way compiler tests are written. You will have a bit of testing code in some scripting language (shell, perl, tcl etc.) that will run compiler on given snippets of code and check whether the right ones compiled and the right ones did not.
gcc uses DejaGnu, which is built on top of expect, which is itself built on top of Tcl.
If you use shell script (probably easier, DejaGnu is probably overkill), you might want to look at shUnit2.
Perl's Test::Harness system should be mostly easy to use as is.
After all, it's not that much more work to run process from C++, so writing a function to try to call compiler on a given string and check whether it outputs error for line where you expect it would not be that hard and than you can integrate it into the other boost.test-based tests.
Testing for a negative feature, hence provide a guarantee that certain construct will fail to compile is possible using c++20 requires expressions as follows:
Simple example
Below, I check if overloads to the function func exist in static assertions, when used with a test framework, the boolean should be used on one of the run time tests in order to not block the other tests from compiling:
#include <concepts>
/// Arbitrary restrictions in order to test:
/// if T != double -> zero args
template <typename T> void func(){};
/// if T == double -> arbitrary args.
template<std::same_as<double> ...T> void func(T... as){};
template <typename T, typename... a> constexpr bool applies_to_func = requires(a... as) {
func<T>(as...);
};
/// compiles:
static_assert(applies_to_func<int>);
static_assert(applies_to_func<double, double>);
static_assert(applies_to_func<double, double, double>);
/// function fails to compile:
static_assert(!applies_to_func<int, int>);
The code is available on Compiler Explorer: https://godbolt.org/z/direWo
C++17
I recently tried tried to do a similar thing for a project in which I can only use c++17. In my code I also check if the function's return type matches the caller's expectations. Aside from some limitations regarding the non-type template parameters, a similiar thing can be achieved as demonstrated below. In this case I could not enfroce double as input to the overload, due to the implicit conversion, applies_to_func(void, int, int) will evaluate to true in the code snipplet below.
#include <utility>
#include <string>
/// Create compile-time tests that allow checking a specific function's type
#define COMPILE_TIME_TEST(func) COMPILE_TIME_TEST_FUNCTION(func, func)
#define COMPILE_TIME_TEST_FUNCTION(name, func) \
namespace detail { \
template<typename R, auto... args> struct name ## FromArgs:std::false_type{}; \
template<auto... args> struct name ## FromArgs<decltype(func(args...)), args...> : std::true_type{}; \
template<typename R, typename... Args> struct name ## FromType:std::false_type{}; \
template<typename... Args> struct name ## FromType<decltype(func(std::declval<Args>()...)), Args...> : std::true_type{};\
} \
template<typename R, auto ...Args> \
static constexpr auto name ## _compiles = detail::name ## FromArgs<R, Args...>::value; \
template<typename ...Args> \
static constexpr auto name ## _compiles_from_type = detail::name ## FromType<Args...>::value;\
int func();
template <typename T> void func(T);
void func(double);
void func(double, double );
void func(double, double, double);
// create the structs from above macros for the function `func`
COMPILE_TIME_TEST(func);
static_assert(!func_compiles<void>);
static_assert(func_compiles<int>);
static_assert(func_compiles_from_type<void, double, double>);
static_assert(!func_compiles_from_type<void, double, double, double, double>);
static_assert(func_compiles<void, 1>);
static_assert(!func_compiles<void, 1, nullptr>);
You would have to rely on an external framework to run a set of compilation tests, e.g. makefiles, or hudson jobs and check for either compiler output or compiler artifacts. If the compilation is supposed to fail then there should not be an object file for the file under compilation. I am guessing you could write a plugin for hudson to do that or a simple batch script that runs a makefile that compiles all the testfiles that should fail or succeed and flag successes or failures accordingly.
In the simplest case you would just check for the existance of the '.o' file to see whether your test succeeded, in more complex cases you might want to look at the compiler output and verify that the error that is produce concurs with the error that you are expecting. That would depend on the compiler that you are using.
Going one level deeper would probably mean writing a compiler extension to do that (LLVM might be able to handle what you are asking for)
You might want to check out metatest - Unit testing framework for C++ template metaprograms (author's original post to the Boost mailing list). Get it here.
Publications related to the libraries here.
[This is a variant of #mutableVoid's answer above, but a little more explicit on two things that I had to think about a little. It concerns only concepts, which is what I wanted to get under test, so it only applies to C++20 onwards].
First, assume we have a concept that we wish to test. This can be simple and just defer to existing type traits:
template <class T>
concept integral = std::is_integral_v<T>;
Or it could be more complex, in this case, a concept that checks that type T has a size function that returns the type's size_type (and you can add more requirements to this "compound requirement", say, that it has a [] operator):
template<typename T>
concept sizeable = requires(T x) {
{x.size()} -> std::same_as<typename T::size_type>;
};
So, now we have some concepts. We want to test that these concepts work like we expect.
What we will need to do is to get a bool to test. Well, that's easy, because that's what concepts gives us naturally:
std::cout << std::string(typeid(integral<int>).name()) << std::endl;
std::cout << std::string(typeid(sizeable<int>).name()) << std::endl;
produces (with GCC):
b
b
So, we can check these, either with static_assert (i.e. mutableVoid's answer), which will fail your compilation if your concept isn't working:
static_assert(integral<int>);
static_assert(!integral<float>);
static_assert(sizeable<std::vector<int>>);
static_assert(!sizeable<int>);
You can prove to yourself that this works by removing a ! and observing that the compilation fails.
However, you may not want compilation to fail with a compiler error. If you'd rather feed this to your unit test framework, it doesn't have to be static_asserted:
void say(bool b, const std::string& s) {
std::cout << s << " = " << b << std::endl;
}
int main() {
say(integral<int>, "integral, int");
say(integral<float>, "integral, float");
say(sizeable<std::vector<int>>, "sizeable, vector of int");
say(sizeable<int>, "sizeable, int");
return 0;
}
This produces something like this:
integral, int = 1
integral, float = 0
sizeable, vector of int = 1
sizeable, int = 0
Now, you can plug this into whatever unit testing library you want, and you can check that your concepts aren't accidentally permitting types that you expect to fail.
However, do note there are some limitations:
You can't check that there are no unexpected type acceptances, because your tests would be infinite in size (the same as any negative test, really).
Although these tests will compile and run even if the concepts are broken, it's possible that other code will choke if the concepts are "allowing" types though that they shouldn't, and your overall build may well still fail. For example, if functions or classes are static_asserting their types for their own purposes.
Indeed, you may still want static_asserts in the main program compilation to prevent non-compliant code even being written in the first place: after all, compile-time correctness is a good thing. However, this gives you a chance to make sure that your concepts and static assertions are working together as expected. If the static_asserts are for some reason not compiled or are changed to be too permissive, you may not notice the concept is now defective.
I am looking for a portable way to implement lazy evaluation in C++ for logging class.
Let's say that I have a simple logging function like
void syslog(int priority, const char *format, ...);
then in syslog() function we can do:
if (priority < current_priority)
return;
so we never actually call the formatting function (sprintf).
On the other hand, if we use logging stream like
log << LOG_NOTICE << "test " << 123;
all the formating is always executed, which may take a lot of time.
Is there any possibility to actually use all the goodies of ostream (like custom << operator for classes, type safety, elegant syntax...) in a way that the formating is executed AFTER the logging level is checked ?
This looks like something that could be handled with expression templates. Beware, however, that expression templates can be decidedly non-trivial to implement.
The general idea of how they work is that the operators just build up a temporary object, and you pass that temporary object to your logging object. The logging object would look at the logging level and decide whether to carry out the actions embodied in the temporary object, or just discard it.
What I've done in our apps is to return a boost::iostreams::null_stream in the case where the logging level filters that statement. That works reasonably well, but will still call all << operators.
If the log level is set at compile time, you could switch to an object with a null << operator.
Otherwise, it's expression templates as Jerry said.
The easiest and most straight-forward way is to simply move the check outside of the formatting:
MyLogger log; // Probably a global variable or similar.
if (log.notice())
log << "notified!\n" << some_function("which takes forever to compute"
" and which it is impossible to elide if the check is inside log's"
" op<< or similar");
if (log.warn()) {
log << "warned!\n";
T x;
longer_code_computing(value_for, x); // easily separate out this logic
log << x;
}
If you really wanted to shorten the common case, you could use a macro:
#define LOG_NOTICE(logger) if (logger.notice()) logger <<
LOG_NOTICE(log) << "foo\n";
// instead of:
if (log.notice()) log << "foo\n";
But the savings is marginal.
One possible MyLogger:
struct MyLogger {
int priority_threshold;
bool notice() const { return notice_priority < current_priority; }
bool warn() const { return warn_priority < current_priority; }
bool etc() const { return etc_priority < current_priority; }
template<class T>
MyLogger& operator<<(T const &x) {
do_something_with(x);
return *this;
}
};
The problem here is mixing iostream-style operator overloading with a printf-like logging function – specifically translating manipulators and formatting flags/fields from iostreams into a format string. You could write to a stringstream and then chunk that to your syslog function, or try something fancier. The above MyLogger works easiest if it also contains an ostream reference to which it can forward, but you'll need a few more op<< overloads for iomanips (e.g. endl) if you do that.
For mine I made a debug_ostream class which has templated << operators. These operators check the debug level before calling the real operator.
You will need to define non-template overrides for const char* and std::ostream& (*x)(std::ostream&) because otherwise those don't work. I'm not sure why.
With inlining and high enough optimization levels the compiler will turn the whole output line into a single check of the debug level instead of one per output item.
I should add to this that this doesn't solve the original problem. For example if part of the debug line is to call an expensive function to get a value for output, that function will still be called. My solution only skips the formatting overhead.
Suppose I have a C++ macro CATCH to replace the catch statement and that macro receive as parameter a variable-declaration regular expression, like <type_name> [*] <var_name> or something like that. Is there a way to recognize those "fields" and use them in the macro definition?
For instance:
#define CATCH(var_declaration) <var_type> <var_name> = (<var_type>) exception_object;
Would work just like:
#define CATCH(var_type, var_name) var_type var_name = (var_type) exception_object;
As questioned, I'm using g++.
You can't do it with just macros, but you can be clever with some helper code.
template<typename ExceptionObjectType>
struct ExceptionObjectWrapper {
ExceptionObjectType& m_unwrapped;
ExceptionObjectWrapper(ExceptionObjectType& unwrapped)
: m_unwrapped(unwrapped) {}
template<typename CastType>
operator CastType() { return (CastType)m_wrapped; }
};
template<typename T>
ExceptionObjectWrapper<T> make_execption_obj_wrapper(T& eobj) {
return ExceptionObjectWrapper<T>(eobj);
}
#define CATCH(var_decl) var_decl = make_exception_obj_wrapper(exception_object);
With these definitions,
CATCH(Foo ex);
should work. I will admit to laziness in not testing this (in my defence, I don't have your exception object test with). If exception_object can only be one type, you can get rid of the ExceptionObjectType template parameter. Further more, if you can define the cast operators on the exception_object itself you can remove the wrappers altogether. I'm guessing exception_object is actually a void* or something though and your casting pointers.
What compiler are you using? I've never seen this done in the gcc preprocessor, but I can't say for sure that no preprocessor out there can implement this functionality.
What you could do however is run your script through something like sed to do your prepreprocessing so to speak before the preprocessor kicks in.
Hmmm... I'm just guessing, but why don't you try something like this : :)
#define CATCH(varType,varName) catch(varType& varName) { /* some code here */ }