I have a program that is running as a service for a long time. I use BOOST_LOG_TRIVIAL(info) << "Message" for logging various events. I would like to cooperate with logrotate and reopen my log file after rotation. In theory the screenplay works as follows:
My program starts, some logging happens
Log is renamed from something.log to something.log.1 by an other program like logrotate or by me manually.
My program continues to log to the something.log.1 file.
I send SIGHUP (or something) to my program so I can reopen the log file. But I don't know how to.
My prepared example so far (may not be necessary):
#include <iostream>
#include <stdexcept>
#include <csignal>
#include <boost/thread.hpp>
#include <boost/chrono.hpp>
#include <boost/log/trivial.hpp>
#include <boost/log/expressions.hpp>
#include <boost/log/sources/severity_logger.hpp>
#include <boost/log/sources/record_ostream.hpp>
#include <boost/log/utility/setup/file.hpp>
#include <boost/log/utility/setup/console.hpp>
#include <boost/log/utility/setup/common_attributes.hpp>
#include <boost/log/support/date_time.hpp>
inline
void setupLogging(std::string const &logFileName)
{
namespace logging = boost::log;
namespace src = boost::log::sources;
namespace expr = boost::log::expressions;
namespace keywords = boost::log::keywords;
logging::add_common_attributes();
auto format = expr::stream
<< expr::format_date_time< boost::posix_time::ptime >("TimeStamp", "%Y-%m-%d %H:%M:%S.%f")
<< " progname " << logging::trivial::severity
<< ": " << expr::smessage;
auto fileOutput = logging::add_file_log(
keywords::file_name = logFileName, keywords::format = format
, keywords::auto_flush = true, keywords::open_mode = std::ios::app
);
auto consoleOutput = logging::add_console_log(
std::cerr, keywords::format = format, keywords::auto_flush = true
);
BOOST_LOG_TRIVIAL(debug) << "CHECKPOINT # setupLogging() after log initialization.";
}
void my_signal_handler(int signal)
{
BOOST_LOG_TRIVIAL(info) << "my_signal_handler BEGIN";
/* REOPEN LOG HERE */
BOOST_LOG_TRIVIAL(info) << "my_signal_handler END";
}
int main()
{
setupLogging("logrotate.test.log");
if(signal(SIGHUP, my_signal_handler) == SIG_ERR)
{
BOOST_LOG_TRIVIAL(error) << "Failed to register signal handler";
return 1;
} else
BOOST_LOG_TRIVIAL(info) << "Signal handler registered.";
BOOST_LOG_TRIVIAL(info) << "CHECKPOINT 0";
for(size_t i=1; i<100; ++i)
{
boost::this_thread::sleep_for( boost::chrono::seconds(1) );
BOOST_LOG_TRIVIAL(info) << "CHECKPOINT " << i;
}
return 0;
}
Compiling:
LINK="-lboost_system -lboost_date_time -lboost_log -lboost_log_setup -lboost_thread -lboost_chrono -lpthread"
g++ -std=c++11 -Wextra -DBOOST_LOG_DYN_LINK -pedantic -O3 logrotate_test.cpp -o logrotate_test $LINK
Source code representation of Adam's answer
namespace detail666777888
{
using namespace boost;
using namespace boost::log;
typedef shared_ptr< sinks::synchronous_sink< sinks::text_file_backend > > T;
}
typedef detail666777888::T SPFileSink;
SPFileSink logFileSink;
void setupLogging(...){
... logFileSink = logging::add_file_log ...
}
void my_sighup_handler(int /*signal*/)
{
BOOST_LOG_TRIVIAL(info) << "my_sighup_handler START";
auto oldLFS = logFileSink;
setupLogging("logrotate.test.log");
boost::log::core::get()->remove_sink(oldLFS);
BOOST_LOG_TRIVIAL(info) << "my_sighup_handler FINISH";
}
I was facing the same problem and found a quite straightforward solution: Just initialize the text_file_backend with a single filename (no pattern whatsoever) and call text_file_backend::rotate_file(). This will close the current file and open a new one with the same name.
// create sink
auto backend = boost::make_shared< boost::log::sinks::text_file_backend >(
boost::log::keywords::file_name = "my.log",
boost::log::keywords::open_mode = std::ios_base::out | std::ios_base::app
);
auto sink = boost::make_shared<sinks::synchronous_sink<boost::log::sinks::text_file_backend>>(backend);
boost::log::core::get()->add_sink(sink);
// ... do some logging
// move log file and...
// reopen log file
sink->locked_backend()->rotate_file();
// ... do more logging
I open the log file with std::ios_base::app to prevent the logfile from being truncated if the file exists.
This solution has the advantage that no files need to be copied and log records will neither be lost nor duplicated.
Just use copytruncate logrotate option.
From manual:
Truncate the original log file in place after creating a copy, instead
of moving the old log file and optionally creating a new one. It can
be used when some program cannot be told to close its logfile and thus
might continue writing (appending) to the previous log file forever.
Note that there is a very small time slice between copying the file
and truncating it, so some logging data might be lost. When this
option is used, the create option will have no effect, as the old log
file stays in place.
http://linux.die.net/man/8/logrotate
EDIT:
Try this. Take a look at add_file_log[1] function source code. Then:
remember the object you have added with add_file_log (it returns sink)
when you receive the signal remove_sink[2] and add new one sink with add_file_log[1] -- (logs entries may leak when they are generated by other thread)
[1] http://www.boost.org/doc/libs/1_55_0/boost/log/utility/setup/file.hpp
[2] http://www.boost.org/doc/libs/1_55_0/boost/log/core/core.hpp
Related
Use the answer in the question: simultaneous read and write to child's stdio using boost.process,
I refactored the code and hybridized the new method using the Boost library. I've been successful in making a pipes connection with Stockfish, but this is also where I get errors I've never seen before, not even Google helps.
Here is what I have tried:
#include <stdio.h>
#include <time.h>
#include <string>
#include <memory.h>
#include <unistd.h>
#include <iostream>
#include <stddef.h>
#include <execinfo.h>
#include <errno.h>
#include <stdlib.h>
#include <sys/stat.h>
#include <fstream>
#include </usr/local/include/backtrace.h>
#include </usr/local/include/backtrace-supported.h>
#include <boost/process.hpp>
#include <boost/asio.hpp>
#include <boost/process/async.hpp>
#include <vector>
#include <iomanip>
#include <stdlib.h>
#include <string.h>
using namespace std;
namespace bp = boost::process;
using boost::system::error_code;
using namespace std::chrono_literals;
string errDetails = "Error Details: ";
void delay(int number_of_seconds) {
int ms = 1000 * number_of_seconds;
clock_t start_time = clock();
while (clock() < start_time + ms)
;
}
static void full_write(int fd, const char* buf, size_t len) {
while (len > 0) {
ssize_t ret = write(fd, buf, len);
if ((ret == -1) && (errno != EINTR)) {
break;
}
buf += (size_t) ret;
len -= (size_t) ret;
}
}
void print_backtrace() {
static const char start[] = "--------BACKTRACE--------\n\n";
static const char end[] = "-------------------------\n\n";
void *bt[1024];
int bt_size;
char **bt_syms;
int i;
bt_size = backtrace(bt, 1024);
bt_syms = backtrace_symbols(bt, bt_size);
full_write(STDERR_FILENO, start, strlen(start));
full_write(STDERR_FILENO, errDetails.c_str(), strlen(errDetails.c_str()));
for (i = 1; i < bt_size; i++) {
size_t len = strlen(bt_syms[i]);
full_write(STDERR_FILENO, bt_syms[i], len);
full_write(STDERR_FILENO, "\n", 1);
}
full_write(STDERR_FILENO, end, strlen(end));
free(bt_syms);
}
void abort_application() {
size_t memLeakCount, staticMemLeakCount;
uint64_t memLeakSize, staticMemLeakSize;
for (int i = 0; i < 3; i++) {
/**
* Delay
*/
delay(1);
}
print_backtrace();
abort();
}
inline bool stockfish_check_exists(const std::string& name) {
struct stat buffer;
return (stat(name.c_str(), &buffer) == 0);
}
int main() {
std::future<std::string> data;
boost::asio::io_service svc;
bp::async_pipe in{svc}, out{svc};
string proc = "";
char command[64];
string output = "";
if (stockfish_check_exists("stockfish")) {
proc = "stockfish"; } else {
errDetails = "Stockfish not found!\n\n";
abort_application();
}
std::string const program_dir = proc;
auto on_exit = [](int code, std::error_code ec) {
std::cout << "Exited " << code << "(" << ec.message() << ")\n";
};
bp::child process(proc, bp::std_in < in, svc);
boost::asio::streambuf recv_buffer;
std::cout << "uci send" << std::endl;
boost::asio::async_write(in, boost::asio::buffer("uci\n"),
[&](boost::system::error_code ec, size_t transferred) {
std::cout << "Write: " << transferred << "\n" << std::endl;
in.close();
}
);
std::cout << "isready send" << std::endl;
boost::asio::async_write(in, boost::asio::buffer("isready\n"),
[&](boost::system::error_code ec, size_t transferred) {
std::cout << "Write: " << transferred << "\n" << std::endl;
in.close();
}
);
cout << "Enter your command: ";
cin >> command;
cout << "Your command is: " << command << endl;
if (strcmp(command, "quit") == 0) {
cout << "Quiting......." << endl;
boost::asio::async_write(in, boost::asio::buffer("quit"),
[&](boost::system::error_code ec, size_t transferred) {
std::cout << "Write: " << transferred << std::endl;
in.close();
cout << "Engine quit!" << endl;
}
);
}
svc.run();
return 0;
}
To make it easier to follow, I left out std::std_out > out at the line:
bp::child process(proc, bp::std_in < in, svc);
so that the engine results are immediately displayed in the Terminal window, so I'll know if I've gone astray. And this is when I discovered the strange thing
When I launch the application, it outputs on Terminal as follows:
[2022-01-14 20:25:55]
duythanh#DuyThanhs-MacBook-Pro:/Volumes/Data/ChessGUI$ ./ChessGUI
uci send
isready send
Enter your command: Stockfish 120122 by the Stockfish developers (see AUTHORS file)
id name Stockfish 120122
id author the Stockfish developers (see AUTHORS file)
option name Debug Log File type string default
option name Threads type spin default 1 min 1 max 512
option name Hash type spin default 16 min 1 max 33554432
option name Clear Hash type button
option name Ponder type check default false
option name MultiPV type spin default 1 min 1 max 500
option name Skill Level type spin default 20 min 0 max 20
option name Move Overhead type spin default 10 min 0 max 5000
option name Slow Mover type spin default 100 min 10 max 1000
option name nodestime type spin default 0 min 0 max 10000
option name UCI_Chess960 type check default false
option name UCI_AnalyseMode type check default false
option name UCI_LimitStrength type check default false
option name UCI_Elo type spin default 1350 min 1350 max 2850
option name UCI_ShowWDL type check default false
option name SyzygyPath type string default <empty>
option name SyzygyProbeDepth type spin default 1 min 1 max 100
option name Syzygy50MoveRule type check default true
option name SyzygyProbeLimit type spin default 7 min 0 max 7
option name Use NNUE type check default true
option name EvalFile type string default nn-ac07bd334b62.nnue
uciok
Unknown command: isready
Contrasting with the code above, the two commands were sent through pipes. is uci and isready, this is fine. The first uci command runs successfully, but the isready command, instead of returning readyok, it returns:
Unknown command: isready
I keep trying to type quit, which sends a quit command to the pipe as the exit engine, and it also fails:
Your command is: quit
Quiting.......
Write: 5
Write: 9
Unknown command: quit
Write: 5
Engine quit!
The program will then exit with the engine. I'm still wondering what was going on at the time, but the clues are really hazy as to what was going on behind the scenes.
Please help me. Any help is highly appreciated. Thank you so much everyone
UPDATE: The error continued when Unknown Command: Quit appeared. I typed these commands in Terminal while running Stockfish directly through Terminal, they work as a result, but my program still can't
You are printing to cout as if the async operations happen immediately. That's not the case. The async operations only happen when the io service runs.
svc.run();
Is at the very end of your code. So no async_ operation ever completes (or even starts) before that.
Other problems:
Your out async pipe is never used (not even connected). It's unclear to me how you intend to communicate with the child process that way.
In fairness, you only every write to the child process, so maybe you're not at all interested in the output. (But then perhaps recv_buffer can be deleted just as well).
Your buffers include the terminating NUL characters. (asio::buffer("uci\n") sends {'u','c','i','\n','\0'}). That's going to mess up the child processes's parsing.
You do in.close() in response to every single async_write completion. This guarantees that subsequent writes never can happen, as you closed the pipe.
Then when you send quit you fail to include the '\n' as well
You are reading into a char[64] with operator>> which makes no sense at all. Maybe you are using c++20 (so width of 64 might be assumed) but you never set a width. Most likely you would want to read into a string instead.
However, doing so cannot accept commands with whitespace (because std::ios::skipws is set by default). So, likely you wanted std::getline instead...
The fact that you include a boatload of C headers makes me think you're porting some C code (badly). That's also exemplified by the strcmp use and others, e.g. no need to use ::stat
Don't use using namespace std; (Why is "using namespace std;" considered bad practice?)
Don't use global variables (errDetails)
Don't use loops to wait for a time delay
No need to manually print backtraces. Instead, use Boost:
void abort_application(std::string const& errDetails) {
std::cerr << errDetails << "\n";
std::cerr << boost::stacktrace::stacktrace{} << std::endl;
std::this_thread::sleep_for(3s);
abort();
}
Existing Stockfish Client: Playing Games
You're in luck: I have a written full demo using stockfish on this site: Interfacing with executable using boost in c++.
This example shows how to correctly await and parse expected replies from the child process(es).
You will note that I chose coroutines for the async version:
Just for completeness, I thought I'd try an asynchronous implementation. Using the default Asio callback style this could become unwieldy, so I thought to use Boost Coroutine for the stackful coroutines. That makes it so the implementation can be 99% similar to the synchronous version
Just for comparison, here's what your code should look like if you didn't use coroutines:
Fixing Up Your Code
Live On Coliru
#include <boost/asio.hpp>
#include <boost/process.hpp>
#include <boost/process/async.hpp>
#include <boost/stacktrace/stacktrace.hpp>
#include <chrono>
#include <iomanip>
#include <iostream>
namespace bp = boost::process;
using boost::system::error_code;
using namespace std::literals;
static void abort_application(std::string const& errDetails) {
std::cerr << errDetails << "\n";
std::cerr << boost::stacktrace::stacktrace{} << std::endl;
std::this_thread::sleep_for(3s);
abort();
}
inline static bool stockfish_check_exists(std::string& name) {
return boost::filesystem::exists(name);
}
int main() {
boost::asio::io_service svc;
bp::async_pipe in{svc};
std::string proc = "/usr/games/stockfish";
if (!stockfish_check_exists(proc)) {
abort_application("Stockfish not found!");
}
auto on_exit = [](int code, std::error_code ec) {
std::cout << "Exited " << code << "(" << ec.message() << ")\n";
};
bp::child process(proc, bp::std_in < in, svc, bp::on_exit = on_exit);
std::function<void()> command_loop;
std::string command_buffer;
command_loop = [&] {
std::cout << "Enter your command: " << std::flush;
// boost::asio::streambuf recv_buffer;
if (getline(std::cin, command_buffer)) {
std::cout << "Your command is: " << command_buffer << std::endl;
command_buffer += '\n';
async_write( //
in, boost::asio::buffer(command_buffer),
[&](error_code ec, size_t transferred) {
std::cout << "Write: " << transferred << " (" << ec.message() << ")" << std::endl;
if (command_buffer == "quit\n") {
std::cout << "Quiting......." << std::endl;
// in.close();
std::cout << "Engine quit!" << std::endl;
} else {
command_loop(); // loop
}
});
}
};
std::cout << "uci send" << std::endl;
async_write(
in, boost::asio::buffer("uci\n"sv),
[&](error_code ec, size_t transferred) {
std::cout << "Write: " << transferred << "\n" << std::endl;
std::cout << "isready send" << std::endl;
async_write(in, boost::asio::buffer("isready\n"sv),
[&](error_code ec, size_t n) {
std::cout << "Write: " << n << std::endl;
command_loop(); // start command loop
});
});
svc.run(); // only here any of the operations start
}
Prints, e.g.
Or if Stockfish is in fact installed:
I have a code similar to this:
const auto jsonFormatter = boost::log::expressions::stream << boost::log::expressions::smessage;
I want to escape the message using nlohmann::json, something like:
nlohmann::json json{boost::log::expressions::smessage};
I can do following to convert boost::log::expressions::smessage to std::string:
std::stringstream ss;
ss << boost::log::expressions::smessage;
std::string message = ss.str();
nlohmann::json json{message};
, but I need to put it inside the formatter, because the
const auto jsonFormatter = boost::log::expressions::stream << nlohmann::json{boost::log::expressions::smessage};
can't convert the boost::log::expressions::smessage argument to any nlohmann::json constructor.
Any suggestions how to make it work?
Log formatters look like normal C++, but they are expression templates that compose deferred calleable that do the corresponding action.
Here's how you can make a wrapper expression that knows how to do this:
namespace {
struct as_json_t {
template <typename E> auto operator[](E fmt) const {
return expr::wrap_formatter(
[fmt](logging::record_view const& rec,
logging::formatting_ostream& strm) {
logging::formatting_ostream tmp;
std::string text;
tmp.attach(text);
fmt(rec, tmp);
strm << nlohmann::json{text};
});
}
};
inline constexpr as_json_t as_json;
} // namespace
Now you can make your formatter e.g.
logging::formatter formatter = expr::stream
<< expr::format_date_time(timestamp, "%Y-%m-%d, %H:%M:%S.%f") << " "
<< logging::trivial::severity
<< " - " << as_json[expr::stream << expr::smessage]
;
ANd the result is e.g.
2021-01-15, 23:34:08.489173 error - ["this is an error message"]
Live Demo
Live On Wandbox
File simpleLogger.h
#ifndef _HOME_SEHE_PROJECTS_STACKOVERFLOW_SIMPLELOGGER_H
#define _HOME_SEHE_PROJECTS_STACKOVERFLOW_SIMPLELOGGER_H
#pragma once
#define BOOST_LOG_DYN_LINK \
1 // necessary when linking the boost_log library dynamically
#include <boost/log/sources/global_logger_storage.hpp>
#include <boost/log/trivial.hpp>
// the logs are also written to LOGFILE
#define LOGFILE "logfile.log"
// just log messages with severity >= SEVERITY_THRESHOLD are written
#define SEVERITY_THRESHOLD logging::trivial::warning
// register a global logger
BOOST_LOG_GLOBAL_LOGGER(logger, boost::log::sources::severity_logger_mt<
boost::log::trivial::severity_level>)
// just a helper macro used by the macros below - don't use it in your code
#define LOG(severity) \
BOOST_LOG_SEV(logger::get(), boost::log::trivial::severity)
// ===== log macros =====
#define LOG_TRACE LOG(trace)
#define LOG_DEBUG LOG(debug)
#define LOG_INFO LOG(info)
#define LOG_WARNING LOG(warning)
#define LOG_ERROR LOG(error)
#define LOG_FATAL LOG(fatal)
#endif
File simpleLogger.cpp
#include "simpleLogger.h"
#include <boost/core/null_deleter.hpp>
#include <boost/log/core/core.hpp>
#include <boost/log/expressions.hpp>
#include <boost/log/expressions/formatters/char_decorator.hpp>
#include <boost/log/expressions/formatters/date_time.hpp>
#include <boost/log/sinks/sync_frontend.hpp>
#include <boost/log/sinks/text_ostream_backend.hpp>
#include <boost/log/sources/severity_logger.hpp>
#include <boost/log/support/date_time.hpp>
#include <boost/log/trivial.hpp>
#include <boost/log/utility/setup/common_attributes.hpp>
#include <nlohmann/json.hpp>
#include <fstream>
namespace logging = boost::log;
namespace src = boost::log::sources;
namespace expr = boost::log::expressions;
namespace sinks = boost::log::sinks;
namespace attrs = boost::log::attributes;
BOOST_LOG_ATTRIBUTE_KEYWORD(timestamp, "TimeStamp", boost::posix_time::ptime)
BOOST_LOG_ATTRIBUTE_KEYWORD(severity, "Severity", logging::trivial::severity_level)
namespace {
struct as_json_t {
template <typename E> auto operator[](E fmt) const {
return expr::wrap_formatter(
[fmt](logging::record_view const& rec,
logging::formatting_ostream& strm) {
logging::formatting_ostream tmp;
std::string text;
tmp.attach(text);
fmt(rec, tmp);
strm << nlohmann::json{text};
});
}
};
inline constexpr as_json_t as_json;
} // namespace
BOOST_LOG_GLOBAL_LOGGER_INIT(logger, src::severity_logger_mt) {
src::severity_logger_mt<boost::log::trivial::severity_level> logger;
// add attributes
logger.add_attribute("TimeStamp", attrs::local_clock()); // each log line gets a timestamp
// add a text sink
using text_sink = sinks::synchronous_sink<sinks::text_ostream_backend>;
boost::shared_ptr<text_sink> sink = boost::make_shared<text_sink>();
// add a logfile stream to our sink
sink->locked_backend()->add_stream(
boost::make_shared<std::ofstream>(LOGFILE));
// add "console" output stream to our sink
sink->locked_backend()->add_stream(
boost::shared_ptr<std::ostream>(&std::clog, boost::null_deleter()));
// specify the format of the log message
logging::formatter formatter = expr::stream
<< expr::format_date_time(timestamp, "%Y-%m-%d, %H:%M:%S.%f") << " "
<< logging::trivial::severity
<< " - " << as_json[expr::stream << expr::smessage]
;
sink->set_formatter(formatter);
// only messages with severity >= SEVERITY_THRESHOLD are written
sink->set_filter(severity >= SEVERITY_THRESHOLD);
// "register" our sink
logging::core::get()->add_sink(sink);
return logger;
}
File test.cpp
#include "simpleLogger.h"
int main() {
LOG_TRACE << "this is a trace message";
LOG_DEBUG << "this is a debug message";
LOG_WARNING << "this is a warning message";
LOG_ERROR << "this is an error message";
LOG_FATAL << "this is a fatal error message";
return 0;
}
Prints
2021-01-15, 23:50:03.130250 warning - ["this is a warning message"]
2021-01-15, 23:50:03.130327 error - ["this is an error message"]
2021-01-15, 23:50:03.130354 fatal - ["this is a fatal error message"]
In addition to sehe's answer, you could achieve the JSON-like format using Boost.Log components. The essential part is the c_decor character decorator, which ensures that its output can be used as a C-style string literal.
namespace expr = boost::log::expressions;
const auto jsonFormatter =
expr::stream << "[\""
<< expr::c_decor[ expr::stream << expr::smessage ]
<< "\"]";
First, c_decor will escape any control characters in the messages to C-style escape sequences, like \n, \t. It will also escape double quote characters. Then, the surrounding brackets and double quotes are added to make the output compatible with JSON format.
If you have non-ASCII characters in your log messages and you want the formatted log records to be strictly ASCII, you can use c_ascii_decor instead of c_decor. In addition to what c_decor does, it will also escape any bytes greater than 127 to their hex escape sequences, e.g. \x8c.
I really appreciate help and the other answers, which may be better in other cases, but long story short, I tried a lot of solutions and went with this one in the end (I can update it if it can be improved):
#include <boost/phoenix/bind/bind_function.hpp>
..
nlohmann::json EscapeMessage(
boost::log::value_ref<std::string, boost::log::expressions::tag::smessage> const& message)
{
return message ? nlohmann::json(message.get()) : nlohmann::json();
}
..
const auto jsonFormatter = boost::log::expressions::stream << boost::phoenix::bind(&EscapeMessage, boost::log::expressions::smessage.or_none())
boost::log::add_console_log(std::cout, boost::log::keywords::format = jsonFormatter);
Need to set custom facet for posix_time::ptime which are printed with BOOST_LOG_TRIVIAL. Say, change default facet to "%Y--%m--%d %H:%M:%S".
I've tried to imbue log sink with new locale but had no success.
The same imbueing trick works perfectly well when applied to std::cout.
What could be the problem and is there a way to deal with it?
#include <boost/log/trivial.hpp>
#include <boost/log/utility/setup.hpp>
#include <boost/log/support/date_time.hpp>
#include <boost/date_time/posix_time/posix_time.hpp>
#include <locale>
namespace logging = boost::log;
namespace pt = boost::posix_time;
void main(void)
{
// console sink
auto sink = logging::add_console_log(std::cout);
// standart output
boost::posix_time::ptime t1(boost::posix_time::time_from_string("2014-11-23 23:59:59.117"));
BOOST_LOG_TRIVIAL(info) << "before " << t1;
// adding custom facet...
pt::time_facet *facet = new boost::posix_time::time_facet("%Y--%m--%d %H:%M:%S");
sink->imbue(std::locale(sink->getloc(), facet));
// ... but output doesn't change
BOOST_LOG_TRIVIAL(info) << "after " << t1;
}
Use sink->set_formatter like this
sink->set_formatter
(
expr::stream << expr::format_date_time< boost::posix_time::ptime >("TimeStamp", "%Y--%m--%d %H:%M:%S")
);
Also, make sure you call logging::add_common_attributes();
Working example: Coliru
#include <boost/log/trivial.hpp>
#include <boost/log/utility/setup.hpp>
#include <boost/log/support/date_time.hpp>
#include <boost/date_time/posix_time/posix_time.hpp>
#include <boost/log/utility/setup/common_attributes.hpp>
#include <boost/log/expressions/formatters/date_time.hpp>
namespace logging = boost::log;
namespace pt = boost::posix_time;
namespace expr = boost::log::expressions;
int main(void)
{
// console sink
auto sink = logging::add_console_log(std::cout);
// standart output
boost::posix_time::ptime t1(boost::posix_time::time_from_string("2014-11-23 23:59:59.117"));
BOOST_LOG_TRIVIAL(info) << "before " << t1;
sink->set_formatter
(
expr::stream << expr::format_date_time< boost::posix_time::ptime >("TimeStamp", "%Y--%m--%d %H:%M:%S")
);
sink->imbue(sink->getloc());
logging::add_common_attributes();
// ... but output doesn't change
BOOST_LOG_TRIVIAL(info) << "after " << t1;
return 0;
}
Output:
before 2014-Nov-23 23:59:59.117000
2015--09--25 10:37:42
The problem is that the locale in the sink is not used for log message string formatting. It can't be used because there can be multiple sinks and the choice is ambiguous. That locale is used for log record formatting, which is performed for every sink individually.
The log message string is composed with a stream that is initialized for every log message, so you can't imbue it with a custom locale in a way that keeps the locale used for multiple records.
What you can do is set your custom locale as the global one somewhere in the initialization code of your application, before you use Boost.Log.
void main(void)
{
pt::time_facet *facet = new boost::posix_time::time_facet("%Y--%m--%d %H:%M:%S");
std::locale::global(std::locale(std::locale(), facet));
// console sink
auto sink = logging::add_console_log(std::cout);
// ...
BOOST_LOG_TRIVIAL(info) << "after " << t1;
}
Note that this will affect all formatting in your application, for logging purposes and not.
If that is not acceptable you can try different approaches. First, you can write a stream manipulator that will imbue the stream with a locale before formatting date/time objects. To automate the use of this manipulator you can define your own logging macro.
struct locale_manip {};
std::ostream& operator<< (std::ostream& strm, locale_manip)
{
pt::time_facet *facet = new boost::posix_time::time_facet("%Y--%m--%d %H:%M:%S");
strm.imbue(std::locale(std::locale(), facet));
return strm;
}
#define MY_LOG(sev) BOOST_LOG_TRIVIAL(sev) << locale_manip()
void main(void)
{
// ...
MY_LOG(info) << "after " << t1;
}
Another solution is to transform your logging statements so that the date/time objects are added to log records as attributes. Then you can use the formatters and the sink-specific locale to customize the output.
I am trying to make my team go away from log4cxx and try to use Boost.Log v2 instead. Our current log4cxx pattern is rather simple:
log4cxx::helpers::Properties prop;
prop.setProperty("log4j.rootLogger","DEBUG, A1");
prop.setProperty("log4j.appender.A1","org.apache.log4j.ConsoleAppender");
prop.setProperty("log4j.appender.A1.layout","org.apache.log4j.PatternLayout");
prop.setProperty("log4j.appender.A1.layout.ConversionPattern","%d{ABSOLUTE} %-5p [%c] %m%n");
log4cxx::PropertyConfigurator::configure(prop);
However I failed to find a solution to support filename and line number printing. I've found so far an old post, but there is no clear solution (no accepted solution). I've looked into using those BOOST_LOG_NAMED_SCOPE but they feel very ugly as it make it impossible to print the proper line number when multiple of those are used within the same function.
I've also found a simpler direct solution, here. But that also feel ugly, since it will print the fullpath, instead of just the basename (it is not configurable). So the fullpath and line number are always printed on a fixed location (before expr::smessage).
I've found also this post, which looks like an old solution.
And finally the most promising solution, I found was here. However it does not even compile for me.
So my question is simply: how can I use Boost.Log v2 to print filename (not fullpath) and line number with minimal formatter flexibility (no solution with MACRO and __FILE__ / __LINE__ accepted, please). I'd appreciate a solution which does not involve BOOST_LOG_NAMED_SCOPE, as described here:
If instead you want to see line numbers of particular log records then
the best way is to define a custom macro which you will use to write
logs. In that macro, you can add the file name and line number as
attributes to the record (you can use manipulators for that). Note
that in this case you won't be able to use these attributes in
filters, but you probably don't need that.
I finally found a simple solution based on add_value. Here is the full source code:
#include <ostream>
#include <fstream>
#include <boost/log/core.hpp>
#include <boost/log/trivial.hpp>
#include <boost/log/sinks/sync_frontend.hpp>
#include <boost/log/sinks/text_ostream_backend.hpp>
#include <boost/log/utility/setup/common_attributes.hpp>
#include <boost/log/utility/manipulators/add_value.hpp>
#include <boost/filesystem.hpp>
namespace logging = boost::log;
namespace src = boost::log::sources;
namespace expr = boost::log::expressions;
namespace sinks = boost::log::sinks;
void my_formatter(logging::record_view const& rec, logging::formatting_ostream& strm)
{
// Get the LineID attribute value and put it into the stream
strm << logging::extract< unsigned int >("LineID", rec) << ": ";
strm << logging::extract< int >("Line", rec) << ": ";
logging::value_ref< std::string > fullpath = logging::extract< std::string >("File", rec);
strm << boost::filesystem::path(fullpath.get()).filename().string() << ": ";
// The same for the severity level.
// The simplified syntax is possible if attribute keywords are used.
strm << "<" << rec[logging::trivial::severity] << "> ";
// Finally, put the record message to the stream
strm << rec[expr::smessage];
}
void init()
{
typedef sinks::synchronous_sink< sinks::text_ostream_backend > text_sink;
boost::shared_ptr< text_sink > sink = boost::make_shared< text_sink >();
sink->locked_backend()->add_stream(
boost::make_shared< std::ofstream >("sample.log"));
sink->set_formatter(&my_formatter);
logging::core::get()->add_sink(sink);
}
#define MY_GLOBAL_LOGGER(log_,sv) BOOST_LOG_SEV( log_, sv) \
<< boost::log::add_value("Line", __LINE__) \
<< boost::log::add_value("File", __FILE__) \
<< boost::log::add_value("Function", BOOST_CURRENT_FUNCTION)
int main(int, char*[])
{
init();
logging::add_common_attributes();
using namespace logging::trivial;
src::severity_logger< severity_level > lg;
MY_GLOBAL_LOGGER(lg,debug) << "Keep";
MY_GLOBAL_LOGGER(lg,info) << "It";
MY_GLOBAL_LOGGER(lg,warning) << "Simple";
MY_GLOBAL_LOGGER(lg,error) << "Stupid";
return 0;
}
On my Linux box, I compiled it using:
$ c++ -otutorial_fmt_custom -DBOOST_LOG_DYN_LINK tutorial_fmt_custom.cpp -lboost_log -lboost_log_setup -lboost_thread -lpthread -lboost_filesystem
If you run and check the log file generated here is what you get:
$ ./tutorial_fmt_custom && cat sample.log
1: 61: tutorial_fmt_custom.cpp: <debug> Keep
2: 62: tutorial_fmt_custom.cpp: <info> It
3: 63: tutorial_fmt_custom.cpp: <warning> Simple
4: 64: tutorial_fmt_custom.cpp: <error> Stupid
My solution is based on two inputs (thanks!):
http://www.boost.org/doc/libs/1_58_0/libs/log/example/doc/tutorial_fmt_custom.cpp
http://d.hatena.ne.jp/yamada28go/20140215/1392470561
In order to output filename and line you will need to register Scopes:
logging::core::get()->add_global_attribute("Scopes", attributes::named_scope());
and add custom formatter:
void my_formatter(logging::record_view const& rec, logging::formatting_ostream& strm)
{
const auto cont = logging::extract< attributes::named_scope::value_type >("Scopes", rec);
if(cont.empty())
return;
auto it = cont->begin();
boost::filesystem::path path(it->file_name.c_str());
strm << path.filename().string() << ":" << it->line << "\t" << rec[expr::smessage];
}
Note: In order for this to work the scopes container needs to be cleared before each logging:
if(!attributes::named_scope::get_scopes().empty()) attributes::named_scope::pop_scope();
I'm using Boost(1.55.0) Logging in my C++ application.
I have been able to generate log of this format
[2014-Jul-15 10:47:26.137959]: <debug> A regular message
I want to be able to add source file name and line number where
the log is generated.
[2014-Jul-15 10:47:26.137959]: <debug> [filename:line_no] A regular message
example:
[2014-Jul-15 10:47:26.137959]: <debug> [helloworld.cpp : 12] A regular message
Source Code:
#include <boost/log/core.hpp>
#include <boost/log/trivial.hpp>
#include <boost/log/expressions.hpp>
#include <boost/log/sinks/text_file_backend.hpp>
#include <boost/log/utility/setup/file.hpp>
#include <boost/log/utility/setup/common_attributes.hpp>
#include <boost/log/sources/severity_logger.hpp>
#include <boost/log/sources/record_ostream.hpp>
#include <boost/log/expressions.hpp>
#include <boost/log/support/date_time.hpp>
#include <boost/log/attributes/attribute.hpp>
#include <boost/log/attributes/attribute_cast.hpp>
#include <boost/log/attributes/attribute_value.hpp>
#include <boost/make_shared.hpp>
#include <boost/property_tree/ptree.hpp>
namespace logging = boost::log;
namespace src = boost::log::sources;
namespace sinks = boost::log::sinks;
namespace keywords = boost::log::keywords;
void init()
{
logging::add_file_log
(
keywords::file_name = "sample_%N.log", /*< file name pattern >*/
keywords::rotation_size = 10*1024*1204, /*< rotate files every 10 MiB... >*/
keywords::time_based_rotation = sinks::file::rotation_at_time_point(0, 0, 0), /*< ...or at midnight >*/
keywords::format =
(
boost::log::expressions::stream
<< boost::log::expressions::format_date_time< boost::posix_time::ptime >("TimeStamp", "%Y-%m-%d_%H:%M:%S.%f")
<< ": <" << boost::log::trivial::severity << "> "
<< boost::log::expressions::smessage
)
);
}
int main(int, char*[])
{
init();
logging::add_common_attributes();
using namespace logging::trivial;
src::severity_logger< severity_level > lg;
BOOST_LOG_SEV(lg, debug) << "A regular message";
return 0;
}
As Horus pointed out, you can use attributes to log file and line numbers. However it is best to avoid using multi-statements macros to avoid problems with expressions like this:
if (something)
LOG_FILE_LINE(debug) << "It's true"; // Only the first statement is conditional!
You can do better creating a macro that leverages the underlying behavior of the Boost Log library. For example, BOOST_LOG_SEV is:
#define BOOST_LOG_SEV(logger, lvl) BOOST_LOG_STREAM_SEV(logger, lvl)
#define BOOST_LOG_STREAM_SEV(logger, lvl)\
BOOST_LOG_STREAM_WITH_PARAMS((logger), (::boost::log::keywords::severity = (lvl)))
Using BOOST_LOG_STREAM_WITH_PARAMS you can set and get more attributes, like this:
// New macro that includes severity, filename and line number
#define CUSTOM_LOG(logger, sev) \
BOOST_LOG_STREAM_WITH_PARAMS( \
(logger), \
(set_get_attrib("File", path_to_filename(__FILE__))) \
(set_get_attrib("Line", __LINE__)) \
(::boost::log::keywords::severity = (boost::log::trivial::sev)) \
)
// Set attribute and return the new value
template<typename ValueType>
ValueType set_get_attrib(const char* name, ValueType value) {
auto attr = logging::attribute_cast<attrs::mutable_constant<ValueType>>(logging::core::get()->get_thread_attributes()[name]);
attr.set(value);
return attr.get();
}
// Convert file path to only the filename
std::string path_to_filename(std::string path) {
return path.substr(path.find_last_of("/\\")+1);
}
The complete source code is:
#include <boost/log/trivial.hpp>
#include <boost/log/sources/severity_logger.hpp>
#include <boost/log/utility/setup/file.hpp>
#include <boost/log/utility/setup/console.hpp>
#include <boost/log/expressions.hpp>
#include <boost/log/utility/setup/common_attributes.hpp>
#include <boost/log/attributes/mutable_constant.hpp>
#include <boost/date_time/posix_time/posix_time_types.hpp>
#include <boost/log/support/date_time.hpp>
#include <boost/log/attributes/mutable_constant.hpp>
namespace logging = boost::log;
namespace attrs = boost::log::attributes;
namespace expr = boost::log::expressions;
namespace src = boost::log::sources;
namespace keywords = boost::log::keywords;
// New macro that includes severity, filename and line number
#define CUSTOM_LOG(logger, sev) \
BOOST_LOG_STREAM_WITH_PARAMS( \
(logger), \
(set_get_attrib("File", path_to_filename(__FILE__))) \
(set_get_attrib("Line", __LINE__)) \
(::boost::log::keywords::severity = (boost::log::trivial::sev)) \
)
// Set attribute and return the new value
template<typename ValueType>
ValueType set_get_attrib(const char* name, ValueType value) {
auto attr = logging::attribute_cast<attrs::mutable_constant<ValueType>>(logging::core::get()->get_thread_attributes()[name]);
attr.set(value);
return attr.get();
}
// Convert file path to only the filename
std::string path_to_filename(std::string path) {
return path.substr(path.find_last_of("/\\")+1);
}
void init() {
// New attributes that hold filename and line number
logging::core::get()->add_thread_attribute("File", attrs::mutable_constant<std::string>(""));
logging::core::get()->add_thread_attribute("Line", attrs::mutable_constant<int>(0));
logging::add_file_log (
keywords::file_name = "sample.log",
keywords::format = (
expr::stream
<< expr::format_date_time<boost::posix_time::ptime>("TimeStamp", "%Y-%m-%d_%H:%M:%S.%f")
<< ": <" << boost::log::trivial::severity << "> "
<< '[' << expr::attr<std::string>("File")
<< ':' << expr::attr<int>("Line") << "] "
<< expr::smessage
)
);
logging::add_common_attributes();
}
int main(int argc, char* argv[]) {
init();
src::severity_logger<logging::trivial::severity_level> lg;
CUSTOM_LOG(lg, debug) << "A regular message";
return 0;
}
This generate a log like this:
2015-10-15_15:25:12.743153: <debug> [main.cpp:61] A regular message
As user2943014 pointed out, using scopes prints the line number where you opened that scope, not the line number where you emitted a log message using BOOST_LOG_SEV.
You can use attributes to log the line numbers etc at the place you actually logged.
Register global attributes in your logging initialization function:
using namespace boost::log;
core::get()->add_global_attribute("Line", attributes::mutable_constant<int>(5));
core::get()->add_global_attribute("File", attributes::mutable_constant<std::string>(""));
core::get()->add_global_attribute("Function", attributes::mutable_constant<std::string>(""));
Setting these attributes in your logging macro:
#define logInfo(methodname, message) \
LOG_LOCATION; \
BOOST_LOG_SEV(_log, boost::log::trivial::severity_level::trace) << message
#define LOG_LOCATION \
boost::log::attribute_cast<boost::log::attributes::mutable_constant<int>>(boost::log::core::get()->get_global_attributes()["Line"]).set(__LINE__); \
boost::log::attribute_cast<boost::log::attributes::mutable_constant<std::string>>(boost::log::core::get()->get_global_attributes()["File"]).set(__FILE__); \
boost::log::attribute_cast<boost::log::attributes::mutable_constant<std::string>>(boost::log::core::get()->get_global_attributes()["Function"]).set(__func__);
Not exactly beautiful, but it works and it was a long way for me. It's a pity boost doesn't offer this feature out of the box.
I would suggest to use the boost::log::add_value() function.
Define:
#define LOG_LOCATION(LEVEL, MSG) \
BOOST_LOG_SEV(logger::get(), LEVEL) \
<< boost::log::add_value("Line", __LINE__) \
<< boost::log::add_value("File", __FILE__) \
<< boost::log::add_value("Function", __FUNCTION__) << MSG
And then you can format it as follows:
boost::log::add_common_attributes();
boost::log::register_simple_filter_factory<boost::log::trivial::severity_level, char>("Severity");
boost::log::register_simple_formatter_factory<boost::log::trivial::severity_level, char>("Severity");
auto syslog_format(
boost::log::expressions::stream <<
"[" << boost::log::expressions::format_date_time< boost::posix_time::ptime >("TimeStamp", "%Y-%m-%d %H:%M:%S") <<
"] [" << boost::log::expressions::attr<boost::log::attributes::current_thread_id::value_type>("ThreadID") <<
"] [" << std::left << std::setw(7) << std::setfill(' ') << boost::log::trivial::severity <<
"] " << boost::log::expressions::smessage <<
" (" << boost::log::expressions::attr<std::string>("Filename") <<
":" << boost::log::expressions::attr<int>("Line") <<
":" << boost::log::expressions::attr<std::string>("Function") <<
")"
);
boost::log::add_file_log(
boost::log::keywords::file_name = "sys_%d_%m_%Y.%N.log",
boost::log::keywords::format = syslog_format
);
No need to add global attributes, and you can format it easily as seen above. I find this is a good compromise between other's solutions and the raw __FILE__ __LINE__ approach.
Full example here.
Define
namespace attrs = boost::logging::attributes;
namespace expr = boost::logging::expressions;
Add
<< expr::format_named_scope("Scope", keywords::format = "[%f:%l]")
to your keywords::format = (...) in init.
Then add
logging::core::get()->add_global_attribute("Scope", attrs::named_scope());
after add_common_attributes() in main.
Then just before the BOOST_LOG_SEV line add BOOST_LOG_NAMED_SCOPE("whatever").
BOOST_LOG_NAMED_SCOPE("whatever") creates a "scope" named "whatever". The scope is implemented by a unused variable that contains the scope name and the file and the line where the scope was defined.
The format_named_scope line specifies how a scope should be formatted in the log line. %f is the file, %l is the line and %n is the scope name.
Note that the file line that appears in the log record is the line where the macro BOOST_LOG_NAMED_SCOPE appears and not the line of the BOOST_LOG_SEV macro.
I am not aware of simple method to record the file and line without using the BOOST_LOG_NAMED_SCOPE.