I have a nice format and a console log:
auto fmtStream = expressions::stream
<< "LineID: " << expressions::attr<unsigned int>("LineID") << " "
<< "Message: " << expressions::message:
boost::log::add_console_log(std::cout, keywords::format = fmtStream);
The format stream is of course somewhat longer..
Now I want to give a user to configure the logging:
std::ifstream file("log.conf");
init_from_stream(file);
A lot of formatting used in fmtStream is not possible using the format string in the config file.
How can I give the user the possibility to modify the console sink, e.g. add a filter? But I want to keep the format string as a default.
Possibilities I see for that:
1) Give the console log that I define in my code a name. The user could now modify it with a sink of the same name.
2) Set a default format that is taken for all sinks. But according to Boost Log changing the default logging::core formatter? this is not possible.
3) Any other ideas?
Thanks!
init_from_stream and init_from_settings functions will initialize the library as specified in the settings. These functions are designed to configure the library from scratch, so they will add new sinks with the specified settings, including filters and formatters. If all you need is to customize formatter for your existing sink and not allow full logging configuration then you should interpret the settings file yourself.
You can parse the settings file with the parse_settings function. From it you will receive a settings (or wsettings) object which you can analyze and modify as described here and here (sorry for the bad formatting in the reference docs). Since you're probably not planning to support all sinks and parameters supported by Boost.Log, you are not bound to the semantics Boost.Log puts in the parameters and can interpret the settings in any way you want. For instance, you may choose to only read the sink formatter:
boost::log::settings setts = boost::log::parse_settings(file);
if (boost::optional<std::string> fmt = setts["MySink"]["Format"])
{
// Sink format is specified in the setting file
}
Now, to turn this format string into a formatter, you will need the parse_formatter function (the string format is described here). This function returns a formatter object that you can install into your sink, provided that you saved a pointer to it.
auto sink = boost::log::add_console_log(std::cout, keywords::format = fmtStream);
boost::log::settings setts = boost::log::parse_settings(file);
if (boost::optional<std::string> fmt = setts["MySink"]["Format"])
{
sink->set_formatter(boost::log::parse_formatter(fmt.get()));
}
There's one more thing to remember though. If you're using attribute values of custom types in your formatter, like enums for severity, for example, you will have to register those types in the library before parsing the formatter. This way the parser will be able to create a formatter that knows your types and uses appropriate formatting operators. There's a tutorial describing how to do that.
Related
I can easily create a config file to send messages to console, to logfile, to DebugView and to Eventlog but I didn't found out how to configure it that - as shared config file - Application "A" has 4 different sink sections than Application "B".
Is that possible?
How to implement that? (any links?)
My basic setup in the main function is following:
std::ifstream logConfiguration(getCwd() + "\\test-logging.ini");
try {
boost::log::init_from_stream(logConfiguration);
boost::log::add_common_attributes();
boost::log::core::get()->add_global_attribute("Scope", boost::log::attributes::named_scope());
}
catch (std::exception e) {
MYTRACE(error) << e.what() << std::endl;
return 1;
}
Kind Regards,
Thomas
Is that possible?
Yes, though not supported by Boost.Log out of the box.
How to implement that? (any links?)
You can implement reading the config file yourself or reuse the config file parser in Boost.Log. For example:
std::ifstream logConfiguration(getCwd() + "\\test-logging.ini");
boost::log::settings unifiedConfig = boost::log::parse_settings(logConfiguration);
Here, settings is a container with all the settings parsed from the file (see here and here for the documentation).
After parsing the file, you will have to construct a filtered settings container that corresponds to each particular application. How you do this depends on the unified settings file format. For example, if each of your application settings is stored in a separate subsection of the unified config, it could be as simple as this:
boost::property_tree::ptree const& unifiedPtree = unifiedConfig.property_tree();
boost::optional<boost::property_tree::ptree const&> ptreeA =
unifiedPtree.get_child_optional("A");
if (ptreeA) {
// We have some settings for application A
boost::log::settings configA(*ptreeA);
// ...
}
If you have a more involved format of the unified config, you may need to iterate over the ptree nodes to filter out the settings that pertain to different applications. See Boost.PropertyTree documentation for more details.
Lastly, when you have constructed the application-specific settings container, you can call init_from_settings in Boost.Log:
boost::log::init_from_settings(configA);
I am attempting to use the C++ StreamWriter class provided by Apache Arrow.
The only example of using StreamWriter uses the low-level Parquet API i.e.
parquet::schema::NodeVector fields;
fields.push_back(parquet::schema::PrimitiveNode::Make(
"string_field", parquet::Repetition::OPTIONAL, parquet::Type::BYTE_ARRAY,
parquet::ConvertedType::UTF8));
fields.push_back(parquet::schema::PrimitiveNode::Make(
"char_field", parquet::Repetition::REQUIRED, parquet::Type::FIXED_LEN_BYTE_ARRAY,
parquet::ConvertedType::NONE, 1));
auto node = std::static_pointer_cast<parquet::schema::GroupNode>(
parquet::schema::GroupNode::Make("schema", parquet::Repetition::REQUIRED, fields));
The end result is a std::shared_ptr<parquet::schema::GroupNode> that can then be passed into StreamWriter.
Is it possible to construct and use "high-level" Arrow schemas with StreamWriter? They are supported when using the WriteTable function (non-streaming) but I've found no examples of its use with the streaming API.
I, of course, can resort to using the low-level API but it is extremely verbose when creating large and complicated schema and I would prefer (but not need) to use the high-level Arrow schema mechanisms.
For example,
std::shared_ptr<arrow::io::FileOutputStream> outfile_;
PARQUET_ASSIGN_OR_THROW(outfile_, arrow::io::FileOutputStream::Open("test.parquet"));
// construct an arrow schema
auto schema = arrow::schema({arrow::field("field1", arrow::int64()),
arrow::field("field2", arrow::float64()),
arrow::field("field3", arrow::float64())});
// build the writer properties
parquet::WriterProperties::Builder builder;
auto properties = builder.build()
// my current best attempt at converting the Arrow schema to a Parquet schema
std::shared_ptr<parquet::SchemaDescriptor> parquet_schema;
parquet::arrow::ToParquetSchema(schema.get(), *properties, &parquet_schema); // parquet_schema is now populated
// and now I try and build the writer - this fails
auto writer = parquet::ParquetFileWriter::Open(outfile_, parquet_schema->group_node(), properties);
The last line fails because parquet_schema->group_node() (which is the only way I'm aware of to get access to the GroupNode for the schema) returns a const GroupNode* whereas ParquetFileWriter::Open) needs a std::shared_ptr<GroupNode>.
I'm not sure casting-away the constness of the returned group node and forcing it into the ::Open() call is an officially supported (or correct) usage of StreamWriter.
Is what I want to do possible?
It seems that you need to use low-level api for StreamWriter.
a very tricky way:
auto writer = parquet::ParquetFileWriter::Open(outfile_, std::shared_ptr<parquet::schema::GroupNode>(const_cast<parquet::schema::GroupNode *>(parquet_schema->group_node())), properties);
you may need to converter schema manually.
source code cpp/src/parquet/arrow/schema.cc may help you.
PARQUET_EXPORT
::arrow::Status ToParquetSchema(const ::arrow::Schema* arrow_schema,
const WriterProperties& properties,
std::shared_ptr<SchemaDescriptor>* out);
This is a kind of follow-up of another question I asked (here) where I was made aware that using the same backend with multiple sinks is not a safe approach.
What I am trying to obtain is to "decouple" the severity levels inside a library/plugin from the applications using them, while being able to write the different logs to the same output (may it be stdout or, more likely, a file or a remote logger); this for the following reasons:
I wouldn't like to tie the severity levels inside my library/plugin to those of the applications using them because the change (for whatever reason) of the severity levels list in one of the applications using the library/plugin would cause the library to be "updated" with the new severities and, as a waterfall, of all other applications which use the library/plugin
I would like to be able to use library specific severity levels (which, for being correctly displayed in the log messaged should be supplied to the sink formatter - thus my need of using different sinks)
Which is the best way to obtain this?
Some afterthoughts: As per Andrey's reply to my previous question the "problem" is that the backend is not synchronized to receive data from multiple sources (sinks); thus the solution might seem to be to create a synchronized version of the backends (e.g. wrapping the writes to the backend in a boost::asio post)...
Is this the only solution?
Edit/Update
I update the question after Andrey's awesome reply, mainly for sake of completeness: the libraries/plugins are meant to be used with internally developed applications only, thus it is assumed that there will be a common API we can shape for defining the log structure and behaviour.
Plus, most applications are meant to run mainly "unmanned", i.e. with really minimal, if not null, user/runtime interaction, so the basic idea is to have the log level set in some plugin specific configuration file, read at startup (or set to be reloaded upon a specific application API command from the application).
First, I'd like to address this premise:
for being correctly displayed in the log messaged should be supplied to the sink formatter - thus my need of using different sinks
You don't need different sinks to be able to filter or format different types of severity levels. Your filters and formatters have to deal with that, not the sink itself. Only create multiple sinks if you need multiple log targets. So to answer your question, you should focus on the protocol of setting up filters and formatters rather than sinks.
The exact way to do that is difficult to suggest because you didn't specify the design of your application/plugin system. What I mean by that is that there must be some common API that must be shared by both the application and the libraries, and the way you set up logging will depend on where that API belongs. Severity levels, among other things, must be a part of that API. For example:
If you're writing plugins for a specific application (e.g. plugins for a media player) then the application is the one that defines the plugin API, including the severity levels and even possibly the attribute names the plugins must use. The application configures sinks, including filters and formatters, using the attributes mandated by the API, and plugins never do any configuration and only emit log records. Note that the API may include some attributes that allow to distinguish plugins from each other (e.g. a channel name), which would allow the application to process logs from different plugins differently (e.g. write to different files).
If you're writing both plugins and application(s) to adhere some common API, possibly defined by a third party, then logging protocol must still be defined by that API. If it's not, then you cannot assume that any other application or plugin not written by you supports logging of any kind, even that it uses Boost.Log at all. In this case every plugin and the application itself must deal with logging independently, which is the worst case scenario because the plugins and the application may affect each other in unpredictable ways. It is also difficult to manage the system like that because every component will have to be configured separately by the user.
If you're writing an application that must be compatible with multiple libraries, each having its own API, then it is the application who should be aware of logging convention taken in each an every library it uses, there's no way around it. This may include setting up callbacks in the libraries, intercepting file output and translating between library's log severity levels and the application severity levels. If the libraries use Boost.Log to emit log records then they should document the attributes they use, including the severity levels, so that the application is able to setup the logging properly.
So, in order to take one approach or the other, you should first decide how your application and plugins interface each other and what API they share and how that API defines logging. The best case scenario is when you define the API, so you can also set the logging conventions you want. In that case, although possible, it is not advisable or typical to have arbitrary severity levels allowed by the API because it significantly complicates implementation and configuration of the system.
However, just in case if for some reason you do need to support arbitrary severity levels and there's no way around that, you can define an API for the library to provide, which can help the application to set up filters and formatters. For example, each plugin can provide API like this:
// Returns the filter that the plugin wishes to use for its records
boost::log::filter get_filter();
// The function extracts log severity from the log record
// and converts it to a string
typedef std::function<
std::string(boost::log::record_view const&)
> severity_formatter;
// Returns the severity formatter, specific for the plugin
severity_formatter get_severity_formatter();
Then the application can use a special filter that will make use of this API.
struct plugin_filters
{
std::shared_mutex mutex;
// Plugin-specific filters
std::vector< boost::log::filter > filters;
};
// Custom filter
bool check_plugin_filters(
boost::log::attribute_value_set const& values,
std::shared_ptr< plugin_filters > const& p)
{
// Filters can be called in parallel, we need to synchronize
std::shared_lock< std::shared_mutex > lock(p->mutex);
for (auto const& f : p->filters)
{
// Call each of the plugin's filter and pass the record
// if any of the filters passes
if (f(values))
return true;
}
// Suppress the record by default
return false;
}
std::shared_ptr< plugin_filters > pf = std::make_shared< plugin_filters >();
// Set the filter
sink->set_filter(std::bind(&check_plugin_filters, std::placeholders::_1, pf));
// Add filters from plugins
std::unique_lock< std::shared_mutex > lock(pf->mutex);
pf->filters.push_back(plugin1->get_filter());
pf->filters.push_back(plugin2->get_filter());
...
And a similar formatter:
struct plugin_formatters
{
std::shared_mutex mutex;
// Plugin-specific severity formatters
std::vector< severity_formatter > severity_formatters;
};
// Custom severity formatter
std::string plugin_severity_formatter(
boost::log::record_view const& rec,
std::shared_ptr< plugin_formatters > const& p)
{
std::shared_lock< std::shared_mutex > lock(p->mutex);
for (auto const& f : p->severity_formatters)
{
// Call each of the plugin's formatter and return the result
// if any of the formatters is able to extract the severity
std::string str = f(rec);
if (!str.empty())
return str;
}
// By default return an empty string
return std::string();
}
std::shared_ptr< plugin_formatters > pf =
std::make_shared< plugin_formatters >();
// Set the formatter
sink->set_formatter(
boost::log::expressions::stream << "["
<< boost::phoenix::bind(&plugin_severity_formatter,
boost::log::expressions::record, pf)
<< "] " << boost::log::expressions::message);
// Add formatters from plugins
std::unique_lock< std::shared_mutex > lock(pf->mutex);
pf->severity_formatters.push_back(plugin1->get_severity_formatter());
pf->severity_formatters.push_back(plugin2->get_severity_formatter());
...
Note, however, that at least with regard to filters, this approach is flawed because you allow the plugins to define the filters. Normally, it should be the application who selects which records are being logged. And for that there must be a way to translate library-specific severity levels to some common, probably defined by the application levels.
How to call sink->imbue for text file sink when using init_from_settings?
I checked the source code and didn't find a way to re-access those sinks.
Seems that register_sink_factory is the extension, but the default factories are all in init_from_settings.cpp, so I'm not able to use an a decorator pattern to implement it easily.
I tried was set global locale, but it breaks RotationSize param (which doesn't accept int with decimal point)
Another way is:
auto previousLocale = std::locale::global(boost::locale::generator()("zh_CN.UTF-8"));
logging::init_from_settings(settings);
logging::add_common_attributes();
std::locale::global(previousLocale);
Any better ideas?
You can register a sink factory that will configure the sink the way you need. You can find an example here.
I would like to know the cleanest way of registering a file extension with my C++ application so that when a data file associated with my program is double clicked, the application is opened and the filename is passed as a parameter to the application.
Currently, I do this through my wix installer, but there are some instances where the application will not be installed on ths user's computer, so I also need the option of creating the registry key through the application.
Additionally, will this also mean that if the application is removed, unused entries in the registry will be left lying around?
Your basic overview of the process is found in this MSDN article. The key parts are at the bottom of the list:
Register the ProgID
A ProgID (essentially, the file type registry key) is what contains your important file type properties, such as icon, description, and context menu items including applications used when the file is double clicked. Many extensions may have the same file type. That mapping is done in the next step:
Register the file name extension for the file type
Here, you set a registry value for your extension, setting that extension's file type to the ProgID you created in the previous step.
The minimum amount of work required to get a file to open with your application is setting/creating two registry keys. In this example .reg file, I create a file type (blergcorp.blergapp.v1) and associate a file extension (.blerg) with it.
Windows Registry Editor Version 5.00
[HKEY_CURRENT_USER\Software\Classes\blergcorp.blergapp.v1\shell\open\command]
#="c:\path\to\app.exe \"%1\""
[HKEY_CURRENT_USER\Software\Classes\.blerg]
#="blergcorp.blergapp.v1"
Now, you probably want to accomplish this programmatically. To be absolutely kosher, you could check for the existence of these keys, and change your program behavior accordingly, especially if you're assuming control of some common file extension. However, the goal can be accomplished by setting those two keys using the SetValue function.
I'm not positive of the exact C++ syntax, but in C# the syntax looks something like this:
Registry.SetValue(#"HKEY_CURRENT_USER\Software\Classes\blergcorp.blergapp.v1\shell\open\command", null, #"c:\path\to\app.exe \"%1\"");
Registry.SetValue(#"HKEY_CURRENT_USER\Software\Classes\.blerg", null, "blergcorp.blergapp.v1");
Of course you could manually open each sub key, manually create the ProgID and extension subkey, and then set the key value, but a nice thing about the SetValue function is that if the keys or values don't exist, they will automatically be created. Very handy.
Now, a quick word about which hive to use. Many file association examples online, including ones on MSDN, show these keys being set in HKEY_CLASSES_ROOT. I don't recommend doing this. That hive is a merged, virtual view of HKEY_LOCAL_MACHINE\Software\Classes (the system defaults) and HKEY_CURRENT_USER\Software\Classes (the per-user settings), and writes to any subkey in the hive are redirected to the same key in HKEY_LOCAL_MACHINE\Software\Classes. Now, there's no direct problem doing this, but you may run into this issue: If you write to HKCR (redirected to HKLM), and the user has specified the same keys with different values in HKCU, the HKCU values will take precedence. Therefore, your writes will succeed but you won't see any change, because HKEY_CURRENT_USER settings take precedence over HKEY_LOCAL_MACHINE settings.
Therefore, you should take this into consideration when designing your application. Now, on the flip side, you can write to only HKEY_CURRENT_USER, as my examples here show. However, that file association setting will only be loaded for the current user, and if your application has been installed for all users, your application won't launch when that other user opens the file in Windows.
That should be a decent primer for what you want to do. For further reading I suggest
Best Practices for File Association
File Types and File Association, especially
How File Associations Work
And see also my similar answer to a similar question:
Associating file extensions with a program
This is a two step process:
1. Define a program that would take care of extension: (unless you want to use existing one)
1.1 create a key in "HKCU\\Software\\Classes\\" for example
"Software\\Classes\\YourProgramName.file.ext"
1.2 create subkey "Software\\Classes\\YourProgramName.file.ext\\DefaultIcon"
1.2.1 set default value ("") to your application full path to get
icon from resources
1.3 create a subkey "Software\\Classes\\YourProgramName.file.ext\\Shell\\OperationName\\Command"
OperationName = for example Open, Print or Other
1.3.1 set default value ("") to your application full path +optional runtime params (filename)
2.Associate file extension with program.
2.1 create a key HKCU\\Software\\Classes\\.ext - here goes your extension
2.2 set default value to the program definition key
("YourProgramName.file.ext")
Below is part of the program written in c# which associate file extension. It is not c++ but i think it is simple enought to explain itself and AFAIK it is verv simmilar if not identical to the code in c++
1.
RegistryKey keyPFCTExt0 = Registry.CurrentUser.OpenSubKey("Software\\Classes\\PFCT.file.enc", true);
if (keyPFCTExt0 == null)
{
keyPFCTExt0 = Registry.CurrentUser.CreateSubKey("Software\\Classes\\PFCT.file.enc");
keyPFCTExt0.CreateSubKey("DefaultIcon");
RegistryKey keyPFCTExt0ext = Registry.CurrentUser.OpenSubKey("Software\\Classes\\PFCT.file.enc\\DefaultIcon", true);
keyPFCTExt0ext.SetValue("", Application.ExecutablePath +",0");
keyPFCTExt0ext.Close();
keyPFCTExt0.CreateSubKey("Shell\\PFCT_Decrypt\\Command");
}
keyPFCTExt0.SetValue("", "PFCT.file.enc");
keyPFCTExt0.Close();
2.
RegistryKey keyPFCTExt1 = Registry.CurrentUser.OpenSubKey("Software\\Classes\\PFCT.file.enc\\Shell\\PFCT_Decrypt\\Command", true);
if (keyPFCTExt1 == null)
keyPFCTExt1 = Registry.CurrentUser.CreateSubKey("Software\\Classes\\PFCT.file.enc\\Shell\\PFCT_Decrypt\\Command");
keyPFCTExt1.SetValue("", Application.ExecutablePath + " !d %1"); //!d %1 are optional params, here !d string and full file path
keyPFCTExt1.Close();
I don't know why people keep saying that HKEY_CURRENT_USER\Software\Classes\<.ext>'s Default value (which will redirect you into another (software-created) class.
It does work, but it will be overridden by
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\FileExts\<.ext>\UserChoice
And I believe Microsoft recommends the second practice- because it's what the built-in "open with" is doing. The value of Progid" key is equal to default value of HKEY_CURRENT_USER\Software\Classes\<.ext> in this case.
I found the following while trying to manipulate associations using C#:
hkcu\software\microsoft\windows\currentVersion\explorer\fileexts.reg\userchoice -> for user specific settings. The values in the openWithProgIds
key point to the keys in the hkcr.
hkcr\xfile\shell\open\muiVerb value or hkcr\xfile\shell\open\command\default value -> affects open handler. This is the value that contains the path to a program.
hkcr\ .x -> affects context menu (new x) among other things related to the menus.
I don't know the C++ code, but given these info you must be able to manipulate the registry using the registry API.