I want to degrade my protobuf version from 3.17.0 to 3.12.0, But I get a problem. For example:
syntax = "proto3";
message Foo {
optional int32 a = 1;
}
message Test {
oneof value {
string str = 1;
Foo foo = 2;
}
}
For this protofile, I compile it with protoc(3.17.0), I get a utility method in generated Test class to check if str is set in the oneof field:
// from generated test.pb.h
// string str = 1;
bool has_str() const;
private:
bool _internal_has_str() const;
public:
void clear_str();
// ...
But when I use protoc(3.12.0) to compile it and set the optional flag:
./protoc test.proto --cpp_out=./ --experimental_allow_proto3_optional
I got the generated code:
// string str = 1;
private:
bool _internal_has_str() const;
public:
void clear_str();
So ,has_str doesn't exist anymore. Does anybody know why? Thanks!
PS:
has_foo will always exist.
The has accessors in C++ for oneof fields were added in 3.15:
Now Proto3 Oneof fields have "has" methods for checking their presence in
C++.
This is coincidentally the same version that includes non-experimental proto3 optional support:
Optional fields for proto3 are enabled by default, and no longer require
the --experimental_allow_proto3_optional flag.
Prior to that version, one can use the generated _case() function to detect which oneof field is set.
Related
I come to you today with another question that my brain can't process by itself:
I got a cpp file that includes optional as a header file. Unfortunately, this works only on c++17 forwards, and I'm trying to compile it in c++14. This cpp file uses optional like this
std::optional<std::string> GetStringPropertyValueFromJson(const std::string& Property, const web::json::value& Json)
{
if (Json.has_field(utility::conversions::to_string_t(Property)))
{
auto& propertyValue = Json.at(utility::conversions::to_string_t(Property));
if (propertyValue.is_string())
{
return std::optional<std::string>{utility::conversions::to_utf8string(propertyValue.as_string())};
}
}
return std::nullopt;
}
and then the function is used to assign values like this:
std::string tokenType = GetStringPropertyValueFromJson("token_type", responseContent).value_or("");
std::string accessToken = GetStringPropertyValueFromJson("access_token", responseContent).value_or("");
Please help me with a proper substitution for OPTIONAL. Thanks and much love
PS: From what i've read, you can replace optional with pair somehow in order to get a similar result, but I don't really know how exactly.
PPS: I am new here so any tips on how to better write my questions or anything else are greatly appreciated :)
I guess in C++14 the optional header could be included by #include <experimental/optional>.
Change your method signature to
std::string GetStringPropertyValueFromJson(const std::string& Property, const web::json::value& Json)
and in the end just return the empty string
return "";
Then later in your code use it without std::optional::value_or:
std::string tokenType = GetStringPropertyValueFromJson("token_type", responseContent);
The logic is exactly the same and you don't use std::optional.
I see now your other question about possibility to use std::pair. Yes, you could also change your method to:
std::pair<std::string, bool> GetStringPropertyValueFromJson(const std::string& Property, const web::json::value& Json)
and return std::make_pair(valueFromJson, true) in case your json property has been found, or std::make_pair("", false) in case it was not. This also solves the problem with empty (but existing) json property.
A poor mans optional string that should be sufficient for your code is this:
struct my_nullopt {};
struct my_optional {
private:
std::string value;
bool has_value = false;
public:
my_optional(my_nullopt) {}
my_optional(const std::string& v) : value(v),has_value(true) {}
T value_or(const std::string& v) {
return has_value ? value : v;
}
};
Its a rather limited interface, for example it is not possible to set the value after construction. But it appears that you do not need that.
Alternatively you can use boost/optional.
Note that the tip you got about using a pair is just what I did above: The value and a bool. Just that std::pair is for cases where you cannot give better names than first and second (eg in generic code), but it is simple to provide a better interface than std::pair does here. With a pair the value_or would be something along the line of x.first ? x.second : "".
PS: Only in the end I realized that the code you present does not actually make use of what std::optional has to offer. As you are calling value_or(""), you cannot distinguish between a field with value "" or "" because the optional had no value. Because of that, the most simple solution is to use a plain std::string and return "" instead of std::nullopt.
I want to use spdlog for my code's logging. In my code, there is a important variable for the step in simulation, and I want it to be always displayed in my logs.
Here is the format I wants.
[log_level][the_special_variable][logger_name] messages
So how could format the logger? Or there isn't any way to do that?
Edited:
Sorry I am not good at asking a question in English.
I've read the Readme.md in spdlog's github, and i saw this
// Log patterns can contain custom flags.
// the following example will add new flag '%*' - which will be bound to a <my_formatter_flag> instance.
#include "spdlog/pattern_formatter.h"
class my_formatter_flag : public spdlog::custom_flag_formatter
{
public:
void format(const spdlog::details::log_msg &, const std::tm &, spdlog::memory_buf_t &dest) override
{
std::string some_txt = "custom-flag";
dest.append(some_txt.data(), some_txt.data() + some_txt.size());
}
std::unique_ptr<custom_flag_formatter> clone() const override
{
return spdlog::details::make_unique<my_formatter_flag>();
}
};
void custom_flags_example()
{
auto formatter = std::make_unique<spdlog::pattern_formatter>();
formatter->add_flag<my_formatter_flag>('*').set_pattern("[%n] [%*] [%^%l%$] %v");
spdlog::set_formatter(std::move(formatter));
}
but I can't understand the usage of it. It seems like it can only add a string for the custom flag. I would like to kwon if it is OK to display a int variable.
Yes, it is okay to add an int to the log message, you just have to stringify it. For example, in the format method:
auto str = std::to_string(my_special_int_variable);
dest.append(...);
The only question is how you make your int var available in the formatter. The example above assumes it's a global variable.
After a few hours, I still can not set a oneof field in a clear (just created) protobuf message using reflection in c++.
I can obtain the needed OneOfDescriptor through the Descriptor of the message. But when I try to 'set' the oneof using Reflection, I found the real problem. There is only three function members related to OneOfDescriptor:
HasOneOf to check if there is a previous defined oneof in the message
GetOneofFieldDescriptor to get a FieldDescriptor from a previous defined oneof in the message
ClearOneof (without documentation) to clear oneof.
So there is not a SetOneofFieldDescriptorand if the oneof in the message is not previously defined, using a mutable_XXXX function member in the message, the GetOneofFieldDescriptor returns nullptr.
Therefore I am really stuck and any idea will be welcome.
Thanks in advance.
You set it the same way you would set the field if it weren't part of a oneof. Get a FieldDescriptor from the message's Descriptor and pass it to the appropriate SetXXX method of the message's Reflection.
Given a message like the following:
message Foo
{
oneof bar
{
int32 a = 1;
string b = 2;
}
}
You can set the a member as follows:
#include "foo.pb.h"
int main()
{
Foo f;
const google::protobuf::Descriptor* d = f.GetDescriptor();
const google::protobuf::FieldDescriptor* a = d->FindFieldByName("a");
const google::protobuf::Reflection* r = f.GetReflection();
r->SetInt32(&f, a, 42);
}
Protobuf will take care of making sure any previously set members of the oneof get unset as needed.
im working into Googles Protocol Buffers right now and have a question. If i have multiple .proto files and thus multiple classes, is it somehow possible when the data is sent over a socket to determine which type it is?
E.g. i have two classes, lets call them person.proto and adress.proto. Now I send one of those over the wire. How can the receiver determine wheather it is a person or an adress?
I am doing this in C++.
My attempt would be adding a frame around the message, containing length and type. But i want to know if there is already some kind of implementation for the type stuff, so i dont reimplement existing stuff.
Yes it is possible. Protobuf supports reflection using so called message descriptors.
But (as stated in the other answer) you'll need a reliable well known root message type. Instead of introducing your own message discrimination mechanism, IMHO it's better to use protobufs extension mechanism
Here's a sample of what we have in production
package Common.ConfigurationCommands;
message UcpConfiguration
{
optional uint32 componentIndex = 1;
optional ConfigCmdStatus configCmdResponseStatus = 2;
optional string configErrorDescription = 3;
extensions 100 to max;
}
The extension looks like
import "Common/ConfigurationCommands.proto";
message AmplifierConfiguration
{
extend Common.ConfigurationCommands.UcpConfiguration
{
optional AmplifierConfiguration amplifierConfiguration = 108;
}
optional uint32 preemphasis = 1;
}
import "Common/ConfigurationCommands.proto";
message FrontendConfiguration
{
extend Common.ConfigurationCommands.UcpConfiguration
{
optional FrontendConfiguration frontendConfiguration = 100;
}
optional bool frontendActive = 1;
optional uint32 refInputComponentIndex = 2;
extensions 100 to max;
}
You can check this part of the documentation to see how to deal with extensions in your C++ code.
It is impossible to detect which object was serialized, Protobuf don't do it. But you can handle that using protobuf very easy:
1) Method: just send message that has type and string body. To body you will serialize your objects, and in type you will show which object is serialized:
Something like that:
package MyGreatPackage;
message Pack
{
required bytes packcode = 1;
//code for data/query
required bytes mess = 2;
}
message Data
{
//anything you need to
}
message Query
{
//anything you need to
}
So, you will always send message Pack, where will be defined which object exactly is in "mess" field.
2) Method: protobuf allows this technique to achieve same thing without pack wrapper, look here: https://developers.google.com/protocol-buffers/docs/techniques?hl=ru#union
message OneMessage {
enum Type { FOO = 1; BAR = 2; BAZ = 3; }
// Identifies which field is filled in.
required Type type = 1;
// One of the following will be filled in.
optional Foo foo = 2;
optional Bar bar = 3;
optional Baz baz = 4;
}
So, you can set all classes you may send as optional and determine their types by required parameter.
Still, for me first varians seems better, choose what you like.
I want to replace external libraries (like boost) as much as possible with their equivalents in standard C++ if they exist and it is possible, to minimize dependencies, therefore I wonder if there exists a safe way to convert boost::system::error_code to std::error_code. Pseudo code example:
void func(const std::error_code & err)
{
if(err) {
//error
} else {
//success
}
}
boost::system::error_code boost_err = foo(); //foo() returns a boost::system::error_code
std::error_code std_err = magic_code_here; //convert boost_err to std::error_code here
func(std_err);
The most important it is not the exactly the same error, just so close to as possible and at last if is an error or not. Are there any smart solutions?
Thanks in advance!
I had this exact same question since I wanted to use std::error_code but was also using other boost libraries that use boost::system::error_code (e.g. boost ASIO). The accepted answer works for the error codes handled by std::generic_category(), as they're a simple cast from boost's generic error codes, but it doesn't work for the general case where you want to handle custom error categories as well.
So I created the following code as a general purpose boost::system::error_code-to-std::error_code converter. It works by dynamically creating an std::error_category shim for each boost::system::error_category, forwarding the calls to the underlying Boost error category. Since error categories need to be singletons (or at least singleton-like as in this case) I don't expect there to be much of a memory explosion.
I also just convert boost::system::generic_category() object to use std::generic_category() since they should behave the same. I had wanted to do the same for system_category(), however in testing on VC++10 it printed out the wrong messages (I assume it should print out what you get from FormatMessage, but it appears to use strerror, Boost uses FormatMessage as expected).
To use it just call BoostToErrorCode(), defined below.
Just a warning, I just wrote this today so it's only had basic testing. You may use it any way you like, but you do so at your own risk.
//==================================================================================================
// These classes implement a shim for converting a boost::system::error_code to a std::error_code.
// Unfortunately this isn't straightforward since it the error_code classes use a number of
// incompatible singletons.
//
// To accomplish this we dynamically create a shim for every boost error category that passes
// the std::error_category calls on to the appropriate boost::system::error_category calls.
//==================================================================================================
#include <boost/system/error_code.hpp>
#include <boost/thread/mutex.hpp>
#include <boost/thread/once.hpp>
#include <boost/thread/locks.hpp>
#include <system_error>
namespace
{
// This class passes the std::error_category functions through to the
// boost::system::error_category object.
class BoostErrorCategoryShim : public std::error_category
{
public:
BoostErrorCategoryShim( const boost::system::error_category& in_boostErrorCategory )
:m_boostErrorCategory(in_boostErrorCategory), m_name(std::string("boost.") + in_boostErrorCategory.name()) {}
virtual const char *name() const;
virtual std::string message(value_type in_errorValue) const;
virtual std::error_condition default_error_condition(value_type in_errorValue) const;
private:
// The target boost error category.
const boost::system::error_category& m_boostErrorCategory;
// The modified name of the error category.
const std::string m_name;
};
// A converter class that maintains a mapping between a boost::system::error_category and a
// std::error_category.
class BoostErrorCodeConverter
{
public:
const std::error_category& GetErrorCategory( const boost::system::error_category& in_boostErrorCategory )
{
boost::lock_guard<boost::mutex> lock(m_mutex);
// Check if we already have an entry for this error category, if so we return it directly.
ConversionMapType::iterator stdErrorCategoryIt = m_conversionMap.find(&in_boostErrorCategory);
if( stdErrorCategoryIt != m_conversionMap.end() )
return *stdErrorCategoryIt->second;
// We don't have an entry for this error category, create one and add it to the map.
const std::pair<ConversionMapType::iterator, bool> insertResult = m_conversionMap.insert(
ConversionMapType::value_type(
&in_boostErrorCategory,
std::unique_ptr<const BoostErrorCategoryShim>(new BoostErrorCategoryShim(in_boostErrorCategory))) );
// Return the newly created category.
return *insertResult.first->second;
}
private:
// We keep a mapping of boost::system::error_category to our error category shims. The
// error categories are implemented as singletons so there should be relatively few of
// these.
typedef std::unordered_map<const boost::system::error_category*, std::unique_ptr<const BoostErrorCategoryShim>> ConversionMapType;
ConversionMapType m_conversionMap;
// This is accessed globally so we must manage access.
boost::mutex m_mutex;
};
namespace Private
{
// The init flag.
boost::once_flag g_onceFlag = BOOST_ONCE_INIT;
// The pointer to the converter, set in CreateOnce.
BoostErrorCodeConverter* g_converter = nullptr;
// Create the log target manager.
void CreateBoostErrorCodeConverterOnce()
{
static BoostErrorCodeConverter converter;
g_converter = &converter;
}
}
// Get the log target manager.
BoostErrorCodeConverter& GetBoostErrorCodeConverter()
{
boost::call_once( Private::g_onceFlag, &Private::CreateBoostErrorCodeConverterOnce );
return *Private::g_converter;
}
const std::error_category& GetConvertedErrorCategory( const boost::system::error_category& in_errorCategory )
{
// If we're accessing boost::system::generic_category() or boost::system::system_category()
// then just convert to the std::error_code versions.
if( in_errorCategory == boost::system::generic_category() )
return std::generic_category();
// I thought this should work, but at least in VC++10 std::error_category interprets the
// errors as generic instead of system errors. This means an error returned by
// GetLastError() like 5 (access denied) gets interpreted incorrectly as IO error.
//if( in_errorCategory == boost::system::system_category() )
// return std::system_category();
// The error_category was not one of the standard boost error categories, use a converter.
return GetBoostErrorCodeConverter().GetErrorCategory(in_errorCategory);
}
// BoostErrorCategoryShim implementation.
const char* BoostErrorCategoryShim::name() const
{
return m_name.c_str();
}
std::string BoostErrorCategoryShim::message(value_type in_errorValue) const
{
return m_boostErrorCategory.message(in_errorValue);
}
std::error_condition BoostErrorCategoryShim::default_error_condition(value_type in_errorValue) const
{
const boost::system::error_condition boostErrorCondition = m_boostErrorCategory.default_error_condition(in_errorValue);
// We have to convert the error category here since it may not have the same category as
// in_errorValue.
return std::error_condition( boostErrorCondition.value(), GetConvertedErrorCategory(boostErrorCondition.category()) );
}
}
std::error_code BoostToErrorCode( boost::system::error_code in_errorCode )
{
return std::error_code( in_errorCode.value(), GetConvertedErrorCategory(in_errorCode.category()) );
}
Since C++-11 (std::errc), boost/system/error_code.hpp maps the same error codes to std::errc, which is defined in the system header system_error.
You can compare both enums and they should be functionally equivalent because they both appear to be based on the POSIX standard. May require a cast.
For example,
namespace posix_error
{
enum posix_errno
{
success = 0,
address_family_not_supported = EAFNOSUPPORT,
address_in_use = EADDRINUSE,
address_not_available = EADDRNOTAVAIL,
already_connected = EISCONN,
argument_list_too_long = E2BIG,
argument_out_of_domain = EDOM,
bad_address = EFAULT,
bad_file_descriptor = EBADF,
bad_message = EBADMSG,
....
}
}
and std::errc
address_family_not_supported error condition corresponding to POSIX code EAFNOSUPPORT
address_in_use error condition corresponding to POSIX code EADDRINUSE
address_not_available error condition corresponding to POSIX code EADDRNOTAVAIL
already_connected error condition corresponding to POSIX code EISCONN
argument_list_too_long error condition corresponding to POSIX code E2BIG
argument_out_of_domain error condition corresponding to POSIX code EDOM
bad_address error condition corresponding to POSIX code EFAULT
I adapted the above solution into a shorter solution.
Using a thread_local std::map to map between the category name to its instance so no locking is needed.
The limitation is that you can't pass error codes between threads since the category pointer will be different.(Converting it to a locking function is simple enough if you don't want to use the thread_local storage)
Also I feed it is more compact.
#include <iostream>
#include <map>
#include <boost/system/system_error.hpp>
namespace std
{
error_code make_error_code(boost::system::error_code error)
{
struct CategoryAdapter : public error_category
{
CategoryAdapter(const boost::system::error_category& category)
: m_category(category)
{
}
const char* name() const noexcept
{
return m_category.name();
}
std::string message(int ev) const
{
return m_category.message(ev);
}
private:
const boost::system::error_category& m_category;
};
static thread_local map<std::string, CategoryAdapter> nameToCategory;
auto result = nameToCategory.emplace(error.category().name(), error.category());
auto& category = result.first->second;
return error_code(error.value(), category);
}
};
int main() {
auto a = boost::system::errc::make_error_code(boost::system::errc::address_family_not_supported);
auto b = std::make_error_code(a);
std::cout << b.message() << std::endl;
}
As of Boost version 1.65, you can convert boost::error_code to its C++11 counterpart:
On a C++11 compiler, Boost.System now provides implicit conversions from boost::system::error_category, error_code, and error_condition to their standard equivalents from <system_error>.
This allows libraries to expose a C++11 interface and report errors via std::error_code even when using Boost.System, directly or through a dependency such as Boost.ASIO.
So it should now be as simple as:
auto ec = static_cast<std::error_code>(boost_error_code);