Assuming I have this proto
message inner_body1{
... // some attr
}
message inner_body2{
... // some attr
}
message body {
oneof inner{
inner_body1 = 1;
inner_body2 = 2;
}
}
message head {
... // some attr
}
message pkg{
head h = 1;
body b = 2;
}
And I design a function like this
void SendPkg(proto::Message& data)
{
pkg p;
auto head = p.mutable_head();
head->fillsomething(); // not important
// My question is, if 'data' is definitely one of the message type defined in 'body'(e.g. 'inner_body1')
// How can I put 'data' into pkg's body field?
}
My question is, if 'data' is definitely one of the message type defined in 'body'(e.g. 'inner_body1')
How can I put 'data' into pkg's body field?
update:
I have tried this way
void SendPkg(proto::Message& data)
{
pkg p;
auto head = p.mutable_head();
head->fillsomething(); // not important
auto body = p.mutable_body();
const Descriptor* desc = data.GetDescriptor();
if (desc.name() == "inner_body1")
{
auto body1 = body->mutable_innerbody1();
body1.CopyFrom(data);
}
else
{
auto body2 = body->mutable_innerbody2();
body2.CopyFrom(data);
}
}
this may works. But the fallback is obviously. I have to maintain this ugly string mapping and it running effienciency is low.
Is there any way could achieve this more elegant?
Related
I have a .proto file which consists of following messages:
message A {
message B {
optional string Header = 1;
optional string Value = 2;
}
repeated B Inputs = 1;
}
message BuildConfig {
optional A Options = 1;
}
In my pb.h file there are following functions:
class BuildConfig:
inline const ::google::protobuf::RepeatedPtrField< ::NBuildModels::NProto::A >&
GetOptions() const { return options(); }
class A:
inline const ::google::protobuf::RepeatedPtrField< ::NBuildModels::NProto::A_B >&
GetInputs() const { return inputs(); }
I am trying to access Head and Value like this:
void foo(const NBuildModels::NProto::BuildConfig& config) {
auto a = config.GetOptions();
auto b = a.GetInputs();
}
However, it does not work with the following error : No member named 'GetInputs' in 'google::protobuf::RepeatedPtrFieldNBuildModels::NProto::A'
What protobuf syntax do you use? What protogen do you use? Neither of known me protogen generates the C++ methods GetOptions and GetInputs. This works for me after Google protogen with syntax = "proto3";:
auto& a = config.options();
auto& b = a.inputs();
I'm working with Protobuf3 in my C++14 project. There have been some functions, which returns the google::protobuf::Message*s as a rpc request, what I need to do is to set their fields. So I need to use the reflection of Protobuf3.
Here is a proto file:
syntax="proto3";
package srv.user;
option cc_generic_services = true;
message BatchGetUserInfosRequest {
uint64 my_uid = 1;
repeated uint64 peer_uids = 2;
map<string, string> infos = 3;
}
message BatchGetUserInfosResponse {
uint64 my_uid = 1;
string info = 2;
}
Service UserSrv {
rpc BatchGetUserInfos(BatchGetUserInfosRequest) returns (BatchGetUserInfosResponse);
};
Now I called a function, which returns a google::protobuf::Message*, pointing an object BatchGetUserInfosRequest and I try to set its fields.
// msg is a Message*, pointing to an object of BatchGetUserInfosRequest
auto descriptor = msg->GetDescriptor();
auto reflection = msg->GetReflection();
auto field = descriptor->FindFieldByName("my_uid");
reflection->SetUInt64(msg, field, 1234);
auto field2 = descriptor->FindFieldByName("peer_uids");
reflection->GetMutableRepeatedFieldRef<uint64_t>(msg, field2).CopyFrom(peerUids); // peerUids is a std::vector<uint64_t>
As you see, I can set my_uid and peer_uids as above, but for the field infos, which is a google::protobuf::Map, I don't know how to set it with the reflection mechanism.
If you dig deep into the source code, you would find out the map in proto3 is implemented on the RepeatedField:
// Whether the message is an automatically generated map entry type for the
// maps field.
//
// For maps fields:
// map<KeyType, ValueType> map_field = 1;
// The parsed descriptor looks like:
// message MapFieldEntry {
// option map_entry = true;
// optional KeyType key = 1;
// optional ValueType value = 2;
// }
// repeated MapFieldEntry map_field = 1;
//
// Implementations may choose not to generate the map_entry=true message, but
// use a native map in the target language to hold the keys and values.
// The reflection APIs in such implementations still need to work as
// if the field is a repeated message field.
//
// NOTE: Do not set the option in .proto files. Always use the maps syntax
// instead. The option should only be implicitly set by the proto compiler
// parser.
optional bool map_entry = 7;
Inspired by the test code from protobuf, this works for me:
BatchGetUserInfosRequest message;
auto *descriptor = message.GetDescriptor();
auto *reflection = message.GetReflection();
const google::protobuf::FieldDescriptor *fd_map_string_string =
descriptor->FindFieldByName("infos");
const google::protobuf::FieldDescriptor *fd_map_string_string_key =
fd_map_string_string->message_type()->map_key();
const google::protobuf::FieldDescriptor *fd_map_string_string_value =
fd_map_string_string->message_type()->map_value();
const google::protobuf::MutableRepeatedFieldRef<google::protobuf::Message>
mmf_string_string =
reflection->GetMutableRepeatedFieldRef<google::protobuf::Message>(
&message, fd_map_string_string);
std::unique_ptr<google::protobuf::Message> entry_string_string(
google::protobuf::MessageFactory::generated_factory()
->GetPrototype(fd_map_string_string->message_type())
->New(message.GetArena()));
entry_string_string->GetReflection()->SetString(
entry_string_string.get(), fd_map_string_string->message_type()->field(0),
"1234");
entry_string_string->GetReflection()->SetString(
entry_string_string.get(), fd_map_string_string->message_type()->field(1),
std::to_string(10));
mmf_string_string.Add(*entry_string_string);
std::cout << "1234: " << message.infos().at("1234") << '\n';
The output:
1234: 10
I have a requirement to build the following Metadata message using serialized key value pairs in C++.
message MetadataValue {
string key = 1;
google.protobuf.Value value = 2;
}
message Metadata {
repeated MetadataValue metadata = 1;
}
So I can have the values for MetadataValue from the following for statement in C++.
Metadata metadata;
if (buffer.has_value()) {
auto pairs = buffer.value()->pairs();
for (auto &p : pairs) {
MetadataValue* metadataValue = metadata.add_metadata();
metadataValue->set_key(std::string(p.first));
// I don't know how to set the value for google.protobuf.Value
}
}
My question is whether my approach is correct ? Are there better alternatives and how to set the google.protobuf.Value in that above scenario ? A simple code snippet with the answer is much appreciated.
I think this code works, I just checked the generated APIs by protoc.
If the typeof(p.second) is not a google::protobuf::Value, you need to add conversion like
auto v = google::protobuf::Value();
v.set_number_value(p.second);
// or p.second is string
// v.set_string_value(p.second);
Metadata metadata;
if (buffer.has_value()) {
auto pairs = buffer.value()->pairs();
for (auto &p : pairs) {
MetadataValue* metadataValue = metadata.add_metadata();
metadataValue->set_key(std::string(p.first));
*metadataValue->mutable_value() = p.second;
// I don't know how to set the value for google.protobuf.Value
}
}
And I am using protoc version 3
syntax = "proto3";
import "google/protobuf/struct.proto";
message MetadataValue {
string key = 1;
google.protobuf.Value value = 2;
}
message Metadata {
repeated MetadataValue metadata = 1;
}
I tried to know what package I got in my tcp socket, so I use protobuf.
But when I SerializeToString my first protobuf class, the ParseFromString method of an other protobuf class returns true.
The two classes are differents
Here are my .proto
syntax = "proto3";
package protobuf;
message Message
{
string content = 1;
}
message Player
{
int32 id = 1;
string name = 2;
}
Here is my c++ code
auto messageProto = new protobuf::Message;
messageProto->set_content("Hello");
std::string data;
messageProto->SerializeToString(&data);
protobuf::Player player;
if (player.ParseFromString(data))
{
qDebug() << "parse player";
}
protobuf::Message message2;
if (message2.ParseFromString(data))
{
qDebug() << "parse message";
}
Output :
parse player
parse message
Why ?
My recommended solution to the problem of multiple different payloads:
syntax = "proto3";
package protobuf;
message RenameMe // the outer payload wrapper
{
oneof payload
{
Foo foo = 1;
Bar bar = 2;
}
}
message Foo // one type of content
{
string content = 1;
}
message Bar // another type of content
{
int32 id = 1;
string name = 2;
}
Now you can just deserialize everything as a RenameMe (naming is hard!) and check the payload discriminated union enum to see how you should interpret the data. Then just access foo or bar respectively.
This approach is clear, obvious, and readily and effectively extensible into additional message types. The testing can be implemented with switch in many programming languages. This style also works well with polymorphism in some environments - for example, in C# with protobuf-net that could be serialized/deserialized with:
[ProtoContract, ProtoInclude(1, typeof(Foo)), ProtoInclude(2, typeof(Bar))]
abstract class RenameMe {}
[ProtoContract]
class Foo : RenameMe {
[ProtoMember(1)] public string Content {get;set;}
}
[ProtoContract]
class Bar : RenameMe {
[ProtoMember(1)] public int Id {get;set;}
[ProtoMember(2)] public string Name {get;set;}
}
EDIT : So, now, is it the best approach ?
syntax = "proto3";
message Header
{
oneof payload
{
Message message = 1;
Player player = 2;
}
}
message Message
{
string content = 1;
}
message Player
{
int32 id = 1;
string name = 2;
}
When I write :
void SocketManager::sendData(Player& player)
{
Header header;
header.set_allocated_player(&player);
write(header);
}
void SocketManager::sendData(Message& message)
{
Header header;
header.set_allocated_message(&message);
write(header);
}
// etc... for each kind of message
When I read :
void read(const std::string& data)
{
protobuf::Header header;
header.ParseFromString(data);
switch (header.payload_case())
{
case protobuf::Header::kMessage:
emit messageProtoReceived(header.message());
break;
case protobuf::Header::kPlayer:
emit playerProtoReceived(header.player());
break;
case protobuf::Header::PAYLOAD_NOT_SET:
qDebug() << "Error, the payload isn't set, please create a header with a payload";
break;
}
}
I'm having some trouble with the following method and I need some help trying to figure out what I am doing wrong.
I want to return a reference to a Value in a document. I am passing the Document from outside the function so that when I read a json file into it I don't "lose it".
const rapidjson::Value& CTestManager::GetOperations(rapidjson::Document& document)
{
const Value Null(kObjectType);
if (m_Tests.empty())
return Null;
if (m_current > m_Tests.size() - 1)
return Null;
Test& the_test = m_Tests[m_current];
CMyFile fp(the_test.file.c_str()); // non-Windows use "r"
if (!fp.is_open())
return Null;
u32 operations_count = 0;
CFileBuffer json(fp);
FileReadStream is(fp.native_handle(), json, json.size());
if (document.ParseInsitu<kParseCommentsFlag>(json).HasParseError())
{
(...)
}
else
{
if (!document.IsObject())
{
(...)
}
else
{
auto tests = document.FindMember("td_tests");
if (tests != document.MemberEnd())
{
for (SizeType i = 0; i < tests->value.Size(); i++)
{
const Value& test = tests->value[i];
if (test["id"].GetInt() == the_test.id)
{
auto it = test.FindMember("operations");
if (it != test.MemberEnd())
{
//return it->value; is this legitimate?
return test["operations"];
}
return Null;
}
}
}
}
}
return Null;
}
Which I am calling like this:
Document document;
auto operations = TestManager().GetOperations(document);
When I inspect the value of test["operations"] inside the function I can see everything I would expect (debug code removed from the abode code).
When I inspect the returned value outside the function I can see that it's an array (which I expect). the member count int the array is correct as well, but when print it out, I only see garbage instead.
When I "print" the Value to a string inside the methods, I get what I expect (i.e. a well formated json), but when I do it outside all keys show up as "IIIIIIII" and values that aren't strings show up correctly.
rapidjson::StringBuffer strbuf2;
rapidjson::PrettyWriter<rapidjson::StringBuffer> writer2(strbuf2);
ops->Accept(writer2);
As this didn't work I decided to change the method to receive a Value as a parameter and do a deep copy into it like this
u32 CTestManager::GetOperationsEx(rapidjson::Document& document, rapidjson::Value& operations)
{
(...)
if (document.ParseInsitu<kParseCommentsFlag>(json).HasParseError())
{
(...)
}
else
{
if (!document.IsObject())
{
(...)
}
else
{
auto tests = document.FindMember("tests");
if (tests != document.MemberEnd())
{
for (SizeType i = 0; i < tests->value.Size(); i++)
{
const Value& test = tests->value[i];
if (test["id"].GetInt() == the_test.id)
{
const Value& opv = test["operations"];
Document::AllocatorType& allocator = document.GetAllocator();
operations.CopyFrom(opv, allocator); //would Swap work?
return operations.Size();
}
}
}
}
}
return 0;
}
Which I'm calling like this:
Document document;
Value operations(kObjectType);
u32 count = TestManager().GetOperationsEx(document, operations);
But... I get same thing!!!!
I know that it's going to be something silly but I can't put my hands on it!
Any ideas?
The problem in this case lies with the use of ParseInSitu. When any of the GetOperations exist the CFileBuffer loses scope and is cleaned up. Because the json is being parsed in-situ when the buffer to the file goes, so goes the data.