Google protobuf repeated fields with C++ - c++

I have a requirement to build the following Metadata message using serialized key value pairs in C++.
message MetadataValue {
string key = 1;
google.protobuf.Value value = 2;
}
message Metadata {
repeated MetadataValue metadata = 1;
}
So I can have the values for MetadataValue from the following for statement in C++.
Metadata metadata;
if (buffer.has_value()) {
auto pairs = buffer.value()->pairs();
for (auto &p : pairs) {
MetadataValue* metadataValue = metadata.add_metadata();
metadataValue->set_key(std::string(p.first));
// I don't know how to set the value for google.protobuf.Value
}
}
My question is whether my approach is correct ? Are there better alternatives and how to set the google.protobuf.Value in that above scenario ? A simple code snippet with the answer is much appreciated.

I think this code works, I just checked the generated APIs by protoc.
If the typeof(p.second) is not a google::protobuf::Value, you need to add conversion like
auto v = google::protobuf::Value();
v.set_number_value(p.second);
// or p.second is string
// v.set_string_value(p.second);
Metadata metadata;
if (buffer.has_value()) {
auto pairs = buffer.value()->pairs();
for (auto &p : pairs) {
MetadataValue* metadataValue = metadata.add_metadata();
metadataValue->set_key(std::string(p.first));
*metadataValue->mutable_value() = p.second;
// I don't know how to set the value for google.protobuf.Value
}
}
And I am using protoc version 3
syntax = "proto3";
import "google/protobuf/struct.proto";
message MetadataValue {
string key = 1;
google.protobuf.Value value = 2;
}
message Metadata {
repeated MetadataValue metadata = 1;
}

Related

How to use reflection of Protobuf to modify a Map

I'm working with Protobuf3 in my C++14 project. There have been some functions, which returns the google::protobuf::Message*s as a rpc request, what I need to do is to set their fields. So I need to use the reflection of Protobuf3.
Here is a proto file:
syntax="proto3";
package srv.user;
option cc_generic_services = true;
message BatchGetUserInfosRequest {
uint64 my_uid = 1;
repeated uint64 peer_uids = 2;
map<string, string> infos = 3;
}
message BatchGetUserInfosResponse {
uint64 my_uid = 1;
string info = 2;
}
Service UserSrv {
rpc BatchGetUserInfos(BatchGetUserInfosRequest) returns (BatchGetUserInfosResponse);
};
Now I called a function, which returns a google::protobuf::Message*, pointing an object BatchGetUserInfosRequest and I try to set its fields.
// msg is a Message*, pointing to an object of BatchGetUserInfosRequest
auto descriptor = msg->GetDescriptor();
auto reflection = msg->GetReflection();
auto field = descriptor->FindFieldByName("my_uid");
reflection->SetUInt64(msg, field, 1234);
auto field2 = descriptor->FindFieldByName("peer_uids");
reflection->GetMutableRepeatedFieldRef<uint64_t>(msg, field2).CopyFrom(peerUids); // peerUids is a std::vector<uint64_t>
As you see, I can set my_uid and peer_uids as above, but for the field infos, which is a google::protobuf::Map, I don't know how to set it with the reflection mechanism.
If you dig deep into the source code, you would find out the map in proto3 is implemented on the RepeatedField:
// Whether the message is an automatically generated map entry type for the
// maps field.
//
// For maps fields:
// map<KeyType, ValueType> map_field = 1;
// The parsed descriptor looks like:
// message MapFieldEntry {
// option map_entry = true;
// optional KeyType key = 1;
// optional ValueType value = 2;
// }
// repeated MapFieldEntry map_field = 1;
//
// Implementations may choose not to generate the map_entry=true message, but
// use a native map in the target language to hold the keys and values.
// The reflection APIs in such implementations still need to work as
// if the field is a repeated message field.
//
// NOTE: Do not set the option in .proto files. Always use the maps syntax
// instead. The option should only be implicitly set by the proto compiler
// parser.
optional bool map_entry = 7;
Inspired by the test code from protobuf, this works for me:
BatchGetUserInfosRequest message;
auto *descriptor = message.GetDescriptor();
auto *reflection = message.GetReflection();
const google::protobuf::FieldDescriptor *fd_map_string_string =
descriptor->FindFieldByName("infos");
const google::protobuf::FieldDescriptor *fd_map_string_string_key =
fd_map_string_string->message_type()->map_key();
const google::protobuf::FieldDescriptor *fd_map_string_string_value =
fd_map_string_string->message_type()->map_value();
const google::protobuf::MutableRepeatedFieldRef<google::protobuf::Message>
mmf_string_string =
reflection->GetMutableRepeatedFieldRef<google::protobuf::Message>(
&message, fd_map_string_string);
std::unique_ptr<google::protobuf::Message> entry_string_string(
google::protobuf::MessageFactory::generated_factory()
->GetPrototype(fd_map_string_string->message_type())
->New(message.GetArena()));
entry_string_string->GetReflection()->SetString(
entry_string_string.get(), fd_map_string_string->message_type()->field(0),
"1234");
entry_string_string->GetReflection()->SetString(
entry_string_string.get(), fd_map_string_string->message_type()->field(1),
std::to_string(10));
mmf_string_string.Add(*entry_string_string);
std::cout << "1234: " << message.infos().at("1234") << '\n';
The output:
1234: 10

How to set a protobuf message to a oneof struct

Assuming I have this proto
message inner_body1{
... // some attr
}
message inner_body2{
... // some attr
}
message body {
oneof inner{
inner_body1 = 1;
inner_body2 = 2;
}
}
message head {
... // some attr
}
message pkg{
head h = 1;
body b = 2;
}
And I design a function like this
void SendPkg(proto::Message& data)
{
pkg p;
auto head = p.mutable_head();
head->fillsomething(); // not important
// My question is, if 'data' is definitely one of the message type defined in 'body'(e.g. 'inner_body1')
// How can I put 'data' into pkg's body field?
}
My question is, if 'data' is definitely one of the message type defined in 'body'(e.g. 'inner_body1')
How can I put 'data' into pkg's body field?
update:
I have tried this way
void SendPkg(proto::Message& data)
{
pkg p;
auto head = p.mutable_head();
head->fillsomething(); // not important
auto body = p.mutable_body();
const Descriptor* desc = data.GetDescriptor();
if (desc.name() == "inner_body1")
{
auto body1 = body->mutable_innerbody1();
body1.CopyFrom(data);
}
else
{
auto body2 = body->mutable_innerbody2();
body2.CopyFrom(data);
}
}
this may works. But the fallback is obviously. I have to maintain this ugly string mapping and it running effienciency is low.
Is there any way could achieve this more elegant?

CouchDB view reduce one doc per key

I'm trying to solve what seems like a fairly simple problem with a couchDb view, but I'm not even getting close to the target with my result set.
Rather than updating documents, I'm creating a new document every time as my versioning strategy, and tying those documents together with a versioning field called ver. The very first document in a version chain will see the ver field and the _id field having the same value. All subsequent documents in the chain will have the same ver field as previous docs in the chain, but will have a unique _id field. These documents also have a createdTime field which is my way of knowing which document is the latest.
Here's my documents:
{
"_id": "abcd-1234-efgh-9876",
"ver": "abcd-1234-efgh-9876",
"createdTime": "2020-01-12 01:15:00 PM -0600",
...
},
{
"_id": "uopa-3849-pmdi-1935",
"ver": "abcd-1234-efgh-9876",
"createdTime": "2020-02-16 02:39:00 PM -0600",
...
}
Here's my map function:
function (doc) {
emit(doc.ver, doc);
}
Here's my reduce function:
function(keys, values, rereduce) {
var latestVersions = {};
for (var i = 0; i < keys.length; i++) {
var found = latestVersions[keys[i][0]];
if (!found || found.createdTime < values[i].createdTime) {
latestVersions[keys[i][0]] = values[i];
}
}
return latestVersions;
}
And finally, here's my desired output from the view (just the doc that I want):
{
"_id": "uopa-3849-pmdi-1935",
"ver": "abcd-1234-efgh-9876",
"createdTime": "2020-02-16 02:39:00 PM -0600",
...
}
What am I missing here? The reduce function is returning both records, which is not what I want. Is what I'm trying to achieve possible or is there a better way to go about this?
Update
I was able to get this to work when a single key is used to access the view, which is one of my use cases.
function (keys, values, rereduce) {
var toReturn = values[0];
for (var i = 1; i < values.length; i++) {
if (values[i].createdTime > toReturn.createdTime) {
toReturn = values[i];
}
}
return toReturn;
}
I have another use case that will be returning all of the data in the view, however. The desired result there is the same as above, but the function I'm using for single keys will only ever return one result. How do I filter multiple values with a shared key such that 1 "shared" key:n values -> 1 key:1 value.
I was finally able to resolve this when I stumbled upon this couchbase article. It was much more articulate than some of the other dry computer-science documentation.
I still do not understand why certain items are grouped in a reduce method and other ones are not. For example, reduce was called 5 times for 6 items that shared an identical key; only one of the keys had actually grouped anything -- an array of two documents. It probably has something to do with those dry computer-science B-tree documents I glossed over.
Anyway, I was able to determine that all I needed to do was group the values by the ver field in both scenarios (the only difference being that rereduce had a 2 dimensional array). Here's what my reduce function ended up looking like:
function (keys, values, rereduce) {
var toValues = function(myMap) {
return Object.keys(myMap).map(function(key) {
return myMap[key];
});
}
if (rereduce) {
// values should look like [[{...}, {...}], [{...}]]
var outputMap = {};
for (var i = 0; i < values.length; i++) {
for (var j = 0; j < values[i].length; j++) {
var currentEl = values[i][j];
var found = outputMap[currentEl.ver];
if ((found && found.createdDate < currentEl.createdDate) || !found) {
outputMap[currentEl.ver] = currentEl;
}
}
}
return toValues(outputMap);
} else {
var outputMap = {};
for (var i = 0; i < values.length; i++) {
var found = outputMap[values[i].ver];
if ((found && found.createdDate < values[i].createdDate) || !found) {
outputMap[values[i].ver] = values[i];
}
}
return toValues(outputMap);
}
}

How to append an attribute-value pair on an existing json11 object (c++)?

For example,
I'm building a json message using following code:
json11::Json my_json = json11::Json::object{
{ "key_val1", val1},
{ "key_val2", val2},
{ "key_val3", val3},
{ "key_val4", val4 }
};
std::string message = my_json.dump();
But if i want to have this json11 object contain different attribute-value pair based on some condition then I've to repeat the same code multiple times.
Is there any way to append attribute-value pair to an existing json11 object?
So that i can build a base object and then append necessary attributes on demand.
Yes it's possible.
json11::Json::object my_json = json11::Json::object{
{ "key_val1", val1},
{ "key_val2", val2},
{ "key_val3", val3},
{ "key_val4", val4 }
};
my_json["newattribute1"] = "newValue1";
my_json["newattribute2"] = 2;
json11::Json json_final = json11::Json{ my_json };
std::string message = json_final .dump();
In your case my_json is an instance of json11::Json. In my case my_json is an instance of json11::Json::object.
json11::Json::object is originally a std::map.

using std::map<int , std::string> with Google Protocol Buffer

I wanted to know if it was possible to use a map with a google protocol buffer.
I currently have something like this in my .proto file
message MsgA
{
required string symbol = 1 ;
optional int32 freq = 2 [default = 0];
}
message MsgB
{
//What should I do to make a map<int,MsgA>
}
My question is in MsgB i would like to create a type that would be a map::
Any suggestion on how I could accomplish this ?
Do this:
message MapEntry
{
required int32 mapKey = 1;
required MsgA mapValue = 2;
}
message MsgB
{
repeated MapEntry = 1;
}
You will have to write your own code to convert the map to and from a MsgB, but that should be basically trivial.