After a few hours, I still can not set a oneof field in a clear (just created) protobuf message using reflection in c++.
I can obtain the needed OneOfDescriptor through the Descriptor of the message. But when I try to 'set' the oneof using Reflection, I found the real problem. There is only three function members related to OneOfDescriptor:
HasOneOf to check if there is a previous defined oneof in the message
GetOneofFieldDescriptor to get a FieldDescriptor from a previous defined oneof in the message
ClearOneof (without documentation) to clear oneof.
So there is not a SetOneofFieldDescriptorand if the oneof in the message is not previously defined, using a mutable_XXXX function member in the message, the GetOneofFieldDescriptor returns nullptr.
Therefore I am really stuck and any idea will be welcome.
Thanks in advance.
You set it the same way you would set the field if it weren't part of a oneof. Get a FieldDescriptor from the message's Descriptor and pass it to the appropriate SetXXX method of the message's Reflection.
Given a message like the following:
message Foo
{
oneof bar
{
int32 a = 1;
string b = 2;
}
}
You can set the a member as follows:
#include "foo.pb.h"
int main()
{
Foo f;
const google::protobuf::Descriptor* d = f.GetDescriptor();
const google::protobuf::FieldDescriptor* a = d->FindFieldByName("a");
const google::protobuf::Reflection* r = f.GetReflection();
r->SetInt32(&f, a, 42);
}
Protobuf will take care of making sure any previously set members of the oneof get unset as needed.
Related
I have a protobuf message roughly like this:
syntax = "proto3";
package state;
message Version
{
uint32 major = 1;
uint32 minor = 2;
uint32 fix = 3;
}
message Parameters
{
optional int32 a = 1;
optional int32 b = 2;
optional bool c = 3;
optional float d = 4;
}
/** A simple tree data structure to hold some state information */
message Tree
{
/** All state value attributes for this node, as a name/value pair map */
map<string, bytes> attributes = 1;
/** All child state trees as a child id / Tree pair map */
map<string, Tree> child_trees = 2;
}
message State
{
string product_name = 1;
Version product_version = 2;
Parameters parameters = 3;
Tree state_tree_a = 4;
Tree state_tree_b = 5;
}
This is already used in shipped software. Now we would like to add further optional fields to the Parameters message (all of them either bool, int or float). This has to be backwards compatible, e.g. a new version of the software should be able to parse a State message written with the older version, where there were less entries in the nested Parameters message. The messages are serialised in C++ via SerializeToArray and restored via ParseFromArray.
I did some tests and this seems to work without problems. Extending the Parameters message did not break the readability of the Tree messages after it and fields not defined in the first version just returned false when checked for existence.
But I won't rely on this limiting amount of try and error testing for shipping code, so I'd like to find some documentation on wether this should be safe by design or if it just worked by pure luck in my tests or only under certain circumstances. In this case, it would be completely okay for me if it worked in a C++ context only.
From my understanding, the Updating A Message Type section from the proto3 language guide does not give a clear answer wether it is safe or not to extend a message which sits nested in the middle of another message without risking to invalidate the data located behind it. But I might be overlooking something or getting something wrong from the documentation. Can anyone point me to a clearly documented answer to my question?
I want to degrade my protobuf version from 3.17.0 to 3.12.0, But I get a problem. For example:
syntax = "proto3";
message Foo {
optional int32 a = 1;
}
message Test {
oneof value {
string str = 1;
Foo foo = 2;
}
}
For this protofile, I compile it with protoc(3.17.0), I get a utility method in generated Test class to check if str is set in the oneof field:
// from generated test.pb.h
// string str = 1;
bool has_str() const;
private:
bool _internal_has_str() const;
public:
void clear_str();
// ...
But when I use protoc(3.12.0) to compile it and set the optional flag:
./protoc test.proto --cpp_out=./ --experimental_allow_proto3_optional
I got the generated code:
// string str = 1;
private:
bool _internal_has_str() const;
public:
void clear_str();
So ,has_str doesn't exist anymore. Does anybody know why? Thanks!
PS:
has_foo will always exist.
The has accessors in C++ for oneof fields were added in 3.15:
Now Proto3 Oneof fields have "has" methods for checking their presence in
C++.
This is coincidentally the same version that includes non-experimental proto3 optional support:
Optional fields for proto3 are enabled by default, and no longer require
the --experimental_allow_proto3_optional flag.
Prior to that version, one can use the generated _case() function to detect which oneof field is set.
I currently need to transfer a Protobuf message of a given type Foo over the network and handle it on reception.
I am using Protobuf 3.5.1 and the proto3 language.
On reception I do the following:
Create an empty instance of the expected type Foo (via a Factory Method, since it may change).
Parse the message from a CodedInputStream (constructed from the data received from the network).
Validate whether the parsing has been successful.
Here is a minimal code snippet illustrating the concrete problem:
// Create the empty Protobuf message of the EXPECTED type.
std::shared_ptr<google::protobuf::Message> message = std::make_unique<Foo>();
// Create the CodedInputStream from the network data.
google::protobuf::io::CodedInputStream stream{receive_buffer_.data(), static_cast<int>(bytes_transferred)};
if (message->ParseFromCodedStream(&stream) && stream.ConsumedEntireMessage() && message->IsInitialized()) {
// Message is of correct type and correctly parsed.
} else {
// Message is NOT VALID.
// TODO: This does not work if sending a message of type `Bar`.
}
Everything works flawless if the sender transfers messages of the expected type Foo, but things get weird if sending a message of another type Bar.
Both messages have a similar structure (they only difference is the oneof type which uses different types (but may use identical field identity values):
message Foo {
First first = 1;
oneof second {
A a = 2;
B b = 3;
}
}
message Bar {
First first = 1;
oneof second {
C c = 2;
D d = 3;
}
}
I haven't found a way yet, that detects whether the received Protobuf message type is "valid".
So my question is: How can I safely detect whether the parsed instance of Foo is valid and it is not an instance of another (similarly structured but completely different) Protobuf message type? Do I need to manually add the name of the message sent as a field (that would be stupid, but should work)?
I am new to GSOAP, so might be missing something obvious here.
But I really couldn't find its solution in GSOAP documentations.
I need to know, how do I silently ignore an unknown node in my xml in GSOAP without affecting other nodes.
For example: I have below class
class gsoap_ex
{
int foo;
char bar;
}
and below XML for it:
<gsoap_ex>
<foo>foo_value</foo>
<unknown>unknown_value</unknown>
<bar>bar_value</bar>
</gsoap_ex>
As of now, my gsoap parses the xml till it reaches the unknown node, after that it returns without further parsing it.
print_after_parsing(gsoap_ex *obj)
{
cout<<obj->foo;
cout<<obj->bar;
}
So in my above function it shows the value of foo but value of bar is not set.
How do I achieve it?
You can configure gSOAP to provide a function pointer to deal with unknown elements, and have your function decide what to do with it.
The https://www.cs.fsu.edu/~engelen/soapfaq.html page discusses handling unknown data. The first part of the paragraph is about detecting unexpected data so that you can figure out what to do about the problem; it seems you've already got that covered, so I've just included the sections detailing how to change the behavior of gSOAP.
My code appears to ignore data when receiving SOAP/XML messages. How can I detect unrecognized element tags at run time and let my
application fault?
...
Another way to control the dropping of unknown elements is to define
the fignore callback. For example:
{
struct soap soap;
soap_init(&soap);
soap.fignore = mustmatch; // overwrite default callback
...
soap_done(&soap); // reset callbacks
}
int mustmatch(struct soap *soap, const char *tag) {
return SOAP_TAG_MISMATCH; // every tag must be handled
}
The tag parameter contains the offending tag name. You can also
selectively return a fault:
int mustmatch(struct soap *soap, const char *tag) {
// all tags in namespace "ns" that start with "login" are optional
if (soap_match_tag(soap, tag, "ns:login*"))
return SOAP_OK;
// every other tag must be understood (handled)
return SOAP_TAG_MISMATCH;
}
Presumably, you'd want to write such a callback that returns SOAP_OK for your unexpected data.
im working into Googles Protocol Buffers right now and have a question. If i have multiple .proto files and thus multiple classes, is it somehow possible when the data is sent over a socket to determine which type it is?
E.g. i have two classes, lets call them person.proto and adress.proto. Now I send one of those over the wire. How can the receiver determine wheather it is a person or an adress?
I am doing this in C++.
My attempt would be adding a frame around the message, containing length and type. But i want to know if there is already some kind of implementation for the type stuff, so i dont reimplement existing stuff.
Yes it is possible. Protobuf supports reflection using so called message descriptors.
But (as stated in the other answer) you'll need a reliable well known root message type. Instead of introducing your own message discrimination mechanism, IMHO it's better to use protobufs extension mechanism
Here's a sample of what we have in production
package Common.ConfigurationCommands;
message UcpConfiguration
{
optional uint32 componentIndex = 1;
optional ConfigCmdStatus configCmdResponseStatus = 2;
optional string configErrorDescription = 3;
extensions 100 to max;
}
The extension looks like
import "Common/ConfigurationCommands.proto";
message AmplifierConfiguration
{
extend Common.ConfigurationCommands.UcpConfiguration
{
optional AmplifierConfiguration amplifierConfiguration = 108;
}
optional uint32 preemphasis = 1;
}
import "Common/ConfigurationCommands.proto";
message FrontendConfiguration
{
extend Common.ConfigurationCommands.UcpConfiguration
{
optional FrontendConfiguration frontendConfiguration = 100;
}
optional bool frontendActive = 1;
optional uint32 refInputComponentIndex = 2;
extensions 100 to max;
}
You can check this part of the documentation to see how to deal with extensions in your C++ code.
It is impossible to detect which object was serialized, Protobuf don't do it. But you can handle that using protobuf very easy:
1) Method: just send message that has type and string body. To body you will serialize your objects, and in type you will show which object is serialized:
Something like that:
package MyGreatPackage;
message Pack
{
required bytes packcode = 1;
//code for data/query
required bytes mess = 2;
}
message Data
{
//anything you need to
}
message Query
{
//anything you need to
}
So, you will always send message Pack, where will be defined which object exactly is in "mess" field.
2) Method: protobuf allows this technique to achieve same thing without pack wrapper, look here: https://developers.google.com/protocol-buffers/docs/techniques?hl=ru#union
message OneMessage {
enum Type { FOO = 1; BAR = 2; BAZ = 3; }
// Identifies which field is filled in.
required Type type = 1;
// One of the following will be filled in.
optional Foo foo = 2;
optional Bar bar = 3;
optional Baz baz = 4;
}
So, you can set all classes you may send as optional and determine their types by required parameter.
Still, for me first varians seems better, choose what you like.