I have created a simple Protobuf based config file and have been using it without any issues until now. The issue is that I added two new items to my settings (Config.proto) and now whatever value I set for the last variable is reflected in the previous one.
The following snapshot demonstrates this better. As you can see below, the value of fv_shape_predictor_path and fv_eyenet_path depend solely on order of being set. the one that is set last changes the others value.
I made sure the cpp files related to Config.proto are built afresh. I also tested this under Linux and there it works just fine. It seems its only happening in windows! it also doesn't affect any other items in the same settings. its just these two new ones.
I have no idea what is causing this or how to go about it. For reference this is how the protobuf looks like:
syntax = "proto3";
package FVConfig;
message Config {
FV Configuration = 4 ;
message FAN_MODELS_WEIGHTS{
string fan_2DFAN_4 = 1;
string fan_3DFAN_4 = 2;
string fan_depth = 3;
}
message S3FD_MODELS_WEIGHTS{
string s3fd = 1;
}
message DLIB_MODELS_WEIGHTS{
string dlib_default = 1;
}
message MTCNN_MODELS_WEIGHTS {
string mt_onet = 1;
string mt_pnet = 2;
string mt_rnet = 3;
}
message FV_MODEL_WEIGHTS {
string r18 = 1;
string r50 = 2;
string r101 = 3;
repeated ModelContainer new_models_weights = 4;
message ModelContainer{
string model_name = 1;
string model_weight_path = 2;
string description = 3;
}
}
message FV {
MTCNNDetectorSettings mtcnn = 1 ;
FaceVerificationSettings fv = 2 ;
}
message MTCNNDetectorSettings {
Settings settings = 1;
MTCNN_MODELS_WEIGHTS model_weights = 4;
message Settings {
string mt_device = 2;
int32 mt_webcam_source = 100;
int32 mt_upper_threshold = 600;
int32 mt_hop = 700;
}
}
message FaceVerificationSettings {
Settings settings = 1;
FV_MODEL_WEIGHTS model_weights = 2;
message Settings {
string fv_model_name = 1;
string fv_model_checkpoint_path = 2;
bool fv_rebuild_cache = 3;
bool fv_short_circut = 6;
bool fv_accumulate_score = 7;
string fv_config_file_path = 10;
string fv_img_bank_folder_root = 11;
string fv_cache_folder = 12;
string fv_postfix = 13;
string fv_device = 14;
int32 fv_idle_interval = 15;
bool fv_show_dbg_info = 16;
// these are the new ones
string fv_shape_predictor_path = 17;
string fv_eyenet_path = 18;
}
}
} //end of Config message
What am I missing here? How should I be going about this? Restarting Windows and Visual Studio didn't do any good either. I'm using protobuf 3.11.4 both on Linux and Windows.
This issue seems to only exist in the Windows version of Protobuf 3.11.4 (didn't test with any newer version though).
Basically what happened was that I use to first create a Config object and initialize it with some default values. When I added these two entries to my Config.proto, I forgot to also add an initialization entry like other entries, thinking I'm fine with the default (which I assumed would be "").
This doesn't pose any issues under Linux/G++ and the program builds and runs just fine and works as intended. However under Windows this results in the behavior you just witnessed i.e. setting any of those newly added entries, would also set the other entries values. So basically I either had to create a whole new Config object or had to explicitly set their values when using the load_default_config.
To be more concrete this is the snippet I used for setting some default values in my Protobuf configs.
These reside in a separate header called Utility.h:
inline FVConfig::Config load_default_config()
{
FVConfig::Config baseCfg;
auto config = baseCfg.mutable_configuration();
load_fv_default_settings(config->mutable_fv());
load_mtcnn_default_settings(config->mutable_mtcnn());
return baseCfg;
}
inline void load_fv_default_settings(FVConfig::Config_FaceVerificationSettings* fv)
{
fv->mutable_settings()->set_fv_config_file_path(fv::config_file_path);
fv->mutable_settings()->set_fv_device(fv::device);
fv->mutable_settings()->set_fv_rebuild_cache(fv::rebuild_cache);
...
// these two lines were missing previously and to my surprise, this was indeed
// the cause of the weird behavior.
fv->mutable_settings()->set_fv_shape_predictor_path(fv::shape_predictor_path);
fv->mutable_settings()->set_fv_eyenet_path(fv::eyenet_path);
auto new_model_list = fv->mutable_model_weights()->mutable_new_models_weights()->Add();
new_model_list->set_model_name("r18");
new_model_list->set_description("default");
new_model_list->set_model_weight_path(fv::model_weights_r18);
}
inline void load_mtcnn_default_settings(FVConfig::Config_MTCNNDetectorSettings* mt)
{
mt->mutable_settings()->set_mt_device(mtcnn::device);
mt->mutable_settings()->set_mt_hop(mtcnn::hop);
....
}
Not sure this counts as a bug in Protobuf, or my wrong approach here.
I am currently trying to rewrite the network code of a MFC application to fit it to a new server, (written in c#) later on.
I'm experimenting with protobuf to ensure that all data models are the same.
My problem is, that the Scalar Value Types, protobuf works with is not understood by the MFC type system. It uses DWORD instead of uint32, BYTE instead of byte or bool and so on. That requires custom type mapping/casting and that's not necessarily taking work from my shoulders.
Is there a way to change those datatypes used in protobuf?
A sample protobuf model would be:
syntax = "proto3";
package CInterop;
option csharp_namespace = "CCX.Protobuf.Login";
message LoginAck {
LoginResultStatus Status = 1;
uint32 dwUserID = 2;
uint32 dwCharID = 3;
uint32 dwKEY = 4;
uint32 dwMapIP = 5;
uint32 wPort = 6;
uint32 bCharCreateCount = 7;
bool IsInPcRoom = 8;
uint32 PremiumPcRoomValue = 9;
uint64 dCurrentTime = 10;
uint64 dKey = 11;
string Reason = 12;
uint64 iDuration = 13;
bool bEternal = 14;
}
The current usage would be
CInterop::LoginAck loginAck;
(*pPacket) >> loginAck; // Reads the packet from the buffer
m_dwUserID = (DWORD)loginAck.dwuserid();
m_dwKickID = (DWORD)loginAck.dwcharid();
m_dwKEY = (DWORD)loginAck.dwkey();
vADDR.sin_addr.S_un.S_addr = (ULONG)loginAck.dwmapip();
vADDR.sin_port = (USHORT)loginAck.wport();
m_bCreateCardCnt = (byte)loginAck.bcharcreatecount();
m_bInPCROOM = (BYTE)loginAck.isinpcroom();
m_dwPrmPCROOM = (DWORD)loginAck.premiumpcroomvalue();
But here, i don't want to cast the types, this should happen automatically somehow.
I am wanting to resolve the following warning when concatenating a vertex declaration for use in Direct3D 9.0
int elements = 1 + useNormals + useTextures + useColors + 1;
D3DVERTEXELEMENT9 customvertex[elements];
customvertex[0] = POSITIONELEMENT;
int i = 1;
if (useNormals) { customvertex[i] = NORMALELEMENT; i += 1; }
if (useTextures) { customvertex[i] = TEXTUREELEMENT; i += 1; }
if (useColors) { customvertex[i] = COLORELEMENT; i += 1; }
customvertex[i] = D3DDECL_END();
The very last line is the one that produces the following warning when compiled with MinGW.
Graphics_Systems/Direct3D9/DX9model.cpp:340:20: warning: extended initializer lists only available with -std=c++11 or -std=gnu++11 [enabled by default]
I have two enums in my proto file which define almost the same values.
Is it possible to cut out one of them and to leave an alias to keep all the code working?
Example:
enum A {
a = 0;
b = 1;
}
enum B {
a = 0;
b = 1;
}
I want to have something like typedef in c++:
enum A {
a = 0;
b = 1;
}
typedef A B;
I haven't found this in documentation. Are there any workarounds ?
This is an old question, but if some people are still interested,
It is possible to create alias on enum with protobuf now
enum EnumAllowingAlias {
option allow_alias = true;
UNKNOWN = 0;
STARTED = 1;
RUNNING = 1;
}
enum EnumNotAllowingAlias {
UNKNOWN = 0;
STARTED = 1;
// RUNNING = 1; // Uncommenting this line will cause a compile error inside Google and a warning message outside.
}
As explained in the official documentation here. You just need to enable the alias option by adding option allow_alias = true;
As of protobuf version 3, this is not possible.
How can I get the value of a primitive literal using libclang?
For example, if I have a CXCursor of cursor kind CXCursor_IntegerLiteral, how can I extract the literal value.
UPDATE:
I've run into so many problems using libclang. I highly recommend avoiding it entirely and instead use the C++ interface clang provides. The C++ interface is highly useable and very well documented: http://clang.llvm.org/doxygen/annotated.html
The only purpose I see of libclang now is to generate the ASTUnit object for you as with the following code (it's not exactly easy otherwise):
ASTUnit * astUnit;
{
index = clang_createIndex(0, 0);
tu = clang_parseTranslationUnit(
index, 0,
clangArgs, nClangArgs,
0, 0, CXTranslationUnit_None
);
astUnit = static_cast<ASTUnit *>(tu->TUData);
}
Now you might say that libclang is stable and the C++ interface isn't. That hardly matters, as the time you spend figuring out the AST with libclang and creating kludges with it wastes so much of your time anyway. I'd just as soon spend a few hours fixing up code that does not compile after a version upgrade (if even needed).
Instead of reparsing the original, you already have all the information you need inside the translation unit :
if (kind == CXCursor_IntegerLiteral)
{
CXSourceRange range = clang_getCursorExtent(cursor);
CXToken *tokens = 0;
unsigned int nTokens = 0;
clang_tokenize(tu, range, &tokens, &nTokens);
for (unsigned int i = 0; i < nTokens; i++)
{
CXString spelling = clang_getTokenSpelling(tu, tokens[i]);
printf("token = %s\n", clang_getCString(spelling));
clang_disposeString(spelling);
}
clang_disposeTokens(tu, tokens, nTokens);
}
You will see that the first token is the integer itself, the next one is not relevant (eg. it's ; for int i = 42;.
If you have access to a CXCursor, you can make use of the clang_Cursor_Evaluate function, for example:
CXChildVisitResult var_decl_visitor(
CXCursor cursor, CXCursor parent, CXClientData data) {
auto kind = clang_getCursorKind(cursor);
switch (kind) {
case CXCursor_IntegerLiteral: {
auto res = clang_Cursor_Evaluate(cursor);
auto value = clang_EvalResult_getAsInt(res);
clang_EvalResult_dispose(res);
std::cout << "IntegerLiteral " << value << std::endl;
break;
}
default:
break;
}
return CXChildVisit_Recurse;
}
Outputs:
IntegerLiteral 42
I found a way to do this by referring to the original files:
std::string getCursorText (CXCursor cur) {
CXSourceRange range = clang_getCursorExtent(cur);
CXSourceLocation begin = clang_getRangeStart(range);
CXSourceLocation end = clang_getRangeEnd(range);
CXFile cxFile;
unsigned int beginOff;
unsigned int endOff;
clang_getExpansionLocation(begin, &cxFile, 0, 0, &beginOff);
clang_getExpansionLocation(end, 0, 0, 0, &endOff);
ClangString filename = clang_getFileName(cxFile);
unsigned int textSize = endOff - beginOff;
FILE * file = fopen(filename.c_str(), "r");
if (file == 0) {
exit(ExitCode::CANT_OPEN_FILE);
}
fseek(file, beginOff, SEEK_SET);
char buff[4096];
char * pBuff = buff;
if (textSize + 1 > sizeof(buff)) {
pBuff = new char[textSize + 1];
}
pBuff[textSize] = '\0';
fread(pBuff, 1, textSize, file);
std::string res(pBuff);
if (pBuff != buff) {
delete [] pBuff;
}
fclose(file);
return res;
}
You can actually use a combination of libclang and the C++ interface.
The libclang CXCursor type contains a data field which contains references to the underlying AST nodes.
I was able to successfully access the IntegerLiteral value by casting data[1] to the IntegerLiteral type.
I'm implementing this in Nim so I will provide Nim code, but you can likely do the same in C++.
let literal = cast[clang.IntegerLiteral](cursor.data[1])
echo literal.getValue().getLimitedValue()
The IntegerLiteral type is wrapped like so:
type
APIntObj* {.importcpp: "llvm::APInt", header: "llvm/ADT/APInt.h".} = object
# https://github.com/llvm-mirror/llvm/blob/master/include/llvm/ADT/APInt.h
APInt* = ptr APIntObj
IntegerLiteralObj* {.importcpp: "clang::IntegerLiteral", header: "clang/AST/Expr.h".} = object
IntegerLiteral* = ptr IntegerLiteralObj
proc getValue*(i: IntegerLiteral): APIntObj {.importcpp: "#.getValue()".}
# This is implemented by the superclass: https://clang.llvm.org/doxygen/classclang_1_1APIntStorage.html
proc getLimitedValue*(a: APInt | APIntObj): culonglong {.importcpp: "#.getLimitedValue()".}
Hope this helps someone :)