Memory issue with using C++ Protobuf in a DLL - c++

This question was posted on Google Group's Protobuf group but unfortunately it has gone unanswered.
A DLL has been created that will accept any Protobuf message which through the use of reflection will be filled with data from a database and then returned to the client. The database table name is mapped to the message name and the corresponding table columns are mapped to the message member names.
The issue appears to be with the 'SetString' method of the reflection class. After the call to the DLL has been made and the protobuf message with all the relevant has been received, a memory error is encountered as the program exists.
A typical message will have a .proto file such as the following:
message OperatingSystemInfo {
string manufacturer;
string name;
string release;
string osversion;
string localename;
string bootdevice;
string serialnumber;
}
The code snippets for the DLL functions are as follow:
void EXPORT_DLL ReadMessage(void* protoBufMessage) {
Message* msg = static_cast<Message*>(protoBufMessage);
const Descriptor* descriptor = msg->GetDescriptor();
const Reflection* reflection = msg->GetReflection();
for (int i = 0; i < descriptor->field_count(); i++) {
const FieldDescriptor* field = descriptor->field(i);
const FieldDescriptor::Type type = field->type();
bool isRepeated = field->is_repeated();
std::string name = field->name();
if (type == FieldDescriptor::TYPE_MESSAGE) {
}
else {
//read values from database for each corresponding field and set value
string fieldValue;
SetFieldValues(reflection, field, msg, fieldValue);
}
}
}
void SetFieldValues(const google::protobuf::Reflection reflection,
const google::protobuf::FieldDescriptor field,
google::protobuf::Message msg,
std::string fieldValue) {
switch (field->type()) {
case FieldDescriptor::TYPE_STRING:
reflection->SetString(msg, field, move(fieldValue))
break;
case FieldDescriptor::TYPE_UINT32:
break;
case FieldDescriptor::TYPE_UINT64:
break;
case FieldDescriptor::TYPE_BOOL:
break;
case FieldDescriptor::TYPE_ENUM:
break;
default:
break;
}
}
The client call to the DLL is:
void CallDLL()
{
OperatingSystemInfo* osi = new OperatingSystemInfo();
ReadMessage(osi);
delete osi;
}
The problem is that when the application is about to terminate, the 'Destroy' method for the last string in the message, 'serialnumber', causes a memory error (__acrt_first_block == header in debug_heap.ccp):
inline void OperatingSystemState::SharedDtor() {
GOOGLE_DCHECK(GetArenaForAllocation() == nullptr);
_impl_.manufacturer_.Destroy();
_impl_.name_.Destroy();
_impl_.release_.Destroy();
_impl_.osversion_.Destroy();
_impl_.localename_.Destroy();
_impl_.bootdevice_.Destroy();
_impl_.serialnumber_.Destroy();
}
void ArenaStringPtr::Destroy() {
//Line 233 in arenastring.cc
//leads to assertion Expession: __acrt_first_block == header in debug_heap.ccp
delete tagged_ptr_.GetIfAllocated();
}
Naturally this is an attempt to free an already freed memory. It is likely that allocating the memory outside of the DLL is a factor but the nature of the project is such that the DLL has no knowledge of Protobuf message types. Hence the reason that reflection is used.
Any helps or hints in resolving this issue is greatly appreciated.

Related

How to determine that type_name of a deleted DataWriter using RTI DDS

I'm writing a tool in c++ using RTI DDS 5.2 that needs to detect when DataWriters are deleted and know the type_name of the related data. I'm using code similar to this and this.
I'm using a DDSWaitSet and it is getting triggered when a DataWriter is deleted with delete_datawriter but the SampleInfo indicates that the data is not valid and sure enough, the data sample type_name is empty.
Is there a way to delete a DataWriter in such a was as to cause the built in topic subscription to get the type_name? Or is there a QOS setting I can set to fix this behavior?
Found a workaround for this problem. I still don't know how to make it so the data sample is valid when a DataWriter goes away, but the SampleInfo does have the field instance_state which uniquely identifies the writer. The solution is to keep track of type_names when the data is valid and just look it up when it's not. Here is the gist of the code I am using to solve the issue:
struct cmp_instance_handle {
bool operator()(const DDS_InstanceHandle_t& a, const DDS_InstanceHandle_t& b) const {
return !DDS_InstanceHandle_equals(&a, &b);
}
};
void wait_for_data_writer_samples()
{
ConditionSeq cs;
DDSWaitSet* ws = new DDSWaitSet();
StatusCondition* condition = _publication_dr->get_statuscondition();
DDS_Duration_t timeout = {DDS_DURATION_INFINITE_SEC, DDS_DURATION_INFINITE_NSEC};
std::map<DDS_InstanceHandle_t, std::string, cmp_instance_handle> instance_handle_map;
ws->attach_condition(condition);
condition->set_enabled_statuses(DDS_STATUS_MASK_ALL);
while(true) {
ws->wait(cs, timeout);
PublicationBuiltinTopicDataSeq data_seq;
SampleInfoSeq info_seq;
_publication_dr->take(
data_seq,
info_seq,
DDS_LENGTH_UNLIMITED,
ANY_SAMPLE_STATE,
ANY_VIEW_STATE,
ANY_INSTANCE_STATE
);
int len = data_seq.length();
for(int i = 0; i < len; ++i) {
DDS_InstanceHandle_t instance_handle = info_seq[i].instance_handle;
if(info_seq[i].valid_data) {
std::string type_name(data_seq[i].type_name);
// store the type_name in the map for future use
instance_handle_map[instance_handle] = type_name;
if(info_seq[i].instance_state == DDS_InstanceStateKind::DDS_ALIVE_INSTANCE_STATE) {
do_data_writer_alive_callback(type_name);
}
else {
// If the data is valid, but not DDS_ALIVE_INSTANCE_STATE, the DataWriter died *and* we can
// directly access the type_name so we can handle that case here
do_data_writer_dead_callback(type_name);
}
}
else {
// If the data is not valid then the DataWriter is definitely not alive but we can't directly access
// the type_name. Fortunately we can look it up in our map.
do_data_writer_dead_callback(instance_handle_map[instance_handle]);
// at this point the DataWriter is gone so we remove it from the map.
instance_handle_map.erase(instance_handle);
}
}
_publication_dr->return_loan(data_seq, info_seq);
}
}

Create a function with unique function pointer in runtime

When calling WinAPI functions that take callbacks as arguments, there's usually a special parameter to pass some arbitrary data to the callback. In case there's no such thing (e.g. SetWinEventHook) the only way we can understand which of the API calls resulted in the call of the given callback is to have distinct callbacks. When we know all the cases in which the given API is called at compile-time, we can always create a class template with static method and instantiate it with different template arguments in different call sides. That's a hell of a work, and I don't like doing so.
How do I create callback functions at runtime so that they have different function pointers?
I saw a solution (sorry, in Russian) with runtime assembly generation, but it wasn't portable across x86/x64 archtectures.
You can use the closure API of libffi. It allows you to create trampolines each with a different address. I implemented a wrapping class here, though that's not finished yet (only supports int arguments and return type, you can specialize detail::type to support more than just int). A more heavyweight alternative is LLVM, though if you're dealing only with C types, libffi will do the job fine.
I've come up with this solution which should be portable (but I haven't tested it):
#define ID_PATTERN 0x11223344
#define SIZE_OF_BLUEPRINT 128 // needs to be adopted if uniqueCallbackBlueprint is complex...
typedef int (__cdecl * UNIQUE_CALLBACK)(int arg);
/* blueprint for unique callback function */
int uniqueCallbackBlueprint(int arg)
{
int id = ID_PATTERN;
printf("%x: Hello unique callback (arg=%d)...\n", id, arg);
return (id);
}
/* create a new unique callback */
UNIQUE_CALLBACK createUniqueCallback(int id)
{
UNIQUE_CALLBACK result = NULL;
char *pUniqueCallback;
char *pFunction;
int pattern = ID_PATTERN;
char *pPattern;
char *startOfId;
int i;
int patterns = 0;
pUniqueCallback = malloc(SIZE_OF_BLUEPRINT);
if (pUniqueCallback != NULL)
{
pFunction = (char *)uniqueCallbackBlueprint;
#if defined(_DEBUG)
pFunction += 0x256; // variable offset depending on debug information????
#endif /* _DEBUG */
memcpy(pUniqueCallback, pFunction, SIZE_OF_BLUEPRINT);
result = (UNIQUE_CALLBACK)pUniqueCallback;
/* replace ID_PATTERN with requested id */
pPattern = (char *)&pattern;
startOfId = NULL;
for (i = 0; i < SIZE_OF_BLUEPRINT; i++)
{
if (pUniqueCallback[i] == *pPattern)
{
if (pPattern == (char *)&pattern)
startOfId = &(pUniqueCallback[i]);
if (pPattern == ((char *)&pattern) + sizeof(int) - 1)
{
pPattern = (char *)&id;
for (i = 0; i < sizeof(int); i++)
{
*startOfId++ = *pPattern++;
}
patterns++;
break;
}
pPattern++;
}
else
{
pPattern = (char *)&pattern;
startOfId = NULL;
}
}
printf("%d pattern(s) replaced\n", patterns);
if (patterns == 0)
{
free(pUniqueCallback);
result = NULL;
}
}
return (result);
}
Usage is as follows:
int main(void)
{
UNIQUE_CALLBACK callback;
int id;
int i;
id = uniqueCallbackBlueprint(5);
printf(" -> id = %x\n", id);
callback = createUniqueCallback(0x4711);
if (callback != NULL)
{
id = callback(25);
printf(" -> id = %x\n", id);
}
id = uniqueCallbackBlueprint(15);
printf(" -> id = %x\n", id);
getch();
return (0);
}
I've noted an interresting behavior if compiling with debug information (Visual Studio). The address obtained by pFunction = (char *)uniqueCallbackBlueprint; is off by a variable number of bytes. The difference can be obtained using the debugger which displays the correct address. This offset changes from build to build and I assume it has something to do with the debug information? This is no problem for the release build. So maybe this should be put into a library which is build as "release".
Another thing to consider whould be byte alignment of pUniqueCallback which may be an issue. But an alignment of the beginning of the function to 64bit boundaries is not hard to add to this code.
Within pUniqueCallback you can implement anything you want (note to update SIZE_OF_BLUEPRINT so you don't miss the tail of your function). The function is compiled and the generated code is re-used during runtime. The initial value of id is replaced when creating the unique function so the blueprint function can process it.

c++ protobuf: how to iterate through fields of message?

I'm new to protobuf and I'm stuck with simple task: I need to iterate through fields of message and check it's type. If type is message I will do same recursively for this message.
For example, I have such messages:
package MyTool;
message Configuration {
required GloablSettings globalSettings = 1;
optional string option1 = 2;
optional int32 option2 = 3;
optional bool option3 = 4;
}
message GloablSettings {
required bool option1 = 1;
required bool option2 = 2;
required bool option3 = 3;
}
Now, to explicitly access a field value in C++ I can do this:
MyTool::Configuration config;
fstream input("config", ios::in | ios::binary);
config.ParseFromIstream(&input);
bool option1val = config.globalSettings().option1();
bool option2val = config.globalSettings().option2();
and so on. This approach is not convenient in case when have big amount of fields.
Can I do this with iteration and get field's name and type? I know there are descriptors of type and somewhat called reflection, but I didn't have success in my attempts.
Can some one give me example of code if it's possible?
Thanks!
This is old but maybe someone will benefit. Here is a method that prints the contents of a protobuf message:
void Example::printMessageContents(std::shared_ptr<google::protobuf::Message> m)
{
const Descriptor *desc = m->GetDescriptor();
const Reflection *refl = m->GetReflection();
int fieldCount= desc->field_count();
fprintf(stderr, "The fullname of the message is %s \n", desc->full_name().c_str());
for(int i=0;i<fieldCount;i++)
{
const FieldDescriptor *field = desc->field(i);
fprintf(stderr, "The name of the %i th element is %s and the type is %s \n",i,field->name().c_str(),field->type_name());
}
}
You can find in FieldDescriptor Enum Values the possible values you get from field->type. For example for the message type you would have to check if type is equal to FieldDescriptor::TYPE_MESSAGE.
This function prints all the "metadata" of the protobuf message. However you need to check separately for each value what the type is and then call the corresponding getter function using Reflection.
So using this condition we could extract the strings :
if(field->type() == FieldDescriptor::TYPE_STRING && !field->is_repeated())
{
std::string g= refl->GetString(*m, field);
fprintf(stderr, "The value is %s ",g.c_str());
}
However fields can be either repeated or not repeated and different methods are used for both field types. So a check is used here to assure that we are using the right method. For repeated fields we have for example this method for strings :
GetRepeatedString(const Message & message, const FieldDescriptor * field, int index)
So it takes the index of the repeated field into consideration.
In the case of FieldDescriptor of type Message, the function provided will only print the name of the message, we better print its contents too.
if(field->type()==FieldDescriptor::TYPE_MESSAGE)
{
if(!field->is_repeated())
{
const Message &mfield = refl->GetMessage(*m, field);
Message *mcopy = mfield.New();
mcopy->CopyFrom(mfield);
void *ptr = new std::shared_ptr<Message>(mcopy);
std::shared_ptr<Message> *m =
static_cast<std::shared_ptr<Message> *>(ptr);
printMessageContents(*m);
}
}
And finally if the field is repeated you will have to call the FieldSize method on the reflection and iterate all repeated fields.
Take a look at how the Protobuf library implements the TextFormat::Printer class, which uses descriptors and reflection to iterate over fields and convert them to text:
https://github.com/google/protobuf/blob/master/src/google/protobuf/text_format.cc#L1473

Threads C++, Access Violation reading location x error

Platform : Windows 7
I'm developing a project for known text cipher attack in which;
Main process creates n child processes
Child processes decrypt an encrypted string, key subspace is partitioned according to number of child processes
Communication between child processes are by a static variable
for(int i = 0; i<info.totalNumberOfChildren; i++)
{
startChild( &info.childInfoList[i]);
//_beginthread(startChild, 0, &info.childInfoList[i]);
}
Above code works fine since:
First child starts execution, the key is set as a number such as 8 for testing purposes which is within the first child's partition, so first child finds the key, reports and sets true the killSwitch.
All the other children that are created are closed even before checking the first key as the killSwitch is true.
When I however do this :
for(int i = 0; i<info.totalNumberOfChildren; i++)
{
//startChild( &info.childInfoList[i]);
_beginthread(startChild, 0, &info.childInfoList[i]);
}
I get an access violation error. What could possibly be my source of error ?
Edit: I will try to share as relevant code as I can
startChild does the following:
void startChild( void* pParams)
{
ChildInfo *ci = (ChildInfo*)pParams;
// cout<<"buraya geldi"<<endl;
ChildProcess cp(*ci);
// write to log
cp.completeNextJob();
}
childInfo holds the following :
// header file
class ChildInfo
{
public:
ChildInfo();
ChildInfo(char * encrypted, char * original, static bool killSwitch, int totalNumOfChildren, int idNum, int orjLen);
void getNextJob();
bool keyIsFound();
Des des;
void printTest();
bool stopExecution;
bool allIsChecked;
char * encyptedString;
char * originalString;
int id;
int orjStrLen;
private:
int lastJobCompleted;
int totalNumberOfChildren;
int jobDistBits;
};
completeNextJob() does the following :
void ChildProcess::completeNextJob()
{
cout<<"Child Id : "<<info.id<<endl;
// cout<<"Trying : "<<info.encyptedString<<endl; // here I got an error
char * newtrial = info.encyptedString;
char * cand = info.des.Decrypt(newtrial); // here I also get an error if I comment out
/*
cout<<"Resultant : "<<cand<<endl;
cout<<"Comparing with : "<<info.originalString<<endl;
*/
bool match = true;
for(int i = 0; i<info.orjStrLen; i++)
{
if(!(cand[i] == info.originalString[i]))
match = false;
}
if(match)
{
cout<<"It has been acknowledged "<<endl;
info.stopExecution = true;
return;
}
else
{
if(!info.keyIsFound())
{
if(!info.allIsChecked)
{
info.getNextJob();
completeNextJob();
}
else
{
}
}
else
{
}
}
}
decrypt() method does the following :
char * Des::Decrypt(char *Text1)
{
int i,a1,j,nB,m,iB,k,K,B[8],n,t,d,round;
char *Text=new char[1000];
unsigned char ch;
strcpy(Text,Text1); // this is where I get the error
i=strlen(Text);
keygen();
int mc=0;
for(iB=0,nB=0,m=0;m<(strlen(Text)/8);m++) //Repeat for TextLenth/8 times.
{
for(iB=0,i=0;i<8;i++,nB++)
{
ch=Text[nB];
n=(int)ch;//(int)Text[nB];
for(K=7;n>=1;K--)
{
B[K]=n%2; //Converting 8-Bytes to 64-bit Binary Format
n/=2;
} for(;K>=0;K--) B[K]=0;
for(K=0;K<8;K++,iB++) total[iB]=B[K]; //Now `total' contains the 64-Bit binary format of 8-Bytes
}
IP(); //Performing initial permutation on `total[64]'
for(i=0;i<64;i++) total[i]=ip[i]; //Store values of ip[64] into total[64]
for(i=0;i<32;i++) left[i]=total[i]; // +--> left[32]
// total[64]--|
for(;i<64;i++) right[i-32]=total[i];// +--> right[32]
for(round=1;round<=16;round++)
{
Expansion(); //Performing expansion on `right[32]' to get `expansion[48]'
xor_oneD(round);
substitution();//Perform substitution on xor1[48] to get sub[32]
permutation(); //Performing Permutation on sub[32] to get p[32]
xor_two(); //Performing XOR operation on left[32],p[32] to get xor2[32]
for(i=0;i<32;i++) left[i]=right[i]; //Dumping right[32] into left[32]
for(i=0;i<32;i++) right[i]=xor2[i]; //Dumping xor2[32] into right[32]
} //rounds end here
for(i=0;i<32;i++) temp[i]=right[i]; // Dumping -->[ swap32bit ]
for(;i<64;i++) temp[i]=left[i-32]; // left[32],right[32] into temp[64]
inverse(); //Inversing the bits of temp[64] to get inv[8][8]
/* Obtaining the Cypher-Text into final[1000]*/
k=128; d=0;
for(i=0;i<8;i++)
{
for(j=0;j<8;j++)
{
d=d+inv[i][j]*k;
k=k/2;
}
final[mc++]=(char)d;
k=128; d=0;
}
} //for loop ends here
final[mc]='\0';
char *final1=new char[1000];
for(i=0,j=strlen(Text);i<strlen(Text);i++,j++)
final1[i]=final[j]; final1[i]='\0';
return(final);
}
Windows is trying to tell you why your program crashed. Please use a debugger to see what Windows is talking about. Location X is important: it should tell you whether your program is dereferencing NULL, overflowing a buffer, or doing something else. The call stack at the time of the crash is also very important.
Debugger is your best friend, try to use it and check step by step what could cause this access violation.
I think that info.encyptedString is not initialized correctly and pointing to not allocated memory, but I cant be sure because you didn't show this part of code.
And of course you must protect your shared resources (info) using some synchronization objects like critical section or mutex or semaphore.
I don't know, the basic issue seems pretty straightforward to me. You have multiple threads executing simultaneously, which access the same information via *pParams, which presumably is of type ChildInfo since that's what you cast it to. That info must be getting accessed elsewhere in the program, perhaps in the main thread. This is corrupting something, which may or may not have to do with Text1 or info.id, these errors can often be 'non-local' and hard to debug for this reason. So start mutex-protecting the entire thread (within your initial loop), and then zero in on the critical sections by trial and error, i.e. mutex-protect as small a region of code as you can get away with without producing errors.

Problem with realloc implementation resulting in Access violation

I have a custom string class that malloc/realloc/free's internally; For certain strings appending works fine, but on certain others it will always fail (small or large allocations). The same code worked fine on a different project, although it was an ANSI build.
I believe I'm implementing this correctly, but most likely I've overlooked something. The error occurs when I attempt to utilize the "szLog" buffer once the log has been opened. This simply contains the path to the program files directory (40 characters total). Using this same buffer without the "Log file '" prefix works fine, so it's an issue with the realloc section. And yes, the log does open properly.
I get 0xC0000005: Access violation reading location 0x00660063. only when realloc is used (but as previously stated, it doesn't always fail - in this situation, when szLog is input - but other variable strings/buffers do it too).
HeapReAlloc is the failing function inside realloc.c, errno being 22.
I've stripped the comments to try and keep the post as small as possible! Any help would be much appreciated.
gData.szLogStr is a UString, IsNull is a definition for "x == NULL" and unichar is simply a typedef for wchar_t
class UString : public Object
{
private:
unichar* mpsz;
unichar* mpszPrev;
UINT muiAlloc;
UINT muiLen;
public:
... other functions ...
UString& operator << (const unichar* pszAdd)
{
if ( IsNull(pszAdd) )
return (*this);
if ( IsNull(mpsz) )
{
muiAlloc = ((str_length(pszAdd)+1) * sizeof(unichar));
if ( IsNull((mpsz = static_cast<unichar*>(malloc(muiAlloc)))) )
{
SETLASTERROR(ERR_NOT_ENOUGH_MEMORY);
muiAlloc = 0;
return (*this);
}
mpszPrev = mpsz;
muiLen = str_copy(mpsz, pszAdd, muiAlloc);
}
else
{
UINT uiNewAlloc = (muiAlloc + (str_length(pszAdd) * sizeof(unichar)));
if ( muiAlloc < uiNewAlloc )
{
uiNewAlloc *= 2;
/* Fails */
if ( IsNull((mpsz = static_cast<unichar*>(realloc(mpsz, uiNewAlloc)))) )
{
SETLASTERROR(ERR_NOT_ENOUGH_MEMORY);
mpsz = mpszPrev;
return (*this);
}
mpszPrev = mpsz;
muiAlloc = uiNewAlloc;
}
muiLen = str_append(mpsz, pszAdd, muiAlloc);
}
return (*this);
}
and this being called from within main via:
UString szConf;
unichar szLog[MAX_LEN_GENERIC];
szConf << ppszCmdline[0];
szConf.replace(_T(".exe"), _T(".cfg"));
if ( GetPrivateProfileString(_T("Application"), _T("LogFile"), NULL, szLog, sizeofbuf(szLog), szConf.str()) == 0 )
{
UINT uiLen = str_copy(szLog, szConf.str(), sizeofbuf(szLog));
szLog[uiLen-3] = 'l';
szLog[uiLen-2] = 'o';
szLog[uiLen-1] = 'g';
}
if ( ApplicationLog::Instance().Open(szLog, CREATE_ALWAYS) )
{
gData.szLogStr.clear();
/* Erroring call */
gData.szLogStr << _T("Log file '") << szLog << _T("' opened");
APP_LOG(LL_WriteAlways, NULL, gData.szLogStr);
ObjMgr::Instance().DumpObjects(LogDumpedObjects);
}
You are programming in c++, so use new and delete. To 'renew', allocate a new memory area large enough to hold the new string, initialize it with the correct values and then delete the old string.
What is str_length() returning when it fails? I'd trace the value of muiAlloc and see what you're actually trying to allocate. It may not be a sane number.
Are you certain that whatever's in szLog is null-terminated, and that the buffer has room for whatever you're copying into it? It's not clear if str_copy is safe or not. It may be a wrapper around strncpy(), which does not guarantee a null terminator, but some people mistakenly assume it does.