How to use shared_pointer as value in a std::map structure? - c++

I have a map structure as below which has shared_pointer of a protobuf structure Stats:
map<string, shared_pointer<vol::proto::Stats> statsMap;
I am storing and retrieving the map entries through a LRU cache implementation
template class LRUCache<string,std::shared_ptr<vol::proto::Stats>>;
Protobuf structure:-
message Stats {
required string oid = 1;
required int64 logical_size = 2;
required int64 committed_size = 3;
}
message StatsRequest {
required string uuid = 1;
}
The function for storing entry into the map:-
template<class K, class V>
void
LRUCache<K, V>::Put(const K& key, V value)
{
Vmacore::System::WriteSynchronized lock(this);
auto val = refMap.find(key);
if (val == refMap.end()) {
if (cacheSize == refMap.size()) {
K leastUsedKey = lruList.back();
lruList.pop_back();
refMap.erase(leastUsedKey);
}
} else {
lruList.erase(val->second.lruKeyRef);
}
lruList.push_front(key);
refMap[key] = LRUValueReference<K, V>(lruList.begin(), value);
}
void PutS(const string& Id, const vol::proto::Stats& info)
{
shared_ptr<vol::proto::Stats> statsInfo = make_shared<vol::proto::Stats>(info);
_StatsCache.Put(Id, statsInfo);
}
void PutStats(vol::proto::StatsRequest &req) {
vol::proto::Stats *stats;
GetStats(stats); //stats gets populated here.
PutS(stats->oid(), *stats);
}
To Get the entry from the map:-
template<class K, class V>
bool
LRUCache<K, V>::Get(const K& key, V& value)
{
Vmacore::System::WriteSynchronized lock(this);
auto val = refMap.find(key);
if (val == refMap.end()) {
return false;
}
lruList.splice(lruList.begin(), lruList, val->second.lruKeyRef);
val->second.lruKeyRef = lruList.begin();
value = val->second.value;
return true;
}
void GetS(const string& Id, shared_ptr<vol::proto::Stats> info)
{
if (!_StatsCache.Get(Id, info))
return false;
return true;
}
void GetStats(vol::proto::StatsRequest &req) {
shared_ptr<vol::proto::Stats> stats;
if (GetS(req.uuid(), stats)) {
Log("object id is %s", stats.get()->oid()); // Crash at oid (this=0x0)
}
}
Now while doing GetStats, I am seeing a crash as to in GetStats's Log statement that is program terminated with signal SIGSEGV, Segmentation fault. Should I do a make_share for Stats in GetStats? Can you please describe as to what has caused this crash and how to fix the crash here?

You're passing a raw pointer from GetStats to GetS, which then wraps it into a shared_ptr. Now lets start ref-counting.
GetS -> increase on entry > 1
_StatsCache.Get -> pass by ref > 1
GetS -> decrease on exit > 0
Ref count 0 -> delete the pointer.
In GetStats you then try to de-reference the stats pointer by calling .get, but it was just deleted.
You should either always use smart pointers, or be very very very careful when you convert from raw pointer to smart pointer, and back.
Actually what I wrote is wrong. What you're doing is actually much worse.
You're wrapping a raw pointer in GetS into a smart pointer, and then pass it by ref to your _StatsCache.Get, which then assigns a different shared_ptr to yours. So you've overwritten the original raw pointer with something else.
But when GetS returns, that shared_ptr is now lost and destructed, but the original stats pointer is accessed in stats.get(). What's in it? Nothing from GetS, that's for sure.

Related

CString used on static method

I am stuck into a problem, and I will be very grateful if you help me.
I have a MDI, and in CDocument class, I have a struct:
CMyDoc.h
class CMyDoc : public CDocument
{
...
struct SRecord
{
SRecord(){}
virtual ~SRecord(){}
CString sName;
CString sState;
CString sDateu;
CString sDatec;
};
CTypedPtrArray<CPtrArray, SRecord*> m_arrRecord;
and somewhere I load this struct with data:
SRecord* pItem = new SRecord;
pItem->sName = saItem.GetAt(ML_ASSETNAME);
pItem->sState = saItem.GetAt(ML_STATE);
pItem->sDateu = saItem.GetAt(ML_DATEU;
pItem->sDatec = saItem.GetAt(ML_DATEC);
m_arrRecord.Add(pItem);
ok. I am trying to sort data:
void CMyDoc::SortData(int nColumn, BOOL bAscending)
{
switch(nColumn)
{
case 9:
if(bAscending)qsort((void*)m_arrRecord.GetData(), m_arrRecord.GetSize(), sizeof(SRecord), CompareDateUAscending);
else qsort((void*)m_arrRecord.GetData(), m_arrRecord.GetSize(), sizeof(SRecord), CompareDateUDescending);
break;
...
}
but the problem become when data is access in static method:
int CMyDoc::CompareDateUDescending(const void* arg1, const void* arg2)
{
SRecord* Record1 = (SRecord*)arg1;
SRecord* Record2 = (SRecord*)arg2;
if(Record1->sDateu.IsEmpty() || Record2->sDateu.IsEmpty()) // <--- crash !
return 0;
COleDateTime dL, dR;
dL.ParseDateTime(Record1->sDateu);
dR.ParseDateTime(Record2->sDateu);
return (dL == dR ? 0 : (dL < dR ? 1 : -1));
}
and the crash take me here (atlsimpstr.h):
CStringData* GetData() const throw()
{
return( reinterpret_cast< CStringData* >( m_pszData )-1 ); // the crash lead me on this line
}
what I am doing wrong ?
Any help will be very appreciated !
Update:
I have tried this:
int CMyDoc::CompareDateUDescending(const void* arg1, const void* arg2)
{
SRecord* Record1 = *(SRecord**)arg1; // <-- OK
SRecord* Record2 = *(SRecord**)arg2; // <-- Unhandled exception* see note below
if(Record1->sDateu.IsEmpty() || Record2->sDateu.IsEmpty())
return 0;
COleDateTime dL, dR;
dL.ParseDateTime(Record1->sDateu);
dR.ParseDateTime(Record2->sDateu);
return (dL == dR ? 0 : (dL < dR ? 1 : -1));
}
and the crash told me:
"An unhandled exception was encountered during a user callback." strange ...
The qsort comparison function receives pointers to the elements in the array. But since the elements in the array are themselves pointers what your specific function receives as arguments are pointers to pointers to SRecord, i.e. SRecord**.
You can solve it by doing e.g.
const SRecord* Record1 = *reinterpret_cast<const SRecord**>(arg1);
That is, you cast arg1 to SRecord** and then dereference that pointer to get a SRecord*.
Example on how to use the C++ standard sort function.
First you need to update your comparison function a little:
// The comparison function should return true if Record1 is *smaller* than Record2,
// and return false otherwise
bool CMyDoc::CompareDateUDescending(const SRecord* Record1, const SRecord* Record2)
{
return Record1->sDateu < Record2->sDateu;
}
Then to call sort:
std::sort(m_arrRecord.GetData(), m_arrRecord.GetData() + m_arrRecord.GetSize(), CompareDateUDescending);
Much simpler!

Rapidjson returning reference to Document Value

I'm having some trouble with the following method and I need some help trying to figure out what I am doing wrong.
I want to return a reference to a Value in a document. I am passing the Document from outside the function so that when I read a json file into it I don't "lose it".
const rapidjson::Value& CTestManager::GetOperations(rapidjson::Document& document)
{
const Value Null(kObjectType);
if (m_Tests.empty())
return Null;
if (m_current > m_Tests.size() - 1)
return Null;
Test& the_test = m_Tests[m_current];
CMyFile fp(the_test.file.c_str()); // non-Windows use "r"
if (!fp.is_open())
return Null;
u32 operations_count = 0;
CFileBuffer json(fp);
FileReadStream is(fp.native_handle(), json, json.size());
if (document.ParseInsitu<kParseCommentsFlag>(json).HasParseError())
{
(...)
}
else
{
if (!document.IsObject())
{
(...)
}
else
{
auto tests = document.FindMember("td_tests");
if (tests != document.MemberEnd())
{
for (SizeType i = 0; i < tests->value.Size(); i++)
{
const Value& test = tests->value[i];
if (test["id"].GetInt() == the_test.id)
{
auto it = test.FindMember("operations");
if (it != test.MemberEnd())
{
//return it->value; is this legitimate?
return test["operations"];
}
return Null;
}
}
}
}
}
return Null;
}
Which I am calling like this:
Document document;
auto operations = TestManager().GetOperations(document);
When I inspect the value of test["operations"] inside the function I can see everything I would expect (debug code removed from the abode code).
When I inspect the returned value outside the function I can see that it's an array (which I expect). the member count int the array is correct as well, but when print it out, I only see garbage instead.
When I "print" the Value to a string inside the methods, I get what I expect (i.e. a well formated json), but when I do it outside all keys show up as "IIIIIIII" and values that aren't strings show up correctly.
rapidjson::StringBuffer strbuf2;
rapidjson::PrettyWriter<rapidjson::StringBuffer> writer2(strbuf2);
ops->Accept(writer2);
As this didn't work I decided to change the method to receive a Value as a parameter and do a deep copy into it like this
u32 CTestManager::GetOperationsEx(rapidjson::Document& document, rapidjson::Value& operations)
{
(...)
if (document.ParseInsitu<kParseCommentsFlag>(json).HasParseError())
{
(...)
}
else
{
if (!document.IsObject())
{
(...)
}
else
{
auto tests = document.FindMember("tests");
if (tests != document.MemberEnd())
{
for (SizeType i = 0; i < tests->value.Size(); i++)
{
const Value& test = tests->value[i];
if (test["id"].GetInt() == the_test.id)
{
const Value& opv = test["operations"];
Document::AllocatorType& allocator = document.GetAllocator();
operations.CopyFrom(opv, allocator); //would Swap work?
return operations.Size();
}
}
}
}
}
return 0;
}
Which I'm calling like this:
Document document;
Value operations(kObjectType);
u32 count = TestManager().GetOperationsEx(document, operations);
But... I get same thing!!!!
I know that it's going to be something silly but I can't put my hands on it!
Any ideas?
The problem in this case lies with the use of ParseInSitu. When any of the GetOperations exist the CFileBuffer loses scope and is cleaned up. Because the json is being parsed in-situ when the buffer to the file goes, so goes the data.

Recursive insert of a chain into memory fails

This meight be a long question but i hope someone can help me figuring out whats going wrong.
I am inserting a JSON Object into already allocated Memory with my own Datatype which basically holds a Union with Data and a ptrdiff_t to the next Datatype in 8bit steps.
template <typename T>
class BaseType
{
public:
BaseType();
explicit BaseType(T& t);
explicit BaseType(const T& t);
~BaseType();
inline void setNext(const ptrdiff_t& next);
inline std::ptrdiff_t getNext();
inline void setData(T& t);
inline void setData(const T& t);
inline T getData() const;
protected:
union DataUnion
{
T data;
::std::ptrdiff_t size;
DataUnion()
{
memset(this, 0, sizeof(DataUnion));
} //init with 0
explicit DataUnion(T& t);
explicit DataUnion(const T& t);
} m_data;
long long m_next;
};
The implementation is streight so nothing special happes there just setting/getting the values of the definition. (i'll skip the impl. here)
So here starts the code where something goes wrong:
std::pair<void*, void*> Page::insertObject(const rapidjson::GenericValue<rapidjson::UTF8<>>& value,
BaseType<size_t>* last)
{
//return ptr to the first element
void* l_ret = nullptr;
//prev element ptr
BaseType<size_t>* l_prev = last;
//position pointer
void* l_pos = nullptr;
//get the members
for (auto it = value.MemberBegin(); it != value.MemberEnd(); ++it)
{
switch (it->value.GetType())
{
case rapidjson::kNullType:
LOG_WARN << "null type: " << it->name.GetString();
continue;
case rapidjson::kFalseType:
case rapidjson::kTrueType:
{
l_pos = find(sizeof(BaseType<bool>));
void* l_new = new (l_pos) BaseType<bool>(it->value.GetBool());
if (l_prev != nullptr)
l_prev->setNext(dist(l_prev, l_new));
}
break;
case rapidjson::kObjectType:
{
//pos for the obj id
//and insert the ID of the obj
l_pos = find(sizeof(BaseType<size_t>));
std::string name = it->name.GetString();
void* l_new = new (l_pos) BaseType<size_t>(common::FNVHash()(name));
if (l_prev != nullptr)
l_prev->setNext(dist(l_prev, l_new));
//TODO something strange happens here!
// pass the objid Object to the insertobj!
// now recursive insert the obj
// the second contains the last element inserted
// l_pos current contains the last inserted element and get set to the
// last element of the obj we insert
l_pos = (insertObject(it->value, reinterpret_cast<BaseType<size_t>*>(l_new)).second);
}
break;
case rapidjson::kArrayType:
{//skip this at the moment till the bug is fixed
}
break;
case rapidjson::kStringType:
{
// find pos where the string fits
// somehow we get here sometimes and it does not fit!
// which cant be since we lock the whole page
l_pos = find(sizeof(StringType) + strlen(it->value.GetString()));
//add the String Type at the pos of the FreeType
auto* l_new = new (l_pos) StringType(it->value.GetString());
if (l_prev != nullptr)
l_prev->setNext(dist(l_prev, l_new));
}
break;
case rapidjson::kNumberType:
{
//doesnt matter since long long and double are equal on x64
//find pos where the string fits
l_pos = find(sizeof(BaseType<long long>));
void* l_new;
if (it->value.IsInt())
{
//insert INT
l_new = new (l_pos) BaseType<long long>(it->value.GetInt64());
}
else
{
//INSERT DOUBLE
l_new = new (l_pos) BaseType<double>(it->value.GetDouble());
}
if (l_prev != nullptr)
l_prev->setNext(dist(l_prev, l_new));
}
break;
default:
LOG_WARN << "Unknown member Type: " << it->name.GetString() << ":" << it->value.GetType();
continue;
}
//so first element is set now, store it to return it.
if(l_ret == nullptr)
{
l_ret = l_pos;
}
//prev is the l_pos now so cast it to this;
l_prev = reinterpret_cast<BaseType<size_t>*>(l_pos);
}
//if we get here its in!
return{ l_ret, l_pos };
}
I am starting to insert like this:
auto firstElementPos = insertObject(value.MemberBegin()->value, nullptr).first;
While value.MemberBegin()->value is Object to be inserted and ->name holds the Name of the object. In the case below its Person and everything between {}.
The problem is, if i insert a JSON Object which has one Object inside like so:
"Person":
{
"age":25,
"double": 23.23,
"boolean": true,
"double2": 23.23,
"firstInnerObj":{
"innerDoub": 12.12
}
}
It works properly and i can reproduce the Object. But if i have more inner objects like so:
"Person":
{
"age":25,
"double": 23.23,
"boolean": true,
"double2": 23.23,
"firstInnerObj":{
"innerDoub": 12.12
},
"secondInnerObj":{
"secInnerDoub": 12.12
}
}
It fails and i lose data so i think that my recursion goes wrong but i dont see why. If you need any more informations let me know. Meight take a look here and the client here.
The test.json need to contain a json object like above. And the find only need to contain {"oid__":2} to get the second object that was inserted.
I could track the issue down to the Point where i recreate the Object recursively in the code. Some of the Nextpointers seem to be incorrect:
void* Page::buildObject(const size_t& hash, void* start, rapidjson::Value& l_obj,
rapidjson::MemoryPoolAllocator<>& aloc)
{
//get the meta information of the object type
//to build it
auto& l_metaIdx = meta::MetaIndex::getInstance();
//get the meta dataset
auto& l_meta = l_metaIdx[hash];
//now we are already in an object here with l_obj!
auto l_ptr = start;
for (auto it = l_meta->begin(); it != l_meta->end(); ++it)
{
//create the name value
rapidjson::Value l_name(it->name.c_str(), it->name.length(), aloc);
//create the value we are going to add
rapidjson::Value l_value;
//now start building it up again
switch (it->type)
{
case meta::OBJECT:
{
auto l_data = static_cast<BaseType<size_t>*>(l_ptr);
//get the hash to optain the metadata
auto l_hash = l_data->getData();
//set to object and create the inner object
l_value.SetObject();
//get the start pointer which is the "next" element
//and call recursive
l_ptr = static_cast<BaseType<size_t>*>(buildObject(l_hash,
(reinterpret_cast<char*>(l_data) + l_data->getNext()), l_value, aloc));
}
break;
case meta::ARRAY:
{
l_value.SetArray();
auto l_data = static_cast<ArrayType*>(l_ptr);
//get the hash to optain the metadata
auto l_size = l_data->size();
l_ptr = buildArray(l_size, static_cast<char*>(l_ptr) + l_data->getNext(), l_value, aloc);
}
break;
case meta::INT:
{
//create the data
auto l_data = static_cast<BaseType<long long>*>(l_ptr);
//with length attribute it's faster ;)
l_value = l_data->getData();
}
break;
case meta::DOUBLE:
{
//create the data
auto l_data = static_cast<BaseType<double>*>(l_ptr);
//with length attribute it's faster ;)
l_value = l_data->getData();
}
break;
case meta::STRING:
{
//create the data
auto l_data = static_cast<StringType*>(l_ptr);
//with length attribute it's faster
l_value.SetString(l_data->getString()->c_str(), l_data->getString()->length(), aloc);
}
break;
case meta::BOOL:
{
//create the data
auto l_data = static_cast<BaseType<bool>*>(l_ptr);
l_value = l_data->getData();
}
break;
default:
break;
}
l_obj.AddMember(l_name, l_value, aloc);
//update the lptr
l_ptr = static_cast<char*>(l_ptr) + static_cast<BaseType<size_t>*>(l_ptr)->getNext();
}
//return the l_ptr which current shows to the next lement. //see line above
return l_ptr;
}
After houers and houres of debugging i found the small issue which causes this. The method which builds up the Object after it was inserted returns a pointer to the actuall last element->next which was inserted and after the switch case i did call the ->next again which causes a loss of data because it scipped one element in the single chained list.
The Fix to this is to put the line
l_ptr = static_cast<char*>(l_ptr) + static_cast<BaseType<size_t>*>(l_ptr)->getNext();
Only into the switch cases where it is not an Object or Array. Fix Commit This actually also gave me the fix for an Issue with inserting Array.
Of cause the real issue could not know someone here who did not took a deep look into the code but i still want to show the fix here. Thanks to #sehe who helped alot with figuring out whats going wrong here.

storing struct as value using CFDictionarySetValue()

Iam new to using Core Foundations. I want to use dictionary to store some key value pair. The value must be a pointer to a struct. This pointer is pointing to dynamically allocated buffer.
CFMutableDictionaryRef init_hash_table() {
return CFDictionaryCreateMutable(NULL, 0, &kCFTypeDictionaryKeyCallBacks, &kCFTypeDictionaryValueCallBacks);
}
This is used to create the dictionary and the return value is stored as global variable.
CFNumberRef
create_hash_key(int sd) {
return CFNumberCreate(NULL, kCFNumberIntType, &sd);
}
int
add_hash_entry(CFMutableDictionaryRef dict, int sd, void *pkt) {
CFNumberRef key = create_hash_key(sd);
CFDictionarySetValue(dict, key, pkt);
return 0;
}
When I execute this code, I get segfault. I see that pkt has a valid address and key seems to be created. Does anyone know how to assign a pointer to value part?
Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x0000000000000011
0x00007fff8c9f339f in objc_msgSend_fixup ()
Any ideas?
The problem is the kCFTypeDictionaryValueCallBacks argument. From the documentation:
kCFTypeDictionaryValueCallBacks
Predefined CFDictionaryValueCallBacks structure containing a set of
callbacks appropriate for use when the values in a CFDictionary are
all CFType-derived objects.
So in your case, CFRetain() is called on the pointer when the value is added to the
dictionary. This causes the crash because the pointer does not point to a CoreFoundation
object.
You can create the dictionary with
CFDictionaryCreateMutable(NULL, 0, &kCFTypeDictionaryKeyCallBacks, NULL);
instead, so that no "reference counting" will be done on the value.
Alternatively, you can wrap the pointer into a CFDataRef and put that into the
dictionary.
In both cases it is your responsibility that the pointer is still valid
when the value is retrieved from the dictionary later.
Here is a simple example how you could implement refcounting for your custom objects:
typedef struct {
int refcount;
int val;
} mystruct;
const void *myretain(CFAllocatorRef allocator, const void *value)
{
mystruct *p = (mystruct *)value;
p->refcount++;
return p;
}
void myrelease(CFAllocatorRef allocator, const void *value)
{
mystruct *p = (mystruct *)value;
if (p->refcount == 1)
free(p);
else
p->refcount--;
}
int main(int argc, const char * argv[])
{
mystruct *p = malloc(sizeof(*p));
p->refcount = 1;
p->val = 13;
CFDictionaryValueCallBacks vcb = { 0 , myretain, myrelease, NULL, NULL };
CFMutableDictionaryRef dict = CFDictionaryCreateMutable(NULL, 0, &kCFTypeDictionaryKeyCallBacks, &vcb);
int sd = 13;
CFNumberRef key = CFNumberCreate(NULL, kCFNumberIntType, &sd);
CFDictionarySetValue(dict, key, p);
// refcount == 2
myrelease(NULL, p);
// refcount == 1
mystruct *q = CFDictionaryGetValue(dict, key);
// refcount is still 1, "GetValue" does not increment the refcount
CFRelease(dict);
// object is deallocated
return 0;
}

Thread Argument Problem

I have working on multithreaded application I am passing IMAGETHREADINFO structure in thread here nSock is showing garbage value. What is the problem here.pointer to IMAGETHREADINFO is declared as member variable of CServerConnectionMgr class.
typedef struct
{
int nScok;
CServerConnectionMgr* pConMgr;
}IMAGETHREADINFO;
void StartImageThread(SOCKET nSock)
{
stThreadInfo = new IMAGETHREADINFO;
stThreadInfo.pConMgr = this;
stThreadInfo.nScok = nSock;
m_hRecordImageThread = CreateThread ( NULL,0, (LPTHREAD_START_ROUTINE)StreamImageThread,(void*)&stThreadInfo, 0,&m_nRecordImageThreadID);
if ( NULL == m_hRecordImageThread)
{
return;
}
int CServerConnectionMgr::StreamImageThread(void *args)
{
IMAGETHREADINFO *pImageThreadInfo = (IMAGETHREADINFO*)&args;
}
This is variable pImageThreadInfo->nSock showing some garbage value
This pImageThreadInfo->pConMgr is coming correctly
I this is showing wrong value
(void*)&stThreadInfo is a pointer to the stThreadInfo pointer. You likely want to remove the &
And then, also change IMAGETHREADINFO *pImageThreadInfo = (IMAGETHREADINFO*)&args;, remove the &