How to controll shared ptr reference count? - c++

I'm creating a Resource Manager for my game engine. Basically it have an unordered_map that stores the path of the resource as key and as value it stores the loaded resource.
This is the code responsible for loading assets.
std::shared_ptr<T> LoadAsset(const String& _path)
{
if (auto asset = m_assets.find(_path); asset != m_assets.end()) {
return std::static_pointer_cast<T>(asset->second);
}
auto asset = std::move(LoadAssetInternal(_path));
if (!asset) {
return nullptr;
}
m_assets[_path] = std::move(asset);
return std::static_pointer_cast<T>(m_assets[_path]);
}
The problem is that when I call the LoadAsset method, the returned shared_ptr variable always has 2 strong ref when I delete the variable holding the resource the ref count goes to 1 and the resource is never freed by the end of the program.
Exemple:
auto tex = LoadAsset<Texture>("Data/Textures/Foo.tga"); // Strong refs = 2
tex = nullptr; // Strong refs = 1 and the loaded Foo.tga is never freed.

Just create a function that runs at the end of your games main loop. Something like this
void PurgeAssets() {
for (auto i = m_assets.begin(); i != m_assets.end();) {
if (i->second.unique()) {
i = m_assets.erase(i);
}
else {
++i;
}
}
}

Related

Copy only necessary objects from PDF file

I've got a huge PDF file with more than 100 pages and I want to separate them to single PDF files (containing only one page each). Problem is, that PoDoFo does not copy just the page, but the whole document because of the references (and so each of the 100 PDF files have same size as the 100-page PDF). A relevant mailing list post can be found, unfortunately there is no solution provided.
In source code of function InsertPages there is explanation:
This function works a bit different than one might expect.
Rather than copying one page at a time - we copy the ENTIRE document
and then delete the pages we aren't interested in.
We do this because
1) SIGNIFICANTLY simplifies the process
2) Guarantees that shared objects aren't copied multiple times
3) offers MUCH faster performance for the common cases
HOWEVER: because PoDoFo doesn't currently do any sort of "object
garbage collection" during a Write() - we will end up with larger
documents, since the data from unused pages will also be in there.
I have tried few methods to copy only relevant objects, but each of them failed.
Copy all pages and remove irrelevant ones
Use XObject wrapping: FillXObjectFromDocumentPage and FillXObjectFromExistingPage
Copy object by object
Use RenumberObjects with bDoGarbageCollection = true
but none of them worked out. Does anybody have an idea or working solution for this problem?
The only solution is to use another PDF library. Or wait for garbage collection to be implemented.
The problem is stated in the quote you mentioned:
> during a Write() - we will end up with larger documents, since the
> data from unused pages will also be in there.
This means podofo always puts the entire PDF content in your file, no matter what. The whole PDF is there, you just don't see parts of it.
Dennis from the podofo support sent me an working example of optimized version of InsertPages function which is actually fixing page references and decreases document size significantly!
void PdfMemDocument::InsertPages2(const PdfMemDocument & rDoc, std::vector<int> pageNumbers)
{
std::unordered_set<PdfObject*> totalSet;
std::vector<pdf_objnum> oldObjNumPages;
std::unordered_map<pdf_objnum, pdf_objnum> oldObjNumToNewObjNum;
std::vector<PdfObject*> newPageObjects;
// Collect all dependencies from all pages that are to be copied
for (int i = 0; i < pageNumbers.size(); ++i) {
PdfPage* page = rDoc.GetPage(pageNumbers[i]);
if (page) {
oldObjNumPages.push_back(page->GetObject()->Reference().ObjectNumber());
std::unordered_set<PdfObject*> *set = page->GetPageDependencies();
totalSet.insert(set->begin(), set->end());
delete set;
}
}
// Create a new page object for every copied page from the old document
// Copy all objects the pages depend on to the new document
for (auto it = totalSet.begin(); it != totalSet.end(); ++it) {
unsigned int length = static_cast<unsigned int>(GetObjects().GetSize() + GetObjects().GetFreeObjects().size());
PdfReference ref(static_cast<unsigned int>(length+1), 0);
PdfObject* pObj = new PdfObject(ref, *(*it));
pObj->SetOwner(&(GetObjects()));
if ((*it)->HasStream()) {
PdfStream *stream = (*it)->GetStream();
pdf_long length;
char* buf;
stream->GetCopy(&buf, &length);
PdfMemoryInputStream inputStream(buf, length);
pObj->GetStream()->SetRawData(&inputStream, length);
free(buf);
}
oldObjNumToNewObjNum.insert(std::pair<pdf_objnum, pdf_objnum>((*it)->Reference().ObjectNumber(), length+1));
GetObjects().push_back(pObj);
newPageObjects.push_back(pObj);
}
// In all copied objects, fix the object numbers so they are valid in the new document
for (auto it = newPageObjects.begin(); it != newPageObjects.end(); ++it) {
FixPageReferences(GetObjects(), *it, oldObjNumToNewObjNum);
}
// Insert the copied pages into the pages tree
for (auto it = oldObjNumPages.begin(); it != oldObjNumPages.end(); ++it) {
PdfObject* pageObject = GetObjects().GetObject(PdfReference(oldObjNumToNewObjNum[(*it)], 0));
PdfPage *page = new PdfPage(pageObject, std::deque<PdfObject*>());
GetPagesTree()->InsertPage(GetPageCount() - 1, page);
}
}
std::unordered_set<PdfObject *>* PdfPage::GetPageDependencies() const
{
std::unordered_set<PdfObject *> *set = new std::unordered_set<PdfObject *>();
const PdfObject* pageObj = GetObject();
if (pageObj) {
PdfVecObjects* objects = pageObj->GetOwner();
if (objects) {
set->insert((PdfObject*)pageObj);
objects->GetObjectDependencies2(pageObj, *set);
}
}
return set;
}
// Optimized version of PdfVecObjects::GetObjectDependencies
void PdfVecObjects::GetObjectDependencies2(const PdfObject* pObj, std::unordered_set<PdfObject*> &refMap) const
{
// Check objects referenced from this object
if (pObj->IsReference())
{
PdfObject* referencedObject = GetObject(pObj->GetReference());
if (referencedObject != NULL && refMap.count(referencedObject) < 1) {
(refMap).insert((PdfObject *)referencedObject); // Insert referenced object
GetObjectDependencies2((const PdfObject*)referencedObject, refMap);
}
}
else {
// Recursion
if (pObj->IsArray())
{
PdfArray::const_iterator itArray = pObj->GetArray().begin();
while (itArray != pObj->GetArray().end())
{
GetObjectDependencies2(&(*itArray), refMap);
++itArray;
}
}
else if (pObj->IsDictionary())
{
TCIKeyMap itKeys = pObj->GetDictionary().GetKeys().begin();
while (itKeys != pObj->GetDictionary().GetKeys().end())
{
if ((*itKeys).first != PdfName("Parent")) {
GetObjectDependencies2((*itKeys).second, refMap);
}
++itKeys;
}
}
}
}
void FixPageReferences(PdfVecObjects& objects, PdfObject* pObject, std::unordered_map<pdf_objnum, pdf_objnum>& oldNumToNewNum) {
if( !pObject)
{
PODOFO_RAISE_ERROR( ePdfError_InvalidHandle );
}
if( pObject->IsDictionary() )
{
TKeyMap::iterator it = pObject->GetDictionary().GetKeys().begin();
while( it != pObject->GetDictionary().GetKeys().end() )
{
if ((*it).first != PdfName("Parent")) {
FixPageReferences(objects, (*it).second, oldNumToNewNum);
}
++it;
}
}
else if( pObject->IsArray() )
{
PdfArray::iterator it = pObject->GetArray().begin();
while( it != pObject->GetArray().end() )
{
FixPageReferences(objects, &(*it), oldNumToNewNum),
++it;
}
}
else if( pObject->IsReference() )
{
//PdfObject* referencedObj = objects.GetObject(pObject->GetReference());
pdf_objnum oldnum = pObject->GetReference().ObjectNumber();
pdf_objnum newnum = oldNumToNewNum[oldnum];
if (!newnum) throw new std::exception("No new object number for old object number");
*pObject = PdfReference(newnum, 0);
}
}

winrt c++/cx concurrency access violation exception

What I'm trying to do is check for the existence of a file in the local folder and then copy it there if it isn't found (the file was previously added to the project as an asset).
Here is the code:
Windows::Storage::StorageFile^ MainPage::GetCustomFileAsync(Platform::String^ fileName)
{
using Windows::Storage::StorageFile;
using Windows::Storage::StorageFolder;
auto localFolder = Windows::Storage::ApplicationData::Current->LocalFolder;
auto localTask = concurrency::create_task(localFolder->GetFileAsync(fileName));
StorageFile^ retVal = nullptr;
localTask.then([&](StorageFile^ t){
retVal = t;
}).then([](concurrency::task<void> t)
{
try
{
t.get();
OutputDebugString(L"Found\n");
}
catch (Platform::COMException^ e)
{
OutputDebugString(e->Message->Data());
}
}).wait();
return retVal;
}
StorageFile^ fileVar;
if ((fileVar = this->GetCustomFileAsync("somefile.txt")) == nullptr)
{
String^ path = Windows::ApplicationModel::Package::Current->InstalledLocation->Path + "\\Assets";
concurrency::create_task(Windows::Storage::StorageFolder::GetFolderFromPathAsync(path)).then([](StorageFolder^ folder){
return (folder->GetFileAsync("somefile.txt"));
}).then([](StorageFile^ file){
return (file->CopyAsync(Windows::Storage::ApplicationData::Current->LocalFolder));
}).then([&](StorageFile^ file){
fileVar = file;
OutputDebugString(file->DisplayName->Data());
});
}
What happens is that I get an access violation exception at the point where "file" is being assigned to "fileVar" (because of cross-thread access perhaps?). How to fix this?
Edit: I can't do all the processing there because the file will be accessed many times. In short I need to know when it has been successfully copied and get a handle to it. Here is the code that works
Windows::Storage::StorageFile^ GetFile(Platform::String^ fileName)
{
using Windows::Storage::StorageFile;
using Windows::Storage::StorageFolder;
using Windows::Foundation::AsyncOperationCompletedHandler;
using Windows::Foundation::AsyncStatus;
using Windows::Foundation::IAsyncOperation;
using Platform::String;
auto localFolder = Windows::Storage::ApplicationData::Current->LocalFolder;
bool completed = false;
StorageFile^ retVal = nullptr;
localFolder->GetFileAsync(fileName)->Completed = ref new AsyncOperationCompletedHandler<StorageFile^>([&completed, &retVal, &fileName](IAsyncOperation<StorageFile^>^ fileOperation, AsyncStatus status)
{
if (status == AsyncStatus::Error)
{
String^ path = Windows::ApplicationModel::Package::Current->InstalledLocation->Path + "\\Assets";
Windows::Storage::StorageFolder::GetFolderFromPathAsync(path)->Completed = ref new AsyncOperationCompletedHandler<Windows::Storage::StorageFolder^>(
[&completed, &retVal, &fileName](IAsyncOperation<Windows::Storage::StorageFolder^>^ folderOperation, AsyncStatus status)->void{
auto assetFolder = folderOperation->GetResults();
assetFolder->GetFileAsync(fileName)->Completed = ref new AsyncOperationCompletedHandler<Windows::Storage::StorageFile^>([&completed, &retVal, &fileName](IAsyncOperation<Windows::Storage::StorageFile^>^ fileOperation, AsyncStatus status)->void{
auto file = fileOperation->GetResults();
file->CopyAsync(Windows::Storage::ApplicationData::Current->LocalFolder)->Completed = ref new AsyncOperationCompletedHandler<Windows::Storage::StorageFile^>
([&completed, &retVal, &fileName](IAsyncOperation<Windows::Storage::StorageFile^>^ fileOperation, AsyncStatus status)->void {
retVal = fileOperation->GetResults();
completed = true;
});
});
});
}
else
{
retVal = fileOperation->GetResults();
completed = true;
}
});
while (completed == false);
return retVal;
}
Rather than passing a delegate as an argument and returning void, make your method return task<StorageFile^> and then the caller can do a .then() to continue working once the operation has succeeded.
Or if this is exposed as a public WinRT method (not an internal / private C++ method) then use IAsyncOperation<StorageFile^>^ as the return type, and wrap the whole thing in create_async():
IAsyncOperation<StorageFile^>^ DoStuff(params)
{
return concurrency::create_async([params]
{
// function body goes here
});
}
Here's a solution I put together. Two things that are important to know:
When executing an asynchronous operation using concurrency::create_task the async operation(s) can still be executing when the parent function returns. So the captured variables MUST outlive the context of the parent function. Which obviously won't happen if they are being passed by reference. It took a while to realize this.
WinRT imposes certain restrictions on the concurrency runtime. Calling concurrency::task::get() or concurrency::task::wait() will throw an exception in an STA thread, unless the call is in a task continuation.
More information in this post:
http://social.msdn.microsoft.com/Forums/windowsapps/en-US/ae54980b-41ce-4337-a059-2213b549be4b/concurrencyinvalidoperation-when-calling-tasktget?forum=winappswithnativecode
In that case how to know when the function has finished doing it's job? I opted to pass in a callback (AKA delegate).
delegate void FileOperation(Windows::Storage::StorageFile^ file);
void GetFileConcurrency(Platform::String^ fileName, FileOperation^ fileOp)
{
using Windows::Storage::StorageFile;
using Windows::Storage::StorageFolder;
using Platform::String;
auto localFolder = Windows::Storage::ApplicationData::Current->LocalFolder;
String^ assetFolderPath = Windows::ApplicationModel::Package::Current->InstalledLocation->Path + "\\Assets";
auto localFolderTask = concurrency::create_task(localFolder->GetFileAsync(fileName));
localFolderTask.then([localFolder, assetFolderPath, fileName, fileOp](concurrency::task<StorageFile^> theTask){
try
{
StorageFile^ theFile = theTask.get();
fileOp(theFile);
}
catch (Platform::Exception^ e)
{
OutputDebugString(e->Message->Data());
auto assetFolderTask = concurrency::create_task(StorageFolder::GetFolderFromPathAsync(assetFolderPath));
assetFolderTask.then([localFolder, assetFolderPath, fileName, fileOp](StorageFolder^ assetFolder){
auto assetFileTask = concurrency::create_task(assetFolder->GetFileAsync(fileName));
assetFileTask.then([localFolder, assetFolderPath, fileName, fileOp](StorageFile^ file){
auto copyFileTask = concurrency::create_task(file->CopyAsync(localFolder));
copyFileTask.then([localFolder, assetFolderPath, fileName, fileOp](StorageFile^ file){
OutputDebugString(file->Path->Data());
fileOp(file);
});
});
});
}
});
}

Memory leak in CLucene

I am using CLucene for creating indexing and searching. The index files created are of more than 5 GB. I have Created the seperate dll for CLucene search.
DLL constructor contains the following code
lucene::index::IndexReader ptrIndexReader;
lucene::search::IndexSearcher searcher; these are defined in class decalaration/
ptrIndexReader = IndexReader::open(pDir.c_str(),false,NULL);
searcher = new IndexSearcher(ptrIndexReader);
I use one search function whose code is as follows
bool LuceneWrapper::SearchIndex(wstring somevalue)
{
lucene::analysis::KeywordAnalyzer fAnalyzer;
Document doc = NULL;
Hits hits = NULL;
Query f_objQuery = NULL;
NistRecord *f_objRecords = NULL;
bool flag = false;
try{
if (ptrIndexReader == NULL)
{
return NULL;
}
// Initialize IndexSearcher
wstring strQuery = _T("+somename:") + somevalue;
// Initialize Query Parser, with Keyword Analyzer
QueryParser *parser = new QueryParser( _T(""),&fAnalyzer);
// Parse Query string
f_objQuery = parser->parse(strQuery.c_str());
// Search Index directory
hits = searcher->search(f_objQuery);
//searcher.
int intHitCount = 0;
intHitCount = hits->length;
if(intHitCount > 0)
{
if(doc!=NULL)
delete [] doc;
flag = true;
}
//searcher.close();
}
catch(CLuceneError& objExp)
{
if(doc!=NULL)
delete doc;
return false;
}
if(hits!=NULL)
delete hits;
if(f_objQuery!=NULL)
delete f_objQuery;
return flag ;
}
I am searching very large number of values. according to the record count the main memory goes high and high and at on level it goes near to 2 GB and application crashes. Can anybody tell me what is wrong with this? Why is memory going so high and application crashing?
You never deallocate parser.
I can't see a reason to allocate it dynamically.
Why don't you just say
QueryParser parser( _T(""), &fAnalyzer);
f_objQuery = parser.parse(strQuery.c_str());
You also need to make sure that you delete both f_objQuery and hits in the event of an exception.
std::unique_ptr can help you here, if you have access to it.
(And you don't have to test for NULL so much - it's OK to delete a null pointer.)

Creation of thread with an dynamic argument

I am creating a thread from main thread with an dynamic allocated object as an argument. But if we are deleting this dynamic memory allocated object in main thread then how can we find out a object is deleted in created thread.
main thread code :
int CLocalReader::Run()
{
TReaderArgument *readerArg = new TReaderArgument;
readerArg->iFinished = &theFinishedACE;
readerArg->iSelf = this;
#ifdef WIN32
if (ACE_Thread::spawn((ACE_THR_FUNC)LocalReaderFunc, readerArg) == -1)
{
ACE_DEBUG((LM_DEBUG,"Could not start reader\n"));
delete readerArg;
readerArg = NULL;
}
#else
if (ACE_Thread_Manager::instance()->spawn(ACE_THR_FUNC (LocalReaderFunc), readerArg, THR_NEW_LWP | THR_DETACHED) < 0)
{
ACE_DEBUG((LM_DEBUG,"Could not start reader\n"));
delete readerArg;
readerArg = NULL;
}
#endif
return KErrNone;
}
static void *ReaderFunc(void *arg)
{
ASSERT(arg);
ACE_Thread::yield();
ACE_OS::sleep(ACE_Time_Value(0, STARTUP_TIME));
TReaderArgument *rarg = (TReaderArgument *)arg;
CLocalReader *self = static_cast<CLocalReader *>(rarg->iSelf);
int *finished = rarg->iFinished;
while (!(*finished))
{
if( self->GetData() != KErrorNone )
{
ACE_DEBUG((LM_DEBUG, "%D LocalReader : Error receiving data\n"));
}
}
return 0;
}
if in above code , this object is deleted then how we can check in thread function self object is deleted
Use reference counting like in COM. When the main thread is done with the object, it can set a flag "deleted" and release the object. The object will not be deleted yet, because the thread is still holding the reference count. But, the thread can check the flag and release the object if the flag is set. The reference count will drop to 0 and the object will commit suicide by calling delete this;

Member variables of a object get overridden when creating another object in the object

I have a memory issue with a class of mine. The issue occurs when I create an object in a member function of a class. It is about the class below. I removed the member functions because they aren’t necessary:
class User
{
private:
bool locked;
bool active;
std::vector<City> * userCitys;
UserData userData;
Credentials credentials;
The problem occurs when I call this function:
int User::addCity(CityData cityData)
{
lockUserObject(); //Everything is fine here
City cityToAdd; //When this object is created, the memory of userCitys will get overridden
cityToAdd.activate();
userCitys->push_back(cityToAdd);
int cityID = userCitys->size() - 1;
userCitys->at(cityID).editCityData(cityData);
unlockUserObject();
return cityID;
}
In the first place I created userCitys on the stack. For test purpose I placed it on the Heap. The address of userCitys get overridden by some data. I can’t find the problem. the City is just a basic class:
Part of the header:
class City
{
private:
bool active;
Supplies supplies;
std::vector<Building> buildings;
std::vector<Company> companies;
std::vector<Share> shares;
std::vector<Troop> troops;
CityData cityData;
Constructor:
City::City()
{
active = false;
}
How is it possible that userCitys get overridden? This all happens on a single Thread so that can’t be a problem. I tried a lot of thing, but I can’t get it to work. What is the best approach to find the problem?
Edit:
Lock function:
void User::lockUserObject()
{
for( int i = 0; locked ; i++)
{
crossSleep(Settings::userLockSleepInterval);
if( i >= Settings::userLockMaxTimes )
Error::addError("User lock is over userLockMaxTimes",2);
}
locked = true;
}
I call the code here (Test function):
City * addCity(User * user)
{
Location location;
location.x = 0;
location.y = 1;
CityData citydata;
citydata.location = location;
citydata.villagers = 0;
citydata.cityName = "test city";
int cityID = user->addCity(citydata); //addCity is called here
City * city = user->cityAction(cityID);;
if( city == NULL)
Error::addError("Could not create a city",2);
return city;
}
The add user (Test code):
User * addUser()
{
UserData test;
test.name = "testtest";
Credentials testc("testtest",3);
//Create object user
int userID = UserControle::addUser(test,testc);
User * user = UserControle::UserAction(userID);
if( user == NULL)
Error::addError("Could not create a user",2);
return user;
}
My test function:
void testCode()
{
User * user = addUser();
City * city = addCity(user);
}
This function in called in main:
int main()
{
testCode();
return 0;
}
Here are UserAction and addUser in UserControle:
int UserControle::addUser(UserData userdata, Credentials credentials)
{
int insertID = -1;
for( int i = 0; i < (int)UserControle::users.size(); i++)
{
if( !UserControle::users.at(i).isActive() )
{
insertID = i;
break;
}
}
User userToInsert(userdata,credentials);
if( insertID != -1 )
{
UserControle::users.insert( UserControle::users.begin() + insertID,userToInsert);
return insertID;
}
else
{
UserControle::users.push_back(userToInsert);
return UserControle::users.size() - 1;
}
}
User* UserControle::UserAction(int userID) //check all indexes if greater then 0!
{
if( (int)UserControle::users.size() <= userID )
{
Error::addError("UserAction is out of range",3);
return NULL;
}
if( !UserControle::users.at(userID).isActive())
{
Error::addError("UserAction, the user is not active.",3);
return NULL;
}
return &UserControle::users[userID];
}
There's a few things you could try:
Remove code until the fault goes away. In other words, distill a minimal example from your code. I guess you'll then see the error yourself, otherwise post that small example program here and others will.
Don't use raw pointers. The question with those is always who owns what they point to. Use smart pointers instead, e.g. unique_ptr (C++11) or auto_ptr (C++98) for exclusive ownership.
If you have pointer members like "userCities", you need to think about what happens when copying instances of that class (you already wrote a proper destructor, or?). So, either prevent copying (make copy-constructor and assignment operator private and without implementing it) or implement them in a way that the vectors are properly cloned and not shared between different instances.
Don't use C-style casts. If those are necessary to get anything through the compiler, the code is probably broken.