I get a data structure like this:
struct My_data
{
MyArray<float> points;
MyArray<float> normals;
MyArray<float> uvCoords;
};
This function can be used to free them:
void ClearAlembicData(My_data* myData)
{
myData->points.clear();
myData->normals.clear();
myData->uvCoords.clear();
}
I want to asynchronously clean the myData so that the program will not wait util all the xxx.clear() are done. Here is my actual code:
My_data myData;
myData.point.push_back(point);
myData.nomals.push_back(nomals);
myData.uvCorrds.push_back(uvCorrds);
ClearAlembicData(&myData);
myData.point.push_back(point);
myData.nomals.push_back(nomals);
myData.uvCorrds.push_back(uvCorrds);
Could you show me how to do it in C++? thanks
Depending on the definition of MyArray, this will either cause undefined behavior or be completely pointless.
If the container is thread-safe, it will block your push_back while the clear is being executed in the other thread (making the 'clearing it asynchronously' completely pointless), if it is not, you are introducing a race condition, which might even end up crashing your program, because you would be concurrently manipulating a shared resource. (Not a good idea).
If you still want to do it, here is a way that might work:
Make 'myData' into a pointer and operate on that.
Upon wanting to clear, store that pointer in another pointer variable and replace the pointer with new My_Data.
Pass the stored pointer to your asynchronous 'free' function.
Continue to work with your new data structure in the original thread.
This way, you are not working on a shared resource and asynchronous freeing becomes feasible.
As for 'how to', if you've got C++11, something like this would work (Pseudo-Code).
My_data *myData = new My_data();
myData->point.push_back(point);
myData->nomals.push_back(nomals);
myData->uvCorrds.push_back(uvCorrds);
std::thread([=](){ ClearAlembicData(myData); delete myData; }).detach();
myData = new My_data()
myData->point.push_back(point);
myData->nomals.push_back(nomals);
myData->uvCorrds.push_back(uvCorrds);
delete myData;
One of the solutions, implement MyArray::swap(MyArray&). Then
void ClearAlembicData(My_data* myData)
{
MyArray<float> old_points;
MyArray<float> old_normal;
MyArray<float> old_coords;
// Fast swap, myData arrays become empty
myData->points.swap(old_points);
myData->normals.clear(old_normals);
myData->uvCoords.clear(old_coords);
// Assumed to be passed to async function
old_points.clear();
old_normals.clear();
old_coords.clear();
}
There is no enough information about MyArray abilities, such as support of move semantics, a real function clear, thus this is just an idea.
If MyArray is a wrapper around std::vector, know that the clear() operation is O(1) as float is trivially destructible and the compiler can eliminate the loop that destroys elements individually.
If it is not, consider implementing MyArray.clear() such that it marks the container as having a non-zero capacity yet zero contents. As stated above, std::vector accomplishes this by simply setting slots_available = capacity.
Related
This problem has bugged me for several days now, and I just cant figure it out. What I am trying to do is get an Entity from the entityMap and make a copy of it. An Entity is essentialy a map of components, so I looped through each component and made a copy of it. I debugged the program, and it worked fine until the very last line, where it said "read access violation this was 0xFFFFFFFFFFFFFFF7." It was very strange as everything was initialized (I checked the debugger)
if (entityMap.find(classname) != entityMap.end()) {
std::shared_ptr<Entity> & prefab = entityMap[classname];
std::shared_ptr<Entity> entity = std::shared_ptr<Entity>(new Entity());
for (auto & component : prefab->GetComponentMap()) {
Component * compPtr = component.second.get();
std::cout << compPtr->GetMemorySize() << "\n";
size_t size = sizeof(compPtr->GetMemorySize());
void * buffer = operator new(size);
memcpy(buffer, compPtr, size);
std::shared_ptr<Component> newComponent = std::shared_ptr<Component>(reinterpret_cast<Component *>(buffer));
entity->AddComponent(newComponent);
newComponent->SetOwner(entity);
}
Here is the offending line
newComponent->SetOwner(entity);
Here is all it does, setting the owner instance variable to the passed in parameter. That was where the debugger complained and sent me to file "memory" at _Decref method.
void Component::SetOwner(std::shared_ptr<Entity> owner) {
this->owner = owner;
}
The problem here is that you can’t copy objects just by copying the memory. For basic plain data objects without any constructors, destructors, or pointers this may work but for anything more complex it most likely won’t.
For example, if the object contains pointers to data and these are released in a destructor then the data is not deep copied, rather the pointer is, and you get double free and also possible pointers to unallocated memory. If the object relies on something being done in the constructor it is never done when copying memory. And depending on how the size is calculated it may not even be a complete copy.
This is why you should always provide a cloning mechanism in the class that takes care of these issues in a way that suits the object and makes sure there’s proper deep/shallow copying depending on the contents.
I'm currently developing application using gSoap library and has some misunderstanding of proper usage library. I has generated proxy object (-j flag) which wrapped my own classes, as you can see below. Application must work 24/7 and connect simultaneously to many cameras (~50 cameras), so after every request i need to clear all temporary data. Is it normal usage to call soap_destroy() and soap_end() after every request? Because it seem's overkill to do it after each request. May be exists another option of proper usage?
DeviceBindingProxy::destroy()
{
soap_destroy(this->soap);
soap_end(this->soap);
}
class OnvifDeviceService : public Domain::IDeviceService
{
public:
OnvifDeviceService()
: m_deviceProxy(new DeviceBindingProxy)
{
soap_register_plugin(m_deviceProxy->soap, soap_wsse);
}
int OnvifDeviceService::getDeviceInformation(const Access::Domain::Endpoint &endpoint, Domain::DeviceInformation *information)
{
_tds__GetDeviceInformation tds__GetDeviceInformation;
_tds__GetDeviceInformationResponse tds__GetDeviceInformationResponse;
setupUserPasswordToProxy(endpoint);
m_deviceProxy->soap_endpoint = endpoint.endpoint().c_str();
int result = m_deviceProxy->GetDeviceInformation(&tds__GetDeviceInformation, tds__GetDeviceInformationResponse);
m_deviceProxy->soap_endpoint = NULL;
if (result != SOAP_OK) {
Common::Infrastructure::printSoapError("Fail to get device information.", m_deviceProxy->soap);
m_deviceProxy->destroy();
return -1;
}
*information = Domain::DeviceInformation(tds__GetDeviceInformationResponse.Manufacturer,
tds__GetDeviceInformationResponse.Model,
tds__GetDeviceInformationResponse.FirmwareVersion);
m_deviceProxy->destroy();
return 0;
}
}
To ensure proper allocation and deallocation of managed data:
soap_destroy(soap);
soap_end(soap);
You want to do this often to avoid memory to fill up with old data. These calls remove all deserialized data and data you allocated with the soap_new_X() and soap_malloc() functions.
All managed allocations are deleted with soap_destroy() followed by soap_end(). After that, you can start allocating again and delete again, etc.
To allocate managed data:
SomeClass *obj = soap_new_SomeClass(soap);
You can use soap_malloc for raw managed allocation, or to allocate an array of pointers, or a C string:
const char *s = soap_malloc(soap, 100);
Remember that malloc is not safe in C++. Better is to allocate std::string with:
std::string *s = soap_new_std__string(soap);
Arrays can be allocated with the second parameter, e.g. an array of 10 strings:
std::string *s = soap_new_std__string(soap, 10);
If you want to preserve data that otherwise gets deleted with these calls, use:
soap_unlink(soap, obj);
Now obj can be removed later with delete obj. But be aware that all pointer members in obj that point to managed data have become invalid after soap_destroy() and soap_end(). So you may have to invoke soap_unlink() on these members or risk dangling pointers.
A new cool feature of gSOAP is to generate deep copy and delete function for any data structures automatically, which saves a HUGE amount of coding time:
SomeClass *otherobj = soap_dup_SomeClass(NULL, obj);
This duplicates obj to unmanaged heap space. This is a deep copy that checks for cycles in the object graph and removes such cycles to avoid deletion issues. You can also duplicate the whole (cyclic) managed object to another context by using soap instead of NULL for the first argument of soap_dup_SomeClass.
To deep delete:
soap_del_SomeClass(obj);
This deletes obj but also the data pointed to by its members, and so on.
To use the soap_dup_X and soap_del_X functions use soapcpp2 with options -Ec and -Ed, respectively.
In principle, static and stack-allocated data can be serialized just as well. But consider using the managed heap instead.
See https://www.genivia.com/doc/databinding/html/index.html#memory2 for more details and examples.
Hope this helps.
The way memory has to be handled is described in Section 9.3 of the GSoap documentation.
I have a function that stores the value of an argument to an std::vector<v8::Local<v8::Value>> property of a C++ class exposes as an ObjectWrap like this:
NAN_METHOD(MyObject::Write) {
MyObject* obj = Nan::ObjectWrap::Unwrap<MyObject>(info.This());
obj->data.push_back(info[0]);
}
However, when I try to read back the value from another C++ function, the value is lost, and becomes undefined.
I'm passing a number to MyObject::Write, and I can confirm info[0]->IsNumber() returns true before pushing it to the vector, however when reading it back, the value it not a number, and in fact returns false for all the types I tested using Is<Type> methods from v8::Value, but still returns true for BooleanValue().
My guess is that the variable is being garbage collected after MyObject::Write returns, however I have no idea how to prevent this from happening.
I'm currently trying to initialise the value as a Persistent value. I tried the following attempts without success:
Nan::CopyablePersistentTraits<v8::Value>::CopyablePersistent p;
Nan::Persistent<v8::Value> persistent(info[0]);
Nan::CopyablePersistentTraits::Copy(persistent, p);
And:
v8::Isolate *isolate = info.GetIsolate();
v8::Persistent<v8::Value, v8::CopyablePersistentTraits<v8::Value>> persistent(isolate, info[0]);
But getting tons of C++ errors.
I was running into problems untangling this mess myself. There's a lot of template stuff going on here that we both missed. Here was the solution I found most readable:
// Define the copyable persistent
v8::CopyablePersistentTraits<v8::Value>::CopyablePersistent p;
// Create the local value
auto val = v8::Local<v8::Value>::New(
v8::Isolate::GetCurrent(), //< Isolate required
v8::Integer::New(v8::Isolate::GetCurrent(), v) //< Isolate required
);
// Reset() is a TEMPLATE FUNCTION, you have to template it with the same
// template type parameter as the v8::Local you are passing
p.Reset<v8::Value>(v8::Isolate::GetCurrent(), val); //< Isolate required
By "info" I assume you are referring to a v8::FunctionCallbackInfo reference. If so the above code would collapse to the following:
void SomeFunc(v8::FunctionCallbackInfo<v8::Value>& info) {
v8::CopyablePersistentTraits<v8::Value>::CopyablePersistent p;
p.Reset<v8::Value>(info[0]);
}
Because the persistent is now copyable you can do things like store it inside a standard library container. This was my use case. This is an example of storing a value in a vector:
std::vector<v8::CopyablePersistentTraits<v8::Value>::CopyablePersistent> vect;
void AccumulateData(v8::FunctionCallbackInfo<v8::Value>& info) {
v8::CopyablePersistentTraits<v8::Value>::CopyablePersistent p;
p.Reset<v8::Value>(info[0]);
vect.push_back(p);
}
I hope this helps someone out there.
If you plan on storing v8 values in C++, you need to make them persistent instead of local so they're independent of handle scope and not garbage-collected when the handle scope is released.
Nan has version-independant wrappers for v8::Persistent and Co. Because of using inside std::vector<>, you'll also need to initialize Nan::Persistent with Nan::CopyablePersistentTraits so it becomes copyable (or make an own reference-counted container for it).
I have a simple recursive-type container object "Level" (such as a directory, which can contain multiples of itself), although I'm not sure that's related to this problem.
//Level.h
class Level
{
public:
Level();
vector<Level*> SubLevels;
Level CreateSubLevel();
}
//Level.cpp
Level::Level()
{
SubLevels = vector<Level*>();
}
Level Level::CreateSubLevel()
{
Level NewLevel = Level();
SubLevels.push_back(&NewLevel);
return NewLevel;
}
If then in my main loop I call
//main.cpp
Level MasterLevel = Level();
MasterLevel.CreateSubLevel();
MasterLevel.CreateSubLevel();
MasterLevel.CreateSubLevel();
I find that indeed the vector MasterLevel.SubLevels contains three pointers to Level objects. However, they are all pointers to the same address!
I'm not sure why this is happening. My memory management skills are lacking - but I'm suspecting that it's because every time CreateSubLevel() is called, a new object is created, but then it is deleted when CreateSubLevel() exits? I thought that ARC would keep track of the fact that the pointer to it still exists, but perhaps I'm mistaken? Or is it another issue entirely?
How can I best fix this problem?
Thanks!
SubLevels is holding onto three pointers to temporaries. It's not a surprise that the compiler chose to reuse the same memory for the temporary each time - why not?
If you want to actually store three different Levels correctly, you will either have to store them by value:
vector<Level> SubLevels;
SubLevels.push_back(Level());
Or actually allocate Levels:
vector<Level*> SubLevels;
SubLevels.push_back(new Level); // don't forget to delete!
The reason you come up with the same value every time is because you are using the address of a temporary variable (on the stack). Every time the function CreateSubLevel() is called, the stack is reused, thus the objects are stored in the same location every call.
You can allocate objects on the heap using operator new():
vector<Level*> SubLevels;
SubLevels.push_back(new Level);
Then you can delete them in a destructor:
Level::~Level()
{
vector<Level*>::iterator i;
for (i = SubLevels.begin(); i != SubLevels.end(); ++i)
delete *i;
}
You have three calls to MasterLevel.CreateSubLevel(); one after the other. Each call creates a stack frame that is of the same size. Hence, the address of the local variable is the same. You are storing the address of the local variable in SubLevels.
If you use the address stored in SubLevels, you will run into undefined behavior. You need to allocate memory from heap.
While you are at it, keep a list of smart pointers, std::unique_ptr or std::shared_ptr instead of storing raw pointers.
Use
vector<std::shared_ptr<Level>> SubLevels;
and use it as:
void Level::CreateSubLevel()
{
SubLevels.push_back(std::make_shared<Level>());
}
I have a code that has a large number of mallocs and device-specific API mallocs (I'm programming on a GPU, so cudaMalloc).
Basically my end of my beginning of my code is a big smorgasbord of allocation calls, while my closing section is deallocation calls.
As I've encapsulated my global data in structures, the deallocations are quite long, but at least I can break them into a separate function. On the other hand, I would like a shorter solution. Additionally an automatic deallocator would reduce the risk of memory leaks created if I forget to explicitly write the deallocation in the global allocator function.
I was wondering whether it'd be possible to write some sort of templated class wrapper that can allow me to "register" variables during the malloc/cudaMalloc process, and then at the end of simulation do a mass loop-based deallocation (deregistration). To be clear I don't want to type out individual deallocations (free/cudaFrees), because again this is long and undesirable, and the assumption would be that anything I register won't be deallocated until the device simulation is complete and main is terminating.
A benefit here is that if I register a new simulation duration variable, it will automatically deallocate, so there's no danger of me forgetting do deallocate it and creating a memory leak.
Is such a wrapper possible?
Would you suggest doing it?
If so, how?
Thanks in advance!
An idea:
Create both functions, one that allocates memory and provides valid pointers after register them in a "list" of allocated pointers. In the second method, loop this list and deallocate all pointers:
// ask for new allocated pointer that will be registered automatically in list of pointers.
pointer1 = allocatePointer(size, listOfPointers);
pointer2 = allocatePointer(size, listOfPointers);
...
// deallocate all pointers
deallocatePointers(listOfPointers);
Even, you may use different listOfPointers depending of your simulation scope:
listOfPointer1 = getNewListOfPointers();
listOfPointer2 = getNewListOfPointers();
....
p1 = allocatePointer(size, listOfPointer1);
p2 = allocatePointer(size, listOfPointer2);
...
deallocatePointers(listOfPointers1);
...
deallocatePointers(listOfPointers2);
There are many ways to skin a cat, as they say.
I would recommend thrust's device_vector as a memory management tool. It abstracts allocation, deallocation, and memcpy in CUDA. It also gives you access to all the algorithms that Thrust provides.
I wouldn't recommend keeping random lists of unrelated pointers as Tio Pepe recommends. Instead you should encapsulate related data into a class. Even if you use thrust::device_vector you may want to encapsulate multiple related vectors and operations on them into a class.
The best choice is probably to use the smart pointers from C++ boost library, if that is an option.
If not, the best you can hope for in C is a program design that allows you to write allocation and deallocation in one place. Perhaps something like the following pseudo code:
while(!terminate_program)
{
switch(state_machine)
{
case STATE_PREOPERATIONAL:
myclass_init(); // only necessary for non-global/static objects
myclass_mem_manager();
state_machine = STATE_RUNNING;
break;
case STATE_RUNNING:
myclass_do_stuff();
...
break;
...
case STATE_EXIT:
myclass_mem_manager();
terminate_program = true;
break;
}
void myclass_init()
{
ptr_x = NULL;
ptr_y = NULL;
/* Where ptr_x, ptr_y are some of the many objects to allocate/deallocate.
If ptr is a global/static, (static storage duration) it is
already set to NULL automatically and this function isn't
necessary */
}
void myclass_mem_manager()
{
ptr_x = mem_manage (ptr_x, items_x*sizeof(Type_x));
ptr_y = mem_manage (ptr_y, items_y*sizeof(Type_y));
}
static void* mem_manage (const void* ptr, size_t bytes_n)
{
if(ptr == NULL)
{
ptr = malloc(bytes_n);
if (ptr == NULL)
{} // error handling
}
else
{
free(ptr);
ptr = NULL;
}
return ptr;
}