Proper memory control in gSoap - c++

I'm currently developing application using gSoap library and has some misunderstanding of proper usage library. I has generated proxy object (-j flag) which wrapped my own classes, as you can see below. Application must work 24/7 and connect simultaneously to many cameras (~50 cameras), so after every request i need to clear all temporary data. Is it normal usage to call soap_destroy() and soap_end() after every request? Because it seem's overkill to do it after each request. May be exists another option of proper usage?
DeviceBindingProxy::destroy()
{
soap_destroy(this->soap);
soap_end(this->soap);
}
class OnvifDeviceService : public Domain::IDeviceService
{
public:
OnvifDeviceService()
: m_deviceProxy(new DeviceBindingProxy)
{
soap_register_plugin(m_deviceProxy->soap, soap_wsse);
}
int OnvifDeviceService::getDeviceInformation(const Access::Domain::Endpoint &endpoint, Domain::DeviceInformation *information)
{
_tds__GetDeviceInformation tds__GetDeviceInformation;
_tds__GetDeviceInformationResponse tds__GetDeviceInformationResponse;
setupUserPasswordToProxy(endpoint);
m_deviceProxy->soap_endpoint = endpoint.endpoint().c_str();
int result = m_deviceProxy->GetDeviceInformation(&tds__GetDeviceInformation, tds__GetDeviceInformationResponse);
m_deviceProxy->soap_endpoint = NULL;
if (result != SOAP_OK) {
Common::Infrastructure::printSoapError("Fail to get device information.", m_deviceProxy->soap);
m_deviceProxy->destroy();
return -1;
}
*information = Domain::DeviceInformation(tds__GetDeviceInformationResponse.Manufacturer,
tds__GetDeviceInformationResponse.Model,
tds__GetDeviceInformationResponse.FirmwareVersion);
m_deviceProxy->destroy();
return 0;
}
}

To ensure proper allocation and deallocation of managed data:
soap_destroy(soap);
soap_end(soap);
You want to do this often to avoid memory to fill up with old data. These calls remove all deserialized data and data you allocated with the soap_new_X() and soap_malloc() functions.
All managed allocations are deleted with soap_destroy() followed by soap_end(). After that, you can start allocating again and delete again, etc.
To allocate managed data:
SomeClass *obj = soap_new_SomeClass(soap);
You can use soap_malloc for raw managed allocation, or to allocate an array of pointers, or a C string:
const char *s = soap_malloc(soap, 100);
Remember that malloc is not safe in C++. Better is to allocate std::string with:
std::string *s = soap_new_std__string(soap);
Arrays can be allocated with the second parameter, e.g. an array of 10 strings:
std::string *s = soap_new_std__string(soap, 10);
If you want to preserve data that otherwise gets deleted with these calls, use:
soap_unlink(soap, obj);
Now obj can be removed later with delete obj. But be aware that all pointer members in obj that point to managed data have become invalid after soap_destroy() and soap_end(). So you may have to invoke soap_unlink() on these members or risk dangling pointers.
A new cool feature of gSOAP is to generate deep copy and delete function for any data structures automatically, which saves a HUGE amount of coding time:
SomeClass *otherobj = soap_dup_SomeClass(NULL, obj);
This duplicates obj to unmanaged heap space. This is a deep copy that checks for cycles in the object graph and removes such cycles to avoid deletion issues. You can also duplicate the whole (cyclic) managed object to another context by using soap instead of NULL for the first argument of soap_dup_SomeClass.
To deep delete:
soap_del_SomeClass(obj);
This deletes obj but also the data pointed to by its members, and so on.
To use the soap_dup_X and soap_del_X functions use soapcpp2 with options -Ec and -Ed, respectively.
In principle, static and stack-allocated data can be serialized just as well. But consider using the managed heap instead.
See https://www.genivia.com/doc/databinding/html/index.html#memory2 for more details and examples.
Hope this helps.

The way memory has to be handled is described in Section 9.3 of the GSoap documentation.

Related

How to return base64 decoded image bytes through a methods formal parameter of type char**

Being new to C++, I am still struggling with pointers-to-pointers and I am not sure if my method below is returning decoded image bytes properly.
This method gets a base64 encoded image string from API. The method has to follow this signature as it is part of legacy code that is not allowed to abbreviate from the way it was written originally. So the signature has to stay the same. Also, I have omitted here async calls and continuations, exceptions etc for code simplicity.
int __declspec(dllexport) GetInfoAndPicture(CString uid, char **image, long *imageSize)
{
CString request = "";
request.Format(url);
http_client httpClient(url);
http_request msg(methods::POST);
...
http_response httpResponse;
httpResponse = httpClient.request(msg).get(); //blocking
web::json::value jsonValue = httpResponse.extract_json().get();
if (jsonValue.has_string_field(L"img"))
{
web::json::value base64EncodedImageValue = jsonValue.at(L"img");
utility::string_t imageString = base64EncodedImageValue.as_string();
std::vector<unsigned char> imageBytes = utility::conversions::from_base64(imageString);
image = (char**)&imageBytes; //Is this the way to pass image bytes back?
*imageSize = imageBytes.size();
}
...
}
The caller calls this method like so:
char mUid[64];
char* mImage;
long mImageSize;
...
resultCode = GetInfoAndPicture(mUid, &mImage, &mImageSize);
//process image given its data and its size
I know what pointer to pointer is, my question is specific to this line
image = (char**)&imageBytes;
Is this the correct way to return the image decoded from base64 into the calling code via the char** image formal parameter given the above method signature and method call?
I do get error "Program .... File: minkernel\crts\ucrt\src\appcrt\convert\isctype.cpp ... "Expression c >= -1 && c <= 255"" which I believe is related to the fact that this line is not correctly passing data back.
Give the requirements there isn't any way to avoid allocating more memory and copying the bytes. You cannot use the vector directly because that is local to the GetInfoAndPicture function and will be destroyed when that function exits.
If I understand the API correctly then this is what you need to do
//*image = new char[imageBytes.size()]; //use this if caller calls delete[] to deallocate memory
*image = (char*)malloc(imageBytes.size()); //use this if caller calls free(image) to deallocate memory
std::copy(imageBytes.begin(), imageBytes.end(), *image);
*imageSize = imageBytes.size();
Maybe there is some way in your utility::conversions functions of decoding directly to a character array instead of to a vector, but only you would know about that.
The problem is with allocating (and freeing) memory for that image; who is responsible for that?
You can't (shouldn't) allocate memory in one module and free it in another.
Your two options are:
Allocate large enough buffer on the caller side, and have DLL use it utility::conversions::from_base64(). The issue here is: what is large enough? Some Win APIs provide an additional method to query the required size. Doesn't fit this scenario as the DLL would either have to get that image for the second time, or hold it (indefinitely) until you ask for it.
Allocate required buffer in the DLL and return a pointer to it. You need to ensure that it won't be freed until the caller request to free it (in a separate API).

Read Access Violation From Using Smart Pointers

This problem has bugged me for several days now, and I just cant figure it out. What I am trying to do is get an Entity from the entityMap and make a copy of it. An Entity is essentialy a map of components, so I looped through each component and made a copy of it. I debugged the program, and it worked fine until the very last line, where it said "read access violation this was 0xFFFFFFFFFFFFFFF7." It was very strange as everything was initialized (I checked the debugger)
if (entityMap.find(classname) != entityMap.end()) {
std::shared_ptr<Entity> & prefab = entityMap[classname];
std::shared_ptr<Entity> entity = std::shared_ptr<Entity>(new Entity());
for (auto & component : prefab->GetComponentMap()) {
Component * compPtr = component.second.get();
std::cout << compPtr->GetMemorySize() << "\n";
size_t size = sizeof(compPtr->GetMemorySize());
void * buffer = operator new(size);
memcpy(buffer, compPtr, size);
std::shared_ptr<Component> newComponent = std::shared_ptr<Component>(reinterpret_cast<Component *>(buffer));
entity->AddComponent(newComponent);
newComponent->SetOwner(entity);
}
Here is the offending line
newComponent->SetOwner(entity);
Here is all it does, setting the owner instance variable to the passed in parameter. That was where the debugger complained and sent me to file "memory" at _Decref method.
void Component::SetOwner(std::shared_ptr<Entity> owner) {
this->owner = owner;
}
The problem here is that you can’t copy objects just by copying the memory. For basic plain data objects without any constructors, destructors, or pointers this may work but for anything more complex it most likely won’t.
For example, if the object contains pointers to data and these are released in a destructor then the data is not deep copied, rather the pointer is, and you get double free and also possible pointers to unallocated memory. If the object relies on something being done in the constructor it is never done when copying memory. And depending on how the size is calculated it may not even be a complete copy.
This is why you should always provide a cloning mechanism in the class that takes care of these issues in a way that suits the object and makes sure there’s proper deep/shallow copying depending on the contents.

C++ initializer constantly creating new objects at same memory location

I have a simple recursive-type container object "Level" (such as a directory, which can contain multiples of itself), although I'm not sure that's related to this problem.
//Level.h
class Level
{
public:
Level();
vector<Level*> SubLevels;
Level CreateSubLevel();
}
//Level.cpp
Level::Level()
{
SubLevels = vector<Level*>();
}
Level Level::CreateSubLevel()
{
Level NewLevel = Level();
SubLevels.push_back(&NewLevel);
return NewLevel;
}
If then in my main loop I call
//main.cpp
Level MasterLevel = Level();
MasterLevel.CreateSubLevel();
MasterLevel.CreateSubLevel();
MasterLevel.CreateSubLevel();
I find that indeed the vector MasterLevel.SubLevels contains three pointers to Level objects. However, they are all pointers to the same address!
I'm not sure why this is happening. My memory management skills are lacking - but I'm suspecting that it's because every time CreateSubLevel() is called, a new object is created, but then it is deleted when CreateSubLevel() exits? I thought that ARC would keep track of the fact that the pointer to it still exists, but perhaps I'm mistaken? Or is it another issue entirely?
How can I best fix this problem?
Thanks!
SubLevels is holding onto three pointers to temporaries. It's not a surprise that the compiler chose to reuse the same memory for the temporary each time - why not?
If you want to actually store three different Levels correctly, you will either have to store them by value:
vector<Level> SubLevels;
SubLevels.push_back(Level());
Or actually allocate Levels:
vector<Level*> SubLevels;
SubLevels.push_back(new Level); // don't forget to delete!
The reason you come up with the same value every time is because you are using the address of a temporary variable (on the stack). Every time the function CreateSubLevel() is called, the stack is reused, thus the objects are stored in the same location every call.
You can allocate objects on the heap using operator new():
vector<Level*> SubLevels;
SubLevels.push_back(new Level);
Then you can delete them in a destructor:
Level::~Level()
{
vector<Level*>::iterator i;
for (i = SubLevels.begin(); i != SubLevels.end(); ++i)
delete *i;
}
You have three calls to MasterLevel.CreateSubLevel(); one after the other. Each call creates a stack frame that is of the same size. Hence, the address of the local variable is the same. You are storing the address of the local variable in SubLevels.
If you use the address stored in SubLevels, you will run into undefined behavior. You need to allocate memory from heap.
While you are at it, keep a list of smart pointers, std::unique_ptr or std::shared_ptr instead of storing raw pointers.
Use
vector<std::shared_ptr<Level>> SubLevels;
and use it as:
void Level::CreateSubLevel()
{
SubLevels.push_back(std::make_shared<Level>());
}

Serial allocators/deallocators

I have a code that has a large number of mallocs and device-specific API mallocs (I'm programming on a GPU, so cudaMalloc).
Basically my end of my beginning of my code is a big smorgasbord of allocation calls, while my closing section is deallocation calls.
As I've encapsulated my global data in structures, the deallocations are quite long, but at least I can break them into a separate function. On the other hand, I would like a shorter solution. Additionally an automatic deallocator would reduce the risk of memory leaks created if I forget to explicitly write the deallocation in the global allocator function.
I was wondering whether it'd be possible to write some sort of templated class wrapper that can allow me to "register" variables during the malloc/cudaMalloc process, and then at the end of simulation do a mass loop-based deallocation (deregistration). To be clear I don't want to type out individual deallocations (free/cudaFrees), because again this is long and undesirable, and the assumption would be that anything I register won't be deallocated until the device simulation is complete and main is terminating.
A benefit here is that if I register a new simulation duration variable, it will automatically deallocate, so there's no danger of me forgetting do deallocate it and creating a memory leak.
Is such a wrapper possible?
Would you suggest doing it?
If so, how?
Thanks in advance!
An idea:
Create both functions, one that allocates memory and provides valid pointers after register them in a "list" of allocated pointers. In the second method, loop this list and deallocate all pointers:
// ask for new allocated pointer that will be registered automatically in list of pointers.
pointer1 = allocatePointer(size, listOfPointers);
pointer2 = allocatePointer(size, listOfPointers);
...
// deallocate all pointers
deallocatePointers(listOfPointers);
Even, you may use different listOfPointers depending of your simulation scope:
listOfPointer1 = getNewListOfPointers();
listOfPointer2 = getNewListOfPointers();
....
p1 = allocatePointer(size, listOfPointer1);
p2 = allocatePointer(size, listOfPointer2);
...
deallocatePointers(listOfPointers1);
...
deallocatePointers(listOfPointers2);
There are many ways to skin a cat, as they say.
I would recommend thrust's device_vector as a memory management tool. It abstracts allocation, deallocation, and memcpy in CUDA. It also gives you access to all the algorithms that Thrust provides.
I wouldn't recommend keeping random lists of unrelated pointers as Tio Pepe recommends. Instead you should encapsulate related data into a class. Even if you use thrust::device_vector you may want to encapsulate multiple related vectors and operations on them into a class.
The best choice is probably to use the smart pointers from C++ boost library, if that is an option.
If not, the best you can hope for in C is a program design that allows you to write allocation and deallocation in one place. Perhaps something like the following pseudo code:
while(!terminate_program)
{
switch(state_machine)
{
case STATE_PREOPERATIONAL:
myclass_init(); // only necessary for non-global/static objects
myclass_mem_manager();
state_machine = STATE_RUNNING;
break;
case STATE_RUNNING:
myclass_do_stuff();
...
break;
...
case STATE_EXIT:
myclass_mem_manager();
terminate_program = true;
break;
}
void myclass_init()
{
ptr_x = NULL;
ptr_y = NULL;
/* Where ptr_x, ptr_y are some of the many objects to allocate/deallocate.
If ptr is a global/static, (static storage duration) it is
already set to NULL automatically and this function isn't
necessary */
}
void myclass_mem_manager()
{
ptr_x = mem_manage (ptr_x, items_x*sizeof(Type_x));
ptr_y = mem_manage (ptr_y, items_y*sizeof(Type_y));
}
static void* mem_manage (const void* ptr, size_t bytes_n)
{
if(ptr == NULL)
{
ptr = malloc(bytes_n);
if (ptr == NULL)
{} // error handling
}
else
{
free(ptr);
ptr = NULL;
}
return ptr;
}

Lazy object creation in C++, or how to do zero-cost validation

I've stumbled across this great post about validating parameters in C#, and now I wonder how to implement something similar in C++. The main thing I like about this stuff is that is does not cost anything until the first validation fails, as the Begin() function returns null, and the other functions check for this.
Obviously, I can achieve something similar in C++ using Validate* v = 0; IsNotNull(v, ...).IsInRange(v, ...) and have each of them pass on the v pointer, plus return a proxy object for which I duplicate all functions.
Now I wonder whether there is a similar way to achieve this without temporary objects, until the first validation fails. Though I'd guess that allocating something like a std::vector on the stack should be for free (is this actually true? I'd suspect an empty vector does no allocations on the heap, right?)
Other than the fact that C++ does not have extension methods (which prevents being able to add in new validations as easily) it should be too hard.
class Validation
{
vector<string> *errors;
void AddError(const string &error)
{
if (errors == NULL) errors = new vector<string>();
errors->push_back(error);
}
public:
Validation() : errors(NULL) {}
~Validation() { delete errors; }
const Validation &operator=(const Validation &rhs)
{
if (errors == NULL && rhs.errors == NULL) return *this;
if (rhs.errors == NULL)
{
delete errors;
errors = NULL;
return *this;
}
vector<string> *temp = new vector<string>(*rhs.errors);
std::swap(temp, errors);
}
void Check()
{
if (errors)
throw exception();
}
template <typename T>
Validation &IsNotNull(T *value)
{
if (value == NULL) AddError("Cannot be null!");
return *this;
}
template <typename T, typename S>
Validation &IsLessThan(T valueToCheck, S maxValue)
{
if (valueToCheck < maxValue) AddError("Value is too big!");
return *this;
}
// etc..
};
class Validate
{
public:
static Validation Begin() { return Validation(); }
};
Use..
Validate::Begin().IsNotNull(somePointer).IsLessThan(4, 30).Check();
Can't say much to the rest of the question, but I did want to point out this:
Though I'd guess that allocating
something like a std::vector on the
stack should be for free (is this
actually true? I'd suspect an empty
vector does no allocations on the
heap, right?)
No. You still have to allocate any other variables in the vector (such as storage for length) and I believe that it's up to the implementation if they pre-allocate any room for vector elements upon construction. Either way, you are allocating SOMETHING, and while it may not be much allocation is never "free", regardless of taking place on the stack or heap.
That being said, I would imagine that the time taken to do such things will be so minimal that it will only really matter if you are doing it many many times over in quick succession.
I recommend to get a look into Boost.Exception, which provides basically the same functionality (adding arbitrary detailed exception-information to a single exception-object).
Of course you'll need to write some utility methods so you can get the interface you want. But beware: Dereferencing a null-pointer in C++ results in undefined behavior, and null-references must not even exist. So you cannot return a null-pointer in a way as your linked example uses null-references in C# extension methods.
For the zero-cost thing: A simple stack-allocation is quite cheap, and a boost::exception object does not do any heap-allocation itself, but only if you attach any error_info<> objects to it. So it is not exactly zero cost, but nearly as cheap as it can get (one vtable-ptr for the exception-object, plus sizeof(intrusive_ptr<>)).
Therefore this should be the last part where one tries to optimize further...
Re the linked article: Apparently, the overhaead of creating objects in C# is so great that function calls are free in comparison.
I'd personally propose a syntax like
Validate().ISNOTNULL(src).ISNOTNULL(dst);
Validate() contructs a temporary object which is basically just a std::list of problems. Empty lists are quite cheap (no nodes, size=0). ~Validate will throw if the list is not empty. If profiling shows even this is too expensive, then you just change the std::list to a hand-rolled list. Remember, a pointer is an object too. You're not saving an object just by sticking to the unfortunate syntax of a raw pointer. Conversely, the overhead of wrapping a raw pointer with a nice syntax is purely a compile-time price.
PS. ISNOTNULL(x) would be a #define for IsNotNull(x,#x) - similar to how assert() prints out the failed condition, without having to repeat it.