I've been writing a test case program to demonstrate a problem with a larger program of mine,
and the test case has a bug that the original program does not.
Here's the header file:
// compiled with g++ -I/usr/local/bin/boost_1_43_0 -Wall -std=c++0x -g test.cpp
#include <bitset>
#include <boost/shared_ptr.hpp>
#include <vector>
typedef std::vector< std::vector< std::bitset<11> > > FlagsVector;
namespace yarl
{
namespace path
{
class Pathfinder;
}
namespace level
{
class LevelMap
{
// Member Variables
private:
int width, height;
FlagsVector flags;
public:
boost::shared_ptr<path::Pathfinder> pathfinder;
// Member Functions
LevelMap(const int, const int);
int getWidth() const {return width;}
int getHeight() const {return height;}
bool getFifthBit(const int x, const int y) const
{
return flags.at(x).at(y).test(5);
}
};
class Level
{
// Member Variables
public:
LevelMap map;
// Member Functions
public:
Level(const int w=50, const int h=50);
};
}
namespace path
{
class Pathfinder
{
// Member Variables
private:
boost::shared_ptr<level::LevelMap> clientMap;
// Member Functions
public:
Pathfinder() {}
Pathfinder(level::LevelMap* cm)
: clientMap(cm) {}
void test() const;
};
}
}
and here's the implementation file:
#include <iostream>
#include "test.hpp"
using namespace std;
namespace yarl
{
namespace level
{
LevelMap::LevelMap(const int w, const int h)
: width(w), height(h), flags(w, vector< bitset<11> >(h, bitset<11>())),
pathfinder(new path::Pathfinder(this))
{}
Level::Level(const int w, const int h)
: map(w,h)
{
map.pathfinder->test();
}
}
namespace path
{
void Pathfinder::test() const
{
int width = clientMap->getWidth();
int height = clientMap->getHeight();
cerr << endl;
cerr << "clientMap->width: " << width << endl;
cerr << "clientMap->height: " << height << endl;
cerr << endl;
for(int x=0; x<width; ++x)
{
for(int y=0; y<height; ++y)
{
cerr << clientMap->getFifthBit(x,y);
}
cerr << "***" << endl; // marker for the end of a line in the output
}
}
}
}
int main()
{
yarl::level::Level l;
l.map.pathfinder->test();
}
I link this program with electric fence, and when I run it it aborts with this error:
ElectricFence Aborting: free(bffff434): address not from malloc().
Program received signal SIGILL, Illegal instruction.
0x0012d422 in __kernel_vsyscall ()
backtracing from gdb shows that the illegal instruction is in the compiler-generated destructor of Pathfinder, which is having trouble destructing its shared_ptr. Anyone see why that is?
yarl::level::Level l;
You instantiate an automatic Level variable, which, in its constructor constructs its member pathfinder like so:
pathfinder(new path::Pathfinder(this))
Then in the Pathfinder constructor, it takes the Level pointer that you pass in and assigns that to a shared_ptr. The shared_ptr then takes ownership of this pointer.
This is incorrect for several reasons:
A shared_ptr should be used to manage dynamically allocated objects, not automatically allocated objects
If you want to use shared_ptr, then you should use it everywhere: as it is now, you pass raw pointers (e.g. to the constructor of Pathfinder, but then store them as shared_ptrs. This just opens a big can of ownership worms.
The correct way to assign this to a shared_ptr is to derive from enable_shared_from_this; note however that you cannot get a shared_ptr from this in a constructor.
When the shared_ptr is destroyed, it will try to delete the pointer it manages. In this case, however, that pointer is not to a dynamically allocated object (i.e., allocated with new), but to an automatically allocated object (i.e., on the stack). Hence, the error.
If you don't need something to take ownership of a resource, there is nothing wrong with using a raw pointer (or a reference, if you have that option).
You are constructing shared_ptr from a pointer which is not supposed to be managed by shared_ptr. (The this pointer)
When the last shared_ptr's copy is destroyed this memory is free'd - when in fact it should not - the this is on stack in this case.
There is a reason the constructor of shared_ptr is explicit - it is exactly to avoid such an unnoticed conversion from regular pointer, which is not to be managed by shared_ptr, to a shared_ptr - once you pass such a pointer into shared_ptr, your program is doomed - the only way out is by deleting the pointer you have not intended to delete.
In general it is advisable to construct shared pointer with new directly - such as ptr(new Somethings(x,y,z) - this way you don't risk an exception leaking your allocated, but unassigned to a shared_ptr memory.
The Level contains a LevelMap member variable. When the Level gets destroyed, it will also destroy its LevelMap.
On the other hand a pointer to this LevelMap member is passed to Pathfinder, which creates a shared_ptr<> from the passed pointer. This newly created shared_ptr<> thinks that it owns the object it points to and will try to destroy it once the Pathfinder gets destroyed.
So the LevelMap is destroyed several times.
In the example the LevelMap is created on the stack. Therefore the delete called by shared_ptr<> can see that the address is not from the heap and you get an error. If your real program also has this problem, but all those objects are dynamically allocated, the error will probably not be detected. You will just get silent memory corruption and weird crashes later on.
Related
TL;DR : why would a pointer to an object in shared memory become invalid after a simple change of scope ? The pointer is not itself in the shared memory. Why would it be rendered useless ?
Let's say I have a simple custom data type for the sake of the example :
class CustomDataType
{
public:
short val;
CustomDataType(short val__) { val = val__; };
virtual ~CustomDataType() {};
short square() { return val*val; };
};
I have some function creating an instance of this object in a shared memory and returning a pointer to the object :
CustomDataType* someFunction()
{
shared_memory_object::remove("Boost");
managed_shared_memory managed_shm{ open_or_create, "Boost", 1024 };
CustomDataType *i = managed_shm.construct<CustomDataType>("CustomDataTypeObject")(7);
std::cout << i->square() << std::endl;
return i;
}
My main calls this function and then tries to access the pointer that is returned :
void main()
{
CustomDataType* customData = someFunction();
std::cout << customData->square() << std::endl; // <------- program crashes
}
The first call to the method 'square' through the pointer within the function works.
However, the second call to the method 'square' through the pointer in the main fails.
Why does the pointer not survive the change of scope ? I am in the same process, the object is still in memory.
Is the pointer rendered useless as soon as the managed_shared_memory gets out of scope, even though it does not change anything in the memory ?
Is there something I am missing to make the pointer survive the change of scope ?
I also tried replacing the standard pointer with an offset_ptr or passing the pointer by reference to the function instead of returning it. Nothing has modified the code's behaviour.
Minimal, Complete, and Verifiable example :
#include <boost/interprocess/managed_shared_memory.hpp>
#include <iostream>
using namespace boost::interprocess;
class CustomDataType
{
public:
short val;
CustomDataType(short val__) { val = val__; };
virtual ~CustomDataType() {};
short square() { return val*val; };
};
CustomDataType* someFunction()
{
shared_memory_object::remove("Boost");
managed_shared_memory managed_shm{ open_or_create, "Boost", 1024 };
CustomDataType *i = managed_shm.construct<CustomDataType>("CustomDataTypeObject")(7);
std::cout << i->square() << std::endl;
return i;
}
void main()
{
CustomDataType* customData = someFunction();
std::cout << customData->square() << std::endl; // <------- program crashes
}
The problem is that when the shared memory management object managed_shm goes out of scope, it is destroyed, and the shared memory is detached (so all pointers into it become invalid).
Normally, you use global variables for shared memeory management objects, so shared memory mappings remain valid until the program completes (unless explicitly destroyed).
From the documentation:
When the managed_shared_memory object is destroyed, the shared memory object is automatically unmapped, and all the resources are freed.
This means that when managed_shm goes out of scope, the pointer i that you return becomes dangling.
Problem
I am currently working on a plugin-library, where one should be able to not only import C-Linkage symbols, but all imported things.
Thus far it works, though the problem is, that gcc screws member-function calls up.
If I export the following:
static member_function(Class* c)
{ c->method();}
it works fine an I can access the class-members. But if I do the following:
void (Class ::*p)() = import("Class::method");
(x.*p)();
i get the right pointer and also am able to call the function and the passed arguments, but the this pointer is pointing into nirvana. I think gcc is taking it from the wrong position of the stack or something like that.
It works just fine with MSVC.
I am using mingw-w64 5.1.
Does anyone have an idea what the error could be?
Simple example:
plugin.cpp
#include <iostream>
namespace space {
class __declspec(dllexport) SomeExportThingy
{
int i = 42;
public:
virtual void __declspec(dllexport) Method(int*) const
{
using namespace std;
cout << "Calling Method" << endl;
cout << pi << endl;
cout << *pi << endl;
cout << this << endl;
cout << this->i << endl;
}
}
}
loader.cpp
namespace space {
class SomeExportThingy
{
///dummy to have some data in the address
int dummy[20];
};
int main()
{
auto h = LoadLibrary("plugin.dll");
auto p = GetProcAddress(h, "_ZNK5space16SomeExportThingy6MethodEPi");
typedef void (space::SomeExportThingy::*mptr)(int*) const;
///used because posix passed void*
auto fp = *reinterpret_cast<mptr*>(&p);
space::SomeExportThingy st;
int value = 22;
cout << "ValueLoc: " << &value << endl;
cout << "StLoc: " << &st << endl;
(st.*fp)(&value);
}
Results
Now what happens is, that the function is called and the pointer to pi is passed correctly. However, the this pointer is completly screwed up.
Again: it works with MSVC, which get's the this pointer correctly, but gcc get's this wrong.
I have no idea why this happens, and removing the virtual from the method doesn't change that either.
I have no idea what causes this, so maybe someone has an idea what the ABI is doing here.
Here are the pointers I am getting:
0x00400000 == GetModuleHandleA(NULL)
0x61840000 == GetModuleHandleA("plugin.dll")
0x0029fcc4 == _&st
0x00ddcd60 == this
I wasn't able to find any relation between the values
This is not going to work with GCC:
typedef void (space::SomeExportThingy::*mptr)(int*) const;
///used because posix passed void*
auto fp = *reinterpret_cast<mptr*>(&p);
The representation of a pointer-to-member is twice the size of a normal function pointer (or a void*) so you are reading two words from a memory location that only contains one word. The second word (which tells the compiler how to adjust the this pointer for the call) is garbage, it is just whatever happens to be after p on the stack.
See https://gcc.gnu.org/onlinedocs/gcc/Bound-member-functions.html:
In C++, pointer to member functions (PMFs) are implemented using a wide pointer of sorts to handle all the possible call mechanisms; the PMF needs to store information about how to adjust the ‘this’ pointer,
p is a void* so it's a memory location on the stack that occupies sizeof(void*) bytes.
&p is a pointer to that memory location.
reinterpret_cast<mptr*>(&p) is a pointer to 2*sizeof(void*) bytes at the same address.
*reinterpret_cast<mptr*>(&p) reads 2*sizeof(void*) bytes from a memory location that is only sizeof(void*) bytes in size.
Bad things happen.
For linux, the functions for dynamic function loading are: dlopen(), dlsym(), and dlclose(). Please reference: dlopen() man page.
Consider that C++ method names are 'mangled' and and they have an invisible '*this' parameter passed before all the others. Together both issues makes trying to directly access C++ objects not trivial when using dynamic linking.
The easiest solution I've found is to use 'C' function(s) that expose access to the C++ object instance.
Secondly, memory management of C++ objects is not trivial when the code to instantiate is within an .so library object, though the referencing code is from the user's app.
For the long answer as to why avoiding Pointer to C++ Member Methods is difficult, please reference: ISO CPP Reference, Pointers to Methods.
/** File: MyClass.h **/
// Explicitly ensure 'MyClassLoaderFunc' is NOT name mangled.
extern 'C' MyClass* MyClassLoaderFunc(p1, p2 ,p3, etc );
extern 'C' MyClass* MyClassDestroyerFunc(MyClass* p);
// Create function pointer typedef named 'LoaderFuncPtr'
typedef MyClass*(MyClassLoaderFunc* LoaderFuncPtr)(p1,p2,p3,etc);
// Define MyClass
class MyClass
{
/** methods & members for the class go here **/
char dummy[25];
int method( const char *data);
};
/** File: MyClass.cpp **/
#include "MyClass.h"
MyClass* MyLoaderFunc(p1, p2 ,p3, etc) {
MyClass* newInstance = new MyClass::CreateInstance( p1, p2, p3, etc);
/** Do something with newInstance **/
return newInstance;
}
MyClass::method(const char* data)
{
}
/** File: MyProgram.cpp **/
#include "MyClass.h"
main()
{
// Dynamically load in the library containing the object's code.
void *myClassLibrary = dlopen("path/to/MyClass.so",RTLD_LOCAL);
// Dynamically resolve the unmangled 'C' function name that
// provides the bootstrap access to the MyClass*
LoaderFuncPtr loaderPtr = dlsym(myClassLibrary,"MyClassLoaderFunc");
DestroyFuncPtr destroyerPtr = dlsym(myClassLibrary,"MyClassDestroyerFunc");
// Use dynamic function to retrieve an instance of MyClass.
MyClass* myClassPtr = loadPtr(p1,p2,p3,etc);
// Do something with MyClass
myClassPtr->method();
// Cleanup of object should happen within original .cpp file
destroyPtr(myClassPtr);
myClassPtr = NULL;
// Release resources
dlclose(myClassLibrary);
return 0;
}
Hope this helps..
I also suggest a factory paradigm as an more robust solution, that I'll leave to the reader to explore.
As Jonathan pointed out, pointer-to-members are bigger than normal function pointers.
The simplest solution is to reserve and initialize the extra space.
typedef void (space::SomeExportThingy::*mptr)(int*) const;
union {
mptr fp;
struct {
FARPROC function;
size_t offset;
};
} combFp;
combFp.function = p;
combFp.offset = 0;
auto fp = combFp.fp;
Now I implement a class and I need to use vector to hold some pointers.This is a global member.
vector g_vIPControlCollection;
When the system finalize. I want to reclaim the memory. Then I define the destroy method.
void Destroy()
{
int size = g_vIPControlCollection.size();
if (size > 1)
{
for (int i = 0; i < size; i++)
{
g_vIPControlCollection[i]->Release();
}
}
g_vIPControlCollection.clear();
g_vIPControlCollection.~vector<IPersistorControl*>(); //Does this line is necessary?
}
My question is whether I need to call destructor of the vector? Thanks in advance. Your help will be greatly appreciated.
No.
If you did that, after the call to Destroy it would be left in an invalid state. It will self destruct when its owner gets destroyed.
And your Destroy function should probably be a destructor, too.
No you should not, what you should do is use a unique_ptr to manage your IPersistorControl objects for example:
#include <iostream>
#include <vector>
#include <memory>
using namespace std;
struct Point{
int x; int y;
Point(int x,int y): x(x), y(y){}
~Point(){ cout<< "destroying " << y<<endl;}
};
int main() {
{
vector<unique_ptr<Point>> ps;
ps.emplace_back(unique_ptr<Point>(new Point(1,2)));
ps.emplace_back(unique_ptr<Point>(new Point(1,3)));
ps.emplace_back(unique_ptr<Point>(new Point(1,4)));
} // will call dtors here
cout << "Example 1" <<endl;
{
vector<unique_ptr<Point>> ps;
ps.emplace_back(unique_ptr<Point>(new Point(1,2)));
ps.emplace_back(unique_ptr<Point>(new Point(1,3)));
ps.emplace_back(unique_ptr<Point>(new Point(1,4)));
ps.clear(); // will call them here
cout << "Example 2" <<endl;
}
return 0;
}
Note that if IPersistorControl is some object that needs a special type that requires some other method of "reclaiming" (e.g windows handles or file handles) you can specify a Deleter e.g:
unique_ptr<FILE, int(*)(FILE*)> filePtr(fopen("LALA.txt", "wb"), fclose);
^ ^
Deleter type the actual Deleter
No you should almost never call the destructor explicitly.
What you need to do here is just g_vIPControlCollection.clear(); that you are already doing. After this std::vector hardly keeps any memory (typically 12 bytes on 32 bit machines) and would be cleaned when the program ends.
No. You shouldn't and needn't call destructor manually. That would
result in destructor invocation two times ( 1 by you and 1 when
object goes out of scope), affecting object state and your program
might crash at best. See live demo
It is good and recommended practice to clear memory allocated. But since it is a
global object, and program is going to terminate, even if you don't
free memory, it will reclaimed as process terminates.
Try using std::unique_ptr<> for storing pointers in std::vector
so that memory will be released when the clear method in
invoked on std::vector. This way you don't have to iterate and manually invoke the Release method of each member of std::vector
I have a class I’d like to be able to set a flag in that says if it is heap allocated so it can properly clean up after itself and not try to delete itself if it’s on the stack. The problem is...I can’t seem to override both new and the constructors at the same time. So it goes from my new overload that sets the isHeapAllocated flag and then into my constructor which resets the flag.
void* String8::operator new(size_t size)
{
String8* string = (String8*)malloc(size);
if(string == null)
Exception("allocation fail : no free memory");
string->isHeapAllocated = true;
return string;
}
String8::String8()
{
isHeapAllocated = false;
}
So new String8() sets the isHeapAllocated flag and then resets it to false. Is there any way to do this?
It will not work as intended:
The new operator return unitialized memory to be given to the constructor.
You -correctly- do String8* string = (String8*)malloc(size);, but *string, at this stage is not yet a String8 object: it is just the memory bulk that will contain it.
So string->isHeapAllocated = true; in fact sets a flag inside a not yet constructed object (that's UB).
Admitting this will not compromise the OS process, so that the program will not crash (you write memory that belongs already to you, after all ...), when you will later do something like String8* ptr = new String8;, after new returns, the String8::String8 constructor is called, and the member will be set back to "false" independently on what you did in the new operator overload.
The idiomatic way to manage C++ objects is let who allocate to be responsible to deallocate.
(and if "who" it is the stack, it just do that by definition).
This is a bad idea, but here’s a way to do it that doesn’t invoke undefined behaviour.
#include <iostream>
#include <memory>
#include <set>
using namespace std;
class C {
public:
void* operator new(size_t size) {
C* c = static_cast<C*>(::operator new(size));
heap_instances.insert(c);
return c;
}
C() : heap_allocated(heap_instances.find(this) != heap_instances.end()) {}
const bool heap_allocated;
private:
static set<const C*> heap_instances;
};
set<const C*> C::heap_instances;
int main(int argc, char** argv) {
cout << boolalpha;
C stack;
cout << stack.heap_allocated << '\n'; // false
C* heap_nozero = new C;
cout << heap_nozero->heap_allocated << '\n'; // true
delete heap_nozero;
C* heap_zero = new C();
cout << heap_zero->heap_allocated << '\n'; // true
delete heap_zero;
}
You can remove pointers from heap_instances when you’re done with them, of course, and use a more suitable container if you’re running in a multithreaded environment. But again, I wouldn’t recommend that you actually do this—deciding behaviour based on allocation is not something an object ought to do.
The only legitimate reason I can think of for this is to enable delete this. While that’s safe if you’re careful not to access members after the object’s suicide, it’s usually saner to let objects manage the lifetimes of other objects.
Note that the construtor gets called if it is allocated on the stack or the heap and there is no way for the object to detect if it was allocated on the stack or in the heap.
To create an object at the stack you don't use any memory allocation functions like this
String8 myString;
To create it on the heap you do
String8 *myString = new String8();
note that you do have to do the cleanup manually after not using the object anymore.
For the use of Heap objects bound to stack scope you can check out the RAII principle which is used intensly by c++ programs (see here for a better explaination of the difference of heap allocation and stack allocation).
Not sure why you need this, really. It's caller's responsibility to call delete if needed and your class's destructor should not be different whether it's called on the object on stack or on the heap... But, maybe, you are doing some special purpose class... Here is my quick take on it.
EDIT: You should also, probably, add custom delete operator to your class, unless you know that global delete calls a deallocation function that matches the allocation function you use in your custom new operator.
#include <cstdlib>
#include <iostream>
namespace so
{
class _test_
{
private:
static bool flag_allocation_heap;
bool flag_heap;
public:
_test_()
: flag_heap( flag_allocation_heap )
{
flag_allocation_heap = 0;
std::cout << flag_heap << std::endl;
}
void * operator new( std::size_t _size )
{
_test_ * test_ = static_cast< _test_ * >( std::malloc( _size ) );
flag_allocation_heap = 1;
return ( test_ );
}
};
bool _test_::flag_allocation_heap = 0;
} // namespace so
int main()
{
so::_test_ test_stack_;
so::_test_ * test_memory_ = new so::_test_;
delete test_memory_;
return( 0 );
}
Output:
0
1
Can you explain what's going on with my code here? I'm not sure if I'm using the destructor correctly or not in the struct.
With the destructor in there I get:
function1: 23
function2: 8.86183e-317
* glibc detected ./a: double free or corruption (fasttop): 0x000000000111b010 **
If I just comment out the destructor I get:
function1: 23
function2: 24
which is what I want. But don't I need the destructor to avoid a memory leak for a more complicated program?
(As you can see I may be a bit confused on pointers/allocation in general)
Thanks!
Edit: oh yeah, and why does the extra allocation step in function1 make a difference?
Edit2: Should I be initializing x = 0 in the constructor? I thought that was proper...should I be allocating it on initialization when I do this? So instead: x = gsl_vector_alloc(1).
#include <iostream>
using namespace std;
#include <cassert>
#include <cmath>
#include <gsl/gsl_vector.h>
struct struct1{
gsl_vector * x;
struct1() {
x = 0;
}
~struct1() {
if (x) gsl_vector_free(x);
}
};
void function1(void *p) {
struct1 s = *(struct1 *) p;
s.x = gsl_vector_alloc(1);
gsl_vector_set(s.x, 0, 24);
}
void function2(void *p) {
struct1 s = *(struct1 *) p;
gsl_vector_set(s.x, 0, 24);
}
int main() {
struct1 s;
s.x = gsl_vector_alloc(1);
gsl_vector_set(s.x, 0, 23);
function1(&s);
cout << "function1: " << gsl_vector_get(s.x, 0) << endl;
function2(&s);
cout << "function2: " << gsl_vector_get(s.x, 0) << endl;
return 0;
}
Inside of function1 and function2 you make a copy of the struct1 object that you create in the main() function. These copies have the same pointer x. When the destructor for each of these copies is called, gsl_vector_free is called, so you try to call it three times on the same pointer:
once in function1 when s is destroyed
once in function2 when s is destroyed
once in main when s is destroyed
You need to implement a copy constructor and copy assignment operator for this class. Any time you have a class that owns a resource, you need to implement these two functions and a destructor. A resource is anything that needs to be cleaned up when you are done using it.
In your sample code, it would be far better to encapsulate all of the allocation and deallocation inside of the class so that you can use the class without worrying about it. Make the class actually manage its resource.