Example:
bool isHeapPtr(void* ptr)
{
//...
}
int iStack = 35;
int *ptrStack = &iStack;
bool isHeapPointer1 = isHeapPtr(ptrStack); // Should be false
bool isHeapPointer2 = isHeapPtr(new int(5)); // Should be true
/* I know... it is a memory leak */
Why, I want to know this:
If I have in a class a member-pointer and I don't know if the pointing object is new-allocated. Then I should use such a utility to know if I have to delete the pointer.
But:
My design isn't made yet. So, I will program it that way I always have to delete it. I'm going to avoid rubbish programming
There is no way of doing this - and if you need to do it, there is something wrong with your design. There is a discussion of why you can't do this in More Effective C++.
In the general case, you're out of luck, I'm afraid - since pointers can have any value, there's no way to tell them apart. If you had knowledge of your stack start address and size (from your TCB in an embedded operating system, for example), you might be able to do it. Something like:
stackBase = myTCB->stackBase;
stackSize = myTCB->stackSize;
if ((ptrStack < stackBase) && (ptrStack > (stackBase - stackSize)))
isStackPointer1 = TRUE;
The only "good" solution I can think of is to overload operator new for that class and track it. Something like this (brain compiled code):
class T {
public:
void *operator new(size_t n) {
void *p = ::operator new(n);
heap_track().insert(p);
return p;
}
void operator delete(void* p) {
heap_track().erase(p);
::operator delete(p);
}
private:
// a function to avoid static initialization order fiasco
static std::set<void*>& heap_track() {
static std::set<void*> s_;
return s_;
}
public:
static bool is_heap(void *p) {
return heap_track().find(p) != heap_track().end();
}
};
Then you can do stuff like this:
T *x = new X;
if(T::is_heap(x)) {
delete x;
}
However, I would advise against a design which requires you to be able to ask if something was allocated on the heap.
Well, get out your assembler book, and compare your pointer's address to the stack-pointer:
int64_t x = 0;
asm("movq %%rsp, %0;" : "=r" (x) );
if ( myPtr < x ) {
...in heap...
}
Now x would contain the address to which you'll have to compare your pointer to. Note that it will not work for memory allocated in another thread, since it will have its own stack.
here it is, works for MSVC:
#define isheap(x, res) { \
void* vesp, *vebp; \
_asm {mov vesp, esp}; \
_asm {mov vebp, ebp}; \
res = !(x < vebp && x >= vesp); }
int si;
void func()
{
int i;
bool b1;
bool b2;
isheap(&i, b1);
isheap(&si, b2);
return;
}
it is a bit ugly, but works. Works only for local variables. If you pass stack pointer from calling function this macro will return true (means it is heap)
In mainstream operating systems, the stack grows from the top while the heap grows from the bottom. So you might heuristically check whether the address is beyond a large value, for some definition of "large." For example, the following works on my 64-bit Linux system:
#include <iostream>
bool isHeapPtr(const void* ptr) {
return reinterpret_cast<unsigned long long int>(ptr) < 0xffffffffull;
}
int main() {
int iStack = 35;
int *ptrStack = &iStack;
std::cout << isHeapPtr(ptrStack) << std::endl;
std::cout << isHeapPtr(new int(5)) << std::endl;
}
Note that is a crude heuristic that might be interesting to play with, but is not appropriate for production code.
First, why do you need to know this? What real problem are you trying to solve?
The only way I'm aware of to make this sort of determination would be to overload global operator new and operator delete. Then you can ask your memory manager if a pointer belongs to it (the heap) or not (stack or global data).
Even if you could determine whether a pointer was on one particular heap, or one particular stack, there can be multiple heaps and multiple stacks for one application.
Based on the reason for asking, it is extremely important for each container to have a strict policy on whether it "owns" pointers that it holds or not. After all, even if those pointers point to heap-allocated memory, some other piece of code might also have a copy of the same pointer. Each pointer should have one "owner" at a time, though ownership can be transferred. The owner is responsible for destructing.
On rare occasions, it is useful for a container to keep track of both owned and non-owned pointers - either using flags, or by storing them separately. Most of the time, though, it's simpler just to set a clear policy for any object that can hold pointers. For example, most smart pointers always own their container real pointers.
Of course smart pointers are significant here - if you want an ownership-tracking pointer, I'm sure you can find or write a smart pointer type to abstract that hassle away.
Despite loud claims to the contrary, it is clearly possible to do what you want, in a platform-dependent way. However just because something is possible, that does not automatically make it a good idea. A simple rule of stack==no delete, otherwise==delete is unlikely to work well.
A more common way is to say that if I allocated a buffer, then I have to delete it, If the program passes me a buffer, it is not my responsibility to delete it.
e.g.
class CSomething
{
public:
CSomething()
: m_pBuffer(new char[128])
, m_bDeleteBuffer(true)
{
}
CSomething(const char *pBuffer)
: m_pBuffer(pBuffer)
, m_bDeleteBuffer(false)
{
}
~CSomething()
{
if (m_bDeleteBuffer)
delete [] m_pBuffer;
}
private:
const char *m_pBuffer;
bool m_bDeleteBuffer;
};
You're trying to do it the hard way. Clarify your design so it's clear who "owns" data and let that code deal with its lifetime.
here is universal way to do it in windows using TIP:
bool isStack(void* x)
{
void* btn, *top;
_asm {
mov eax, FS:[0x08]
mov btn, eax
mov eax, FS:[0x04]
mov top, eax
}
return x < top && x > btn;
}
void func()
{
int i;
bool b1;
bool b2;
b1 = isStack(&i);
b2 = isStack(&si);
return;
}
The only way I know of doing this semi-reliably is if you can overload operator new for the type for which you need to do this. Unfortunately there are some major pitfalls there and I can't remember what they are.
I do know that one pitfall is that something can be on the heap without having been allocated directly. For example:
class A {
int data;
};
class B {
public:
A *giveMeAnA() { return &anA; }
int data;
A anA;
};
void foo()
{
B *b = new B;
A *a = b->giveMeAnA();
}
In the above code a in foo ends up with a pointer to an object on the heap that was not allocated with new. If your question is really "How do I know if I can call delete on this pointer." overloading operator new to do something tricky might help you answer that question. I still think that if you have to ask that question you've done something very wrong.
How could you not know if something is heap-allocated or not? You should design the software to have a single point of allocation.
Unless you're doing some truly exotic stuff in an embedded device or working deep in a custom kernel, I just don't see the need for it.
Look at this code (no error checking, for the sake of example):
class A
{
int *mysweetptr;
A()
{
mysweetptr = 0; //always 0 when unalloc'd
}
void doit()
{
if( ! mysweetptr)
{
mysweetptr = new int; //now has non-null value
}
}
void undoit()
{
if(mysweetptr)
{
delete mysweetptr;
mysweetptr = 0; //notice that we reset it to 0.
}
}
bool doihaveit()
{
if(mysweetptr)
return true;
else
return false;
}
~A()
{
undoit();
}
};
In particular, notice that I am using the null value to determine whether the pointer has been allocated or not, or if I need to delete it or not.
Your design should not rely on determining this information (as others have pointed out, it's not really possible). Instead, your class should explicitly define the ownership of pointers that it takes in its constructor or methods. If your class takes ownership of those pointers, then it is incorrect behavior to pass in a pointer to the stack or global, and you should delete it with the knowledge that incorrect client code may crash. If your class does not take ownership, it should not be deleting the pointer.
Related
I am working on an open-source library that has a memory leak in it. The library is a data streaming service built around boost::asio. The server side uses heap memory management system which provides memory to hold a finite number of samples while they wait to get pushed accross a tcp connection. When the server is first constructed, a heap of memory for all the old samples is allocated. From this heap, after a sample is passed accross the socket, the memory is returned to the heap.
This is fine, unless all that pre-allocated heap is already taken. Here is the function that creates a 'sample':
sample_p new_sample(double timestamp, bool pushthrough) {
sample *result = pop_freelist();
if (!result){
result = new(new char[sample_size_]) sample(fmt_, num_chans_, this);
}
return sample_p(result);
}
sample_p is just a typedef'd smart pointer templated to the sample class.
The offending line is in the middle. When there isn't a chunk of memory on the freelist, we need to make some. This leaks memory.
My question is why is this happening? Since I shove the new sample into a smart pointer, shouldn't the memory be freed when it goes out of scope (it gets popped off of a stack later on.)? Do I need to somehow handle the memory allocated on the inside---i.e. the memory allocated by new char[sample_size_]? If yes, how can I do that?
Edit:
#RichardHodges here is a compile-able MCVE. This is highly simplified but I think it captures exactly the problem I am facing in the original code.
#include <boost/intrusive_ptr.hpp>
#include <boost/lockfree/spsc_queue.hpp>
#include <iostream>
typedef boost::intrusive_ptr<class sample> sample_p;
typedef boost::lockfree::spsc_queue<sample_p> buffer;
class sample {
public:
double data;
class factory{
public:
friend class sample;
sample_p new_sample(int size, double data) {
sample* result = new(new char[size]) sample(data);
return sample_p(result);
}
};
sample(double d) {
data = d;
}
void operator delete(void *x) {
delete[](char*)x;
}
/// Increment ref count.
friend void intrusive_ptr_add_ref(sample *s) {
}
/// Decrement ref count and reclaim if unreferenced.
friend void intrusive_ptr_release(sample *s) {
}
};
void push_sample(buffer &buff, const sample_p &samp) {
while (!buff.push(samp)) {
sample_p dummy;
buff.pop(dummy);
}
}
int main(void){
buffer buff(1);
sample::factory factory_;
for (int i = 0; i < 10; i++)
push_sample(buff, factory_.new_sample(100,0.0));
std::cout << "press any key to exit" << std::endl;
char foo;
std::cin >> foo;
return 0;
}
When I step through the code, I note that my delete operator never gets called on the sample pointers. I guess that the library I'm working on (which again, I didn't write, so I am still learning its ways) is mis-using the intrusive_ptr type.
You are allocating the memory with new[] so you need to deallocate it with delete[] (on a char*). The smart pointer probably calls delete by default, so you should provide a custom deleter that calls delete[] (after manually invoking the destructor of the sample). Here is an example using std::shared_ptr.
auto s = std::shared_ptr<sample>(
new (new char[sizeof(sample)]) sample,
[](sample* p) {
p->~sample();
delete[] reinterpret_cast<char*>(p);
}
);
However, why you are using placement new when your buffer only contains one object? Why not just use regular new instead?
auto s = std::shared_ptr<sample>(new sample);
Or even better (with std::shared_ptr), use a factory function.
auto s = std::make_shared<sample>();
I have a program that contains a processing phase that needs to use a bunch of different object instances (all allocated on the heap) from a tree of polymorphic types, all eventually derived from a common base class.
As the instances may cyclically reference each other, and do not have a clear owner, I want allocated them with new, handle them with raw pointers, and leave them in memory for the phase (even if they become unreferenced), and then after the phase of the program that uses these instances, I want to delete them all at once.
How I thought to structure it is as follows:
struct B; // common base class
vector<unique_ptr<B>> memory_pool;
struct B
{
B() { memory_pool.emplace_back(this); }
virtual ~B() {}
};
struct D : B { ... }
int main()
{
...
// phase begins
D* p = new D(...);
...
// phase ends
memory_pool.clear();
// all B instances are deleted, and pointers invalidated
...
}
Apart from being careful that all B instances are allocated with new, and that noone uses any pointers to them after the memory pool is cleared, are there problems with this implementation?
Specifically I am concerned about the fact that the this pointer is used to construct a std::unique_ptr in the base class constructor, before the derived class constructor has completed. Does this result in undefined behaviour? If so is there a workaround?
In case you haven't already, familiarize yourself with Boost.Pool. From the Boost documentation:
What is Pool?
Pool allocation is a memory allocation scheme that is very fast, but
limited in its usage. For more information on pool allocation (also
called simple segregated storage, see concepts concepts and Simple Segregated Storage.
Why should I use Pool?
Using Pools gives you more control over how memory is used in your
program. For example, you could have a situation where you want to
allocate a bunch of small objects at one point, and then reach a point
in your program where none of them are needed any more. Using pool
interfaces, you can choose to run their destructors or just drop them
off into oblivion; the pool interface will guarantee that there are no
system memory leaks.
When should I use Pool?
Pools are generally used when there is a lot of allocation and
deallocation of small objects. Another common usage is the situation
above, where many objects may be dropped out of memory.
In general, use Pools when you need a more efficient way to do unusual
memory control.
Which pool allocator should I use?
pool_allocator is a more general-purpose solution, geared towards
efficiently servicing requests for any number of contiguous chunks.
fast_pool_allocator is also a general-purpose solution but is geared
towards efficiently servicing requests for one chunk at a time; it
will work for contiguous chunks, but not as well as pool_allocator.
If you are seriously concerned about performance, use
fast_pool_allocator when dealing with containers such as std::list,
and use pool_allocator when dealing with containers such as
std::vector.
Memory management is tricky business (threading, caching, alignment, fragmentation, etc. etc.) For serious production code, well-designed and carefully optimized libraries are the way to go, unless your profiler demonstrates a bottleneck.
Your idea is great and millions of applications are already using it. This pattern is most famously known as «autorelease pool». It forms a base for ”smart” memory management in Cocoa and Cocoa Touch Objective-C frameworks. Despite the fact that C++ provides hell of a lot of other alternatives, I still think this idea got a lot of upside. But there are few things where I think your implementation as it stands may fall short.
The first problem that I can think of is thread safety. For example, what happens when objects of the same base are created from different threads? A solution might be to protect the pool access with mutually exclusive locks. Though I think a better way to do this is to make that pool a thread-specific object.
The second problem is invoking an undefined behavior in case where derived class's constructor throws an exception. You see, if that happens, the derived object won't be constructed, but your B's constructor would have already pushed a pointer to this to the vector. Later on, when the vector is cleared, it would try to call a destructor through a virtual table of the object that either doesn't exist or is in fact a different object (because new could reuse that address).
The third thing I don't like is that you have only one global pool, even if it is thread-specific, that just doesn't allow for a more fine grained control over the scope of allocated objects.
Taking the above into account, I would do a couple of improvements:
Have a stack of pools for more fine-grained scope control.
Make that pool stack a thread-specific object.
In case of failures (like exception in derived class constructor), make sure the pool doesn't hold a dangling pointer.
Here is my literally 5 minutes solution, don't judge for quick and dirty:
#include <new>
#include <set>
#include <stack>
#include <cassert>
#include <memory>
#include <stdexcept>
#include <iostream>
#define thread_local __thread // Sorry, my compiler doesn't C++11 thread locals
struct AutoReleaseObject {
AutoReleaseObject();
virtual ~AutoReleaseObject();
};
class AutoReleasePool final {
public:
AutoReleasePool() {
stack_.emplace(this);
}
~AutoReleasePool() noexcept {
std::set<AutoReleaseObject *> obj;
obj.swap(objects_);
for (auto *p : obj) {
delete p;
}
stack_.pop();
}
static AutoReleasePool &instance() {
assert(!stack_.empty());
return *stack_.top();
}
void add(AutoReleaseObject *obj) {
objects_.insert(obj);
}
void del(AutoReleaseObject *obj) {
objects_.erase(obj);
}
AutoReleasePool(const AutoReleasePool &) = delete;
AutoReleasePool &operator = (const AutoReleasePool &) = delete;
private:
// Hopefully, making this private won't allow users to create pool
// not on stack that easily... But it won't make it impossible of course.
void *operator new(size_t size) {
return ::operator new(size);
}
std::set<AutoReleaseObject *> objects_;
struct PrivateTraits {};
AutoReleasePool(const PrivateTraits &) {
}
struct Stack final : std::stack<AutoReleasePool *> {
Stack() {
std::unique_ptr<AutoReleasePool> pool
(new AutoReleasePool(PrivateTraits()));
push(pool.get());
pool.release();
}
~Stack() {
assert(!stack_.empty());
delete stack_.top();
}
};
static thread_local Stack stack_;
};
thread_local AutoReleasePool::Stack AutoReleasePool::stack_;
AutoReleaseObject::AutoReleaseObject()
{
AutoReleasePool::instance().add(this);
}
AutoReleaseObject::~AutoReleaseObject()
{
AutoReleasePool::instance().del(this);
}
// Some usage example...
struct MyObj : AutoReleaseObject {
MyObj() {
std::cout << "MyObj::MyObj(" << this << ")" << std::endl;
}
~MyObj() override {
std::cout << "MyObj::~MyObj(" << this << ")" << std::endl;
}
void bar() {
std::cout << "MyObj::bar(" << this << ")" << std::endl;
}
};
struct MyObjBad final : AutoReleaseObject {
MyObjBad() {
throw std::runtime_error("oops!");
}
~MyObjBad() override {
}
};
void bar()
{
AutoReleasePool local_scope;
for (int i = 0; i < 3; ++i) {
auto o = new MyObj();
o->bar();
}
}
void foo()
{
for (int i = 0; i < 2; ++i) {
auto o = new MyObj();
bar();
o->bar();
}
}
int main()
{
std::cout << "main start..." << std::endl;
foo();
std::cout << "main end..." << std::endl;
}
Hmm, I needed almost exactly the same thing recently (memory pool for one phase of a program that gets cleared all at once), except that I had the additional design constraint that all my objects would be fairly small.
I came up with the following "small-object memory pool" -- perhaps it will be of use to you:
#pragma once
#include "defs.h"
#include <cstdint> // uintptr_t
#include <cstdlib> // std::malloc, std::size_t
#include <type_traits> // std::alignment_of
#include <utility> // std::forward
#include <algorithm> // std::max
#include <cassert> // assert
// Small-object allocator that uses a memory pool.
// Objects constructed in this arena *must not* have delete called on them.
// Allows all memory in the arena to be freed at once (destructors will
// be called).
// Usage:
// SmallObjectArena arena;
// Foo* foo = arena::create<Foo>();
// arena.free(); // Calls ~Foo
class SmallObjectArena
{
private:
typedef void (*Dtor)(void*);
struct Record
{
Dtor dtor;
short endOfPrevRecordOffset; // Bytes between end of previous record and beginning of this one
short objectOffset; // From the end of the previous record
};
struct Block
{
size_t size;
char* rawBlock;
Block* prevBlock;
char* startOfNextRecord;
};
template<typename T> static void DtorWrapper(void* obj) { static_cast<T*>(obj)->~T(); }
public:
explicit SmallObjectArena(std::size_t initialPoolSize = 8192)
: currentBlock(nullptr)
{
assert(initialPoolSize >= sizeof(Block) + std::alignment_of<Block>::value);
assert(initialPoolSize >= 128);
createNewBlock(initialPoolSize);
}
~SmallObjectArena()
{
this->free();
std::free(currentBlock->rawBlock);
}
template<typename T>
inline T* create()
{
return new (alloc<T>()) T();
}
template<typename T, typename A1>
inline T* create(A1&& a1)
{
return new (alloc<T>()) T(std::forward<A1>(a1));
}
template<typename T, typename A1, typename A2>
inline T* create(A1&& a1, A2&& a2)
{
return new (alloc<T>()) T(std::forward<A1>(a1), std::forward<A2>(a2));
}
template<typename T, typename A1, typename A2, typename A3>
inline T* create(A1&& a1, A2&& a2, A3&& a3)
{
return new (alloc<T>()) T(std::forward<A1>(a1), std::forward<A2>(a2), std::forward<A3>(a3));
}
// Calls the destructors of all currently allocated objects
// then frees all allocated memory. Destructors are called in
// the reverse order that the objects were constructed in.
void free()
{
// Destroy all objects in arena, and free all blocks except
// for the initial block.
do {
char* endOfRecord = currentBlock->startOfNextRecord;
while (endOfRecord != reinterpret_cast<char*>(currentBlock) + sizeof(Block)) {
auto startOfRecord = endOfRecord - sizeof(Record);
auto record = reinterpret_cast<Record*>(startOfRecord);
endOfRecord = startOfRecord - record->endOfPrevRecordOffset;
record->dtor(endOfRecord + record->objectOffset);
}
if (currentBlock->prevBlock != nullptr) {
auto memToFree = currentBlock->rawBlock;
currentBlock = currentBlock->prevBlock;
std::free(memToFree);
}
} while (currentBlock->prevBlock != nullptr);
currentBlock->startOfNextRecord = reinterpret_cast<char*>(currentBlock) + sizeof(Block);
}
private:
template<typename T>
static inline char* alignFor(char* ptr)
{
const size_t alignment = std::alignment_of<T>::value;
return ptr + (alignment - (reinterpret_cast<uintptr_t>(ptr) % alignment)) % alignment;
}
template<typename T>
T* alloc()
{
char* objectLocation = alignFor<T>(currentBlock->startOfNextRecord);
char* nextRecordStart = alignFor<Record>(objectLocation + sizeof(T));
if (nextRecordStart + sizeof(Record) > currentBlock->rawBlock + currentBlock->size) {
createNewBlock(2 * std::max(currentBlock->size, sizeof(T) + sizeof(Record) + sizeof(Block) + 128));
objectLocation = alignFor<T>(currentBlock->startOfNextRecord);
nextRecordStart = alignFor<Record>(objectLocation + sizeof(T));
}
auto record = reinterpret_cast<Record*>(nextRecordStart);
record->dtor = &DtorWrapper<T>;
assert(objectLocation - currentBlock->startOfNextRecord < 32768);
record->objectOffset = static_cast<short>(objectLocation - currentBlock->startOfNextRecord);
assert(nextRecordStart - currentBlock->startOfNextRecord < 32768);
record->endOfPrevRecordOffset = static_cast<short>(nextRecordStart - currentBlock->startOfNextRecord);
currentBlock->startOfNextRecord = nextRecordStart + sizeof(Record);
return reinterpret_cast<T*>(objectLocation);
}
void createNewBlock(size_t newBlockSize)
{
auto raw = static_cast<char*>(std::malloc(newBlockSize));
auto blockStart = alignFor<Block>(raw);
auto newBlock = reinterpret_cast<Block*>(blockStart);
newBlock->rawBlock = raw;
newBlock->prevBlock = currentBlock;
newBlock->startOfNextRecord = blockStart + sizeof(Block);
newBlock->size = newBlockSize;
currentBlock = newBlock;
}
private:
Block* currentBlock;
};
To answer your question, you're not invoking undefined behaviour since nobody is using the pointer until the object is fully constructed (the pointer value itself is safe to copy around until then). However, it's a rather intrusive method, as the object(s) themselves need to know about the memory pool. Additionally, if you're constructing a large number of small objects, it would likely be faster to use an actual pool of memory (like my pool does) instead of calling out to new for every object.
Whatever pool-like approach you use, be careful that the objects are never manually deleteed, because that would lead to a double free!
I still think this is an interesting question without a definitive reply, but please let me break it down into the different questions you are actually asking:
1.) Does inserting a pointer to a base class into a vector before initialisation of a subclass prevent or cause issues with retrieving inherited classes from that pointer. [slicing for example.]
Answer: No, so long as you are 100% sure of the relevant type that is being pointed to, this mechanism does not cause these issues however note the following points:
If the derived constructor fails, you are left with an issue later when you are likely to have a dangling pointer at least sitting in the vector, as that address space it [the derived class] thought it was getting would be freed to the operating environment on failure, but the vector still has the address as being of the base class type.
Note that a vector, although kind of useful, is not the best structure for this, and even if it was, there should be some inversion of control involved here to allow the vector object to control initialisation of your objects, so that you have awareness of success/failure.
These points lead to the implied 2nd question:
2.) Is this a good pattern for pooling?
Answer: Not really, for the reasons mentioned above, plus others (Pushing a vector past it's end point basically ends up with a malloc which is unnecessary and will impact performance.) Ideally you want to use a pooling library, or a template class, and even better, separate the allocation/de-allocation policy implementation away from the pool implementation, with a low level solution already being hinted at, which is to allocate adequate pool memory from pool initialisation, and then use this using pointers to void from within the pool address space (See Alex Zywicki's solution above.) Using this pattern, the pool destruction is safe as the pool which will be contiguous memory can be destroyed en masse without any dangling issues, or memory leaks through losing all references to an object (losing all reference to an object whose address is allocated through the pool by the storage manager leaves you with dirty chunk/s, but will not cause a memory leak as it is managed by the pool implementation.
In the early days of C/C++ (before mass proliferation of the STL), this was a well discussed pattern and many implementations and designs can be found out there in good literature: As an example:
Knuth (1973 The art of computer programming: Multiple volumes), and for a more complete list, with more on pooling, see:
http://www.ibm.com/developerworks/library/l-memory/
The 3rd implied question seems to be:
3) Is this a valid scenario to use pooling?
Answer: This is a localised design decision based on what you are comfortable with, but to be honest, your implementation (no controlling structure/aggregate, possibly cyclic sharing of sub sets of objects) suggests to me that you would be better off with a basic linked list of wrapper objects, each of which contains a pointer to your superclass, used only for addressing purposes. Your cyclical structures are built on top of this, and you simply amend/grow shrink the list as required to accommodate all of your first class objects as required, and when finished, you can then easily destroy them in effectively an O(1) operation from within the linked list.
Having said that, I would personally recommend that at this time (when you have a scenario where pooling does have a use and so you are in the right mind-set) to carry out the building of a storage management/pooling set of classes that are paramaterised/typeless now as it will hold you in good stead for the future.
This sounds what I have heard called a Linear Allocator.
I will explain the basics of how I understand how it works.
Allocate a block of memory using ::operator new(size);
Have a void* that is your Pointer to the next free space in memory.
You will have an alloc(size_t size) function that will give you a pointer to the location in the block from step one for you to construct on to using Placement New
Placement new looks like... int* i = new(location)int(); where location is a void* to a block of memory you alloced from the allocator.
when you are done with all of your memory you will call a Flush() function that will dealloc the memory from the pool or at least wipe the data clean.
I programmed one of these recently and i will post my code here for you as well as do my best to explain.
#include <iostream>
class LinearAllocator:public ObjectBase
{
public:
LinearAllocator();
LinearAllocator(Pool* pool,size_t size);
~LinearAllocator();
void* Alloc(Size_t size);
void Flush();
private:
void** m_pBlock;
void* m_pHeadFree;
void* m_pEnd;
};
don't worry about what i'm inheriting from. i have been using this allocator in conjunction with a memory pool. but basically instead of getting the memory from operator new i am getting memory from a memory pool. the internal workings are the same essentially.
Here is the implementation:
LinearAllocator::LinearAllocator():ObjectBase::ObjectBase()
{
m_pBlock = nullptr;
m_pHeadFree = nullptr;
m_pEnd=nullptr;
}
LinearAllocator::LinearAllocator(Pool* pool,size_t size):ObjectBase::ObjectBase(pool)
{
if (pool!=nullptr) {
m_pBlock = ObjectBase::AllocFromPool(size);
m_pHeadFree = * m_pBlock;
m_pEnd = (void*)((unsigned char*)*m_pBlock+size);
}
else{
m_pBlock = nullptr;
m_pHeadFree = nullptr;
m_pEnd=nullptr;
}
}
LinearAllocator::~LinearAllocator()
{
if (m_pBlock!=nullptr) {
ObjectBase::FreeFromPool(m_pBlock);
}
m_pBlock = nullptr;
m_pHeadFree = nullptr;
m_pEnd=nullptr;
}
MemoryBlock* LinearAllocator::Alloc(size_t size)
{
if (m_pBlock!=nullptr) {
void* test = (void*)((unsigned char*)m_pEnd-size);
if (m_pHeadFree<=test) {
void* temp = m_pHeadFree;
m_pHeadFree=(void*)((unsigned char*)m_pHeadFree+size);
return temp;
}else{
return nullptr;
}
}else return nullptr;
}
void LinearAllocator::Flush()
{
if (m_pBlock!=nullptr) {
m_pHeadFree=m_pBlock;
size_t size = (unsigned char*)m_pEnd-(unsigned char*)*m_pBlock;
memset(*m_pBlock,0,size);
}
}
This code is fully functional except for a few lines which will need to be changed because of my inheritance and use of the memory pool. but I bet you can figure out what needs to change and just let me know if you need a hand changing the code. This code has not been tested in any sort of professional manor and is not guaranteed to be thread safe or anything fancy like that. i just whipped it up and thought i could share it with you since you seemed to need help.
I also have a working implementation of a fully generic memory pool if you think it may help you. I can explain how it works if you need.
Once again if you need any help let me know. Good luck.
Say I have a pointer like this:
int *thingy;
At some point, this code may or may not be called:
thingy=new int;
How do I know if I can do this:
delete thingy;
I could use a bool for every pointer and mark the bool as true whenever the I use new, but I have many pointers and that would get very unwieldy.
If I have not called new on thingy, calling delete on it would likely cause a crash, right?
I searched around quite a bit but could find no answer that clearly fit my situation.
EDIT: I need to be able to delete the pointers as many times as I like without the pointers necessarily pointing to any data. If this is impossible I'll have to re-write my code.
Initialize it to NULL always
int *thingy = NULL;
and then
delete thingy;
thingy = NULL;
is valid even if thingy is NULL. You can do the delete as many times as you want as long as thingy is NULL delete will have no unwanted side effects.
There's no built-in way to tell if a particular pointer value is deleteable. Instead you simply have to design the program to do the right thing, preferably by carefully designing resource ownership policies in line with your requirements and them implementing them with something like RAII.
Given appropriate RAII types you will not need to scatter deletes or other resource management commands around your code. You will simply initialize and use objects of the appropriate types, and leave clean up to the objects themselves. For example if the RAII type unique_ptr corresponds to an ownership policy you want to use then you can manage an object this way:
unique_ptr<int> thingy {new int};
// use thingy ...
There's no need to manually cleanup, because unique_ptr takes care of that for you.
On the other hand if you try to manage resources directly you end up with lots of code like:
int *thingy = nullptr;
// ...
thingy = new int;
try {
// something that might throw
} catch(...) {
delete thingy;
thingy = nullptr;
throw;
}
delete thingy;
thingy = nullptr;
There is no builtin C++ tool to identify if a pointer points to heap data and can safely deleted. It's safe to delete a NULL pointer and you can set every pointer whose data has been deleted to NULL. But this doesn't help to differentiate between pointers to heap data and pointers to other data or to code.
When your operation system starts a process it will locate the code and data sections to specific data areas. In Windows this is partially controlled by the PE header of the EXE file. Therefore the actual address of the memory regions may vary. But you can identify where theses regions are located:
code
bss
data
stack
heap
After obtaining the address range for each region you can differentiate between a pointer to the heap data (where delete is appropriate) and a pointer to stack data. This allows you to differetiate between deleteable and data whose pointer you must not delete.
Write a wrapper class that does the tracking for you, eg:
template<typename T>
class ptr_t
{
private:
T* m_ptr;
bool m_delete;
ptr_t(const ptr_t&) {}
ptr_t& operator=(const ptr_t&) { return *this; }
public:
ptr_t()
: m_ptr(NULL), m_delete(false)
{
}
ptr_t(T *ptr, bool del)
: m_ptr(ptr), m_delete(del)
{
}
~ptr_t()
{
reset();
}
void assign(T *ptr, bool del)
{
if (m_delete)
delete m_ptr;
m_ptr = ptr;
m_delete = del;
}
void reset()
{
assign(NULL, false);
}
operator T*() { return m_ptr; }
bool operator!() const { return (!m_ptr); }
};
typedef ptr_t<int> int_ptr;
.
int_ptr thingy;
...
thingy.assign(new int, true);
...
thingy.reset();
.
int i;
int_ptr pi;
...
pi.assign(&i, false);
...
pi.reset();
This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
Can a local variable's memory be accessed outside its scope?
Is there worrying thing to do a code such (getIDs() returns a pointer):
class Worker{
private:
int workerID;
int departID;
int supervisorID;
public:
Worker()
{
workerID=0;
departID=0;
supervisorID=0;
name="anonymous";
workerAddress="none";
}
void setIDs(int worker, int depart, int supervisor)
{
workerID=worker;
departID=depart;
supervisorID=supervisor;
}
int* getIDs()
{
int id[3];
id[0]=workerID;
id[1]=departID;
id[2]=supervisorID;
return id;
}
};
And then, use it such:
Worker obj;
obj.setIDs(11,22,33);
cout<<(*obj.getIDs())<<endl;
cout<<++(*obj.getIDs())<<endl;
cout<<++(++(*obj.getIDs()))<<endl;
I am wondering about that because the compiler shows:
Warning 1 warning C4172: returning address of local variable or
temporary
Your int id[3] is allocated on a stack and gets destroyed when your int* getIDs() returns.
You're return a pointer to a variable that gets destroyed immediately after getIDs() returns. The pointer then becomes dangling and is practically useless as doing anyting with it is undefined behaviour.
Suppose you defined your class like this:
class Worker{
private:
int IDs[3];
public
// ...
int* getIDs() { return IDs; }
};
This partially solves your problem, as the pointer remains valid as long the Worker object is in scope, but it's still bad practice. Example:
int* ptr;
while (true) {
Worker obj;
obj.setIDs(11,22,33);
ptr = obj.getIDs();
cout << *ptr; // ok, obj is still alive.
break;
} // obj gets destroyed here
cout << *ptr; // NOT ok, dereferencing a dangling pointer
A better way of solving this is to implement your custom operator << for your class. Something like this:
class Worker {
private:
int workerID;
int departID;
int supervisorID;
public:
// ...
friend ostream& operator<<(ostream& out, Worker w);
};
ostream& operator<<(ostream& out, const Worker& w)
{
out << w.workerID << "\n" << w.departID << "\n" << w.supervisorID;
return out;
}
Even if this would work, it wouldn't be good practice to do it this way in c++ unless there is some profound reason why you want pointers to int. Raw c-syle arrays are more difficult to handle than, for instance, std::vectors, so use those, like
std::vector<int> getIDs(){
std::vector<int> id(3);
id[0]=workerID; id[1]=departID; id[2]=supervisorID;
return id;
}
If you're worried about the overhead: this is likely to be optimized away completely by modern compilers.
A local (also caled automatic) variable is destroyed once you leave the function where it is defined. So your pointer will point to this destroyed location, and of course referencing such a location outside the function is incorect and will cause undefined behaviour.
The basic problem here is that when you enter a function call, you get a new frame on your stack (where all your local variables will be kept). Anything that is not dynamically allocated (using new/malloc) in your function will exist in that stack frame, and it gets destroyed when your function returns.
Your function returns a pointer to the start of your 3-element-array which you declared in that stack frame that will go away. So, this is undefined behavior.
While you may get "lucky/unlucky" and still have your data around where the pointer points when you use it, you may also have the opposite happen with this code. Since the space is given up when the stack frame is destroyed, it can be reused - so another part of your code could likely use the memory location where your three elements in that array is stored, which would mean they would have completely different values by the time you dereferenced that pointer.
If you're lucky, your program would just seg-fault/crash so you knew you made a mistake.
Redesign your function to return a structure of 3 ints, a vector, or at the very least (and I don't recommend this), dynamically allocate the array contents with new so it persists after the function call (but you better delete it later or the gremlins will come and get you...).
Edit: My apologies, I completely misread the question. Shouldn't be answering StackOverflow before my coffee.
When you want to return an array, or a pointer rather, there are two routes.
One route: new
int* n = new int[3];
n[0] = 0;
// etc..
return n;
Since n is now a heap object, it is up to YOU to delete it later, if you don't delete it, eventually it will cause memory leaks.
Now, route two is a somewhat easier method I find, but it's kind of riskier. It is where you pass an array in and copy the values in.
void copyIDs(int arr[3] /* or int* arr */)
{
arr[0] = workerID;
/* etc */
}
Now your array is populated, and there was no heap allocation, so no problem.
Edit: Returning a local variable as an address is bad. Why?
Given the function:
int* foo() {
int x = 5;
return &x; // Returns the address (in memory) of x
} // At this point, however, x is popped off the stack, so its address is undefined
// (Garbage)
// So here's our code calling it
int *x = foo(); // points to the garbage memory, might still contain the values we need
// But what if I go ahead and do this?
int bar[100]; // Pushed onto the stack
bool flag = true; // Pushed onto the stack
std::cout << *x << '\n'; // Is this guaranteed to be the value we expect?
Overall, it is too risky. Don't do it.
I have
class Foo {
....
}
Is there a way for Foo to be able to separate out:
function blah() {
Foo foo; // on the stack
}
and
function blah() {
Foo foo* = new Foo(); // on the heap
}
I want Foo to be able to do different things depending on whether it's allocated on the Stack or the Heap.
Edit:
Alof of people have asked me "why do this?"
The answer:
I'm using a ref-counted GC right now. However, I want to have ability to run mark & sweep too. For this, I need to tag a set of "root" pointers -- these are the pointers on the stack. Thus, for each class, I'd like to know whether they're in the stack or in the heap.
A hacky way to do it:
struct Detect {
Detect() {
int i;
check(&i);
}
private:
void check(int *i) {
int j;
if ((i < &j) == ((void*)this < (void*)&j))
std::cout << "Stack" << std::endl;
else
std::cout << "Heap" << std::endl;
}
};
If the object was created on the stack it must live somewhere in the direction of the outer functions stack variables. The heap usually grows from the other side, so that stack and heap would meet somewhere in the middle.
(There are for sure systems where this wouldn't work)
You need to actually ask us the real question(a) :-) It may be apparent to you why you think this is necessary but it almost certainly isn't. In fact, it's almost always a bad idea. In other words, why do you think you need to do this?
I usually find it's because developers want to delete or not delete the object based on where it was allocated but that's something that should usually be left to the client of your code rather than your code itself.
Update:
Now that you've clarified your reasons in the question, I apologise, you've probably found one of the few areas in which what you're asking makes sense (running your own garbage collection processes). Ideally, you'd override all the memory allocation and de-allocation operators to keep track of what is created and removed from the heap.
However, I'm not sure it's a simple matter of intercepting the new/delete for the class since there could be situations where delete is not called and, since mark/sweep relies on a reference count, you need to be able to intercept pointer assignments for it to work correctly.
Have you thought about how you're going to handle that?
The classic example:
myobject *x = new xclass();
x = 0;
will not result in a delete call.
Also, how will you detect the fact that the pointer to one of your instances is on the stack? The interception of new and delete can let you store whether the object itself is stack or heap-based but I'm at a loss as to how you tell where the pointer is going to be assigned to, especially with code like:
myobject *x1 = new xclass(); // yes, calls new.
myobject *x2 = x; // no, it doesn't.
Perhaps you may want to look into C++'s smart pointers, which go a long way toward making manual memory management obsolete. Shared pointers on their own can still suffer from problems like circular dependencies but the judicious use of weak pointers can readily solve that.
It may be that manual garbage collection is no longer required in your scenario.
(a) This is known as the X/Y problem. Many times, people will ask a question that pre-supposes a class of solution whereas a better approach would be just to describe the problem with no preconceptions of what the best solution will be.
The answer is no, there is no standard/portable way to do this. Hacks involving overloading the new operator tend to have holes. Hacks that depend on checking pointer addresses are OS specific and heap implementation specific, and may change with future versions of the OS. You may be comfortable with that, but I wouldn't build any sort of system around this behavior.
I would start looking at different ways to accomplish your goal - perhaps you can have a totally different type to serve as the "root" in your scheme, or require the users to (properly) annotate the stack allocated types as such with a special constructor.
It is possible if you compare the value of 'this' with the current value of the stack pointer. If this < sp then you have been allocated in the stack.
Try this out (using gcc in x86-64):
#include <iostream>
class A
{
public:
A()
{
int x;
asm("movq %1, %%rax;"
"cmpq %%rsp, %%rax;"
"jbe Heap;"
"movl $1,%0;"
"jmp Done;"
"Heap:"
"movl $0,%0;"
"Done:"
: "=r" (x)
: "r" (this)
);
std::cout << ( x ? " Stack " : " Heap " ) << std::endl;
}
};
class B
{
private:
A a;
};
int main()
{
A a;
A *b = new A;
A c;
B x;
B *y = new B;
return 0;
}
It should output:
Stack
Heap
Stack
Stack
Heap
A more direct, and less intrusive method would be to look up the pointer in the memory region maps (such as /proc/<pid>/maps). Each thread has a region allocated to its stack. Static and global variables will live in the .bss section, constants in a rodata or const segment, and so on.
I am not positive what you are asking, but overriding the new operator may be what you are trying to do. As the only safe way to create an object on the heap in C++ is to use the new operator, you can differentiate between objects that exist on the heap versus other forms of memory. Google "overloading new in c++" for more information.
You should, however, consider if differentiating between the two types of memory is really necessary from inside the class. Having an object behave differently depending upon where it is stored sounds like a recipe for disaster if you are not careful!
As mentioned above, you need to control how your object is allocated through overloaded new operator. Watch out for two things however, first the 'placement new' operator that initializes your object inside the memory buffer preallocated by user; second, nothing stops the user from simply casting arbitrary memory buffer into your object type:
char buf[0xff]; (Foo*)buf;
Another way is the fact that most runtimes use a bit more memory than asked when doing heap allocations. They usually place some service structure there to identify proper deallocations by pointer. You could inspect your runtime implementation for these patterns, although it will make your code really unportable, dangerous and unsupportable overkill.
Again, as mentioned above, you really are asking for solution details ("how") when you should ask about the initial problem you devised this solution for ("why").
Nope, it can't be done reliably or sensibly.
You may be able to detect when an object is allocated with new by overloading new.
But then what if the object is constructed as a class member, and the owning class is allocated on the heap?
Here's a third code example to add to the two you've got:
class blah {
Foo foo; // on the stack? Heap? Depends on where the 'blah' is allocated.
};
What about static/global objects? How would you tell them apart from stack/heap ones?
You could look at the address of the object, and use that to determine if it is within the range that defines the stack. But the stack may be resized at runtime.
So really, the best answer is that "there's a reason why mark & sweep GC's aren't used with C++".
If you want a proper garbage collector, use a different language, one which supports it.
On the other hand, most experienced C++ programmers find that the need for a garbage collector pretty much vanishes when you learn the necessary techniques for resource management (RAII).
The meta question as asked by pax is asked "why would you want to do that" you'll likely get a more informative answer.
Now assuming you're doing this for "a good reason" (perhaps just curiousity) can get this behaviour by overriding operators new and delete, but don't forget to override all 12 variants including:
new, delete, new no throw, delete no throw, new array, delete array, new array no throw, delete array no throw, placement new, placement delete, placement new array, placement delete array.
One thing you can do is put this in a base class and derive from it.
This is kind of a pain, so what different behavior did you want?
A way for MFC classes:
.H
class CTestNEW : public CObject
{
public:
bool m_bHasToBeDeleted;
__declspec(thread) static void* m_lastAllocated;
public:
#ifdef _DEBUG
static void* operator new(size_t size, LPCSTR file, int line) { return internalNew(size, file, line); }
static void operator delete(void* pData, LPCSTR file, int line) { internalDelete(pData, file, line); }
#else
static void* operator new(size_t size) { return internalNew(size); }
static void operator delete(void* pData) { internalDelete(pData); }
#endif
public:
CTestNEW();
public:
#ifdef _DEBUG
static void* internalNew(size_t size, LPCSTR file, int line)
{
CTestNEW* ret = (CTestNEW*)::operator new(size, file, line);
m_lastAllocated = ret;
return ret;
}
static void internalDelete(void* pData, LPCSTR file, int line)
{
::operator delete(pData, file, line);
}
#else
static void* internalNew(size_t size)
{
CTestNEW* ret = (CTestNEW*)::operator new(size);
return ret;
}
static void internalDelete(void* pData)
{
::operator delete(pData);
}
#endif
};
.CPP
#include "stdafx.h"
.
.
.
#ifdef _DEBUG
#define new DEBUG_NEW
#endif
void* CTestNEW::m_lastAllocated = NULL;
CTestNEW::CTestNEW()
{
m_bHasToBeDeleted = (this == m_lastAllocated);
m_lastAllocated = NULL;
}
Overload new() for your class. This way you'll be able to tell between heap and stack allocation, but not between stack and static/global.
I would recommend using smart pointers instead. By design, the class should have data and information about class. Book-keeping tasks should be delegated outside the class.
overloading new and delete can lead to more holes than you can imagine.
To answer your question, a reliable way (assuming your aplication isn't using more thant one thread), assuming that everithing wich is not contained by your smart pointer isn't on the heap :
-> Overloading new, so that you ca store a list of all blocs allocated, with the size of each block.
-> When the constructor of your smart pointer, search in wich block your this pointer belong. If it isn't in any block, you can say it's "on the stack" (actualy, it means it's not managed by you). Otherwise, you know where and when your pointer was allocated (if you wan't to look for orphan pointers and lasily free memory, or things like that..)
It do not depend from the architechture.
There is a solution, but it forces inheritance. See Meyers, "More Effective C++", Item 27.
EDIT:
Meyers' suggestion is summarized in an article written by Ron van der Wal, which Meyers himself linked to in his blog (in this post):
Tracking heap based objects
As an alternative to the global variable
approach, Meyers presents a HeapTracked class that uses a list to keep
track of the addresses of class instances allocated off the heap, then
uses this information to determine if a particular object resides on
the heap. The implementation goes like this:
class HeapTracked {
// Class-global list of allocated addresses
typedef const void *RawAddress;
static list<RawAddress> addresses;
public:
// Nested exception class
class MissingAddress {};
// Virtual destructor to allow dynamic_cast<>; pure to make
// class HeapTracked abstract.
virtual ~HeapTracked()=0;
// Overloaded operator new and delete
static void *operator new(size_t sz)
{
void *ptr=::operator new(sz);
addresses.push_front(ptr);
return ptr;
}
static void operator delete(void *ptr)
{
// Remove ‘ptr’ from ‘addresses’
list<RawAddress>::iterator it=find(addresses.begin(),
addresses.end(), ptr);
if (it !=addresses.end()) {
addresses.erase(it);
::operator delete(ptr);
} else
throw MissingAddress();
}
// Heap check for specific object
bool isOnHeap() const
{
// Use dynamic cast to get start of object block
RawAddress ptr=dynamic_cast<RawAddress>(this);
// See if it’s in ‘addresses’
return find(addresses.begin(), addresses.end(), ptr) !=
addresses.end();
}
};
// Meyers omitted first HeapTracked:: qualifier...
list<HeapTracked::RawAddress> HeapTracked::addresses;
There is more to read on the original article: Ron van der Wal comments on this suggestion, and then demonstrates other alternative heap tracking methods.
Take a look at the program here: http://alumni.cs.ucr.edu/~saha/stuff/memaddr.html. With a few casts, it ouputs:
Address of main: 0x401090
Address of afunc: 0x401204
Stack Locations:
Stack level 1: address of stack_var: 0x28ac34
Stack level 2: address of stack_var: 0x28ac14
Start of alloca()'ed array: 0x28ac20
End of alloca()'ed array: 0x28ac3f
Data Locations:
Address of data_var: 0x402000
BSS Locations:
Address of bss_var: 0x403000
Heap Locations:
Initial end of heap: 0x20050000
New end of heap: 0x20050020
Final end of heap: 0x20050010