c++ overloading new[], get number of requested objects - c++

I want to overload the operator new[] to get the number of how many objects are to be created.
the best possible solution is to overload new[] in a way that the when creating an object using new[] would not necessitate the caller to write any extra code.
i.e using simply:
Myclass objectsArray = new Myclass[5];
example overload:
class Myclass
{
...
public:
void* operator new[](size_t size)
{
int numberOfObjects = figure out how to get number of objects to be created by this call to new[]
std::cout << "requesting: " << numberOfObjects << " objects" << std::endl;
void * temp = malloc(size) ;
return temp ;
}
...
}
the following example program:
Myclass objectsArray = new Myclass[4];
would output:
requesting: 4 objects
edit:
I want to do this just in MyClass not globaly
edit:
I'm on ‎Lubuntu 12.0 with GCC version 4.8.1
edit: Purpose of the question:
I'm using a custom memory pool allocator that can't allocate arrays out of the box. so I need to know how many objects are requested and create them one by one using a loop with the pool allocator.

You need to define a class-specific operator new. This works for me, but beware of alignment, derived classes sizes, and allocation overhead issues in the count/sizeof(MyClass) expression:
#include <iostream>
#include <memory>
#include <new>
struct MyClass {
int dummy[8];
void *operator new[](std::size_t count) {
std::cout << "allocating " << count << " bytes, " << ((count-sizeof(void*))/sizeof(MyClass)) << " items\n";
return ::operator new[](count);
}
};
int main() {
auto x = std::make_unique<MyClass[]>(10);
}
(prints: allocating 10 items) I think one can determine the exact overhead for your compiler/stdlib combination empirically by doing something like:
#include <iostream>
#include <memory>
#include <new>
struct MyClass {
char c;
char d;
virtual ~MyClass() {} // needed for introducing overhead
void *operator new[](std::size_t count) {
void *p = ::operator new[](count);
std::cout << "allocating " << count << " bytes\n";
return p;
}
};
int main() {
for( auto count : { 1, 5, 10, 20 } ) {
std::cout << "allocating " << sizeof(MyClass) << " * " << count << " items\n";
auto x = new MyClass[count];
delete[] x;
}
}
This will print, in my machine (and on Coliru):
allocating 16 * 1 items
allocating 24 bytes, 1 items
allocating 16 * 5 items
allocating 88 bytes, 5 items
allocating 16 * 10 items
allocating 168 bytes, 10 items
allocating 16 * 20 items
allocating 328 bytes, 20 items
So, our count/sizeof(MyClass) (here) should be something like
(count-sizeof(void*))/sizeof(MyClass)
But YMMV, etc.
LAST EDIT (I hope)
I had success on 64-bits linux (clang-3.4, clang-3.5 & gcc-4.8, libstdc++ & libc++) with the following how_many_elements function:
template<class T>
std::size_t how_many_elements(std::size_t bytes) {
return (bytes -
((!std::is_trivially_destructible<T>::value)*sizeof(void*))
) / sizeof(T);
}
So your operator new becomes something like:
void *operator new[](std::size_t count) {
std::cout << "allocating " << count << " bytes, " << how_many_elements<MyClass>(count) << " items\n";
return ::operator new[](count);
}
But I have not tested it in 32-bits environments and the YMMV still applies. And remember to replicate the operator new for every derived class of MyClass, obviously replacing MyClass for MyDerivedClass or whatever...

Related

Will moving from released pointers leak memory?

I have the following code:
std::unique_ptr<T> first = Get();
…
T* ptr_to_class_member = GetPtr(obj);
*ptr_to_class_member = std::move(*first.release());
Will this behave as expected with no copies, 1 move and without memory leak?
*ptr_to_class_member = std::move(*first.release());
just calls the move assignment operator of T with the object pointed to by first as argument. This may properly transfer some data, but delete is not called or the object so neither T::~T is executed nor does the memory of the object get freed.
In the example of T = std::string this would result in the backing storage of the string object properly being transfered from the rhs to the lhs of the move assignment, but dynamically allocated memory of size sizeof(std::string) would still be leaked.
For some classes the lack of a destructor invocation for the object could result in additional trouble, since move assignment simply needs to leave the rhs in an unspecified, but valid state which could still require freeing of additional resources.
You need to do
*ptr_to_class_member = std::move(*first);
first.reset();
in order to prevent memory leaks.
To show what's going wrong here, the following code implements prints for memory (de)allocation and special member functions:
#include <iostream>
#include <memory>
#include <new>
#include <utility>
struct TestObject
{
TestObject()
{
std::cout << "TestObject::TestObject() : " << this << '\n';
}
TestObject(TestObject&& other)
{
std::cout << "TestObject::TestObject(TestObject&&) : " << this << ", " << &other << '\n';
}
TestObject& operator=(TestObject&& other)
{
std::cout << "TestObject::operator=(TestObject&&) : " << this << ", " << &other << '\n';
return *this;
}
~TestObject()
{
std::cout << "TestObject::~TestObject() : " << this << '\n';
}
void* operator new(size_t size)
{
void* const result = ::operator new(size);
std::cout << "memory allocated for TestObject: " << result << '\n';
return result;
}
void operator delete(void* mem)
{
std::cout << "memory of TestObject deallocated: " << mem << '\n';
::operator delete(mem);
}
};
template<class Free>
void Test(Free free, char const* testName)
{
std::cout << testName << " begin -------------------------------------------\n";
{
auto ptr = std::make_unique<TestObject>();
std::cout << "object creation done\n";
free(ptr);
}
std::cout << testName << " end ---------------------------------------------\n";
}
int main()
{
TestObject lhs;
Test([&lhs](std::unique_ptr<TestObject>& ptr)
{
lhs = std::move(*ptr);
ptr.reset();
}, "good");
Test([&lhs](std::unique_ptr<TestObject>& ptr)
{
lhs = std::move(*ptr.release());
}, "bad");
}
Possible output:
TestObject::TestObject() : 0000009857AFF994
good begin -------------------------------------------
memory allocated for TestObject: 000001C1D5715EF0
TestObject::TestObject() : 000001C1D5715EF0
object creation done
TestObject::operator=(TestObject&&) : 0000009857AFF994, 000001C1D5715EF0
TestObject::~TestObject() : 000001C1D5715EF0
memory of TestObject deallocated: 000001C1D5715EF0
good end ---------------------------------------------
bad begin -------------------------------------------
memory allocated for TestObject: 000001C1D5715EF0
TestObject::TestObject() : 000001C1D5715EF0
object creation done
TestObject::operator=(TestObject&&) : 0000009857AFF994, 000001C1D5715EF0
bad end ---------------------------------------------
TestObject::~TestObject() : 0000009857AFF994
You can clearly see the destructor call and deallocation missing in the second case, which is the one matching the code you're asking about.

double free while using vector and strings

I wrote the following function and a simple class, while trying to understand how expensive a work with vector can be.
void gen_random(string & str, const int len)
{
static const char alphanum[] =
"0123456789"
"ABCDEFGHIJKLMNOPQRSTUVWXYZ"
"abcdefghijklmnopqrstuvwxyz";
srand((unsigned int)time(NULL));
str.reserve(len);
for (int i = 0; i < len; ++i)
{
str += alphanum[rand() % (sizeof(alphanum) - 1)];
}
}
class Person
{
public:
//CTOR with parameter
Person(u_int32_t Id)
{
std::cout << "\033[1;32mPerson CTOR: " << Id << "\033[0m" << std::endl;
m_Id = Id;
m_RandSid = new string;
gen_random(*m_RandSid, 10);
}
//CCTOR
Person(const Person & p)
{
std::cout << "\033[1;31mPerson CCTOR: " << p.m_Id << "\033[0m" << std::endl;
m_Id = p.m_Id;
m_RandSid = p.m_RandSid; //trigger string operator=()
}
//MCTOR
Person(Person&& p)
{
std::cout << "\033[1;34mPerson MCTOR: " << p.m_Id << "\033[0m" << std::endl;
m_Id = p.m_Id;
m_RandSid = p.m_RandSid;
p.m_RandSid = nullptr;
}
//DTOR
~Person()
{
std::cout << "\033[1;33mPerson DTOR: "<<m_Id <<"\033[0m"<< std::endl;
if (nullptr != m_RandSid)
{
delete m_RandSid;
}
}
u_int32_t m_Id;
string * m_RandSid;
};
and driver :
int main()
{
int a;
vector<Person> v;
for (int i = 0; i < 2; ++i)
{
std::cout <<std::endl<< "inserting person #" << i << std::endl;
std::cout << "Vector size = " << v.size()<< " Vector capacity = " << v.capacity() << std ::endl;
v.emplace_back(i);
std::cout << *v[i].m_RandSid << std::endl;
std::cout << "Vector size = " << v.size()<< " Vector capacity = " << v.capacity() << std ::endl;
}
std::cout<<std::endl<<"--------------------------------------------------------"<<std::endl;
return 0;
}
when I run this program, I the following output:
inserting person #0
Vector size = 0 Vector capacity = 0
Person CTOR: 0
07QoUmgEe6
Vector size = 1 Vector capacity = 1
inserting person #1
Vector size = 1 Vector capacity = 1
Person CTOR: 1
Person CCTOR: 0
Person DTOR: 0
07QoUmgEe6
Vector size = 2 Vector capacity = 2
--------------------------------------------------------
Person DTOR: 0
free(): double free detected in tcache 2
I don't understand where else I perform another free :\
The other problem is that the string is randomized per execution and not per object,
If srand is performed on each execution, why all the string look the same ?
In your copy and move constructor you simply copy raw pointers, which makes 2 pointers to point to the same memory, and when both objects destroyed you get double deallocation:
Person(const Person & p)
{
std::cout << "\033[1;31mPerson CCTOR: " << p.m_Id << "\033[0m" << std::endl;
m_Id = p.m_Id;
m_RandSid = p.m_RandSid; // now both pointers point to the same memory
}
It is not clear why you need dynamically allocated string objects, you should just store objects by value, but if you do need that you should use smart pointers (either std::shared_ptr or std::unique_ptr depends of what ownership you need). That will not only make your problem disappear but you would not have to provide copy and move constructor manually, compiler generated ones would be sufficient.
Note, your class is also missing proper copy and move assignment operator, though it is not exposed in code shown it still violates the rule of three/five/zero and you may have problems with your code later.
Your copy constructor copies the pointer value, and should be doing a deep copy (allocate a new string). In a simple design like this I would avoid using new/free for the string.
You initialize (actually reset) rand with time with second precision - time(). Your app probably finishes in less than a second, hence the similar strings. Initialize rand only once, eg when starting the app.

Memory allocation for objects in arrays

so this is my code [ notice that I commented the cstr and destructor]
#include <iostream>
#include <stdlib.h>
#include <array>
class MyIntClass
{
int _mymember;
public:
// MyIntClass(){}
// ~MyIntClass(){}
void *operator new(size_t size)
{
std::cout << "new: Allocating " << size << " bytes of memory" << std::endl;
void *p = malloc(size);
return p;
}
void operator delete(void *p)
{
std::cout << "delete: Memory is freed again " << std::endl;
free(p);
}
void *operator new[](size_t size)
{
std::cout << "new: Allocating " << size << " bytes of memory" << std::endl;
void *p = malloc(size);
return p;
}
void operator delete[](void *p)
{
std::cout << "delete: Memory is freed again " << std::endl;
free(p);
}
};
void line(){
std::cout << "\n--------------------------------------------------\n" << std::endl;
}
int main()
{
line();
std::cout << "Using new overloading and malloc\nWe will create one object of MyIntClass that is supposed to be 4 bytes" << std::endl;
MyIntClass *m1 = new MyIntClass();
line();
//I want to create an array of the MyIntClass of two objects
std::cout << "Now we create array of MyIntClass using <array> header" << std::endl;
std::array<MyIntClass, 2> z = {};
std::cout << " The elements in the array z = "<< z.size() <<std::endl;
std::cout << "The memory allocated for array z = " << sizeof(z) << std::endl;
line();
std::cout << "\nNow we create array using new[] overloading and malloc " << std::endl;
MyIntClass *i = new MyIntClass[2]();
delete[] i;
}
now the result is as follow:
Using new overloading and malloc
We will create one object of MyIntClass that is supposed to be 4 bytes
new: Allocating 4 bytes of
memory
Now we create array of MyIntClass using <array> header
The elements in the array z = 2
The memory allocated for array z = 8
Now we create array using new[] overloading and malloc
new: Allocating 8 bytes of memory
To me as inexperienced C++ programmer I think every thing is working as expected
Now If I uncomment the constructor the same result will happen
however when I uncomment the destructor different result will occur
Now we create array using new[] overloading and malloc
new: Allocating 12 bytes of memory
So my question is what is the explanation for this:
creating array of two objects each is 4 bytes will result in 8byte memory allocation for the array in both methods using the array library or the overloading the new[] and malloc.
however when we have a destructor for the object the malloc will allocate 12 bytes not 8 bytes for this array of 2 elements.
I saw this question on SO but it didn't explain my case
following is my compiler version :
gcc version 8.2.0 (MinGW.org GCC-8.2.0-3)
The reason for extra allocated memory is that the compiler needs to know the number of elements in the array in order to be able to call destructors on each element of the array when you call delete[]. For trivially destructible types the compiler doesn't need to call destructor, so the extra space for the array size is not needed.

Any way to detect whether an object of my class is create on stack?

Now I need to detect whether my class is created as a stack/global/thread_local variable, for example:
class Foo {
public:
Foo() {
if(im_on_stack) {
std::cout << "I'm on stack" << std::endl;
} else if(im_in_global) {
std::cout << "I'm in global" << std::endl;
} else if(im_a_thread_local) {
std::cout << "I'm a thread_local" << std::endl;
} else {
std::cout << "I'm on ohters location" << std::endl;
}
}
};
class Bar {
Foo mFoo;
};
Foo gFoo;
thread_local Foo tFoo;
int main() {
Foo lFoo;
}
and the out put should be:
I'm on ohters location
I'm in global
I'm a thread_local
I'm on stack
This there any way in C++ I can do this?
Edit:
why I'm doing this:
I'm writing a garbage collection library, and I got a class, let's call it gc_ptr, I need to know if this gc_ptr is a gc root (which is create on the location I mentioned) or not (which is a member of another class)
Edit2:
According to the concept of gc root, which is a reference which is not on a heap, I should probably asked in this way: can I detect if my class is create on the heap? But I think on heap or on stack make this question no difference.
Short answer: No. Not with standard C++. There may be compiler or OS specific solutions, but nothing portable.
I guess you could make a heuristic to detect stack allocated objects by detecting in their constructor whether their address is close to a stack allocated variable. Assuming that the stack and the heap have completely different memory addresses it should work. Completely undefined behaviour though according to the standard. e.g.:
#include <iostream>
#include <memory>
struct A
{
A()
{
int test = 0; // test is on the stack
auto distance = reinterpret_cast<char*>(this) - reinterpret_cast<char*>(&test);
isStack = std::abs(distance) < 1024; // completely arbitrary magic number, will need to experiment
}
bool isStack;
};
int main()
{
std::cout << "stack: " << A().isStack << "\n";
std::cout << "stack: " << std::make_unique<A>()->isStack << "\n";
}
I don't think you could expand this technique to thread local variables. You'd also need to take care in copy constructors and assignment operators to handle copying from a stack to a heap object and vice versa.
This is not part of the spec, so no.
If you told us on what system/os you are working, maybe we can assist - some systems may provide the stack address and size - usually this is available in embedded devices as part of the compiler output (the address), and as input (the size) as part of the environment/project configuration.
What I wrote in my comment:
Can be done to some extent: Assuming, you need it only for a certain kind of classes: You have to overload new (or wrap dynamic construction in a static create). You have to derive all considered classes from a base class with a specific constructor. To pass info from new to the constructor is a bit tricky. In our case, we used a global set where new remembered pointers to created instances and the corresponding constructor looked into it to determine whether creation was done by new. That was sufficient for us. (About the other topics - no idea...)
A demo:
#include <iostream>
#include <set>
class Object {
private:
static thread_local std::set<void*> pNewSet;
bool _isNewed;
public:
Object();
Object(const Object&) = delete;
const Object& operator=(const Object&) = delete;
~Object() = default;
static void* operator new(size_t size);
bool isNewed() const { return _isNewed; }
private:
static std::set<void*>& getNewPtrs()
{
static thread_local std::set<void*> pNewSet;
return pNewSet;
}
};
void* Object::operator new(size_t size)
{
std::set<void*> &pNewSet = getNewPtrs();
void *p = ::operator new(size);
if (p) pNewSet.insert(p);
return p;
}
Object::Object(): _isNewed(false)
{
std::set<void*> &pNewSet = getNewPtrs();
std::set<void*>::iterator iter = pNewSet.find((void*)this);
if (iter != pNewSet.end()) {
_isNewed = true;
pNewSet.erase(iter);
}
}
#define DEBUG(...) std::cout << #__VA_ARGS__ << ";\n"; __VA_ARGS__
// a global static instance
static Object o;
int main()
{
DEBUG(std::cout << o.isNewed() << '\n');
// a static instance (local scope)
DEBUG(static Object o1);
DEBUG(std::cout << o1.isNewed() << '\n');
// a local instance
DEBUG(Object o2);
DEBUG(std::cout << o2.isNewed() << '\n');
// as members
DEBUG(struct Composed { Object o1, o2; } comp);
DEBUG(std::cout << comp.o1.isNewed() << ' ' << comp.o2.isNewed() << '\n');
// created with new
DEBUG(Object *pO = new Object());
DEBUG(std::cout << pO->isNewed() << '\n');
DEBUG(delete pO);
// created as members in an object created with new
DEBUG(Composed *pComp = new Composed());
DEBUG(std::cout << pComp->o1.isNewed() << ' ' << pComp->o2.isNewed() << '\n');
DEBUG(delete pComp);
}
Output:
std::cout << o.isNewed() << '\n';
0
static Object o1;
std::cout << o1.isNewed() << '\n';
0
Object o2;
std::cout << o2.isNewed() << '\n';
0
struct Composed { Object o1, o2; } comp;
std::cout << comp.o1.isNewed() << ' ' << comp.o2.isNewed() << '\n';
0 0
Object *pO = new Object();
std::cout << pO->isNewed() << '\n';
1
delete pO;
Composed *pComp = new Composed();
std::cout << pComp->o1.isNewed() << ' ' << pComp->o2.isNewed() << '\n';
0 0
delete pComp;
Live Demo on Compiler Explorer
Notes:
The copy constructor and assignment operator of Object are deleted intenionally.
Derived classes may provide a copy constructor but it has to call Object::Object().
The sample doesn't consider other flavors of new (e.g. operator new[] or the align variants since C++17). That should be done in productive code.
No, but you can prevent creation of objects on heap/stack.
To prevent creation on stack make the destructor private/protected:
class heap_only
{
public:
void release() const { delete this; }
protected:
~heap_only() {}
};
struct heap_only_deleter
{
void operator()( heap_only* p ) { p->release(); }
};
using up_heap_only = std::unique_ptr<heap_only, heap_only_deleter>;
//...
heap_only ho; // fails
heap_only* pho = new heap_only();
pho->release();
up_heap_only up{ new heap_only() };
To prevent creation on heap make the new operators private/protected:
class stack_only
{
protected:
static void* operator new( std::size_t );
static void* operator new[]( std::size_t );
};
//...
stack_only* pso = new stack_only(); // fails
stack_only so;
Regarding your edits, and just for fun:
void* pstart;
bool is_on_stack( void* p )
{
int end;
return pstart >= p && p > &end;
}
int globalint;
int main()
{
//
int start;
pstart = &start;
//
int stackint;
std::cout << std::boolalpha
<< is_on_stack( &globalint ) << std::endl
<< is_on_stack( &stackint ) << std::endl
<< is_on_stack( new int );
//...
}

Where is the `size` coming from?

I know there should be a delete operator with which I'm not concerned in somewhere. I just wondering, wow, it worked. Where is the argument "size" coming from?
#include<iostream>
#include<string>
class Base {
public:
Base() { }
void *operator new( unsigned int size, std::string str ) {
std::cout << "Logging an allocation of ";
std::cout << size;
std::cout << " bytes for new object '";
std::cout << str;
std::cout << "'";
std::cout << std::endl;
return malloc( size );
}
private:
int var1;
double var2;
};
int main(int argc, char** argv){
Base* b = new ("Base instance 1") Base;
}
Here is the result:
Logging an allocation of 16 bytes for new object 'Base instance 1'
It is provided by the compiler at compile time. When the compiler sees:
new ("Base instance 1") Base;
it will add a call to:
Base::operator new(sizeof(Base), "Base instance 1");
EDIT: The compiler will of course also add a call to Base::Base()
on 32 bit arch int is 4 bytes, double is 8, but the double is going to be aligned to 8 byte boundary so size = 4 + 4(empty space) + 8 = 16