I have two programs. The first allocates a Shared-Memory file and the second reads from it.. I am using placement-new to place objects into this memory guaranteeing that the objects do NOT use new or allocate any memory outside of the Shared-Memory file.
My Array structure:
template<typename T, size_t Size>
struct SHMArray {
SHMArray() : ptr(elements) {}
SHMArray(const SHMArray& other) { std::copy(other.begin(), other.end(), begin()); }
SHMArray(SHMArray&& other)
{
std::swap(other.ptr, ptr);
std::fill_n(ptr.get(), Size, T());
}
~SHMArray()
{
std::fill_n(ptr.get(), Size, T());
}
constexpr bool empty() const noexcept
{
return Size == 0;
}
constexpr size_type size() const noexcept
{
return Size;
}
T& operator[](std::size_t pos)
{
return *(ptr.get() + pos);
}
constexpr const T& operator[](std::size_t pos) const
{
return *(ptr.get() + pos);
}
T* data() noexcept
{
return ptr.get();
}
constexpr const T* data() const noexcept
{
return ptr.get();
}
private:
offset_ptr<T> ptr;
T elements[];
};
Program 1:
int main()
{
//Allocate a shared memory file of 1mb..
auto memory_map = SharedMemoryFile("./memory.map", 1024 * 1024, std::ios::in | std::ios::out);
memory_map.lock();
//Pointer to the shared memory.
void* data = memory_map.data();
//Place the object in the memory..
SHMArray<int, 3>* array = ::new(data) SHMArray<int, 3>();
(*array)[0] = 500;
(*array)[1] = 300;
(*array)[2] = 200;
memory_map.unlock(); //signals other program it's okay to read..
}
Program 2:
int main()
{
//Open the file..
auto memory_map = SharedMemoryFile("./memory.map", 1024 * 1024, std::ios::in | std::ios::out);
memory_map.lock();
//Pointer to the shared memory.
void* data = memory_map.data();
//Place the object in the memory..
//I already understand that I could just cast the `data` to an SHMArray..
SHMArray<int, 3>* array = ::new(data) SHMArray<int, 3>();
for (int i = 0; i < array.size(); ++i)
{
std::cout<<(*array)[i]<<"\n";
}
memory_map.unlock(); //signals other program it's okay to read..
}
Program One placed the SHMArray in memory with placement new. Program Two does the same thing on top of program one's already placed object (overwriting it). Is this undefined behaviour? I don't think it is but I want to confirm.
Neither program calls the destructor array->~SHMVEC(); I also don't think this leaks as long as I close the MemoryMapped file then it should all be fine.. but I want to make sure this is fine. If I ran the programs again on the same file, it shouldn't be a problem.
I am essentially making the assumption that placement new is working as if I placed a C struct in memory in this particular scenario via: struct SHMArray* array = (struct SHMArray*)data;.. Is this correct?
I am essentially making the assumption that placement new is working
as if I placed a C struct in memory in this particular scenario via:
struct SHMArray* array = (struct SHMArray*)data;.. Is this correct?
No, this is not correct. Placement new also invokes the object's appropriate constructor. "struct SHMArray* array = (struct SHMArray*)data;" does not invoke any object's constructor. It's just a pointer conversion cast. Which does not invoke anyone's constructor. Key difference.
In your sample code, you do actually want to invoke the templated object's constructor. Although the shown example has other issues, as already mentioned in the comments, this does appear to be what needs to be done in this particular situation.
But insofar as the equivalent of placement new versus a pointer cast, no they're not the same. One invokes a constructor, one does not. new always invokes the constructor, whether it's placement new, or not. This is a very important detail, that's not to be overlooked.
Related
Lets take custom vector implementation as an example:
template<typename Object>
class myVector {
public:
explicit myVector(int size = 0) :
_size{ size },
_capasity{ size + SPARE_CAPACITY }
{
_buff = new Object[_capasity];
if (_size > 0) {
for (int i = 0; i < _size; i++) {
//_buff[i] = 0;
}
}
}
// more code
private:
Object * _buff = nullptr;
int _size;
int _capasity;
};
So my question is, how to make myVector be value-initialized in case I'll initialize it as:
int main() {
myVector<int> v02(5);
}
Here, it contains 5 int values, so I need it to be all zeros; same with other types. I commented out _buff[i] = 0; as it's specific to int. Please give me some hints.
It's as simple as
for (int i = 0; i < _size; i++)
_buff[i] = Object{};
Alternatively, you could get rid of the loop and add a pair of {} (or ()) here:
_buff = new Object[_capasity]{};
// ^^
But this option would value-initialize all _capasity objects, rather than the first _size ones, as noted by #bipll.
Also, note that if you want to mimic the behavior of std::vector, you need to allocate raw storate (probably std::aligned_storage) and call constructors (via placement-new) and destructors manually.
If Object is a class type, _buff = new Object[_capasity]; calls default constructors for all _capasity objects, rather than for the first _size objects as std::vector does.
Note that when calling
_buff = new Object[_capasity];
(btw, why have you moved this initialization out of init-list, into constructor body?) you already have default-initialized _capasity objects. Default initialization has the following effects here: while elements of scalar type would remain uninitialized (and reading from them UB), for class types you have already called _capasity constructors.
To avoid unnecessary constructions you have the following possible options, among others:
Use std::aligned_alloc to allocate non-initialized memory:
explicit myVector(std::size_t size = 0) :
size_{ size }
, capacity_{ size + SPARE_CAPACITY }
, buff_{std::aligned_alloc(alignof(Object), _capacity)}
{
if(!buff_) throw std::bad_alloc();
if(size) new (buff_) Object[size]{}; // empty braces answer your original query
}
Remember that again buff_ should be aligned_alloced when vector grows (can be std::realloc()ed for trivial types), and in destructor it should be std::free()d — and prior to that size_ objects inside it should be destructed (with an explicit call to ~Object()).
Change buff_'s type to something more trivial yet properly aligned:
using Storage = std::aligned_storage_t<sizeof(Object), alignof(Object)>;
Storage *buff_;
Object *data_ = nullptr;
public:
explicit myVector(std::size_t size = 0) :
size_{ size }
, capacity_{ size + SPARE_CAPACITY }
, buff_{new Storage(_capacity)}
{
if(size) data_ = new (buff_) Object[size]{};
}
Again, in destructor, objects should be manually destroyed, but this time buff_ can be simply delete[]d afterwards.
I currently have a c++ class as follows:
template<class T>
class MyQueue {
T** m_pBuffer;
unsigned int m_uSize;
unsigned int m_uPendingCount;
unsigned int m_uAvailableIdx;
unsigned int m_uPendingndex;
public:
MyQueue(): m_pBuffer(NULL), m_uSize(0), m_uPendingCount(0), m_uAvailableIdx(0),
m_uPendingndex(0)
{
}
~MyQueue()
{
delete[] m_pBuffer;
}
bool Initialize(T *pItems, unsigned int uSize)
{
m_uSize = uSize;
m_uPendingCount = 0;
m_uAvailableIdx = 0;
m_uPendingndex = 0;
m_pBuffer = new T *[m_uSize];
for (unsigned int i = 0; i < m_uSize; i++)
{
m_pBuffer[i] = &pItems[i];
}
return true;
}
};
So, I have this pointer to arrays m_pBuffer object and I was wondering if it is possible to replace this way of doing things with the c++ smart pointer perhaps? I know I can do things like:
std::unique_ptr<T> buffer(new T[size]);
Is using a vector of smart pointers the way to go? Is this recommended and safe?
[EDIT]
Based on the answers and the comments, I have tried to make a thread-safe buffer array. Here it is. Please comment.
#ifndef __BUFFER_ARRAY_H__
#define __BUFFER_ARRAY_H__
#include <memory>
#include <vector>
#include <mutex>
#include <thread>
#include "macros.h"
template<class T>
class BufferArray
{
public:
class BufferArray()
:num_pending_items(0), pending_index(0), available_index(0)
{}
// This method is not thread-safe.
// Add an item to our buffer list
void add(T * buffer)
{
buffer_array.push_back(std::unique_ptr<T>(buffer));
}
// Returns a naked pointer to an available buffer. Should not be
// deleted by the caller.
T * get_available()
{
std::lock_guard<std::mutex> lock(buffer_array_mutex);
if (num_pending_items == buffer_array.size()) {
return NULL;
}
T * buffer = buffer_array[available_index].get();
// Update the indexes.
available_index = (available_index + 1) % buffer_array.size();
num_pending_items += 1;
return buffer;
}
T * get_pending()
{
std::lock_guard<std::mutex> lock(buffer_array_mutex);
if (num_pending_items == 0) {
return NULL;
}
T * buffer = buffer_array[pending_index].get();
pending_index = (pending_index + 1) % buffer_array.size();
num_pending_items -= 1;
}
private:
std::vector<std::unique_ptr<T> > buffer_array;
std::mutex buffer_array_mutex;
unsigned int num_pending_items;
unsigned int pending_index;
unsigned int available_index;
// No copy semantics
BufferArray(const BufferArray &) = delete;
void operator=(const BufferArray &) = delete;
};
#endif
Vector of smart pointers is good idea. It is safe enough inside your class - automatic memory deallocation is provided.
It is not thread-safe though, and it's not safe in regard of handling external memory given to you by simple pointers.
Note that you current implementation does not delete pItems memory in destructor, so if after refactoring you mimic this class, you should not use vector of smart pointers as they will delete memory referenced by their pointers.
On the other side you cannot garantee that noone outside will not deallocate memory for pItems supplied to your Initialize. IF you want to use vector of smart pointers, you should formulate contract for this function that clearly states that your class claims this memory etc. - and then you should rework outside code that calls your class to fit into new contract.
If you don't want to change memory handling, vector of simple pointers is the way to go. Nevertheless, this piece of code is so simple, that there is no real benefit of vector.
Note that overhead here is creation of smart pointer class for each buffer and creation of vector class. Reallocation of vector can take up more memory and happens without your direct control.
The code has two issues:
1) Violation of the rule of zero/three/five:
To fix that you do not need a smart pointer here. To represent a dynamic array with variable size use a std:vector<T*>. That allows you to drop m_pBuffer, m_uSize and the destructor, too.
2) Taking the addresses of elements of a possible local array
In Initialize you take the addresses of the elements of the array pItems passed as argument to the function. Hence the queue does not take ownership of the elements. It seems the queue is a utility class, which should not be copyable at all:
template<class T>
class MyQueue
{
std::vector<T*> m_buffer;
unsigned int m_uPendingCount;
unsigned int m_uAvailableIdx;
unsigned int m_uPendingndex;
public:
MyQueue(T *pItems, unsigned int uSize)
: m_buffer(uSize, nullptr), m_uPendingCount(0), m_uAvailableIdx(0), m_uPendingndex(0)
{
for (unsigned int i = 0; i < uSize; i++)
{
m_buffer[i] = &pItems[i];
}
}
private:
MyQueue(const MyQueue&); // no copy (C++11 use: = delete)
MyQueue& operator = (const MyQueue&); // no copy (C++11 use: = delete)
};
Note:
The red herring is the local array.
You may consider a smart pointer for that, but that is another question.
I have a class that contains several arrays whose sizes can be determined by parameters to its constructor. My problem is that instances of this class have sizes that can't be determined at compile time, and I don't know how to tell a new method at run time how big I need my object to be. Each object will be of a fixed size, but different instances may be different sizes.
There are several ways around the problem:- use a factory- use a placement constructor- allocate arrays in the constructor and store pointers to them in my object.
I am adapting some legacy code from an old application written in C. In the original code, the program figures out how much memory will be needed for the entire object, calls malloc() for that amount, and proceeds to initialize the various fields.
For the C++ version, I'd like to be able to make a (fairly) normal constructor for my object. It will be a descendant of a parent class, and some of the code will be depending on polymorphism to call the right method. Other classes descended from the same parent have sizes known at compile time, and thus present no problem.
I'd like to avoid some of the special considerations necessary when using placement new, and I'd like to be able to delete the objects in a normal way.
I'd like to avoid carrying pointers within the body of my object, partially to avoid ownership problems associated with copying the object, and partially because I would like to re-use as much of the existing C code as possible. If ownership were the only issue, I could probably just use shared pointers and not worry.
Here's a very trimmed-down version of the C code that creates the objects:
typedef struct
{
int controls;
int coords;
} myobject;
myobject* create_obj(int controls, int coords)
{
size_t size = sizeof(myobject) + (controls + coords*2) * sizeof(double);
char* mem = malloc(size);
myobject* p = (myobject *) mem;
p->controls = controls;
p->coords = coords;
return p;
}
The arrays within the object maintain a fixed size of the life of the object. In the code above, memory following the structure of myobject will be used to hold the array elements.
I feel like I may be missing something obvious. Is there some way that I don't know about to write a (fairly) normal constructor in C++ but be able to tell it how much memory the object will require at run time, without resorting to a "placement new" scenario?
How about a pragmatic approach: keep the structure as is (if compatibility with C is important) and wrap it into a c++ class?
typedef struct
{
int controls;
int coords;
} myobject;
myobject* create_obj(int controls, int coords);
void dispose_obj(myobject* obj);
class MyObject
{
public:
MyObject(int controls, int coords) {_data = create_obj(controls, coords);}
~MyObject() {dispose_obj(_data);}
const myobject* data() const
{
return _data;
}
myobject* data()
{
return _data;
}
int controls() const {return _data->controls;}
int coords() const {return _data->coords;}
double* array() { return (double*)(_data+1); }
private:
myobject* _data;
}
While I understand the desire to limit the changes to the existing C code, it would be better to do it correctly now rather than fight with bugs in the future. I suggest the following structure and changes to your code to deal with it (which I suspect would mostly be pulling out code that calculates offsets).
struct spots
{
double x;
double y;
};
struct myobject
{
std::vector<double> m_controls;
std::vector<spots> m_coordinates;
myobject( int controls, int coordinates ) :
m_controls( controls ),
m_coordinates( coordinates )
{ }
};
To maintain the semantics of the original code, where the struct and array are in a single contigious block of memory, you can simply replace malloc(size) with new char[size] instead:
myobject* create_obj(int controls, int coords)
{
size_t size = sizeof(myobject) + (controls + coords*2) * sizeof(double);
char* mem = new char[size];
myobject* p = new(mem) myobject;
p->controls = controls;
p->coords = coords;
return p;
}
You will have to use a type-cast when freeing the memory with delete[], though:
myobject *p = create_obj(...);
...
p->~myobject();
delete[] (char*) p;
In this case, I would suggest wrapping that logic in another function:
void free_obj(myobject *p)
{
p->~myobject();
delete[] (char*) p;
}
myobject *p = create_obj(...);
...
free_obj(p);
That being said, if you are allowed to, it would be better to re-write the code to follow C++ semantics instead, eg:
struct myobject
{
int controls;
int coords;
std::vector<double> values;
myobject(int acontrols, int acoords) :
controls(acontrols),
coords(acoords),
values(acontrols + acoords*2)
{
}
};
And then you can do this:
std::unique_ptr<myobject> p = std::make_unique<myobject>(...); // C++14
...
std::unique_ptr<myobject> p(new myobject(...)); // C++11
...
std::auto_ptr<myobject> p(new myobject(...)); // pre C++11
...
New Answer (given comment from OP):
Allocate a std::vector<byte> of the correct size. The array allocated to back the vector will be contiguous memory. This vector size can be calculated and the vector will manage your memory correctly. You will still need to be very careful about how you manage your access to that byte array obviously, but you can use iterators and the like at least (if you want).
By the way here is a little template thing I use to move along byte blobs with a little more grace (note this has aliasing issues as pointed out by Sergey in the comments below, I'm leaving it here because it seems to be a good example of what not to do... :-) ) :
template<typename T>
T readFromBuf(byte*& ptr) {
T * const p = reinterpret_cast<T*>(ptr);
ptr += sizeof(T);
return *p;
}
Old Answer:
As the comments suggest, you can easily use a std::vector to do what you want. Also I would like to make another suggestion.
size_t size = sizeof(myobject) + (controls + coords*2) * sizeof(double);
The above line of code suggests to me that you have some "hidden structure" in your code. Your myobject struct has two int values from which you are calculating the size of what you actually need. What you actually need is this:
struct ControlCoord {
double control;
std::pair<double, double> coordinate;
};
std::vector<ControlCoord>> controlCoords;
When the comments finally scheded some light on the actual requirements, the solution would be following:
allocate a buffer large enough to hold your object and the array
use placement new in the beginning of the buffer
Here is how:
class myobject {
myobject(int controls, int coords) : controls(controls), coords(coords) {}
~myobject() {};
public:
const int controls;
const int coords;
static myobject* create(int controls, int coords) {
std::unique_ptr<char> buffer = new char[sizeof(myobject) + (controls + coords*2) * sizeof(double)];
myobject obj* = new (buffer.get()) myobject(controls, coords);
buffer.release();
return obj;
}
void dispose() {
~myobject();
char* p = (char*)this;
delete[] p;
}
};
myobject *p = myobject::create(...);
...
p->dispose();
(or suitably wrapped inside deleter for smart pointer)
I'm trying to make a mechanism that could tell where the object of the class is allocated.
Thought about making a flag in the class, but it's not possible to set a value because object's lifetime is not started during the call of "new" operator.
Is it possible in C++ to tell if an object is on stack or heap (runtime)?
There is no portable way to do this, but if we assume you have a limited amount of system types where you are going to do this on, you could try the following:
Take the address of some local variable in main (or other "low in the callstack"). Store this in a global variable, lets call char *stackbase;
Then take the address of a local variable in your function that you are checking in, let's call it char *stacktop;
Now, if we have a char *obj = reinterpret_cast<char *>(object_in_test);, then:
if (obj > stacktop && obj < stackbase) on_stack = true;
else on_stack = false;
Note that there are SEVERAL flaws with this:
It's technically undefined behaviour. It will work on most systems, because the whole memory space is contiguous. But there are systems where the stack and other sections of memory have separate "address spaces", which means that two pointers to different types of memory can have the same address.
Threads will need to have a "per thread stackbase".
The stack is assumed to "grow towards zero" (if not, you'll have to invert the > and < in the if.
Global variables will be seen as not on stack.
USE AT YOUR OWN RISK!
I fully expect to have to delete this answer as it will be downvoted by language lawyers, despite the disclaimer below.
I have been doing some experimentation, and have discovered that this seems to work for being able to always tell at runtime if an object was allocated on the stack or not.
The interface is as follows:
#ifndef HEAPAWARE_H
#define HEAPAWARE_H
#include <cintttypes>
class HeapAware
{
public:
HeapAware();
void *operator new(std::size_t size);
void *operator new[](std::size_t size);
void operator delete(void *ptr, std::size_t);
void operator delete[](void *ptr, std::size_t);
bool is_on_heap() const { return on_heap; }
std::ptrdiff_t get_heap_array_index() const { return heap_array_index; }
private:
const bool on_heap;
const std::ptrdiff_t heap_array_index;
static thread_local HeapAware * last_alloc;
static thread_local std::size_t allocated_size;
};
#endif
And the implementation is:
void *HeapAware::operator new(std::size_t size)
{
auto result = last_alloc = reinterpret_cast<HeapAware*>(malloc(size));
allocated_size = 1;
return result;
}
void *HeapAware::operator new[](std::size_t size)
{
auto result = last_alloc = reinterpret_cast<HeapAware*>(malloc(size));
allocated_size = size;
return result;
}
void HeapAware::operator delete(void *ptr, std::size_t)
{
free(ptr);
}
void HeapAware::operator delete[](void *ptr, std::size_t)
{
free(ptr);
}
HeapAware::HeapAware():on_heap(this>=last_alloc && this<last_alloc+allocated_size),heap_array_index(allocated_size>1?this-last_alloc:-1)
{
}
thread_local HeapAware * HeapAware::last_alloc = nullptr;
thread_local std::size_t HeapAware::allocated_size = 0;
This seems to always work correctly. For arrays allocated on the heap, the index of the entry is also available. For values that are allocated on the stack, or for entries that are just allocated singly, the get_heap_array_index() function returns -1.
The assumption that this code makes is that the new operator is called immediately before construction on any given thread. This assumption seems to hold true for everything I have tried, however.
EDITED: reworded question.
When new and malloc are called, the size of the block of memory to be allocated is passed:
void* malloc(size_t);
void* operator new(size_t);
is it possible to get type information, i.e. so you could do a sizeof(T) where T was the type the memory was being allocated for:
T t = new T;
I'd like to overload new and malloc but require the type information not just the size of the memory being allocated.
FURTHER EDIT:
The reason I am doing this is that I will overload malloc. This function will inspect the size of the memory being allocated and allocate from a particualr memory pool:
template <int v> struct int2type { value = v };
inline void* malloc(const std::size_t sz)
{
if(sz <= 64)
{
size_obj* ptr = malloc(nggt::core::int2type<sizeof(size_obj) + (sz % 8)>);
ptr->sz = sz;
return ptr+1;
}
else if(sz <= 128)
{
size_obj* ptr = malloc(nggt::core::int2type<sizeof(size_obj) + 128>);
ptr->sz = sz;
return ptr+1;
}
else
{
return TSuper::malloc(sz);
}
}
inline void* malloc(const nggt::core::int2type<sizeof(size_obj) + 8> sz)
{
return m_heap8.malloc(sz.value);
}
inline void* malloc(const nggt::core::int2type<sizeof(size_obj) + 16> sz)
{
return m_heap16.malloc(sz.value);
}
There are also supporintg free overloads using a freelist to return memory to a pool. Problem is I can't use size_t sz in a template as it's not known at compile time. If I could get the type information I could do sizeof(T) and be done!
Cheers,
Graeme
The new with the std::size_t argument is commonly known as non-placement new. The std::size_t argument is passed in by default to let the operating system know how many bytes to allocate for the object you are creating. You, yourself, don't need to provide it to the new declaration.
Edit:
In response to your edit. It's common practice if you are overloading the new operator to
place it within the namespace you would like it to be called from
Avoid a namespace new if you can make specific news for your objects. (that is, there is behavior specific to each class, why not place it with the class itself)
The first argument should be the std::size_t which you don't need to worry about. It'll be handled for you.
Everything else after that is all yours:
struct MyObject{
MyObject* operator new(std::size_t s, const char* message){
cout << "Creating an object of size " << size_of(MyObject) << endl;
return new MyObject();
}
};
You can even use the "old" new in your call of your "new" new.
Further Edit:
In response to your next edit, don't call your own malloc, malloc. Instead, call it some other name. If you're writing C++, malloc has no place within your code what so ever unless you're dealing with some legacy application. In C++, you should only be touching the new operator.