How can I realloc in C++? It seems to be missing from the language - there is new and delete but not resize!
I need it because as my program reads more data, I need to reallocate the buffer to hold it. I don't think deleteing the old pointer and newing a new, bigger one, is the right option.
Use ::std::vector!
Type* t = (Type*)malloc(sizeof(Type)*n)
memset(t, 0, sizeof(Type)*m)
becomes
::std::vector<Type> t(n, 0);
Then
t = (Type*)realloc(t, sizeof(Type) * n2);
becomes
t.resize(n2);
If you want to pass pointer into function, instead of
Foo(t)
use
Foo(&t[0])
It is absolutely correct C++ code, because vector is a smart C-array.
The right option is probably to use a container that does the work for you, like std::vector.
new and delete cannot resize, because they allocate just enough memory to hold an object of the given type. The size of a given type will never change. There are new[] and delete[] but there's hardly ever a reason to use them.
What realloc does in C is likely to be just a malloc, memcpy and free, anyway, although memory managers are allowed to do something clever if there is enough contiguous free memory available.
Resizing in C++ is awkward because of the potential need to call constructors and destructors.
I don't think there's a fundamental reason why in C++ you couldn't have a resize[] operator to go with new[] and delete[], that did something similar to this:
newbuf = new Type[newsize];
std::copy_n(oldbuf, std::min(oldsize, newsize), newbuf);
delete[] oldbuf;
return newbuf;
Obviously oldsize would be retrieved from a secret location, same is it is in delete[], and Type would come from the type of the operand. resize[] would fail where the Type is not copyable - which is correct, since such objects simply cannot be relocated. Finally, the above code default-constructs the objects before assigning them, which you would not want as the actual behaviour.
There's a possible optimisation where newsize <= oldsize, to call destructors for the objects "past the end" of the newly-ensmallened array and do nothing else. The standard would have to define whether this optimisation is required (as when you resize() a vector), permitted but unspecified, permitted but implementation-dependent, or forbidden.
The question you should then ask yourself is, "is it actually useful to provide this, given that vector also does it, and is designed specifically to provide a resize-able container (of contiguous memory--that requirement omitted in C++98 but fixed in C++03) that's a better fit than arrays with the C++ ways of doing things?"
I think the answer is widely thought to be "no". If you want to do resizeable buffers the C way, use malloc / free / realloc, which are available in C++. If you want to do resizeable buffers the C++ way, use a vector (or deque, if you don't actually need contiguous storage). Don't try to mix the two by using new[] for raw buffers, unless you're implementing a vector-like container.
Here's a std::move example implementing a simple vector with a realloc (*2 each time we hit the limit). If there's a way to do better than the copy I have below, pls let me know.
Compile as:
g++ -std=c++2a -O2 -Wall -pedantic foo.cpp
Code:
#include <iostream>
#include <algorithm>
template<class T> class MyVector {
private:
T *data;
size_t maxlen;
size_t currlen;
public:
MyVector<T> () : data (nullptr), maxlen(0), currlen(0) { }
MyVector<T> (int maxlen) : data (new T [maxlen]), maxlen(maxlen), currlen(0) { }
MyVector<T> (const MyVector& o) {
std::cout << "copy ctor called" << std::endl;
data = new T [o.maxlen];
maxlen = o.maxlen;
currlen = o.currlen;
std::copy(o.data, o.data + o.maxlen, data);
}
MyVector<T> (const MyVector<T>&& o) {
std::cout << "move ctor called" << std::endl;
data = o.data;
maxlen = o.maxlen;
currlen = o.currlen;
}
void push_back (const T& i) {
if (currlen >= maxlen) {
maxlen *= 2;
auto newdata = new T [maxlen];
std::copy(data, data + currlen, newdata);
if (data) {
delete[] data;
}
data = newdata;
}
data[currlen++] = i;
}
friend std::ostream& operator<<(std::ostream &os, const MyVector<T>& o) {
auto s = o.data;
auto e = o.data + o.currlen;;
while (s < e) {
os << "[" << *s << "]";
s++;
}
return os;
}
};
int main() {
auto c = new MyVector<int>(1);
c->push_back(10);
c->push_back(11);
}
Related
I have two programs. The first allocates a Shared-Memory file and the second reads from it.. I am using placement-new to place objects into this memory guaranteeing that the objects do NOT use new or allocate any memory outside of the Shared-Memory file.
My Array structure:
template<typename T, size_t Size>
struct SHMArray {
SHMArray() : ptr(elements) {}
SHMArray(const SHMArray& other) { std::copy(other.begin(), other.end(), begin()); }
SHMArray(SHMArray&& other)
{
std::swap(other.ptr, ptr);
std::fill_n(ptr.get(), Size, T());
}
~SHMArray()
{
std::fill_n(ptr.get(), Size, T());
}
constexpr bool empty() const noexcept
{
return Size == 0;
}
constexpr size_type size() const noexcept
{
return Size;
}
T& operator[](std::size_t pos)
{
return *(ptr.get() + pos);
}
constexpr const T& operator[](std::size_t pos) const
{
return *(ptr.get() + pos);
}
T* data() noexcept
{
return ptr.get();
}
constexpr const T* data() const noexcept
{
return ptr.get();
}
private:
offset_ptr<T> ptr;
T elements[];
};
Program 1:
int main()
{
//Allocate a shared memory file of 1mb..
auto memory_map = SharedMemoryFile("./memory.map", 1024 * 1024, std::ios::in | std::ios::out);
memory_map.lock();
//Pointer to the shared memory.
void* data = memory_map.data();
//Place the object in the memory..
SHMArray<int, 3>* array = ::new(data) SHMArray<int, 3>();
(*array)[0] = 500;
(*array)[1] = 300;
(*array)[2] = 200;
memory_map.unlock(); //signals other program it's okay to read..
}
Program 2:
int main()
{
//Open the file..
auto memory_map = SharedMemoryFile("./memory.map", 1024 * 1024, std::ios::in | std::ios::out);
memory_map.lock();
//Pointer to the shared memory.
void* data = memory_map.data();
//Place the object in the memory..
//I already understand that I could just cast the `data` to an SHMArray..
SHMArray<int, 3>* array = ::new(data) SHMArray<int, 3>();
for (int i = 0; i < array.size(); ++i)
{
std::cout<<(*array)[i]<<"\n";
}
memory_map.unlock(); //signals other program it's okay to read..
}
Program One placed the SHMArray in memory with placement new. Program Two does the same thing on top of program one's already placed object (overwriting it). Is this undefined behaviour? I don't think it is but I want to confirm.
Neither program calls the destructor array->~SHMVEC(); I also don't think this leaks as long as I close the MemoryMapped file then it should all be fine.. but I want to make sure this is fine. If I ran the programs again on the same file, it shouldn't be a problem.
I am essentially making the assumption that placement new is working as if I placed a C struct in memory in this particular scenario via: struct SHMArray* array = (struct SHMArray*)data;.. Is this correct?
I am essentially making the assumption that placement new is working
as if I placed a C struct in memory in this particular scenario via:
struct SHMArray* array = (struct SHMArray*)data;.. Is this correct?
No, this is not correct. Placement new also invokes the object's appropriate constructor. "struct SHMArray* array = (struct SHMArray*)data;" does not invoke any object's constructor. It's just a pointer conversion cast. Which does not invoke anyone's constructor. Key difference.
In your sample code, you do actually want to invoke the templated object's constructor. Although the shown example has other issues, as already mentioned in the comments, this does appear to be what needs to be done in this particular situation.
But insofar as the equivalent of placement new versus a pointer cast, no they're not the same. One invokes a constructor, one does not. new always invokes the constructor, whether it's placement new, or not. This is a very important detail, that's not to be overlooked.
My question is not a duplicate of Is it safe to `free()` memory allocated by `new`?.
I'm writing a toy garbage collector for PODs, in which I'm defining my own custom operator new/new[] and operator delete/delete[]. Code below:
#include <iostream>
#include <map>
std::map<void*, std::size_t> memory; // globally allocated memory map
struct collect_t {} collect; // tag for placement new
void* operator new(std::size_t size, const collect_t&)
{
void* addr = malloc(size);
memory[addr] = size;
return addr;
}
void* operator new[](std::size_t size, const collect_t&)
{
return operator new(size, collect);
}
void operator delete(void *p, const collect_t&) noexcept
{
memory.erase(p); // should call ::operator delete, no recursion
free(p);
}
void operator delete[](void *p, const collect_t&) noexcept
{
operator delete(p, collect);
}
void display_memory()
{
std::cout << "Allocated heap memory: " << std::endl;
for (auto && elem : memory)
{
std::cout << "\tADDR: " << elem.first << " "
<< "SIZE: " << elem.second << std::endl;
}
}
void clear()
{
for (auto && elem : memory)
free(elem.first); // is this safe for arrays?
memory.clear();
}
int main()
{
// use the garbage collector
char *c = new(collect) char;
int *p = new(collect) int[1024]; // true size: sizeof(int)*1024 + y (unknown overhead)
display_memory();
clear();
display_memory();
}
The idea is simple: I store all allocated tracked addresses (the ones allocated with my custom new) in a std::map, and make sure that at the end of the day I clear all memory in my clear() function. I use a tag for my new and delete (and don't overload the global ones) so that std::map's allocator can call the global ones without recurring.
My question is the following: in my clear() function, I de-allocate the memory in the line
for (auto && elem : memory)
free(elem.first); // is this safe for arrays?
Is this safe for arrays, e.g. for int *p = new(collect) int[1024];?. I believe it is, since void* operator new[](std::size_t size, const collect_t&) calls operator new(size, collect);, and the latter calls malloc. I am not 100% sure though, can something go wrong here?
It appears to me that in order for memory to be in your memory container it must have been allocated with your custom allocator that always calls malloc. Therefore I believe your code calling free should be ok.
Obviously if someone goes around stuffing random addresses into the memory map you will wind up with all sorts of undefined behavior.
Assuming the objects using your garbage collector never implement a destructor, and this holds true for any members that those objects may contain, the code as it were is "safe" in the sense that the call to free() directly is just by-passing the work the compiler would have done to achieve the same thing as it inlined the delete calls.
However, the code is not really safe.
If you ever changed how your garbage collector worked, or how the new function worked, then you would have to hunt down all the direct calls to free() to head off any problems. If the code was ever cut-and-pasted or otherwise reused in a context outside of your garbage collector, you would face a similar problem.
It is just better practice to always match new to delete and malloc to free.
I have a class that is described that way :
class Foo {
int size;
int data[0];
public:
Foo(int _size, int* _data) : size(_size) {
for (int i = 0 ; i < size ; i++) {
data[i] = adapt(_data[i]);
}
}
// Other, uninteresting methods
}
I cannot change the design of that class.
How can I create an instance of that class ? Before calling the constructor, I have to make it reserve enough memory to store its data, so it has to be on the heap, not on the stack. I guess I want something like
Foo* place = static_cast<Foo*>(malloc(sizeof(int) + sizeof(int) * size));
*place = new Foo(size, data); // I mean : "use the memory allocated in place to do your stuff !"
But I can't find a way to make it work.
EDIT : as commentators have noticed, this is not a very good overall design (with non-standards tricks such as data[0]), alas this is a library I am forced to use...
You could malloc the memory for the object and then use a placement new to create the object in the previously allocated memory :
void* memory = malloc(sizeof(Foo) + sizeof(int) * size);
Foo* foo = new (memory) Foo(size, data);
Note that in order to destroy this object, you can't use delete. You would have to manually call the destructor and then use free on the memory allocated with malloc :
foo->~Foo();
free(memory); //or free(foo);
Also note that, as #Nikos C. and #GManNickG suggested, you can do the same in a more C++ way using ::operator new :
void* memory = ::operator new(sizeof(Foo) + sizeof(int) * size);
Foo* foo = new (memory) Foo(size, data);
...
foo->~Foo();
::operator delete(memory); //or ::operator delete(foo);
You have a library that does this thing but doesn't supply a factory function? For shame!
Anyway, while zakinster's method is right (I'd directly call operator new instead of newing an array of chars, though), it's also error-prone, so you should wrap it up.
struct raw_delete {
void operator ()(void* ptr) {
::operator delete(ptr);
}
};
template <typename T>
struct destroy_and_delete {
void operator ()(T* ptr) {
if (ptr) {
ptr->~T();
::operator delete(ptr);
}
}
};
template <typename T>
using dd_unique_ptr = std::unique_ptr<T, destroy_and_delete<T>>;
using FooUniquePtr = dd_unique_ptr<Foo>;
FooUniquePtr CreateFoo(int* data, int size) {
std::unique_ptr<void, raw_delete> memory{
::operator new(sizeof(Foo) + size * sizeof(int))
};
Foo* result = new (memory.get()) Foo(size, data);
memory.release();
return FooUniquePtr{result};
}
Yes, there's a bit of overhead here, but most of this stuff is reusable.
If you really want to be lazy simply use a std::vector<Foo>. It will use more space (I think 3 pointers instead of 1) but you get all the benefits of a container and really no downsides if you know it is never going to change in size.
Your objects will be movable given your definition so you can safely do the following to eliminate reallocation of the vector during initial fill...
auto initialFooValue = Foo(0, 0)
auto fooContainer = std::vector<Foo>(size, initialFooValue);
int i = 0;
for (auto& moveFoo : whereverYouAreFillingFrom)
{
fooContainer[i] = std::move(moveFoo);
++i;
}
Since std::vector is contiguous you can also just memcopy into it safely since your objects are trivially-copyable.
I think a good C++ solution is to get raw memory with new and then use placement new to embed your class into it.
getting raw memory works like this:
Foo *f = static_cast<Foo *>(operator new(sizeof(Foo));
constructing the object in received memory works like this:
new (f) Foo(size, data); // placement new
remember that this also means that you have to manually clean up the place.
f->~Foo(); // call the destructor
operator delete(f); // free the memory again
My personal opinion is, that it is bad to use malloc and free in newly written C++ code.
EDITED: reworded question.
When new and malloc are called, the size of the block of memory to be allocated is passed:
void* malloc(size_t);
void* operator new(size_t);
is it possible to get type information, i.e. so you could do a sizeof(T) where T was the type the memory was being allocated for:
T t = new T;
I'd like to overload new and malloc but require the type information not just the size of the memory being allocated.
FURTHER EDIT:
The reason I am doing this is that I will overload malloc. This function will inspect the size of the memory being allocated and allocate from a particualr memory pool:
template <int v> struct int2type { value = v };
inline void* malloc(const std::size_t sz)
{
if(sz <= 64)
{
size_obj* ptr = malloc(nggt::core::int2type<sizeof(size_obj) + (sz % 8)>);
ptr->sz = sz;
return ptr+1;
}
else if(sz <= 128)
{
size_obj* ptr = malloc(nggt::core::int2type<sizeof(size_obj) + 128>);
ptr->sz = sz;
return ptr+1;
}
else
{
return TSuper::malloc(sz);
}
}
inline void* malloc(const nggt::core::int2type<sizeof(size_obj) + 8> sz)
{
return m_heap8.malloc(sz.value);
}
inline void* malloc(const nggt::core::int2type<sizeof(size_obj) + 16> sz)
{
return m_heap16.malloc(sz.value);
}
There are also supporintg free overloads using a freelist to return memory to a pool. Problem is I can't use size_t sz in a template as it's not known at compile time. If I could get the type information I could do sizeof(T) and be done!
Cheers,
Graeme
The new with the std::size_t argument is commonly known as non-placement new. The std::size_t argument is passed in by default to let the operating system know how many bytes to allocate for the object you are creating. You, yourself, don't need to provide it to the new declaration.
Edit:
In response to your edit. It's common practice if you are overloading the new operator to
place it within the namespace you would like it to be called from
Avoid a namespace new if you can make specific news for your objects. (that is, there is behavior specific to each class, why not place it with the class itself)
The first argument should be the std::size_t which you don't need to worry about. It'll be handled for you.
Everything else after that is all yours:
struct MyObject{
MyObject* operator new(std::size_t s, const char* message){
cout << "Creating an object of size " << size_of(MyObject) << endl;
return new MyObject();
}
};
You can even use the "old" new in your call of your "new" new.
Further Edit:
In response to your next edit, don't call your own malloc, malloc. Instead, call it some other name. If you're writing C++, malloc has no place within your code what so ever unless you're dealing with some legacy application. In C++, you should only be touching the new operator.
So having
struct ResultStructure
{
ResultStructure(const ResultStructure& other)
{
// copy code in here ? using memcpy ? how???
}
ResultStructure& operator=(const ResultStructure& other)
{
if (this != &other) {
// copy code in here ?
}
return *this
}
int length;
char* ptr;
};
How to implement "copy constructor" and "assignment operator"? (sorry - I am C++ nube)
Update: sbi and others ask - why do I want to manually deal with raw memory? My answer is simple - In a students project I am part of now we use lots of C library's such as for example OpenCV OpenAL and FFmpeg and there are more to come. Currently using C++ we try to create a graph based direct show like cross platform library that would be helpful in live video broadcasting and processing. Our graph elements currently use char* and int pairs for data exchange. To cast data to subscribed elements we use raw memcpy now. I want to go further and make it possible for us to make our graph elements base C++ template. So that one graph element would be capable of of sharing current graph element data with other Graph elements and that data it shares would be a structure that would contain not one char* one int but any number of data fields and nearly any elements inside. So that is why I need to understand how to create a basic C++ structure that implements "copy constructor" and "assignment operator" for me to be capable to use new for us data casting algorithms like
void CastData(T item){
for(size_t i = 0 ; i < FuncVec.size(); i++){
T dataCopy = item;
FuncVec[i](dataCopy);
}
}
instead of currently used
void CastData(char * data, int length){
for(size_t i = 0 ; i < FuncVec.size(); i++){
char* dataCopy = new char[length];
memcpy(dataCopy, data, length);
FuncVec[i](dataCopy, length);
delete[] dataCopy;
}
}
You might want to explain why you want to manually deal with raw memory. I haven't done this in a long time, it's what std::string and std::vector where designed for:
struct ResultStructure
{
// nothing else needed
std::string data; // or std::vector<char>
};
But if you really need to do this the hard way (is this homework?), then be advised that it is, at first, surprisingly hard to get this right. For example, a naive implementation of the assignment operator might be like this:
// DON'T TRY THIS AT HOME!!
ResultStructure& ResultStructure::operator=(const ResultStructure& rhs)
{
delete[] ptr; // free old ressource
ptr = new char[rhs.length]; // allocate new resourse
std::copy(rhs.ptr, rhs.ptr+rhs.length, ptr; // copy data
length = rhs.length;
}
If someone accidentally assigns an object to itself (which might happen if all you have is two references and you don't suspect them to refer to the same object), then this will fail fatally.
Also, what if new throws an exception? (It might throw std::bad_alloc if memory is exhausted.) Then we have already deleted the old data and have not allocated new data. The pointer, however, still points at where the old data used to be (actually, I think this is implementation-defined, but I have yet to see an implementation that changes a ptr upon deletion), and the class' destructor (you knew that class would need a destructor, right?) would then attempt to delete a piece of data at an address where no data is allocated. That's Undefined Behavior. The best you can hope for is that it crashes immediately.
The easiest way to do this is to employ the Copy-And-Swap idiom:
struct ResultStructure
{
ResultStructure(const ResultStructure& other)
: ptr(new char[rhs.length]), length(rhs.length)
{
std::copy(rhs.ptr, rhs.ptr+rhs.length, ptr);
}
~ResultStructure() // your class needs this
{
delete[] ptr;
}
ResultStructure& operator=(ResultStructure rhs) // note: passed by copy
{
this->swap(rhs);
return *this
}
void swap(const ResultStruct& rhs)
{
using std::swap;
swap(length, rhs.length);
swap(ptr, rhs.ptr);
}
std::size_t length;
char* ptr;
};
Note that I have added a destructor, changed the assignment operator to pass the argument per copy (we need the copy constructor invoked to allocate memory), and added a swap member function. Per convention a swap() function never throws and is fast, preferably O(1).
I know that GMan's discussion of the Copy-And-Swap idiom is exhaustive enough to be and exhausting while mine is probably too terse for you, and you will likely not understand a lot of it at first, but try to persevere and to understand as much as possible.
If you use std::string, instead of char*, you would not even need to write operator= or copy-constructor. The compiler generated code would do your job very well.
But as a general solution (for some other scenario), use copy-and-swap idiom:
Copy-and-Swap Idiom
What is the copy-and-swap idiom?
Exceptional C++ by Herb Sutter has described these in great detail. I would recommend you to read items from this book. For the time being, you can read this article online:
Exception-Safe Generic Containers
The easy solution is to use a std::string instead of char* member.
Then the compiler-generated copy constructor and copy assignment operator just work.
As a rule, and especially as a novice, don't have raw pointer members.
Cheers & hth.,
As has been said, and as was recommending in the question this emanated from, you should probably reuse an existing container. At least the code would be right :)
For educational purposes though, let's examine this structure:
class Result
{
public:
private:
size_t length; // can't really be negative, right ?
char* data;
};
Here, we need explicit memory management. This implies, notably, following the Rule Of Three (say thanks to Fred)
Let's begin with actually building our object:
Result::Result(): length(0), data(0) {}
Result::Result(size_t l, char* d): length(0), data(0)
{
if (!d) { return; }
data = new char[l]; // this may throw, so we do it first
length = l;
memcpy(data, d, l);
}
Now we can implement the traditional operators:
// Swap
void Result::swap(Result& other)
{
using std::swap;
swap(length, other.length);
swap(data, other.data);
}
// Copy Constructor
Result::Result(Result const& other): length(0), data(0)
{
if (!other.length) { return; }
data = new char[other.length];
length = other.length;
mempcy(data, other.data, length);
}
// Assignemt Operator
Result& Result::operator=(Result other)
{
this->swap(other);
return *this;
}
// !IMPORTANT!
// Destructor
Result::~Result()
{
delete[] data; // use the array form of delete
// no need to test for nullity, it's handled
}
this is std::vector<char>'s job - or is this homework?
the vector would replace both length and the allocation behind ptr. the vector is the c++ idiom, and you'll not make friends with other c++ devs if you implement your classes like you've described. of course, there are corner cases, but standard containers such as vector are the default.
the vector knows how to copy chars as well as itself, and the implementations are optimized and tested.
here's how to explicitly implement copy ctor/assign using a vector:
struct ResultStructure {
ResultStructure(const ResultStructure& other) : d_chars(other.d_chars) {
}
ResultStructure& operator=(const ResultStructure& other) {
if (this != &other) {
this->d_chars = other.d_chars;
}
return *this;
}
std::vector<char> d_chars;
};
I think this should do the work:
struct ResultStructure
{
ResultStructure(const ResultStructure& other);
ResultStructure& operator=(const ResultStructure& other);
int length;
char* ptr;
};
ResultStructure::ResultStructure(const ResultStructure& other):length(other.length)
{
ptr = (char*)malloc(length);
memcpy(ptr, other.ptr, length);
}
ResultStructure& ResultStructure::operator=(const ResultStructure& other)
{
length = other.length;
ptr = (char*)malloc(length);
memcpy(ptr, other.ptr, length);
return *this;
}
Please remember about freeing ptr in destructor.
What is stored under ptr? If text, why not to use std::string? Anyway you can use std::vector. The constructors will be much easier then...
How is the memory to which ptr points allocated?
if using new, allocate with new, set length and then copy
other.length = length;
other.ptr = new char[length];
memcpy( other.ptr, ptr, length );
If you're allocating the memory with malloc, substitute a call to malloc in place of the call to new. If you're using some other memory scheme, figure it out. :)