Wrap existing memory with const std::vector? - c++

OK, so I recently learned that (a) std::vector uses contiguous memory by definition/standard, and thus (b) &(v[0]) is the address of that contiguous block of memory, which you can read/write to as an old-skool C-array. Like...
void printem(size_t n, int* iary)
{ for (size_t i=0; i<n; ++i) std::cout << iary[i] << std::endl; }
void doublem(size_t n, int* iary)
{ for (size_t i=0; i<n; ++i) iary[i] *= 2; }
std::vector<int> v;
for (size_t i=0; i<100; ++i) v.push_back(i);
int* iptr = &(v[0]);
doublem(v.size(), iptr);
printem(v.size(), iptr);
OK, so that's cool, but I want to go in the other direction. I have lots and lots of existing code like
double computeSomething(const std::vector<SomeClass>& v) { ... }
If I have a C-array of objects, I can use such code like this:
SomeClass cary[100]; // 100*sizeof(SomeClass)
// populate this however
std::vector<SomeClass> v;
for (size_t i=0; i<100; ++i) v.push_back(cary[i]);
// now v is also using 100*sizeof(SomeClass)
double x = computeSomething(v);
I would like to do that (a) without the extra space and (b) without the extra time of inserting a redundant copy of all that data into the vector. Note that "just change your stupid computeSomething, idiot" is not sufficient, because there are thousands of such functions/methods that exhibit this pattern that are not under my control and, even if they were are too many to go and change all of them.
Note also that because I am only interested in const std::vector& usage, there is no worry that my original memory will ever need to be resized, or even modified. I would want something like a const std::vector constructor, but I don't know if the language even allows special constructors for const instances of a class, like:
namespace std { template <typename T> class vector {
vector() { ... }
vector(size_t n) { ... }
vector(size_t n, const T& t) { ... }
const vector(size_t n, T*) { ... } // can this be done?
...
If that is not possible, how about a container derived off of std::vector called std::const_vector, which (a) could construct from a pointer to a c-array and a size, and (b) purposefully did not implement non-const methods (push_back, resize, etc.), so then even if the object with a typename of const_vector is not actually a const object, the interface which only offers const methods makes it practically const (and any erroneous attempts to modify would be caught at compile time)?
UPDATE: A little messing around shows that this "solves" my problem wrt Windows-implementation of std::vector:
template <typename T>
class vector_tweaker : public std::vector<T> {
public:
vector_tweaker(size_t n, T* t) {
_saveMyfirst = _Myfirst;
_saveMylast = _Mylast;
_saveMyend = _Myend;
_Myfirst = t;
_Mylast = t + n;
_Myend = t + n;
}
~vector_tweaker() {
_Myfirst = _saveMyfirst;
_Mylast = _saveMylast;
_Myend = _saveMyend; // and proceed to std::vector destructor
}
private:
T* _saveMyfirst;
T* _saveMylast;
T* _saveMyend;
};
But of course that "solution" is hideous because (a) it offers no protection against the base class deleting the original memory by doing a resize() or push_back() (except for a careful user that only constructs const vector_tweaker()) -- and (b) it is specific to a particular implementation of std::vector, and would have to be reimplemented for others -- if indeed other platforms only declare their std::vector member data as protected: as microsoft did (seems a Bad Idea).

You can try reference-logic storing introduced in C++11 with std::reference_wrapper<>:
SomeClass cary[100];
// ...
std::vector<std::reference_wrapper<SomeClass>> cv;
cv.push_back(cary[i]); // no object copying is done, reference wrapper is stored
Or without C11, you can create a specialization of such template class for bytes - char. Then for the constructor from char* C-array you can use ::memcpy: which unfortunately will then use twice as much memory.
::memcpy(&v[0], c_arr, n);
Something like this:
template <typename T> class MyVector : public std::vector<T> {
};
template <> class MyVector<char> : public std::vector<char> {
public:
MyVector<char>(char* carr, size_t n) : std::vector<char>(n) {
::memcpy(&operator[](0), carr, n);
}
};
What I would recommend - replace all C-arrays to vectors where possible, then no extra copying will be needed.

Related

Custom allocator for nested containers

I am working on custom allocators. So far, I have tried to work on simple containers: std::list, std::vector, std::basic_string, etc...
My custom allocator is a static buffer allocator, its implementation is straightforward:
#include <memory>
template <typename T>
class StaticBufferAlloc : std::allocator<T>
{
private:
T *memory_ptr;
std::size_t memory_size;
public:
typedef std::size_t size_type;
typedef T *pointer;
typedef T value_type;
StaticBufferAlloc(T *memory_ptr, size_type memory_size) : memory_ptr(memory_ptr), memory_size(memory_size) {}
StaticBufferAlloc(const StaticBufferAlloc &other) throw() : memory_ptr(other.memory_ptr), memory_size(other.memory_size){};
pointer allocate(size_type n, const void *hint = 0) { return memory_ptr; } // when allocate return the buffer
void deallocate(T *ptr, size_type n) {} // empty cause the deallocation is buffer creator's responsability
size_type max_size() const { return memory_size; }
};
I am using it in this fashion:
using inner = std::vector<int, StaticBufferAlloc<int>>;
int buffer[201];
auto alloc1 = StaticBufferAlloc<int>(&buffer[100], 50);
inner v1(0, alloc1);
assert(v1.size() == 0);
const int N = 10;
// insert 10 integers
for (size_t i = 0; i < N; i++) {
v1.push_back(i);
}
assert(v1.size() == N);
All good so far, when I grow N past the max buffer size it throws and that's expected.
Now, I am trying to work with nested containers. In short, am trying to have a vector of the vector (matrix), where the parent vector and all its underlying elements (that are vectors i.e. containers) share the same static buffer for allocation. It looks like scoped_allocator can be a solution for my problem.
using inner = std::vector<int, StaticBufferAlloc<int>>;
using outer = std::vector<inner, std::scoped_allocator_adaptor<StaticBufferAlloc<inner>>>;
int buffer[201];
auto alloc1 = StaticBufferAlloc<int>(&buffer[100], 50);
auto alloc2 = StaticBufferAlloc<int>(&buffer[150], 50);
inner v1(0, alloc1);
inner v2(0, alloc2);
assert(v1.size() == 0);
assert(v2.size() == 0);
const int N = 10;
// insert 10 integers
for (size_t i = 0; i < N; i++)
{
v1.push_back(i);
v2.push_back(i);
}
assert(v1.size() == N);
assert(v2.size() == N);
outer v // <- how to construct this vector with the outer buffer?
v.push_back(v1);
v.push_back(v2);
...
My question is how to initialize the outer vector on its constructor call with its static buffer?
Creating a scoped allocator in C++11/C++14 was a little bit challenging. So I opted for a very modern solution introduced in C++17. Instead of implementing an allocator, I used polymorphic_allocator. Polymorphic allocators are scoped allocators, standard containers will automatically pass the allocators to sub-objects.
Basically, the idea was to use a polymorphic allocator and inject it with monotonic_buffer_resource. The monotonic_buffer_resource can be initialized with a memory resource.
Writing a custom memory resource was very simple:
class custom_resource : public std::pmr::memory_resource
{
public:
explicit custom_resource(std::pmr::memory_resource *up = std::pmr::get_default_resource())
: _upstream{up}
{
}
void *do_allocate(size_t bytes, size_t alignment) override
{
return _upstream; //do nothing, don't grow just return ptr
}
void do_deallocate(void *ptr, size_t bytes, size_t alignment) override
{
//do nothing, don't deallocate
}
bool do_is_equal(const std::pmr::memory_resource &other) const noexcept override
{
return this == &other;
}
private:
std::pmr::memory_resource *_upstream;
};
Using it is even simpler:
std::byte buffer[512];
custom_resource resource;
std::pmr::monotonic_buffer_resource pool{std::data(buffer), std::size(buffer), &resource};
std::pmr::vector<std::pmr::vector<int>> outer(&pool)
It is important to note that std::pmr::vector<T> is just std::vector<T, polymorphic_allocator>.
Useful resources:
CppCon 2017: Pablo Halpern “Allocators: The Good Parts”
C++ Weekly - Ep 222 - 3.5x Faster Standard Containers With PMR
Purpose of scoped allocator
std::pmr is cool but it requires modern versions of gcc to run (9+). Fortunately, Reddit is full of kind strangers. A C++14 solution can be found here.

Can I force std::vector to leave a memory leak?

Can I force std::vector to not deallocate its memory after the vector goes out of scope?
For example, if I have
int* foo() {
std::vector<int> v(10,1); // trivial vector
return &v[0];
}
int main()
{
int* bar = foo();
std::cout << bar[5] << std::endl;
}
There is no guarantee that the values will still be accessible here.
I am currently simply doing this
int* foo() {
std::vector<int> v(10,1);
int* w = new int[10];
for (int i=0; i<10; i++) {
w[i] = v[i];
}
return w;
}
but it is a little wasteful to repopulate a whole new array. Is there a way to force std::vector to not delete its array?
Note: I am not returning the vector itself because I am interfacing c++ with python using SWIG, and ARG_OUTVIEW_ARRAY requires a raw pointer and, in fact, an intentional memory leak. I would still however like to be able to make use of vector features while constructing the data itself.
It is possible but you should never do it. Forcing a vector to leave memory leak is a terrible idea and if you need such a thing then you need to re-think your design. std::vector is a resource managing type whose one of the main goals is to ensure that we don't have a leak. Never try to break that.
Now, to answer your specific question: std::vector takes an allocator type as second template parameter which is default to std::allocator<T>. Now you can write a custom allocator that doesn't release any memory and use that with your vector. Writing a custom allocator is not very trivial work, so I'm not going to describe that here (but you can Google to find the tutorials).
If you really want to use custom allocator then you must ensure that your vector never triggers a grow operation. Cause during growing capacity the vector will move/copy data to new location and release the old memories using the allocator. If you use an allocator that leaks then during growing you not only retain the final data, but also retain the old memories which I'm sure that you don't want to retain. So make sure that you create the vector with full capacity.
The vector is desiged to prevent leaks.
But if you want to shoot yourself in the foot, it's possible. Here's how you prevent the vector from deallocating its internal array:
int *foo()
{
std::vector<int> v(10,1);
int *ret = v.data();
new (&v) std::vector<int>; // Replace `v` with an empty vector. Old storage is leaked.
return ret;
}
As the other answers say, you should never do it.
No.
Vectors are not implemented to have memory leaks, and the interface does not provide a way to create one.
You can't "steal" the memory (removing ownership of it from the vector), which is possibly a bit of a shame.
Sorry, but you are going to have to either copy (as you're doing now), or not use vector.
This is a bad idea, but possible by creating a custom allocator that does not deallocate as said in other answers.
For example : (boilerplate mostly from cppref)
#include <cstdlib>
#include <new>
#include <vector>
template <typename T>
struct LeakingAllocator
{
using value_type = T;
LeakingAllocator() = default;
template <typename U> constexpr LeakingAllocator(const LeakingAllocator<U>&) noexcept {}
T* allocate(std::size_t n)
{
if(n > std::size_t(-1) / sizeof(T)) throw std::bad_alloc(); // check for overflow
if(auto p = static_cast<T*>(std::malloc(n*sizeof(T)))) return p; // return p if malloc returns a valid object
throw std::bad_alloc(); // otherwise just throw.
}
void deallocate(T* p, std::size_t) noexcept { /*leak intentionally*/ }
};
template <typename T, typename U>
bool operator==(const LeakingAllocator<T>&, const LeakingAllocator<U>&) { return true; }
template <typename T, typename U>
bool operator!=(const LeakingAllocator<T>&, const LeakingAllocator<U>&) { return false; }
template <typename T>
using LeakingVector = std::vector<T, LeakingAllocator<T>>;
Then code like
int* ret()
{
LeakingVector<int> a;
a.resize(10);
return &a[0];
}
int main()
{
auto ptr = ret();
*ptr = 10;
std::cout << *ptr;
}
becomes valid.
Not sure but, yes.
You can create a custum allocator who do nothing when deallocate => leak
Or may be you can jsut create your vectoron the heap so it will leak anyway.
int* foo() {
std::vector<int>* v = new std::vector<int>(10,1);
return &((*v)[0]);
// no delete
}
int main()
{
int* bar = foo();
std::cout << bar[5] << std::endl;
}
No.
And you're doing it wrong. Return the vector instead so the lifetime works out:
Write your own special Python memory vector class, something like (most crudely):
template <typename T>
class python_vector
{
T* buffer_;
public:
python_vector(size_t n, const T& value) : buffer_{new T(n)}
{}
// copy, assignment, operator[](), *etc*
~python_vector()
{
// DO NOTHING!
}
}
python_vector<int> foo() {
python_vector<int> v(10,1);
// process v
return v;
}
int main()
{
python_vector<int> bar = foo(); // copy allusion will build only one python_vector here
std::cout << bar[5] << std::endl;
}
In C++ you would most probably write:
auto foo()
{
std::vector<int> v(10,1); // trivial vector
return v;
}
int main()
{
const auto bar = foo();
std::cout << bar[5] << std::endl;
}

c++ memory allocation and rewriting the vector class

I'm doing an assignment in c++ where we have to rewrite the std::vector class that inherits from an abstract Container class. We have to write methods for 'Vector' such as begin(), end(), push_back(), etc...
Many of the basic std::vector Member functions found here: http://www.cplusplus.com/reference/vector/vector/
My question is multi-faceted mainly because I am a beginner and the memory of computers is fascinating. Also, sorry if this question is super specific or has similar answers on here but I'm really stuck because this is the first time I've dealt with memory.
Here are my Container and Vector class implementations:
template <typename T>
class Container {
protected:
T* elem;
int cap;
public:
virtual T& operator[](int i) = 0;
virtual int size() = 0;
~ Container() { delete[] elem; }
};
template <typename T>
class Vector: public Container<T> {
protected:
using Container<T>::elem;
using Container<T>::cap;
private:
int sz;
public:
// constructors
Vector(); // default constructor
Vector(int n); // fill constructor #1
Vector(int n, T fill_val); // fill constructor #2
/* // ~ Vector(); // destructors
unsure if the above is correct, because of the Container destructor
*/
// overloaded operators
T& operator[](int i); // subscript overloading '[]'
T& operator=(const Vector<T>& V); // assignment overloading '='
// access methods
T& front(); //returns a reference to the first element
T& back(); //returns a reference to the last element
// iterator methods
T* begin(); // returns memory address of the first element
T* end(); // returns memory address at the end
// size methods
int size() { return sz; }
int capacity() { return cap; }
};
Part 2.
When/why do I need the destructor? I've tried to read up on destructors on Stack, but I feel more confused than clarified. I'm assuming the SegFault is coming from memory needing to be deallocated, and I know I'm supposed to be learning how to dynamically allocate memory with this assignment. The constructors and destructors are crucial for this process but can someone break it down as simply as possible for me?
EDIT:
I've fixed the segmentation fault by implementing Fureeish's answer. My code has been updated above.
The only remaining inquiry is to the destructor and its purpose. Do I need to implement one for Vector or will it be called automatically by Container?
You decided to manage the memory by yourself, but you never allocated any. You're missing some news here and there. For example, given a:
Vector(int n, T fill_val) {
sz = n;
cap = 2*n;
for(int i = 0; i < sz; i++) {
elem[i] = fill_val;
}
}
the for() loop iterates over elem, which is an uninitialized pointer. You treat it like a pointer to a dynamically allocated array, but it simply points to some garbage value. Remember - first allocate, then work on (only when you decide to manage the memory yourself. Typically you should prefer using standard implementations).
The correct version should look like this:
Vector(int n, T fill_val) {
sz = n;
cap = 2*n;
elem = new T[cap]; // or cap-1, depending on what you do with `end()`.
for(int i = 0; i < sz; i++) {
elem[i] = fill_val;
}
}
The fact that you never encountered this problem while using other constructors is just you being extremely lucky and undefined behavior being sneaky

Create an object array without calling the constructors

for an embedded system we need a custom vector class, where the capacity is set during compile-time through a template parameter.
Until now we had an array of objects as a member variable.
template<class T, size_t SIZE>
class Vector {
...
T data[SIZE];
}
The problem here of course is that if T isn't a POD, the default constructors of T are called. Is there any way to let data be uninitialized until a corresponding push() call (with placement new inside)? Just using
uint8_t data[SIZE * sizeof(T)];
possibly breaks the alignment of T. We absolutely cannot use dynamic memory, the total container size always needs to be known at compile-time. We also cannot use C++'s alignas specifier since the compiler does not support C++11 yet :(
First I would check if the compiler has support for alignment, ie gcc has __attribute__(aligned(x)), there is likely something similar.
Then if you absolutely have to have aligned uninitialized data without such support, you will have to waste some space
// Align must be power of 2
template<size_t Len, size_t Align>
class aligned_memory
{
public:
aligned_memory()
: data((void*)(((std::uintptr_t)mem + Align - 1) & -Align)) {}
void* get() const {return data;}
private:
char mem[Len + Align - 1];
void* data;
};
And you'd use placement new with it
template<typename T, size_t N>
class Array
{
public:
Array() : sz(0) {}
void push_back(const T& t)
{
new ((T*)data.get() + sz++) T(t);
}
private:
aligned_memory<N * sizeof(T), /* alignment */> data;
size_t sz;
};
Live
The alignment of T can be found with C++11 alignof, check your compiler to see if it supports anything that can be used to find out its alignment. You can also just take a guess from printed pointer values and hope that's enough.
Another way is to use std::vector<> with a custom allocator that allocates on the stack.
This way you would create an empty vector, reserve the required space, which should be equal to the space your allocator allocates for you on the stack, and then populate the vector using vector<>::emplace_back. Your element type can be non-copyable but must be movable in this case.
E.g.:
#include <vector>
struct X {
X(int, int);
// Non-copyable.
X(X const&) = delete;
X& operator=(X const&) = delete;
// But movable.
X(X&&);
X& operator=(X&&);
};
template<class T, std::size_t N>
struct MyStackAllocator; // Implement me.
int main() {
std::vector<X, MyStackAllocator<X, 10>> v;
v.reserve(10);
v.emplace_back(1, 2);
v.emplace_back(3, 4);
}
Information about how to implement an allocator is widely available, for example, search YouTube for "c++ allocator".
You are going to have to use placement new along with a union trick to get the alignment properly set.
// use `std::max_align_t` and `std::aligned_storage` when you have it
// since don't have access to alignof(), use the presumably max
// alignment value
using MaxAlign = long;
template <typename T, int size>
class UninitializedArray {
union Node {
char data[sizeof(T)];
MaxAlign alignment;
};
Node aligned_data[size];
bool initialized;
public:
UninitializedArray() : initialized(false) {}
void initialize() {
for (int i = 0; i < static_cast<int>(size); ++i) {
new (&this->aligned_data[i].data) T();
}
this->initialized = true;
}
~UninitializedArray() {
if (this->initialized) {
for (int i = 0; i < static_cast<int>(size); ++i) {
T* ptr = reinterpret_cast<T*>(&this->aligned_data[i].data);
ptr->~T();
}
}
}
T& operator[](int index) {
if (!this->initialized) {
this->initialize();
}
T* ptr = reinterpret_cast<T*>(&this->aligned_data[i].data);
return *ptr;
}
};
And then use it like this
UninitializedArray<Something, 5> arr;
arr[0].do_something();
If you ever get C++17 working, then you can use std::array and std::optional to make this easy
std::optional<std::array<T, N>> optional_array;
// construct the optional, this will construct all your elements
optional_array.emplace();
// then use the value in the optional by "treating" the optional like
// a pointer
optional_array->at(0); // returns the 0th object

Nested templates and constructor

Edit:
Note that my final purpose here is not having the class working, is just learning more about templates :-)
Suppose you have a template class which implements a vector:
template <typename T>
class Vector
{
public:
Vector(size_t dim) {
dimension = dim;
elements = new T[dim];
}
/* Here more stuff, like operator[] etc... */
private:
size_t dimension;
T * elements;
}
And suppose you want to build a matrix with it. A matrix is just a vector of vectors, thus it can be designed as follows:
template <typename T>
class Matrix : public Vector<Vector<T> >
{
/*...*/
}
And here comes trouble: In the constructor I need to provide rows and columns as parameter to the internal vectors. It should be something like
template <typename T>
Matrix<T>::Matrix (size_t ncols, size_t nrows)
: Vector<Vector<T> > /* Here I need to specify size for both
* internal and external vectors */
{
}
Obviously I cannot write Vector<Vector<T>(nrows)>(ncols), but that's what I would need!
A possible solution would be including size inside the template:
template <typename T, size_t N>
class Vector
{
public:
Vector() {
elements = new T[N];
}
/* Here more stuff, like operator[] etc... */
private:
T * elements;
}
Hence I would no longer need constructor parameters, but this also forces me to write clumsy code with templates everywhere (by exmample, every function using a Vector should be declared as
template <typename T, size_t N>
void foo (Vector<T,N> &vec) {...}
Do you have better solutions?
EDIT:
As solution I took inspiration from Mr Fooz's and chubsdad's posts. That's how I fixed the problem:
/* The RowAccess class is just an array wrapper which raises an exception
* if you go out of bounds */
template <typename T>
class RowAccess
{
public:
RowAccess (T * values, unsigned cols) : values(vals), cols(c) {}
T & operator[] (unsigned i) throw (MatrixError) {
if (i < cols) return values[i];
else throw MatrixError("Column out of bound");
}
private:
T * values;
unsigned cols;
};
template <typename T>
class Matrix
{
public:
Matrix (unsigned rows, unsigned cols) {...}
virtual ~Matrix () {...}
RowAccess<T> operator[] (unsigned row) {
if (row < rows) return RowAccess<T>(values + cols * row, cols);
else throw MatrixError("Row out of boundary");
}
private:
unsigned rows;
unsigned cols;
T * values;
};
Thanks a lot to everyone!
In OO terms, I would vote for "has" a relationship between Matrix and a Vector. A Matrix has vectors, rather than a Matrix "is a" vector, which means that Matrix should not derive from Vector.
EDIT 1: A small correction. "..which means that Matrix should not derive "publicly" from Vector". Private inheritance may be still fine.
Use placement-new like this (burried behind the uninitialized_fill call)
template <typename T>
class Vector
{
public:
Vector(size_t dim, T const& c = T()) {
dimension = dim;
elements =
static_cast<T*>(operator new(sizeof(T) * dim));
std::uninitialized_fill(elements, elements + dim, c);
}
/* Here more stuff, like operator[] etc... */
private:
size_t dimension;
T * elements;
};
Then you can call the constructor with Matrix::Vector(ncols, Vector<T>(nrows)) (you don't need to repeat the argument for the outer Vector, because Vector refers to Vector< Vector<T> > automatically since you inherit from the outer Vector. You need to make sure to call destructors manually before doing operator delete(elements) in the destructor then.
You might also want to embed the vector as a member, which i would probably prefer because I imagine not necessarily all operations of the outer vector make sense for a matrix. Then the initialization looks like m(ncols, Vector<T>(nrows)).
It should be noted that std::vector can also be used for this
template <typename T>
class Vector
{
public:
Vector(size_t dim, T const& c = T()):elements(dim, c)
{ }
private:
std::vector<T> elements;
};
This is an easy and safe way to accomplish that, and you get automatic memory management.
This isn't what you asked, but there's a good chance the matrix would be better of implemented as a single linear vector where you provide high-level access methods that do the indexing (e.g. elmLoc=row*ncols+col). This way you don't need to create and initialize a vector of vectors. You also don't need to worry about accidentally having some inner vectors of the differing size. All dense matrix implementations I have ever used use a single linear vector as the underlying implementation.
It depends on what you expect from your Vector (and Matrix) class.
Either you want the size to be determined at runtime, and in that case, I'd suggest adding a resize() function which would allow you to size the Vector in the constructor as you like.
template <typename T>
class Vector
{
public:
Vector(size_t dim) {
dimension = dim;
elements = new T[dim];
}
Vector() : dimension(0), elements(0) {} // you need default constructor
void resize(size_t dim) { // note: this could be implemented a lot better
T* new_elements=new T[dim];
for(int i=0; i<dim && i<dimension; i++)
new_elements[i]=elements[i];
delete [] elements;
elements=new_elements; dimension=dim;
}
/* Here more stuff, like operator[] etc... */
private:
size_t dimension;
T * elements;
}
You would then resize your Vectors in the Matrix constructor in a loop.
If you want the size of your vector or matrix to be determined at compile time, the best thing probably would be to use the template non-type argument as you suggested.