Array changes when element retrieved - c++

Edit: Added constructors and add function
Consider the following three blocks:
//ONE
template<typename T>
class List1 {
private:
uint32_t capacity;
uint32_t used;
T* data;
void checkGrow() {
if (used < capacity)
return;
T* old = data;
capacity*=2;
data = (T*)new char[sizeof(T) * capacity];
memcpy(data, old, sizeof(T)*used);
delete (void*)old;
}
public:
List1() : capacity(1), used(0), data((T*)new char [sizeof(T)]) {}
List1(uint32_t initialSize) :
capacity(initialSize), used(0), data((T*)new char[sizeof(T)]) {}
List1(const List1& orig) :
capacity(orig.capacity), used(orig.used), data((T*)new char[used * sizeof(T)]) {
memcpy(data, orig.data, used * sizeof(T));
}
uint32_t serializeSize() const { return sizeof(used) + used*sizeof(T); }
char* read(char* p) {
used = *(uint32_t*)p;
p += sizeof(uint32_t);
data = (T*)new char[used*sizeof(T)];
memcpy(p, data, used * sizeof(T));
return p + used*sizeof(T);
}
char* write(char* p) {
*(uint32_t*)p = used;
p += sizeof(uint32_t);
memcpy(p, data, used * sizeof(T));
return p + used * sizeof(T);
}
~List1() { delete [] data; }
void add(const T& v) {
checkGrow();
data[used++] = v;
}
uint32_t getUsed() const{
return used;
}
uint32_t getCapacity() const{
return capacity;
}
//const T& operator [](int i) const { return data[i]; }
//T& operator [](int i) { return data[i]; }
T getData (int i) const{
return data[i];
}
uint32_t size() const { return used * sizeof(T); }
};
//TWO
List1<uint32_t> temp=in.readList<uint32_t>(); // <List1<uint32_t>>();
//BREAKPOINT HERE
for(uint i=0;i<15;i++){
//cout<<temp[i]<<endl;
cout<<temp.getData(i)<<endl;
}
//THREE
template<typename T>
T _read() {
T temp = *(T*)p;
p += sizeof(T);
availRead -= sizeof(T);
return temp;
}
template<typename T>
T read(){
//cout << (uint64_t)p << endl;
checkAvailableRead(sizeof(T));
return _read<T>();
}
template<typename T>
List1<T> readList(){
uint32_t len = read<uint32_t>();
List1<T> temp(len);
for (uint i = 0 ; i < len; i++){
T val =read<T>();
temp.add(val);
//HERE: For some reason code does not work without this print statement
//cout<<temp.getData(i)<<endl;
}
return temp;
}
Basically the issue is that the value of data changes from after returning from getData as shown below.
gdb) p/u *temp.data#15
$1 = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14}
(gdb) p/u *temp.data#15
$2 = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14}
(gdb) p/u *data#15
$3 = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14}
[New Thread 8444.0xad8]
(gdb) p/u *data#15
$4 = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14}
[New Thread 8444.0x214c]
(gdb) p/u *temp.data#15
$5 = {0, 1, 2, 3, 4, 5, 83, 0, 2150008464, 1, 3742232646, 0, 168272, 6, 0}
For some reason adding a print statement to readList fixes the issue, but that isn't a reasonable solution. I've tried a few different variations on the code but none of them worked.
I'm not sure what the issue is or how to even begin debugging as the problem occurs between the return statement and the next iteration of the loop (there's nothing to step into there).
Any advice would be greatly appreciated.

List1(const List1& orig) :
capacity(orig.capacity), used(orig.used), data((T*)new char[used * sizeof(T)]) {
memcpy(data, orig.data, used * sizeof(T));
}
For List1 to work correctly, there must never be a List1 whose capacity is greater than the actual allocated size. However, this creates a new List1 that violates this invariant if orig has a capacity greater than its used.
You probably meant capacity(orig.used).
Same issue here:
List1(uint32_t initialSize) :
capacity(initialSize), used(0), data((T*)new char[sizeof(T)]) {}
If you set capacity to initialSize, you can't allocate space for just 1 T.
This is also broken delete (void*)old;. What you allocate with new[], you must free with delete[].
Note that List1 can only be used to hold POD types (plain old data) that don't have constructors or destructors. If you're trying to use List1 to hold anything more complex, you're way off base with your design.

Related

Assigning a subsection of C-style array using a std::array& without violating "strict aliasing" and hence invoking UB?

Can I use a std::array<int, N> to alias parts of a int[] without invoking UB?
https://en.cppreference.com/w/cpp/container/array
"This container is an aggregate type with the same semantics as a struct holding a C-style array T[N] as its only non-static data member."
Motivation: The copy function below is not under my control and needs to make a single assignment via references. Only a struct { int[N]; } like a std::array<int, N> can make that kind of "multiple object assignment"?
Is this UB?
Is there another way?
#include <iostream>
#include <array>
template <std::size_t N>
void print(int (&arr)[N], std::size_t number_rows, std::size_t number_cols) {
assert(number_rows * number_cols == N);
for (std::size_t r = 0; r != number_rows; ++r) {
for (std::size_t c = 0; c != number_cols; ++c) {
std::cout << arr[r * number_cols + c] << ' ';
}
std::cout << '\n';
}
std::cout << '\n';
}
void copy(std::array<int, 4>& a, std::array<int, 4>& b) {
b = a;
}
int main() {
int vals[16] = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16};
print(vals, 4, 4);
auto s1 = reinterpret_cast<std::array<int, 4>*>(&vals[0]);
auto s2 = reinterpret_cast<std::array<int, 4>*>(&vals[4]);
copy(*s2, *s1);
print(vals, 4, 4);
}
Output
1 2 3 4
5 6 7 8
9 10 11 12
13 14 15 16
5 6 7 8
5 6 7 8
9 10 11 12
13 14 15 16
Edit: The Wider problem
Thanks for all the comments / answers. By popular request I am posting the wider problem for more context.
I am going to do that in 2 levels.
Level 1
This is the next layer out of what I would like to do:
#include "range/v3/algorithm/remove.hpp"
#include "range/v3/view/chunk.hpp"
#include <vector>
int main() {
std::vector<int> v{
1, 2, 3, 4,
5, 6, 7, 8,
9, 10, 11, 12,
13, 14, 15, 16
};
auto chunked = ranges::views::chunk(v, 4);
auto it = ranges::remove(chunked, 9, [](const auto& e) { return e[0]; }); // <== compile error
// expected result (xx = unspecified)
// std::vector<int> v{
// 1, 2, 3, 4,
// 5, 6, 7, 8,
// 13, 14, 15, 16,
// xx, xx, xx, xx
// };
// and it pointing at chunked[3] (ie the row of xx)
}
But ranges::remove complains that the ranges::view::chunk is "not permutable". This was confirmed here:
https://github.com/ericniebler/range-v3/issues/1760
So my next attempt was writing a "chunked range" which I could pass to ranges::remove. I did this in multiple ways. Several "worked" but are based on UB, including this way of using std::array<int,4> as a "chunk proxy" (and hence the OP above):
#include "range/v3/algorithm/remove.hpp"
#include "range/v3/view/chunk.hpp"
#include "range/v3/view/zip.hpp"
#include <iostream>
#include <iterator>
#include <vector>
class Integers {
public:
struct Iterator {
using chunk = std::array<int, 4>;
using iterator_category = std::forward_iterator_tag;
using difference_type = std::ptrdiff_t;
using value_type = chunk;
using pointer = value_type*;
using reference = value_type&;
template <class ForwardIt, class UnaryPredicate, class ChunkedForwardIt>
ForwardIt remove_if_par(ForwardIt first, ForwardIt last, UnaryPredicate p,
ChunkedForwardIt chunked_first, std::ptrdiff_t chunk_size) {
auto first_orig = first;
first = std::find_if(first, last, p);
// advance chunked_first in lockstep. TODO this is linear compelxity unless random_access_iter
std::advance(chunked_first, std::distance(first_orig, first) * chunk_size);
if (first != last) {
ForwardIt i = first;
ChunkedForwardIt chunk_i = chunked_first;
while (++i != last) {
std::advance(chunk_i, chunk_size);
if (!p(*i)) {
*first++ = std::move(*i);
// move chunk
auto loop_chunk_i = chunk_i;
for (std::ptrdiff_t ci = 0; ci != chunk_size; ++ci)
*chunked_first++ = std::move(*loop_chunk_i++);
}
}
}
return first;
}
Iterator();
Iterator(int* ptr) : current_row_(reinterpret_cast<chunk*>(ptr)) {} // <== UB here
reference operator*() const { return *current_row_; }
pointer operator->() { return current_row_; }
Iterator& operator++() {
++current_row_;
return *this;
}
Iterator operator++(int) {
Iterator tmp = *this;
++(*this);
return tmp;
}
friend bool operator==(const Iterator& a, const Iterator& b) {
return a.current_row_ == b.current_row_;
}
friend bool operator!=(const Iterator& a, const Iterator& b) {
return a.current_row_ != b.current_row_;
}
private:
chunk* current_row_;
};
Iterator begin() { return Iterator(&data_[0]); }
Iterator end() { return Iterator(&data_[16]); }
int data_[16];
};
template <std::size_t N>
void print(int (&arr)[N], std::size_t number_rows, std::size_t number_cols) {
assert(number_rows * number_cols == N);
for (std::size_t r = 0; r != number_rows; ++r) {
for (std::size_t c = 0; c != number_cols; ++c) {
std::cout << arr[r * number_cols + c] << ' ';
}
std::cout << '\n';
}
std::cout << '\n';
}
int main() {
Integers chunked{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16};
print(chunked.data_, 4, 4);
auto it = ranges::remove(chunked, 9, [](const auto& e) { return e[0]; });
print(chunked.data_, 4, 4);
Output (as desired but based on UB)
1 2 3 4
5 6 7 8
9 10 11 12
13 14 15 16
1 2 3 4
5 6 7 8
13 14 15 16
13 14 15 16
Level 2
The reason for being quite keen on using ranges, is because there is another layer outwards for my desired algorithm, in that there is actually a 1D parallel vector which I zipped together with the chunked one and then the remove condition is based on the 1D vector.
Note that both vectors are reasonably large here (~100-500k items), so I want to avoid making a copy. This is why I am not using | composition and the lazy ranges::views::filter, but using the eager ranges::remove instead which modifies the original containers (both need modifying).
The code below "works for me", but contains the UB as per OP:
#include "range/v3/algorithm/remove.hpp"
#include "range/v3/view/zip.hpp"
#include <cstddef>
#include <iostream>
#include <iterator>
#include <vector>
class Integers {
public:
struct Iterator {
using chunk = std::array<int, 4>;
using iterator_category = std::random_access_iterator_tag; // some requirements ommitted for brevity
using difference_type = std::ptrdiff_t;
using value_type = chunk;
using pointer = value_type*;
using reference = value_type&;
Iterator();
Iterator(int* ptr) : current_row_(reinterpret_cast<chunk*>(ptr)) {}
reference operator*() const { return *current_row_; }
pointer operator->() { return current_row_; }
Iterator& operator++() {
++current_row_;
return *this;
}
Iterator operator++(int) {
Iterator tmp = *this;
++(*this);
return tmp;
}
friend std::ptrdiff_t operator-(const Iterator& lhs, const Iterator& rhs) {
return lhs.current_row_ - rhs.current_row_;
}
friend bool operator==(const Iterator& a, const Iterator& b) {
return a.current_row_ == b.current_row_;
}
friend bool operator!=(const Iterator& a, const Iterator& b) {
return a.current_row_ != b.current_row_;
}
private:
chunk* current_row_;
};
Iterator begin() { return Iterator(&data_[0]); }
Iterator end() { return Iterator(&data_[16]); }
// fake the initialisation for brevity
std::vector<int> data_{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16};
};
void print(const auto& zipped) {
for (const auto& t: zipped) {
for (auto i: t.first) std::cout << i << ' ';
std::cout << " | " << t.second << '\n';
}
std::cout << '\n';
}
// no control over this api
void external_api(int* /* ptr */, std::size_t /* size */) {}
int main() {
Integers chunked;
std::vector<int> b{10, 20, 30, 40};
auto zipped = ranges::views::zip(chunked, b);
print(zipped);
auto it = ranges::remove(zipped, 30, [](const auto& e) { return e.second; });
auto relidx = it - zipped.begin();
chunked.data_.erase(chunked.data_.begin() + relidx * 4, chunked.data_.end());
b.erase(b.begin() + relidx, b.end());
print(zipped);
external_api(&chunked.data_[0], chunked.data_.size());
}
Output (as desired but based on UB):
1 2 3 4 | 10
5 6 7 8 | 20
9 10 11 12 | 30
13 14 15 16 | 40
1 2 3 4 | 10
5 6 7 8 | 20
13 14 15 16 | 40
Current best alternative
My best current alternative is to hand code both "zip" and "remove" using messy raw loops that deal with the "chunk 4" logic. Below is one version of this, which is basically a modified version of the std::remove implementation:
// no control over this api
void external_api(int* /* ptr */, std::size_t /* size */) {}
template <class ForwardIt, class UnaryPredicate, class ChunkedForwardIt>
ForwardIt remove_if_par(ForwardIt first, ForwardIt last, UnaryPredicate p,
ChunkedForwardIt chunked_first, std::ptrdiff_t chunk_size) {
auto first_orig = first;
first = std::find_if(first, last, p);
// advance chunked_first in lockstep. TODO this is linear compelxity unless random_access_iter
std::advance(chunked_first, std::distance(first_orig, first) * chunk_size);
if (first != last) {
ForwardIt i = first;
ChunkedForwardIt chunk_i = chunked_first;
while (++i != last) {
std::advance(chunk_i, chunk_size);
if (!p(*i)) {
*first++ = std::move(*i);
// move chunk
auto loop_chunk_i = chunk_i;
for (std::ptrdiff_t ci = 0; ci != chunk_size; ++ci)
*chunked_first++ = std::move(*loop_chunk_i++);
}
}
}
return first;
}
void print(const std::vector<int>& a, const std::vector<int>& chunked, std::size_t chunk_size) {
for (std::size_t i = 0; i != a.size(); ++i) {
std::cout << a[i] << " | ";
for (std::size_t j = 0; j != chunk_size; ++j)
std::cout << chunked[i * chunk_size + j] << ' ';
std::cout << '\n';
}
std::cout << '\n';
}
int main() {
std::vector<int> a{10, 20, 30, 40, 50, 60, 70};
std::vector<int> chunked{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14,
15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28};
static constexpr std::ptrdiff_t chunk_size = 4;
print(a, chunked, chunk_size);
auto it = remove_if_par(
a.begin(), a.end(), [](auto e) { return e % 20 == 0; }, chunked.begin(), chunk_size);
print(a, chunked, chunk_size);
a.erase(it, a.end());
chunked.erase(chunked.begin() + (it - a.begin()) * chunk_size, chunked.end());
print(a, chunked, chunk_size);
external_api(&chunked[0], chunked.size());
}
Output (as desired and without UB)
10 | 1 2 3 4
20 | 5 6 7 8
30 | 9 10 11 12
40 | 13 14 15 16
50 | 17 18 19 20
60 | 21 22 23 24
70 | 25 26 27 28
10 | 1 2 3 4
30 | 9 10 11 12
50 | 17 18 19 20
70 | 25 26 27 28
50 | 17 18 19 20
60 | 21 22 23 24
70 | 25 26 27 28
10 | 1 2 3 4
30 | 9 10 11 12
50 | 17 18 19 20
70 | 25 26 27 28
I haven't checked, but I suspect that the assembly code generated for this "raw iterator loop" version is at least as good as any range based alternative; probably better.
Can I use a std::array<int, N> to alias parts of a int[] without invoking UB?
No.
Is this UB?
Yes.
Is there another way?
Depends on what parts of your scenario you can change.
The copy function below is not under my control
Simplest solution would be to not use the copy function that's not useful for your use case.
the std::array<int, N>& is the return value of the operator*() of a custom iterator over the raw int[]
Seems that there's probably no good way to implement such operator*().
This seems like a good opportunity to define a custom range adaptor with custom assignment operators that you could return from the custom iterator.
As you have noticed, chunked_view is not mutable. What you can do instead is to use adjacent<N> | stride(n) to create a mutable chunked view(assuming N is known at compile time):
std::vector data {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16};
std::vector b{10, 20, 30, 40};
constexpr std::size_t chunk_size = 4;
auto chunked_view = data
| std::views::adjacent<chunk_size>
| std::views::stride(chunk_size);
auto zipped_view = std::views::zip(chunked_view, b);
auto removed_range = std::ranges::remove(zipped_view, 30, [](auto pair){ return std::get<1>(pair); });
data.resize(data.size() - removed_range.size() * chunk_size);
b.resize(b.size() - removed_range.size());
Now data will be 1,2,3,4,5,6,7,8,13,14,15,16.
One thing to note is that adjacent creates a tuple of references of all the elements, so you can't iterate through a tuple. However, you can create a span or views::counted by using the address of the first element and chunk_size, since the underlying data were contiguous:
for (auto& [tuple, key] : zipped_view) {
for (auto i : std::span(&std::get<0>(tuple), chunk_size)) std::cout << i << ' ';
std::cout << " | " << key << '\n';
}
You can also remove all the temporary views if desired, since all you needed from the remove function is how many elements were removed:
std::vector data {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16};
std::vector b{10, 20, 30, 40};
constexpr std::size_t chunk_size = 4;
auto removal_size =
std::ranges::remove(
std::views::zip(
data | std::views::adjacent<chunk_size> | std::views::stride(chunk_size)
, b)
, 30, [](auto pair){ return std::get<1>(pair); }
).size();
data.resize(data.size() - removal_size * chunk_size);
b.resize(b.size() - removal_size);

How to traverse shared_ptr<std::vector<int>>?

Can someone help me to finish this?
I have a vector of int. How to get back this vector<int> or traverse it in the foo() function?
std::vector<int> myIntVec = { 1, 2, 3, 4};
auto mBlock = std::make_shared<std::vector<int>>(myIntVec);
std::shared_ptr<void> in = std::static_pointer_cast<void>(mBlock);
foo(in);
void foo(std::shared_ptr<void> input) {
//TODO:: print 1, 2, 3, 4 extract from std::shared_ptr<void> input parameter
auto sp_vec = std::static_pointer_cast<std::vector<int>>(input);
auto vec = sp_vec.get();
}

Array/List/Vector type which fills intermediate values in C++?

I read about the vector/list/map types contained in the stardard template library of C++ and I couldn't find one that behaved as such:
myMap::autofillingmap m;
m[0] = 2; // m[] = [2]
m[1] = 3; // m[] = [2, 3]
m[4] = 5; // m[] = [2, 3, 0, 0, 5]
m[10] = 1 // m[] = [2, 3, 0, 0, 5, 0, 0, 0, 0, 0, 1]
Is there any cpp lib that emulates this behavior?
(beforehand observation: Yes, I do know this is not optimal methodology to work data structures out. Thanks)
 I have wrote myMap struct which use a std::vector<int> to store values, myMap I also have tried a basic Insert() method that interpret your example to assign values.
struct myMap {
vector<int> _data;
size_t _capacity;
myMap(size_t capacity);
const myMap& operator=(const myMap&);
int operator [] (int index);
void Insert(int idx, int value);
};
myMap::myMap(size_t capacity) : _capacity(capacity) {
_data.resize(capacity);
}
const myMap& myMap::operator=(const myMap& rhs){
_data = rhs._data;;
return *this;
}
int myMap::operator[](int idx) {
return _data[idx];
}
void myMap::Insert(int idx, int value) {
if (idx >= _data.size())
_data.resize(idx+1);
_data[idx] = value;
}
demo

Preserving block-allocator for boost::deque?

I want to make boost::container::deque to reuse the freed blocks instead of deallocating them and then allocating new ones.
The only configuration option allowed by boost::container::deque is the compile-time specification of the block size (in terms of either items or bytes count).
Indeed, specifying the block size is something that I want to use, but I also would like to specify the number of blocks that will be preserved after becoming free and reused when new blocks will be required. However, as illustrated here, for boost::container::deque this number is 0, so it deallocates as soon as a block becomes free! I want to make a deque with this number equal to 1.
I see an opportunity to achieve this by specifying a custom allocator. Consider this ugly one:
template < typename Block >
struct PreservingAllocator : std::allocator<Block>
{
using std::allocator<Block>::allocator;
Block* allocate(size_t nn)
{
if (nn == 1) if (auto oldBlock = m_reserve->exchange(nullptr); !!oldBlock) return oldBlock;
return std::allocator<Block>::allocate(nn);
}
void deallocate(Block* block, size_t nn)
{
if (nn == 1) block = m_reserve->exchange(block);
if (!!block) std::allocator<Block>::deallocate(block, nn);
}
private:
static constexpr auto Deleter = [](std::atomic<Block*>* pointer)
{
if (!!pointer) if (auto block = pointer->exchange(nullptr); !!block)
std::allocator<Block>{}.deallocate(block,1);
delete pointer;
};
std::shared_ptr<std::atomic<Block*>> m_reserve = {new std::atomic<Block*>{nullptr},Deleter};
};
So the questions are.
How can I specify an allocator of blocks for boost::container::deque (blocks, not items!)?
If there is a way, then will such specification support allocators with state?
If yes, then will the above-mentioned allocator go for it?
After all, if not in this way, how can I make a deque that will not deallocate at least one of its freed blocks and will reuse it later when a new block will be needed?
This gave me an excuse to play more with allocators. I opted for the polymorphic allocators - although that's only tangentially related¹.
Aside: The relation is that with custom allocators you often want to propagate the allocator to nested types that are allocator-aware. See "Advanced" below
Sample Element Type
struct X {
std::string key, value;
};
It doesn't get much simpler, although it allows us to experiment with sharing
the allocator with the nested strings later.
Tracing Memory Resource
Let's create a tracing memory resource. That's pretty straight forward, and we'll just forward to standard new/delete:
namespace pmr = boost::container::pmr;
struct tracing_resource : pmr::memory_resource {
uint64_t n = 0, total_bytes = 0;
virtual void* do_allocate(std::size_t bytes, std::size_t alignment) override {
n += 1;
total_bytes += bytes;
return pmr::new_delete_resource()->allocate(bytes, alignment);
}
virtual void do_deallocate(void* p, std::size_t bytes, std::size_t alignment) override {
if (p) {
n -= 1;
total_bytes -= bytes;
}
return pmr::new_delete_resource()->deallocate(p, bytes, alignment);
}
virtual bool do_is_equal(const memory_resource& other) const noexcept override {
return pmr::new_delete_resource()->is_equal(other);
}
};
We can inspect n (number of allocation) as well as the total bytes allocated at various points throughout our test code.
A Test Routine
Let's put it together in main, starting with our tracer:
tracing_resource tracer;
Let's mount a pooled resource on that:
pmr::unsynchronized_pool_resource res(&tracer);
auto allocations = [&] {
fmt::print("alloc: #{}, {} bytes, cached = {}\n", tracer.n, tracer.total_bytes, cache_buckets(res));
};
allocations();
This will print
alloc: #0, 0 bytes, cached = {0, 0, 0, 0, 0, 0, 0, 0, 0}
right out of the gate.
Now, let's start (re)allocating some deques in various patterns:
pmr::deque<X> collection(&res);
auto report = [&] {
fmt::print("collection = {}\nalloc: #{}, {} bytes, cached = {}\n", collection, tracer.n, tracer.total_bytes, cache_buckets(res));
};
std::vector data1 { X{"1", "eins"}, {"2", "zwei"}, {"3", "drei"}, };
std::vector data2 { X{"4", "vier"}, {"5", "fuenf"}, {"6", "sechs"}, };
std::vector data3 { X{"7", "sieben"}, {"8", "acht"}, {"9", "neun"}, };
auto i = 0;
for (auto const& data : {data1, data2, data3}) {
for (auto el : data) {
(i%2)
? collection.push_back(el)
: collection.push_front(el);
}
report();
collection.clear();
report();
}
This will append varying sequences at varying ends of the container. We won't
do much mutation, as this will become interesting when we make the strings also
use the pooled resource).
Live Demo
Live On Compiler Explorer
#include <boost/container/pmr/deque.hpp>
#include <boost/container/pmr/unsynchronized_pool_resource.hpp>
// debug output
#include <range/v3/all.hpp>
#include <fmt/ranges.h>
#include <fmt/ostream.h>
#include <iomanip>
namespace pmr = boost::container::pmr;
struct tracing_resource : pmr::memory_resource {
uint64_t n = 0, total_bytes = 0;
virtual void* do_allocate(std::size_t bytes, std::size_t alignment) override {
n += 1;
total_bytes += bytes;
return pmr::new_delete_resource()->allocate(bytes, alignment);
}
virtual void do_deallocate(void* p, std::size_t bytes, std::size_t alignment) override {
if (p) {
n -= 1;
total_bytes -= bytes;
}
return pmr::new_delete_resource()->deallocate(p, bytes, alignment);
}
virtual bool do_is_equal(const memory_resource& other) const noexcept override {
return pmr::new_delete_resource()->is_equal(other);
}
};
struct X {
std::string key, value;
friend std::ostream& operator<<(std::ostream& os, X const& x) {
return os << "(" << std::quoted(x.key) << ", " << std::quoted(x.value) << ")";
}
};
auto cache_buckets(pmr::unsynchronized_pool_resource& res) {
using namespace ::ranges;
return views::iota(0ull)
| views::take_exactly(res.pool_count())
| views::transform([&](auto idx) {
return res.pool_cached_blocks(idx);
});
}
int main() {
tracing_resource tracer;
{
pmr::unsynchronized_pool_resource res(&tracer);
auto allocations = [&] {
fmt::print("alloc: #{}, {} bytes, cached = {}\n", tracer.n, tracer.total_bytes, cache_buckets(res));
};
allocations();
{
pmr::deque<X> collection(&res);
auto report = [&] {
fmt::print("collection = {}\nalloc: #{}, {} bytes, cached = {}\n", collection, tracer.n, tracer.total_bytes, cache_buckets(res));
};
std::vector data1 { X{"1", "eins"}, {"2", "zwei"}, {"3", "drei"}, };
std::vector data2 { X{"4", "vier"}, {"5", "fuenf"}, {"6", "sechs"}, };
std::vector data3 { X{"7", "sieben"}, {"8", "acht"}, {"9", "neun"}, };
auto i = 0;
for (auto const& data : {data1, data2, data3}) {
for (auto el : data) {
(i%2)
? collection.push_back(el)
: collection.push_front(el);
}
report();
collection.clear();
report();
}
}
allocations();
}
fmt::print("alloc: #{}, {} bytes\n", tracer.n, tracer.total_bytes);
}
Prints
alloc: #0, 0 bytes, cached = {0, 0, 0, 0, 0, 0, 0, 0, 0}
collection = {("3", "drei"), ("2", "zwei"), ("1", "eins")}
alloc: #4, 1864 bytes, cached = {0, 0, 0, 0, 0, 1, 0, 0, 0}
collection = {}
alloc: #4, 1864 bytes, cached = {0, 0, 0, 0, 0, 2, 0, 0, 0}
collection = {("6", "sechs"), ("5", "fuenf"), ("4", "vier")}
alloc: #4, 1864 bytes, cached = {0, 0, 0, 0, 0, 2, 0, 0, 0}
collection = {}
alloc: #4, 1864 bytes, cached = {0, 0, 0, 0, 0, 2, 0, 0, 0}
collection = {("9", "neun"), ("8", "acht"), ("7", "sieben")}
alloc: #4, 1864 bytes, cached = {0, 0, 0, 0, 0, 2, 0, 0, 0}
collection = {}
alloc: #4, 1864 bytes, cached = {0, 0, 0, 0, 0, 2, 0, 0, 0}
alloc: #4, 1864 bytes, cached = {0, 0, 1, 0, 0, 3, 0, 0, 0}
alloc: #0, 0 bytes
Advanced
As promised, we can make the element type allocator aware and show propagation:
#include <boost/container/pmr/string.hpp>
// ...
struct X {
using allocator_type = pmr::polymorphic_allocator<X>;
template<typename K, typename V>
explicit X(K&& key, V&& value, allocator_type a = {})
: key(std::forward<K>(key), a), value(std::forward<V>(value), a) {}
pmr::string key, value;
};
Let's amend the test driver to emplace the elements from string literals:
std::vector data1 { std::pair{"1", "eins"}, {"2", "zwei"}, {"3", "drei"}, };
std::vector data2 { std::pair{"4", "vier"}, {"5", "fuenf"}, {"6", "sechs"}, };
std::vector data3 { std::pair{"7", "sieben"}, {"8", "acht"}, {"9", "neun"}, };
auto i = 0;
for (auto const& data : {data1, data2, data3}) {
for (auto [k,v] : data) {
(i%2)
? collection.emplace_back(k, v)
: collection.emplace_front(k, v);
}
For good measure, let's also mutate one of the nested string values:
collection.at(1).value.append(50, '*'); // thwart SSO
report();
collection.at(1).value = "sept";
report();
Again, demoing Live On Compiler Explorer
#include <boost/container/pmr/deque.hpp>
#include <boost/container/pmr/string.hpp>
#include <boost/container/pmr/unsynchronized_pool_resource.hpp>
// debug output
#include <range/v3/all.hpp>
#include <fmt/ranges.h>
#include <fmt/ostream.h>
#include <iomanip>
namespace pmr = boost::container::pmr;
struct tracing_resource : pmr::memory_resource {
uint64_t n = 0, total_bytes = 0;
virtual void* do_allocate(std::size_t bytes, std::size_t alignment) override {
n += 1;
total_bytes += bytes;
return pmr::new_delete_resource()->allocate(bytes, alignment);
}
virtual void do_deallocate(void* p, std::size_t bytes, std::size_t alignment) override {
if (p) {
n -= 1;
total_bytes -= bytes;
}
return pmr::new_delete_resource()->deallocate(p, bytes, alignment);
}
virtual bool do_is_equal(const memory_resource& other) const noexcept override {
return pmr::new_delete_resource()->is_equal(other);
}
};
struct X {
using allocator_type = pmr::polymorphic_allocator<X>;
template<typename K, typename V>
explicit X(K&& key, V&& value, allocator_type a = {})
: key(std::forward<K>(key), a), value(std::forward<V>(value), a) {}
pmr::string key, value;
friend std::ostream& operator<<(std::ostream& os, X const& x) {
return os << "(" << std::quoted(x.key.c_str()) << ", " << std::quoted(x.value.c_str()) << ")";
}
};
auto cache_buckets(pmr::unsynchronized_pool_resource& res) {
using namespace ::ranges;
return views::iota(0ull)
| views::take_exactly(res.pool_count())
| views::transform([&](auto idx) {
return res.pool_cached_blocks(idx);
});
}
int main() {
tracing_resource tracer;
{
pmr::unsynchronized_pool_resource res(&tracer);
auto allocations = [&] {
fmt::print("alloc: #{}, {} bytes, cached = {}\n", tracer.n, tracer.total_bytes, cache_buckets(res));
};
allocations();
{
pmr::deque<X> collection(&res);
auto report = [&] {
fmt::print("collection = {}\nalloc: #{}, {} bytes, cached = {}\n", collection, tracer.n, tracer.total_bytes, cache_buckets(res));
};
std::vector data1 { std::pair{"1", "eins"}, {"2", "zwei"}, {"3", "drei"}, };
std::vector data2 { std::pair{"4", "vier"}, {"5", "fuenf"}, {"6", "sechs"}, };
std::vector data3 { std::pair{"7", "sieben"}, {"8", "acht"}, {"9", "neun"}, };
auto i = 0;
for (auto const& data : {data1, data2, data3}) {
for (auto [k,v] : data) {
(i%2)
? collection.emplace_back(k, v)
: collection.emplace_front(k, v);
}
report();
collection.at(1).value.append(50, '*'); // thwart SSO
report();
collection.at(1).value = "sept";
report();
collection.clear();
report();
}
}
allocations();
}
fmt::print("alloc: #{}, {} bytes\n", tracer.n, tracer.total_bytes);
}
Prints:
alloc: #0, 0 bytes, cached = {0, 0, 0, 0, 0, 0, 0, 0, 0}
collection = {("3", "drei"), ("2", "zwei"), ("1", "eins")}
alloc: #4, 1864 bytes, cached = {0, 0, 0, 0, 0, 1, 0, 0, 0}
collection = {("3", "drei"), ("2", "zwei**************************************************"), ("1", "eins")}
alloc: #5, 2008 bytes, cached = {0, 0, 0, 0, 0, 1, 0, 0, 0}
collection = {("3", "drei"), ("2", "sept"), ("1", "eins")}
alloc: #5, 2008 bytes, cached = {0, 0, 0, 0, 0, 1, 0, 0, 0}
collection = {}
alloc: #5, 2008 bytes, cached = {0, 0, 0, 1, 0, 2, 0, 0, 0}
collection = {("6", "sechs"), ("5", "fuenf"), ("4", "vier")}
alloc: #5, 2008 bytes, cached = {0, 0, 0, 1, 0, 2, 0, 0, 0}
collection = {("6", "sechs"), ("5", "fuenf**************************************************"), ("4", "vier")}
alloc: #5, 2008 bytes, cached = {0, 0, 0, 0, 0, 2, 0, 0, 0}
collection = {("6", "sechs"), ("5", "sept"), ("4", "vier")}
alloc: #5, 2008 bytes, cached = {0, 0, 0, 0, 0, 2, 0, 0, 0}
collection = {}
alloc: #5, 2008 bytes, cached = {0, 0, 0, 1, 0, 2, 0, 0, 0}
collection = {("9", "neun"), ("8", "acht"), ("7", "sieben")}
alloc: #5, 2008 bytes, cached = {0, 0, 0, 1, 0, 1, 0, 0, 0}
collection = {("9", "neun"), ("8", "acht**************************************************"), ("7", "sieben")}
alloc: #5, 2008 bytes, cached = {0, 0, 0, 0, 0, 1, 0, 0, 0}
collection = {("9", "neun"), ("8", "sept"), ("7", "sieben")}
alloc: #5, 2008 bytes, cached = {0, 0, 0, 0, 0, 1, 0, 0, 0}
collection = {}
alloc: #5, 2008 bytes, cached = {0, 0, 0, 1, 0, 2, 0, 0, 0}
alloc: #5, 2008 bytes, cached = {0, 0, 1, 1, 0, 3, 0, 0, 0}
alloc: #0, 0 bytes
Conclusions
While I chose poymorphic allocators because they support scoped_allocator_adaptor<>-style propagation by default, all of the above can be created with statically typed allocators.
It does demonstrate that if you use a pool allocator, the deque behaviour becomes pooled.
Aside: Pool allocators exist that are able to forgo cleanup, which can be valid in some scenarios, e.g. where the entire memory pool lives on the stack anyways. This is a common allocation optimization technique, allowing large numbers of deallocations/destructions to be skipped.
Addressing some more points of your list:
Q. How can I specify an allocator of blocks for boost::container::deque
(blocks, not items!)?
A. I think the allocator is intrinsically always called for blocks,
because of how deques work.
Q. If there is a way, then will such specification support allocators
with state?
A. Standard library as well as Boost Container should all support
stateful allocators these days. When in doubt, Boost Container has your
back.
Q. If yes, then will the above-mentioned allocator go for it?
A. I didn't look closely at it, but you may put it in the same test-bed
to find out
Q. After all, if not in this way, how can I make a deque that will not
deallocate at least one of its freed blocks and will reuse it later when a
new block will be needed?
A. See above. I'm not sure I understand the exact intent of the "at
least one" clause, but I did note that Boost's deque implementation does do
a "private map" allocation — presumably for some block accounting overhead
— that stays around until the deque object is destroyed. This allocation
doesn't happen at (default) construction, but later.
Couldn't leave well enough alone - I wanted to prove that it can be done with stateful statically-typed allocators just the same.
This builds on
Boost Container's basic_string and deque
Boost Container's scoped_allocator_adaptor to get allocator propagation on uses_allocator-construction
Boost Pool's object_pool for a scoped pool that uses segregated storage, optional block size hinting and ordered allocations with any UserAllocator
Finally, I used my own (surprise! I forgot about that earlier) stateful allocator to complement object_pool¹. It's the allocator in the non_boost namespace.
Note that if you don't mind singleton pools, you can just use Boist's own pool allocators. This is obviously recommended because mine is not maintained by Boost.
Note: I sacrificed the pooling statistics for the sake of cleanliness (it's way messier to add these to the allocator than it is to decorate a polymorphic memory_resource), but I guess the profiler knows best in the end
Live On Compiler Explorer
#include <boost/container/deque.hpp>
#include <boost/container/string.hpp>
#include <boost/container/scoped_allocator.hpp>
#include <boost/pool/pool_alloc.hpp>
// debug output
#include <range/v3/all.hpp>
#include <fmt/ranges.h>
#include <fmt/ostream.h>
#include <iomanip>
namespace bc = boost::container;
namespace non_boost {
template <typename T, typename UserAllocator = boost::default_user_allocator_new_delete>
class fast_pool_allocator
{
public:
typedef T value_type;
typedef UserAllocator user_allocator;
typedef value_type * pointer;
typedef const value_type * const_pointer;
typedef value_type & reference;
typedef const value_type & const_reference;
typedef boost::pool<UserAllocator> pool_type;
typedef typename pool_type::size_type size_type;
typedef typename pool_type::difference_type difference_type;
template <typename U>
struct rebind {
typedef fast_pool_allocator<U, UserAllocator> other;
};
pool_type* _ref;
public:
fast_pool_allocator(pool_type& ref) : _ref(&ref) { }
fast_pool_allocator(fast_pool_allocator const&) = default;
fast_pool_allocator& operator=(fast_pool_allocator const&) = default;
// Not explicit, mimicking std::allocator [20.4.1]
template <typename U>
fast_pool_allocator(const fast_pool_allocator<U, UserAllocator> & other) : _ref(other._ref)
{ }
// Default destructor used.
static pointer address(reference r) { return &r; }
static const_pointer address(const_reference s) { return &s; }
static size_type max_size() { return (std::numeric_limits<size_type>::max)(); }
void construct(const pointer ptr, const value_type & t) { new (ptr) T(t); }
void destroy(const pointer ptr) { ptr->~T(); }
bool operator==(fast_pool_allocator const& rhs) const { return _ref == rhs._ref; }
bool operator!=(fast_pool_allocator const& rhs) const { return _ref != rhs._ref; }
pointer allocate(const size_type n)
{
const pointer ret = (n == 1)
? static_cast<pointer>( (_ref->malloc)() )
: static_cast<pointer>( _ref->ordered_malloc(n) );
if (ret == 0)
boost::throw_exception(std::bad_alloc());
return ret;
}
pointer allocate(const size_type n, const void * const) { return allocate(n); }
pointer allocate()
{
const pointer ret = static_cast<pointer>( (_ref->malloc)() );
if (ret == 0)
boost::throw_exception(std::bad_alloc());
return ret;
}
void deallocate(const pointer ptr, const size_type n)
{
#ifdef BOOST_NO_PROPER_STL_DEALLOCATE
if (ptr == 0 || n == 0)
return;
#endif
if (n == 1)
(_ref->free)(ptr);
else
(_ref->free)(ptr, n);
}
void deallocate(const pointer ptr) { (_ref->free)(ptr); }
};
//Specialization of fast_pool_allocator<void> required to make the allocator standard-conforming.
template<typename UserAllocator>
class fast_pool_allocator<void, UserAllocator> {
public:
typedef void* pointer;
typedef const void* const_pointer;
typedef void value_type;
template <class U> struct rebind {
typedef fast_pool_allocator<U, UserAllocator> other;
};
};
}
template <typename T> using Alloc
= bc::scoped_allocator_adaptor< non_boost::fast_pool_allocator<T> >;
struct X {
using allocator_type = Alloc<X>;
template<typename K, typename V>
explicit X(K&& key, V&& value, allocator_type a)
: key(std::forward<K>(key), a), value(std::forward<V>(value), a) {}
bc::basic_string<char, std::char_traits<char>, Alloc<char> > key, value;
friend std::ostream& operator<<(std::ostream& os, X const& x) {
return os << "(" << std::quoted(x.key.c_str()) << ", " << std::quoted(x.value.c_str()) << ")";
}
};
int main() {
boost::pool<boost::default_user_allocator_new_delete> _pool { sizeof(X) };
Alloc<X> alloc { _pool };
bc::deque<X, Alloc<X> > collection(alloc);
auto dump = [&] { fmt::print("collection = {}\n", collection); };
std::vector data1 { std::pair{"1", "eins"}, {"2", "zwei"}, {"3", "drei"}, };
std::vector data2 { std::pair{"4", "vier"}, {"5", "fuenf"}, {"6", "sechs"}, };
std::vector data3 { std::pair{"7", "sieben"}, {"8", "acht"}, {"9", "neun"}, };
auto i = 0;
for (auto const& data : {data1, data2, data3}) {
for (auto [k,v] : data) {
(i%2)
? collection.emplace_back(k, v)
: collection.emplace_front(k, v);
}
dump();
collection.at(1).value.append(50, '*'); // thwart SSO
dump();
collection.at(1).value = "sept";
dump();
collection.clear();
dump();
}
}
¹ (see Is there some way to use boost::obect_pool with faster free operations and Is there a BOOST pool fixed-sized allocator?)

C++ double pointer not known conversion

I am literally freaking out on the following :
template <class T> // int, float, double etc..
class Graph {
public:
// Documentation, it has to be between [0..100]
Graph(int size = 10, int density = 10, T range = 0):
m_size(size),
m_density(density),
m_range(range) {
generate();
}
~Graph() {
for (int i = 0; i < m_size; i++)
delete[] m_graph[i];
delete[] m_graph;
}
[..]
static Graph<T>* custom(T** cgraph, int size, T range) {
Graph<T> *graph = new Graph<T>(size, 10, range);
for (int i = 0; i < size; i++)
delete[] graph->m_graph[i];
delete[] graph->m_graph;
graph->m_graph = cgraph;
}
private:
T** m_graph;
[ .. ]
};
int nodes[4][4] = {
{ 6, 5, 2, 5 },
{ 5, 6, 3, 3 },
{ 1, 3, 6, 1 },
{ 5, 3, 1, 6 }
};
int main() {
Graph<int> *graph = Graph<int>::custom(nodes, 4, 5);
}
What is it poorly failing to compile reporting the following errors ?
g++ graph.cpp -o test_graph
graph.cpp: In function ‘int main()’:
graph.cpp:191:55: error: no matching function for call to ‘Graph<int>::custom(int [4][4], int, int)’
Graph<int> *graph = Graph<int>::custom(nodes, 4, 5);
^
graph.cpp:60:20: note: candidate: static Graph<T>* Graph<T>::custom(T**, int, T) [with T = int]
static Graph<T>* custom(T** cgraph, int size, T range) {
^
graph.cpp:60:20: note: no known conversion for argument 1 from ‘int [4][4]’ to ‘int**’
It looks so right to me, what's wrong ?
You need to make nodes by an array of pointers to int.
int nodes_v[4][4] = {
{ 6, 5, 2, 5 },
{ 5, 6, 3, 3 },
{ 1, 3, 6, 1 },
{ 5, 3, 1, 6 }
};
int *nodes[4] = { nodes_v[0], nodes_v[1], nodes_v[2], nodes_v[3] };
You also need to add an additional member to the Graph variable to mark that it is a custom graph, and if set, the destructor should not delete the memory.
Best to give Graph a private constructor which passes in the custom flag and doesn't bother allocating the memory if set.
Graph(int size, int density, T range, bool custom):
m_size(size),
m_density(density),
m_range(range),
m_custom {
}
Graph(int size = 10, int density = 10, T range = 0):
Graph(size, density, range, false) {
generate();
}
static Graph<T>* custom(T** cgraph, int size, T range) {
Graph<T> *graph = new Graph<T>(size, 10, range, true);
graph->m_graph = cgraph;
}
Finally, you need to handle copy constructor and assignment operator (start by just deleting them).