Why my code doesn't work with custom allocator? - c++

What is the problem and how to fix it?
Without trying to squeeze a custom allocator, my vector, at first glance, works correctly.
I will also be happy to point out any mistakes in my code. Or an example of the correct implementation of a custom vector or another container, that would help me.
This code doesn't work:
using MyLib::Vector;
int main()
{
//Vector<int> v; //Works fine
Vector<int> v(CustomAllocator<int>());
for (size_t i = 0; i < 256; i++) {
v.push_back(i); //Error: "expression must have class type"
}
}
CustomAllocator implementation (it should be fine):
template <typename T>
class CustomAllocator : public std::allocator<T>
{
private:
using Base = std::allocator<T>;
public:
T* allocate(size_t count){
std::cout << ">> Allocating " << count << " elements" << std::endl;
return Base::allocate(count);
}
T* allocate(size_t count, const void* p)
{
std::cout << ">> Allocating " << count << " elements" << std::endl;
return Base::allocate(count, p);
}
void deallocate(T* p, size_t count)
{
if (p != nullptr)
{
std::cout << ">> Deallocating " << count << " elements" << std::endl;
Base::deallocate(p, count);
}
}
};
Vector implementation:
namespace MyLib
{
template <typename T,
template <typename Y> class Allocator = std::allocator>
class Vector
{
private:
std::size_t capacityV;
std::size_t sizeV;
Allocator<T> alloc;
T* arr;
public:
typedef Allocator<T> AllocatorType;
typedef Vector<T, Allocator> VectorType;
using AllocTraits = std::allocator_traits<Allocator<T>>;
public:
explicit Vector(const AllocatorType& allocator = AllocatorType()) {
capacityV = 0;
sizeV = 0;
alloc = allocator;
arr = nullptr;
}
Vector(const std::initializer_list<T>& values,
const AllocatorType& allocator = AllocatorType()) {
sizeV = values.size();
alloc = allocator;
if (sizeV < 128)
capacityV = 128;
else
capacityV = (sizeV / 128) * 256; //that makes sense
arr = AllocTraits::allocate(alloc, capacityV);
AllocTraits::construct(alloc, arr, capacityV);
std::copy(values.begin(), values.end(), arr);
}
~Vector() {
if (arr)
AllocTraits::deallocate(alloc, arr, capacityV);
}
Vector(const Vector& rhs) {
capacityV = rhs.capacityV;
sizeV = rhs.sizeV;
arr = AllocTraits::allocate(alloc, capacityV);
std::copy(rhs.arr, rhs.arr + rhs.sizeV, arr);
}
Vector(Vector&& rhs) noexcept {
capacityV = std::move(rhs.capacityV);
sizeV = std::move(rhs.sizeV);
arr = std::move(rhs.arr);
}
Vector& operator = (const Vector& rhs) {
capacityV = rhs.capacityV;
sizeV = rhs.sizeV;
arr = AllocTraits::allocate(alloc, capacityV);
std::copy(rhs.arr, rhs.arr + rhs.sizeV, arr);
}
Vector& operator = (Vector&& rhs) {
capacityV = std::move(rhs.capacityV);
sizeV = std::move(rhs.sizeV);
arr = std::move(rhs.arr);
}
T& operator [](std::size_t i) noexcept {
if (i < sizeV)
return arr[i];
else
throw std::out_of_range("Wrong index!");
}
const T& operator [](std::size_t i) const noexcept {
if (i < sizeV)
return arr[i];
else
throw std::out_of_range("Wrong index!");
}
T* data() noexcept {
return arr;
}
const T* data() const noexcept {
return arr;
}
void push_back(const T& value) {
++sizeV;
if (!arr) {
if (!capacityV)
capacityV = 128;
arr = AllocTraits::allocate(alloc, capacityV);
}
if (sizeV > capacityV) {
if(capacityV > UINT32_MAX - 256)
throw std::runtime_error("Vector overflowed!");
size_t tmpCap = capacityV;
capacityV = (sizeV / 128) * 256; //Увеличим capacityV
T* buf = AllocTraits::allocate(alloc, capacityV);
std::move(arr, arr + sizeV - 1, buf);
AllocTraits::deallocate(alloc, arr, capacityV); //sizeof
arr = buf;
}
arr[sizeV - 1] = value;
}
void push_back(T&& value) {
++sizeV;
if (!arr) {
if (!capacityV)
capacityV = 128;
arr = AllocTraits::allocate(alloc, capacityV);
}
if (sizeV > capacityV) {
if (capacityV > UINT32_MAX - 256)
throw std::runtime_error("Vector overflowed!");
size_t tmpCap = capacityV;
capacityV = (sizeV / 128) * 256; //Увеличим capacityV
T* buf = AllocTraits::allocate(alloc, capacityV);
std::move(arr, arr + sizeV - 1, buf);
AllocTraits::deallocate(alloc, arr, capacityV); //sizeof
arr = buf;
}
arr[sizeV - 1] = std::move(value);
}
void pop_back() {
--sizeV;
}
void resize(std::size_t size) {
if (this->sizeV == size)
return;
if (this->sizeV > size) {
this->sizeV = size;
}
else {
size_t tmpSize = size;
if (capacityV >= size) {
this->sizeV = size;
for (size_t i = tmpSize - 1; i < this->sizeV; i++)
arr[i] = 0;
}
else {
size_t tmpCap = capacityV;
capacityV = (size / 128) * 256; //that makes sense
T* buf = AllocTraits::allocate(alloc, capacityV);
std::move(arr, arr + sizeV - 1, buf);
AllocTraits::deallocate(alloc, arr, capacityV); //sizeof
arr = buf;
this->sizeV = size;
for (size_t i = tmpSize - 1; i < this->sizeV; i++)
arr[i] = 0;
}
}
}
void reserve(std::size_t capacity) {
if (capacity > this->capacityV)
{
size_t tmpCap = capacity;
this->capacityV = capacity;
T* buf = AllocTraits::allocate(alloc, capacityV);
std::move(arr, arr + sizeV - 1, buf);
AllocTraits::deallocate(alloc, arr, capacityV); //sizeof
arr = buf;
}
}
std::size_t size() const noexcept {
return sizeV;
}
std::size_t capacity() const noexcept {
return capacityV;
}
bool empty() const noexcept {
return (sizeV == 0);
}
};
}

Vector<int> v(CustomAllocator<int>());
You got hit by the most vexing parse. Vector<int> v(CustomAllocator<int>()); can be parsed as a variable declaration or a function declaration and the grammar prefers the latter. Therefore, the compiler thinks that v is a function and this it why you get the "expression must have class type" error -- you can only invoke methods on values with a class type, but v is a function.
Even if you fixed that error using one of these options:
// C++03 solution (extra parens)
Vector<int> v((CustomAllocator<int>()));
// C++11 solution (uniform initialization)
Vector<int> v{CustomAllocator<int>{}};
Your code still wouldn't do what you expected, though it would run. Vector<int> is the same thing as Vector<int, std::allocator> and so v will still use the standard allocator.
Why doesn't this cause a compilation error? Because CustomAllocator<int> inherits std::allocator<int> (which it shouldn't!), so the std::allocator<int> copy constructor is used to slice your custom allocator into an std::allocator<int> and then the program proceeds using the standard allocator. Your CustomAllocator<int> temporary is basically converted into an std::allocator<int>.
To illustrate, both of the above "fixed" examples are roughly equivalent to this code (if we disregard some value copies/moves that are irrelevant to the program's observable behavior):
// Creates a custom allocator value.
CustomAllocator<int> a;
// Custom allocator is converted to std::allocator<int>.
std::allocator<int> b(a);
// std::allocator<int> is provided to the Vector constructor.
Vector<int> v(b);
The correct fix is to specify the second Vector type parameter and then the constructor arguments aren't even needed since the default constructor will do the right thing:
Vector<int, CustomAllocator> v;

Related

How to define a RandomAccessIterator over a pointer to a vector of chars?

I am implementing a kind of dataframe and I want to define a RandomAccessIterator over it, in order to execute the different std algorithms, such as the sorting one. The dataframe of the example contains two column "a" and "b":
a; b;
20; 21;
20; 19;
10; 11;
40; 41;
10; 11;
After sorting with a trivial selection sort this is the result:
a; b;
10; 11;
10; 11;
20; 19;
20; 21;
40; 41;
The problem that I am facing is that the std::sort does not work properly. And I don't know weather the implementation of the iterator is sound or not.
This is the code.
File: dataframe.hpp
#pragma once
#include <iostream>
#include <charconv>
#include <vector>
#include <memory>
#include <cstring>
#include <numeric>
#include "iterator.hpp"
namespace df
{
class Record;
class Column;
class Dataframe;
namespace types
{
enum class Base : char
{
CHAR = 'A',
UNSIGNED = 'U',
// Other types..
};
class Dtype
{
public:
Dtype(types::Base base, std::size_t size) : m_base_dtype{base}, m_size{size} {}
[[nodiscard]] auto name() const
{
return std::string{static_cast<char>(m_base_dtype)} + std::to_string(m_size);
}
[[nodiscard]] auto base() const { return m_base_dtype; }
[[nodiscard]] auto size() const { return m_size; }
[[nodiscard]] auto is_primitive() const
{
switch (base())
{
case types::Base::CHAR:
return size() == 1;
case types::Base::UNSIGNED:
return size() == 1 or size() == 2 or size() == 4 or size() == 8;
}
return false;
}
private:
types::Base m_base_dtype;
std::size_t m_size;
};
[[nodiscard]] static auto CHAR(const std::size_t size) { return Dtype(types::Base::CHAR, size); }
[[nodiscard]] static auto UNSIGNED(const std::size_t size) { return Dtype(types::Base::UNSIGNED, size); }
}
class Column
{
public:
Column(std::vector<char> &raw, const types::Dtype dtype) : m_raw{std::move(raw)}, m_dtype{dtype} {}
Column &operator=(Column &&c) = default; // Move constructor
[[nodiscard]] const auto &dtype() const { return m_dtype; }
[[nodiscard]] auto &raw() { return m_raw; }
[[nodiscard]] const auto &raw() const { return m_raw; }
[[nodiscard]] auto *data() { return m_raw.data(); }
[[nodiscard]] const auto *data() const { return m_raw.data(); }
private:
std::vector<char> m_raw;
types::Dtype m_dtype;
};
class Dataframe
{
public:
Dataframe(std::vector<char> &raw, std::vector<std::string> names, std::vector<types::Dtype> dtypes)
{
m_raw = std::move(raw);
m_column_dtypes = dtypes;
m_column_names = names;
m_record_size = 0;
for (const auto dt : dtypes)
{
m_column_offsets.emplace_back(m_record_size);
m_record_size += dt.size();
}
m_record_count = m_raw.size() / m_record_size;
}
Dataframe(std::vector<char> &raw, std::vector<types::Dtype> dtypes) : Dataframe(raw, {}, dtypes) {}
Dataframe &operator=(Dataframe &&c) = default; // Move constructor
[[nodiscard]] auto &raw() { return m_raw; }
[[nodiscard]] const auto &raw() const { return m_raw; }
[[nodiscard]] auto *data() { return m_raw.data(); }
[[nodiscard]] const auto *data() const { return m_raw.data(); }
// Iterators
[[nodiscard]] df::Iterator begin()
{
return df::Iterator{m_raw.data(), m_record_size};
}
[[nodiscard]] df::Iterator end()
{
return df::Iterator{m_raw.data() + m_raw.size(), m_record_size};
}
[[nodiscard]] auto shape() const { return std::make_pair(m_record_count, m_column_dtypes.size()); }
[[nodiscard]] auto record_count() const { return m_record_count; }
[[nodiscard]] auto record_size() const { return m_record_size; }
[[nodiscard]] const auto &names() const { return m_column_names; }
[[nodiscard]] const auto &dtypes() const { return m_column_dtypes; }
[[nodiscard]] const auto &offsets() const { return m_column_offsets; }
void print() { print(m_record_count); }
void print(const std::size_t initial_records)
{
// Print header
for (auto column_name : m_column_names)
{
std::cout << column_name << "; ";
}
std::cout << std::endl;
// Print rows
std::size_t records_to_print = std::min(initial_records, m_record_count);
for (std::size_t i = 0; i < records_to_print; i++)
{
const auto start_p = i * record_size();
auto start_field = 0;
auto end_field = 0;
for (auto field : m_column_dtypes)
{
end_field += field.size();
switch (field.base())
{
case types::Base::UNSIGNED:
{
std::uint64_t uint_value = 0;
memcpy(&uint_value, m_raw.data() + start_p + start_field, field.size());
std::cout << uint_value;
break;
}
case types::Base::CHAR:
{
std::string str_value = std::string(m_raw.data() + start_p + start_field, field.size());
std::cout << str_value;
break;
}
}
start_field = end_field;
// New column
std::cout << "; ";
}
// New row
std::cout << std::endl;
}
}
std::shared_ptr<Dataframe> copy() const
{
auto x = std::vector<char>(m_raw);
return std::make_shared<Dataframe>(x, std::vector<std::string>(m_column_names), std::vector<types::Dtype>(m_column_dtypes));
}
private:
std::vector<char> m_raw = {};
std::vector<std::string> m_column_names = {};
std::vector<types::Dtype> m_column_dtypes = {};
std::vector<std::size_t> m_column_offsets = {};
std::size_t m_record_size = {};
std::size_t m_record_count = {};
};
using namespace types;
static std::shared_ptr<Dataframe> read_from_vector(const std::vector<std::vector<std::string>> values, const std::vector<std::string> names, const std::vector<Dtype> dtypes)
{
const auto record_size = std::accumulate(dtypes.begin(), dtypes.end(), std::size_t{0},
[](std::size_t accum, const auto &m)
{ return accum + m.size(); });
const auto total_size = values.size() * record_size;
const std::size_t INCR_RECORDS = std::max(total_size / (10 * record_size), std::size_t{65536});
auto raw = std::vector<char>{};
std::size_t written_records = 0;
auto offsets = std::vector<std::size_t>{};
for (int offset = 0; const auto &kd : dtypes)
{
offsets.push_back(offset);
offset += kd.size();
}
for (auto value : values)
{
if (written_records >= raw.size() / record_size)
{
raw.resize(raw.size() + INCR_RECORDS * record_size, char{' '});
}
for (int i = 0; i < names.size(); i++)
{
const auto name = names[i];
const auto dtype = dtypes[i];
const auto offset = offsets[i];
const auto pos = written_records * record_size + offset;
switch (dtype.base())
{
case df::Base::CHAR:
{
const auto v = value[i];
const auto byte_to_copy = std::min(v.size(), dtype.size());
std::memcpy(raw.data() + pos,
v.data() + v.size() - byte_to_copy, byte_to_copy); // Prendo gli ultimi byte
break;
}
case df::Base::UNSIGNED:
{
const auto v = std::stoull(value[i]);
const auto byte_to_copy = dtype.size();
std::memcpy(raw.data() + pos, &v, byte_to_copy); // Prendo gli ultimi byte
break;
}
default:
throw std::runtime_error("ColumnType non riconosciuto");
}
}
written_records++;
}
raw.resize(written_records * record_size);
raw.shrink_to_fit();
return std::make_shared<Dataframe>(raw, names, dtypes);
}
}
File: iterator.hpp
#pragma once
#include <iostream>
#include <cstring>
namespace df
{
class Iterator
{
std::size_t size;
char *ptr;
public:
struct record_reference;
struct record_value
{
std::size_t size;
char *ptr;
record_value(const record_reference &t) : record_value(t.size, t.ptr){};
record_value(const std::size_t m_size, char *m_ptr)
{
this->size = m_size;
this->ptr = new char[this->size];
std::memcpy(ptr, m_ptr, this->size);
}
~record_value()
{
delete[] this->ptr;
}
};
struct record_reference
{
std::size_t size;
char *ptr;
record_reference(const std::size_t m_size, char *m_ptr)
{
this->size = m_size;
this->ptr = m_ptr;
}
record_reference(const record_reference &t)
{
this->size = t.size;
this->ptr = t.ptr;
}
// record_reference(const record_value &t) : record_reference(t.size, t.ptr) {};
record_reference &operator=(const record_value &t)
{
std::memcpy(ptr, t.ptr, size);
return *this;
}
record_reference &operator=(const record_reference &t)
{
std::memcpy(ptr, t.ptr, size);
return *this;
}
record_reference &operator=(char *t)
{
std::memcpy(ptr, t, size);
return *this;
}
operator char *()
{
return ptr;
}
operator const char *() const { return ptr; }
};
using iterator_category = std::random_access_iterator_tag;
using value_type = record_value;
using reference = record_reference;
using difference_type = std::ptrdiff_t;
// default constructible
Iterator() : size(0), ptr(nullptr)
{
}
// copy assignable
Iterator &operator=(const Iterator &t)
{
size = t.size;
ptr = t.ptr;
return *this;
}
Iterator(char *ptr, const std::size_t size) : size{size}, ptr(ptr)
{
}
record_reference operator*() const
{
return {size, ptr};
}
// Prefix
Iterator &operator++()
{
ptr += size;
return *this;
}
// Postfix
Iterator operator++(int)
{
auto tmp = *this;
++*this;
return tmp;
}
Iterator &operator--()
{
ptr -= size;
return *this;
}
difference_type operator-(const Iterator &it) const
{
return (this->ptr - it.ptr) / size;
}
Iterator operator+(const difference_type &offset) const
{
return Iterator(ptr + offset * size, size);
}
friend Iterator operator+(const difference_type &diff, const Iterator &it)
{
return it + diff;
}
Iterator operator-(const difference_type &diff) const
{
return Iterator(ptr - diff * size, size);
}
reference operator[](const difference_type &offset) const
{
return {size, ptr + offset * size};
}
bool operator==(const Iterator &it) const
{
return this->ptr == it.ptr;
}
bool operator!=(const Iterator &it) const
{
return !(*this == it);
}
bool operator<(const Iterator &it) const
{
return this->ptr < it.ptr;
}
bool operator>=(const Iterator &it) const
{
return this->ptr >= it.ptr;
}
bool operator>(const Iterator &it) const
{
return this->ptr > it.ptr;
}
bool operator<=(const Iterator &it) const
{
return this->ptr <= it.ptr;
}
Iterator &operator+=(const difference_type &diff)
{
ptr += diff * size;
return *this;
}
operator Iterator() const
{
return Iterator(ptr, size);
}
};
void swap(df::Iterator::record_reference a, df::Iterator::record_reference b)
{
unsigned char *p;
unsigned char *q;
unsigned char *const sentry = (unsigned char *)a.ptr + a.size;
for (p = (unsigned char *)a.ptr, q = (unsigned char *)b.ptr; p < sentry; ++p, ++q)
{
const unsigned char t = *p;
*p = *q;
*q = t;
}
}
}
File: comparator.hpp
#pragma once
#include <memory>
#include <functional>
#include "dataframe.hpp"
#include "iterator.hpp"
namespace compare
{
using comparator_fn = std::function<int(const df::Iterator::record_reference, const df::Iterator::record_reference)>;
template <typename T, std::size_t offset = 0, std::size_t size = sizeof(T)>
static inline comparator_fn make_comparator()
{
if constexpr (size == 3 or size == 5 or size == 7 or size > 8)
return [=](const df::Iterator::record_reference a, const df::Iterator::record_reference b)
{ return std::memcmp(a + offset, b + offset, size); };
return [](const df::Iterator::record_reference a, const df::Iterator::record_reference b)
{ return *(T *)(a + offset) < *(T *)(b + offset) ? -1 : *(T *)(b + offset) < *(T *)(a + offset) ? +1
: 0; };
}
template <typename T>
static inline comparator_fn make_comparator(const std::size_t offset)
{
return [=](const df::Iterator::record_reference a, const df::Iterator::record_reference b)
{ return *(T *)(a + offset) < *(T *)(b + offset) ? -1 : *(T *)(b + offset) < *(T *)(a + offset) ? +1
: 0; };
}
static inline comparator_fn make_column_comparator(const df::Dtype dtype, const std::size_t offset)
{
switch (dtype.base())
{
case df::Base::CHAR:
{
if (dtype.size() == 1)
return make_comparator<std::uint8_t>(offset);
else if (dtype.size() == 2)
return [=](const df::Iterator::record_reference a, const df::Iterator::record_reference b)
{ return std::memcmp(a + offset, b + offset, 2); }; // C'� qualche beneficio a fissare il 2? o conviene trattarlo come uno unsigned short?
return [=](const df::Iterator::record_reference a, const df::Iterator::record_reference b)
{ return std::memcmp(a + offset, b + offset, dtype.size()); };
}
case df::Base::UNSIGNED:
{
return [=](const df::Iterator::record_reference a, const df::Iterator::record_reference b)
{
std::uint64_t uint_value_a = 0;
std::uint64_t uint_value_b = 0;
std::memcpy(&uint_value_a, a + offset, dtype.size());
std::memcpy(&uint_value_b, b + offset, dtype.size());
return (uint_value_a < uint_value_b ? -1 : uint_value_a > uint_value_b ? +1
: 0);
};
}
default:
throw std::runtime_error("Unsupported dtype");
break;
}
}
static inline comparator_fn make_composite_two_way_comparator(const std::shared_ptr<df::Dataframe> &T)
{
const auto K = T->dtypes().size();
std::vector<comparator_fn> F;
for (int i = 0; i < K; i++)
{
F.emplace_back(make_column_comparator(T->dtypes()[i], T->offsets()[i]));
}
const auto comparator = [=](const df::Iterator::record_reference a, const df::Iterator::record_reference b)
{
for (int i = 0; i < K; i++)
{
// If equal go to the next column, otherwise return the result
// The return value is true if the first argument is less than the second
// and false otherwise
if (const auto result = F[i](a, b); result != 0)
return result < 0;
}
return false;
};
return comparator;
}
}
File: main.cpp
#include <iostream>
#include <vector>
#include "dataframe.hpp"
#include "comparator.hpp"
template <typename RandomAccessIterator, typename Comparator>
static void selection_sort(RandomAccessIterator first, RandomAccessIterator last, Comparator comp)
{
for (auto i = first; i != last; ++i)
{
auto min = i;
for (auto j = i + 1; j != last; ++j)
{
if (comp(*j, *min))
min = j;
}
df::Iterator::value_type temp = *i;
*i = *min;
*min = temp;
// Alternative
// std::iter_swap(i, min);
}
}
int main(int argc, char const *argv[])
{
std::vector<std::string> values{"20", "21", "20", "19", "10", "11", "40", "41", "10", "11"};
// Create a vector that contains values grouped by 2
std::vector<std::vector<std::string>> v;
for (int i = 0; i < values.size(); i += 2)
{
std::vector<std::string> temp;
temp.push_back(values[i]);
temp.push_back(values[i + 1]);
v.push_back(temp);
}
std::vector<std::string> column_names = {"a", "b"};
df::Dtype d = df::Dtype(df::Base::UNSIGNED, 4);
std::vector dtypes = {d, d};
// Create a dataframe
std::shared_ptr<df::Dataframe> df = df::read_from_vector(v, column_names, dtypes);
std::cout << "Before sorting" << std::endl;
df->print();
// This comparator sorts the dataframe first by column a and then by column b in ascending order
auto comparator = compare::make_composite_two_way_comparator(df);
selection_sort(df->begin(), df->end(), comparator);
std::cout << "\nAfter sorting" << std::endl;
df->print();
// With the std::sort it does not work
std::sort(df->begin(), df->end(), comparator);
return 0;
}
Your type is not a C++17 RandomAccessIterator, because it isn't a C++17 ForwardIterator, because reference is an object type, not a reference type.
The type It satisfies ForwardIterator if
Let T be the value type of It. The type std::iterator_traits<It>::reference must be either
T& or T&& if It satisfies OutputIterator (It is mutable), or
const T& or const T&& otherwise (It is constant),
(Other requirements elided)
You will be able to satisfy the C++20 concept std::random_access_iterator, because that relaxes the requirement on It::reference.
In C++17, the reference type of an iterator must be precisely value_type& in order for that iterator to be random access. Only input iterators can have the reference type be something other than value_type&. So in C++17, proxy iterators are limited to input iterators. And every algorithm written against C++17 has this expectation.
The C++20 ranges library adds the ability to have random access proxy iterators. And the C++20 algorithms that use those range concepts will respect them.

Move semantics for a custom vector and allocators

I'm implementing a basic std::vector by using an allocator, following PPP by Stroustrup, and in particular I'm sure that all the functions like resize, push_back, emplace_back, and so on are okay.
What is really puzzling me a bit is the copy and move semantics: it works now, but I find it a bit too hard-coded and, more importantly, it's leaking 1 byte in the move assignment, as can be seen from this
==7853== LEAK SUMMARY:
==7853== definitely lost: 1 bytes in 1 blocks
==7853== indirectly lost: 0 bytes in 0 blocks
==7853== possibly lost: 0 bytes in 0 blocks
In the following the whole code. I'd like to see especially how the move assignment should be implemented, because I believe that a problem is the fact that I have to move an allocator, and I'm totally new to this topic.
#include <iostream>
#include <memory>
#include <utility>
#include <initializer_list>
#include <string>
template <typename T, typename Allocator = std::allocator<T>>
class Vector
{
private:
Allocator allocator; //used to handle memory for the elements
T *elem;
std::size_t _size{};
std::size_t _capacity{};
void reserve(const std::size_t n)
{
if (_capacity < n) //if the capacity is smaller than what I want to allocate
{
T *tmp{std::allocator_traits<Allocator>::allocate(allocator, n)};
for (std::size_t i = 0; i < _size; ++i)
{
// allocator.construct(tmp + i, std::move(elem[i])); //tmp[i] = std::move(elem[i]);
std::allocator_traits<Allocator>::construct(allocator, tmp + i, elem[i]);
}
for (std::size_t i = 0; i < _size; ++i)
{
std::allocator_traits<Allocator>::destroy(allocator, elem + i);
}
std::allocator_traits<Allocator>::deallocate(allocator, elem, _capacity);
elem = tmp; //move the pointer
_capacity = n; //size increased!
}
}
void check_and_increase_capacity()
{
if (_capacity == 0)
{
reserve(8);
}
else if (_size == _capacity)
{
reserve(2 * _size);
}
}
template <typename O>
void _push_back(O &&x)
{
check_and_increase_capacity();
std::allocator_traits<Allocator>::construct(allocator, elem + _size, std::forward<O>(x));
++_size;
}
public:
explicit Vector(std::initializer_list<T> list) : elem{std::allocator_traits<Allocator>::allocate(allocator, list.size())}, _size{list.size()}, _capacity{list.size()}
{
std::uninitialized_copy(list.begin(), list.end(), begin());
std::cout << "custom cstr"
<< "\n";
}
~Vector() noexcept
{
for (std::size_t i = 0; i < _size; ++i)
{
std::allocator_traits<Allocator>::destroy(allocator, elem + i); //call the destructor
}
std::allocator_traits<Allocator>::deallocate(allocator, elem, _capacity); //deallocate memory
}
T *begin() { return elem; }
const T *begin() const { return elem; }
T *end() { return elem + _capacity; }
const T *end() const { return elem + _capacity; }
Vector(const Vector &v) : allocator{std::allocator_traits<Allocator>::select_on_container_copy_construction(v.allocator)}, elem{std::allocator_traits<Allocator>::allocate(allocator, v._capacity)}
{
T *tmp{std::allocator_traits<Allocator>::allocate(allocator, _capacity)};
_size = v._size;
_capacity = v._capacity;
for (std::size_t i = 0; i < _size; ++i)
{
std::allocator_traits<Allocator>::construct(allocator, tmp + i, std::move(v[i]));
}
std::uninitialized_copy(v.begin(), v.end(), begin()); //copy the elements
//destroy and deallocate tmp
for (std::size_t i = 0; i < _size; ++i)
{
std::allocator_traits<Allocator>::destroy(allocator, tmp + i);
}
std::allocator_traits<Allocator>::deallocate(allocator, tmp, _capacity);
}
Vector &operator=(const Vector &v)
{
T *tmp{std::allocator_traits<Allocator>::allocate(allocator, v._capacity)};
std::uninitialized_copy(v.begin(), v.end(), begin()); //copy the elements
for (std::size_t i = 0; i < v._size; ++i)
{
std::allocator_traits<Allocator>::destroy(allocator, elem + i);
}
std::allocator_traits<Allocator>::deallocate(allocator, elem, _capacity);
elem = tmp;
_size = v._size;
_capacity = v._capacity;
return *this;
}
Vector(Vector &&v) noexcept : allocator{std::move(v.allocator)}
{
elem = v.elem;
v.elem = nullptr;
_size = v._size;
v._size = 0;
_capacity = v._capacity;
v._capacity = 0;
std::cout << elem << "\n";
}
Vector &operator=(Vector &&v) noexcept
{
std::cout << "move assignment"
<< "\n";
allocator = std::move(v.allocator);
elem = v.elem;
v.elem = nullptr;
_size = v._size;
v._size = 0;
_capacity = v._capacity;
v._capacity = 0;
return *this;
}
void push_back(const T &x)
{
_push_back(x);
}
void push_back(T &&x)
{
_push_back(std::move(x));
}
template <typename... Types>
void emplace_back(Types &&...args)
{
check_and_increase_capacity();
std::allocator_traits<Allocator>::construct(allocator, elem + _size, std::forward<Types>(args)...);
}
T &operator[](const std::size_t i) noexcept { return elem[i]; }
const T &operator[](const std::size_t i) const noexcept { return elem[i]; }
friend std::ostream &operator<<(std::ostream &os, const Vector &v)
{
for (std::size_t i = 0; i < v._size; i++)
{
std::cout << v[i] << "\n";
}
return os;
}
void resize(const std::size_t newsize, T val = T{})
{
reserve(newsize);
for (std::size_t i = _size; i < newsize; ++i)
{
std::allocator_traits<Allocator>::construct(allocator, elem + i, val);
}
//destroy all the new extra elements
for (std::size_t i = newsize; i < _size; ++i)
{
std::allocator_traits<Allocator>::destroy(allocator, elem + i); //just destroy them, don't call release the memory!
}
_size = newsize;
}
};
struct Foo
{
std::string _s;
Foo()
{
std::cout << "foo cstr"
<< "\n";
};
explicit Foo(const std::string &s) : _s{s} {}
~Foo() = default;
};
int main()
{
Vector<int> v{1, 2, 3, 4, 5, 6, 7, 8, 9, 10};
std::cout << v << "\n";
v.push_back(11);
std::cout << "After push_back \n"
<< v << "\n";
Vector<Foo> w{{}, {}};
w.emplace_back();
v.resize(6);
std::cout << "After resize \n"
<< v << "\n";
// Copy/Move semantics tests
Vector<int> v1{1, 2, 3, 4, 5, 6, 7, 8, 9, 10};
Vector<int> v2{v1};
std::cout << "After copy cstr \n"
<< v2 << "\n";
v2.push_back(20);
std::cout << "after push_back \n"
<< v2 << "\n";
Vector<int> v3{};
v3 = v1;
std::cout << v3 << "and v1: \n"
<< v1 << "\n";
Vector<int> v4{std::move(v1)};
std::cout << v4 << "and v1: \n"
<< v1 << "\n";
Vector<int> v5{};
v5 = std::move(v4);
std::cout << v5 << "and v4: \n"
<< v4 << "\n";
return 0;
}

array of unknown size as class member for making objects of array at runtime(object creating time)

I want to construct a class of an array whose size is not known,
i want to make objects of array whose size would be initialized at the time when i would create an object(runtime)
class array_class{
public:
int size_of_array;
int arr[size_of_array];
array_class(int p_size_of_array){ //constructor
this->size_of_array=p_size_of_array;
}
};
it say's error of invalid use of non static data member, what wrong am i doing(what should i have known)?.
You can't have something like
void foo(int n)
{
int vals[n];
}
in C++.
The thing is called variable-length array and it's supported in C99.
By the way, it is the only thing I know which is supported in C99 and not supported in C++17 :).
std::vector is a nice alternative.
If you know the size of an array at the compile time, you may use std::array.
I want to construct a class of an array whose size is not known
(what should i have known)?
You should have known that is not not possible in C++. The size of all classes is compile time constant.
i want to make objects of array whose size would be initialized at the time when i would create an object(runtime)
You need to allocate the array dynamically.
A simple solution:
struct array_class {
std::vector<int> arr;
array_class(int p_size_of_array) : arr(p_size_of_array) {}
};
#eerorika and #Nestor have good answers. To add to them here is a simple implementation of a vector i did some time ago you can take as a reference.
As you will see a basic way to have a variable length data structure, is to delete the underlying array and create a larger one if more or less memory is needed.
#pragma once
#include <ostream>
#include <string>
#include <initializer_list>
#include <stdexcept>
static constexpr size_t min_sz = 5;
template<typename T>
class Vector
{
public:
class ConstIterator;
class Iterator;
using value_type = T;
using size_type = std::size_t;
using difference_type = std::ptrdiff_t;
using reference = value_type&;
using const_reference = const value_type &;
using pointer = value_type *;
using const_pointer = const value_type *;
using iterator = Vector::Iterator;
using const_iterator = Vector::ConstIterator;
/// constructors, destructor, assign
Vector()
{
allocEmpty(min_sz);
}
Vector(Vector<value_type> &vec)
{
allocEmpty(vec.max_sz);
for (size_type i = 0; i < vec.sz; ++i)
values[i] = vec.values[i];
sz = vec.sz;
}
Vector(size_type size)
{
allocEmpty(size);
}
Vector(const std::initializer_list<value_type> &list)
{
allocEmpty(list.size());
size_type i = 0;
for (const auto &val : list)
{
values[i] = val;
++i;
}
sz = list.size();
}
~Vector()
{
delete[] values;
}
Vector &operator=(const Vector &other)
{
reserve(other.sz);
for (size_type i = 0; i < other.sz; ++i)
values[i] = other.values[i];
sz = other.sz;
return *this;
}
/// element access
reference operator[](size_type position)
{
if (position >= sz)
throw std::out_of_range ("operator[] out of range: sz = " + std::to_string(sz) + " but position = " + std::to_string(position));
return values[position];
}
const_reference operator[](size_type position) const
{
if (position >= sz)
throw std::out_of_range ("operator[] out of range: sz = " + std::to_string(sz) + " but position = " + std::to_string(position));
return values[position];
}
const_reference front() const
{
if (sz == 0)
throw std::out_of_range ("front called on empty container");
return values[0];
}
const_reference back() const
{
if (sz == 0)
throw std::out_of_range ("back called on empty container");
return values[sz - 1];
}
pointer data()
{
if (sz > 0)
return values;
return nullptr;
}
const_pointer data() const
{
if (sz > 0)
return values;
return nullptr;
}
/// capacity
size_type size() const
{
return sz;
}
size_type capacity() const
{
return max_sz;
}
bool empty() const
{
return (sz == 0);
}
void reserve(size_type size)
{
if (size <= min_sz)
return;
value_type *newArray = new value_type[size];
for (size_type i = 0; i < sz; ++i)
newArray[i] = values[i];
delete[] values;
values = newArray;
max_sz = size;
}
void shrink_to_fit()
{
size_type newSize = (sz >= min_sz) ? sz : min_sz;
value_type *newArray = new value_type[newSize];
for (size_type i = 0; i < sz; ++i)
newArray[i] = values[i];
delete[] values;
values = newArray;
max_sz = newSize;
}
/// modifiers
void push_back(const value_type &value)
{
if (sz >= max_sz)
reserve(max_sz * 2);
values[sz] = value;
++sz;
}
void resize(size_type count)
{
reserve(count);
sz = count;
}
void resize(size_type count, const value_type &value)
{
reserve(count);
for (size_type i = sz; i < count; ++i)
values[i] = value;
sz = count;
}
void insert(size_type position, const value_type &value)
{
if (position > sz)
throw std::out_of_range ("insert out of range: sz = " + std::to_string(sz) + " but position = " + std::to_string(position));
if (sz >= max_sz)
reserve(max_sz * 2);
for (size_type i = sz; i > position; --i)
values[i] = values[i - 1];
values[position] = value;
++sz;
}
iterator insert(const_iterator pos, const_reference val)
{
auto diff = pos - begin();
if (diff < 0 || static_cast<size_type>(diff) > sz)
throw std::runtime_error("Iterator out of bounds");
size_type current = static_cast<size_type>(diff);
if (sz >= max_sz)
reserve(max_sz * 2);
for (size_t i = sz; i-->current;)
values[i + 1] = values[i];
values[current] = val;
++sz;
return iterator(values + current);
}
void erase(size_type position)
{
if (position >= sz)
throw std::out_of_range ("erase out of range: sz = " + std::to_string(sz) + " but position = " + std::to_string(position));
for (size_type i = position; i < sz - 1; ++i)
values[i] = values[i + 1];
--sz;
}
iterator erase(const_iterator pos)
{
auto diff = pos - begin();
if (diff < 0 || static_cast<size_type>(diff) >= sz)
throw std::runtime_error("Iterator out of bounds");
size_type current = static_cast<size_type>(diff);
for (size_type i = current; i < sz - 1; ++i)
values[i] = values[i + 1];
--sz;
return iterator(values + current);
}
void pop_back()
{
if (sz == 0)
throw std::out_of_range ("pop_back on empty container");
if (sz > 0) --sz;
}
void clear()
{
sz = 0;
}
/// iterators
iterator begin()
{
if (sz == 0)
return end();
return iterator(values);
}
iterator end()
{
return iterator(values + sz);
}
const_iterator begin() const
{
if (sz == 0)
return end();
return ConstIterator(values);
}
const_iterator end() const
{
return ConstIterator(values + sz);
}
/// private section
private:
void allocEmpty(size_type size)
{
auto newSize = (size > min_sz) ? size : min_sz;
sz = 0;
values = new value_type[newSize];
max_sz = newSize;
}
private:
value_type *values;
size_type sz;
size_type max_sz;
/// iterator implementations
public:
class Iterator
{
public:
using value_type = Vector::value_type;
using reference = Vector::reference;
using pointer = Vector::pointer;
using difference_type = Vector::difference_type;
using iterator_category = std::forward_iterator_tag;
public:
Iterator()
{
ptr = nullptr;
}
Iterator(pointer ptr)
{
this->ptr = ptr;
}
reference operator*() const
{
return *ptr;
}
pointer operator->() const
{
return ptr;
}
iterator& operator++()
{
++ptr;
return *this;
}
iterator operator++(int)
{
iterator it = *this;
++ptr;
return it;
}
iterator operator+ (difference_type difference) const
{
return iterator(ptr + difference);
}
bool operator==(const const_iterator &it) const
{
return it == ptr;
}
bool operator!=(const const_iterator &it) const
{
return it != ptr;
}
operator const_iterator() const
{
return const_iterator(ptr);
}
private:
pointer ptr;
};
class ConstIterator
{
public:
using value_type = Vector::value_type;
using reference = Vector::const_reference;
using pointer = Vector::const_pointer;
using difference_type = Vector::difference_type;
using iterator_category = std::forward_iterator_tag;
public:
ConstIterator()
{
ptr = nullptr;
}
ConstIterator(pointer ptr)
{
this->ptr = ptr;
}
reference operator*() const
{
return *ptr;
}
pointer operator->() const
{
return ptr;
}
const_iterator& operator++()
{
++ptr;
return *this;
}
const_iterator operator++(int)
{
const_iterator it = *this;
++ptr;
return it;
}
bool operator==(const const_iterator &it) const
{
return it.ptr == ptr;
}
bool operator!=(const const_iterator &it) const
{
return it.ptr != ptr;
}
Vector::difference_type operator-(const const_iterator &rop)
{
return ptr - rop.ptr;
}
private:
pointer ptr;
};
};
/// non-member functions
template<typename T>
bool operator==(const Vector<T> &lop, const Vector<T> &rop)
{
if (lop.size() != rop.size()) return false;
for (size_t i = 0; i < lop.size(); ++i)
{
if (lop[i] != rop[i])
return false;
}
return true;
}
template<typename T>
std::ostream& operator<<(std::ostream &out, const Vector<T> &vec)
{
out << '[';
for (size_t i = 0; i < vec.size(); ++i)
{
if (i > 0) out << ", ";
out << vec[i];
}
out << ']';
return out;
}

xutility read access violation on returning isstream with custom vector

So i have a custom made vector class with also an custom allocator:
#define _SCL_SECURE_NO_WARNINGS
#include <iostream>
#include <exception>
#include <sstream>
#include <string>
#include <iostream>
namespace mem {
template<typename T>
class allocator {
public:
//
T * allocate(int n); //allocate space for n objects of type T
void deallocate(T* p); //deallocate n objects of type T starting at p
void construct(T* p, const T& v); // construct a T with the value v in p
void destroy(T* p); // destroy the T in p
};
template<typename T> T* allocator<T>::allocate(int n) //allocate space for n objects of type T
{
T* res = (T*)malloc(sizeof(T)*n);
if (res == nullptr)
throw std::bad_alloc();
return res;
}
template<typename T> void allocator<T>::deallocate(T* p/*, int n*/) //deallocate n objects of type T starting at p
{
free(p);
}
template<typename T> void allocator<T>::construct(T* p, const T& v) // construct a T with the value v in p
{
new(p) T(v);
}
template<typename T> void allocator<T>::destroy(T* p) // destroy the T in p
{
p->~T();
}
}
namespace vec {
template<typename T, typename A = mem::allocator<T>>
class vector {
A alloc; //use allocate to handle memory for elements
int sz; //the size
T* elem; //a pointer to the elements
int space; //size + free space
public:
using size_type = unsigned long;
using value_type = T;
using iterator = T * ;
using const_iterator = const T*;
vector() :sz{ 0 }, elem{ nullptr }, space{ 0 } {}
explicit vector(int s) :sz{ s }, elem{ new T[s] }, space{ s }
{
for (int i = 0; i < sz; ++i) elem[i] = 0; // elements are initalized
}
vector(const vector& arg); //copy constructor
vector& operator =(const vector& a); //copy assignment
vector(vector&& a); //move constructor
vector& operator=(vector&& a); //move assignment
~vector() { delete[] elem; } //destructor
void reserve(int newalloc);
void resize(int newsize, T val = T()); //growth
void push_back(const T& val);
iterator begin() { return elem; }
const_iterator begin()const { return elem; }
iterator end() { return elem + sz; }
const_iterator end() const { return elem + sz; }
size_type size() { return sz; }
iterator back() { return end() - 1; }
const_iterator back() const { return end() - 1; }
};
template<typename T, typename A> vector<T, A>::vector(const vector<T, A>& a) //copy constructor
:sz{ a.sz }, elem{ new T[a.sz] }
{
copy(a.elem, a.elem + sz, elem);
}
template<typename T, typename A> vector<T, A>& vector<T, A>::operator =(const vector<T, A>& a) //copy assignment
{
if (this == &a) return *this; //return if self assignment
if (a.sz <= space) { //enough space no need for new allocation
for (int i = 0; i < a.sz; ++i) //copy elements
alloc.construct(&elem[i], a.elem[i]);
sz = a.sz;
return *this;
}
T* p = alloc.allocate(a.sz);//allocate new size
for (int i = 0; i < a.sz; ++i) //copy elements
alloc.construct(&p[i], a.elem[i]);
for (int i = 0; i < sz; ++i)
alloc.destroy(&elem[i]); //destroy
alloc.deallocate(elem);
space = sz = a.sz; //set new size
elem = p; //set new elements
return *this; //return a self reference
}
template<typename T, typename A> vector<T, A>::vector(vector<T, A>&& a) //move constructor
:sz{ a.sz }, elem{ a.elem }
{
a.sz = 0; //make a the empty vector
a.elem = nullptr;
}
template<typename T, typename A> vector<T, A>& vector<T, A>::operator=(vector<T, A>&& a) //move assignment
{
delete[] elem; //deallocate old space
elem = a.elem; //copy a's elem and sz
sz = a.sz;
a.elem = nullptr; //make a the empty vector
a.sz = 0;
return *this;
}
template<typename T, typename A> void vector<T, A>::reserve(int newalloc)
{
if (newalloc <= space) return; //never decrease allocation
T* p = alloc.allocate(newalloc); //alocate new space
for (int i = 0; i < sz; ++i)
alloc.construct(&p[i], elem[i]); //copy
for (int i = 0; i < sz; ++i)
alloc.destroy(&elem[i]); //destroy
alloc.deallocate(elem/*, space*/); //deallocate old space
elem = p;
space = newalloc;
}
template<typename T, typename A> void vector<T, A>::push_back(const T& val)
{
if (space == 0)
reserve(8);
else if (space == sz)
reserve(2 * space);
alloc.construct(&elem[sz], val); //add val at the end
++sz;
}
template<typename T, typename A> void vector<T, A>::resize(int newsize, T val)
{
reserve(newsize);
for (int i = sz; i < newsize; ++i)
alloc.construct(&elem[i], val); //construct
for (int i = newsize; i < sz; ++i)
alloc.destroy(&elem[i]); //destroy
sz = newsize;
}
template<typename T, typename A> std::ostream& operator<<(std::ostream& os, const vector<T, A>& obj) //requires Element<T> && Allocator<A>
{
for (const auto& x : obj)
os << x << ' ';
os << '\n';
return os;
}
template<typename T, typename A> std::istream& operator>>(std::istream& is, vector<T, A>& obj)
{
std::string line;
std::getline(is, line);
if (is.bad() || is.fail())
return is;
std::istringstream istr{ line };
vector<T, A> tmp;
while (true) {
T tmp_in = T();
istr >> tmp_in;
if (istr.fail()) {
istr.clear();
std::string invalid;
istr >> invalid;
continue;
}
tmp.push_back(tmp_in);
if (istr.eof())break;
}
for (const auto& x : tmp)
obj.push_back(x);
return is; //its sends me to xutility in this step
}
}
int main()
{
//vec::vector <int> test; //input "1 2 3 4 5" works fine with cin >>test
vec::vector <std::string> test; //input "this is a test" breaks into xutility
while (true) {
std::cin >> test; //breaks into xutility
std::cout << test << '\n';
}
}
Now the following problem. I can read in vector values if the vector is vector. So adding 1 2 3 4 5 is no problem.
Also vector works fine.
If i change the container type to vector and read in "hello this is a test" via cin >> test; I get a crahs in xutility.h:
// MEMBER FUNCTIONS FOR _Container_base12
inline void _Container_base12::_Orphan_all() _NOEXCEPT
{ // orphan all iterators
#if _ITERATOR_DEBUG_LEVEL == 2
if (_Myproxy != 0) //It points on this line
{ // proxy allocated, drain it
_Lockit _Lock(_LOCK_DEBUG);
for (_Iterator_base12 **_Pnext = &_Myproxy->_Myfirstiter;
*_Pnext != 0; *_Pnext = (*_Pnext)->_Mynextiter)
(*_Pnext)->_Myproxy = 0;
_Myproxy->_Myfirstiter = 0;
}
#endif /* _ITERATOR_DEBUG_LEVEL == 2 */
}
Visual Studio 2017 throws:
Exception thrown: read access violation.this was 0xC83FB858.
How can this header even get called and what is the meaning of it?
I really have no idea whats going on.
I replaced the custom vector with std::vector and then it also works.
edit: I minimized the code a bit and added a namespace for the vector. still the same behavior:
vec::vector<int> test;
std::cin >> test; //Ok: Example input 1 2 3 4 5
vec::vector<std::string> test;
std::cin >> test; //Mem access violation in xutility: Example input "this is a test"
The application crashes because:
The code does not use allocator consistently and ends up doing free() on memory that was allocated with new[].
The code does not maintain vector::space variable consistently.

Algorithm to unify contiguous chunks in a collection of chunks

I'm creating a pre-allocator with dynamic memory chunk size, and I need to unify contiguous memory chunks.
struct Chunk // Chunk of memory
{
Ptr begin, end; // [begin, end) range
}
struct PreAlloc
{
std::vector<Chunk> chunks; // I need to unify contiguous chunks here
...
}
I tried a naive solution, that, after sorting the chunks based on their begin, basically did a pass through the vector checking if the next chunk's begin was equal to the current chunk's end. I'm sure it could be improved.
Is there a good algorithm to unify contiguous ranges?
Information:
Chunks can never "overlap".
Chunks can have any size greater than 0.
Performance is the most important factor.
NOTE: there was an error in my original algorithm, where I only considered blocks to the left of the current block.
Use two associative tables (e.g. unordered_map), one mapping the begin address to the Chunk, another mapping the end to the Chunk. This lets you find the neighbouring blocks quickly. Alternatively, you can change the Chunk struct to store a pointer/id/whatever to the neighbouring Chunk, plus a flag to mark to tell if it's free.
The algorithm consists of scanning the vector of chunks once, while maintaining the invariant: if there is a neighbour to the left, you merge them; if there is a neighbour to the right, you merge them. At the end, just collect the remaining chunks.
Here's the code:
void unify(vector<Chunk>& chunks)
{
unordered_map<Ptr, Chunk> begins(chunks.size() * 1.25); // tweak this
unordered_map<Ptr, Chunk> ends(chunks.size() * 1.25); // tweak this
for (Chunk c : chunks) {
// check the left
auto left = ends.find(c.begin);
if (left != ends.end()) { // found something to the left
Chunk neighbour = left->second;
c.begin = neighbour.begin;
begins.erase(neighbour.begin);
ends.erase(left);
}
// check the right
auto right = begins.find(c.end);
if (right != begins.end()) { // found something to the right
Chunk neighbour = right->second;
c.end = neighbour.end;
begins.erase(right);
ends.erase(neighbour.end);
}
begins[c.begin] = c;
ends[c.end] = c;
}
chunks.clear();
for (auto x : begins)
chunks.push_back(x.second);
}
The algorithm has O(n) complexity assuming constant time access to the begins and ends tables (which is nearly what you get if you don't trigger rehashing, hence the "tweak this" comments). There are quite a few options to implement associative tables, make sure to try a few different alternatives; as pointed out in the comment by Ben Jackson, a hash table doesn't always make good use of cache, so even a sorted vector with binary searches might be faster.
If you can change the Chunk structure to store left/right pointers, you get a guaranteed O(1) lookup/insert/remove. Assuming you are doing this to consolidate free chunks of memory, the left/right checking can be done in O(1) during the free() call, so there is no need to consolidate it afterwards.
I think you can not do better then N log(N) - the naive approach. The idea using an unordered associative container I dislike - the hashing will degenerate performance. An improvement might be: keep the chunks sorted at each insert, making 'unify' O(N).
It seems you are writing some allocator, hence I dig up some old code of mine (with some adjustment regarding C++ 11 and without any warranty). The allocator is for small objects having a size <= 32 * sizeof(void*).
Code:
// Copyright (c) 1999, Dieter Lucking.
//
// Permission is hereby granted, free of charge, to any person or organization
// obtaining a copy of the software and accompanying documentation covered by
// this license (the "Software") to use, reproduce, display, distribute,
// execute, and transmit the Software, and to prepare derivative works of the
// Software, and to permit third-parties to whom the Software is furnished to
// do so, all subject to the following:
//
// The copyright notices in the Software and this entire statement, including
// the above license grant, this restriction and the following disclaimer,
// must be included in all copies of the Software, in whole or in part, and
// all derivative works of the Software, unless such copies or derivative
// works are solely in the form of machine-executable object code generated by
// a source language processor.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT
// SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE
// FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
// DEALINGS IN THE SOFTWARE.
//
#include <limits>
#include <chrono>
#include <iomanip>
#include <iostream>
#include <mutex>
#include <thread>
#include <vector>
// raw_allocator
// =============================================================================
class raw_allocator
{
// Types
// =====
public:
typedef std::size_t size_type;
typedef std::ptrdiff_t difference_type;
typedef void value_type;
typedef void* pointer;
typedef const void* const_pointer;
typedef unsigned char byte_type;
typedef byte_type* byte_pointer;
typedef const unsigned char* const_byte_pointer;
// Information
// ===========
public:
static size_type max_size() noexcept {
return std::numeric_limits<size_type>::max();
}
static size_type mem_size(size_type) noexcept;
// Allocation.System
// =================
public:
static pointer system_allocate(size_type) noexcept;
static void system_allocate(size_type, pointer&, size_type&) noexcept;
static void system_deallocate(pointer) noexcept;
// Allocation
// ==========
public:
static void allocate(size_type, pointer& result, size_type& capacity) noexcept;
static pointer allocate(size_type n) noexcept {
pointer result;
allocate(n, result, n);
return result;
}
static void deallocate(pointer p, size_type n) noexcept;
// Allocation.Temporary:
//======================
public:
static void allocate_temporary(size_type, pointer& result,
size_type& capacity) noexcept;
static pointer allocate_temporary(size_type n) noexcept {
pointer result;
allocate_temporary(n, result, n);
return result;
}
static void deallocate_temporary(pointer, size_type) noexcept;
// Logging
// =======
public:
static void log(std::ostream& stream);
};
// static_allocator
// =============================================================================
template<class T> class static_allocator;
template<>
class static_allocator<void>
{
public:
typedef void value_type;
typedef void* pointer;
typedef const void* const_pointer;
template<class U> struct rebind
{
typedef static_allocator<U> other;
};
};
template<class T>
class static_allocator
{
// Types
// =====
public:
typedef raw_allocator::size_type size_type;
typedef raw_allocator::difference_type difference_type;
typedef T value_type;
typedef T& reference;
typedef const T& const_reference;
typedef T* pointer;
typedef const T* const_pointer;
template<class U> struct rebind
{
typedef static_allocator<U> other;
};
// Construction/Destruction
// ========================
public:
static_allocator() noexcept {};
static_allocator(const static_allocator&) noexcept {};
~static_allocator() noexcept {};
// Information
// ===========
public:
static size_type max_size() noexcept {
return raw_allocator::max_size() / sizeof(T);
}
static size_type mem_size(size_type n) noexcept {
return raw_allocator::mem_size(n * sizeof(T)) / sizeof(T);
}
static pointer address(reference x) {
return &x;
}
static const_pointer address(const_reference x) {
return &x;
}
// Construct/Destroy
//==================
public:
static void construct(pointer p, const T& value) {
new ((void*) p) T(value);
}
static void destroy(pointer p) {
((T*) p)->~T();
}
// Allocation
//===========
public:
static pointer allocate(size_type n) noexcept {
return (pointer)raw_allocator::allocate(n * sizeof(value_type));
}
static void allocate(size_type n, pointer& result, size_type& capacity) noexcept
{
raw_allocator::pointer p;
raw_allocator::allocate(n * sizeof(value_type), p, capacity);
result = (pointer)(p);
capacity /= sizeof(value_type);
}
static void deallocate(pointer p, size_type n) noexcept {
raw_allocator::deallocate(p, n * sizeof(value_type));
}
// Allocation.Temporary
// ====================
static pointer allocate_temporary(size_type n) noexcept {
return (pointer)raw_allocator::allocate_temporary(n * sizeof(value_type));
}
static void allocate_temporary(size_type n, pointer& result,
size_type& capacity) noexcept
{
raw_allocator::pointer p;
raw_allocator::allocate_temporary(n * sizeof(value_type), p, capacity);
result = (pointer)(p);
capacity /= sizeof(value_type);
}
static void deallocate_temporary(pointer p, size_type n) noexcept {
raw_allocator::deallocate_temporary(p, n);
}
// Logging
// =======
public:
static void log(std::ostream& stream) {
raw_allocator::log(stream);
}
};
template <class T1, class T2>
inline bool operator ==(const static_allocator<T1>&,
const static_allocator<T2>&) noexcept {
return true;
}
template <class T1, class T2>
inline bool operator !=(const static_allocator<T1>&,
const static_allocator<T2>&) noexcept {
return false;
}
// allocator:
// =============================================================================
template<class T> class allocator;
template<>
class allocator<void>
{
public:
typedef static_allocator<void>::value_type value_type;
typedef static_allocator<void>::pointer pointer;
typedef static_allocator<void>::const_pointer const_pointer;
template<class U> struct rebind
{
typedef allocator<U> other;
};
};
template<class T>
class allocator
{
// Types
// =====
public:
typedef typename static_allocator<T>::size_type size_type;
typedef typename static_allocator<T>::difference_type difference_type;
typedef typename static_allocator<T>::value_type value_type;
typedef typename static_allocator<T>::reference reference;
typedef typename static_allocator<T>::const_reference const_reference;
typedef typename static_allocator<T>::pointer pointer;
typedef typename static_allocator<T>::const_pointer const_pointer;
template<class U> struct rebind
{
typedef allocator<U> other;
};
// Constructor/Destructor
// ======================
public:
template <class U>
allocator(const allocator<U>&) noexcept {}
allocator() noexcept {};
allocator(const allocator&) noexcept {};
~allocator() noexcept {};
// Information
// ===========
public:
size_type max_size() const noexcept {
return static_allocator<T>::max_size();
}
pointer address(reference x) const {
return static_allocator<T>::address(x);
}
const_pointer address(const_reference x) const {
return static_allocator<T>::address(x);
}
// Construct/Destroy
// =================
public:
void construct(pointer p, const T& value) {
static_allocator<T>::construct(p, value);
}
void destroy(pointer p) {
static_allocator<T>::destroy(p);
}
// Allocation
// ==========
public:
pointer allocate(size_type n, typename allocator<void>::const_pointer = 0) {
return static_allocator<T>::allocate(n);
}
void deallocate(pointer p, size_type n) {
static_allocator<T>::deallocate(p, n);
}
// Logging
// =======
public:
static void log(std::ostream& stream) {
raw_allocator::log(stream);
}
};
template <class T1, class T2>
inline bool operator ==(const allocator<T1>&, const allocator<T2>&) noexcept {
return true;
}
template <class T1, class T2>
inline bool operator !=(const allocator<T1>&, const allocator<T2>&) noexcept {
return false;
}
// Types
// =============================================================================
typedef raw_allocator::size_type size_type;
typedef raw_allocator::byte_pointer BytePointer;
struct LinkType
{
LinkType* Link;
};
struct FreelistType
{
LinkType* Link;
};
// const
// =============================================================================
// Memory layout:
// ==============
//
// Freelist
// Index Request Alignment
// =============================================================================
// [ 0 ... 7] [ 0 * align ... 8 * align] every 1 * align bytes
// [ 8 ... 11] ( 8 * align ... 16 * align] every 2 * align bytes
// [12 ... 13] ( 16 * align ... 24 * align] every 4 * align bytes
// [14] ( 24 * align ... 32 * align] 8 * align bytes
//
// temporary memory:
// [15] [ 0 * align ... 256 * align] 256 * align
static const unsigned FreeListArraySize = 16;
static const size_type FreelistInitSize = 16;
static const size_type MinAlign =
(8 < 2 * sizeof(void*)) ? 2 * sizeof(void*) : 8;
static const size_type MaxAlign = 32 * MinAlign;
static const size_type MaxIndex = 14;
static const size_type TmpIndex = 15;
static const size_type TmpAlign = 256 * MinAlign;
static const size_type IndexTable[] = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 8, 9, 9, 10,
10, 11, 11, 12, 12, 12, 12, 13, 13, 13, 13, 14, 14, 14, 14, 14, 14, 14, 14 };
static_assert(sizeof(IndexTable) / sizeof(size_type) == MaxAlign / MinAlign, "Invalid Index Table");
inline size_type get_index(size_type n) {
return IndexTable[long(n - 1) / MinAlign];
}
static const size_type AlignTable[] = { MinAlign * 1, MinAlign * 2, MinAlign
* 3, MinAlign * 4, MinAlign * 5, MinAlign * 6, MinAlign * 7, MinAlign * 8,
MinAlign * 10, MinAlign * 12, MinAlign * 14, MinAlign * 16, MinAlign * 20,
MinAlign * 24, MinAlign * 32, TmpAlign, };
static_assert(sizeof(AlignTable) / sizeof(size_type) == TmpIndex + 1, "Invalid Align Table");
inline size_type get_align(size_type i) {
return AlignTable[i];
}
// Thread
// ============================================================================
static LinkType* Freelist[FreeListArraySize];
static BytePointer HeapBeg;
static BytePointer HeapEnd;
static size_type TotalHeapSize;
static std::mutex FreelistMutex[FreeListArraySize] = { };
inline void lock_free_list(size_type i) {
FreelistMutex[i].lock();
}
inline void unlock_free_list(size_type i) {
FreelistMutex[i].unlock();
}
// Allocation
// ============================================================================
// Requiers: freelist[index] is locked
LinkType* allocate_free_list(size_type index) noexcept {
static std::mutex mutex;
const size_type page_size = 4096; // FIXME some system_page_size();
std::lock_guard<std::mutex> guard(mutex);
size_type heap_size = HeapEnd - HeapBeg;
size_type align = get_align(index);
if(heap_size < align) {
LinkType* new_list = (LinkType*)(HeapBeg);
// If a temporary list:
if(MaxAlign <= heap_size) {
LinkType* current = new_list;
LinkType* next;
while(2*MaxAlign <= heap_size) {
next = (LinkType*)(BytePointer(current) + MaxAlign);
current->Link = next;
current = next;
heap_size -= MaxAlign;
}
if(index != MaxIndex) lock_free_list(MaxIndex);
current->Link = Freelist[MaxIndex];
Freelist[MaxIndex] = new_list;
if(index != MaxIndex) unlock_free_list(MaxIndex);
new_list = (LinkType*)(BytePointer(current) + MaxAlign);
heap_size -= MaxAlign;
}
if(MinAlign <= heap_size) {
std::cout << "heap_size: " << heap_size << std::endl;
size_type i = get_index(heap_size);
if(heap_size < get_align(i)) --i;
if(index != i) lock_free_list(i);
new_list->Link = Freelist[i];
Freelist[i] = new_list;
if(index != i) unlock_free_list(i);
}
heap_size = FreelistInitSize * align + TotalHeapSize / FreelistInitSize;
heap_size = (((heap_size - 1) / page_size) + 1) * page_size;
HeapBeg = BytePointer(raw_allocator::system_allocate(heap_size));
if(HeapBeg) {
HeapEnd = HeapBeg + heap_size;
TotalHeapSize += heap_size;
}
else {
HeapEnd = 0;
size_type i = FreeListArraySize;
while(HeapBeg == 0) {
--i;
if(i <= index) return 0;
lock_free_list(i);
if(Freelist[i]) {
heap_size = get_align(i);
HeapBeg = (BytePointer)(Freelist[i]);
HeapEnd = HeapBeg + heap_size;
Freelist[i] = Freelist[i]->Link;
}
unlock_free_list(i);
}
}
}
size_type size = FreelistInitSize * align;
size_type count = FreelistInitSize;
if(heap_size < size) {
count = heap_size / align;
size = align * count;
}
LinkType* beg_list = (LinkType*)(HeapBeg);
LinkType* end_list = beg_list;
while(--count) {
LinkType* init = (LinkType*)(BytePointer(end_list) + align);
end_list->Link = init;
end_list = init;
}
LinkType*& freelist = Freelist[index];
end_list->Link = freelist;
freelist = beg_list;
HeapBeg += size;
return freelist;
}
// raw_allocator
// ============================================================================
// size
// ====
raw_allocator::size_type
raw_allocator::mem_size(size_type n) noexcept {
if( ! n) return 0;
else {
if(n <= MaxAlign) return get_align(get_index(n));
else return ((difference_type(n) - 1) / difference_type(MaxAlign)) * MaxAlign
+ MaxAlign;
}
}
// allocation.system
// =================
raw_allocator::pointer raw_allocator::system_allocate(size_type n) noexcept
{
return ::malloc(n);
}
void raw_allocator::system_allocate(size_type n, pointer& p, size_type& capacity) noexcept
{
capacity = mem_size(n);
p = ::malloc(capacity);
if(p == 0) capacity = 0;
}
void raw_allocator::system_deallocate(pointer p) noexcept {
::free(p);
}
// allocation
// ==========
void raw_allocator::allocate(size_type n, pointer& p, size_type& capacity) noexcept
{
if(n == 0 || MaxAlign < n) system_allocate(n, p, capacity);
else {
p = 0;
capacity = 0;
size_type index = get_index(n);
lock_free_list(index);
LinkType*& freelist = Freelist[index];
if(freelist == 0) {
freelist = allocate_free_list(index);
}
if(freelist != 0) {
p = freelist;
capacity = get_align(index);
freelist = freelist->Link;
}
unlock_free_list(index);
}
}
void raw_allocator::deallocate(pointer p, size_type n) noexcept {
if(p) {
if(n == 0 || MaxAlign < n) system_deallocate(p);
else {
size_type index = get_index(n);
lock_free_list(index);
LinkType*& freelist = Freelist[index];
LinkType* new_list = ((LinkType*)(p));
new_list->Link = freelist;
freelist = new_list;
unlock_free_list(index);
}
}
}
// allocation.temporary
// ====================
void raw_allocator::allocate_temporary(size_type n, pointer& p,
size_type& capacity) noexcept
{
if(n == 0 || size_type(TmpAlign) < n) system_allocate(n, p, capacity);
else {
p = 0;
capacity = 0;
lock_free_list(TmpIndex);
LinkType*& freelist = Freelist[TmpIndex];
if(freelist == 0) freelist = allocate_free_list(TmpIndex);
if(freelist != 0) {
p = freelist;
freelist = freelist->Link;
capacity = TmpAlign;
}
unlock_free_list(TmpIndex);
}
}
void raw_allocator::deallocate_temporary(pointer p, size_type n) noexcept {
if(p) {
if(n == 0 || size_type(TmpAlign) < n) system_deallocate(p);
else {
lock_free_list(TmpIndex);
LinkType*& freelist = Freelist[TmpIndex];
LinkType* new_list = ((LinkType*)(p));
new_list->Link = freelist;
freelist = new_list;
unlock_free_list(TmpIndex);
}
}
}
void raw_allocator::log(std::ostream& stream) {
stream << " Heap Size: " << TotalHeapSize << '\n';
size_type total_size = 0;
for (unsigned i = 0; i < FreeListArraySize; ++i) {
size_type align = get_align(i);
size_type size = 0;
size_type count = 0;
lock_free_list(i);
LinkType* freelist = Freelist[i];
while (freelist) {
size += align;
++count;
freelist = freelist->Link;
}
total_size += size;
unlock_free_list(i);
stream << " Freelist: " << std::setw(4) << align << ": " << size
<< " [" << count << ']' << '\n';
}
size_type heap_size = HeapEnd - HeapBeg;
stream << " Freelists: " << total_size << '\n';
stream << " Free Heap: " << heap_size << '\n';
stream << " Allocated: " << TotalHeapSize - total_size - heap_size
<< '\n';
}
int main() {
const unsigned sample_count = 100000;
std::vector<char*> std_allocate_pointers;
std::vector<char*> allocate_pointers;
std::vector<unsigned> sample_sizes;
typedef std::chrono::nanoseconds duration;
duration std_allocate_duration;
duration std_deallocate_duration;
duration allocate_duration;
duration deallocate_duration;
std::allocator<char> std_allocator;
allocator<char> allocator;
for (unsigned i = 0; i < sample_count; ++i) {
if (std::rand() % 2) {
unsigned size = unsigned(std::rand()) % MaxAlign;
//std::cout << " Allocate: " << size << std::endl;
sample_sizes.push_back(size);
{
auto start = std::chrono::high_resolution_clock::now();
auto p = std_allocator.allocate(size);
auto end = std::chrono::high_resolution_clock::now();
std_allocate_pointers.push_back(p);
std_allocate_duration += std::chrono::duration_cast<duration>(
end - start);
}
{
auto start = std::chrono::high_resolution_clock::now();
auto p = allocator.allocate(size);
auto end = std::chrono::high_resolution_clock::now();
allocate_pointers.push_back(p);
allocate_duration += std::chrono::duration_cast<duration>(
end - start);
}
}
else {
if (!sample_sizes.empty()) {
char* std_p = std_allocate_pointers.back();
char* p = allocate_pointers.back();
unsigned size = sample_sizes.back();
//std::cout << "Deallocate: " << size << std::endl;
{
auto start = std::chrono::high_resolution_clock::now();
std_allocator.deallocate(std_p, size);
auto end = std::chrono::high_resolution_clock::now();
std_deallocate_duration += std::chrono::duration_cast<
duration>(end - start);
}
{
auto start = std::chrono::high_resolution_clock::now();
allocator.deallocate(p, size);
auto end = std::chrono::high_resolution_clock::now();
deallocate_duration += std::chrono::duration_cast<duration>(
end - start);
}
std_allocate_pointers.pop_back();
allocate_pointers.pop_back();
sample_sizes.pop_back();
}
}
}
for (unsigned i = 0; i < sample_sizes.size(); ++i) {
unsigned size = sample_sizes[i];
std_allocator.deallocate(std_allocate_pointers[i], size);
allocator.deallocate(allocate_pointers[i], size);
}
std::cout << "std_allocator: "
<< (std_allocate_duration + std_deallocate_duration).count() << " "
<< std_allocate_duration.count() << " "
<< std_deallocate_duration.count() << std::endl;
std::cout << " allocator: "
<< (allocate_duration + deallocate_duration).count() << " "
<< allocate_duration.count() << " " << deallocate_duration.count()
<< std::endl;
raw_allocator::log(std::cout);
return 0;
}
Note: The raw allocator never release memory to the system (That
might be a bug).
Note: Without optimizations enabled the performance
is lousy (g++ -std=c++11 -O3 ...)
Result:
std_allocator: 11645000 7416000 4229000
allocator: 5155000 2758000 2397000
Heap Size: 94208
Freelist: 16: 256 [16]
Freelist: 32: 640 [20]
Freelist: 48: 768 [16]
Freelist: 64: 1024 [16]
Freelist: 80: 1280 [16]
Freelist: 96: 1536 [16]
Freelist: 112: 1792 [16]
Freelist: 128: 2176 [17]
Freelist: 160: 5760 [36]
Freelist: 192: 6144 [32]
Freelist: 224: 3584 [16]
Freelist: 256: 7936 [31]
Freelist: 320: 10240 [32]
Freelist: 384: 14208 [37]
Freelist: 512: 34304 [67]
Freelist: 4096: 0 [0]
Freelists: 91648
Free Heap: 2560
Allocated: 0
It seemed like an interesting problem so I invested some time in it. The aproach you took is far from being naive. Actually it has pretty good results. It can definetly be optimized further though. I will assume the list of chunks is not already sorted because your algo is probably optimal then.
To optimize it my aproach was to optimize the sort itself eliminating the chunks that can be combined during the sort, thus making the sort faster for the remaining elements.
The code below is basically a modified version of bubble-sort. I also implemented your solution using std::sort just for comparison.
The results are suprisingly good using my also. For a data set of 10 million chunks the combined sort with the merge of chunks performs 20 times faster.
The output of the code is (algo1 is std::sort followed by merging consecutive elements, algo 2 is the sort optimized with removing the chunks that can be merged):
generating input took: 00:00:19.655999
algo 1 took 00:00:00.968738
initial chunks count: 10000000, output chunks count: 3332578
algo 2 took 00:00:00.046875
initial chunks count: 10000000, output chunks count: 3332578
You can probably improve it further using a better sort algo like introsort.
full code:
#include <vector>
#include <map>
#include <set>
#include <iostream>
#include <boost\date_time.hpp>
#define CHUNK_COUNT 10000000
struct Chunk // Chunk of memory
{
char *begin, *end; // [begin, end) range
bool operator<(const Chunk& rhs) const
{
return begin < rhs.begin;
}
};
std::vector<Chunk> in;
void generate_input_data()
{
std::multimap<int, Chunk> input_data;
Chunk chunk;
chunk.begin = 0;
chunk.end = 0;
for (int i = 0; i < CHUNK_COUNT; ++i)
{
int continuous = rand() % 3; // 66% chance of a chunk being continuous
if (continuous)
chunk.begin = chunk.end;
else
chunk.begin = chunk.end + rand() % 100 + 1;
int chunk_size = rand() % 100 + 1;
chunk.end = chunk.begin + chunk_size;
input_data.insert(std::multimap<int, Chunk>::value_type(rand(), chunk));
}
// now we have the chunks randomly ordered in the map
// will copy them in the input vector
for (std::multimap<int, Chunk>::const_iterator it = input_data.begin(); it != input_data.end(); ++it)
in.push_back(it->second);
}
void merge_chunks_sorted(std::vector<Chunk>& chunks)
{
if (in.empty())
return;
std::vector<Chunk> res;
Chunk ch = in[0];
for (size_t i = 1; i < in.size(); ++i)
{
if (in[i].begin == ch.end)
{
ch.end = in[i].end;
} else
{
res.push_back(ch);
ch = in[i];
}
}
res.push_back(ch);
chunks = res;
}
void merge_chunks_orig_algo(std::vector<Chunk>& chunks)
{
std::sort(in.begin(), in.end());
merge_chunks_sorted(chunks);
}
void merge_chunks_new_algo(std::vector<Chunk>& chunks)
{
size_t new_last_n = 0;
Chunk temp;
do {
int last_n = new_last_n;
new_last_n = chunks.size() - 1;
for (int i = chunks.size() - 2; i >= last_n; --i)
{
if (chunks[i].begin > chunks[i + 1].begin)
{
if (chunks[i].begin == chunks[i + 1].end)
{
chunks[i].begin = chunks[i + 1].begin;
if (i + 1 != chunks.size() - 1)
chunks[i + 1] = chunks[chunks.size() - 1];
chunks.pop_back();
} else
{
temp = chunks[i];
chunks[i] = chunks[i + 1];
chunks[i + 1] = temp;
}
new_last_n = i + 1;
} else
{
if (chunks[i].end == chunks[i + 1].begin)
{
chunks[i].end = chunks[i + 1].end;
if (i + 1 != chunks.size() - 1)
chunks[i + 1] = chunks[chunks.size() - 1];
chunks.pop_back();
}
}
}
} while (new_last_n < chunks.size() - 1);
}
void run_algo(void (*algo)(std::vector<Chunk>&))
{
static int count = 1;
// allowing the algo to modify the input vector is intentional
std::vector<Chunk> chunks = in;
size_t in_count = chunks.size();
boost::posix_time::ptime start = boost::posix_time::microsec_clock::local_time();
algo(chunks);
boost::posix_time::ptime stop = boost::posix_time::microsec_clock::local_time();
std::cout<<"algo "<<count++<<" took "<<stop - start<<std::endl;
// if all went ok, statistically we should have around 33% of the original chunks count in the output vector
std::cout<<" initial chunks count: "<<in_count<<", output chunks count: "<<chunks.size()<<std::endl;
}
int main()
{
boost::posix_time::ptime start = boost::posix_time::microsec_clock::local_time();
generate_input_data();
boost::posix_time::ptime stop = boost::posix_time::microsec_clock::local_time();
std::cout<<"generating input took:\t"<<stop - start<<std::endl;
run_algo(merge_chunks_orig_algo);
run_algo(merge_chunks_new_algo);
return 0;
}
I've seen below you mention n is not that high. so I rerun the test with 1000 chunks, 1000000 runs to make the times significant. The modified bubble sort still performs 5 times better. Basically for 1000 chunks total run time is 3 microseconds. Numbers below.
generating input took: 00:00:00
algo 1 took 00:00:15.343456, for 1000000 runs
initial chunks count: 1000, output chunks count: 355
algo 2 took 00:00:03.374935, for 1000000 runs
initial chunks count: 1000, output chunks count: 355
Add pointers to the chunk struct for previous and next adjacent chunk in contiguous memory, if such exists, null otherwise. When a chunk is released you check if adjacent chunks are free, and if they are you merge them and update prev->next and next->prev pointers. This procedure is O(1) and you do it each time a chunk is released.
Some memory allocators put the size of current and previous chunk at the memory position immediately before the address returned by malloc. It is then possible calculate the offset to adjacent chunks without explicit pointers.
The following doesn't require sorted input or provide sorted output. Treat the input as a stack. Pop a chunk off and check if it is adjacent to a member of the initially empty output set. If not, add it to the output set. If it is adjacent, remove the adjacent chunk from the output set and push the new combined chunk onto the input stack. Repeat until input is empty.
vector<Chunk> unify_contiguous(vector<Chunk> input)
{
vector<Chunk> output;
unordered_set<Ptr, Chunk> begins;
unordered_set<Ptr, Chunk> ends;
while (!input.empty())
{
// pop chunk from input
auto chunk = input.back();
input.pop_back();
// chunk end adjacent to an output begin?
auto it = begins.find(chunk.end);
if (it != begins.end())
{
auto end = it->second.end;
Chunk combined{chunk.begin, end};
ends.erase(end);
begins.erase(it);
input.push_back(combined);
continue;
}
// chunk begin adjacent to an output end?
it = ends.find(chunk.begin);
if (it != ends.end())
{
auto begin = it->second.begin;
Chunk combined{begin, chunk.end};
begins.erase(begin);
ends.erase(it);
input.push_back(combined);
continue;
}
// if not add chunk to output
begins[chunk.begin] = chunk;
ends[chunk.end] = chunk;
}
// collect output
for (auto kv : begins)
output.push_back(kv.second);
return output;
}