Related
I was looking at STL containers and trying to figure what they really are (i.e. the data structure used), and the deque stopped me: I thought at first that it was a double linked list, which would allow insertion and deletion from both ends in constant time, but I am troubled by the promise made by the operator [] to be done in constant time. In a linked list, arbitrary access should be O(n), right?
And if it's a dynamic array, how can it add elements in constant time? It should be mentioned that reallocation may happen, and that O(1) is an amortized cost, like for a vector.
So I wonder what is this structure that allows arbitrary access in constant time, and at the same time never needs to be moved to a new bigger place.
A deque is somewhat recursively defined: internally it maintains a double-ended queue of chunks of fixed size. Each chunk is a vector, and the queue (“map” in the graphic below) of chunks itself is also a vector.
There’s a great analysis of the performance characteristics and how it compares to the vector over at CodeProject.
The GCC standard library implementation internally uses a T** to represent the map. Each data block is a T* which is allocated with some fixed size __deque_buf_size (which depends on sizeof(T)).
From overview, you can think deque as a double-ended queue
The datas in deque are stored by chuncks of fixed size vector, which are
pointered by a map(which is also a chunk of vector, but its size may change)
The main part code of the deque iterator is as below:
/*
buff_size is the length of the chunk
*/
template <class T, size_t buff_size>
struct __deque_iterator{
typedef __deque_iterator<T, buff_size> iterator;
typedef T** map_pointer;
// pointer to the chunk
T* cur;
T* first; // the begin of the chunk
T* last; // the end of the chunk
//because the pointer may skip to other chunk
//so this pointer to the map
map_pointer node; // pointer to the map
}
The main part code of the deque is as below:
/*
buff_size is the length of the chunk
*/
template<typename T, size_t buff_size = 0>
class deque{
public:
typedef T value_type;
typedef T& reference;
typedef T* pointer;
typedef __deque_iterator<T, buff_size> iterator;
typedef size_t size_type;
typedef ptrdiff_t difference_type;
protected:
typedef pointer* map_pointer;
// allocate memory for the chunk
typedef allocator<value_type> dataAllocator;
// allocate memory for map
typedef allocator<pointer> mapAllocator;
private:
//data members
iterator start;
iterator finish;
map_pointer map;
size_type map_size;
}
Below i will give you the core code of deque, mainly about three parts:
iterator
How to construct a deque
1. iterator(__deque_iterator)
The main problem of iterator is, when ++, -- iterator, it may skip to other chunk(if it pointer to edge of chunk). For example, there are three data chunks: chunk 1,chunk 2,chunk 3.
The pointer1 pointers to the begin of chunk 2, when operator --pointer it will pointer to the end of chunk 1, so as to the pointer2.
Below I will give the main function of __deque_iterator:
Firstly, skip to any chunk:
void set_node(map_pointer new_node){
node = new_node;
first = *new_node;
last = first + chunk_size();
}
Note that, the chunk_size() function which compute the chunk size, you can think of it returns 8 for simplify here.
operator* get the data in the chunk
reference operator*()const{
return *cur;
}
operator++, --
// prefix forms of increment
self& operator++(){
++cur;
if (cur == last){ //if it reach the end of the chunk
set_node(node + 1);//skip to the next chunk
cur = first;
}
return *this;
}
// postfix forms of increment
self operator++(int){
self tmp = *this;
++*this;//invoke prefix ++
return tmp;
}
self& operator--(){
if(cur == first){ // if it pointer to the begin of the chunk
set_node(node - 1);//skip to the prev chunk
cur = last;
}
--cur;
return *this;
}
self operator--(int){
self tmp = *this;
--*this;
return tmp;
}
iterator skip n steps / random access
self& operator+=(difference_type n){ // n can be postive or negative
difference_type offset = n + (cur - first);
if(offset >=0 && offset < difference_type(buffer_size())){
// in the same chunk
cur += n;
}else{//not in the same chunk
difference_type node_offset;
if (offset > 0){
node_offset = offset / difference_type(chunk_size());
}else{
node_offset = -((-offset - 1) / difference_type(chunk_size())) - 1 ;
}
// skip to the new chunk
set_node(node + node_offset);
// set new cur
cur = first + (offset - node_offset * chunk_size());
}
return *this;
}
// skip n steps
self operator+(difference_type n)const{
self tmp = *this;
return tmp+= n; //reuse operator +=
}
self& operator-=(difference_type n){
return *this += -n; //reuse operator +=
}
self operator-(difference_type n)const{
self tmp = *this;
return tmp -= n; //reuse operator +=
}
// random access (iterator can skip n steps)
// invoke operator + ,operator *
reference operator[](difference_type n)const{
return *(*this + n);
}
2. How to construct a deque
common function of deque
iterator begin(){return start;}
iterator end(){return finish;}
reference front(){
//invoke __deque_iterator operator*
// return start's member *cur
return *start;
}
reference back(){
// cna't use *finish
iterator tmp = finish;
--tmp;
return *tmp; //return finish's *cur
}
reference operator[](size_type n){
//random access, use __deque_iterator operator[]
return start[n];
}
template<typename T, size_t buff_size>
deque<T, buff_size>::deque(size_t n, const value_type& value){
fill_initialize(n, value);
}
template<typename T, size_t buff_size>
void deque<T, buff_size>::fill_initialize(size_t n, const value_type& value){
// allocate memory for map and chunk
// initialize pointer
create_map_and_nodes(n);
// initialize value for the chunks
for (map_pointer cur = start.node; cur < finish.node; ++cur) {
initialized_fill_n(*cur, chunk_size(), value);
}
// the end chunk may have space node, which don't need have initialize value
initialized_fill_n(finish.first, finish.cur - finish.first, value);
}
template<typename T, size_t buff_size>
void deque<T, buff_size>::create_map_and_nodes(size_t num_elements){
// the needed map node = (elements nums / chunk length) + 1
size_type num_nodes = num_elements / chunk_size() + 1;
// map node num。min num is 8 ，max num is "needed size + 2"
map_size = std::max(8, num_nodes + 2);
// allocate map array
map = mapAllocator::allocate(map_size);
// tmp_start，tmp_finish poniters to the center range of map
map_pointer tmp_start = map + (map_size - num_nodes) / 2;
map_pointer tmp_finish = tmp_start + num_nodes - 1;
// allocate memory for the chunk pointered by map node
for (map_pointer cur = tmp_start; cur <= tmp_finish; ++cur) {
*cur = dataAllocator::allocate(chunk_size());
}
// set start and end iterator
start.set_node(tmp_start);
start.cur = start.first;
finish.set_node(tmp_finish);
finish.cur = finish.first + num_elements % chunk_size();
}
Let's assume i_deque has 20 int elements 0~19 whose chunk size is 8, and now push_back 3 elements (0, 1, 2) to i_deque:
i_deque.push_back(0);
i_deque.push_back(1);
i_deque.push_back(2);
It's internal structure like below:
Then push_back again, it will invoke allocate new chunk:
push_back(3)
If we push_front, it will allocate new chunk before the prev start
Note when push_back element into deque, if all the maps and chunks are filled, it will cause allocate new map, and adjust chunks.But the above code may be enough for you to understand deque.
Imagine it as a vector of vectors. Only they aren't standard std::vectors.
The outer vector contains pointers to the inner vectors. When its capacity is changed via reallocation, rather than allocating all of the empty space to the end as std::vector does, it splits the empty space to equal parts at the beginning and the end of the vector. This allows push_front and push_back on this vector to both occur in amortized O(1) time.
The inner vector behavior needs to change depending on whether it's at the front or the back of the deque. At the back it can behave as a standard std::vector where it grows at the end, and push_back occurs in O(1) time. At the front it needs to do the opposite, growing at the beginning with each push_front. In practice this is easily achieved by adding a pointer to the front element and the direction of growth along with the size. With this simple modification push_front can also be O(1) time.
Access to any element requires offsetting and dividing to the proper outer vector index which occurs in O(1), and indexing into the inner vector which is also O(1). This assumes that the inner vectors are all fixed size, except for the ones at the beginning or the end of the deque.
(This is an answer I've given in another thread. Essentially I'm arguing that even fairly naive implementations, using a single vector, conform to the requirements of "constant non-amortized push_{front,back}". You might be surprised, and think this is impossible, but I have found other relevant quotes in the standard that define the context in a surprising way. Please bear with me; if I have made a mistake in this answer, it would be very helpful to identify which things I have said correctly and where my logic has broken down. )
In this answer, I am not trying to identify a good implementation, I'm merely trying to help us to interpret the complexity requirements in the C++ standard. I'm quoting from N3242, which is, according to Wikipedia, the latest freely available C++11 standardization document. (It appears to be organized differently from the final standard, and hence I won't quote the exact page numbers. Of course, these rules might have changed in the final standard, but I don't think that has happened.)
A deque<T> could be implemented correctly by using a vector<T*>. All the elements are copied onto the heap and the pointers stored in a vector. (More on the vector later).
Why T* instead of T? Because the standard requires that
"An insertion at either end of the deque invalidates all the iterators
to the deque, but has no effect on the validity of references to
elements of the deque."
(my emphasis). The T* helps to satisfy that. It also helps us to satisfy this:
"Inserting a single element either at the beginning or end of a deque always ..... causes a single call to a constructor of T."
Now for the (controversial) bit. Why use a vector to store the T*? It gives us random access, which is a good start. Let's forget about the complexity of vector for a moment and build up to this carefully:
The standard talks about "the number of operations on the contained objects.". For deque::push_front this is clearly 1 because exactly one T object is constructed and zero of the existing T objects are read or scanned in any way. This number, 1, is clearly a constant and is independent of the number of objects currently in the deque. This allows us to say that:
'For our deque::push_front, the number of operations on the contained objects (the Ts) is fixed and is independent of the number of objects already in the deque.'
Of course, the number of operations on the T* will not be so well-behaved. When the vector<T*> grows too big, it'll be realloced and many T*s will be copied around. So yes, the number of operations on the T* will vary wildly, but the number of operations on T will not be affected.
Why do we care about this distinction between counting operations on T and counting operations on T*? It's because the standard says:
All of the complexity requirements in this clause are stated solely in terms of the number of operations on the contained objects.
For the deque, the contained objects are the T, not the T*, meaning we can ignore any operation which copies (or reallocs) a T*.
I haven't said much about how a vector would behave in a deque. Perhaps we would interpret it as a circular buffer (with the vector always taking up its maximum capacity(), and then realloc everything into a bigger buffer when the vector is full. The details don't matter.
In the last few paragraphs, we have analyzed deque::push_front and the relationship between the number of objects in the deque already and the number of operations performed by push_front on contained T-objects. And we found they were independent of each other. As the standard mandates that complexity is in terms of operations-on-T, then we can say this has constant complexity.
Yes, the Operations-On-T*-Complexity is amortized (due to the vector), but we're only interested in the Operations-On-T-Complexity and this is constant (non-amortized).
The complexity of vector::push_back or vector::push_front is irrelevant in this implementation; those considerations involve operations on T* and hence are irrelevant. If the standard was referring to the 'conventional' theoretical notion of complexity, then they wouldn't have explicitly restricted themselves to the "number of operations on the contained objects". Am I overinterpreting that sentence?
deque = double ended queue
A container which can grow in either direction.
Deque is typically implemented as a vector of vectors (a list of vectors can't give constant time random access). While the size of the secondary vectors is implementation dependent, a common algorithm is to use a constant size in bytes.
I was reading "Data structures and algorithms in C++" by Adam Drozdek, and found this useful.
HTH.
A very interesting aspect of STL deque is its implementation. An STL deque is not implemented as a linked list but as an array of pointers to blocks or arrays of data. The number of blocks changes dynamically depending on storage needs, and the size of the array of pointers changes accordingly.
You can notice in the middle is the array of pointers to the data (chunks on the right), and also you can notice that the array in the middle is dynamically changing.
An image is worth a thousand words.
While the standard doesn't mandate any particular implementation (only constant-time random access), a deque is usually implemented as a collection of contiguous memory "pages". New pages are allocated as needed, but you still have random access. Unlike std::vector, you're not promised that data is stored contiguously, but like vector, insertions in the middle require lots of relocating.
deque could be implemented as a circular buffer of fixed size array:
Use circular buffer so we could grow/shrink at both end by adding/removing a fixed sized array with O(1) complexity
Use fixed sized array so it is easy to calculate the index, hence access via index with two pointer dereferences - also O(1)
I've written some code to decrease the capacity of a templated container class. After an element is removed from the container, the erase function checks to see whether or not 25% of the total space is in use, and whether reducing the capacity by half would cause it to be less than the default size I've set. If these two return true, then the downsize function runs. However, if this happens while I'm in the middle of a const_iterator loop, I get a segfault.
edit again: I'm thinking it's because the const_iterator pointer is pointing to the old array and needs to be pointed to the new one created by downsize()...now how to go about doing that...
template <class T>
void sorted<T>::downsize(){
// Run the same process as resize, except
// in reverse (sort of).
int newCapacity = (m_capacity / 2);
T *temp_array = new T[newCapacity];
for (int i = 0; i < m_size; i++)
temp_array[i] = m_data[i];
// Frees memory, points m_data at the
// new, smaller array, sets the capacity
// to the proper (lower) value.
delete [] m_data;
m_data = temp_array;
setCap(newCapacity);
cout << "Decreased array capacity to " << newCapacity << "." << endl;
}
// Implementation of the const_iterator erase method.
template <class T>
typename sorted<T>::const_iterator sorted<T>::erase(const_iterator itr){
// This section is reused from game.cpp, a file provided in the
// Cruno project. It handles erasing the element pointed to
// by the constant iterator.
T *end = &m_data[m_capacity]; // one past the end of data
T *ptr = itr.m_current; // element to erase
// to erase element at ptr, shift elements from ptr+1 to
// the end of the array down one position
while ( ptr+1 != end ) {
*ptr = *(ptr+1);
ptr++;
}
m_size--;
// Once the element is removed, check to
// see if a size reduction of the array is
// necessary.
// Initialized some new values here to make
// sure downsize only runs when the correct
// conditions are met.
double capCheck = m_capacity;
double sizeCheck = m_size;
double usedCheck = (sizeCheck / capCheck);
int boundCheck = (m_capacity / 2);
if ((usedCheck <= ONE_FOURTH) && (boundCheck >= DEFAULT_SIZE))
downsize();
return itr;
}
// Chunk from main that erases.
int i = 0;
for (itr = x.begin(); itr != x.end(); itr++) {
if (i < 7) x.erase(itr);
i++;
}
To prevent issues with invalidated iterators during an erase loop, you can use:
x.erase(itr++);
instead of separate increment and increment calls (obviously if you're not erasing everything you loop over you'd need an else case to increment past the non-erased items.) Note this is a case where you need to use post-increment rather than pre-increment.
The whole thing looks a bit inefficient though. The way you convert to an array suggests that it's only going to work with certain container types anyway. If you expect lots of erasing from the middle of the container you might be able to avoid the memory issue by just choosing a different container; list maybe.
If you choose vector, you might find that calling shrink_to_fit is what you want.
Also note that erase has a return value that will point to the (new) location of the element after the erased one.
If you go down the line of returning an iterator to a newly created down-sized version of the container, note that comparing to the original containers end() iterator wouldn't work.
I figured out the segfault problem!
What was going on was that my x.end() was using the size of the array (NOT the capacity), where the size is the number of data elements stored in the array, instead of the capacity which is actually the size of the array. So when it was looking for the end of the array, it was seeing it as numerous elements before the actual end.
Now on to more interesting problems!
I am writing a program in C++ that will be used with Windows Embedded Compact 7. I have heard that it is best not to dynamically allocate arrays when writing embedded code. I will be keeping track of between 0 and 50 objects, so I am initially allocating 50 objects.
Object objectList[50];
int activeObjectIndex[50];
static const int INVALID_INDEX = -1;
int activeObjectCount=0;
activeObjectCount tells me how many objects I am actually using, and activeObjectIndex tells me which objects I am using. If the 0th, 7th, and 10th objects were being used I would want activeObjectIndex = [0,7,10,-1,-1,-1,...,-1]; and activeObjectCount=3;
As different objects become active or inactive I would like activeObjectIndex list to remain ordered.
Currently I am just sorting the activeObjectIndex at the end of each loop that the values might change in.
First, is there a better way to keep track of objects (that may or may not be active) in an embedded system than what I am doing? If not, is there an algorithm I can use to keep the objects sorted each time I add or remove and active object? Or should I just periodically do a bubble sort or something to keep them in order?
You have a hard question, where the answer requires quite a bit of knowledge about your system. Without that knowledge, no answer I can give would be complete. However, 15 years of embedded design has taught me the following:
You are correct, you generally don't want to allocate objects during runtime. Preallocate all the objects, and move them to active/inactive queues.
Keeping things sorted is generally hard. Perhaps you don't need to. You don't mention it, but I'll bet you really just need to keep your Objects in "used" and "free" pools, and you're using the index to quickly find/delete Objects.
I propose the following solution. Change your object to the following:
class Object {
Object *mNext, *mPrevious;
public:
Object() : mNext(this), mPrevious(this) { /* etc. */ }
void insertAfterInList(Object *p2) {
mNext->mPrev = p2;
p2->mNext = mNext;
mNext = p2;
p2->mPrev = this;
}
void removeFromList() {
mPrev->mNext = mNext;
mNext->mPrev = mPrev;
mNext = mPrev = this;
}
Object* getNext() {
return mNext;
}
bool hasObjects() {
return mNext != this;
}
};
And use your Objects:
#define NUM_OBJECTS (50)
Object gObjects[NUM_OBJECTS], gFree, gUsed;
void InitObjects() {
for(int i = 0; i < NUM_OBJECTS; ++i) {
gFree.insertAfter(&mObjects[i]);
}
}
Object* GetNewObject() {
assert(mFree.hasObjects());
Object obj = mFree->getNext();
obj->removeFromList();
gUsed.insertAfter(obj);
return obj;
}
void ReleaseObject(Object *obj) {
obj->removeFromList();
mFree.insertAfter(obj);
}
Edited to fix a small glitch. Should work now, although not tested. :)
The overhead of a std::vector is very small. The problem you can have is that dynamic resizing will allocate more memory than needed. However, as you have 50 elements, this shouldn't be a problem at all. Give it a try, and change it only if you see a strong impact.
If you cannot/do not want to remove unused objects from a std::vector, you can maybe add a boolean to your Object that indicates if it is active? This won't require more memory than using activeObjectIndex (maybe even less depending on alignment issues).
To sort the data with a boolean (not active at the end), write a function :
bool compare(const Object & a, const Object & b) {
if(a.active && !b.active) return true;
else return false;
}
std::sort(objectList,objectList + 50, &compare); // if you use an array
std::sort(objectList.begin(),objectList.end(), &compare); // if you use std::vector
If you want to sort using activeObjectIndex it will be more complicated.
If you want to use a structure that is always ordered, use std::set. However it will require more memory (but for 50 elements, it won't be an issue).
Ideally, implement the following function :
bool operator<(const Object & a, const Object & b) {
if(a.active && !b.active) return true;
else return false;
}
This will allow to use directly std::sort(objectList.begin(), objectList.end()) or declare an std::set that will stay sorted.
One way to keep track of active / inactive is to have the active Objects be on a doubly linked list. When an object goes from inactive to active then add to the list, and active to inactive remove from the list. You can add these to Object
Object * next, * prev;
so this does not require memory allocation.
If no dynamic memory allocation is allowed, I would use simple c-array or std::array and an index, which points into last+1 object. Objects are always kept in sorted order.
Addition is done by inserting new object into correct position of sorted list. To find insert position lower_bound or find_if can be used. For 50 element, second probably will be faster. Removal is similar.
You should not worry about having the list sorted, as writing a method to search in a list of indices what are the ones active would be O(N), and, in your particular case, amortized to O(1), as your array seems to be small enough for this little extra verification.
You could maintain the index of the last element checked, until it reaches the limit:
unsigned int next(const unsigned int& last) {
for (unsigned int i = last + 1; i < MAX_ARRAY_SIZE; i++) {
if (activeObjectIndex[i] != -1) {
return i;
}
}
return -1;
}
However, if you really want to have a side index, you can simply double the size of the array, creating a double linked list to the elements:
activeObjectIndex[MAX_ARRAY_SIZE * 3] = {-1};
activeObjectIndex[i] = "element id";
activeObjectIndex[i + 1] = "position of the previous element";
activeObjectIndex[i + 2] = "position of the next element";
I'm creating a custom vector class as part of a homework assignment. What I am currently trying to do is implement a function called erase, which will take an integer as an argument, decrease my array length by 1, remove the element at the position specified by the argument, and finally shift all the elements down to fill in the gap left by "erased" element.
What I am not completely understanding, due to my lack of experience with this language, is how you can delete a single element from an array of pointers.
Currently, I have the following implemented:
void myvector::erase(int i)
{
if(i != max_size)
{
for(int x = i; x < max_size; x++)
{
vec_array[x] = vec_array[x+1];
}
vec_size --;
//delete element from vector;
}
else
//delete element from vector
}
The class declaration and constructors look like this:
template <typename T>
class myvector
{
private:
T *vec_array;
int vec_size;
int max_size;
bool is_empty;
public:
myvector::myvector(int max_size_input)
{
max_size = max_size_input;
vec_array = new T[max_size];
vec_size = 0;
}
I have tried the following:
Using delete to try and delete an element
delete vec_size[max_size];
vec_size[max_size] = NULL;
Setting the value of the element to NULL or 0
vec_size[max_size] = NULL
or
vec_size[max_size] = 0
None of which are working for me due to either operator "=" being ambiguous or specified type not being able to be cast to void *.
I'm probably missing something simple, but I just can't seem to get passed this. Any help would be much appreciated. Again, sorry for the lack of experience if this is something silly.
If your custom vector class is supposed to work like std::vector, then don't concern yourself with object destruction. If you need to erase an element, you simply copy all elements following it by one position to the left:
void myvector::erase(int i)
{
for (int x = i + 1; x < vec_size; x++) {
vec_array[x - 1] = vec_array[x];
}
vec_size--;
}
That's all the basic work your erase() function has to do.
If the elements happen to be pointers, you shouldn't care; the user of your vector class is responsible for deleting those pointers if that's needed. You cannot determine if they can actually be deleted (the pointers might point to automatic stack variables, which are not deletable.)
So, do not ever call delete on an element of your vector.
If your vector class has a clear() function, and you want to make sure the elements are destructed, simply:
delete[] vec_array;
vec_array = new T[max_size];
vec_size = 0;
And this is how std::vector works, actually. (Well, the basic logic of it; of course you can optimize a hell of a lot of stuff in a vector implementation.)
Since this is homework i wont give you a definitive solution, but here is one method of erasing a value:
loop through and find value specified in erase function
mark values position in the array
starting from that position, move all elements values to the previous element(overlapping 'erased' value)
for i starting at position, i less than size minus one, i plus plus
element equals next element
reduce size of vector by 1
see if this is a big enough hint.
I was looking at STL containers and trying to figure what they really are (i.e. the data structure used), and the deque stopped me: I thought at first that it was a double linked list, which would allow insertion and deletion from both ends in constant time, but I am troubled by the promise made by the operator [] to be done in constant time. In a linked list, arbitrary access should be O(n), right?
And if it's a dynamic array, how can it add elements in constant time? It should be mentioned that reallocation may happen, and that O(1) is an amortized cost, like for a vector.
So I wonder what is this structure that allows arbitrary access in constant time, and at the same time never needs to be moved to a new bigger place.
A deque is somewhat recursively defined: internally it maintains a double-ended queue of chunks of fixed size. Each chunk is a vector, and the queue (“map” in the graphic below) of chunks itself is also a vector.
There’s a great analysis of the performance characteristics and how it compares to the vector over at CodeProject.
The GCC standard library implementation internally uses a T** to represent the map. Each data block is a T* which is allocated with some fixed size __deque_buf_size (which depends on sizeof(T)).
From overview, you can think deque as a double-ended queue
The datas in deque are stored by chuncks of fixed size vector, which are
pointered by a map(which is also a chunk of vector, but its size may change)
The main part code of the deque iterator is as below:
/*
buff_size is the length of the chunk
*/
template <class T, size_t buff_size>
struct __deque_iterator{
typedef __deque_iterator<T, buff_size> iterator;
typedef T** map_pointer;
// pointer to the chunk
T* cur;
T* first; // the begin of the chunk
T* last; // the end of the chunk
//because the pointer may skip to other chunk
//so this pointer to the map
map_pointer node; // pointer to the map
}
The main part code of the deque is as below:
/*
buff_size is the length of the chunk
*/
template<typename T, size_t buff_size = 0>
class deque{
public:
typedef T value_type;
typedef T& reference;
typedef T* pointer;
typedef __deque_iterator<T, buff_size> iterator;
typedef size_t size_type;
typedef ptrdiff_t difference_type;
protected:
typedef pointer* map_pointer;
// allocate memory for the chunk
typedef allocator<value_type> dataAllocator;
// allocate memory for map
typedef allocator<pointer> mapAllocator;
private:
//data members
iterator start;
iterator finish;
map_pointer map;
size_type map_size;
}
Below i will give you the core code of deque, mainly about three parts:
iterator
How to construct a deque
1. iterator(__deque_iterator)
The main problem of iterator is, when ++, -- iterator, it may skip to other chunk(if it pointer to edge of chunk). For example, there are three data chunks: chunk 1,chunk 2,chunk 3.
The pointer1 pointers to the begin of chunk 2, when operator --pointer it will pointer to the end of chunk 1, so as to the pointer2.
Below I will give the main function of __deque_iterator:
Firstly, skip to any chunk:
void set_node(map_pointer new_node){
node = new_node;
first = *new_node;
last = first + chunk_size();
}
Note that, the chunk_size() function which compute the chunk size, you can think of it returns 8 for simplify here.
operator* get the data in the chunk
reference operator*()const{
return *cur;
}
operator++, --
// prefix forms of increment
self& operator++(){
++cur;
if (cur == last){ //if it reach the end of the chunk
set_node(node + 1);//skip to the next chunk
cur = first;
}
return *this;
}
// postfix forms of increment
self operator++(int){
self tmp = *this;
++*this;//invoke prefix ++
return tmp;
}
self& operator--(){
if(cur == first){ // if it pointer to the begin of the chunk
set_node(node - 1);//skip to the prev chunk
cur = last;
}
--cur;
return *this;
}
self operator--(int){
self tmp = *this;
--*this;
return tmp;
}
iterator skip n steps / random access
self& operator+=(difference_type n){ // n can be postive or negative
difference_type offset = n + (cur - first);
if(offset >=0 && offset < difference_type(buffer_size())){
// in the same chunk
cur += n;
}else{//not in the same chunk
difference_type node_offset;
if (offset > 0){
node_offset = offset / difference_type(chunk_size());
}else{
node_offset = -((-offset - 1) / difference_type(chunk_size())) - 1 ;
}
// skip to the new chunk
set_node(node + node_offset);
// set new cur
cur = first + (offset - node_offset * chunk_size());
}
return *this;
}
// skip n steps
self operator+(difference_type n)const{
self tmp = *this;
return tmp+= n; //reuse operator +=
}
self& operator-=(difference_type n){
return *this += -n; //reuse operator +=
}
self operator-(difference_type n)const{
self tmp = *this;
return tmp -= n; //reuse operator +=
}
// random access (iterator can skip n steps)
// invoke operator + ,operator *
reference operator[](difference_type n)const{
return *(*this + n);
}
2. How to construct a deque
common function of deque
iterator begin(){return start;}
iterator end(){return finish;}
reference front(){
//invoke __deque_iterator operator*
// return start's member *cur
return *start;
}
reference back(){
// cna't use *finish
iterator tmp = finish;
--tmp;
return *tmp; //return finish's *cur
}
reference operator[](size_type n){
//random access, use __deque_iterator operator[]
return start[n];
}
template<typename T, size_t buff_size>
deque<T, buff_size>::deque(size_t n, const value_type& value){
fill_initialize(n, value);
}
template<typename T, size_t buff_size>
void deque<T, buff_size>::fill_initialize(size_t n, const value_type& value){
// allocate memory for map and chunk
// initialize pointer
create_map_and_nodes(n);
// initialize value for the chunks
for (map_pointer cur = start.node; cur < finish.node; ++cur) {
initialized_fill_n(*cur, chunk_size(), value);
}
// the end chunk may have space node, which don't need have initialize value
initialized_fill_n(finish.first, finish.cur - finish.first, value);
}
template<typename T, size_t buff_size>
void deque<T, buff_size>::create_map_and_nodes(size_t num_elements){
// the needed map node = (elements nums / chunk length) + 1
size_type num_nodes = num_elements / chunk_size() + 1;
// map node num。min num is 8 ，max num is "needed size + 2"
map_size = std::max(8, num_nodes + 2);
// allocate map array
map = mapAllocator::allocate(map_size);
// tmp_start，tmp_finish poniters to the center range of map
map_pointer tmp_start = map + (map_size - num_nodes) / 2;
map_pointer tmp_finish = tmp_start + num_nodes - 1;
// allocate memory for the chunk pointered by map node
for (map_pointer cur = tmp_start; cur <= tmp_finish; ++cur) {
*cur = dataAllocator::allocate(chunk_size());
}
// set start and end iterator
start.set_node(tmp_start);
start.cur = start.first;
finish.set_node(tmp_finish);
finish.cur = finish.first + num_elements % chunk_size();
}
Let's assume i_deque has 20 int elements 0~19 whose chunk size is 8, and now push_back 3 elements (0, 1, 2) to i_deque:
i_deque.push_back(0);
i_deque.push_back(1);
i_deque.push_back(2);
It's internal structure like below:
Then push_back again, it will invoke allocate new chunk:
push_back(3)
If we push_front, it will allocate new chunk before the prev start
Note when push_back element into deque, if all the maps and chunks are filled, it will cause allocate new map, and adjust chunks.But the above code may be enough for you to understand deque.
Imagine it as a vector of vectors. Only they aren't standard std::vectors.
The outer vector contains pointers to the inner vectors. When its capacity is changed via reallocation, rather than allocating all of the empty space to the end as std::vector does, it splits the empty space to equal parts at the beginning and the end of the vector. This allows push_front and push_back on this vector to both occur in amortized O(1) time.
The inner vector behavior needs to change depending on whether it's at the front or the back of the deque. At the back it can behave as a standard std::vector where it grows at the end, and push_back occurs in O(1) time. At the front it needs to do the opposite, growing at the beginning with each push_front. In practice this is easily achieved by adding a pointer to the front element and the direction of growth along with the size. With this simple modification push_front can also be O(1) time.
Access to any element requires offsetting and dividing to the proper outer vector index which occurs in O(1), and indexing into the inner vector which is also O(1). This assumes that the inner vectors are all fixed size, except for the ones at the beginning or the end of the deque.
(This is an answer I've given in another thread. Essentially I'm arguing that even fairly naive implementations, using a single vector, conform to the requirements of "constant non-amortized push_{front,back}". You might be surprised, and think this is impossible, but I have found other relevant quotes in the standard that define the context in a surprising way. Please bear with me; if I have made a mistake in this answer, it would be very helpful to identify which things I have said correctly and where my logic has broken down. )
In this answer, I am not trying to identify a good implementation, I'm merely trying to help us to interpret the complexity requirements in the C++ standard. I'm quoting from N3242, which is, according to Wikipedia, the latest freely available C++11 standardization document. (It appears to be organized differently from the final standard, and hence I won't quote the exact page numbers. Of course, these rules might have changed in the final standard, but I don't think that has happened.)
A deque<T> could be implemented correctly by using a vector<T*>. All the elements are copied onto the heap and the pointers stored in a vector. (More on the vector later).
Why T* instead of T? Because the standard requires that
"An insertion at either end of the deque invalidates all the iterators
to the deque, but has no effect on the validity of references to
elements of the deque."
(my emphasis). The T* helps to satisfy that. It also helps us to satisfy this:
"Inserting a single element either at the beginning or end of a deque always ..... causes a single call to a constructor of T."
Now for the (controversial) bit. Why use a vector to store the T*? It gives us random access, which is a good start. Let's forget about the complexity of vector for a moment and build up to this carefully:
The standard talks about "the number of operations on the contained objects.". For deque::push_front this is clearly 1 because exactly one T object is constructed and zero of the existing T objects are read or scanned in any way. This number, 1, is clearly a constant and is independent of the number of objects currently in the deque. This allows us to say that:
'For our deque::push_front, the number of operations on the contained objects (the Ts) is fixed and is independent of the number of objects already in the deque.'
Of course, the number of operations on the T* will not be so well-behaved. When the vector<T*> grows too big, it'll be realloced and many T*s will be copied around. So yes, the number of operations on the T* will vary wildly, but the number of operations on T will not be affected.
Why do we care about this distinction between counting operations on T and counting operations on T*? It's because the standard says:
All of the complexity requirements in this clause are stated solely in terms of the number of operations on the contained objects.
For the deque, the contained objects are the T, not the T*, meaning we can ignore any operation which copies (or reallocs) a T*.
I haven't said much about how a vector would behave in a deque. Perhaps we would interpret it as a circular buffer (with the vector always taking up its maximum capacity(), and then realloc everything into a bigger buffer when the vector is full. The details don't matter.
In the last few paragraphs, we have analyzed deque::push_front and the relationship between the number of objects in the deque already and the number of operations performed by push_front on contained T-objects. And we found they were independent of each other. As the standard mandates that complexity is in terms of operations-on-T, then we can say this has constant complexity.
Yes, the Operations-On-T*-Complexity is amortized (due to the vector), but we're only interested in the Operations-On-T-Complexity and this is constant (non-amortized).
The complexity of vector::push_back or vector::push_front is irrelevant in this implementation; those considerations involve operations on T* and hence are irrelevant. If the standard was referring to the 'conventional' theoretical notion of complexity, then they wouldn't have explicitly restricted themselves to the "number of operations on the contained objects". Am I overinterpreting that sentence?
deque = double ended queue
A container which can grow in either direction.
Deque is typically implemented as a vector of vectors (a list of vectors can't give constant time random access). While the size of the secondary vectors is implementation dependent, a common algorithm is to use a constant size in bytes.
I was reading "Data structures and algorithms in C++" by Adam Drozdek, and found this useful.
HTH.
A very interesting aspect of STL deque is its implementation. An STL deque is not implemented as a linked list but as an array of pointers to blocks or arrays of data. The number of blocks changes dynamically depending on storage needs, and the size of the array of pointers changes accordingly.
You can notice in the middle is the array of pointers to the data (chunks on the right), and also you can notice that the array in the middle is dynamically changing.
An image is worth a thousand words.
While the standard doesn't mandate any particular implementation (only constant-time random access), a deque is usually implemented as a collection of contiguous memory "pages". New pages are allocated as needed, but you still have random access. Unlike std::vector, you're not promised that data is stored contiguously, but like vector, insertions in the middle require lots of relocating.
deque could be implemented as a circular buffer of fixed size array:
Use circular buffer so we could grow/shrink at both end by adding/removing a fixed sized array with O(1) complexity
Use fixed sized array so it is easy to calculate the index, hence access via index with two pointer dereferences - also O(1)