I want a standard C++ container to use as an accumulator for reading from a network socket (that is, T = byte or unsigned char). I want the container to reserve a capacity on construction, and not initialize the elements. That is, I want to be able to do:
container c(1024);
and get the container to reserve 1024 octets. One-step construction/capacity is important because I want to use it in an initializer.
I also want contiguous storage. If the containers must grow, the new storage should be contiguous.
I also want to be able to append bytes to the container. And I want to be able to search for byte strings in the container.
vector and string don't really fit because construction and reserve are two step process, they use an extra allocation, and they initialize elements. Plus, vector is missing the search functionality. (EDIT: vector is fine thanks to <algorithm>; thanks DYP and Lightness Races in Orbit).
Are there any C++ standard containers that have the properties?
You are asking the wrong question.
The Standard Library does not provide a container for each and every use, instead it provides a selection of useful building bricks.
If you want contiguous storage, then you should be using std::vector and look into making it match your other requirements:
non-initialization is accomplished by providing an allocator with a no-op construct method
and searching for a pattern is accomplished by using std::search or rolling your own specific search algorithm (Knuth Morris Pratt, Boyer Moore, ...)
In any case, I advise encapsulating the vector into a dedicated class with a clear semantic role.
Related
I search something which is std::vector but without the big overhead and a bit more than std::array, because with std::array I did not have the size stored anyway ( it is only known from the type itself ).
What I want to achieve:
Written with "dynamic" containers it is like:
std::map< int, std::vector< std::pair<int,int>>>;
I need no modification during runtime, but I need the size information during runtime. Replacing std::vector with std::array could not work, because the array must be the same size for all map entries which is not what I need.
I only! want to ask if there is already an available implementation around. If the answer is simply "No", there is no need for a suggestion how to do the job. I only want to not reinvent the wheel again :-)
Background: I can use the stl on my small avr controllers, but the overhead is "a bit" to high. So I search for a hopefully standard implementation which fit the needs for compile time constant representation with implemented features like begin()/end() and iterators to fulfill the minimum container requirements to get them used with range based for and others.
c++14 is also available, if there is something I search for.
All what I found is full template implemented where the access to the data is also compile time constant like:
container.get<2>()
which I also could not use, because I need runtime vars to access my data.
EDIT: Which problem/overhead comes up while using std::vector:
While using std::vector I need also new/delete which results in having malloc/free for avr. I also found that on avr the initialization of the vector itself takes arround 350 bytes code for each template instance I use. The access functions like operator[] and also the iterators are very small.
To avoid the overhead of std::vector, you may use instead std::initializer_list
const std::map<int, std::initializer_list<std::pair<int, int>>>
Not entirely clear to me what you're looking for, but I think you could achieve what you need with the following:
You don't know the exact size of each contained element (say N elements), and you will know it when program starts. Then, whenever you know the size of the containers, reserve just one contiguous memory block to hold all the inner elements of all the containers. (one dynamic allocation)
Create the map with vectors constructed via range: std::vector(start, end), where start and end are calculated via blocks of N elements.
Then populate the map. If you don't strictly need a map, you can also calculate where each vector will begin and end, and just create an initial array with indexes of the positions of each vector...
What you are asking seems to be impossible in principle. You say you want to avoid heap allocation of your vector like type, but the size of a type on the stack must be known at compile time, and the same for all members of that type. Since map is a homogeneous container, the value type must be a single type with a constant size. You could replace the map with a tuple, but then all of your keys would have to be known at compile time.
I'm looking for a container to store a dynamically growing and shrinking family of objects the size of which I know to come near to but never exceed a given bound. The container need not be ordered, so that I'm happy with any kind of insertion, no matter where it takes place. Moreover, I want all the objects to be stored in a some fixed contiguous memory-pool, but I do not require the memory that is actually occupied at some point in time to be a connected interval in the memory-pool.
Is there any container/allocator in the STL or boost that provides the above?
It seems that a reasonable approach would be to use a linked list with memory taken from a fixed-size memory-pool, but I'd rather use some already existing and well-established implementation for this than trying to do it myself.
Thank you!
As you need elements to be contiguous, I think you should go for std::vector, calling reserve at the beginning.
As I said in comment, as soon as you need contiguous memory you'll have to move something when you delete in the middle and that behavior is already handled by std::vectorusing remove/erase idiom.
Apart from this, if you use only the vector insertion or the lookup will be costly according to your design:
Either you always add new element at the end and the lookup of an element will cost you but the insertion will be painless
Or you sort the vector after every insertion (that will cost) and your lookup will be a lot faster using std::equal_range
Otherwise if you can afford an additional std::unordered_set<std::vector<your_element>::iterator> with a custom hash/equal you have a fair insertion/lookup ratio by looking up with the std::unordered_set<> to find where your element is stored.
Recapping, your requirements are:
dynamically growing and shrinking
no need to be ordered
fixed contiguous memory-pool
With the third requirement you rule out most of the node based containers (such as lists or queues). What you are left with are array-like containers. Specifically std::array and std::vector (or even std::valarray/boost::valarray).
But with the first requirement you rule out std::array (unless you want to implement some weird looking std::array<std::optional<T>> that mimics the functionality of an std::vector).
What you are left with is std::vector, which happens to fit requirement number two as well. Of course you can manage the capacity with std::vector::reserve and std::vector::shrink_to_fit.
Is there an STL container similar to a list in that elements of lists are not stored contiguously? The size of this container can be up to 1000x1000 elements with each element being a vector containing 36 doubles. This would be a large chunk to store together (like ~200 megabytes). Is there a variant that instead stores pointers to its contents as a separate vector so it would allow for random access. Is there an STL container class for this that already exists or should I just store the pointers manually?
The container I need is actually a constant size so I think implementing it myself wouldnt be too difficult, but I was wondering if an STL container already exists for this. I'd like to avoid a vector because the list is large and the contents will be of medium size. If the vectors in the container don't need to reside next to each other then wouldn't it be better to separate them in a list to prevent running out of memory from fragmentation?
Both deque<array<double, 36>> and vector<vector<double>> would avoid the need for any really huge contiguous allocations.
The vector<vector<double>> is worse in those terms. For the numbers you specify it needs a contiguous allocation of 1000*1000*sizeof(vector<double>), which is low 10s of MB (most likely a vector is the size of 3 pointers). That's rarely a problem on a "proper computer" (desktop or server). The places where it would be a concern for fragmentation reasons (small virtual address space or no virtual addressing at all), you might also have a more fundamental problem that you don't have 300MB-ish of RAM anyway. But you could play extra-safe by avoiding it, since clearly there can exist environments where you could allocate 300MB total but not 12MB contiguously.
There is no std::array in C++03, but there's boost::array or you could easily write a class to represent 36 doubles.
vector<array<double, 36>> suffers worst from fragmentation, it requires a contiguous 250-MB allocation. Personally I don't find it easy to simulate in testing "the worst possible memory fragmentation we will ever face", but I'm not the best tester. That size of block is about where I start feeling a bit uneasy in a 32 bit process, but it will work fine in good conditions.
I highly recommend you to use the std::array class. It is constant sized, it supports random access to all elements, and has implementations of iterator, const_iterator, reverse_iterator, const_reverse_iterator. More about it: http://www.cplusplus.com/reference/stl/array/
It isn't clear what characteristic of std::list<T> you are after exactly. If you want a container whose elements stay put when adding or removing elements, you might want to have a look at std::deque<T>: when adding/removing elements at the front or the back all other element stay at the same location. That is, pointers and references to elements stay valid, unless elements are add or removed in the middle. Iterators get invalid on any insertion or removal. std::deque<T> provides random access.
There is no container directly given random access and support addition/removal at any poistion with the elements staying put. However, as others have pointed out, using a container of pointers provides such an interface. It may be necessary to wrap it to hide the use of pointers.
I'm searching for a container with the following functionality:
Fixed size at runtime. Thus, memory shouldn't be allocated in little chunks (like std::list does).
Elements should be swappable (Something like std::list::splice offers).
EDIT:
Thinking of a list: I just need to move elements from an arbitrary position to the front.
EDIT2:
I would like to use something like a std::list, but takes advantage of a runtime fixed size.
I'd think of TR1 array:
std::array<T, int>
Or, if you haven't got that yet,
boost::array<T, int>
Which is identical for all intents and purposes. Of course the validity of std::swap on elements depends on availability of proper copy constructor/assignment operator.
It's not too clear to me what you are looking for. Two solutions come to mind: std::vector (created with the maximum size), coupled with a good implementation of swap for your objects, or a custom allocator for std::list, which pre-allocates the number of nodes you'll need in a single block.
You can specify the (initial) size of an std::list at construction time :
std::list<Type> myList = std::list<Type>(10);
You can still grow/shrink it afterwards, but the list will contain 10 default constructed Type objects from the start.
Does that suit your needs ?
std::vector ?
Array based. Can specify the size and has a swap function.
You haven't said what you'll be using it for so I don't really know if you're going to have any speed issues with it.
For small collections std::vector is almost certainly the best container whatever the operations applied to it are. Is it possible to have std::vector as underlying storage for the elements set container instead red-black tree involving a lot of heap allocations (maybe boost has something?) or do I have to invent it myself?
Plain std::vector and std::sort is not an option due to performance reasons and std::inplace_merge is prone to coding errors (invalidation of iterators, etc..).
EDIT: clarified the question
There is no way to specify the underlying structure of an STL set. At best you can write an allocator that uses a vector to provide the memory used by set which may or may not be what you want.
for small size all containers are pretty efficient; just use set unless you know that you have a performance problem
in your case
using vector trades functionality (sorting, uniqueness) for storage size
using set does the opposite
If you need sorting and uniqueness then choose the container with that feature unless you are sure its a bad trade
If you mean can you have
std::set<std::vector<MyType> > myIdealContainer;
then the answer is yes, provided you are able to meaningfully wrap the vector in something that makes it sortable (so set can order its members). Watch out for copying inefficiency though.
If you mean can I instantiate set with vector as the storage for a custom allocator, then I don't know how you would do that (or why you would want to).
If you mean can you treat a vector the same way you would a set, then the answer is no. if your dataset is small and matching the container member is cheap, use vector, preserve ordering on inserts and scan linearly for matches using std::find. If dataset is large and/or matching is expensive, use set.
I think you are looking for boost::container::flat_set
flat_set is similar to std::set but it's implemented like an ordered vector.
No, it is not possible to specify the container to use for std::set, you can do that only with container adapters like std::queue or std::stack. std::set is one of the basic container with its own performance requirement. std::vector may not be the best container for all cases. For example, of you want a good lookup performance you would chose set, as find is O(log n) where as for vector it is O(n)
Maybe i've misunderstood you but if you are trying to use a std::set which has a std::vector for data storage (so all data of the set is actually stored int the vector), then the answer should be "no".
The reason for this is simply that the c++ std::set implementation is a binary search tree and a std::vector manages just a simple array/memory block.