I search something which is std::vector but without the big overhead and a bit more than std::array, because with std::array I did not have the size stored anyway ( it is only known from the type itself ).
What I want to achieve:
Written with "dynamic" containers it is like:
std::map< int, std::vector< std::pair<int,int>>>;
I need no modification during runtime, but I need the size information during runtime. Replacing std::vector with std::array could not work, because the array must be the same size for all map entries which is not what I need.
I only! want to ask if there is already an available implementation around. If the answer is simply "No", there is no need for a suggestion how to do the job. I only want to not reinvent the wheel again :-)
Background: I can use the stl on my small avr controllers, but the overhead is "a bit" to high. So I search for a hopefully standard implementation which fit the needs for compile time constant representation with implemented features like begin()/end() and iterators to fulfill the minimum container requirements to get them used with range based for and others.
c++14 is also available, if there is something I search for.
All what I found is full template implemented where the access to the data is also compile time constant like:
container.get<2>()
which I also could not use, because I need runtime vars to access my data.
EDIT: Which problem/overhead comes up while using std::vector:
While using std::vector I need also new/delete which results in having malloc/free for avr. I also found that on avr the initialization of the vector itself takes arround 350 bytes code for each template instance I use. The access functions like operator[] and also the iterators are very small.
To avoid the overhead of std::vector, you may use instead std::initializer_list
const std::map<int, std::initializer_list<std::pair<int, int>>>
Not entirely clear to me what you're looking for, but I think you could achieve what you need with the following:
You don't know the exact size of each contained element (say N elements), and you will know it when program starts. Then, whenever you know the size of the containers, reserve just one contiguous memory block to hold all the inner elements of all the containers. (one dynamic allocation)
Create the map with vectors constructed via range: std::vector(start, end), where start and end are calculated via blocks of N elements.
Then populate the map. If you don't strictly need a map, you can also calculate where each vector will begin and end, and just create an initial array with indexes of the positions of each vector...
What you are asking seems to be impossible in principle. You say you want to avoid heap allocation of your vector like type, but the size of a type on the stack must be known at compile time, and the same for all members of that type. Since map is a homogeneous container, the value type must be a single type with a constant size. You could replace the map with a tuple, but then all of your keys would have to be known at compile time.
Related
I need a data structure that satisfies the following:
stores an arbitrary number of elements, where each element is described by 10 numeric metrics
allows fast (log n) search of elements by any of the metrics
allows fast (log n) insertion of new elements
allows fast (log n) removal of elements
And let's assume that the elements are expensive to construct.
I came up with the following plan
store all elements in a vector called DATA.
use 10 std::sets, one for each of 10 metrics. Each std:set is light-weight, it contains only integers, which are indexes into the vector DATA. The comparison operators 'look up' the appropriate element in DATA and then select the appropriate metric
template< int C >
struct Cmp
{
bool operator() (int const a, int const b)
{
return ( DATA[a].coords[C] != DATA[b].coords[C] )
? ( DATA[a].coords[C] < DATA[b].coords[C] )
: ( a < b );
}
};
Elements are never modified or removed from a vector. A new element is pushed back to DATA and then its index (DATA.size()-1) is inserted into the sets (set<int, Cmp<..> >). To remove an element, I set a flag in the element saying that it is deleted (without actually removing it from the DATA vector) and then remove the element index from all ten std::sets.
This works fine as long as DATA is a global variable. (It also somewhat abuses the type system by making the templated struct Cmp dependent on a global variable.)
However, I was not able to enclose the DATA vector and std::set's (set<int, Cmp<...> >) inside a class and then 'index' DATA with those std::sets. For starters, the comparison operator Cmp defined inside an outer class has no access to the outer class' fields (so it cannot assess DATA). I also cannot pass the vector to the Cmp constructor because Cmp is being constructed by std::set and std::set expects a comparison operator with a constructor that has no arguments.
I have a feeling I'm working against C++ type system and trying to achieve something that the type system is purposely preventing me from doing. (I'm trying to make std::set depend on a variable that is going to be constructed only at runtime.) And while I understand why the type system might not like what I do, I think this is a legitimate use case.
Is there a way to implement the data structure/class I described above without providing a re-implementation of std::set/red-black tree? I hope there may be a trick I have not thought of yet. (And yes, I know that boost has something, but I'd like to stick to the standard library.)
When I read something like "look up foo by a value bar", my initial reaction is to use a map<> or something similar. There are some implications to this though:
Keys in an std::map (or values in an std::set) are unique, so no two elements can share the same key and accordingly no two data objects would be able to have the same metric. If multiple data objects can have the same metric (this isn't clear from your question), using an std::multimap (or std::multiset) would work though.
If the keys are constant and stored within the elements themselves, using a set<data*,cmp> is a common approach. The comparator then just retrieves the according field from the objects and compares them. Lookup then requires creating a temporary object and using find() with it. Some implementations also have an extension that allows searching with a different type, which would make this much easier but also make porting require actual work.
It is important that the fields used as keys remain constant though, because if you modify them, you implicitly change the order of the set<>. This is the reason that a set<>'s elements are effectively constant, i.e. even a plain iterator has a constant as value type. If you store pointers though, you can easily get around that, because a constant pointer is something different than a pointer to a constant. Don't shoot yourself into the foot with that!
If the metrics are not so much a property of the objects themselves (or you don't mind redundantly storing them), using an std::map would be a natural choice. Storing the same object under multiple keys, depending on the metric, can be done in separate containers (map<int,data*> c[10];). However, you can do that in a single map using e.g. a pair<metric,value> as key (map<pair<int,int>,data*> c;).
Using a vector<> to store the actual elements and only referencing them as either pointers or indices in a map surely works. I'd take the pointers though, as this is what allows the above approaches using a set or map to work. Without that, the comparator would have to store a reference to the container, where at the moment it just uses the global DATA container. Getting this to work with a vector is tricky though, since it reallocates its elements when growing, as you correctly pointed out. I'd consider a different container type, like std::list or std::deque. The former would allow erasing elements, too, but it has a higher per-element overhead. The latter has a relatively low per-element overhead, only slightly above std::vector. You could then even go so far as to store iterators instead of pointers, which helps debugging provided you use a "checked STL" for that. Still, you will have to do some manual bookkeeping which object is still referenced somewhere and which one isn't.
Instead of using a separate container, you could also allocate the elements dynamically, although that itself has some overhead. If the overhead per element is not an issue, you could then use reference-counted smart pointers. If the application is a one-shot process, you could also use raw pointers and let the OS reclaim the memory on exit.
Note that I assume that storing multiple copies of the data objects is not an option. If that was the case, you could just as well have a map<int,data> m[10];, where each map stores its own copy of the data objects. All the bookkeeping issues would then be resolved, but at the price of a 10x overhead.
I want a standard C++ container to use as an accumulator for reading from a network socket (that is, T = byte or unsigned char). I want the container to reserve a capacity on construction, and not initialize the elements. That is, I want to be able to do:
container c(1024);
and get the container to reserve 1024 octets. One-step construction/capacity is important because I want to use it in an initializer.
I also want contiguous storage. If the containers must grow, the new storage should be contiguous.
I also want to be able to append bytes to the container. And I want to be able to search for byte strings in the container.
vector and string don't really fit because construction and reserve are two step process, they use an extra allocation, and they initialize elements. Plus, vector is missing the search functionality. (EDIT: vector is fine thanks to <algorithm>; thanks DYP and Lightness Races in Orbit).
Are there any C++ standard containers that have the properties?
You are asking the wrong question.
The Standard Library does not provide a container for each and every use, instead it provides a selection of useful building bricks.
If you want contiguous storage, then you should be using std::vector and look into making it match your other requirements:
non-initialization is accomplished by providing an allocator with a no-op construct method
and searching for a pattern is accomplished by using std::search or rolling your own specific search algorithm (Knuth Morris Pratt, Boyer Moore, ...)
In any case, I advise encapsulating the vector into a dedicated class with a clear semantic role.
I am confused. I don't know what containers should I use. I tell you what I need first. Basically I need a container that can stored X number of Object (and the number of objects is unknown, it could be 1 - 50k).
I read a lot, over here array vs list its says: array need to be resized if the number of objects is unknown (I am not sure how to resize an array in C++), and it also stated that if using a linked list, if you want to search certain item, it will loop through (iterate) from first to end (or vice versa) while an array can specify "array object at index".
Then I went for an other solution, map, vector, etc. Like this one: array vs vector. Some responder says never use array.
I am new to C++, I only used array, vector, list and map before. Now, for my case, what kind of container you will recommend me to use? Let me rephrase my requirements:
Need to be a container
The number of objects stored is unknown but is huge (1 - 40k maybe)
I need to loop through the containers to find specific object
std::vector is what you need.
You have to consider 2 things when selecting a stl container.
Data you want to store
Operations you want to perform on the stored data
There wasa good diagram in a question here on SO, which depitcs this, I cannot find the link to it but I had it saved long time ago, here it is:
You cannot resize an array in C++, not sure where you got that one from. The container you need is std::vector.
The general rule is: use std::vector until it doesn't work, then shift to something that does. There are all sorts of theoretical rules about which one is better, depending on the operations, but I've regularly found that std::vector outperforms the others, even when the most frequent operations are things where std::vector is supposedly worse. Locality seems more important than most of the theoretical considerations on a modern machine.
The one reason you might shift from std::vector is because of iterator validity. Inserting into an std::vector may invalidate iterators; inserting into a std::list never.
Do you need to loop through the container, or you have a key or ID for your objects?
If you have a key or ID - you can use map to be able to quickly access the object by it, if the id is the simple index - then you can use vector.
Otherwise you can iterate through any container (they all have iterators) but list would be the best if you want to be memory efficient, and vector if you want to be performance oriented.
You can use vector. But if you need to find objects in the container, then consider using set, multiset or map.
I'm searching for a container with the following functionality:
Fixed size at runtime. Thus, memory shouldn't be allocated in little chunks (like std::list does).
Elements should be swappable (Something like std::list::splice offers).
EDIT:
Thinking of a list: I just need to move elements from an arbitrary position to the front.
EDIT2:
I would like to use something like a std::list, but takes advantage of a runtime fixed size.
I'd think of TR1 array:
std::array<T, int>
Or, if you haven't got that yet,
boost::array<T, int>
Which is identical for all intents and purposes. Of course the validity of std::swap on elements depends on availability of proper copy constructor/assignment operator.
It's not too clear to me what you are looking for. Two solutions come to mind: std::vector (created with the maximum size), coupled with a good implementation of swap for your objects, or a custom allocator for std::list, which pre-allocates the number of nodes you'll need in a single block.
You can specify the (initial) size of an std::list at construction time :
std::list<Type> myList = std::list<Type>(10);
You can still grow/shrink it afterwards, but the list will contain 10 default constructed Type objects from the start.
Does that suit your needs ?
std::vector ?
Array based. Can specify the size and has a swap function.
You haven't said what you'll be using it for so I don't really know if you're going to have any speed issues with it.
I require a linear vector container to behave like a multi-dimensional vector container. That is, a vector such as std::vector< std::vector< std::vector<int> > > be declared as std::vector<int> but behave as if it is multi-dimensional. This requires that I have set and get functions that allow accessing by multiple indices, rather than just one (that is, x, y, z and not just i).
The conversion is not the problem (that is, converting a 3D or 2D index to 1D), I am already doing that. However, I am starting to require the same conversion for several arrays. To make matters more complicated, I require a linear array to behave as if it is 2D, 3D and 4D. Now I want to make a container that will take care of the conversion internally. For all intents and purposes, the container will look like it is multi-dimensional, but it is in fact a linear array underneath.
So, first of, I am not sure whether libraries like boost have already something similar implemented. If so, please point me in the right direction. If not, then I would like to create one for my use but have a few questions.
My current approach is to make a class that has a vector container. There is a GetVec() function that will return the address of the std::vector so that I don't have to write all the functions that std::vector already has (getting the iterators, size etc.).
I want the user to be able to use the container from 1-D to N-D. When accessing, the user should be allowed to pass indexes that can represent a 1-D array or an N-D array. How do I ensure that the user is passing the correct number of indexes (in the function parameters)? Also, when initializing, the user should be able to specify the dimensions (variable parameter list?).
If this is a bad idea in general, please let me know but with good reasons. If not, any pointers on how to go about implementing this class would be much appreciated.
You have two choices: std::valarray (and its splice adapters), or boost::multi_array.
The first one lacks many things, and is not often used (it should deserve more love though). The second one is heavy but quite clean to use.
Also, you can have a look at Eigen if you intend to do linear algebra on those beasts.
I can also mention Intel Array Building Blocks which is simply awesome, and could be what you want in fine. It depends on what you do with those containers.