I am working on a Recurrent Neural Network in C++ using the Qt library. It gets constructed with a vector containing the topology where the number of elements define the number of layers and each element specifies the number of neurons in each layer.
The problem at hand is that the class needs to store some weights in matrices which depend on the size of the elements as well as their number is depended on the size of the vector. Required is a fast and efficient, index based access, 3 dimensional (of which one dimension differs in size) container. It does not necessarily be able to be resizable but AFAIK it can not be preallocated bevor the constructor, since size and number is given with the constructor.
I tried to store them in a nested vector but except from problems how to work with that I already read that nested vectors do not have a good performance, e.g.
typedef QVector < QVector < QVector < double> > > Weights;
I read something about ‘Dynamically Allocated Multi-Dimensional Arrays’ using arrays of pointer pointing to arrays of pointers pointing to ... and so on. This did not look very elegant.
Qt also provides some Matrix classes but since I can not preallocate them I neglected those.
Taking some additional library like Boost.
Another solution might be writing some class or custom container for my needs but I would rather not unless this is really the best way to do it.
So I am asking for advise what is the way to go here? Is there even a possibility to preallocate the memory or does it need to be dynamic?
Disclaimer: I am aware that there are a lot of similar questions around but they seem to lack comparing possibilities also I am not always sure if the underlying situation is the same.
Related
Because of external dependencies, I'm required to write some data in a Eigen::VectorXi container.
However, there are other times in which I'm not required to use en Eigen::VectorXi as a container, and I may use a standard std::vector<int>
Currently I have Eigen and std vectors sprinkled through my code. The only times in which I decide to use an std:vector is when I'll be making insertions on the fly, since resizing and reinserting is more tedious (and possibly computationally expensive) than using push_back(<value>)
I'd like to be sure that I'm using one or the other in the right time, instead of randomly choosing. What are the right instances to use Eigen::Vector instead of std::vector?
Classes in Eigen represent mathematical objects. In this case, a mathematical vector, with all the typical operations on vectors, like multiplication with a matrix or scalar product.
std::vector is essentially an (dynamic) array, storing a list of data in consecutive space. Typical operations would be appending data or sorting it.
When you decide what you want to use, simply ask yourself if this is a mathematical vector. You could extend both to hold what the other is supposed to, creating a scalar product over std::vector, but why would you? Eigen does that already. Using it for what it is intended makes it clear to everybody what you represent, increasing readability.
I search something which is std::vector but without the big overhead and a bit more than std::array, because with std::array I did not have the size stored anyway ( it is only known from the type itself ).
What I want to achieve:
Written with "dynamic" containers it is like:
std::map< int, std::vector< std::pair<int,int>>>;
I need no modification during runtime, but I need the size information during runtime. Replacing std::vector with std::array could not work, because the array must be the same size for all map entries which is not what I need.
I only! want to ask if there is already an available implementation around. If the answer is simply "No", there is no need for a suggestion how to do the job. I only want to not reinvent the wheel again :-)
Background: I can use the stl on my small avr controllers, but the overhead is "a bit" to high. So I search for a hopefully standard implementation which fit the needs for compile time constant representation with implemented features like begin()/end() and iterators to fulfill the minimum container requirements to get them used with range based for and others.
c++14 is also available, if there is something I search for.
All what I found is full template implemented where the access to the data is also compile time constant like:
container.get<2>()
which I also could not use, because I need runtime vars to access my data.
EDIT: Which problem/overhead comes up while using std::vector:
While using std::vector I need also new/delete which results in having malloc/free for avr. I also found that on avr the initialization of the vector itself takes arround 350 bytes code for each template instance I use. The access functions like operator[] and also the iterators are very small.
To avoid the overhead of std::vector, you may use instead std::initializer_list
const std::map<int, std::initializer_list<std::pair<int, int>>>
Not entirely clear to me what you're looking for, but I think you could achieve what you need with the following:
You don't know the exact size of each contained element (say N elements), and you will know it when program starts. Then, whenever you know the size of the containers, reserve just one contiguous memory block to hold all the inner elements of all the containers. (one dynamic allocation)
Create the map with vectors constructed via range: std::vector(start, end), where start and end are calculated via blocks of N elements.
Then populate the map. If you don't strictly need a map, you can also calculate where each vector will begin and end, and just create an initial array with indexes of the positions of each vector...
What you are asking seems to be impossible in principle. You say you want to avoid heap allocation of your vector like type, but the size of a type on the stack must be known at compile time, and the same for all members of that type. Since map is a homogeneous container, the value type must be a single type with a constant size. You could replace the map with a tuple, but then all of your keys would have to be known at compile time.
I would like to share the contents of an array of doubles a of size k with one or more STL vectors v1, v2...vn.
The effect that I want from this shared storage is that if the underlying array gets modified the change can be observed from all the vectors that share its contents with the array.
I can do that by defining the vectors v1...vn as vectors of pointers
vector<double*> v1;
and copy the pointers a to a + k into this vector. However, I do not like that solution. I want the vectors to be a vector of doubles.
Given that you can extract the underlying pointer from a vector I am assuming one could initialize a vector with an array in such a way that the contents are shared. Would appreciate help about how to do this.
Given that you can extract the underlying pointer from a vector I am assuming one could initialize a vector with an array in such a way that the contents are shared.
No, you can't do this. The Standard Library containers always manage their own memory.
Your best option is to create the std::vector<double> and then use it as an array where you need to do so (via &v[0], assuming the vector is non-empty).
If you just want to have the container interface, consider using std::array (or boost::array or std::tr1::array) or writing your own container interface to encapsulate the array.
This sounds to me like you want to alias the array with a vector. So logically you want a vector of references (which doesn't work for syntactical reasons). If you really really need this feature, you can write your own ref wrapper class, that behaves exactly like an actual C++ reference, so the users of your vn vectors wont be able to distinguish between vector<T> and vector<ref<T> > (e.g. with T = double). But internally, you could link the items in the vectors to the items in your "master" array.
But you should have darned good reasons to do this overhead circus :)
OK, Standard Library containers are both holders of information, and enumerators for those elements. That is, roughly any container can be used in almost any algorithm, and at least, you can go through them using begin() and end().
When you separate both (element holding and element enumeration), as in your case, you may consider boost.range. boost.range gives you a pair of iterators that delimit the extent to which algorithms will be applied, and you have the actual memory store in your array. This works mostly to read-access them, because normally, modifying the structure of the vector will invalidate the iterators. You can recreate them, though.
To answer your question, as far as I know std::vector can not be given an already constructed array to use. I can not even think how that could be done since there are also the size/capacity related variables. You can possibly try to hack a way to do it using a custom allocator but I feel it will be ugly, error prone and not intuitive for future maintenance.
That said, if I may rephrase your words a bit, you are asking for multiple references to the same std::vector. I would either do just that or maybe consider using a shared_ptr to a vector.
I am confused. I don't know what containers should I use. I tell you what I need first. Basically I need a container that can stored X number of Object (and the number of objects is unknown, it could be 1 - 50k).
I read a lot, over here array vs list its says: array need to be resized if the number of objects is unknown (I am not sure how to resize an array in C++), and it also stated that if using a linked list, if you want to search certain item, it will loop through (iterate) from first to end (or vice versa) while an array can specify "array object at index".
Then I went for an other solution, map, vector, etc. Like this one: array vs vector. Some responder says never use array.
I am new to C++, I only used array, vector, list and map before. Now, for my case, what kind of container you will recommend me to use? Let me rephrase my requirements:
Need to be a container
The number of objects stored is unknown but is huge (1 - 40k maybe)
I need to loop through the containers to find specific object
std::vector is what you need.
You have to consider 2 things when selecting a stl container.
Data you want to store
Operations you want to perform on the stored data
There wasa good diagram in a question here on SO, which depitcs this, I cannot find the link to it but I had it saved long time ago, here it is:
You cannot resize an array in C++, not sure where you got that one from. The container you need is std::vector.
The general rule is: use std::vector until it doesn't work, then shift to something that does. There are all sorts of theoretical rules about which one is better, depending on the operations, but I've regularly found that std::vector outperforms the others, even when the most frequent operations are things where std::vector is supposedly worse. Locality seems more important than most of the theoretical considerations on a modern machine.
The one reason you might shift from std::vector is because of iterator validity. Inserting into an std::vector may invalidate iterators; inserting into a std::list never.
Do you need to loop through the container, or you have a key or ID for your objects?
If you have a key or ID - you can use map to be able to quickly access the object by it, if the id is the simple index - then you can use vector.
Otherwise you can iterate through any container (they all have iterators) but list would be the best if you want to be memory efficient, and vector if you want to be performance oriented.
You can use vector. But if you need to find objects in the container, then consider using set, multiset or map.
I require a linear vector container to behave like a multi-dimensional vector container. That is, a vector such as std::vector< std::vector< std::vector<int> > > be declared as std::vector<int> but behave as if it is multi-dimensional. This requires that I have set and get functions that allow accessing by multiple indices, rather than just one (that is, x, y, z and not just i).
The conversion is not the problem (that is, converting a 3D or 2D index to 1D), I am already doing that. However, I am starting to require the same conversion for several arrays. To make matters more complicated, I require a linear array to behave as if it is 2D, 3D and 4D. Now I want to make a container that will take care of the conversion internally. For all intents and purposes, the container will look like it is multi-dimensional, but it is in fact a linear array underneath.
So, first of, I am not sure whether libraries like boost have already something similar implemented. If so, please point me in the right direction. If not, then I would like to create one for my use but have a few questions.
My current approach is to make a class that has a vector container. There is a GetVec() function that will return the address of the std::vector so that I don't have to write all the functions that std::vector already has (getting the iterators, size etc.).
I want the user to be able to use the container from 1-D to N-D. When accessing, the user should be allowed to pass indexes that can represent a 1-D array or an N-D array. How do I ensure that the user is passing the correct number of indexes (in the function parameters)? Also, when initializing, the user should be able to specify the dimensions (variable parameter list?).
If this is a bad idea in general, please let me know but with good reasons. If not, any pointers on how to go about implementing this class would be much appreciated.
You have two choices: std::valarray (and its splice adapters), or boost::multi_array.
The first one lacks many things, and is not often used (it should deserve more love though). The second one is heavy but quite clean to use.
Also, you can have a look at Eigen if you intend to do linear algebra on those beasts.
I can also mention Intel Array Building Blocks which is simply awesome, and could be what you want in fine. It depends on what you do with those containers.