I have a vector of vectors:
thrust::device_vector weights_;
which is a continuous amount of memory, where every w items,
represent a vector.
In one of my functions, I pass as parameters the begin and end of that range, like so:
__host__ ann::d_vector ann::prop_layer (
unsigned int weights_begin,
unsigned int weights_end,
ann::d_vector & input
) const
and then, I go and copy into a new vector that range,
and then get a raw pointer which I can use in a kernel:
thrust::device_vector<float> weights ( weights_.begin() + weights_begin,
weights_.begin() + weights_end );
float * weight_ptr = thrust::raw_pointer_cast( weights.data() );
some_kernel<<<numBlocks,numThreads>>>( weight_ptr, weight.size() );
Can I get a pointer from that range, without first copying it to a new vector? That seems like a waste of copy-realloc to me.
In case I can't get a pointer from that range, can I at least assign a vector to that range, without copying the actual values?
Can I get a pointer from that range, without first copying it to a new vector? That seems like a waste of copy-realloc to me.
Yes, you can get a pointer to that range.
float * weight_ptr = thrust::raw_pointer_cast( weights_.data() ) + weights_begin;
In case I can't get a pointer from that range, can I at least assign a vector to that range, without copying the actual values?
No, a thrust vector cannot be instantiated "on top" of existing data.
Related
What is the expected difference (if any) in memory taken by vvint1 and vvint2?
Is vitest1 copied into a new memory position each time a push_back takes place?
Is vitest2 copied into a new memory position each time a push_back takes place?
typedef vector<int> vint_t;
typedef vector<vint_t> vvint_t;
size_t nvec = 2;
size_t nvvec = 3;
vvint_t vvint1(nvvec), vvint2(nvvec);
vint_t vitest2(nvec, 1);
for ( j = 0; j < nvvec; j++ ) {
vint_t vitest1(nvec, 2);
vvint1.push_back(vitest1);
vvint2.push_back(vitest2);
}
Both vvint1 and vvint2 are initially created with nvvec = 3 default-constructed members (i.e. empty vectors of int).
push_back always either copies or moves, but in this case you're not supplying rvalue references so you'll get copies. Look into std::move for more on that.
You're pushing the same number of things to both vectors. Therefore both vvint1 and vvint2 will end up being the same size.
vvint1 and vvint2 memory requirements are:
(on stack, in the example) sizeof(vector<vector<int>>) for the objects themselves, which is the same (vector is 2–3 pointers usually, regardless of the inner type);
(on heap) 2 * nvvec * sizeof(vector<int>) for the contents (nvvec initially and nvvec push_back-ed in the loop); again, that’s the same for vvint1 and vvint2;
(on heap) contents of each vector stored in these vectors. Since vectors don’t share memory, and you store them by value, nvec * nnvec * sizeof(int). Again, the same.
So the overall requirements are the same:
sizeof(vector<vector<int>>) + nvvec * sizeof(vector<int>) + nvec * nnvec * sizeof(int)
Plain vector<int> would take less space ofc as item 2 wouldn’t apply. But what is more important is that in vvint_t, inner vectors may be of different lengths, and resizing any inner vector doesn’t affect others. But that adds complexity, so unless you really need that, it’s simpler to use flat vector and calculate index; imaging libraries do it that way.
Regarding second part, both vitests are copied on each push_back. But since C++11, you can write vvint1.push_back(std::move(vitest1)); (or vvint1.emplace_back(std::move(vitest1));) to move instead. For vectors that means the newly-constructed vector takes ownership of vitest1 contents without copying it (so vitest1 becomes empty). That doesn’t change memory requirements but reduces allocations as the space allocated by vitest (at construction) would be reused instead of being freed (at destruction, at end of each iteration).
I am trying to copy an array to a vector.
int A[1000]; //This array is filled by some function
vector<int> vec;
//some position from which I want to write the contents of the array into vector
int x = some_position;
vec.resize(sizeof(A)+x);
vec.insert(vec.begin()+x, A, A+sizeof(A));
The problem is that every fourth element is not copied correctly. The rest of the elements are copied correctly. i.e vec[x+3] != A[x+3]
for x=0,1,2,3....
First off, you need to check your understanding of sizeof. It returns the number of bytes needed for A as a whole, not the number of items in A, for that you would need sizeof(A)/sizeof(*A).
int A[1000];
vector<int> vec;
int x = 5;
vec.resize(x + sizeof(A) / sizeof(*A));
vec.insert(vec.begin()+x, A, A + sizeof(A) / sizeof(*A));
It's also worth noting that 'insert' may not be what you want. If your objective is to treat the vector like an array and overwrite a 1000 element long section of the vector, then you should use std::copy instead. Insert will resize the array even more, so if the resize will make the vector 1005 elements long, and them you start inserting at position 5, then the final vector will be 2005 elements long, with the contents of A going from 5 - 1004.
You could instead replace the insert line with this:
std::copy(A, A + sizeof(A) / sizeof(*A), vec.begin() + x);
This would overwrite the contents of the vector starting at position 5 and leave the vector sized at 1005.
The better way to copy array to vector:
vec.resize(1000+some_position);//if needed
std::copy(A,A+1000,vec.begin()+some_position);
It seems you believe sizeof() gives number of elements
e.g.
vec.resize(sizeof(A)+x);
but it doesn't. it gives the number of bytes.
the correct resizing should be something along the lines of
vec.resize(sizeof(A)/sizeof(int)+x);
of that follows that
vec.insert(vec.begin()+x, A, A+sizeof(A)/sizeof(int));
although I agree with Sergey that copy() is the better (more elegant) way to do it.
Your use of sizeof is wrong. sizeof is a very primitive operator,
which returns the number of bytes in the shallow image of the object
or type. This is totally useless except for very low level programming.
If you need to deal with C style arrays, there functions std::begin()
and std::end() in C++11; in earlier versions of C++, we just wrote
them ourselves. (I usually also wrote a size() function, which
basically returned the number of elements.) And std::vector works in
number of elements, not number of bytes. So your last two lines of code
should be:
vec.resize( x );
vec.insert( vec.end(), std::begin( A ), std::end( A ) );
At least, that's what I think you're trying to do, based on the
comments: create an std::vector<int> with x elements initialized to
0, followed by the contents of A.
Replace sizeof(A) with sizeof(A) / sizeof(A[0]) and it will work.
And as #Sergey pointed out, vec.resize(); in unnecessary in this case as insert() also resizes the vector.
don't copy an array into a vector. Use C++ to avoid that altogether. Instead of
void fill_array(int*, size_t);
int A[1000];
fill_array(A,1000);
std::vector<int> vec;
my_copy(vec,A);
simply do
std::vector<int> vec;
vec.resize(1000); // or whatever
fill_array(vec.data(),vec.size()); // std::vector::data() is C++11
In C++ (also pre C++11) you would actually do this more like this:
template<typename iterator> fill_data(iterator begin, iterator end);
std::vector<int> vec;
vec.resize(n); // make space (otherwise fill_data cannot fill in anything)
fill_data(vec.begin(), vec.end());
then your fill_data is generic enough to be re-used for any type of container.
I was searching about this topic and I found many ways to convert an array[] to an std::vector, like using:
assign(a, a + n)
or, direct in the constructor:
std::vector<unsigned char> v ( a, a + n );
Those solve my problem, but I am wondering if it is possible (and correct) to do:
myvet.resize( 10 );
memcpy( &myvet[0], buffer, 10 );
I am wondering this because I have the following code:
IDiskAccess::ERetRead nsDisks::DiskAccess::Read( std::vector< uint8_t >& bufferRead, int32_t totalToRead )
{
uint8_t* data = new uint8_t[totalToRead];
DWORD totalRead;
ReadFile( mhFile, data, totalToRead, &totalRead, NULL );
bufferRead.resize( totalRead );
bufferRead.assign( data, data + totalRead );
delete[] data;
return IDiskAccess::READ_OK;
}
And I would like to do:
IDiskAccess::ERetRead nsDisks::DiskAccess::Read( std::vector< uint8_t >& bufferRead, int32_t totalToRead )
{
bufferRead.resize( totalToRead );
DWORD totalRead;
ReadFile( mhFile, &bufferRead[0], totalToRead, &totalRead, NULL );
bufferRead.resize( totalRead );
return IDiskAccess::READ_OK;
}
(I have removed the error treatment of the ReadFile function to simplify the post).
It is working, but I am affraid that it is not safe. I believe it is ok, as the memory used by the vector is continuous, but I've never seen someone using vectors this way.
Is it correct to use vectors like this? Is there any other better option?
Yes it is safe with std::vector C++ standard guarantees that the elements will be stored at contiguous memory locations.
C++11 Standard:
23.3.6.1 Class templatevector overview [vector.overview]
A vector is a sequence container that supports random access iterators. In addition,itsupports(amortized) constant time insert and erase operations at the end; insert and erase in the middle take linear time. Storage management is handled automatically, though hints can be given to improve efficiency. The elements of a vector are stored contiguously, meaning that ifv is avector whereT is some type other than bool, then it obeys the identity&v[n] == &v[0] + n for all0 <= n < v.size().
Yes, it is fine to do that. You might want to do myvet.data() instead of &myvet[0] if it looks better to you, but they both have the same effect. Also, if circumstances permit, you can use std::copy instead and have more type-safety and all those other C++ standard library goodies.
The storage that a vector uses is guaranteed to be contiguous, which makes it suitable for use as a buffer or with other functions.
Make sure that you don't modify the vector (such as calling push_back on it, etc) while you are using the pointer you get from data or &v[0] because the vector could resize its buffer on one of those operations and invalidate the pointer.
That approach is correct, it only depends on the vector having contiguous memory which is required by the standard. I believe that in c++11 there is a new data() member function in vectors that returns a pointer to the buffer. Also note that in the case of `memcpy you need to pass the size in bytes not e size of the array
The memory in vector is guaranteed to be allocated contiguously, and unsigned char is POD, therefore it is totally safe to memcpy into it (assuming you don't copy more than you have allocated, of course).
Do your resize first, and it should work fine.
vector<int> v;
v.resize(100);
memcpy(&v[0], someArrayOfSize100, 100 * sizeof(int));
Yes, the solution using memcpy is correct; the buffer held by a vector is contiguous. But it's not quite type-safe, so prefer assign or std::copy.
Need for two dimensional array of objects that could be something like:
myContainer<myObject*> *a = new myArray<myObject*>( 20, 20 ); // passing int's as width & height
later accessing values would be done with methods such as:
mylist<myObject*> getRow( int );
mylist<myObject*> getColumn( int );
mylist<myObject*> getDiagonalRow( int );
implementations of those could be something like:
myList<myObject*> myContainer::getRow( int a ){
if( a < 0 && a>=this->height )
return;
myList<myObject*> hlp;
for( int i=0; i<this->width; i++)
hlp.append( this->arr[a][i] );
return hlp; // returns a copy. Could also be a pointer if created with new.
}
Other methods could follow similar lines, ie. creating a list object and filling it with what was requested.
My questions: Can anyone think of elegant way to create a container class I'm describing here. Which could for example avoid creating and filling of list-objects but still maintaining the abstraction and/or usability. Or please advice if I have missed something in STL etc. that has something like this.
STL has the valarray container which can be viewed as row and columns using slices, but you have to do it manually or wrap it into a wrapper class. Also the slices represents the values of the valarray (they are not a copy), but it is designed to be used with numbers and to be a bit optimized, it doesn't have any iterator and cannot be grown. it's doesn't follow the usual STL container concept. but it can still be used as a quick and dirty workaround if you can't use boost.
std::valarray<float> array(16);
// we can view it as a 4x4 matrix.
// this represents the first line
array[std::slice(0,4,1)];
// and the second column
array[std::slice(1,4,4)];
// you cannot use the sliced array directly. they don't
// have operator[], but they have operator=(valarray), and
// valarray has a constructor that takes sliced arrays as input.
What is the speed you are aiming for?
I can image to get a getRow() and getColumn() and getDiagonal()
in constant time when you handle with something list-like in the background.e.g. vector.
(if you just return a pointer)
Should the container grow or is the size fixed after the initialization?
If your container size is m and n you can store m std::vectors of size n,
n std::vectors of size m, and m+n std::vectors of different sizes.
Describing the rows, coloumns, and diagonals.
Then the get methods are easy to implement and in a at() or [] method/operator
three entries are to be changed.
i have a std::vector, namely
vector<vector<vector> > > mdata;
i want pass data from my mdata vector to the GSL function
gsl_spline_init(gsl_spline * spline, const double xa[], const double ya[], size_t size);
as ya. i already figured out that i can do things like
gsl_spline_init(spline, &(mgrid.front()), &(mdata[i][j][k].front()), mgrid.size());
this is fine if i want to pass the data from mdata for fixed i,j to gsl_spline_init().
however, now i would need to pass along the first dimension of mdata, so for fixed j,k.
i know that for any two fixed indices, all vectors along the remaining dimensions have the same length, so my vector is a 'regular cube'. so the offset between all the values i need should be the same.
of course i could create a temporary vector
int j = 123;
int k = 321;
vector<double> tmp;
for (int i = 0: i < mdata.size(); i++)
tmp.push_back(mdata[i][j][k]);
gsl_spline_init(spline, &(mgrid.front()), &(tmp.front()), mgrid.size());
but this seems too complicated. perhaps there is a way to achieve my goal with pointer arithmetic?
any help is greatly appreciated :)
You really can't do that without redesigning the array consumer function gsl_spline_init() - it relies on the data passed being a contiguous block of data. This is not the case with you three-level vector - not only it is a cube but also each level has a separate buffer allocated on heap.
This can't be done. Not only with vectors, but even with plain arrays only the last dimension is a contiguous block of data. If gsl_spline_init took an iterator instead of array, you could try to craft some functor to choose appropriate data but I'm not sure it's worth trying. No pointer arithmetic can help you.