Decimate vector in eigen - c++

I have a float array Eigen::ArrayXf which I need to decimate (i.e. pick 1 out of f.i. 8 samples).
Eigen::ArrayXf decimatedSignal = Eigen::Map<Eigen::ArrayXf, 0, Eigen::InnerStride<8> >(signal.data(), length, 1).eval();
which works, with a caveat: I need to know how long length is, and it can be specified too long, leading to runtime errors.
Q: is there a way to decimate all that is possible, so that resultant length is == signal.size() / 8 ?

Two things. You are using the c'tor for mapping a matrix:
Map (
PointerArgType dataPtr,
Index nbRows,
Index nbCols,
const StrideType & a_stride = StrideType()
)
Constructor in the dynamic-size matrix case.
Parameters
dataPtr pointer to the array to map
nbRows the number of rows of the matrix expression
nbCols the number of columns of the matrix expression
a_stride optional Stride object, passing the strides.
I think you want the c'tor for a vector:
Map ( PointerArgType dataPtr,
Index a_size,
const StrideType & a_stride = StrideType()
)
Constructor in the dynamic-size vector case.
Parameters
dataPtr pointer to the array to map
a_size the size of the vector expression
a_stride optional Stride object, passing the strides.
The second thing is that you want length == signal.size())/8. Is that always a whole integer, or are you rounding up? If the data is 16 in length and you want the positions [0] and [8], then use 1+(signal.size()-1)/8 as the length parameter:
Eigen::ArrayXf decimatedSignal = Eigen::Map<Eigen::ArrayXf, 0, Eigen::InnerStride<8> >(signal.data(), 1+((signal.size()-1)/8) ).eval();
For example:
#include <Eigen/Core>
#include <iostream>
using std::cout;
using std::endl;
int main(int argc, char *argv[])
{
Eigen::VectorXf signal;
signal.setLinSpaced(64, 0.0, 63.);
cout << "Original signal:" << endl << signal.transpose() << endl;
Eigen::ArrayXf decimatedSignal = Eigen::Map<Eigen::ArrayXf, 0,
Eigen::InnerStride<8> >(signal.data(), 1+((signal.size()-1)/8)).eval();
cout << endl << "Decimated:" << endl << decimatedSignal.transpose() << endl;
return 0;
}
outputs
Original signal:
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63
Decimated:
0 8 16 24 32 40 48 56
which I think is exactly what you want.

Related

How to extract data from Minecraft .mca files

I would like to generate a map for my own world, so I wrote a program try to analyze data from files in C++.
According to this page, the region file begins with a 4KB head which tells the positions of each chunk.
I wrote a program, but it outputs the wrong stuff.
This is my program
#include<bits/stdc++.h>
#include<fstream>
using namespace std;
char DataRaw[4100];
uint8_t DataUnsigned8[4100];
struct F4K{int pos,sz,dfn;} _chunk[40][40];
int main()
{
ifstream file("first4K.sample",ios::binary|ios::in|ios::ate);
ifstream::pos_type n=file.tellg();
freopen("offset.out","w",stdout);
file.seekg(0);
file.read((char*)(&DataRaw),n);
DataRaw[n]='\0';
file.close();
for (int i=0;i<n;i++)
DataUnsigned8[i+1]=uint8_t(DataRaw[i]);
for (int i=0;i<32;i++)
{
for (int j=0;j<32;j++)
{
int id=4*((i&31)+(j&31)*32);
_chunk[i][j].pos=(DataUnsigned8[id+1]<<16)|(DataUnsigned8[id+2]<<8)|(DataUnsigned8[id+3]);
_chunk[i][j].sz=DataUnsigned8[id+4];
}
}
cout<<" X Z offset sz\n-------------------"<<endl;
for (int i=0;i<32;i++)
{
for (int j=0;j<32;j++)
cout<<setw(2)<<i<<setw(3)<<j<<setw(9)<<_chunk[i][j].pos<<setw(4)<<_chunk[i][j].sz<<endl;
}
return 0;
}
And it outputs this
X Z offset sz
-------------------
0 0 2098751 32
0 1 2098252 2
0 2 139296 73
0 3 2113312 32
0 4 286978 32
0 5 2099058 3
0 6 2098275 2
0 7 139271 65
...
31 25 7602464 32
31 26 2556192 1
31 27 2105407 32
31 28 6546821 168
31 29 10927590 179
31 30 15109023 35
31 31 15315359 230
I expect that the offset is sorted and begins with 8192,but it was totally wrong! Some addressed (for example X:31,Y:31) is even bigger than the file size (The size is only 8,048,640 Bytes)
May anyone tell me why?

How do can I get rid of the blank lines that are created c++?

Edit:
Thank you all for the quick and helpful replies. I got it working now. It was because I had to reset the counter.
I have come to ask for help as my professor is not giving me the help I need. I am new to c++ and I am trying to program a program that displays all the integers from 1 to 100 that are divisible by 6 or 7, but not both. and I have to display 5 numbers per row. I got it working except I have blank lines forming in certain areas. I don't know if it's because of how I set up the counter or what.
Here is what I got.
#include <iostream>
using namespace std;
int main()
{
int counter = 0; // Counter for creating new lines after 5 numbers
for (int numRange = 1; numRange <= 100; ++numRange) // Starts the loop of number 1 to 100
{
if (numRange % 6 == 0 || numRange % 7 == 0) // Makes the numbers divisible by 6 and 7
{
cout << numRange << " "; // Displays the output of the divisible numbers
counter++; // Starts the counter
}
if (counter % 5 == 0) // using the counter to create new lines after 5 numbers displayed
{
cout << endl; // Creates a new line
}
}
return 0;
}
This is what is outputted:
6 7 12 14 18
21 24 28 30 35
36 42 48 49 54
56 60 63 66 70
72 77 78 84 90
91 96 98
and this is what it's supposed to look like
6 7 12 14 18
21 24 28 30 35
36 48 49 54 56
60 63 66 70 72
77 78 90 91 96
98
The problem that you're seeing is due to the fact that you are checking for "5 outputs" on every loop, rather than only on ones where a number has been output! So, to fix this issue (there are others), put the counter % 5 == 0 test inside the preceding if block:
for (int numRange = 1; numRange <= 100; ++numRange) // Starts the loop of number 1 to 100
{
if (numRange % 6 == 0 || numRange % 7 == 0) // Makes the numbers divisible by 6 and 7
{
cout << numRange << " "; // Displays the output of the divisible numbers
counter++; // Increments the counter
if (counter % 5 == 0) // Only need this if we have done some output!
{
cout << endl; // Creates a new line
}
}
}
Another problem is that, in this requirement:
that are divisible by 6 or 7, but not both
your code doesn't check for the "but not both" part (but that's not the 'title' question, and I'm not going to do all your homework in one fell swoop).

high performance calculations and saving of the threads identificators

I write grid-stride loop to have High Performance Calculations, where large N, for example long long N 1<<36, or even more. From total grid I need only some indexes, which have to satisfy the define condition.
__global__ void Indexes(int *array, int N) {
int index = blockIdx.x * blockDim.x + threadIdx.x;
while( index<N)
{
if (condition)
{....//do something to save index in array}
index += blockDim.x * gridDim.x;
}
}
Of course, it is possible use the Thrust, which allows to have both host and device arrays. But in this case obviously the calculation will be extremely ineffective, because need firstly to create a lot of non-needed elements, then to delete these.
What is the most effective way to save the indexes directly in array in device to pass in CPU?
If your output is relatively dense (i.e. a lot of indices and relatively few zeros), then the stream compaction approach suggested in comments is a good solution. There are a lot of ready-to-go stream compaction implementations which you can probably adapt to your purposes.
If your output is sparse, so you need to save relatively few indices for a lot of inputs, then stream compaction isn't such a great solution because it will waste a lot of GPU memory. In that case (and you can roughly estimate an upper bound of the number of output indices) something like this:
template <typename T>
struct Array
{
T* p;
int Nmax;
int* next;
Array() = default;
__host__ __device__
Array(T* _p, int _Nmax, int* _next) : p(_p), Nmax(_Nmax), next(_next) {};
__device__
int append(T& val)
{
int pos = atomicAdd(next, 1);
if (pos > Nmax) {
atomicExch(next, Nmax);
return -1;
} else {
p[pos] = val;
return pos;
}
};
};
is probably more appropriate. Here, the idea is to use an atomically incremented position in the output array to keep track of where a thread should store its index. The code will signal if you fill the index array, and there will be information from which you can work out a restart strategy to stop the current kernel and then start from the last known index which you were able to store.
A complete example:
$ cat append.cu
#include <iostream>
#include <thrust/device_ptr.h>
#include <thrust/device_vector.h>
#include <thrust/iterator/counting_iterator.h>
#include <thrust/copy.h>
namespace AppendArray
{
template <typename T>
struct Array
{
T* p;
int Nmax;
int* next;
Array() = default;
__host__ __device__
Array(T* _p, int _Nmax, int* _next) : p(_p), Nmax(_Nmax), next(_next) {};
__device__
int append(T& val)
{
int pos = atomicAdd(next, 1);
if (pos > Nmax) {
atomicExch(next, Nmax);
return -1;
} else {
p[pos] = val;
return pos;
}
};
};
}
__global__
void kernelfind(int* input, int N, AppendArray::Array<int> indices)
{
int idx = threadIdx.x + blockIdx.x * blockDim.x;
for(; idx < N; idx += gridDim.x*blockDim.x) {
if (input[idx] % 10000 == 0) {
if (indices.append(idx) < 0) return;
}
}
}
int main()
{
const int Ninputs = 1 << 20;
thrust::device_vector<int> inputs(Ninputs);
thrust::counting_iterator<int> vals(1);
thrust::copy(vals, vals + Ninputs, inputs.begin());
int* d_input = thrust::raw_pointer_cast(inputs.data());
int Nindices = Ninputs >> 12;
thrust::device_vector<int> indices(Nindices);
int* d_indices = thrust::raw_pointer_cast(indices.data());
int* pos; cudaMallocManaged(&pos, sizeof(int)); *pos = 0;
AppendArray::Array<int> index(d_indices, Nindices-1, pos);
int gridsize, blocksize;
cudaOccupancyMaxPotentialBlockSize(&gridsize, &blocksize, kernelfind, 0, 0);
kernelfind<<<gridsize, blocksize>>>(d_input, Ninputs, index);
cudaDeviceSynchronize();
for(int i = 0; i < *pos; ++i) {
int idx = indices[i];
std::cout << i << " " << idx << " " << inputs[idx] << std::endl;
}
return 0;
}
$ nvcc -std=c++11 -arch=sm_52 -o append append.cu
$ ./append
0 9999 10000
1 19999 20000
2 29999 30000
3 39999 40000
4 49999 50000
5 69999 70000
6 79999 80000
7 59999 60000
8 89999 90000
9 109999 110000
10 99999 100000
11 119999 120000
12 139999 140000
13 129999 130000
14 149999 150000
15 159999 160000
16 169999 170000
17 189999 190000
18 179999 180000
19 199999 200000
20 209999 210000
21 219999 220000
22 239999 240000
23 249999 250000
24 229999 230000
25 279999 280000
26 269999 270000
27 259999 260000
28 319999 320000
29 329999 330000
30 289999 290000
31 299999 300000
32 339999 340000
33 349999 350000
34 309999 310000
35 359999 360000
36 379999 380000
37 399999 400000
38 409999 410000
39 369999 370000
40 429999 430000
41 419999 420000
42 389999 390000
43 439999 440000
44 459999 460000
45 489999 490000
46 479999 480000
47 449999 450000
48 509999 510000
49 539999 540000
50 469999 470000
51 499999 500000
52 569999 570000
53 549999 550000
54 519999 520000
55 589999 590000
56 529999 530000
57 559999 560000
58 619999 620000
59 579999 580000
60 629999 630000
61 669999 670000
62 599999 600000
63 609999 610000
64 699999 700000
65 639999 640000
66 649999 650000
67 719999 720000
68 659999 660000
69 679999 680000
70 749999 750000
71 709999 710000
72 689999 690000
73 729999 730000
74 779999 780000
75 799999 800000
76 809999 810000
77 739999 740000
78 849999 850000
79 759999 760000
80 829999 830000
81 789999 790000
82 769999 770000
83 859999 860000
84 889999 890000
85 879999 880000
86 819999 820000
87 929999 930000
88 869999 870000
89 839999 840000
90 909999 910000
91 939999 940000
92 969999 970000
93 899999 900000
94 979999 980000
95 959999 960000
96 949999 950000
97 1019999 1020000
98 1009999 1010000
99 989999 990000
100 1029999 1030000
101 919999 920000
102 1039999 1040000
103 999999 1000000

How do I call the value in array and change the value in array?

class DataStorage{
// 0 1 2 3 4 5 6 7 8
string Data[20][4]={{"Wee","50","1","First"},{"Wee","22","2","First"},
// 9 10 11 12 13 14 15 16
{"Jason","26","3","First"},{"Krappa","12","4","First"},
// 17 18 19 20 21 22 23 24
{" "," ","5","First"},{" "," ","6","Economy"},
//25 26 27 28 29 30 31 32
{"Kappa","15","7","Economy"},{"Eraser","17","8","Economy"},
//33 34 35 36 37 38 39 40
{" "," ","9","Economy"},{"Morty"," ","10","Economy"},
//41 42 43 44 45 46 47 48
{"Rick"," ","11","Economy"},{"Amanda","10","12","Economy"},
//49 50 51 52 53 54 55 56
{"Lee","","13","Economy"},{"MingLee"," ","14","Economy"},
//57 58 59 60 61 62 63 64
{"Beauty"," ","15","Economy"},{"S4head"," ","16","Economy"},
//65 66 67 68 69 70 71 72
{"Ivan"," ","17","Economy"},{"Dex"," ","18","Economy"},
//73 74 75 76 77 78 79 80
{"Chua"," ","19","Economy"},{"Haha"," ","20","Economy"},};
};
int main(){
}
How do I call the value in array and change the value in array? Do I need to make some function to get value from the input and pass it into a variable in class and set it into my array?
I'm not sure what you're asking when you say How do I call the value in array and change the value in array? but I think you're asking how do you change the value of an array element.
To modify an array element you assign the array's index to what you're changing the array's element to; however, remember that C++ arrays are 0-index arrays meaning when you start counting their elements at 0. For example the following code modifies the element at index 5. Live preview
#include <iostream>
int array[10] = {1, 5, 33, 7, -23, 2, 8, 54, 19, 2};
int main() {
std::cout << array[5] << std::endl;
array[5] = 100; // Set the value of the element at index 5 to 100
std::cout << array[5] << std::endl;
return 0;
}
If you want to have Data as a class member of DataStorage you have to initialize it in the member initialization list. I also highly recommend to use an abstraction for the bare array, like std::array. This allows to use bounds-checked access with the at() function. You can then access Data and change it's contents.
#include <array>
#include <iostream>
#include <string>
class DataStorage
{
public:
std::array<std::array<std::string,4>,20> Data;
DataStorage() : Data({{
{{"Wee","50","1","First"}},
{{"Wee","22","2","First"}},
{{"Jason","26","3","First"}},
{{"Krappa","12","4","First"}},
{{" "," ","5","First"}},
{{" "," ","6","Economy"}},
{{"Kappa","15","7","Economy"}},
{{"Eraser","17","8","Economy"}},
{{" "," ","9","Economy"}},
{{"Morty"," ","10","Economy"}},
{{"Rick"," ","11","Economy"}},
{{"Amanda","10","12","Economy"}},
{{"Lee","","13","Economy"}},
{{"MingLee"," ","14","Economy"}},
{{"Beauty"," ","15","Economy"}},
{{"S4head"," ","16","Economy"}},
{{"Ivan"," ","17","Economy"}},
{{"Dex"," ","18","Economy"}},
{{"Chua"," ","19","Economy"}},
{{"Haha"," ","20","Economy"}}
}}) {}
};
int main()
{
DataStorage d;
std::cout << d.Data.at(10).at(2) << '\n'; // prints 11
d.Data.at(10).at(2) = "1729";
std::cout << d.Data.at(10).at(2) << '\n'; // prints 1729
}

c++ class return reference to itself

Can a c++ object have a method that return a reference to itself?
I want multiple independent instance of the class of the object.
It is for implementing the [] operator of a multi-dimensional array.
I want something like Array[0][1][2] to work.
Thanks.
Update:
Implementation of a multi-dimensional array:
Intended usage:
A[0][1] = 4; //the element at 0th row and 1st column is now 4.
A[0] = 5; //all elements of the 0th row are now 5.
A["all"][1] = 10; //all elements of the 1st column are now 10;
Object A has pointer ptr_buffer to a chunk of memory with correct simd alignment. Constructor of A would allocate the memory. Destructor of A deallocate the memory. A[] returns object B.
Object B has a pointer ptr_A_buffer to a subsection of the memory of A. B[] modifies ptr_A_buffer and also returns reference to itself. I don't want to constantly make a bunch of object B for each [] operation.
Both A and B belong to the same abstract class. Math functions take the abstract class as argument.
I was curious about the idea. Here is a data type which seems to fit the bill. It's an array type which, when called with three integers for the dimensions, allocates contiguous memory and then allows the user to define "views" into the data which have smaller dimensions (a plane, a row, a single value).
I made use of shared pointers which I have not really used before, so I may have made mistakes. I welcome corrections.
The idea is that copying views around is shallow; they all operate on the same underlying data. This allows me to pass them around by value with, as I believe, reasonable efficiency.
#include <iostream>
#include <iomanip>
#include <memory>
using namespace std;
/// This class is a three-dimensional array of doubles.
/// It defines an index operator which returns a view into that
/// data that is of one lesser dimension, just like the standard
/// index operator on plain old arrays. The last index operation
/// yields an "array" which is a single value.
/// Converting to double and assigning from double is defined as
/// using the first double in the view.
class NDimArrT
{
/// All slices hold a shared pointer to the base data they
/// are looking into so that their view stays valid.
const shared_ptr<double> baseData;
/// The data the view is looking at. Identical to the shared
/// ptr for the original object.
double *const slice;
/// The three dimensions, in the order of indexing.
/// All three of them may be zero, indicating a single value.
const int dim1, dim2, dim3;
public:
/// A single double value view into a one-dimensional array
NDimArrT(const shared_ptr<double> base, double *sliceStart)
: baseData(base), slice(sliceStart), dim1(0), dim2(0), dim3(0) {}
/// A 1D vector/row view into a two-dimensional array.
/// #param dim1Arg is the length of the row.
NDimArrT(const shared_ptr<double> base, double *sliceStart, int dim1Arg)
: baseData(base), slice(sliceStart), dim1(dim1Arg), dim2(0), dim3(0) {}
/// A 2D matrix plane view into the cube
NDimArrT(const shared_ptr<double> base, double *sliceStart, int dim1Arg, int dim2Arg)
: baseData(base), slice(sliceStart), dim1(dim1Arg), dim2(dim2Arg), dim3(0) {}
/// A 3D cube. This actually allocates memory.
NDimArrT(int dim1Arg, int dim2Arg, int dim3Arg)
: baseData(new double[dim1Arg*dim2Arg*dim3Arg], std::default_delete<double[]>() ),
slice(baseData.get()), // the data view is the whole array
dim1(dim1Arg), dim2(dim2Arg), dim3(dim3Arg) {}
/// Perform a shallow copy. We assume that e.g. returning a slice means
/// essentially returning another view into the main base array.
NDimArrT(const NDimArrT &rhs) = default;
/// Use standard move semantics. The rhs will be invalidated, and the
/// reference count to the base array does not grow. Can be used to return results from
/// functions.
NDimArrT(NDimArrT &&rhs) = default;
/// Default destructor; destroy baseData if it's the last reference.
~NDimArrT() = default;
/// Get the value of the first element we look at. Handy for
/// single value views.
operator double() const { return *slice; }
/// Return an instance of NDimArrT representing a view
/// with one dimension less than this. If we have a single value
/// already, simply return this single value. (We should
/// perhaps throw an exception there.)
NDimArrT operator[](int ind)
{
// This could be regarded an error, because this view
// is already a single element.
if(GetDims() == 0) { return *this; }
// This view is a 1-dimensional vector. Return the element at ind.
if(dim2==0) { return NDimArrT(baseData, slice + ind); } // return a single value.
// This view is a 2D matrix. Return the row no. ind.
// The row length is dim2. (Dim 1 indicates how many rows.)
if(dim3==0) { return NDimArrT(baseData, slice + dim2*ind, dim2); } // return a 1D vector
// This view is a true 3D cube matrix. dim1 is the number of planes,
// dim2 is the number of rows in a plane, dim3 is the length of each row.
// Return the plane no. ind, which starts at ind*planesize, i.e. ind*dim2*dim3.
// The number of rows and row length are dim2 and dim3, respectively.
return NDimArrT(baseData, slice + dim2*dim3*ind, dim2, dim3); // return a 2D matrix.
}
NDimArrT &operator=(double d) { *slice = d; }
int Len() { return dim1 ? dim1 : 1; } // interestingly, length is always dim1.
int GetDims() const
{
return dim1
? dim2
? dim3
? 3
: 2
: 1
: 0;
}
};
/// An example function which initializes an NDimArr of unknown
/// dimensionality with nice numbers..
void initNDimArr(NDimArrT arr, int &startVal)
{
// Single value? Give it the start value and increment that.
if( arr.GetDims() == 0 ) { arr = startVal++; }
else
{
for(int ind=0; ind<arr.Len(); ind++) { initNDimArr(arr[ind], startVal); }
}
}
// An example function doing something with
// an unknown n-dimensional array
void printNdimArr( NDimArrT nDimArr)
{
if( nDimArr.GetDims() == 0) { cout << setw(4) << nDimArr << " "; }
else
{
for(int i=0; i<nDimArr.Len(); i++) { printNdimArr(nDimArr[i]); }
cout << endl;
}
}
int main()
{
NDimArrT arr(3,4,5);
int start = 1;
initNDimArr(arr, start);
printNdimArr(arr);
// now use the middle plane of the 3 4x5 planes
cout << "Middle plane, values starting at 100:" << endl;
auto middlePlane = arr[1];
start = 100;
initNDimArr(middlePlane, start);
printNdimArr(middlePlane);
cout << "Whole array now:" << endl;
printNdimArr(arr);
cout << "Print line 2 of the 3rd plane:" << endl;
printNdimArr(arr[2][1]);
cout << endl << "Last number in that row is " << arr[2][1][4] << endl;
}
Sample session:
$>g++ -std=c++11 -o ndimContiguousArr ndimContiguousArr.cpp && ./ndimContiguousArr
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
16 17 18 19 20
21 22 23 24 25
26 27 28 29 30
31 32 33 34 35
36 37 38 39 40
41 42 43 44 45
46 47 48 49 50
51 52 53 54 55
56 57 58 59 60
Middle plane, values starting at 100:
100 101 102 103 104
105 106 107 108 109
110 111 112 113 114
115 116 117 118 119
Whole array now:
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
16 17 18 19 20
100 101 102 103 104
105 106 107 108 109
110 111 112 113 114
115 116 117 118 119
41 42 43 44 45
46 47 48 49 50
51 52 53 54 55
56 57 58 59 60
Print line 2 of the 3rd plane:
46 47 48 49 50
Last number in that row is 50