I'm trying to figure out the best C++ library/package for array manipulations in a manner of python. Basically I need a simplicity like this:
values = numpy.array(inp.data)
idx1 = numpy.where(values > -2.14)
idx2 = numpy.where(values < 2.0)
res1 = (values[idx1] - diff1)/1000
res2 = (values[idx2] - diff2)*1000
In python it's just 5 lines, but the simplest way in C++ i can think of is quite a number of nested loops. Pls advise..
Basically my question is concerning the array/vector operations like array multiplications, operations on indexs, etc.. In the example above, res1 is an array, containing a set of elements filtered out of values array and some arithmetics applied afterward (subtraction, multiplication for all selected elements). In this python example I'm not copying elements of values array as it could be big enough in terms of memory, i'm keeping only the indexes and want to be able to run arithmetic operations on a selected set of elements of the original array.
You should not be using arrays at all. Please sit down and learn about the std::vector class and about iterators and Standard Library algorithms. I strongly suggest reading the book The C++ Standard Library.
You can achieve something similar in C++ but you shouldn't use plain C arrays for it.
The easiest way I can see this work would be using a std::set of floats (your code looks like it assumes that the data is sorted in ascending order). You could also use a std::vector of float but you'll have to sort that yourself, probably by using std::sort.
In that case, your example code could look like this - the set assumes the values are unique, if they aren't, you could use a std::multiset instead;
std::set<float> values(inp.data.begin(), inp.data.end());
std::set<float>::iterator idx1 = values.lower_bound(-2.14);
std::set<float>::iterator idx2 = values.upper_bound(2.0);
float res1 = (*idx1 - diff1) / 1000.0;
float res2 = (*idx2 - diff2) / 1000.0;
Please note that the above code sample is not a 100% conversion of your original code - lower_boundgives you the first element that's equal or larger than -2.14. I also didn't put any checking code in for failures - if lower_bound or upper_bound can't find matching elements, they would return values.end(), for example.
Using vector, the example would look very similar, just one line more to pre-sort the vector:
std::vector<float> values(inp.data.begin(), inp.data.end());
std::sort(values.begin(), values.end();
std::vector<float>::iterator idx1 = std::lower_bound(values.begin(), values.end(), -2.14);
std::vector<float>::iterator idx2 = std::upper_bound(values.begin(), values.end(), 2.0);
float res1 = (*idx1 - diff1) / 1000.0;
float res2 = (*idx2 - diff2) / 1000.0;
I suggest you to check the algorithm header.
Also don't work with arrays, you have std::vector or boost(soon to be std)::array.
wikipedia article
Reference for all algorithms
If I'm not mistaken, numpy is written mostly in C (with a Python wrapper), so you could be able to use it directly from C++ without much effort.
If you combine std::vector and boost::lambda, you can come really close to your example:
#include <algorithm>
#include <iostream>
#include <vector>
#include <boost/lambda/lambda.hpp>
using boost::lambda::_1;
int main() {
float ary[10] = { -4, -3, -2, -1, 0, 1, 2, 3, 4, 5 };
std::vector<float> v(&ary[0], &ary[10]);
std::vector<float>::iterator iter1, iter2;
iter1 = std::find_if(v.begin(), v.end(), (_1 > -2.14));
iter2 = std::find_if(v.begin(), v.end(), (_1 < 2.0));
// output:
// iter1 = -2.000
// iter2 = 1.000
std::cout
<< "iter1 = " << *iter1 << "\n"
<< "iter2 = " << *iter2 << "\n"
<< std::endl;
return 0;
}
Related
I am learning C++ for fun. I am trying to understand iterator arithmetic and have written a simple program that sums the first element in a vector with the last element. Next the second and second-to-last elements are summed. The process continues until all elements have been summed in this manner.
The issue seems to be with the *(a.end() - 1 - it) portion of my code. If I remove the - it, then I get the result I would expect. Adding the - it results in an error saying that
there were build errors. Would you like to continue to run the last successful build?
The errors I receive from Visual Studio are as follows;
E0075 Operand of * must be a pointer
C2100 illegal indirection
I have already approached this problem in a different way and was able to accomplish my goal. My question is why is this code an error, and could it be modified slightly to execute properly?
I appreciate any advice.
#include <iostream>
#include <vector>
using std::cin;
using std::cout;
using std::endl;
using std::vector;
int main()
{
vector<int> a{1, 2, 3, 4, 5, 6, 7, 8, 9, 10};
for (auto it = a.begin(); it != a.begin() + (a.end() - a.begin()) / 2; ++it)
cout << (*it) + *(a.end() - 1 - it) << endl;
return 0;
} // END MAIN
Error message
Error list
Subtracting 2 iterators with a.end() - 1 - it gives you the distance between them. You can't dereference a distance with * and so you get a compiler error.
This distance from a.begin() to the element you want to get at is needed in the expression. You can adjust the expression slightly by just adding a.begin() to that distance:
cout << (*it) + *(a.end() - 1 - it + a.begin()) << endl;
// ^^^^^^^^^^^
Here's a demo.
From C++20, instead of dealing with iterators and manipulating them, you can deal with ranges directly, which can be easier to read once you get used to it:
// just for convenience
namespace sv = std::views;
namespace sr = std::ranges;
// size of half the range
auto half = a.size() / 2;
sr::transform(a | sv::take(half), // first half
a | sv::reverse | sv::take(half), // second half reversed
std::ostream_iterator<int>(std::cout, "\n"), // print out
std::plus{}); // the sum
Here's a demo.
You should change code like this:
auto fix = it - a.begin();
cout << (*it) + *(a.end() - 1 - fix) << endl;
you can't manipulate on iterators like integers, you just can simply add or sub integers or use starting point from begin() - as end() isn't a real pointer for some value in vector.
Suppose
b = ["good ", "bad "]
a = ["apple","mango"]
then output = ["good apple","good mango","bad apple","bad mango"]
I know this can be done with nested for loops but is there some elegant one liner for doing this using C++ STL?
Here is a one-liner (copied from Jonathan Mee's answer posted here):
for(size_t i = 0, s = a.size(); i < output.size(); ++i) output[i] = b[i/s] + ' ' + a[i%s];
Full example here.
Given vector<string> a and vector<string> b you can use for_each:
vector<string> output(size(a) * size(b));
for_each(begin(output), end(output), [&, it = 0U](auto& i) mutable {
i = a[it / size(b)] + ' ' + b[it % size(b)];
++it;
});
Live Example
EDIT:
We've initialized output with enough room to contain every combination of a and b. Then we'll step through each element of output and assign it.
We'll want to use the 1st element of a for the first size(b) elements of output, and the 2nd element of a for the second size(b) elements, and so on. So we'll do this by indexing with it / size(b). We'll want to combine that by iteration through b's elements.
it will move to the next index for each element of output but the indexing needs to wrap or it will be out of bounds when it == size(b), to do that we use it % size(b).
EDIT2:
In this question through benchmarking I'd discovered the phenomenon that modulo and division are expensive operations for iteration. I've done the same test here. For the purpose of isolating the algorithms I'm just doing the Cartesian summation on a vector<int> not vector<string>.
First off we can see the two algorithms result in differing assembly. My algorithm as written above requires 585 lines of assembly. 588 lines were required by my interpretation of MSalter's code
vector<string> output(size(testValues1) * size(testValues2));
auto i = begin(output);
std::for_each(cbegin(a), cend(a), [&](const auto& A) { std::for_each(cbegin(b), cend(b), [&](const auto& B) { *i++ = A + ' ' + B; }); });
I have placed a pretty solid benchmarking test here: http://ideone.com/1YpzIO In the test I've only got it set to do 100 tests yet MSalters' algorithm always wins. Locally using Visual Studio 2015 in release with 10,000,000 tests MSalters algorithm finishes in about 2/3 the time it takes mine.
Clearly modulo isn't a great method of indexing :(
There's no direct solution; I checked the whole of <algorithm>. None of the functions produce an output of length M*N.
What you can do is call std::for_each on the first range, using a lambda which calls std::for_each on the second range (!)
std::vector<std::string> a, b;
std::for_each(a.begin(), a.end(),
[&](std::string A) { std::for_each(b.begin(), b.end(),
[A](std::string B) { std::cout << A << '/' << B << '\n'; }
);});
But that's just a nested loop in STL.
I am trying to do the following mathematical operation with two vectors:
v1 = [a1][a2][a3][a4][a5]
v2 = [b1][b2][b3][b4]b5]
Want to compute:
v = [a2*b2][a3*b3][a4*b4][a5*b5]
Note that I did not want the first element in the new vector.
I was wondering if there is a more efficient (one-liner) way to multiply (element-wise) two vectors in c++ than a for-loop (using push back). My current approach is as follows,
for(long i=1;i < v1.size();++i){
v.push_back(v1[i]*v2[i]);
}
I also tried the following,
for (long i = 1; i < v1.size(); ++i){
v[i-1] = v1[i]*v2[i];
}
Any suggestions?
std::transform( v1.begin()+1, v1.end(),
v2.begin()+1, v.begin(), // assumes v1,v2 of same size > 1,
// v one element smaller
std::multiplies<int>() ); // assumes values are 'int'
You can replace v.begin() with std::back_inserter(v) if v is empty, you should reserve() memory upfront to avoid multiple allocations.
You could look into std::valarray. It's designed to allow mathematical operations on every element in the array.
Say I have a vector / list whatever of ints populated with 2300 values
I want to be able to easily slice this into 4 vectors /lists (not necessarily of equal size).
e.g.
vec1 ( elements 0 - 500 )
vec2 ( elements 501 - 999)
vec3 ( elements 1001 - 1499)
etc.
A common way to do it would be to use the one container, and just define separate iterator ranges over it.
std::vector<int> vec(2300);
it0 = vec.begin();
it1 = vec.begin() + 500;
it2 = vec.begin() + 1000;
it3 = vec.begin() + 1500;
it4 = vec.begin() + 2000;
it5 = vec.end();
Now, the first range is simply defined by the iterators it0 and it1. The second by it1 and it2, and so on.
So, if you want to apply a function to every element in the third range, you'd simply do this:
std::for_each(it2, it3, somefunc);
Actually copying the elements into separate containers may be unnecessary, and would carry a performance cost.
std::list would be the best choice, as you just build lists by joining pointers. Finding the exact place to slice would be the problem, though, because you have to reach that point in the list iterator to make the cut.
EDIT:
As per comments (thanks for the insights), maybe using std::vector<int> and iterators is a good idea. However, with plain iterators, you loose the length of the vector, so I propose using, for instance, a boost::range_iterator:
std::vector<int> vec(2300);
it0 = vec.begin();
it1 = vec.begin() + 500;
it2 = vec.begin() + 1000;
it3 = vec.begin() + 1500;
it4 = vec.begin() + 2000;
it5 = vec.end;
typedef boost::iterator_range< std::vector<int>::iterator > my_slice_t;
my_slice_t slice1 = boost::make_iterator_range(it0, it1);
...
Then, you can use slice1 as a normal, underlying std::vector<int> as per iteration:
std::for_each(slice1.begin(), slice1.end(), /* stuff */);
See the fourth std::vector<> constructor documented here.
// given std::vector<T> vec with 2300 elements
std::vector<T> vec1(vec.begin(), vec.begin() + 500);
std::vector<T> vec2(vec.begin() + 500, vec.begin() + 1000);
std::vector<T> vec3(vec.begin() + 1000, vec.begin() + 1500);
std::vector<T> vec4(vec.begin() + 1500, vec.begin() + 2000);
std::vector<T> vec5(vec.begin() + 2000, vec.end());
Actually it is doable with the vector container
#include <vector>
#include <iostream>
using namespace std ;
int main()
{
vector<int> ints ;
vector<int> ints_sliced;
int i ;
// populate
for( i = 0 ; i < 100 ; i++ )
ints.push_back(i) ;
// slice from 10-19
ints_sliced.insert(ints_sliced.begin(), ints.begin()+10, ints.begin()+20) ;
// inspect
vector<int>::iterator it ;
for( it = ints_sliced.begin() ; it != ints_sliced.end() ; it++ )
cout << *it << endl ;
}
Oh, if you happen to be using g++ and GNU stdlibc++, you can use roughly
g++ -march=native -O3 -ftree-vectorize ...
If you also throw in GNU OpenMP support (libgomp) you can benefit (evaluate, profile!) from automatic parallelization of standard algorithms,
g++ -D_GLIBCXX_PARALLEL -fopenmp -march=native -O3 .... -lgomp
YMMV - I wanted to just throw this out there, because e.g. the parallel for_each seems to be close to what you want (but, automagic and self-adapting to container size, iterator type and number of processors)
In addition to #jalf's correct observation that actually copying the subvectors into fresh vectors might be a waste of time and space, let me point at valarray.
Intro: valarray
Valarray may be more complicated, but especially in the face of parallel processing, might lead to better ways to subvector work for the different threads. Things to look for:
algorithmic pre-science (if locations in a certain pattern have a certain property (e.g. are known to be zero), you can hand it to an optimized worker for those values)
subvector alignment (the aligment can make or break the availability of SIMD, SSE4 optimized versions; have a look at gcc -ftree-vectorizer for more background)
Now valarrays have quite a number of 'obscure' operations and tricks to them (gslices; basically revectored array dimensions to address the original array) that I won't go into here, but suffice it to say, if you want to do number crunching across subsets of contiguous arrays of (mainly) floating points[1], it will pay to read up on those.
Mandatory (braindead) teaser
// mask_array example
#include <iostream>
#include <valarray>
using namespace std;
int main ()
{
valarray<int> myarray (10);
for (int i=0; i<10; ++i) myarray[i]=i; // 0 1 2 3 4 5 6 7 8 9
valarray<bool> mymask (10);
for (int i=0; i<10; ++i)
mymask[i]= ((i%2)==1); // f t f t f t f t f t
myarray[mymask] *= valarray<int>(10,5); // 0 10 2 30 4 50 6 70 8 90
myarray[!mymask] = 0; // 0 10 0 30 0 50 0 70 0 90
cout << "myarray:\n";
for (size_t i=0; i<myarray.size(); ++i)
cout << myarray[i] << ' ';
cout << endl;
return 0;
}
This was copied verbatim from the above link, you will want to adapt to your specific need. There was probably a good reason why you kept the endgoal a bit vague, so I'll happily leave the rest of the work to you!
Wrapup
If you really want to go all the way, however, you should start looking at the big guns (Blitz++, et al.).
[1] these have historically been the focus for vectorized CPU instruction sets. However, als #jalf notes, SSE2 and higher includes SIMD integer instructions as well
How can I find the smallest value of each column in the given set of vectors efficiently ?
For example, consider the following program:
#include <iostream>
#include <vector>
#include <iterator>
#include <cstdlib>
using namespace std;
typedef vector<double> v_t;
int main(){
v_t v1,v2,v3;
for (int i = 1; i<10; i++){
v1.push_back(rand()%10);
v2.push_back(rand()%10);
v3.push_back(rand()%10);
}
copy(v1.begin(), v1.end(), ostream_iterator<double>(cout, " "));
cout << endl;
copy(v2.begin(), v2.end(), ostream_iterator<double>(cout, " "));
cout << endl;
copy(v3.begin(), v3.end(), ostream_iterator<double>(cout, " "));
cout << endl;
}
Let the output be
3 5 6 1 0 6 2 8 2
6 3 2 2 9 0 6 7 0
7 5 9 7 3 6 1 9 2
In this program I want to find the smallest value of every column (of the 3 given vectors) and put it into a vector. In this program I want to define a vector v_t vfinal that will have the values :
3 3 2 1 0 0 1 7 0
Is there an efficient way to do this ? I mention efficient because my program may have to find the smallest values among very large number of vectors. Thank you.
Update:
I'm trying to use something like this which I used in one of my previous programs
int count = std::inner_product(A, A+5, B, 0, std::plus<int>(), std::less<int>());
This counts the number of minimum elements between two arrays A and B. Wouldn't it be efficient enough if I could loop through and use similar kind of function to find the minimal values ? I'm not claiming it can be done or not. It's just an idea that may be improved upon but I don't know how.
You can use std::transform for this. The loops are still there, they're just hidden inside the algorithm. Each additional vector to process is a call to std::transform.
This does your example problem in two linear passes.
typedef std::vector<double> v_t;
int main()
{
v_t v1,v2,v3,vfinal(9); // note: vfinal sized to accept results
for (int i = 1; i < 10; ++i) {
v1.push_back(rand() % 10);
v2.push_back(rand() % 10);
v3.push_back(rand() % 10);
}
std::transform(v1.begin(), v1.end(), v2.begin(), vfinal.begin(), std::min<double>);
std::transform(v3.begin(), v3.end(), vfinal.begin(), vfinal.begin(), std::min<double>);
}
Note: this works in MSVC++ 2010. I had to provide a min functor for gcc 4.3.
I think that the lower bound of your problem is O(n*m), where n is the number of vectors and m the elements of each vector.
The trivial algorithm (comparing the elements at the same index of the different vectors) is as efficient as it can be, I think.
The easiest way to implement it would be to put all your vectors in some data structure (a simple C-like array, or maybe a vector of vectors).
The bst way to do this would be to use a vector of vectors, and just simple looping.
void find_mins(const std::vector<std::vector<int> >& inputs, std::vector<int>& outputs)
{
// Assuming that each vector is the same size, resize the output vector to
// change the size of the output vector to hold enough.
output.resize(inputs[0].size());
for (std::size_t i = 0; i < inputs.size(); ++i)
{
int min = inputs[i][0];
for (std::size_t j = 1; j < inputs[i].size(); ++j)
if (inputs[i][j] < min) min = inputs[i][j];
outputs[i] = min;
}
}
To find the smallest number in a vector, you simply have to examine each element in turn; there's no quicker way, at least from an algorithmic point-of-view.
In terms of practical performance, cache issues may affect you here. As has been mentioned in a comment, it will probably be more cache-efficient if you could store your vectors column-wise rather than row-wise. Alternatively, you may want to do all min searches in parallel, so as to minimise cache misses. i.e. rather than this:
foreach (col)
{
foreach (row)
{
x_min[col] = std::min(x_min[col], x[col][row]);
}
}
you should probably do this:
foreach (row)
{
foreach (col)
{
x_min[col] = std::min(x_min[col], x[col][row]);
}
}
Note that STL already provides a nice function to do this: min_element().