Related
Is there something like the streams in Java 8 also in C++?
For Example, in Java you can iterate over a collection using streams like so:
int arr[] = { 1, 2, 3 };
Arrays.stream(arr); // this returns a stream
// on these streams you can apply some functions like the filter function:
int evenArray[] = Arrays.stream(arr).filter(a -> { return a % 2 == 0; }).toArray();
// the result of filter is another stream. You can also collect to a new array after
So, is there something like this in C++? I really like this feature of Java and all I found was an article explaining, there wasn't.
I am hoping, that C++ maybe got an update that slipped by me.
As pointed out by #NathanOliver, C++20 ranges come to the rescue.
If one wanted to first filter a std::vector<int>, then multiply ever element by 2, one could write:
#include <ranges>
#include <algorithm>
#include <vector>
#include <iostream>
namespace vi = std::ranges::views;
namespace rn = std::ranges;
int main()
{
std::vector<int> vec = { 2, 3, 1, 8, 5, 4, 6 };
auto result = vec | vi::filter([](int i) { return i % 2 == 0; })
| vi::transform([](int i) { return i * 2; });
// | rn::to<std::vector> // would collect to vector, but is C++23
for (auto r : result)
std::cout << r << " "; // output: 4 16 8 12
std::cout << std::endl;
}
However the output is a range, not a vector.
As far as I could find out, there was no standard way of converting back into a vector in C++20, however std::ranges::to was added in C++23. Up until then you need a workaround.
So I recently watched this talk on c++:
https://www.youtube.com/watch?v=mFUXNMfaciE
And I was very interested on trying it out. So after some toy program I am stuck on how to properly flatten a vector of vectors into a vector. According to the documentation here: https://ericniebler.github.io/range-v3/ This is possible using ranges::view::for_each. However I just can't seem to get it to work. Here is some minimal code.
#include <range/v3/all.hpp>
#include <iostream>
#include <vector>
int main()
{
auto nums = std::vector<std::vector<int>>{
{0, 1, 2, 3},
{5, 6, 7, 8},
{10, 20},
{30},
{55}
};
auto filtered = nums
| ranges::view::for_each([](std::vector<int> num) { return ranges::yield_from(num); })
| ranges::view::remove_if([](int i) { return i % 2 == 1; })
| ranges::view::transform([](int i) { return std::to_string(i); });
for (const auto i : filtered)
{
std::cout << i << std::endl;
}
}
range-v3 error messages tend to be pretty horrible, so much so that this one is actually better than most:
prog.cc: In lambda function:
prog.cc:16:90: error: no match for call to '(const ranges::v3::yield_from_fn) (std::vector<int>&)'
| ranges::view::for_each([](std::vector<int> num) { return ranges::yield_from(num); })
^
In file included from /opt/wandbox/range-v3/include/range/v3/view.hpp:38:0,
from /opt/wandbox/range-v3/include/range/v3/all.hpp:21,
from prog.cc:1:
/opt/wandbox/range-v3/include/range/v3/view/for_each.hpp:133:17: note: candidate: template<class Rng, int _concept_requires_132, typename std::enable_if<((_concept_requires_132 == 43) || ranges::v3::concepts::models<ranges::v3::concepts::View, T>()), int>::type <anonymous> > Rng ranges::v3::yield_from_fn::operator()(Rng) const
Rng operator()(Rng rng) const
^~~~~~~~
to someone with a bit of knowledge of range-v3's concepts emulation layer, this "clearly" states that the call to yield_from failed because the type of the argument you passed to it - std::vector<int> - does not satisfy the View concept.
The View concept characterizes a subset of ranges that do not own their elements, and therefore have all operations - move/copy construction/assignment, begin, end, and default construction - computable in O(1). The range composition algrebra in range-v3 works only on views to avoid having to deal with element lifetimes and to provide predictable performance.
yield_from rejects the std::vectors you are trying to pass since they are not views, but you could easily provide views by (1) taking the vectors as lvalues instead of by value in for_each, and (2) yielding view::all of those lvalues [DEMO]:
auto filtered = nums
| ranges::view::for_each([](std::vector<int>& num) {
return ranges::yield_from(ranges::view::all(num)); })
| ranges::view::remove_if([](int i) { return i % 2 == 1; })
| ranges::view::transform([](int i) { return std::to_string(i); });
But in this simple case, flattening a range of ranges of elements into a range of elements already has a purpose-specific view in range-v3: view::join. You may as well use that [DEMO]:
auto filtered = nums
| ranges::view::join
| ranges::view::remove_if([](int i) { return i % 2 == 1; })
| ranges::view::transform([](int i) { return std::to_string(i); });
Given:
struct Object {
int id;
...
};
list<Object> objectList;
list<int> idList;
What is the best way to order objectList depending on order of idList?
Example (pseudo code):
INPUT
objectList = {o1, o2, o3};
idList = {2, 3, 1};
ACTION
sort(objectList, idList);
OUTPUT
objectList = {o2, o3, o1};
I searched in documentation but I only found methods to order elements comparing among themselves.
You can store the objects in an std::map, with id as key. Then traverse idList, get the object out of map with its id.
std::map<int, Object> objectMap;
for (auto itr = objectList.begin(); itr != objectList.end(); itr++)
{
objectMap.insert(std::make_pair(itr->id, *itr));
}
std::list<Object> newObjectList;
for (auto itr = idList.begin(); itr != idList.end(); itr++)
{
// here may fail if your idList contains ids which does not appear in objectList
newObjectList.push_back(objectMap[*itr]);
}
// now newObjectList is sorted as order in idList
Here is another variant, which works in O(n log n). This is asymptotcally optimal.
#include <list>
#include <vector>
#include <algorithm>
#include <iostream>
#include <cassert>
int main() {
struct O {
int id;
};
std::list<O> object_list{{1}, {2}, {3}, {4}};
std::list<int> index_list{4, 2, 3, 1};
assert(object_list.size() == index_list.size());
// this vector is optional. It is needed if sizeof(O) is quite large.
std::vector<std::pair<int, O*>> tmp_vector(object_list.size());
// this is O(n)
std::transform(begin(object_list), end(object_list), begin(tmp_vector),
[](auto& o) { return std::make_pair(o.id, &o); });
// this is O(n log n)
std::sort(begin(tmp_vector), end(tmp_vector),
[](const auto& o1, const auto& o2) {
return o1.first < o2.first;
});
// at this point, tmp_vector holds pairs in increasing index order.
// Note that this may not be a contiguous list.
std::list<O> tmp_list(object_list.size());
// this is again O (n log n), because lower_bound is O (n)
// we then insert the objects into a new list (you may also use some
// move semantics here).
std::transform(begin(index_list), end(index_list), begin(tmp_list),
[&tmp_vector](const auto& i) {
return *std::lower_bound(begin(tmp_vector), end(tmp_vector),
std::make_pair(i, nullptr),
[](const auto& o1, const auto& o2) {
return o1.first < o2.first;
})->second;
});
// As we just created a new list, we swap the new list with the old one.
std::swap(object_list, tmp_list);
for (const auto& o : object_list)
std::cout << o.id << std::endl;
}
I assumed that O is quite large and not easily movable. Therefore i first create tmp_vector which only contains of pairs. Then I sort this vector.
Afterwards I can simply go through the index_list and find the matching indices using binary search.
Let me elaborate on why a map is not the best solution eventhough you get a quite small piece of code. If you use a map you need to rebalance your tree after each insertion. This doesn't cost asympatotically (because n times rebalancing costs you the same as sorting once), but the constant is way larger. A "constant map" makes not that much sense (except accessing it may be easier).
I then timed the "simple" map-approach against my "not-so-simple" vector-approach. I created a randomly sorted index_list with N entries. And this is what I get (in us):
N map vector
1000 90 75
10000 1400 940
100000 24500 15000
1000000 660000 250000
NOTE: This test shows the worst case as in my case only index_list was randomly sorted, while the object_list (which is inserted into the map in order) is sorted. So rebalancing shows all its effect. If the object_list is kind of random, performance will behave more similar, eventhough performance will always be worse. The vector list will even behave better when the object list is completely random.
So already with 1000 entries the difference is already quite large. So I would strongly vote for a vector-based approach.
Assuming the data is handled to you externally and you don't have the choice of the containers:
assert( objectList.size() == idList.size() );
std::vector<std::pair<int,Object>> wrapper( idList.size() );
auto idList_it = std::begin( idList );
auto objectList_it = std::begin( objectList );
for( auto& e: wrapper )
e = std::make_pair( *idList_it++, *objectList_it++ );
std::sort(
std::begin(wrapper),
std::end(wrapper),
[]
(const std::pair<int,Object>& a, const std::pair<int,Object>& b) -> bool
{ return a.first<b.first; }
);
Then, copy back to original container.
{
auto objectList_it = std::begin( objectList );
for( const auto& e: wrapper )
*objectList_it++ = e;
}
But this solution is not optimal, I'm sure somebody will come with a better solution.
Edit: The default comparison operator for pairs requires that it is defined both for first and second members. Thus the easiest way is to provide a lambda.
Edit2: for some reason, this doesn't build if using a std::list for the wrapper. But it's ok if you use a std::vector (see here).
std::list has a sort member function you can use with a custom comparison functor.
That custom functor has to look up an object's id in the idList and can then use std::distance to calculate the position of the element in idList. It does so for both objects to be compared and returns true if the first position is smaller than the second.
Here is an example:
#include <iostream>
#include <list>
#include <algorithm>
#include <stdexcept>
struct Object
{
int id;
};
int main()
{
Object o1 = { 1 };
Object o2 = { 2 };
Object o3 = { 3 };
std::list<Object> objectList = { o1, o2, o3 };
std::list<int> const idList = { 2, 3, 1 };
objectList.sort([&](Object const& first, Object const& second)
{
auto const id_find_iter1 = std::find(begin(idList), end(idList), first.id);
auto const id_find_iter2 = std::find(begin(idList), end(idList), second.id);
if (id_find_iter1 == end(idList) || id_find_iter2 == end(idList))
{
throw std::runtime_error("ID not found");
}
auto const pos1 = std::distance(begin(idList), id_find_iter1);
auto const pos2 = std::distance(begin(idList), id_find_iter2);
return pos1 < pos2;
});
for (auto const& object : objectList)
{
std::cout << object.id << '\n';
}
}
It's probably not terribly efficient, but chances are you will never notice. If it still bothers you, you might want to look for a solution with std::vector, which unlike std::list provides random-access iterators. That turns std::distance from O(n) to O(1).
I would find it strange to end up in this situation as I would use the pointers instead of the ids. Though; there might be usecases for this.
Note that in all examples below, I assume that the ids-list contains all ids exactly ones.
Writing it yourself
The issue you like to solve is creating/sorting a list of objects based on the order of the ids in another list.
The naive way of doing this, is simply writing it yourself:
void sortByIdVector(std::list<Object> &list, const std::list<int> &ids)
{
auto oldList = std::move(list);
list = std::list<Object>{};
for (auto id : ids)
{
auto itElement = std::find_if(oldList.begin(), oldList.end(), [id](const Object &obj) { return id == obj.id; });
list.emplace_back(std::move(*itElement));
oldList.erase(itElement);
}
}
If you use a sorted vector as input, you can optimize this code to get the best performance out of it. I'm leaving it up-to you to do so.
Using sort
For this implementation, I'm gonna assume this are std::vector instead of std::list, as this is the better container to request the index of an element. (You can with some more code do the same for list)
size_t getIntendedIndex(const std::vector<int> &ids, const Object &obj)
{
auto itElement = std::find_if(ids.begin(), ids.end(), [obj](int id) { return id == obj.id; });
return itElement - ids.begin();
}
void sortByIdVector(std::list<Object> &list, const std::vector<int> &ids)
{
list.sort([&ids](const Object &lhs, const Object &rhs){ return getIntendedIndex(ids, lhs) < getIntendedIndex(ids, rhs); });
}
Insertion
Another approach, also more suitable for std::vector would be simply inserting the elements at the right place and will be more performant than the std::sort.
void sortByIdVector(std::vector<Object> &list, const std::vector<int> &ids)
{
auto oldList = std::move(list);
list = std::vector<Object>{};
list.resize(oldList.size());
for (Object &obj : oldList)
{
auto &newLocation = list[getIntendedIndex(ids, obj)];
newLocation = std::move(obj);
}
}
objectList.sort([&idList] (const Object& o1, const Object& o2) -> bool
{ return std::find(++std::find(idList.begin(), idList.end(), o1.id),
idList.end(), o2.id)
!= idList.end();
});
The idea is to check if we find o1.id before o2.id in the idList.
We search o1.id, increment the found position then we search o2.id: if found, that implies o1 < o2.
Test
#include <iostream>
#include <string>
#include <list>
#include <algorithm>
struct Object {
int id;
string name;
};
int main()
{
list<Object> objectList {{1, "one_1"}, {2, "two_1"}, {3, "three_1"}, {2, "two_2"}, {1, "one_2"}, {4, "four_1"}, {3, "Three_2"}, {4, "four_2"}};
list<int> idList {3, 2, 4, 1};
objectList.sort([&idList] (const Object& o1, const Object& o2) -> bool
{ return std::find(++std::find(idList.begin(), idList.end(), o1.id), idList.end(), o2.id) != idList.end(); });
for(const auto& o: objectList) cout << o.id << " " << o.name << "\n";
}
/* OUTPUT:
3 three_1
3 Three_2
2 two_1
2 two_2
4 four_1
4 four_2
1 one_1
1 one_2
*/
Suppose we have a function key_to_index that maps keys of a map to indices of a vector. For an example, let's just make it trivial:
std::map<int, int> source = {{1,55}, {4, 20}, {6, 25}};
std::vector<int> target;
int key_to_index(int key) {return key;}
What would be a version of the following loop that uses STL algorithms?
for (const auto &el: source) {
int index = key_to_index(el.first);
if (index > (int)target.size() - 1) target.resize(index + 1);
target[index] = el.second;
}
#Edgar answer is good, however, I do not like second map creation. Assuming that key_to_index is reasonably fast, it is better to just run it twice more than create map with converted indices.
Obvious optimization for your code (unless key_to_index is too complex) is to avoid more than one resizing. Then apply std::for_each to original map
auto max = std::max_element(source.cbegin(), source.cend(), [](auto& lhs, auto& rhs) {
return key_to_index(lhs.first) < key_to_index(rhs.first); });
target.resize(key_to_index(max->first) + 1);
std::for_each(source.cbegin(), source.cend(), [&target](const auto& e) {
target[key_to_index(e.first)] = e.second; });
Basically you can create a new map storing the same values with transformed keys:
std::map<int, int> transformed;
std::transform(std::cbegin(source), std::cend(source),
std::inserter(transformed, transformed.end()),
[](const auto& e) {
return std::make_pair(key_to_index(e.first), e.second);
}
);
And then fill the target:
std::vector<int> target;
target.resize(transformed.rbegin()->first + 1);
std::for_each(std::cbegin(transformed), std::cend(transformed),
[&target](const auto& e) {
target[e.first] = e.second;
}
);
wandbox example
Anyway I believe that the initial version is better. STL does not always make your code more efficient or even more readable.
You can create an output iterator, very similar to std::insert_iterator, with the value type of std::pair<int,int> and operator= that mutates your array. Then your function can be written as std::transform.
Sorry for my potential nOOb'ness but have been trying to get this for hours and cant seem to find an elegant solution for c++ 98.
My question is, say i have a vector of strings { a,b,c,d,e,f } and i want to move 'e' to the 2nd element how would i do so? Obviously the expected output would now print out { a,e,b,c,d,f }
Ideally looking for a single operation that lets me do this just for efficiency reasons but would love to hear some suggestions on how to achieve this.
Thanks.
It's not possible to do this "efficiently" with std::vector<>, because it is stored in contiguous memory and you must therefore move everything between the old and new locations by one element. So it's linear time in the length of the vector (or at least the distance moved).
The naive solution would be to insert() then erase(), but that requires moving everything after the rightmost location you modified, twice! So instead you can do it "by hand", by copying b through d one position to the right (e.g. with std::copy(), then overwriting b. At least then you avoid shifting anything outside the modified range. It looks like you may be able to make std::rotate() do this, as #WhozCraig mentioned in a comment.
I'd try with std::rotate first and only try other manual stuff (or a container other than vector) if that turns out not be efficient enough:
#include <vector>
#include <iostream>
#include <algorithm>
int main()
{
// move 5 from 4th to 1st index
std::vector<int> v {1,2,3,4,5,6};
// position: 0 1 2 3 4 5
std::size_t i_old = 4;
std::size_t i_new = 1;
auto it = v.begin();
std::rotate( it + i_new, it + i_old, it + i_old + 1);
for (int i : v) std::cout << i << ' ';
}
Live demo.
EDIT As noted in the comments, the below code actually mimics std::rotate, which is of course preferred above my hand-rolled code in all cases.
You can accomplish this with K swaps where K is the distance between the elements:
#include <iostream>
#include <string>
using namespace std;
int main()
{
string v = "abcdef"; // use string here so output is trivial
string::size_type insert_index = 1; // at the location of 'b'
string::size_type move_index = 4; // at the location of 'e'
while(move_index > insert_index)
{
std::swap(v[move_index], v[move_index-1]);
--move_index;
}
std::cout << v;
}
Live demo here. Note I used std::string, but the algorithm remains the same for std::vector. The same can be done with iterators, so you can generalize to containers that don't have operator[].
Expanding on jrok's answer, here's a wrapper around std::rotate() for moving a single element around. This is more general than jrok's example, in that it supports moving an element forward in the vector too (rather than only backward).
See the comments within rotate_single() explaining how you have to swap the logic around when moving the element forward versus back.
#include <vector>
#include <stdexcept> // for std::domain_error in range-checking assertion
#include <algorithm> // for std::rotate()
template<class ContiguousContainer>
void assert_valid_idx(ContiguousContainer & v, size_t index)
{
// You probably have a preferred assertion mechanism in your code base...
// This is just a sample.
if(index >= v.size())
{
throw std::domain_error("Invalid index");
}
}
template<class ContiguousContainer>
void rotate_single(ContiguousContainer & v, size_t from_index, size_t to_index)
{
assert_valid_idx(v, from_index);
assert_valid_idx(v, to_index);
const auto from_it = v.begin() + from_index;
const auto to_it = v.begin() + to_index;
if(from_index < to_index)
{
// We're rotating the element toward the back, so we want the new
// front of our range to be the element just after the "from" iterator
// (thereby making our "from" iterator the new end of the range).
std::rotate(from_it, from_it + 1, to_it + 1);
}
else if(to_index < from_index)
{
// We're rotating the element toward the front,
// so we want the new front of the range to be the "from" iterator.
std::rotate(to_it, from_it, from_it + 1);
}
// else the indices were equal, no rotate necessary
}
You can play with this in Compiler Explorer—there are (extensive) unit tests there, but here's an illustrative sample:
TEST_CASE("Handful of elements in the vector")
{
std::vector<int> v{1, 2, 3, 4, 5, 6}; // Note: this gets recreated for each SECTION() below
// position: 0 1 2 3 4 5
SECTION("Interior moves")
{
SECTION("Move 5 from 4th to 1st index")
{
rotate_single(v, 4, 1);
CHECK(v == std::vector<int>{1, 5, 2, 3, 4, 6});
}
SECTION("Move 2 from 1st to 4th index")
{
rotate_single(v, 1, 4);
CHECK(v == std::vector<int>{1, 3, 4, 5, 2, 6});
}
}
SECTION("Swap adjacent")
{
rotate_single(v, 4, 5);
rotate_single(v, 0, 1);
CHECK(v == std::vector<int>{2, 1, 3, 4, 6, 5});
}
}