Often I find myself working with some STL container and wishing to modify it based on some external condition. By external, I mean something that cannot be derived from the object in the container alone.
For example, let's say I have worked out which elements of the container I want to keep based on some elaborate criterion involving not only the elements themselves. The keep flags are stored in a container of the same size as the original container. Now I want to use std::remove_if to remove those for which the flag is zero. How can I do that?
std::vector<Foo> container_of_foos;
std::vector<int> to_keep(container_of_foos.size(), 0);
// ... code calculates which foos to keep and stores a flag for each one
// NOTE: the condition relies information external to the Foo class (could be relation to other Foo instances)
auto i_new_end = std::remove_if(begin(container_of_foos), end(container_of_foos), [&to_keep](const Foo& foo) {
// can't tell whether to keep, because I don't know which object is iterated now
});
Using boost::zip_iterator
boost::zip_iterator over a set of tuple iterators really simplifies the removal.
std::vector<int> numbers = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9};
std::vector<bool> selectors = {0, 1, 0, 1, 0, 1, 0, 1, 0, 1};
auto zip_first = boost::make_zip_iterator(
boost::make_tuple(selectors.begin(), numbers.begin()));
auto zip_last = zip_first + std::min(selectors.size(), numbers.size());
auto removed_first = std::remove_if(zip_first, zip_last, [](const auto& tup) {
return boost::get<0>(tup) == 0;
});
Live example on Wandbox.
Using algorithms (non-boost)
Write your own remove/remove_if that takes two ranges (values and selectors) and a value/predicate. You'll have to decide on how you want to handle mismatched range distances. Once you have two ranges in your expected state, you can run compress both by erasing. Below is an example of compressing a range that is filtered using another range, terminating on the shortest sequence.
Remove by value and predicate
template <typename FwdIt1, typename FwdIt2, typename ValueType>
auto remove(FwdIt1 first1, FwdIt1 last1, FwdIt2 first2, FwdIt2 last2,
const ValueType value) {
FwdIt1 curr1 = first1;
FwdIt2 curr2 = first2;
for (; curr1 != last1 && curr2 != last2; ++curr1, ++curr2) {
if (value != *curr2) {
*first1++ = std::move(*curr1);
*first2++ = std::move(*curr2);
}
}
return std::make_pair(first1, first2);
}
template <typename FwdIt1, typename FwdIt2, typename Predicate>
auto remove_if(FwdIt1 first1, FwdIt1 last1, FwdIt2 first2, FwdIt2 last2,
Predicate pred) {
FwdIt1 curr1 = first1;
FwdIt2 curr2 = first2;
for (; curr1 != last1 && curr2 != last2; ++curr1, ++curr2) {
if (!pred(*curr2)) {
*first1++ = std::move(*curr1);
*first2++ = std::move(*curr2);
}
}
return std::make_pair(first1, first2);
}
Container-based Compression (remove & erase) helpers
template <typename Container, typename Selector, typename ValueType>
auto compress(Container& values, Selector& selectors, const ValueType& value) {
const auto remove_iters =
remove(std::begin(values), std::end(values), std::begin(selectors),
std::end(selectors), value);
return std::make_pair(
values.erase(remove_iters.first, std::end(values)),
selectors.erase(remove_iters.second, std::end(selectors)));
}
template <typename Container, typename Selector, typename Predicate>
auto compress_if(Container& values, Selector& selectors, Predicate pred) {
const auto remove_iters =
remove_if(std::begin(values), std::end(values), std::begin(selectors),
std::end(selectors), pred);
return std::make_pair(
values.erase(remove_iters.first, std::end(values)),
selectors.erase(remove_iters.second, std::end(selectors)));
}
Live Example on Wandbox.
In C++11, use a lambda as the predicate. For example;
void func()
{
std::vector<int> container;
// populate container
int flag = 42;
// modify flag as needed
auto lambda = [flag](int element) {return element < flag;} // whatever
std::remove_if(container.begin(), container.end(), lambda);
}
Before C++11, use a functor.
struct remover
{
int flag;
remover(int flag_value) : flag(flag_value) {};
bool operator()(int element) {return element < flag;};
};
void func()
{
std::vector<int> container;
// populate container
int flag = 42;
// modify flag as needed
remover functor(flag);
std::remove_if(container.begin(), container.end(), functor);
}
These two samples are essentially equivalent.
In C++11, look up lambda capture specifications to work out how to pass other variables to the lambda. Before that, change the functor so it is constructed using whatever variables are needed.
&x-vec.data() is the index of x in vec.
Assuming, of course, that x is in vec: otherwise, is UB.
Related
Say I have a vector with various entries, which I want to insert into another vector, while leaving out entries that satisfy a condition.
For example, I want to insert a vector while leaving out all three's.
{1, 3, 2, 3, 4, 5, 3} -> { /* previous content, */ 1, 2, 4, 5}
What I came up with so far uses std::partition, which does not preserve the relative order and rearranges the source vector.
std::vector<int> source({1, 3, 2, 3, 4, 5, 3});
std::vector<int> target;
auto partition = std::partition(std::begin(source),
std::end(source), [](const auto& a) { return a == 3; });
target.insert(std::begin(target), partition, std::end(source));
What I am looking for is more of an iterator that checks a condition and moves on if the condition is not satisfied. Something like this:
target.insert(std::begin(target),
conditional_begin(source, [](const auto& a) { return a != 3; }),
conditional_end(source));
I suppose a conditional_end function would be necessary, since std::end would return a different iterator type than conditional_begin.
Maybe I have overlooked something, so my questions are:
Does the standard library provide something similar?
Is there a different easy way to achieve my goal?
Is there an easy way to implement the conditional iterator functionality?
Is there a different easy way to achieve my goal?
Yes, the standard already has this functionality built in. The function you are looking for is std::copy_if.
std::vector<int> source({1, 3, 2, 3, 4, 5, 3});
std::vector<int> target;
std::copy_if(source.begin(),
source.end(),
std::back_inserter(target), [](auto val){ return val != 3; });
Here, std::back_inserter(target), will call push_back on target for each element that the predicate returns true.
Yes, you can create a custom iterator that does what you want but it is currently a little tedious to create custom iterators using standard C++. It would look something like this:
template <typename Itr, typename F>
struct ConditionalIterator {
Itr itr;
Itr end;
F condition;
using value_type = typename Itr::value_type;
using difference_type = typename Itr::difference_type;
using pointer = typename Itr::pointer;
using reference = typename Itr::reference;
using iterator_category = std::forward_iterator_tag;
ConditionalIterator() = default;
ConditionalIterator(Itr itr, Itr end, F condition): itr(itr), end(end), condition(condition) {}
bool operator!=(const ConditionalIterator &other) const { return other.itr != itr; }
reference operator*() const { return *itr; }
pointer operator->() const { return &(*itr); }
ConditionalIterator& operator++() {
for (; ++itr != end;) {
if (condition(*itr))
break;
}
return *this;
}
ConditionalIterator operator++(int) {
ConditionalIterator ret(*this);
operator++();
return ret;
}
};
You can then create something like the conditional_begin and conditional_end helper functions you asked for. The only issue is that std::vector::insert expects the two iterators to have the same type. If we use a lambda for our condition then this will be part of the type of our conditional iterator. So we need to pass the lambda to both helper functions so that they return iterators with matching types:
template <typename C, typename F>
auto conditional_begin(const C &source, F f) {
return ConditionalIterator<typename C::const_iterator, F>(source.begin(),
source.end(), f);
}
template <typename C, typename F>
auto conditional_end(const C &source, F f) {
return ConditionalIterator<typename C::const_iterator, F>(source.end(),
source.end(), f);
}
Which you could call with a lambda like this:
auto condition = [](const auto &a) { return a != 3; };
target.insert(std::begin(target),
conditional_begin(source, std::ref(condition)),
conditional_end(source, std::ref(condition)));
Live demo.
My crude tests show, in this case, this ends up being significantly faster than simply using copy_if and back_inserter because std::vector::insert first works out how much memory to allocate before inserting. Just using back_inserter will cause multiple memory allocations. The difference in performance will depend on how expensive the condition is to evaluate. You can get the same speedup by using count_if to reserve enough space before using copy_if:
auto count = static_cast<size_t>(std::count_if(source.begin(),
source.end(), condition));
target.reserve(target.size() + count);
std::copy_if(source.begin(),
source.end(),
std::back_inserter(target), condition);
Live demo.
As ranges will be standardized soon, this is an alternative using range-v3, the reference library for the proprosal:
#include <range/v3/view/concat.hpp>
#include <range/v3/view/filter.hpp>
using namespace ranges;
const std::vector<int> source{1, 3, 2, 3, 4, 5, 3};
const std::vector<int> target = view::concat(source,
source | view::filter([](auto i){ return i != 3; }));
Is there a simpler way to write this, e.g. by using an STL or boost algorithm?
std::vector<int> v { 0, 1, 2, 3 }; // any generic STL container
std::vector<int> result;
std::transform(v.begin(), v.end() - 1, // (0, 1, 2)
v.begin() + 1, // (1, 2, 3)
std::back_inserter(result),
[](int a, int b){ return a + b; }); // any binary function
// result == { 1, 3, 5 }
I propose using a for loop:
for(std::vector::size_type i = 0; i < v.size() - 1; i++)
result.push_back(v[i] + v[i+1])
A more generic loop for bidirectional iterators:
// let begin and end be iterators to corresponding position
// let out be an output iterator
// let fun be a binary function
for (auto it = begin, end_it = std::prev(end); it != end_it; ++it)
*out++ = fun(*it, *std::next(it));
We can go a bit further and write a loop for forward iterators:
if(begin != end) {
for (auto curr = begin,
nxt = std::next(begin); nxt != end; ++curr, ++nxt) {
*out++ = fun(*curr, *nxt);
}
}
Finally, and algorithm for input iterators. However, this one requires that the value type is copyable.
if(begin != end) {
auto left = *begin;
for (auto it = std::next(begin); it != end; ++it) {
auto right = *it;
*out++ = fun(left, right);
left = right;
}
}
The binary version of std::transform can be used.
The std::adjacent_find/std::adjacent_difference algorithms can be abused.
std::adjacent_difference is for exactly this, but as you mentioned, it copies the first element to the result, which you don't want.
Using Boost.Iterator, it's pretty easy to make a back_inserter which throws away the first element.
#include <boost/function_output_iterator.hpp>
template <class Container>
auto mybackinsrtr(Container& cont) {
// Throw away the first element
return boost::make_function_output_iterator(
[&cont](auto i) -> void {
static bool first = true;
if (first)
first = false;
else
cont.push_back(i);
});
}
Then you can #include <boost/range/numeric.hpp> and do this:
std::vector<int> v { 0, 1, 2, 3 }; // any generic STL container
std::vector<int> result;
boost::adjacent_difference(v, mybackinsrtr(result), std::plus<>{}); // any binary function
See it on ideone
When you want your binary function to return a different type (such as a string), the above solution won't work because, even though the insertion cont.push_back(i) is never called for the first copied element, it still must be compiled and it won't go.
So, you can instead make a back_inserter that ignores any elements of a different type than go in the container. This will ignore the first, copied, element, and accept the rest.
template <class Container>
struct ignore_insert {
// Ignore any insertions that don't match container's type
Container& cont;
ignore_insert(Container& c) : cont(c) {}
void operator() (typename Container::value_type i) {
cont.push_back(i);
}
template <typename T>
void operator() (T) {}
};
template <class Container>
auto ignoreinsrtr(Container& cont) {
return boost::make_function_output_iterator(ignore_insert<Container>{cont});
}
Then you can use it similarly.
std::vector<int> v { 0, 1, 2, 3 }; // any generic STL container
std::vector<std::string> result;
boost::adjacent_difference(v, ignoreinsrtr(result), [](int a, int b){ return std::to_string(a+b); });
On ideone
I would write your own algorithm to apply a functor to each pair of elements in the container.
(Shameless blurb) In my ACCU presentation this year, "STL Algorithms – How to Use Them and How to Write Your Own", showed how to write one like this. I called it adjacent_pair (about 25:00 into the video)
template <typename ForwardIterator, typename Func>
void adjacent_pair(ForwardIterator first, ForwardIterator last, Func f)
{
if (first != last)
{
ForwardIterator trailer = first;
++first;
for (; first != last; ++first, ++trailer)
f(*trailer, *first);
}
}
Stephan T. Lavavej has written a nice adjacent_iterator class here:
How do I loop over consecutive pairs in an STL container using range-based loop syntax?
This could also be used here.
Given
std::vector<T> first = /* some given data */, second;
I want to move all elements e which satisfy some condition cond(e) from first to second, i.e. something like
move_if(std::make_move_iterator(first.begin()),
std::make_move_iterator(first.end()),
std::back_inserter(second), [&](T const& e)
{
return cond(e);
});
I wasn't able to establish this with the algorithms library. So, how can I do that?
If the moved-from elements can stay where they are in first, then just use copy_if with move_iterator.
std::copy_if(std::make_move_iterator(first.begin()),
std::make_move_iterator(first.end()),
std::back_inserter(second), cond);
If the moved-from elements should be erased from first, I'd do
// partition: all elements that should not be moved come before
// (note that the lambda negates cond) all elements that should be moved.
// stable_partition maintains relative order in each group
auto p = std::stable_partition(first.begin(), first.end(),
[&](const auto& x) { return !cond(x); });
// range insert with move
second.insert(second.end(), std::make_move_iterator(p),
std::make_move_iterator(first.end()));
// erase the moved-from elements.
first.erase(p, first.end());
Or partition_copy with a move_iterator, followed by assignment:
std::vector<T> new_first;
std::partition_copy(std::make_move_iterator(first.begin()),
std::make_move_iterator(first.end()),
std::back_inserter(second), std::back_inserter(new_first), cond);
first = std::move(new_first);
The reason why move_if doesn't exist is because it would bloat the library. Either use copy_if with move iterator or write it yourself.
copy_if(move_iterator<I>(f), move_iterator<I>(l), out);
Here is an implementation by Jonas_No found at channel9.
template <typename FwdIt, typename Container, typename Predicate>
inline FwdIt move_if(FwdIt first, FwdIt last, Container &cont, Predicate pred)
{
if (first == last)
return last; // Empty so nothing to move
const size_t size = count_if(first, last, pred);
if (size == 0)
return last; // Nothing to move
cont.resize(size);
FwdIt new_end = first;
auto c = cont.begin();
for (auto i = first; i != last; ++i)
{
if (pred(*i)) // Should it move it ?
*c++ = move(*i);
else
*new_end++ = move(*i);
}
return new_end;
}
#T.C. has provided a perfectly working solution. However, at a first glance, one may not understand what the intend of that code is. So, it might be not perfect, but I tend to prefer something like this:
template<class InputIt, class OutputIt, class InputContainer, class UnaryPredicate>
OutputIt move_and_erase_if(InputIt first, InputIt last, InputContainer& c, OutputIt d_first, UnaryPredicate pred)
{
auto dist = std::distance(first, last);
while (first != last)
{
if (pred(*first))
{
*d_first++ = std::move(*first);
first = c.erase(first);
last = std::next(first, --dist);
}
else
{
++first;
--dist;
}
}
return d_first;
}
I need a method to remove all elements fulfilling a certain criteria from a range (an std::vector in this particular case) and copy those removed elements to a new range (so something like std::remove_if with an output parameter.) Neither the order of the input range nor the order of the output range after this operation is relevant.
One naive approach would be using std::partition to find all "evil" elements, then copy those and last remove them, but this would touch all the "evil" elements twice without need.
Alternatively I could write the desired remove_if variant myself, but why reinvent the wheel (plus I do not know if I can match the efficiency of a high quality library implementation).
So the question is:
Does a such a function already exist?
Boost is allowed, but standard C++ is preferred (the project does not depend on boost yet).
If not, is there a smart algorithm that is faster than a naive handcrafted remove_if variant would be?
No it doesn't. There's functions that do one (remove elements which match a predicate) or the other (copy elements which match a predicate) but not both. But it's easy enough to write our own in two steps:
template <typename InputIter, typename OutputIter, typename UnaryPredicate>
InputIter remove_and_copy(InputIter first, InputIter last,
OutputIter d_first, UnaryPredicate pred)
{
std::copy_if(first, last, d_first, pred);
return std::remove_if(first, last, pred);
}
To be used like:
std::vector<int> v = {1, 2, 3, 4, 5, 6, 7};
std::vector<int> u;
v.erase(
remove_and_copy(v.begin(), v.end(), std::back_inserter(u),
[](int i) { return i%2 == 0; }),
v.end()
);
// now v is {1, 3, 5, 7} and u is {2, 4, 6}
If you only need vector you can write it a bit differently, but still just 2 lines:
template <typename T, typename UnaryPredicate>
void remove_and_copy(std::vector<T>& from, std::vector<T>& to, UnaryPredicate pred)
{
std::copy_if(from.begin(), from.end(), std::back_inserter(to), pred);
from.erase(std::remove_if(from.begin(), from.end(), pred), from.end());
}
Or write your own loop:
template <typename T, typename UnaryPredicate>
void remove_and_copy(std::vector<T>& from, std::vector<T>& to, UnaryPredicate pred)
{
for (auto it = from.begin(); it != from.end(); ) {
if (pred(*it)) {
to.push_back(*it);
it = from.erase(it);
}
else {
++it;
}
}
}
Usage of either:
remove_and_copy(v, u, [](int i) { return i%2 == 0; });
The problem with removing while using iterators is that you don't have access to the actual container, so you can't actually get rid of the elements. Rather, for instance, what std::remove() does is move the target range to the end of the range, which the container will later use to actually remove the elements.
Instead, you can have your function take the stream as a parameter so you can call its removal method once the target value is found:
#include <algorithm>
#include <iterator>
#include <string>
#include <iostream>
template <typename Container, typename OutputIt, typename UnaryPredicate>
auto remove_and_copy_if(Container& c, OutputIt d_first, UnaryPredicate pred)
-> decltype(c.begin())
{
auto it = std::begin(c);
for (; it != std::end(c); )
{
while (it != std::end(c) && pred(*it))
{
d_first++ = *it;
it = c.erase(it);
}
if (it != std::end(c)) ++it;
}
return it;
}
template <typename Container, typename OutputIt, typename T>
auto remove_and_copy(Container& c, OutputIt d_first, T const& value)
-> decltype(c.begin())
{
return remove_and_copy_if(c, d_first,
[&] (T const& t) { return t == value; });
}
int main()
{
std::string str = "Text with some spaces ";
std::string output;
std::cout << "Before: " << str << '\n';
remove_and_copy(str, std::back_inserter(output), ' ');
std::cout << "After: " << str << '\n';
std::cout << "Characters removed: " << output << '\n';
}
Demo
Is there a couple of std::algorithm/lambda function to access the nth element satisfying a given condition. Because std::find_if will access the first one, so is there an equivalend to find the nth one ?
You need to create a stateful predicate that will count the number of instances and then complete when the expected count is reached. Now the problem is that there are no guarantees as of how many times the predicate will be copied during the evaluation of the algorithm, so you need to maintain that state outside of the predicate itself, which makes it a bit ugly, but you can do:
iterator which;
{ // block to limit the scope of the otherwise unneeded count variable
int count = 0;
which = std::find_if(c.begin(), c.end(), [&count](T const & x) {
return (condition(x) && ++count == 6)
});
};
If this comes up frequently, and you are not concerned about performance, you could write a predicate adapter that created a shared_ptr to the count internally and updated it. Multiple copies of the same adapter would share the same actual count object.
Another alternative would be to implement find_nth_if, which could be simpler.
#include <iterator>
#include <algorithm>
template<typename Iterator, typename Pred, typename Counter>
Iterator find_if_nth( Iterator first, Iterator last, Pred closure, Counter n ) {
typedef typename std::iterator_traits<Iterator>::reference Tref;
return std::find_if(first, last, [&](Tref x) {
return closure(x) && !(--n);
});
}
http://ideone.com/EZLLdL
An STL-like function template would be:
template<class InputIterator, class NthOccurence class UnaryPredicate>
InputIterator find_nth_if(InputIterator first, InputIterator last, NthOccurence Nth, UnaryPredicate pred)
{
if (Nth > 0)
while (first != last) {
if (pred(*first))
if (!--Nth)
return first;
++first;
}
return last;
}
And if you absolutely want to use the std::find_if, you could have something like:
template<class InputIterator, class NthOccurence class UnaryPredicate>
InputIterator find_nth_if(InputIterator first, InputIterator last, NthOccurence Nth, UnaryPredicate pred)
{
if (Nth > 0) {
do
first = std::find_if(first, last, pred);
while (!--Nth && ++first != last);
return first;
}
else
return last;
}
David's answer is fine as it is. Let me just point out that the predicate can be abstracted into the iterators by using the Boost.Iterator library, in particular the boost::filter_iterator adaptor, which has the advantage that it can be used for a lot more algorithms as well (counting e.g.):
#include <iostream>
#include <vector>
#include <algorithm>
#include <boost/iterator/filter_iterator.hpp>
template<class ForwardIt, class Predicate, class Size>
ForwardIt find_if_nth(ForwardIt first, ForwardIt last, Predicate pred, Size n)
{
auto vb = boost::make_filter_iterator(pred, first, last);
auto const ve = boost::make_filter_iterator(pred, last, last);
while (vb != ve && --n)
++vb;
return vb.base();
}
int main()
{
auto const v = std::vector<int>{ 0, 0, 3, 0, 2, 4, 5, 0, 7 };
auto const n = 2;
auto const pred = [](int i){ return i > 0; };
auto const nth_match = find_if_nth(v.begin(), v.end(), pred, n);
if (nth_match != v.end())
std::cout << *nth_match << '\n';
else
std::cout << "less than n elements in v matched predicate\n";
}
Live example. This will print 2 (the 2nd element > 0, counting starting at 1, so that find_if matches find_if_nth with n==1. If the predicate is changed to i > 10 or if the nth element is changed to n = 6, it will return the end iterator.