The Boost ICL interval_set can join right-open intervals, which touch each other, during adding them to the set. For example, intervals [0,4) and [4,8) will be joined to become an interval [0,8).
This is more complicated for the interval_map - intervals, which touch each other and have different associated values, won't be joined:
#include <iostream>
#include <utility>
#include <boost/icl/interval_map.hpp>
namespace icl = boost::icl;
using IMap = icl::interval_map<int, int>;
int main()
{
IMap m;
m += std::make_pair(IMap::interval_type::right_open(0, 4), 1);
m += std::make_pair(IMap::interval_type::right_open(4, 8), 2);
std::cout << m << std::endl;
}
Output of this test program is below:
{([0,4)->1)([4,8)->2)}
I know how to customize the process of aggregating on overlap, however I need to customize another case - aggregating on touch. For example, if intervals touch each other and value of the left interval is equal to the value of the right interval minus 1, then intervals must be joined, and the resulting interval must have a value of the left interval. So, the program above should print:
{([0,8)->1)}
Is it possible to do that with currently available Boost ICL?
I can do what I want using weird manipulations with the interval_map, but I think it'd be cumbersome and non-efficient. I'd prefer to be pointed in right direction to use currently available ICL customizations, functors etc.
This is more complicated for the interval_map - intervals, which touch each other and have different associated values, won't be joined:
There's no difference, really.
I know how to customize the process of aggregating on overlap, however I need to customize another case - aggregating on touch.
You seem to imply that
m += std::make_pair(IMap::interval_type::right_open(4, 8), 2);
will insert [4, 8) -> 2.
That's simply not the case. It's a codomain combination operation, and the results depend on prior state of the map.
Of course, you can write it:
m.set({Ival::right_open(4, 8), 2});
If you need to, you can query the preceding slot, so your operation might look like:
// returns true if joined with preceding slot
bool fill_slot(IMap& m, int from, int till, int value) {
bool joined = false;
auto slot = Ival::right_open(from, till);
if (within(slot, m)) {
// There is overlap, I don't know how you want to handle this.
// You can add some logic here.
} else {
auto preceding = m(from - 1);
if (preceding && value == preceding + 1) {
joined = true;
value = preceding;
}
}
m.set({slot, value});
return joined;
}
Now you can write test cases like:
int main() {
{
IMap m;
fill_slot(m, 0, 4, 1);
fill_slot(m, 4, 8, 2);
std::cout << m << std::endl;
}
{
IMap m;
fill_slot(m, 0, 4, 1);
fill_slot(m, 4, 8, 3);
std::cout << m << std::endl;
}
{
IMap m;
fill_slot(m, 0, 4, 1);
fill_slot(m, 5, 8, 2);
std::cout << m << std::endl;
}
}
And they print Live On Coliru
{([0,8)->1)}
{([0,4)->1)([4,8)->3)}
{([0,4)->1)([5,8)->2)}
Related
I want to merge k sorted pairwise key/value vectors by keys. Typically, the size n of the vectors is very large (e.g., n >= 4,000,000,000).
Consider the following example for k = 2:
// Input
keys_1 = [1, 2, 3, 4], values_1 = [11, 12, 13, 14]
keys_2 = [3, 4, 5, 6], values_2 = [23, 24, 25, 26]
// Output
merged_keys = [1, 2, 3, 3, 4, 4, 5, 6], merged_values = [11, 12, 13, 23, 14, 24, 25, 26]
Since __gnu_parallel::multiway_merge is a highly efficient k-way merge algorithm, I tried to utilize a state-of-the-art zip iterator (https://github.com/dpellegr/ZipIterator) to "combine" the key-value pair vectors.
#include <iostream>
#include <vector>
#include <parallel/algorithm>
#include "ZipIterator.hpp"
int main(int argc, char* argv[]) {
std::vector<int> keys_1 = {1, 2, 3, 4};
std::vector<int> values_1 = {11, 12, 13, 14};
std::vector<int> keys_2 = {3, 4, 5, 6};
std::vector<int> values_2 = {23, 24, 25, 26};
std::vector<int> merged_keys(8);
std::vector<int> merged_values(8);
auto kv_it_1 = Zip(keys_1, values_1);
auto kv_it_2 = Zip(keys_2, values_2);
auto mkv_it = Zip(merged_keys, merged_values);
auto it_pairs = {std::make_pair(kv_it_1.begin(), kv_it_1.end()),
std::make_pair(kv_it_2.begin(), kv_it_2.end())};
__gnu_parallel::multiway_merge(it_pairs.begin(), it_pairs.end(), mkv_it.begin(), 8, std::less<>());
for (size_t i = 0; i < 8; ++i) {
std::cout << merged_keys[i] << ":" << merged_values[i] << (i == 7 ? "\n" : ", ");
}
return 0;
}
However, I get various compilation errors (building with -O3):
error: cannot bind non-const lvalue reference of type' std::__iterator_traits<ZipIter<__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator > >, __gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator > > >, void>::value_type&' {aka 'std::tuple<int, int>&'} to an rvalue of type' std::tuple<int, int>'
error: cannot convert ‘ZipIter<__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator > >, __gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator > > >::reference*’ {aka ‘ZipRef<int, int>’} to ‘_ValueType’ {aka ‘std::tuple<int, int>*’}
Is it possible to modify the ZipIterator to make it work?
Or is there a more efficient way of merging k sorted pairwise key/value vectors by keys?
Considered Alternatives
Define a KeyValuePair struct with int key and int value members as well as operator< and operator<= operators. Move the elements of the key/value vectors into std::vector<KeyValuePair>s. Call __gnu_parallel::multiway_merge on the std::vector<KeyValuePair>s. Move the merged elements back into the key/value vectors.
[Verdict: slow execution, high memory overhead, even with -O3]
Use std::merge(std::execution::par_unseq, kv_it_1.begin(), kv_it_1.end(), kv_it_2.begin(), kv_it_2.end(), mkv_it.begin()); instead of __gnu_parallel::multiway_merge.
[Verdict: supports only two key/value vectors]
Is it possible to modify the ZipIterator to make it work?
Yes, but it would require patching __gnu_parallel::multiway_merge. The source of error is this line:
/** #brief Dereference operator.
* #return Referenced element. */
typename std::iterator_traits<_RAIter>::value_type&
operator*() const
{ return *_M_current; }
This is a member function of _GuardedIterator - an auxiliary structure used in the multiway_merge implementation. It wraps _RAIter class which in your case is ZipIter. By definition, when an iterator is dereferenced (*_M_current), the type of the returned expression is supposed to be reference type. However, this code expects it to be value_type&. In most cases, these are the same types. Indeed, when you dereference an item you expect to get a reference to this very item. However, it is impossible to do with a zip iterator, because its elements are virtual, they are created on the fly. That's why reference type of ZipIter is not a reference type at all, it is actually a value type called ZipRef:
using reference = ZipRef<std::remove_reference_t<typename std::iterator_traits<IT>::reference>...>;
Kind of the same practice that is used with (much hated) vector<bool>.
So, there is no problem with ZipIterator, or with how you use the algorithm, it is a non-trivial requirement for the algorithm itself. The next question is, can we get rid of it?
And the answer is yes. You can change _GuardedIterator::operator*() to return reference instead of value_type&. Then you will have a compile error in this line:
// Default value for potentially non-default-constructible types.
_ValueType* __arbitrary_element = 0;
for (_SeqNumber __t = 0; __t < __k; ++__t)
{
if(!__arbitrary_element
&& _GLIBCXX_PARALLEL_LENGTH(__seqs_begin[__t]) > 0)
__arbitrary_element = &(*__seqs_begin[__t].first);
}
Here the address of an element is taken for some __arbitrary_element. We can store a copy of this element instead since we know ZipRef is cheap to copy and it is default-constructible:
// Local copy of the element
_ValueType __arbitrary_element_val;
_ValueType* __arbitrary_element = 0;
for (_SeqNumber __t = 0; __t < __k; ++__t)
{
if(!__arbitrary_element
&& _GLIBCXX_PARALLEL_LENGTH(__seqs_begin[__t]) > 0) {
__arbitrary_element_val = *__seqs_begin[__t].first;
__arbitrary_element = &__arbitrary_element_val;
}
}
The same errors will appear in several places in the file multiseq_selection.h, e.g. here and here. Fix all of them using the similar technique.
Then you will see multiple errors like this one:
./parallel/multiway_merge.h:879:29: error: passing ‘const ZipIter<__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int> > >, __gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int> > > >’ as ‘this’ argument discards qualifiers [-fpermissive]
They are about const incorrectness. They are due to the fact that you declared it_pairs as auto, which in this particular scenario deduced the type to be std::inializer_list. This is a very peculiar type. For instance, it provides only constant access to its members, even though it itself is not declared const. That's the source of these errors. Change auto to e.g. std::vector and these errors are gone.
It should compile find at this point. Just don't forget to compile with -fopenmp or you will get "undefined reference to `omp_get_thread_num'" error.
Here is the output that I see:
$ ./a.out
1:11, 2:12, 3:13, 3:23, 4:14, 4:24, 5:25, 6:26
Since you need low memory overhead, one possible solution is to have the multiway_merge algorithm only operate on unique range identifiers and range indices and to supply the comparison and copy operators as lambda functions.
That way the merge algorithm is completely independent of the actual container types and key and value types used.
Here is a C++17 solution which is based on the heap based algorithm described here:
#include <cassert>
#include <cstdint>
#include <functional>
#include <initializer_list>
#include <iostream>
#include <iterator>
#include <queue>
#include <vector>
using range_type = std::pair<std::uint32_t,std::size_t>;
void multiway_merge(
std::initializer_list<std::size_t> range_sizes,
std::function<bool(const range_type&, const range_type&)> compare_func,
std::function<void(const range_type&)> copy_func)
{
// lambda compare function for priority queue of ranges
auto queue_less = [&](const range_type& range1, const range_type& range2) {
// reverse comparison order of range1 and range2 here,
// because we require the smallest element to be on top
return compare_func(range2, range1);
};
// create priority queue from all non-empty ranges
std::priority_queue<
range_type, std::vector<range_type>,
decltype(queue_less)> queue{ queue_less };
for (std::uint32_t range_id = 0; range_id < range_sizes.size(); ++range_id) {
if (std::data(range_sizes)[range_id] > 0) {
queue.emplace(range_id, 0);
}
}
// merge ranges until priority queue is empty
while (!queue.empty()) {
range_type top_range = queue.top();
queue.pop();
copy_func(top_range);
if (++top_range.second != std::data(range_sizes)[top_range.first]) {
// re-insert non-empty range
queue.push(top_range);
}
}
}
int main() {
std::vector<int> keys_1 = { 1, 2, 3, 4 };
std::vector<int> values_1 = { 11, 12, 13, 14 };
std::vector<int> keys_2 = { 3, 4, 5, 6, 7 };
std::vector<int> values_2 = { 23, 24, 25, 26, 27 };
std::vector<int> merged_keys;
std::vector<int> merged_values;
multiway_merge(
{ keys_1.size(), keys_2.size() },
[&](const range_type& left, const range_type& right) {
if (left == right) return false;
switch (left.first) {
case 0:
assert(right.first == 1);
return keys_1[left.second] < keys_2[right.second];
case 1:
assert(right.first == 0);
return keys_2[left.second] < keys_1[right.second];
}
return false;
},
[&](const range_type& range) {
switch (range.first) {
case 0:
merged_keys.push_back(keys_1[range.second]);
merged_values.push_back(values_1[range.second]);
break;
case 1:
merged_keys.push_back(keys_2[range.second]);
merged_values.push_back(values_2[range.second]);
break;
}
});
// copy result to stdout
std::cout << "keys: ";
std::copy(
merged_keys.cbegin(), merged_keys.cend(),
std::ostream_iterator<int>(std::cout, " "));
std::cout << "\nvalues: ";
std::copy(
merged_values.cbegin(), merged_values.cend(),
std::ostream_iterator<int>(std::cout, " "));
std::cout << "\n";
}
The algorithm has a time complexity of O(n log(k)) and a space complexity of O(k), where n is the total size of all ranges and k is the number of ranges.
The sizes of all input ranges need to be passed as an initializer list.
The example only passes the two input ranges from your example. Extending the example for more than two ranges is straightforward.
You will have to implement one that fits that exact case you have and with such a large arrays multi threatening may not be that good if you can afford to allocate a full or close to full copy of the arrays, one optimization you can do is to use large pages and ensure that the memory you are accessing is not paged (it is not ideal to not have swap if you plan to run at capacity).
This simple low memory example works just fine, it is hard to beat sequential i/o, the main bottle neck it has is the use of realloc, when displacing the used values from the arrs to the ret multiple reallocs at every step_size are made but only one is expensive, ret.reserve() can consume a "large" amount of time just because shortening a buffer is always available but extending one might not and multiple memory movements might be need to be made by the os.
#include <vector>
#include <chrono>
#include <stdio.h>
template<typename Pair, typename bool REVERSED = true>
std::vector<Pair> multi_merge_lm(std::vector<std::vector<Pair>>& arrs, float step){
size_t final_size = 0, max, i;
for (i = 0; i < arrs.size(); i++){
final_size += arrs[i].size();
}
float original = (float)final_size;
size_t step_size = (size_t)((float)(final_size) * step);
printf("Merge of %zi (%zi bytes) with %zi step size \n",
final_size, sizeof(Pair), step_size
);
printf("Merge operation size %.*f mb + %.*f mb \n",
3, ((float)(sizeof(Pair) * (float)final_size) / 1000000),
3, ((float)(sizeof(Pair) * (float)final_size * step) / 1000000)
);
std::vector<Pair> ret;
while (final_size --> 0){
for (max = 0, i = 0; i < arrs.size(); i++){
// select the next biggest item from all the arrays
if (arrs[i].back().first > arrs[max].back().first){
max = i;
}
}
// This does not actualy resize the vector
// unless the capacity is too small
ret.push_back(arrs[max].back());
arrs[max].pop_back();
// This check could be extracted of the while
// with a unroll and sort to little
for (i = 0; i < arrs.size(); i++){
if (arrs[i].empty()){
arrs[i] = arrs.back();
arrs.pop_back();
break;
}
}
if (ret.size() == ret.capacity()) {
// Remove the used memory from the arrs and
// realloc more to the ret
for (std::vector<Pair>& chunk : arrs){
chunk.shrink_to_fit();
}
ret.reserve(ret.size() + step_size);
// Dont move this to the while loop, it will slow down
// the execution, leave it just for debugging
printf("\rProgress %i%c / Merge size %zi",
(int)((1 - ((float)final_size / original) ) * 100),
'%', ret.size()
);
}
}
printf("\r%*c\r", 40, ' ');
ret.shrink_to_fit();
arrs.clear();
if (REVERSED){
std::reverse(ret.begin(), ret.end());
}
return ret;
}
int main(void) {
typedef std::pair<uint64_t, uint64_t> Pair;
int inc = 1;
int increment = 100000;
int test_size = 40000000;
float step_size = 0.05f;
auto arrs = std::vector<std::vector<Pair>>(5);
for (auto& chunk : arrs){
// makes the arrays big and asymmetric and adds
// some data to check if it works
chunk.resize(test_size + increment * inc++);
for (int i = 0; i < chunk.size(); i++){
chunk[i] = std::make_pair(i, i * -1);
}
}
printf("Generation done \n");
auto start = std::chrono::steady_clock::now();
auto merged = multi_merge_lm<Pair>(arrs, step_size);
auto end = std::chrono::steady_clock::now();
printf("Time taken: %lfs \n",
(std::chrono::duration<double>(end - start)).count()
);
for (size_t i = 1; i < merged.size(); i++){
if (merged[i - 1] > merged[i]){
printf("Miss placed at index: %zi \n", i - 1);
}
}
merged.clear();
return 0;
}
Merge of 201500000 (16 bytes) with 10075000 step size
Merge operation size 3224.000 mb + 161.200 mb
Time taken: 166.197639s
Running this thru a profiler (ANDuProf in my case) shows that the resizing is quite expensive, the larger you make the step_size the more efficient it becomes.
(Names are duplicated because they are from different parts of the code that call the same functions, and in this case, calls that the std functions make)
This rerun is with 0.5x, it is ~2x faster but now the function consumes 10x more memory than before and you should keep in mind that this values are not generic, they might change depending on what hardware you are running but the proportion are not going to change that much.
Merge of 201500000 (16 bytes) with 100750000 step size
Merge operation size 3224.000 mb + 1612.000 mb
Time taken: 72.062857s
Two other thing you shouldn't forget is that std::vector is dynamic and it's actual size might be bigger and O2 cant really do much optimization to the heap memory access, if you cant make it secuencial then the instruction can only wait.
I barely remember this, but you might find it helpful - I'm pretty sure I have seen merging K sorted linked lists problem. It was using something similar to Divide and Conquer and was close to logarithmic time complexity. I doubt it's any possible to get a better time complexity.
The logic behind this was to minimize iterations over merged lists. If you merge 1st and 2nd lists, then merging it with 3rd involves going through the longer, merged list. This method avoided this by merging all little lists at first, then moving to(what I like to call) '2nd layer merging' by merging 1-time merged lists.
This way, if your lists' length on average is n, you have to do at most logn iterators, resulting in K*log(n) complexity, where K is amount of the lists you have.
Sorry for being a little 'not-so-precise', but I think you might find this piece of information helpful. Although, I'm not familiar with multiway_merge by gnu, so whatever I said might be quite useless too.
Is there something like the streams in Java 8 also in C++?
For Example, in Java you can iterate over a collection using streams like so:
int arr[] = { 1, 2, 3 };
Arrays.stream(arr); // this returns a stream
// on these streams you can apply some functions like the filter function:
int evenArray[] = Arrays.stream(arr).filter(a -> { return a % 2 == 0; }).toArray();
// the result of filter is another stream. You can also collect to a new array after
So, is there something like this in C++? I really like this feature of Java and all I found was an article explaining, there wasn't.
I am hoping, that C++ maybe got an update that slipped by me.
As pointed out by #NathanOliver, C++20 ranges come to the rescue.
If one wanted to first filter a std::vector<int>, then multiply ever element by 2, one could write:
#include <ranges>
#include <algorithm>
#include <vector>
#include <iostream>
namespace vi = std::ranges::views;
namespace rn = std::ranges;
int main()
{
std::vector<int> vec = { 2, 3, 1, 8, 5, 4, 6 };
auto result = vec | vi::filter([](int i) { return i % 2 == 0; })
| vi::transform([](int i) { return i * 2; });
// | rn::to<std::vector> // would collect to vector, but is C++23
for (auto r : result)
std::cout << r << " "; // output: 4 16 8 12
std::cout << std::endl;
}
However the output is a range, not a vector.
As far as I could find out, there was no standard way of converting back into a vector in C++20, however std::ranges::to was added in C++23. Up until then you need a workaround.
class Solution
{
public:
void sort012(int a[], int n)
{
// code here
int low = 0;
int high = n-1;
int mid = 0;
while(mid<high)
{
int high = n-1;
if(a[mid]==0 && mid<=high)
{ swap(a[mid++],a[low++]);
}
else if(a[mid]==2 && mid<=high)
{ swap(a[mid],a[high--]);
}
else if(a[mid]==1 && mid<=high)
{
mid++;
}
}
}
};
Problem number one is you are redefining the int high = n - 1 inside of the while loop, at each iteration it's reset to this value, so high-- has no effect, and you're getting inside an infinite loop.
Problem number two is that potentially if you pass an array a which has a single value that is not a 0, 1 or 2, you are 100% getting into an infinite loop as well.
Check out this compiler explorer link for an interactive demo: https://godbolt.org/z/EbKPqrxz4
For what it's worth, you program looks like bad C instead of being C++. Non exhaustive list of issues:
The sort012 is an instance method on a class while it doesn't use the instance state. It's probably better as a free function, or at worse a static method on that class.
You're using C arrays.
As a result, you're also not using the algorithms provided by the STL.
I'm assuming this is a kind of coding exercise, but anyways, for the sake of completeness you could achieve the same thing (and more, it'd work with several containers, and regardless of your values/types) with fewer lines of code with this (Compiler Explorer):
#include <fmt/format.h>
#include <algorithm>
#include <array>
int main() {
std::array<int, 10> a{1, 2, 0, 1, 2, 1, 2, 1, 0, 2};
// Could also be a vector: `std::vector<int> a{1, 2, 0, 1, 2, 1, 2, 1, 0, 2};`
std::sort(a.begin(), a.end());
fmt::print("sorted a=");
for (auto x: a) {
fmt::print("{}, ", x);
}
}
Sorry for my potential nOOb'ness but have been trying to get this for hours and cant seem to find an elegant solution for c++ 98.
My question is, say i have a vector of strings { a,b,c,d,e,f } and i want to move 'e' to the 2nd element how would i do so? Obviously the expected output would now print out { a,e,b,c,d,f }
Ideally looking for a single operation that lets me do this just for efficiency reasons but would love to hear some suggestions on how to achieve this.
Thanks.
It's not possible to do this "efficiently" with std::vector<>, because it is stored in contiguous memory and you must therefore move everything between the old and new locations by one element. So it's linear time in the length of the vector (or at least the distance moved).
The naive solution would be to insert() then erase(), but that requires moving everything after the rightmost location you modified, twice! So instead you can do it "by hand", by copying b through d one position to the right (e.g. with std::copy(), then overwriting b. At least then you avoid shifting anything outside the modified range. It looks like you may be able to make std::rotate() do this, as #WhozCraig mentioned in a comment.
I'd try with std::rotate first and only try other manual stuff (or a container other than vector) if that turns out not be efficient enough:
#include <vector>
#include <iostream>
#include <algorithm>
int main()
{
// move 5 from 4th to 1st index
std::vector<int> v {1,2,3,4,5,6};
// position: 0 1 2 3 4 5
std::size_t i_old = 4;
std::size_t i_new = 1;
auto it = v.begin();
std::rotate( it + i_new, it + i_old, it + i_old + 1);
for (int i : v) std::cout << i << ' ';
}
Live demo.
EDIT As noted in the comments, the below code actually mimics std::rotate, which is of course preferred above my hand-rolled code in all cases.
You can accomplish this with K swaps where K is the distance between the elements:
#include <iostream>
#include <string>
using namespace std;
int main()
{
string v = "abcdef"; // use string here so output is trivial
string::size_type insert_index = 1; // at the location of 'b'
string::size_type move_index = 4; // at the location of 'e'
while(move_index > insert_index)
{
std::swap(v[move_index], v[move_index-1]);
--move_index;
}
std::cout << v;
}
Live demo here. Note I used std::string, but the algorithm remains the same for std::vector. The same can be done with iterators, so you can generalize to containers that don't have operator[].
Expanding on jrok's answer, here's a wrapper around std::rotate() for moving a single element around. This is more general than jrok's example, in that it supports moving an element forward in the vector too (rather than only backward).
See the comments within rotate_single() explaining how you have to swap the logic around when moving the element forward versus back.
#include <vector>
#include <stdexcept> // for std::domain_error in range-checking assertion
#include <algorithm> // for std::rotate()
template<class ContiguousContainer>
void assert_valid_idx(ContiguousContainer & v, size_t index)
{
// You probably have a preferred assertion mechanism in your code base...
// This is just a sample.
if(index >= v.size())
{
throw std::domain_error("Invalid index");
}
}
template<class ContiguousContainer>
void rotate_single(ContiguousContainer & v, size_t from_index, size_t to_index)
{
assert_valid_idx(v, from_index);
assert_valid_idx(v, to_index);
const auto from_it = v.begin() + from_index;
const auto to_it = v.begin() + to_index;
if(from_index < to_index)
{
// We're rotating the element toward the back, so we want the new
// front of our range to be the element just after the "from" iterator
// (thereby making our "from" iterator the new end of the range).
std::rotate(from_it, from_it + 1, to_it + 1);
}
else if(to_index < from_index)
{
// We're rotating the element toward the front,
// so we want the new front of the range to be the "from" iterator.
std::rotate(to_it, from_it, from_it + 1);
}
// else the indices were equal, no rotate necessary
}
You can play with this in Compiler Explorer—there are (extensive) unit tests there, but here's an illustrative sample:
TEST_CASE("Handful of elements in the vector")
{
std::vector<int> v{1, 2, 3, 4, 5, 6}; // Note: this gets recreated for each SECTION() below
// position: 0 1 2 3 4 5
SECTION("Interior moves")
{
SECTION("Move 5 from 4th to 1st index")
{
rotate_single(v, 4, 1);
CHECK(v == std::vector<int>{1, 5, 2, 3, 4, 6});
}
SECTION("Move 2 from 1st to 4th index")
{
rotate_single(v, 1, 4);
CHECK(v == std::vector<int>{1, 3, 4, 5, 2, 6});
}
}
SECTION("Swap adjacent")
{
rotate_single(v, 4, 5);
rotate_single(v, 0, 1);
CHECK(v == std::vector<int>{2, 1, 3, 4, 6, 5});
}
}
I'm trying to implement a class that will generate all possible unordered n-tuples or combinations given a number of elements and the size of the combination.
In other words, when calling this:
NTupleUnordered unordered_tuple_generator(3, 5, print);
unordered_tuple_generator.Start();
print() being a callback function set in the constructor.
The output should be:
{0,1,2}
{0,1,3}
{0,1,4}
{0,2,3}
{0,2,4}
{0,3,4}
{1,2,3}
{1,2,4}
{1,3,4}
{2,3,4}
This is what I have so far:
class NTupleUnordered {
public:
NTupleUnordered( int k, int n, void (*cb)(std::vector<int> const&) );
void Start();
private:
int tuple_size; //how many
int set_size; //out of how many
void (*callback)(std::vector<int> const&); //who to call when next tuple is ready
std::vector<int> tuple; //tuple is constructed here
void add_element(int pos); //recursively calls self
};
and this is the implementation of the recursive function, Start() is just a kick start function to have a cleaner interface, it only calls add_element(0);
void NTupleUnordered::add_element( int pos )
{
// base case
if(pos == tuple_size)
{
callback(tuple); // prints the current combination
tuple.pop_back(); // not really sure about this line
return;
}
for (int i = pos; i < set_size; ++i)
{
// if the item was not found in the current combination
if( std::find(tuple.begin(), tuple.end(), i) == tuple.end())
{
// add element to the current combination
tuple.push_back(i);
add_element(pos+1); // next call will loop from pos+1 to set_size and so on
}
}
}
If I wanted to generate all possible combinations of a constant N size, lets say combinations of size 3 I could do:
for (int i1 = 0; i1 < 5; ++i1)
{
for (int i2 = i1+1; i2 < 5; ++i2)
{
for (int i3 = i2+1; i3 < 5; ++i3)
{
std::cout << "{" << i1 << "," << i2 << "," << i3 << "}\n";
}
}
}
If N is not a constant, you need a recursive function that imitates the above
function by executing each for-loop in it's own frame. When for-loop terminates,
program returns to the previous frame, in other words, backtracking.
I always had problems with recursion, and now I need to combine it with backtracking to generate all possible combinations. Any pointers of what am I doing wrong? What I should be doing or I am overlooking?
P.S: This is a college assignment that also includes basically doing the same thing for ordered n-tuples.
Thanks in advance!
/////////////////////////////////////////////////////////////////////////////////////////
Just wanted to follow up with the correct code just in case someone else out there is wondering the same thing.
void NTupleUnordered::add_element( int pos)
{
if(static_cast<int>(tuple.size()) == tuple_size)
{
callback(tuple);
return;
}
for (int i = pos; i < set_size; ++i)
{
// add element to the current combination
tuple.push_back(i);
add_element(i+1);
tuple.pop_back();
}
}
And for the case of ordered n-tuples:
void NTupleOrdered::add_element( int pos )
{
if(static_cast<int>(tuple.size()) == tuple_size)
{
callback(tuple);
return;
}
for (int i = pos; i < set_size; ++i)
{
// if the item was not found in the current combination
if( std::find(tuple.begin(), tuple.end(), i) == tuple.end())
{
// add element to the current combination
tuple.push_back(i);
add_element(pos);
tuple.pop_back();
}
}
}
Thank you Jason for your thorough response!
A good way to think about forming N combinations is to look at the structure like a tree of combinations. Traversing that tree then becomes a natural way to think about the recursive nature of the algorithm you wish to implement, and how the recursive process would work.
Let's say for instance that we have the sequence, {1, 2, 3, 4}, and we wish to find all the 3-combinations in that set. The "tree" of combinations would then look like the following:
root
________|___
| |
__1_____ 2
| | |
__2__ 3 3
| | | |
3 4 4 4
Traversing from the root using a pre-order traversal, and identifying a combination when we reach a leaf-node, we get the combinations:
{1, 2, 3}
{1, 2, 4}
{1, 3, 4}
{2, 3, 4}
So basically the idea would be to sequence through an array using an index value, that for each stage of our recursion (which in this case would be the "levels" of the tree), increments into the array to obtain the value that would be included in the combination set. Also note that we only need to recurse N times. Therefore you would have some recursive function whose signature that would look something like the following:
void recursive_comb(int step_val, int array_index, std::vector<int> tuple);
where the step_val indicates how far we have to recurse, the array_index value tells us where we're at in the set to start adding values to the tuple, and the tuple, once we're complete, will be an instance of a combination in the set.
You would then need to call recursive_comb from another non-recursive function that basically "starts off" the recursive process by initializing the tuple vector and inputting the maximum recursive steps (i.e., the number of values we want in the tuple):
void init_combinations()
{
std::vector<int> tuple;
tuple.reserve(tuple_size); //avoids needless allocations
recursive_comb(tuple_size, 0, tuple);
}
Finally your recusive_comb function would something like the following:
void recursive_comb(int step_val, int array_index, std::vector<int> tuple)
{
if (step_val == 0)
{
all_combinations.push_back(tuple); //<==We have the final combination
return;
}
for (int i = array_index; i < set.size(); i++)
{
tuple.push_back(set[i]);
recursive_comb(step_val - 1, i + 1, tuple); //<== Recursive step
tuple.pop_back(); //<== The "backtrack" step
}
return;
}
You can see a working example of this code here: http://ideone.com/78jkV
Note that this is not the fastest version of the algorithm, in that we are taking some extra branches we don't need to take which create some needless copying and function calls, etc. ... but hopefully it gets across the general idea of recursion and backtracking, and how the two work together.
Personally I would go with a simple iterative solution.
Represent you set of nodes as a set of bits. If you need 5 nodes then have 5 bits, each bit representing a specific node. If you want 3 of these in your tupple then you just need to set 3 of the bits and track their location.
Basically this is a simple variation on fonding all different subsets of nodes combinations. Where the classic implementation is represent the set of nodes as an integer. Each bit in the integer represents a node. The empty set is then 0. Then you just increment the integer each new value is a new set of nodes (the bit pattern representing the set of nodes). Just in this variation you make sure that there are always 3 nodes on.
Just to help me think I start with the 3 top nodes active { 4, 3, 2 }. Then I count down. But it would be trivial to modify this to count in the other direction.
#include <boost/dynamic_bitset.hpp>
#include <iostream>
class TuppleSet
{
friend std::ostream& operator<<(std::ostream& stream, TuppleSet const& data);
boost::dynamic_bitset<> data; // represents all the different nodes
std::vector<int> bitpos; // tracks the 'n' active nodes in the tupple
public:
TuppleSet(int nodes, int activeNodes)
: data(nodes)
, bitpos(activeNodes)
{
// Set up the active nodes as the top 'activeNodes' node positions.
for(int loop = 0;loop < activeNodes;++loop)
{
bitpos[loop] = nodes-1-loop;
data[bitpos[loop]] = 1;
}
}
bool next()
{
// Move to the next combination
int bottom = shiftBits(bitpos.size()-1, 0);
// If it worked return true (otherwise false)
return bottom >= 0;
}
private:
// index is the bit we are moving. (index into bitpos)
// clearance is the number of bits below it we need to compensate for.
//
// [ 0, 1, 1, 1, 0 ] => { 3, 2, 1 }
// ^
// The bottom bit is move down 1 (index => 2, clearance => 0)
// [ 0, 1, 1, 0, 1] => { 3, 2, 0 }
// ^
// The bottom bit is moved down 1 (index => 2, clearance => 0)
// This falls of the end
// ^
// So we move the next bit down one (index => 1, clearance => 1)
// [ 0, 1, 0, 1, 1]
// ^
// The bottom bit is moved down 1 (index => 2, clearance => 0)
// This falls of the end
// ^
// So we move the next bit down one (index =>1, clearance => 1)
// This does not have enough clearance to move down (as the bottom bit would fall off)
// ^ So we move the next bit down one (index => 0, clearance => 2)
// [ 0, 0, 1, 1, 1]
int shiftBits(int index, int clerance)
{
if (index == -1)
{ return -1;
}
if (bitpos[index] > clerance)
{
--bitpos[index];
}
else
{
int nextBit = shiftBits(index-1, clerance+1);
bitpos[index] = nextBit-1;
}
return bitpos[index];
}
};
std::ostream& operator<<(std::ostream& stream, TuppleSet const& data)
{
stream << "{ ";
std::vector<int>::const_iterator loop = data.bitpos.begin();
if (loop != data.bitpos.end())
{
stream << *loop;
++loop;
for(; loop != data.bitpos.end(); ++loop)
{
stream << ", " << *loop;
}
}
stream << " }";
return stream;
}
Main is trivial:
int main()
{
TuppleSet s(5,3);
do
{
std::cout << s << "\n";
}
while(s.next());
}
Output is:
{ 4, 3, 2 }
{ 4, 3, 1 }
{ 4, 3, 0 }
{ 4, 2, 1 }
{ 4, 2, 0 }
{ 4, 1, 0 }
{ 3, 2, 1 }
{ 3, 2, 0 }
{ 3, 1, 0 }
{ 2, 1, 0 }
A version of shiftBits() using a loop
int shiftBits()
{
int bottom = -1;
for(int loop = 0;loop < bitpos.size();++loop)
{
int index = bitpos.size() - 1 - loop;
if (bitpos[index] > loop)
{
bottom = --bitpos[index];
for(int shuffle = loop-1; shuffle >= 0; --shuffle)
{
int index = bitpos.size() - 1 - shuffle;
bottom = bitpos[index] = bitpos[index-1] - 1;
}
break;
}
}
return bottom;
}
In MATLAB:
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% combinations.m
function combinations(n, k, func)
assert(n >= k);
n_set = [1:n];
k_set = zeros(k, 1);
recursive_comb(k, 1, n_set, k_set, func)
return
function recursive_comb(k_set_index, n_set_index, n_set, k_set, func)
if k_set_index == 0,
func(k_set);
return;
end;
for i = n_set_index:length(n_set)-k_set_index+1,
k_set(k_set_index) = n_set(i);
recursive_comb(k_set_index - 1, i + 1, n_set, k_set, func);
end;
return;
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Test:
>> combinations(5, 3, #(x) printf('%s\n', sprintf('%d ', x)));
3 2 1
4 2 1
5 2 1
4 3 1
5 3 1
5 4 1
4 3 2
5 3 2
5 4 2
5 4 3