Compare two unsorted std::vector [closed] - c++

Question

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 1 year ago.
Improve this question
What is the best way to compare two unsorted std::vector
std::vector<int> v1 = {1, 2, 3, 4, 5};
std::vector<int> v2 = {2, 3, 4, 5, 1};
What I am currently doing is
const auto is_similar = v1.size() == v2.size() && std::is_permutation(v1.begin(), v1.end(), v2.begin());
Here two vectors are similar only when the size of both vectors are equal and they contain the same elements
What would be a better approach for
two small std::vectors (size well under 50 elements)
two large std::vectors

Answer 1

std::is_permutation appears to be very-very slow for large arrays. Already for 64 K elements for simlar arrays it takes around 5 seconds to give answer. While regular sorting takes 0.007 seconds for this size of arrays. Timings are provided in my code below.
I suggest to do following thing - compute any simple (and fast) hash function of elements that is independent of elements order. If hash of two arrays is not equal then they are not similar, in other words two arrays as sets are not equal. Only if hashes are same then do regular sorting and compare arrays for equality.
Things suggested in my answer are meant for large arrays, to make computation fast, for tiny arrays probably std::is_permutation is enough. Although everything in this answer applies well to small arrays too.
In following code there are three functions implemented SimilarProd(), SimilarSort(), SimilarIsPermutation(), first of them uses my suggestion about first computing hash function and then sorting.
As a position-independent hash function I took regular product (multiplication) of all elements shifted (added to) by some fixed random 64-bit value. This kind of computation applied to integer arrays will be computed very fast due to good auto-vectorization capabilities of modern compilers (like CLang and GCC) which use SIMD instructions to boost computation.
In below code I did timings for all three implementations of similarity functions. It appeared that in case of similar arrays (same set of numbers) for arrays 64 K in size it takes 5 seconds for std::is_permutation(), while both hash approach and sort approach take 0.007 seconds. For unsimilar arrays std::is_permutation is very fast, below 0.00005 seconds, while sort is also 0.007 seconds and hash is 100x times faster, 0.00008 seconds.
So conclusion is that std::is_permutation is very slow for large similar arrays, and very fast for unsimilar. Sort approach is same fast speed for similar and unsimilar. While hash approach is fast for similar and blazingly fast for unsimilar. Hash approach is about the same speed as std::is_permutation for the case of unsimilar arrays, but for similar arrays is a clear win.
So out of three approaches hash approach is a clear win.
See timings below after code.
Update. For comparison just now added one more method SimilarMap(). Counting number of occurances of each integer in arrays using std::unordered_map. It appeared to be a bit slower than sorting. So still Hash+Sort method is the fastest. Although for very large arrays this map-counting method should outperform sorting speed.
Try it online!
#include <cstdint>
#include <array>
#include <vector>
#include <algorithm>
#include <random>
#include <chrono>
#include <iomanip>
#include <iostream>
#include <unordered_map>
bool SimilarProd(std::vector<int> const & a, std::vector<int> const & b) {
using std::size_t;
using u64 = uint64_t;
if (a.size() != b.size())
return false;
u64 constexpr randv = 0x6A7BE8CD0708EC4CULL;
size_t constexpr num_words = 8;
std::array<u64, num_words> prodA = {}, prodB = {};
std::fill(prodA.begin(), prodA.end(), 1);
std::fill(prodB.begin(), prodB.end(), 1);
for (size_t i = 0; i < a.size() - a.size() % num_words; i += num_words)
for (size_t j = 0; j < num_words; ++j) {
prodA[j] *= (randv + u64(a[i + j])) | 1;
prodB[j] *= (randv + u64(b[i + j])) | 1;
}
for (size_t i = a.size() - a.size() % num_words; i < a.size(); ++i) {
prodA[0] *= (randv + u64(a[i])) | 1;
prodB[0] *= (randv + u64(b[i])) | 1;
}
for (size_t i = 1; i < num_words; ++i) {
prodA[0] *= prodA[i];
prodB[0] *= prodB[i];
}
if (prodA[0] != prodB[0])
return false;
auto a2 = a, b2 = b;
std::sort(a2.begin(), a2.end());
std::sort(b2.begin(), b2.end());
return a2 == b2;
}
bool SimilarSort(std::vector<int> a, std::vector<int> b) {
if (a.size() != b.size())
return false;
std::sort(a.begin(), a.end());
std::sort(b.begin(), b.end());
return a == b;
}
bool SimilarIsPermutation(std::vector<int> const & a, std::vector<int> const & b) {
return a.size() == b.size() && std::is_permutation(a.begin(), a.end(), b.begin());
}
bool SimilarMap(std::vector<int> const & a, std::vector<int> const & b) {
if (a.size() != b.size())
return false;
std::unordered_map<int, int> m;
for (auto x: a)
++m[x];
for (auto x: b)
--m[x];
for (auto const & p: m)
if (p.second != 0)
return false;
return true;
}
void Test() {
using std::size_t;
auto TimeCur = []{ return std::chrono::high_resolution_clock::now(); };
auto const gtb = TimeCur();
auto Time = [&]{ return std::chrono::duration_cast<
std::chrono::microseconds>(TimeCur() - gtb).count() / 1000000.0; };
std::mt19937_64 rng{123};
auto RandV = [&](size_t n) {
std::vector<int> v(n);
for (size_t i = 0; i < v.size(); ++i)
v[i] = rng() % (1 << 30);
return v;
};
size_t constexpr n = 1 << 16;
auto a = RandV(n), b = a, c = RandV(n);
std::shuffle(b.begin(), b.end(), rng);
std::cout << std::boolalpha << std::fixed;
double tb = 0;
tb = Time(); std::cout << "Prod "
<< SimilarProd(a, b) << " time " << (Time() - tb) << std::endl;
tb = Time(); std::cout << "Sort "
<< SimilarSort(a, b) << " time " << (Time() - tb) << std::endl;
tb = Time(); std::cout << "IsPermutation "
<< SimilarIsPermutation(a, b) << " time " << (Time() - tb) << std::endl;
tb = Time(); std::cout << "Map "
<< SimilarMap(a, b) << " time " << (Time() - tb) << std::endl;
tb = Time(); std::cout << "Prod "
<< SimilarProd(a, c) << " time " << (Time() - tb) << std::endl;
tb = Time(); std::cout << "Sort "
<< SimilarSort(a, c) << " time " << (Time() - tb) << std::endl;
tb = Time(); std::cout << "IsPermutation "
<< SimilarIsPermutation(a, c) << " time " << (Time() - tb) << std::endl;
tb = Time(); std::cout << "Map "
<< SimilarMap(a, c) << " time " << (Time() - tb) << std::endl;
}
int main() {
Test();
}
Output:
Prod true time 0.009208
Sort true time 0.008080
IsPermutation true time 4.436632
Map true time 0.010382
Prod false time 0.000082
Sort false time 0.008750
IsPermutation false time 0.000036
Map false time 0.016390

Answer 2

What would be a better approach
Remove the v1.size() == v2.size() && expression and instead pass end iterator to std::is_permutation.
You tagged C++11, but to those who can use C++20, I recommend following:
std::ranges::is_permutation(v1, v2)
If you can modify the vectors, then it will be asymptotically faster to sort them and compare equality. If you cannot modify, then you could create a sorted copy if you can afford the storage cost.