Is gcc std::unordered_map implementation slow? If so - why? - c++

We are developing a highly performance critical software in C++. There we need a concurrent hash map and implemented one. So we wrote a benchmark to figure out, how much slower our concurrent hash map is compared with std::unordered_map.
But, std::unordered_map seems to be incredibly slow... So this is our micro-benchmark (for the concurrent map we spawned a new thread to make sure that locking does not get optimized away and note that I never inser 0 because I also benchmark with google::dense_hash_map, which needs a null value):
boost::random::mt19937 rng;
boost::random::uniform_int_distribution<> dist(std::numeric_limits<uint64_t>::min(), std::numeric_limits<uint64_t>::max());
std::vector<uint64_t> vec(SIZE);
for (int i = 0; i < SIZE; ++i) {
uint64_t val = 0;
while (val == 0) {
val = dist(rng);
}
vec[i] = val;
}
std::unordered_map<int, long double> map;
auto begin = std::chrono::high_resolution_clock::now();
for (int i = 0; i < SIZE; ++i) {
map[vec[i]] = 0.0;
}
auto end = std::chrono::high_resolution_clock::now();
auto elapsed = std::chrono::duration_cast<std::chrono::milliseconds>(end - begin);
std::cout << "inserts: " << elapsed.count() << std::endl;
std::random_shuffle(vec.begin(), vec.end());
begin = std::chrono::high_resolution_clock::now();
long double val;
for (int i = 0; i < SIZE; ++i) {
val = map[vec[i]];
}
end = std::chrono::high_resolution_clock::now();
elapsed = std::chrono::duration_cast<std::chrono::milliseconds>(end - begin);
std::cout << "get: " << elapsed.count() << std::endl;
(EDIT: the whole source code can be found here: http://pastebin.com/vPqf7eya)
The result for std::unordered_map is:
inserts: 35126
get : 2959
For google::dense_map:
inserts: 3653
get : 816
For our hand backed concurrent map (which does locking, although the benchmark is single threaded - but in a separate spawn thread):
inserts: 5213
get : 2594
If I compile the benchmark program without pthread support and run everything in the main thread, I get the following results for our hand backed concurrent map:
inserts: 4441
get : 1180
I compile with the following command:
g++-4.7 -O3 -DNDEBUG -I/tmp/benchmap/sparsehash-2.0.2/src/ -std=c++11 -pthread main.cc
So especially inserts on std::unordered_map seem to be extremely expensive - 35 seconds vs 3-5 seconds for other maps. Also the lookup time seems to be quite high.
My question: why is this? I read another question on stackoverflow where someone asks, why std::tr1::unordered_map is slower than his own implementation. There the highest rated answer states, that the std::tr1::unordered_map needs to implement a more complicated interface. But I can not see this argument: we use a bucket approach in our concurrent_map, std::unordered_map uses a bucket-approach too (google::dense_hash_map does not, but than std::unordered_map should be at least as fast than our hand backed concurrency-safe version?). Apart from that I can not see anything in the interface that force a feature which makes the hash map perform badly...
So my question: is it true that std::unordered_map seems to be very slow? If no: what is wrong? If yes: what is the reason for that.
And my main question: why is inserting a value into a std::unordered_map so terrible expensive (even if we reserve enough space at the beginning, it does not perform much better - so rehashing seems not to be the problem)?
EDIT:
First of all: yes the presented benchmark is not flawless - this is because we played around a lot with it and it is just a hack (for example the uint64 distribution to generate ints would in practice not be a good idea, exclude 0 in a loop is kind of stupid etc...).
At the moment most comments explain, that I can make the unordered_map faster by preallocating enough space for it. In our application this is just not possible: we are developing a database management system and need a hash map to store some data during a transaction (for example locking information). So this map can be everything from 1 (user just makes one insert and commits) to billions of entries (if full table scans happen). It is just impossible to preallocate enough space here (and just allocate a lot in the beginning will consume too much memory).
Furthermore, I apologize, that I did not state my question clear enough: I am not really interested in making unordered_map fast (using googles dense hash map works fine for us), I just don't really understand where this huge performance differences come from. It can not be just preallocation (even with enough preallocated memory, the dense map is an order of magnitude faster than unordered_map, our hand backed concurrent map starts with an array of size 64 - so a smaller one than unordered_map).
So what is the reason for this bad performance of std::unordered_map? Or differently asked: Could one write an implementation of the std::unordered_map interface which is standard conform and (nearly) as fast as googles dense hash map? Or is there something in the standard that enforces the implementer to chose an inefficient way to implement it?
EDIT 2:
By profiling I see that a lot of time is used for integer divions. std::unordered_map uses prime numbers for the array size, while the other implementations use powers of two. Why does std::unordered_map use prime-numbers? To perform better if the hash is bad? For good hashes it does imho make no difference.
EDIT 3:
These are the numbers for std::map:
inserts: 16462
get : 16978
Sooooooo: why are inserts into a std::map faster than inserts into a std::unordered_map... I mean WAT? std::map has a worse locality (tree vs array), needs to make more allocations (per insert vs per rehash + plus ~1 for each collision) and, most important: has another algorithmic complexity (O(logn) vs O(1))!

I found the reason: it is a Problem of gcc-4.7!!
With gcc-4.7
inserts: 37728
get : 2985
With gcc-4.6
inserts: 2531
get : 1565
So std::unordered_map in gcc-4.7 is broken (or my installation, which is an installation of gcc-4.7.0 on Ubuntu - and another installation which is gcc 4.7.1 on debian testing).
I will submit a bug report.. until then: DO NOT use std::unordered_map with gcc 4.7!

I am guessing that you have not properly sized your unordered_map, as Ylisar suggested. When chains grow too long in unordered_map, the g++ implementation will automatically rehash to a larger hash table, and this would be a big drag on performance. If I remember correctly, unordered_map defaults to (smallest prime larger than) 100.
I didn't have chrono on my system, so I timed with times().
template <typename TEST>
void time_test (TEST t, const char *m) {
struct tms start;
struct tms finish;
long ticks_per_second;
times(&start);
t();
times(&finish);
ticks_per_second = sysconf(_SC_CLK_TCK);
std::cout << "elapsed: "
<< ((finish.tms_utime - start.tms_utime
+ finish.tms_stime - start.tms_stime)
/ (1.0 * ticks_per_second))
<< " " << m << std::endl;
}
I used a SIZE of 10000000, and had to change things a bit for my version of boost. Also note, I pre-sized the hash table to match SIZE/DEPTH, where DEPTH is an estimate of the length of the bucket chain due to hash collisions.
Edit: Howard points out to me in comments that the max load factor for unordered_map is 1. So, the DEPTH controls how many times the code will rehash.
#define SIZE 10000000
#define DEPTH 3
std::vector<uint64_t> vec(SIZE);
boost::mt19937 rng;
boost::uniform_int<uint64_t> dist(std::numeric_limits<uint64_t>::min(),
std::numeric_limits<uint64_t>::max());
std::unordered_map<int, long double> map(SIZE/DEPTH);
void
test_insert () {
for (int i = 0; i < SIZE; ++i) {
map[vec[i]] = 0.0;
}
}
void
test_get () {
long double val;
for (int i = 0; i < SIZE; ++i) {
val = map[vec[i]];
}
}
int main () {
for (int i = 0; i < SIZE; ++i) {
uint64_t val = 0;
while (val == 0) {
val = dist(rng);
}
vec[i] = val;
}
time_test(test_insert, "inserts");
std::random_shuffle(vec.begin(), vec.end());
time_test(test_insert, "get");
}
Edit:
I modified the code so that I could change out DEPTH more easily.
#ifndef DEPTH
#define DEPTH 10000000
#endif
So, by default, the worst size for the hash table is chosen.
elapsed: 7.12 inserts, elapsed: 2.32 get, -DDEPTH=10000000
elapsed: 6.99 inserts, elapsed: 2.58 get, -DDEPTH=1000000
elapsed: 8.94 inserts, elapsed: 2.18 get, -DDEPTH=100000
elapsed: 5.23 inserts, elapsed: 2.41 get, -DDEPTH=10000
elapsed: 5.35 inserts, elapsed: 2.55 get, -DDEPTH=1000
elapsed: 6.29 inserts, elapsed: 2.05 get, -DDEPTH=100
elapsed: 6.76 inserts, elapsed: 2.03 get, -DDEPTH=10
elapsed: 2.86 inserts, elapsed: 2.29 get, -DDEPTH=1
My conclusion is that there is not much significant performance difference for any initial hash table size other than making it equal to the entire expected number of unique insertions. Also, I don't see the order of magnitude performance difference that you are observing.

I have run your code using a 64 bit / AMD / 4 cores (2.1GHz) computer and it gave me the following results:
MinGW-W64 4.9.2:
Using std::unordered_map:
inserts: 9280
get: 3302
Using std::map:
inserts: 23946
get: 24824
VC 2015 with all the optimization flags I know:
Using std::unordered_map:
inserts: 7289
get: 1908
Using std::map:
inserts: 19222
get: 19711
I have not tested the code using GCC but I think it may be comparable to the performance of VC, so if that is true, then GCC 4.9 std::unordered_map it's still broken.
[EDIT]
So yes, as someone said in the comments, there is no reason to think that the performance of GCC 4.9.x would be comparable to VC performance.
When I have the change I will be testing the code on GCC.
My answer is just to establish some kind of knowledge base to other answers.

Related

Using std::async slower than non-async method to populate a vector

I am experimenting with std::async to populate a vector. The idea behind it is to use multi-threading to save time. However, running some benchmark tests I find that my non-async method is faster!
#include <algorithm>
#include <vector>
#include <future>
std::vector<int> Generate(int i)
{
std::vector<int> v;
for (int j = i; j < i + 10; ++j)
{
v.push_back(j);
}
return v;
}
Async:
std::vector<std::future<std::vector<int>>> futures;
for (int i = 0; i < 200; i+=10)
{
futures.push_back(std::async(
[](int i) { return Generate(i); }, i));
}
std::vector<int> res;
for (auto &&f : futures)
{
auto vec = f.get();
res.insert(std::end(res), std::begin(vec), std::end(vec));
}
Non-async:
std::vector<int> res;
for (int i = 0; i < 200; i+=10)
{
auto vec = Generate(i);
res.insert(std::end(res), std::begin(vec), std::end(vec));
}
My benchmark test shows that the async method is 71 times slower than non-async. What am I doing wrong?
std::async has two modes of operation:
std::launch::async
std::launch::deferred
In this case, you've called std::async without specifying either one, which means it's allowed to choose either one. std::launch::deferred basically means do the work on the calling thread. So std::async returns a future, and with std::launch::deferred, the action you've requested won't be carried out until you call .get on that future. It can be kind of handy under a few circumstances, but it's probably not what you want here.
Even if you specify std::launch::async, you need to realize that this starts up a new thread of execution to carry out the action you've requested. It then has to create a future, and use some sort of signalling from the thread to the future to let you know when the computation you've requested is done.
All of that adds a fair amount of overhead--anywhere from microseconds to milliseconds or so, depending on the OS, CPU, etc.
So, for asynchronous execution to make sense, the "stuff" you do asynchronously typically needs to take tens of milliseconds at the very least (and hundreds of milliseconds might be a more sensible lower threshold). I wouldn't get too wrapped up in the exact cutoff, but it needs to be something that takes a while.
So, filling an array asynchronously probably only makes sense if the array is quite a lot larger than you're dealing with here.
For filling memory, you'll quickly run into another problem though: most CPUs are enough faster than main memory that if all you're doing is writing to memory, there's a pretty good chance that a single thread will already saturate the path to memory, so even at best doing the job asynchronously will only gain a little, and may still pretty easily cause a slow-down.
The ideal case for asynchronous operation would be something like one thread that's heavily memory bound, but another that (for example) reads a little bit of data, and does a lot of computation on that small amount of data. In this case, the computation thread will mostly operate on its data in the cache, so it won't get in the way of the memory thread doing its thing.
There are multiple factors that are causing the Multithreaded code to perform (much) slower than the Singlethreaded code.
Your array sizes are too small
Multithreading often has negligible-to-no effect on datasets that are particularly small. In both versions of your code, you're generating 2000 integers, and each Logical Thread (which, because std::async is often implemented in terms of thread pools, might not be the same as a Software Thread) is only generating 10 integers. The cost of spooling up a thread every 10 integers way offsets the benefit of generating those integers in parallel.
You might see a performance gain if each thread were instead responsible for, say, 10,000 integers each, but you'll probably instead have a different issue:
All your code is bottlenecked by an inherently serial process
Both versions of the code copy the generated integers into a host vector. It would be one thing if the act of generating those integers was itself a time consuming process, but in your case, it's likely just a matter of a small, fast bit of assembly generating each integer.
So the act of copying each integer into the final vector is probably not inherently faster than generating each integer, meaning a sizable chunk of the "work" being done is completely serial, defeating the whole purpose of multithreading your code.
Fixing the code
Compilers are very good at their jobs, so in trying to revise your code, I was only barely able to get multithreaded code that was faster than the serial code. Multiple executions had varying results, so my general assessment is that this kind of code is bad at being multithreaded.
But here's what I came up with:
#include <algorithm>
#include <vector>
#include <future>
#include<chrono>
#include<iostream>
#include<iomanip>
//#1: Constants
constexpr int BLOCK_SIZE = 500000;
constexpr int NUM_OF_BLOCKS = 20;
std::vector<int> Generate(int i) {
std::vector<int> v;
for (int j = i; j < i + BLOCK_SIZE; ++j) {
v.push_back(j);
}
return v;
}
void asynchronous_attempt() {
std::vector<std::future<void>> futures;
//#2: Preallocated Vector
std::vector<int> res(NUM_OF_BLOCKS * BLOCK_SIZE);
auto it = res.begin();
for (int i = 0; i < NUM_OF_BLOCKS * BLOCK_SIZE; i+=BLOCK_SIZE)
{
futures.push_back(std::async(
[it](int i) {
auto vec = Generate(i);
//#3 Copying done multithreaded
std::copy(vec.begin(), vec.end(), it + i);
}, i));
}
for (auto &&f : futures) {
f.get();
}
}
void serial_attempt() {
//#4 Changes here to show fair comparison
std::vector<int> res(NUM_OF_BLOCKS * BLOCK_SIZE);
auto it = res.begin();
for (int i = 0; i < NUM_OF_BLOCKS * BLOCK_SIZE; i+=BLOCK_SIZE) {
auto vec = Generate(i);
it = std::copy(vec.begin(), vec.end(), it);
}
}
int main() {
using clock = std::chrono::steady_clock;
std::cout << "Theoretical # of Threads: " << std::thread::hardware_concurrency() << std::endl;
auto begin = clock::now();
asynchronous_attempt();
auto end = clock::now();
std::cout << "Duration of Multithreaded Attempt: " << std::setw(10) << (end - begin).count() << "ns" << std::endl;
begin = clock::now();
serial_attempt();
end = clock::now();
std::cout << "Duration of Serial Attempt: " << std::setw(10) << (end - begin).count() << "ns" << std::endl;
}
This resulted in the following output:
Theoretical # of Threads: 2
Duration of Multithreaded Attempt: 361149213ns
Duration of Serial Attempt: 364785676ns
Given that this was on an online compiler (here) I'm willing to bet the multithreaded code might win out on a dedicated machine, but I think this at least demonstrates the improvement in performance that we're at least on par between the two methods.
Below are the changes I made, that are ID'd in the code:
We've dramatically increased the number of integers being generated, to force the threads to do actual meaningful work, instead of getting bogged down on OS-level housekeeping
The vector has its size pre-allocated. No more frequent resizing.
Now that the space has been preallocated, we can multithread the copying instead of doing it in serial later.
We have to change the serial code so it also preallocates + copies so that it's a fair comparison.
Now, we've ensured that all the code is indeed running in parallel, and while it's not amounting to a substantial improvement over the serial code, it's at least no longer exhibiting the degenerate performance losses we were seeing before.
First of all, you are not forcing the std::async to work asynchronously (you would need to specify std::launch::async policy to do so). Second of all, it'd be kind of an overkill to asynchronously create an std::vector of 10 ints. It's just not worth it. Remember - using more threads does not mean that you will see performance benefit! Creating a thread (or even using a threadpool) introduces some overhead, which, in this case, seems to dwarf the benefits of running tasks asynchronously.
Thanks #NathanOliver ;>

prime number below 2 billion - usage of std::list hinders performance

Problem Statement is to find prime number below 2 billion in timeframe < 20 sec.
I followed below approaches.
Divide the number n with list of number k ( k < sqrt(n)) - took 20 sec
Divide the number n with list of prime number below sqrt(n).In this scenario I stored prime numbers in std::list - took more than 180 sec
Can someone help me understand why did 2nd approach take longtime even though we reduced no of divisions by 50%(approx)? or Did I choose wrong Data Structure?
Approach 1:
#include <iostream>
#include<list>
#include <ctime>
using namespace std;
list<long long> primeno;
void ListPrimeNumber();
int main()
{
clock_t time_req = clock();
ListPrimeNumber();
time_req = clock() - time_req;
cout << "time taken " << static_cast<float>(time_req) / CLOCKS_PER_SEC << " seconds" << endl;
return 0;
}
void check_prime(int i);
void ListPrimeNumber()
{
primeno.push_back(2);
primeno.push_back(3);
primeno.push_back(5);
for (long long i = 6; i <= 20000000; i++)
{
check_prime(i);
}
}
void check_prime(int i)
{
try
{
int j = 0;
int limit = sqrt(i);
for (j = 2 ; j <= limit;j++)
{
if(i % j == 0)
{
break;
}
}
if( j > limit)
{
primeno.push_back(i);
}
}
catch (exception ex)
{
std::cout << "Message";
}
}
Approach 2 :
#include <iostream>
#include<list>
#include <ctime>
using namespace std;
list<long long> primeno;
int noofdiv = 0;
void ListPrimeNumber();
int main()
{
clock_t time_req = clock();
ListPrimeNumber();
time_req = clock() - time_req;
cout << "time taken " << static_cast<float>(time_req) / CLOCKS_PER_SEC << " seconds" << endl;
cout << "No of divisions : " << noofdiv;
return 0;
}
void check_prime(int i);
void ListPrimeNumber()
{
primeno.push_back(2);
primeno.push_back(3);
primeno.push_back(5);
for (long long i = 6; i <= 10000; i++)
{
check_prime(i);
}
}
void check_prime(int i)
{
try
{
int limit = sqrt(i);
for (int iter : primeno)
{
noofdiv++;
if (iter <= limit && (i%iter) == 0)
{
break;
}
else if (iter > limit)
{
primeno.push_back(i);
break;
}
}
}
catch (exception ex)
{
std::cout << "Message";
}
}
The reason your second example takes longer is you're iterating a std::list.
A std::list in C++ is a linked list, which means it doesn't use contiguous memory. This is bad because to iterate the list you must jump from node to node in a (to the CPU/prefetcher) unpredictable way. Also, You're most likely only "using" a few bytes of each cacheline. RAM is slow. Fetching a byte from RAM takes a lot longer than fetching it from L1. CPUs are fast these days, so your program is most of the time not doing anything and waiting for memory to arrive.
Use a std::vector instead. It stores all values one after the other and iterating is very cheap. Since you're iterating forward in memory without jumping, you're using the full cacheline and your prefetcher will be able to fetch further pages before you need them because your access of memory is predictable.
It has been proven by numerous people, including Bjarne Stroustrup, that std::vector is in a lot of cases faster than std::list, even in cases where the std::list has "theoretically" better complexity (random insert, delete, ...) just because caching helps a lot. So always use std::vector as your default. And if you think a linked list would be faster in your case, measure it and be surprised that - most of the time - std::vector dominates.
Edit: as others have noted, your method of finding primes isn't very efficient. I just played around a bit and implemented a Sieve of Eratosthenes using a bitset.
constexpr int max_prime = 1000000000;
std::bitset<max_prime> *bitset = new std::bitset<max_prime>{};
// Note: Bit SET means NO prime
bitset->set(0);
bitset->set(1);
for(int i = 4; i < max_prime ; i += 2)
bitset->set(i); // set all even numbers
int max = sqrt(max_prime);
for(int i = 3; i < max; i += 2) { // No point testing even numbers as they can't be prime
if(!bitset->test(i)) { // If i is prime
for(int j = i * 2; j < no_primes; j += i)
bitset->set(j); // set all multiples of i to non-prime
}
}
This takes between 4.2 and 4.5 seconds 30 seconds (not sure why it changed that much after slight modifications... must be an optimization I'm not hitting anymore) to find all primes below one Billion (1,000,000,000) on my machine. Your approach took way too long even for 100 million. I cancelled the 1 Billion search after about two minutes.
Comparison for 100 million:
time taken: 63.515 seconds
time taken bitset: 1.874 seconds
No of divisions : 1975961174
No of primes found: 5761455
No of primes found bitset: 5761455
I'm not a mathematician so I'm pretty sure there are still ways to optimize it further, I only optimize for even numbers.
The first thing to do is make sure you are compiling with optimisations enabled. The c++ standard library template classes tend to perform very poorly with unoptimised code as they generate lots of function calls. The optimiser inlines most of these function calls which makes them much cheaper.
std::list is a linked list. Its is mostly useful where you want to insert or remove elements randomly (i.e. not from the end).
For the case where you are only appending to the end of a list std::list has the following issues:
Iterating through the list is relatively expensive as the code has to follow node pointers and then retrieve the data
The list uses quite a lot more memory, each element needs a pointer to the previous and next nodes in addition to the actual data. On a 64-bit system this equates to 20 bytes per element rather than 4 for a list of int
As the elements in the list are not contiguous in memory the compiler can't perform as many SIMD optimisations and you will suffer more from CPU cache misses
A std::vector would solve all of the above as its memory is contiguous and iterating through it is basically just a case of incrementing an array index. You do need to make sure that you call reserve on your vector at the beginning with a sufficiently large value so that appending to the vector doesn't cause the whole array to be copied to a new larger array.
A bigger optimisation than the above would be to use the Sieve of Eratosthenes to calculate your primes. As generating this light require random deletions (depending on your exact implementation) std::list might perform better than std::vector though even in this case the overheads of std::list might not outweigh its costs.
A test at Ideone (the OP code with few superficial alterations) completely contradicts the claims made in this question:
/* check_prime__list:
time taken No of divisions No of primes
10M: 0.873 seconds 286144936 664579
20M: 2.169 seconds 721544444 1270607 */
2B: projected time: at least 16 minutes but likely much more (*)
/* check_prime__nums:
time taken No of divisions No of primes
10M: 4.650 seconds 1746210131 664579
20M: 12.585 seconds 4677014576 1270607 */
2B: projected time: at least 3 hours but likely much more (*)
I also changed the type of the number of divisions counter to long int because it was wrapping around the data type limit. So they could have been misinterpreting that.
But the run time wasn't being affected by that. A wall clock is a wall clock.
Most likely explanation seems to be a sloppy testing by the OP, with different values used in each test case, by mistake.
(*) The time projection was made by the empirical orders of growth analysis:
100**1.32 * 2.169 / 60 = 15.8
100**1.45 * 12.585 / 3600 = 2.8
Empirical orders of growth, as measured on the given range of sizes, were noticeably better for the list algorithm, n1.32 vs. the n1.45 for the testing by all numbers. This is entirely expected from theoretical complexity, since there are fewer primes than all numbers up to n, by a factor of log n, for a total complexity of O(n1.5/log n) vs. O(n1.5). It is also highly unlikely for any implementational discrepancy to beat an actual algorithmic advantage.

How to zero a vector<bool>?

I have a vector<bool> and I'd like to zero it out. I need the size to stay the same.
The normal approach is to iterate over all the elements and reset them. However, vector<bool> is a specially optimized container that, depending on implementation, may store only one bit per element. Is there a way to take advantage of this to clear the whole thing efficiently?
bitset, the fixed-length variant, has the set function. Does vector<bool> have something similar?
There seem to be a lot of guesses but very few facts in the answers that have been posted so far, so perhaps it would be worthwhile to do a little testing.
#include <vector>
#include <iostream>
#include <time.h>
int seed(std::vector<bool> &b) {
srand(1);
for (int i = 0; i < b.size(); i++)
b[i] = ((rand() & 1) != 0);
int count = 0;
for (int i = 0; i < b.size(); i++)
if (b[i])
++count;
return count;
}
int main() {
std::vector<bool> bools(1024 * 1024 * 32);
int count1= seed(bools);
clock_t start = clock();
bools.assign(bools.size(), false);
double using_assign = double(clock() - start) / CLOCKS_PER_SEC;
int count2 = seed(bools);
start = clock();
for (int i = 0; i < bools.size(); i++)
bools[i] = false;
double using_loop = double(clock() - start) / CLOCKS_PER_SEC;
int count3 = seed(bools);
start = clock();
size_t size = bools.size();
bools.clear();
bools.resize(size);
double using_clear = double(clock() - start) / CLOCKS_PER_SEC;
int count4 = seed(bools);
start = clock();
std::fill(bools.begin(), bools.end(), false);
double using_fill = double(clock() - start) / CLOCKS_PER_SEC;
std::cout << "Time using assign: " << using_assign << "\n";
std::cout << "Time using loop: " << using_loop << "\n";
std::cout << "Time using clear: " << using_clear << "\n";
std::cout << "Time using fill: " << using_fill << "\n";
std::cout << "Ignore: " << count1 << "\t" << count2 << "\t" << count3 << "\t" << count4 << "\n";
}
So this creates a vector, sets some randomly selected bits in it, counts them, and clears them (and repeats). The setting/counting/printing is done to ensure that even with aggressive optimization, the compiler can't/won't optimize out our code to clear the vector.
I found the results interesting, to say the least. First the result with VC++:
Time using assign: 0.141
Time using loop: 0.068
Time using clear: 0.141
Time using fill: 0.087
Ignore: 16777216 16777216 16777216 16777216
So, with VC++, the fastest method is what you'd probably initially think of as the most naive -- a loop that assigns to each individual item. With g++, the results are just a tad different though:
Time using assign: 0.002
Time using loop: 0.08
Time using clear: 0.002
Time using fill: 0.001
Ignore: 16777216 16777216 16777216 16777216
Here, the loop is (by far) the slowest method (and the others are basically tied -- the 1 ms difference in speed isn't really repeatable).
For what it's worth, in spite of this part of the test showing up as much faster with g++, the overall times were within 1% of each other (4.944 seconds for VC++, 4.915 seconds for g++).
Try
v.assign(v.size(), false);
Have a look at this link:
http://www.cplusplus.com/reference/vector/vector/assign/
Or the following
std::fill(v.begin(), v.end(), 0)
You are out of luck. std::vector<bool> is a specialization that apparently does not even guarantee contiguous memory or random access iterators (or even forward?!), at least based on my reading of cppreference -- decoding the standard would be the next step.
So write implementation specific code, pray and use some standard zeroing technique, or do not use the type. I vote 3.
The recieved wisdom is that it was a mistake, and may become deprecated. Use a different container if possible. And definitely do not mess around with the internal guts, or rely on its packing. Check if you have dynamic bitset in your std library mayhap, or roll your own wrapper around std::vector<unsigned char>.
I ran into this as a performance issue recently. I hadn't tried looking for answers on the web but did find that using assignment with the constructor was 10x faster using g++ O3 (Debian 4.7.2-5) 4.7.2. I found this question because I was looking to avoid the additional malloc. Looks like the assign is optimized as well as the constructor and about twice as good in my benchmark.
unsigned sz = v.size(); for (unsigned ii = 0; ii != sz; ++ii) v[ii] = false;
v = std::vector(sz, false); // 10x faster
v.assign(sz, false); > // 20x faster
So, I wouldn't say to shy away from using the specialization of vector<bool>; just be very cognizant of the bit vector representation.
Use the std::vector<bool>::assign method, which is provided for this purpose.
If an implementation is specific for bool, then assign, most likely, also implemented appropriately.
If you're able to switch from vector<bool> to a custom bit vector representation, then you can use a representation designed specifically for fast clear operations, and get some potentially quite significant speedups (although not without tradeoffs).
The trick is to use integers per bit vector entry and a single 'rolling threshold' value that determines which entries actually then evaluate to true.
You can then clear the bit vector by just increasing the single threshold value, without touching the rest of the data (until the threshold overflows).
A more complete write up about this, and some example code, can be found here.
It seems that one nice option hasn't been mentioned yet:
auto size = v.size();
v.resize(0);
v.resize(size);
The STL implementer will supposedly have picked the most efficient means of zeroising, so we don't even need to know which particular method that might be. And this works with real vectors as well (think templates), not just the std::vector<bool> monstrosity.
There can be a minuscule added advantage for reused buffers in loops (e.g. sieves, whatever), where you simply resize to whatever will be needed for the current round, instead of to the original size.
As an alternative to std::vector<bool>, check out boost::dynamic_bitset (https://www.boost.org/doc/libs/1_72_0/libs/dynamic_bitset/dynamic_bitset.html). You can zero one (ie, set each element to false) out by calling the reset() member function.
Like clearing, say, std::vector<int>, reset on a boost::dynamic_bitset can also compile down to a memset, whereas you probably won't get that with std::vector<bool>. For example, see https://godbolt.org/z/aqSGCi

Efficiency of vector index access vs iterator access

I have question to correct my understanding of efficiency of accessing elements of a vector by using index access (with operator []) or using an iterator.
My understanding is "iterator" is more efficient than "index access".
(also I think vector::end() is more efficient than vector::size()).
Now I wrote sample code measure it (under Windows 7 using Cygwin, with g++ 4.5.3)
The index access loop version (formerly labeled as random access):
int main()
{
std::vector< size_t > vec ( 10000000 );
size_t value = 0;
for( size_t x=0; x<10; ++x )
{
for ( size_t idx = 0; idx < vec.size(); ++idx )
{
value += vec[idx];
}
return value;
}
}
The iterator loop code is this:
for (std::vector< size_t >::iterator iter = vec.begin(); iter != vec.end(); ++iter) {
value = *iter;
}
I am surprised to see that the "index access" version is much quicker. I used the time command to "measure". The numbers were :
results using g++ source.cpp (no optimizations)
index access
real 800ms
iterator access
real 2200ms
Do these numbers make sense? (I repeated the runs multiple times) And I wondered what details I miss and why I am mistaken...
results using g++ -O2
index access, time real: ~200ms
iterator access, time real: ~200ms
I repeated tests on different platform (amd64 w/ g++ and power7 w xlC) and see that all the time I used optimized code the example programs have similar execution time.
edit changed code to add values ( value += *iter ) instead of just using assignment. Added details about compiler options. Added new numbers for using -O2.
*edit2 changed title correcting "iterator efficiency" to "accesses efficiency".
Without seeing the test harnesses, the compiler options, and how you
measured the time, it's hard to say anything. Also, a good compiler may
be able eliminate the loop in one case or the other, since the loop has
no effect on the value returned. Still, depending on the
implementation, it wouldn't surprise me to see iterators significantly
faster than indexing (or vice versa).
With regards to your "understanding", there's nothing inherent about the
type of iterator and its performance. You can write forward iterators
which are very fast, or very slow, just as you can write random access
iterators which are very fast or very slow. Globally, the types of data
structures which will support random access iterators are likely to have
better locality than those which don't, which might work in favor of
random access iterators; but that's really not enough to be able to make
any reasonable generalizations.
When I compile both programs with -O2 (Linux, GCC 4.6.1), they run equally fast.
Then: your first program is not using iterators, it is using indices. These are different concepts.
Your second program is in fact using random access iterators, because that is what std::vector<T>::iterators are. The restrictions on std::vector are designed in such a way that an iterator can be implemented as a simple pointer into the dynamic array that a vector encapsulates.
begin should be just as fast as size. The only difference between the two in a typical implementation of std::vector is that end might need to compute begin() + size(), though size might also be implemented as (roughly) end() - begin(). The compiler might optimize both away in the loop, though.
With optimizations the two codes should be (near) identical. Try -O2.
Without optimizations and added debug information your measurements will be quite misleading.
In your first example, you dereference each individual item using value = vec[idx];, which causes an offset of element_size * index to be calculated each time you access an element.
Since a vector consists of elements lined up in a continuous block of memory, a vector iterator is usually just implemented as a simple pointer, so iterating through a vector (like in your second example) just involves advancing the pointer one element after each iteration.
If you enable optimizations (try -O2 or -O3), however, the compiler will likely optimize your loop in the first example to something similar to the second example, making the performance nearly identical.
I have found iterators to be faster, actually. Try refactoring your iterator loop to something like the following and you may see this as well:
#include <ctime>
#include <vector>
#include <iostream>
using namespace std;
int main()
{
std::vector< size_t > vec ( 1000000 );
size_t value = 0;
srand ( time(NULL) );
clock_t start,stop;
int ncycle = 10000;
start = std::clock();
for( size_t x=0; x<ncycle; ++x ) {
for ( size_t idx = 0; idx < vec.size(); ++idx )
vec[idx] += rand();
}
stop = std::clock();
cout << start << " " << stop << endl;
cout << "INDEX: " << (double((stop - start)) / CLOCKS_PER_SEC) / ncycle << " seconds per cycle" << endl;
start = std::clock();
for( size_t x=0; x<ncycle; ++x ) {
for (std::vector< size_t >::iterator iter = vec.begin(), end = vec.end(); iter != end; ++iter)
*iter += rand();
}
stop = std::clock();
cout << "ITERATOR: " << (double((stop - start)) / CLOCKS_PER_SEC) / ncycle << " seconds per cycle" << endl;
}
The result is the following on my pc, showing that iterators have a slight lead:
INDEX: 0.012069 seconds per cycle
ITERATOR: 0.011482 seconds per cycle
You should note that I used an addition of rand(); this prevents the compiler from optimizing out something that it can calculate at compile time. Compilers seem to have a much easier time doing so with intrinsic arrays than with vectors, and that can misleadingly give arrays an advantage over vectors.
I compiled the above with "icpc -fast". slavik was right on about having to do calculations on indices vs incrementing when using iterators (ala pointers).

When does a map become better than two vectors?

A map does binary search on all its elements, which has logarithmic complexity — this means that for a small enough collection of objects, a map will perform worse than two vectors that have linear search.
How large should the object (key) pool be for a map to start performing better than two vectors?
Edit: A more generalized version of the question: how large should the object pool be for binary search to perform better than linear search?
I'm using strings as keys and the values are pointers, but my particular use case probably shouldn't matter. I'm more curious to understand how to use the two tools properly.
If you'll forgive my saying so, most of the answers sound to me like various ways of saying: "I don't know", without actually admitting that they don't know. While I generally agree with the advice they've given, none of them seems to have attempted to directly address the question you asked: what is the break-even point.
To be fair, when I read the question, I didn't really know either. It's one of those things what we all know the basics: for a small enough collection, a linear search will probably be faster, and for a large enough collection, a binary search will undoubtedly be faster. I, however, have never really had much reason to investigate anything about what the break-even point would really be. Your question got me curious, however, so I decided to write a bit of code to get at least some idea.
This code is definitely a very quick hack (lots of duplication, only currently supports one type of key, etc.) but at least it might give some idea of what to expect:
#include <set>
#include <vector>
#include <string>
#include <time.h>
#include <iostream>
#include <algorithm>
int main() {
static const int reps = 100000;
std::vector<int> data_vector;
std::set<int> data_set;
std::vector<int> search_keys;
for (int size=10; size<100; size += 10) {
data_vector.clear();
data_set.clear();
search_keys.clear();
int num_keys = size / 10;
for (int i=0; i<size; i++) {
int key = rand();
if (i % num_keys == 0)
search_keys.push_back(key);
data_set.insert(key);
data_vector.push_back(key);
}
// Search for a few keys that probably aren't present.
for (int i=0; i<10; i++)
search_keys.push_back(rand());
long total_linear =0, total_binary = 0;
clock_t start_linear = clock();
for (int k=0; k<reps; k++) {
for (int j=0; j<search_keys.size(); j++) {
std::vector<int>::iterator pos = std::find(data_vector.begin(), data_vector.end(), search_keys[j]);
if (pos != data_vector.end())
total_linear += *pos;
}
}
clock_t stop_linear = clock();
clock_t start_binary = clock();
for (int k=0; k<reps; k++) {
for (int j=0; j<search_keys.size(); j++) {
std::set<int>::iterator pos = data_set.find(search_keys[j]);
if (pos != data_set.end())
total_binary += *pos;
}
}
clock_t stop_binary = clock();
std::cout << "\nignore: " << total_linear << " ignore also: " << total_binary << "\n";
std::cout << "For size = " << size << "\n";
std::cout << "\tLinear search time = " << stop_linear - start_linear << "\n";
std::cout << "\tBinary search time = " << stop_binary - start_binary << "\n";
}
return 0;
}
Here are the results I get running this on my machine:
ignore: 669830816 ignore also: 669830816
For size = 10
Linear search time = 37
Binary search time = 45
ignore: 966398112 ignore also: 966398112
For size = 20
Linear search time = 60
Binary search time = 47
ignore: 389263520 ignore also: 389263520
For size = 30
Linear search time = 83
Binary search time = 63
ignore: -1561901888 ignore also: -1561901888
For size = 40
Linear search time = 106
Binary search time = 65
ignore: -1741869184 ignore also: -1741869184
For size = 50
Linear search time = 127
Binary search time = 69
ignore: 1130798112 ignore also: 1130798112
For size = 60
Linear search time = 155
Binary search time = 75
ignore: -1949669184 ignore also: -1949669184
For size = 70
Linear search time = 173
Binary search time = 83
ignore: -1991069184 ignore also: -1991069184
For size = 80
Linear search time = 195
Binary search time = 90
ignore: 1750998112 ignore also: 1750998112
For size = 90
Linear search time = 217
Binary search time = 79
Obviously that's not the only possible test (or even close to the best one possible), but it seems to me that even a little hard data is better than none at all.
Edit: I would note for the record that I see no reason that code using two vectors (or a vector of pairs) can't be just as clean as code using a set or map. Obviously you'd want to put the code for it into a small class of its own, but I see no reason at all that it couldn't present precisely the same interface to the outside world that map does. In fact, I'd probably just call it a "tiny_map` (or something on that order).
One of the basic points of OO programming (and it remains the case in generic programming, to least some degree) is to separate the interface from the implementation. In this case, you're talking about purely an implementation detail that need not affect the interface at all. In fact, if I were writing a standard library, I'd be tempted to incorporate this as a "small map optimization" analogous to the common small string optimization. Basically, just allocate an array of 10 (or so) objects of value_type directly in the map object itself, and use them when/if the map is small, then move the data to a tree iff it grows large enough to justify it. The only real question is whether people us tiny maps often enough to justify the work.
The code for map will be much cleaner than the code for two vectors; that should be your primary concern. Only once you've determined that the performance of map is a problem in your own application should you worry about the difference, and at that point you should benchmark it yourself.
A map will never equal a sorted vector search speed, because the vector memory is better packed, and memory access is more straightforward.
However, it's only half the tale. As soon as you start modifying the container, node-base containers (which do not need moving around their elements to accomodate insertions in the middle), get a theoritecal advantage. Like the search, the better cache locality still gives an edge to vector for a small number of elements.
Exact measurements are hard to provide and depend on specific optimizations, context usage and machine, so I would advise to simply go by functionality. A map has a natural interface for searching by key, so using it will make your life easier.
If you really find yourself measuring and need to really squeeze performance out of the use case, then you probably need a more specialized data structure. Look up B-Trees.