Speed of associative array (map) in STL [closed] - c++

Question

Closed. This question is not reproducible or was caused by typos. It is not currently accepting answers.
This question was caused by a typo or a problem that can no longer be reproduced. While similar questions may be on-topic here, this one was resolved in a way less likely to help future readers.
Closed 8 years ago.
Improve this question
Wrote a simple program to measure the speed of STL. The following code showed that it took 1.49sec on my Corei7-2670QM PC (2.2GHz and turbo 3.1GHz). If I remove the Employees[buf] = i%1000; part in the loop, it only took 0.0132sec. So the hashing part took 1.48sec. Why is it that slow?
#include <string.h>
#include <iostream>
#include <map>
#include <utility>
#include <stdio.h>
#include <sys/time.h>
using namespace std;
extern "C" {
int get(map<string, int> e, char* s){
return e[s];
}
int set(map<string, int> e, char* s, int value) {
e[s] = value;
}
}
double getTS() {
struct timeval tv;
gettimeofday(&tv, NULL);
return tv.tv_sec + tv.tv_usec/1000000.0;
}
int main()
{
map<string, int> Employees;
char buf[10];
int i;
double ts = getTS();
for (i=0; i<1000000; i++) {
sprintf(buf, "%08d", i);
Employees[buf] = i%1000;
}
printf("took %f sec\n", getTS() - ts);
cout << Employees["00001234"] << endl;
return 0;
}

Answer 1

Here's a C++ version of your code. Note that you should obviously take the maps by reference when passing them in get/set.
UPDATE Taking things a bit further and seriously optimizing for the given test case:
Live On Coliru
#include <iostream>
#include <boost/container/flat_map.hpp>
#include <chrono>
using namespace std;
using Map = boost::container::flat_map<string, int>;
int get(Map &e, char *s) { return e[s]; }
int set(Map &e, char *s, int value) { return e[s] = value; }
using Clock = std::chrono::high_resolution_clock;
template <typename F, typename Reso = std::chrono::microseconds, typename... Args>
Reso measure(F&& f, Args&&... args) {
auto since = Clock::now();
std::forward<F>(f)(std::forward<Args>(args)...);
return chrono::duration_cast<Reso>(Clock::now() - since);
}
#include <boost/iterator/iterator_facade.hpp>
using Pair = std::pair<std::string, int>;
struct Gen : boost::iterators::iterator_facade<Gen, Pair, boost::iterators::single_pass_traversal_tag, Pair>
{
int i;
Gen(int i = 0) : i(i) {}
value_type dereference() const {
char buf[10];
std::sprintf(buf, "%08d", i);
return { buf, i%1000 };
}
bool equal(Gen const& o) const { return i==o.i; }
void increment() { ++i; }
};
int main() {
Map Employees;
const auto n = 1000000;
auto elapsed = measure([&] {
Employees.reserve(n);
Employees.insert<Gen>(boost::container::ordered_unique_range, {0}, {n});
});
std::cout << "took " << elapsed.count() / 1000000.0 << " sec\n";
cout << Employees["00001234"] << endl;
}
Prints
took 0.146575 sec
234
Old answer
This just used C++ where appropriate
Live On Coliru
#include <iostream>
#include <map>
#include <chrono>
#include <cstdio>
using namespace std;
int get(map<string, int>& e, char* s){
return e[s];
}
int set(map<string, int>& e, char* s, int value) {
return e[s] = value;
}
using Clock = std::chrono::high_resolution_clock;
template <typename Reso = std::chrono::microseconds>
Reso getElapsed(Clock::time_point const& since) {
return chrono::duration_cast<Reso>(Clock::now() - since);
}
int main()
{
map<string, int> Employees;
std::string buf(10, '\0');
auto ts = Clock::now();
for (int i=0; i<1000000; i++) {
buf.resize(std::sprintf(&buf[0], "%08d", i));
Employees[buf] = i%1000;
}
std::cout << "took " << getElapsed(ts).count()/1000000.0 << " sec\n";
cout << Employees["00001234"] << endl;
}
Prints:
took 0.470009 sec
234

Answer 2

The notion of "slow" depends of course in comparison to what.
I ran your benchmark (using the standard chrono::high_resolution_clock instead of gettimeofday() ) on MSVC2013 with release configuration on an Corei7-920 at 2.67 GHz and find very similar results (1.452 s).
In your code, you do basically 1 millions of:
insertion in the map: Employees\[buf\]
update in the map (copying a new element to exisitng element): = i%1000
SO I tried to understand better where the time is spent:
first, the map needs to store the ordered keys, which is typically implemented with a binary tree. So I tried to use an unordered_map which uses a flatter hash table and gave it a very large bucket size to avoid clisions and rehashing. The result is then 1.198 s.
So roughly 20% of the time (here) is needed for making possibile a sorted access to the map data (i.e. you can iterate through your map using the order of the keys: do you need this ?)
next, playing with the order of insertion can really influence significantly the timing. As Thomas Matthews pointed out in the comments: for benchmarking purpose you should use random order.
then, making only and optimised insertion of data (no search no update) using emplace_hint() brings us to a time of 1.100 s.
So 75% of the time is needed to allocate and insert the data
finally, elaborating on the previous test, if you add an additional search and update after emplace_hint(), then the time goes up slightly above the original time (1.468 s). This confirms that access to the map is only a fraction of the time and most of the execution time is needed for the insertion.
Here the test for the point above:
chrono::high_resolution_clock::time_point ts = chrono::high_resolution_clock::now();
for (i = 0; i<1000000; i++) {
sprintf(buf, "%08d", i);
Employees.emplace_hint(Employees.end(), buf, 0);
Employees[buf] = i % 1000; // matters for 300
}
chrono::high_resolution_clock::time_point te = chrono::high_resolution_clock::now();
cout << "took " << chrono::duration_cast<chrono::milliseconds>(te - ts).count() << " millisecs\n";
Now your benchmark not only depends performance of the map: you do 1 million of sprintf() to set your buffer, and 1 million of conversion to a string. If you'd use a map instead, you'd notice that the whole test would take only 0.950s instead of 1.450s:
30% of your benchmark time is caused not by the map, but by the many strings you handle !
Of course, all this is much slower than a vector. But a vector doesn't sort its elements, and cannot provide for associative store.