Hashing Techniques, using unordered_map - c++

Question

So my professor just assigned this homework assignment. I know my fare share of hashing techniques, but I have absolutely no idea how to go about not losing a lot of points due to collisions, because 1 Million strings will literally brute force collisions into my hash table.
What should I focus on?
Creating a really good re-hashing technique to detect when a collision occurs and appropriately re-hash
Focus on how to convert the strings into unique integers as to avoid collisions using some kind of prime number based modulus.
Or maybe I'm just misunderstanding the assignment completely. How would you guys go about solving this. Any ideas would be really helpful.

Answer 1

The task is to create a hashfunction with zero collisions. TonyD just calculated the expected collisions to be 116. According to the grading you will get zero points for a hashfunction with 116 collisions.
The professor gave a hint to use unordered_map which doesnt help for designing hashfunctions. It may be a trick question...
How would you design a function which returns a repeatable, unique number for 1 million inputs?

Answer 2

Your teacher's asking you to hash 1 million strings and you have 2^32 = 4,294,967,296 distinct 32-bit integer values available.
With 20 character random strings, there's massively more possible strings than hash values, so you can't map specific strings onto specific hash values in a way that limits the collision potential (i.e. say you had <= 2^32 potential strings because the string length was shorter, or the values each character was allowed to take were restricted - you'd have a chance at a perfect hash function: a formula mapping each to a known distinct number).
So, you're basically left having to try to randomly but repeatably map from strings to hash values. The "Birthday Paradox" then kicks in, meaning you must expect quite a lot of collisions. How many? Well - this answer provides the formula - for m buckets (2^32) and n inserts (1,000,000):
expected collisions = n - m * (1 - ((m-1)/m)^n)
= 1,000,000 - 2^32 * (1 - ((2^32 - 1) / 2^32) ^ 1,000,000)
= 1,000,000 - 2^32 * (1 - 0.99976719645926983712557804052625)
~= 1,000,000 - 999883.6
~= 116.4
Put another way, the very best possible hash function would on average - for random string inputs - still have 116 collisions.
Your teacher says:
final score for you is max{0, 200 – 5*T}
So, there's no point doing the assignment: you're more likely to have a 24 carat gold meteor land in your front garden than get a positive score.
That said, if you want to achieve the lowest number of collisions for the class, a lowish-performance (not particularly cache friendly) but minimal collision option is simply to have an array of random data...
uint32_t data[20][256] = { ... };
Download some genuinely random data from an Internet site to populate it with. Discard any duplicate numbers (in C++, you can use a std:set<> to find them). Index by character position (0..19) then character value, generating your hash by XORing the values.
Illustration of collisions
If unconvinced by the information above, you can generate a million random 32-bit values - as if they were hashes of distinct strings - and see how often the hash values repeat. Any given run should produce output not too far from the 116 collision average calculated above.
#include <iostream>
#include <map>
#include <random>
int main()
{
std::random_device rd;
std::map<unsigned, int> count;
for (int i = 0; i < 1000000; ++i)
++count[rd()];
std::map<int, int> histogram;
for (auto& c : count)
++histogram[c.second];
for (auto& h : histogram)
std::cout << h.second << " hash values generated by " << h.first << " key(s)\n";
}
A few runs produced output...
$ ./poc
999752 hash values generated by 1 key(s)
124 hash values generated by 2 key(s)
$ ./poc
999776 hash values generated by 1 key(s)
112 hash values generated by 2 key(s)
$ ./poc
999796 hash values generated by 1 key(s)
102 hash values generated by 2 key(s)
$ ./poc
999776 hash values generated by 1 key(s)
112 hash values generated by 2 key(s)
$ ./poc
999784 hash values generated by 1 key(s)
108 hash values generated by 2 key(s)
$ ./poc
999744 hash values generated by 1 key(s)
128 hash values generated by 2 key(s)