Related
At every moment t, I have a different set of positive integers. I need to randomly select one of them, satisfying the criteria that the probability of a particular number to be selected from the set must be proportionally higher, the lower is the value of the number. At moment t+1, we have another set of positive integers, and again we need to select one of them satisfying the same criteria. So on, so forth. How to do this in c++?
One way would be to assign ranges to each number, randomize a value, and pick the number which has the range in which the randomized value is included.
Example
Input:
1 2 3
Ranges:
[0,300) [300, 450) [450, 550)
Probability for each number:
~55% ~27% ~18%
With random value between 0 and 550, ex. 135 the selected number would be 1 since 0 <= 135 < 300.
As to how you do this in C++, try it for yourself first.
What you're talking about is basically the Geometric distribution, or at least a chopped and normalized variant of it.
C++11 includes a geometric distribution generator, and you can use that directly if you like, but to keep things simple, you can also do something like this:
int genRandom(int count)
{
while(true)
{
int x = 0;
while(x < count-1)
{
if(rand() < RAND_MAX/2) // with probability 0.5...
{
return x;
}
x++;
}
}
}
Note that here I'm using rejection sampling to ensure that a value gets picked, rather than return count-1 if I run past it (which would give the last two items equal probability).
When reading how to use std::rand, I found this code on cppreference.com
int x = 7;
while(x > 6)
x = 1 + std::rand()/((RAND_MAX + 1u)/6); // Note: 1+rand()%6 is biased
What is wrong with the expression on the right? Tried it and it works perfectly.
There are two issues with rand() % 6 (the 1+ doesn't affect either problem).
First, as several answers have pointed out, if the low bits of rand() aren't appropriately uniform, the result of the remainder operator is also not uniform.
Second, if the number of distinct values produced by rand() is not a multiple of 6, then the remainder will produce more low values than high values. That's true even if rand() returns perfectly distributed values.
As an extreme example, pretend that rand() produces uniformly distributed values in the range [0..6]. If you look at the remainders for those values, when rand() returns a value in the range [0..5], the remainder produces uniformly distributed results in the range [0..5]. When rand() returns 6, rand() % 6 returns 0, just as if rand() had returned 0. So you get a distribution with twice as many 0's as any other value.
The second is the real problem with rand() % 6.
The way to avoid that problem is to discard values that would produce non-uniform duplicates. You calculate the largest multiple of 6 that's less than or equal to RAND_MAX, and whenever rand() returns a value that's greater than or equal to that multiple you reject it and call `rand() again, as many times a needed.
So:
int max = 6 * ((RAND_MAX + 1u) / 6)
int value = rand();
while (value >= max)
value = rand();
That's a different implementation of the code in question, intended to more clearly show what's going on.
There are hidden depths here:
The use of the small u in RAND_MAX + 1u. RAND_MAX is defined to be an int type, and is often the largest possible int. The behaviour of RAND_MAX + 1 would be undefined in such instances as you'd be overflowing a signed type. Writing 1u forces type conversion of RAND_MAX to unsigned, so obviating the overflow.
The use of % 6 can (but on every implementation of std::rand I've seen doesn't) introduce any additional statistical bias above and beyond the alternative presented. Such instances where % 6 is hazardous are cases where the number generator has correlation plains in the low order bits, such as a rather famous IBM implementation (in C) of rand in, I think, the 1970s which flipped the high and low bits as "a final flourish". A further consideration is that 6 is very small cf. RAND_MAX, so there will be a minimal effect if RAND_MAX is not a multiple of 6, which it probably isn't.
In conclusion, these days, due to its tractability, I'd use % 6. It's not likely to introduce any statistical anomalies beyond those introduced by the generator itself. If you are still in doubt, test your generator to see if it has the appropriate statistical properties for your use case.
This example code illustrates that std::rand is a case of legacy cargo cult balderdash that should make your eyebrows raise every time you see it.
There are several issues here:
The contract people usually assume—even the poor hapless souls who don't know any better and won't think of it in precisely these terms—is that rand samples from the uniform distribution on the integers in 0, 1, 2, …, RAND_MAX, and each call yields an independent sample.
The first problem is that the assumed contract, independent uniform random samples in each call, is not actually what the documentation says—and in practice, implementations historically failed to provide even the barest simulacrum of independence. For example, C99 §7.20.2.1 ‘The rand function’ says, without elaboration:
The rand function computes a sequence of pseudo-random integers in the range 0 to RAND_MAX.
This is a meaningless sentence, because pseudorandomness is a property of a function (or family of functions), not of an integer, but that doesn't stop even ISO bureaucrats from abusing the language. After all, the only readers who would be upset by it know better than to read the documentation for rand for fear of their brain cells decaying.
A typical historical implementation in C works like this:
static unsigned int seed = 1;
static void
srand(unsigned int s)
{
seed = s;
}
static unsigned int
rand(void)
{
seed = (seed*1103515245 + 12345) % ((unsigned long)RAND_MAX + 1);
return (int)seed;
}
This has the unfortunate property that even though a single sample may be uniformly distributed under a uniform random seed (which depends on the specific value of RAND_MAX), it alternates between even and odd integers in consecutive calls—after
int a = rand();
int b = rand();
the expression (a & 1) ^ (b & 1) yields 1 with 100% probability, which is not the case for independent random samples on any distribution supported on even and odd integers. Thus, a cargo cult emerged that one should discard the low-order bits to chase the elusive beast of ‘better randomness’. (Spoiler alert: This is not a technical term. This is a sign that whosever prose you are reading either doesn't know what they're talking about, or thinks you are clueless and must be condescended to.)
The second problem is that even if each call did sample independently from a uniform random distribution on 0, 1, 2, …, RAND_MAX, the outcome of rand() % 6 would not be uniformly distributed in 0, 1, 2, 3, 4, 5 like a die roll, unless RAND_MAX is congruent to -1 modulo 6. Simple counterexample: If RAND_MAX = 6, then from rand(), all outcomes have equal probability 1/7, but from rand() % 6, the outcome 0 has probability 2/7 while all other outcomes have probability 1/7.
The right way to do this is with rejection sampling: repeatedly draw an independent uniform random sample s from 0, 1, 2, …, RAND_MAX, and reject (for example) the outcomes 0, 1, 2, …, ((RAND_MAX + 1) % 6) - 1—if you get one of those, start over; otherwise, yield s % 6.
unsigned int s;
while ((s = rand()) < ((unsigned long)RAND_MAX + 1) % 6)
continue;
return s % 6;
This way, the set of outcomes from rand() that we accept is evenly divisible by 6, and each possible outcome from s % 6 is obtained by the same number of accepted outcomes from rand(), so if rand() is uniformly distributed then so is s. There is no bound on the number of trials, but the expected number is less than 2, and the probability of success grows exponentially with the number of trials.
The choice of which outcomes of rand() you reject is immaterial, provided that you map an equal number of them to each integer below 6. The code at cppreference.com makes a different choice, because of the first problem above—that nothing is guaranteed about the distribution or independence of outputs of rand(), and in practice the low-order bits exhibited patterns that don't ‘look random enough’ (never mind that the next output is a deterministic function of the previous one).
Exercise for the reader: Prove that the code at cppreference.com yields a uniform distribution on die rolls if rand() yields a uniform distribution on 0, 1, 2, …, RAND_MAX.
Exercise for the reader: Why might you prefer one or the other subsets to reject? What computation is needed for each trial in the two cases?
A third problem is that the seed space is so small that even if the seed is uniformly distributed, an adversary armed with knowledge of your program and one outcome but not the seed can readily predict the seed and subsequent outcomes, which makes them seem not so random after all. So don't even think about using this for cryptography.
You can go the fancy overengineered route and C++11's std::uniform_int_distribution class with an appropriate random device and your favorite random engine like the ever-popular Mersenne twister std::mt19937 to play at dice with your four-year-old cousin, but even that is not going to be fit for generating cryptographic key material—and the Mersenne twister is a terrible space hog too with a multi-kilobyte state wreaking havoc on your CPU's cache with an obscene setup time, so it is bad even for, e.g., parallel Monte Carlo simulations with reproducible trees of subcomputations; its popularity likely arises mainly from its catchy name. But you can use it for toy dice rolling like this example!
Another approach is to use a simple cryptographic pseudorandom number generator with a small state, such as a simple fast key erasure PRNG, or just a stream cipher such as AES-CTR or ChaCha20 if you are confident (e.g., in a Monte Carlo simulation for research in the natural sciences) that there are no adverse consequences to predicting past outcomes if the state is ever compromised.
I'm not an experienced C++ user by any means, but was interested to see if the other answers regarding
std::rand()/((RAND_MAX + 1u)/6) being less biased than 1+std::rand()%6 actually holds true. So I wrote a test program to tabulate the results for both methods (I haven't written C++ in ages, please check it). A link for running the code is found here. It's also reproduced as follows:
// Example program
#include <cstdlib>
#include <iostream>
#include <ctime>
#include <string>
int main()
{
std::srand(std::time(nullptr)); // use current time as seed for random generator
// Roll the die 6000000 times using the supposedly unbiased method and keep track of the results
int results[6] = {0,0,0,0,0,0};
// roll a 6-sided die 20 times
for (int n=0; n != 6000000; ++n) {
int x = 7;
while(x > 6)
x = 1 + std::rand()/((RAND_MAX + 1u)/6); // Note: 1+rand()%6 is biased
results[x-1]++;
}
for (int n=0; n !=6; n++) {
std::cout << results[n] << ' ';
}
std::cout << "\n";
// Roll the die 6000000 times using the supposedly biased method and keep track of the results
int results_bias[6] = {0,0,0,0,0,0};
// roll a 6-sided die 20 times
for (int n=0; n != 6000000; ++n) {
int x = 7;
while(x > 6)
x = 1 + std::rand()%6;
results_bias[x-1]++;
}
for (int n=0; n !=6; n++) {
std::cout << results_bias[n] << ' ';
}
}
I then took the output of this and used the chisq.test function in R to run a Chi-square test to see if the results are significantly different than expected. This stackexchange question goes into more detail of using the chi-square test to test die fairness: How can I test whether a die is fair?. Here are the results for a few runs:
> ?chisq.test
> unbias <- c(100150, 99658, 100319, 99342, 100418, 100113)
> bias <- c(100049, 100040, 100091, 99966, 100188, 99666 )
> chisq.test(unbias)
Chi-squared test for given probabilities
data: unbias
X-squared = 8.6168, df = 5, p-value = 0.1254
> chisq.test(bias)
Chi-squared test for given probabilities
data: bias
X-squared = 1.6034, df = 5, p-value = 0.9008
> unbias <- c(998630, 1001188, 998932, 1001048, 1000968, 999234 )
> bias <- c(1000071, 1000910, 999078, 1000080, 998786, 1001075 )
> chisq.test(unbias)
Chi-squared test for given probabilities
data: unbias
X-squared = 7.051, df = 5, p-value = 0.2169
> chisq.test(bias)
Chi-squared test for given probabilities
data: bias
X-squared = 4.319, df = 5, p-value = 0.5045
> unbias <- c(998630, 999010, 1000736, 999142, 1000631, 1001851)
> bias <- c(999803, 998651, 1000639, 1000735, 1000064,1000108)
> chisq.test(unbias)
Chi-squared test for given probabilities
data: unbias
X-squared = 7.9592, df = 5, p-value = 0.1585
> chisq.test(bias)
Chi-squared test for given probabilities
data: bias
X-squared = 2.8229, df = 5, p-value = 0.7273
In the three runs that I did, the p-value for both methods was always greater than typical alpha values used to test significance (0.05). This means that we wouldn't consider either of them to be biased. Interestingly, the supposedly unbiased method has consistently lower p-values, which indicates that it might actually be more biased. The caveat being that I only did 3 runs.
UPDATE: While I was writing my answer, Konrad Rudolph posted an answer that takes the same approach, but gets a very different result. I don't have the reputation to comment on his answer, so I'm going to address it here. First, the main thing is that the code he uses uses the same seed for the random number generator every time it's run. If you change the seed, you actually get a variety of results. Second, if you don't change the seed, but change the number of trials, you also get a variety of results. Try increasing or decreasing by an order of magnitude to see what I mean. Third, there is some integer truncation or rounding going on where the expected values aren't quite accurate. It probably isn't enough to make a difference, but it's there.
Basically, in summary, he just happened to get the right seed and number of trials that he might be getting a false result.
One can think of a random number generator as working on a stream of binary digits. The generator turns the stream into numbers by slicing it up into chunks. If the std:rand function is working with a RAND_MAX of 32767, then it is using 15 bits in each slice.
When one takes the modules of a number between 0 and 32767 inclusive one finds that 5462 '0's and '1's but only 5461 '2's, '3's, '4's, and '5's. Hence the result is biased. The larger the RAND_MAX value is, the less bias there will be, but it is inescapable.
What is not biased is a number in the range [0..(2^n)-1]. You can generate a (theoretically) better number in the range 0..5 by extracting 3 bits, converting them to an integer in the range 0..7 and rejecting 6 and 7.
One hopes that every bit in the bit stream has an equal chance of being a '0' or a '1' irrespective of where it is in the stream or the values of other bits. This is exceptionally difficult in practice. The many different implementations of software PRNGs offer different compromises between speed and quality. A linear congruential generator such as std::rand offers fastest speed for lowest quality. A cryptographic generator offers highest quality for lowest speed.
I have seen this question asked a lot but never seen a true concrete answer to it. So I am going to post one here which will hopefully help people understand why exactly there is "modulo bias" when using a random number generator, like rand() in C++.
So rand() is a pseudo-random number generator which chooses a natural number between 0 and RAND_MAX, which is a constant defined in cstdlib (see this article for a general overview on rand()).
Now what happens if you want to generate a random number between say 0 and 2? For the sake of explanation, let's say RAND_MAX is 10 and I decide to generate a random number between 0 and 2 by calling rand()%3. However, rand()%3 does not produce the numbers between 0 and 2 with equal probability!
When rand() returns 0, 3, 6, or 9, rand()%3 == 0. Therefore, P(0) = 4/11
When rand() returns 1, 4, 7, or 10, rand()%3 == 1. Therefore, P(1) = 4/11
When rand() returns 2, 5, or 8, rand()%3 == 2. Therefore, P(2) = 3/11
This does not generate the numbers between 0 and 2 with equal probability. Of course for small ranges this might not be the biggest issue but for a larger range this could skew the distribution, biasing the smaller numbers.
So when does rand()%n return a range of numbers from 0 to n-1 with equal probability? When RAND_MAX%n == n - 1. In this case, along with our earlier assumption rand() does return a number between 0 and RAND_MAX with equal probability, the modulo classes of n would also be equally distributed.
So how do we solve this problem? A crude way is to keep generating random numbers until you get a number in your desired range:
int x;
do {
x = rand();
} while (x >= n);
but that's inefficient for low values of n, since you only have a n/RAND_MAX chance of getting a value in your range, and so you'll need to perform RAND_MAX/n calls to rand() on average.
A more efficient formula approach would be to take some large range with a length divisible by n, like RAND_MAX - RAND_MAX % n, keep generating random numbers until you get one that lies in the range, and then take the modulus:
int x;
do {
x = rand();
} while (x >= (RAND_MAX - RAND_MAX % n));
x %= n;
For small values of n, this will rarely require more than one call to rand().
Works cited and further reading:
CPlusPlus Reference
Eternally Confuzzled
Keep selecting a random is a good way to remove the bias.
Update
We could make the code fast if we search for an x in range divisible by n.
// Assumptions
// rand() in [0, RAND_MAX]
// n in (0, RAND_MAX]
int x;
// Keep searching for an x in a range divisible by n
do {
x = rand();
} while (x >= RAND_MAX - (RAND_MAX % n))
x %= n;
The above loop should be very fast, say 1 iteration on average.
#user1413793 is correct about the problem. I'm not going to discuss that further, except to make one point: yes, for small values of n and large values of RAND_MAX, the modulo bias can be very small. But using a bias-inducing pattern means that you must consider the bias every time you calculate a random number and choose different patterns for different cases. And if you make the wrong choice, the bugs it introduces are subtle and almost impossible to unit test. Compared to just using the proper tool (such as arc4random_uniform), that's extra work, not less work. Doing more work and getting a worse solution is terrible engineering, especially when doing it right every time is easy on most platforms.
Unfortunately, the implementations of the solution are all incorrect or less efficient than they should be. (Each solution has various comments explaining the problems, but none of the solutions have been fixed to address them.) This is likely to confuse the casual answer-seeker, so I'm providing a known-good implementation here.
Again, the best solution is just to use arc4random_uniform on platforms that provide it, or a similar ranged solution for your platform (such as Random.nextInt on Java). It will do the right thing at no code cost to you. This is almost always the correct call to make.
If you don't have arc4random_uniform, then you can use the power of opensource to see exactly how it is implemented on top of a wider-range RNG (ar4random in this case, but a similar approach could also work on top of other RNGs).
Here is the OpenBSD implementation:
/*
* Calculate a uniformly distributed random number less than upper_bound
* avoiding "modulo bias".
*
* Uniformity is achieved by generating new random numbers until the one
* returned is outside the range [0, 2**32 % upper_bound). This
* guarantees the selected random number will be inside
* [2**32 % upper_bound, 2**32) which maps back to [0, upper_bound)
* after reduction modulo upper_bound.
*/
u_int32_t
arc4random_uniform(u_int32_t upper_bound)
{
u_int32_t r, min;
if (upper_bound < 2)
return 0;
/* 2**32 % x == (2**32 - x) % x */
min = -upper_bound % upper_bound;
/*
* This could theoretically loop forever but each retry has
* p > 0.5 (worst case, usually far better) of selecting a
* number inside the range we need, so it should rarely need
* to re-roll.
*/
for (;;) {
r = arc4random();
if (r >= min)
break;
}
return r % upper_bound;
}
It is worth noting the latest commit comment on this code for those who need to implement similar things:
Change arc4random_uniform() to calculate 2**32 % upper_bound as
-upper_bound % upper_bound. Simplifies the code and makes it the
same on both ILP32 and LP64 architectures, and also slightly faster on
LP64 architectures by using a 32-bit remainder instead of a 64-bit
remainder.
Pointed out by Jorden Verwer on tech#
ok deraadt; no objections from djm or otto
The Java implementation is also easily findable (see previous link):
public int nextInt(int n) {
if (n <= 0)
throw new IllegalArgumentException("n must be positive");
if ((n & -n) == n) // i.e., n is a power of 2
return (int)((n * (long)next(31)) >> 31);
int bits, val;
do {
bits = next(31);
val = bits % n;
} while (bits - val + (n-1) < 0);
return val;
}
Definition
Modulo Bias is the inherent bias in using modulo arithmetic to reduce an output set to a subset of the input set. In general, a bias exists whenever the mapping between the input and output set is not equally distributed, as in the case of using modulo arithmetic when the size of the output set is not a divisor of the size of the input set.
This bias is particularly hard to avoid in computing, where numbers are represented as strings of bits: 0s and 1s. Finding truly random sources of randomness is also extremely difficult, but is beyond the scope of this discussion. For the remainder of this answer, assume that there exists an unlimited source of truly random bits.
Problem Example
Let's consider simulating a die roll (0 to 5) using these random bits. There are 6 possibilities, so we need enough bits to represent the number 6, which is 3 bits. Unfortunately, 3 random bits yields 8 possible outcomes:
000 = 0, 001 = 1, 010 = 2, 011 = 3
100 = 4, 101 = 5, 110 = 6, 111 = 7
We can reduce the size of the outcome set to exactly 6 by taking the value modulo 6, however this presents the modulo bias problem: 110 yields a 0, and 111 yields a 1. This die is loaded.
Potential Solutions
Approach 0:
Rather than rely on random bits, in theory one could hire a small army to roll dice all day and record the results in a database, and then use each result only once. This is about as practical as it sounds, and more than likely would not yield truly random results anyway (pun intended).
Approach 1:
Instead of using the modulus, a naive but mathematically correct solution is to discard results that yield 110 and 111 and simply try again with 3 new bits. Unfortunately, this means that there is a 25% chance on each roll that a re-roll will be required, including each of the re-rolls themselves. This is clearly impractical for all but the most trivial of uses.
Approach 2:
Use more bits: instead of 3 bits, use 4. This yield 16 possible outcomes. Of course, re-rolling anytime the result is greater than 5 makes things worse (10/16 = 62.5%) so that alone won't help.
Notice that 2 * 6 = 12 < 16, so we can safely take any outcome less than 12 and reduce that modulo 6 to evenly distribute the outcomes. The other 4 outcomes must be discarded, and then re-rolled as in the previous approach.
Sounds good at first, but let's check the math:
4 discarded results / 16 possibilities = 25%
In this case, 1 extra bit didn't help at all!
That result is unfortunate, but let's try again with 5 bits:
32 % 6 = 2 discarded results; and
2 discarded results / 32 possibilities = 6.25%
A definite improvement, but not good enough in many practical cases. The good news is, adding more bits will never increase the chances of needing to discard and re-roll. This holds not just for dice, but in all cases.
As demonstrated however, adding an 1 extra bit may not change anything. In fact if we increase our roll to 6 bits, the probability remains 6.25%.
This begs 2 additional questions:
If we add enough bits, is there a guarantee that the probability of a discard will diminish?
How many bits are enough in the general case?
General Solution
Thankfully the answer to the first question is yes. The problem with 6 is that 2^x mod 6 flips between 2 and 4 which coincidentally are a multiple of 2 from each other, so that for an even x > 1,
[2^x mod 6] / 2^x == [2^(x+1) mod 6] / 2^(x+1)
Thus 6 is an exception rather than the rule. It is possible to find larger moduli that yield consecutive powers of 2 in the same way, but eventually this must wrap around, and the probability of a discard will be reduced.
Without offering further proof, in general using double the number
of bits required will provide a smaller, usually insignificant,
chance of a discard.
Proof of Concept
Here is an example program that uses OpenSSL's libcrypo to supply random bytes. When compiling, be sure to link to the library with -lcrypto which most everyone should have available.
#include <iostream>
#include <assert.h>
#include <limits>
#include <openssl/rand.h>
volatile uint32_t dummy;
uint64_t discardCount;
uint32_t uniformRandomUint32(uint32_t upperBound)
{
assert(RAND_status() == 1);
uint64_t discard = (std::numeric_limits<uint64_t>::max() - upperBound) % upperBound;
RAND_bytes((uint8_t*)(&randomPool), sizeof(randomPool));
while(randomPool > (std::numeric_limits<uint64_t>::max() - discard)) {
RAND_bytes((uint8_t*)(&randomPool), sizeof(randomPool));
++discardCount;
}
return randomPool % upperBound;
}
int main() {
discardCount = 0;
const uint32_t MODULUS = (1ul << 31)-1;
const uint32_t ROLLS = 10000000;
for(uint32_t i = 0; i < ROLLS; ++i) {
dummy = uniformRandomUint32(MODULUS);
}
std::cout << "Discard count = " << discardCount << std::endl;
}
I encourage playing with the MODULUS and ROLLS values to see how many re-rolls actually happen under most conditions. A sceptical person may also wish to save the computed values to file and verify the distribution appears normal.
Mark's Solution (The accepted solution) is Nearly Perfect.
int x;
do {
x = rand();
} while (x >= (RAND_MAX - RAND_MAX % n));
x %= n;
edited Mar 25 '16 at 23:16
Mark Amery 39k21170211
However, it has a caveat which discards 1 valid set of outcomes in any scenario where RAND_MAX (RM) is 1 less than a multiple of N (Where N = the Number of possible valid outcomes).
ie, When the 'count of values discarded' (D) is equal to N, then they are actually a valid set (V), not an invalid set (I).
What causes this is at some point Mark loses sight of the difference between N and Rand_Max.
N is a set who's valid members are comprised only of Positive Integers, as it contains a count of responses that would be valid. (eg: Set N = {1, 2, 3, ... n } )
Rand_max However is a set which ( as defined for our purposes ) includes any number of non-negative integers.
In it's most generic form, what is defined here as Rand Max is the Set of all valid outcomes, which could theoretically include negative numbers or non-numeric values.
Therefore Rand_Max is better defined as the set of "Possible Responses".
However N operates against the count of the values within the set of valid responses, so even as defined in our specific case, Rand_Max will be a value one less than the total number it contains.
Using Mark's Solution, Values are Discarded when: X => RM - RM % N
EG:
Ran Max Value (RM) = 255
Valid Outcome (N) = 4
When X => 252, Discarded values for X are: 252, 253, 254, 255
So, if Random Value Selected (X) = {252, 253, 254, 255}
Number of discarded Values (I) = RM % N + 1 == N
IE:
I = RM % N + 1
I = 255 % 4 + 1
I = 3 + 1
I = 4
X => ( RM - RM % N )
255 => (255 - 255 % 4)
255 => (255 - 3)
255 => (252)
Discard Returns $True
As you can see in the example above, when the value of X (the random number we get from the initial function) is 252, 253, 254, or 255 we would discard it even though these four values comprise a valid set of returned values.
IE: When the count of the values Discarded (I) = N (The number of valid outcomes) then a Valid set of return values will be discarded by the original function.
If we describe the difference between the values N and RM as D, ie:
D = (RM - N)
Then as the value of D becomes smaller, the Percentage of unneeded re-rolls due to this method increases at each natural multiplicative. (When RAND_MAX is NOT equal to a Prime Number this is of valid concern)
EG:
RM=255 , N=2 Then: D = 253, Lost percentage = 0.78125%
RM=255 , N=4 Then: D = 251, Lost percentage = 1.5625%
RM=255 , N=8 Then: D = 247, Lost percentage = 3.125%
RM=255 , N=16 Then: D = 239, Lost percentage = 6.25%
RM=255 , N=32 Then: D = 223, Lost percentage = 12.5%
RM=255 , N=64 Then: D = 191, Lost percentage = 25%
RM=255 , N= 128 Then D = 127, Lost percentage = 50%
Since the percentage of Rerolls needed increases the closer N comes to RM, this can be of valid concern at many different values depending on the constraints of the system running he code and the values being looked for.
To negate this we can make a simple amendment As shown here:
int x;
do {
x = rand();
} while (x > (RAND_MAX - ( ( ( RAND_MAX % n ) + 1 ) % n) );
x %= n;
This provides a more general version of the formula which accounts for the additional peculiarities of using modulus to define your max values.
Examples of using a small value for RAND_MAX which is a multiplicative of N.
Mark'original Version:
RAND_MAX = 3, n = 2, Values in RAND_MAX = 0,1,2,3, Valid Sets = 0,1 and 2,3.
When X >= (RAND_MAX - ( RAND_MAX % n ) )
When X >= 2 the value will be discarded, even though the set is valid.
Generalized Version 1:
RAND_MAX = 3, n = 2, Values in RAND_MAX = 0,1,2,3, Valid Sets = 0,1 and 2,3.
When X > (RAND_MAX - ( ( RAND_MAX % n ) + 1 ) % n )
When X > 3 the value would be discarded, but this is not a vlue in the set RAND_MAX so there will be no discard.
Additionally, in the case where N should be the number of values in RAND_MAX; in this case, you could set N = RAND_MAX +1, unless RAND_MAX = INT_MAX.
Loop-wise you could just use N = 1, and any value of X will be accepted, however, and put an IF statement in for your final multiplier. But perhaps you have code that may have a valid reason to return a 1 when the function is called with n = 1...
So it may be better to use 0, which would normally provide a Div 0 Error, when you wish to have n = RAND_MAX+1
Generalized Version 2:
int x;
if n != 0 {
do {
x = rand();
} while (x > (RAND_MAX - ( ( ( RAND_MAX % n ) + 1 ) % n) );
x %= n;
} else {
x = rand();
}
Both of these solutions resolve the issue with needlessly discarded valid results which will occur when RM+1 is a product of n.
The second version also covers the edge case scenario when you need n to equal the total possible set of values contained in RAND_MAX.
The modified approach in both is the same and allows for a more general solution to the need of providing valid random numbers and minimizing discarded values.
To reiterate:
The Basic General Solution which extends mark's example:
// Assumes:
// RAND_MAX is a globally defined constant, returned from the environment.
// int n; // User input, or externally defined, number of valid choices.
int x;
do {
x = rand();
} while (x > (RAND_MAX - ( ( ( RAND_MAX % n ) + 1 ) % n) ) );
x %= n;
The Extended General Solution which Allows one additional scenario of RAND_MAX+1 = n:
// Assumes:
// RAND_MAX is a globally defined constant, returned from the environment.
// int n; // User input, or externally defined, number of valid choices.
int x;
if n != 0 {
do {
x = rand();
} while (x > (RAND_MAX - ( ( ( RAND_MAX % n ) + 1 ) % n) ) );
x %= n;
} else {
x = rand();
}
In some languages ( particularly interpreted languages ) doing the calculations of the compare-operation outside of the while condition may lead to faster results as this is a one-time calculation no matter how many re-tries are required. YMMV!
// Assumes:
// RAND_MAX is a globally defined constant, returned from the environment.
// int n; // User input, or externally defined, number of valid choices.
int x; // Resulting random number
int y; // One-time calculation of the compare value for x
y = RAND_MAX - ( ( ( RAND_MAX % n ) + 1 ) % n)
if n != 0 {
do {
x = rand();
} while (x > y);
x %= n;
} else {
x = rand();
}
There are two usual complaints with the use of modulo.
one is valid for all generators. It is easier to see in a limit case. If your generator has a RAND_MAX which is 2 (that isn't compliant with the C standard) and you want only 0 or 1 as value, using modulo will generate 0 twice as often (when the generator generates 0 and 2) as it will generate 1 (when the generator generates 1). Note that this is true as soon as you don't drop values, whatever the mapping you are using from the generator values to the wanted one, one will occurs twice as often as the other.
some kind of generator have their less significant bits less random than the other, at least for some of their parameters, but sadly those parameter have other interesting characteristic (such has being able to have RAND_MAX one less than a power of 2). The problem is well known and for a long time library implementation probably avoid the problem (for instance the sample rand() implementation in the C standard use this kind of generator, but drop the 16 less significant bits), but some like to complain about that and you may have bad luck
Using something like
int alea(int n){
assert (0 < n && n <= RAND_MAX);
int partSize =
n == RAND_MAX ? 1 : 1 + (RAND_MAX-n)/(n+1);
int maxUsefull = partSize * n + (partSize-1);
int draw;
do {
draw = rand();
} while (draw > maxUsefull);
return draw/partSize;
}
to generate a random number between 0 and n will avoid both problems (and it avoids overflow with RAND_MAX == INT_MAX)
BTW, C++11 introduced standard ways to the the reduction and other generator than rand().
With a RAND_MAX value of 3 (in reality it should be much higher than that but the bias would still exist) it makes sense from these calculations that there is a bias:
1 % 2 = 1
2 % 2 = 0
3 % 2 = 1
random_between(1, 3) % 2 = more likely a 1
In this case, the % 2 is what you shouldn't do when you want a random number between 0 and 1. You could get a random number between 0 and 2 by doing % 3 though, because in this case: RAND_MAX is a multiple of 3.
Another method
There is much simpler but to add to other answers, here is my solution to get a random number between 0 and n - 1, so n different possibilities, without bias.
the number of bits (not bytes) needed to encode the number of possibilities is the number of bits of random data you'll need
encode the number from random bits
if this number is >= n, restart (no modulo).
Really random data is not easy to obtain, so why use more bits than needed.
Below is an example in Smalltalk, using a cache of bits from a pseudo-random number generator. I'm no security expert so use at your own risk.
next: n
| bitSize r from to |
n < 0 ifTrue: [^0 - (self next: 0 - n)].
n = 0 ifTrue: [^nil].
n = 1 ifTrue: [^0].
cache isNil ifTrue: [cache := OrderedCollection new].
cache size < (self randmax highBit) ifTrue: [
Security.DSSRandom default next asByteArray do: [ :byte |
(1 to: 8) do: [ :i | cache add: (byte bitAt: i)]
]
].
r := 0.
bitSize := n highBit.
to := cache size.
from := to - bitSize + 1.
(from to: to) do: [ :i |
r := r bitAt: i - from + 1 put: (cache at: i)
].
cache removeFrom: from to: to.
r >= n ifTrue: [^self next: n].
^r
Modulo reduction is a commonly seen way to make a random integer generator avoid the worst case of running forever.
When the range of possible integers is unknown, however, there is no way in general to "fix" this worst case of running forever without introducing bias. It's not just modulo reduction (rand() % n, discussed in the accepted answer) that will introduce bias this way, but also the "multiply-and-shift" reduction of Daniel Lemire, or if you stop rejecting an outcome after a set number of iterations. (To be clear, this doesn't mean there is no way to fix the bias issues present in pseudorandom generators. For example, even though modulo and other reductions are biased in general, they will have no issues with bias if the range of possible integers is a power of 2 and if the random generator produces unbiased random bits or blocks of them.)
The following answer of mine discusses the relationship between running time and bias in random generators, assuming we have a "true" random generator that can produce unbiased and independent random bits. The answer doesn't even involve the rand() function in C because it has many issues. Perhaps the most serious here is the fact that the C standard does not explicitly specify a particular distribution for the numbers returned by rand(), not even a uniform distribution.
How to generate a random integer in the range [0,n] from a stream of random bits without wasting bits?
As the accepted answer indicates, "modulo bias" has its roots in the low value of RAND_MAX. He uses an extremely small value of RAND_MAX (10) to show that if RAND_MAX were 10, then you tried to generate a number between 0 and 2 using %, the following outcomes would result:
rand() % 3 // if RAND_MAX were only 10, gives
output of rand() | rand()%3
0 | 0
1 | 1
2 | 2
3 | 0
4 | 1
5 | 2
6 | 0
7 | 1
8 | 2
9 | 0
So there are 4 outputs of 0's (4/10 chance) and only 3 outputs of 1 and 2 (3/10 chances each).
So it's biased. The lower numbers have a better chance of coming out.
But that only shows up so obviously when RAND_MAX is small. Or more specifically, when the number your are modding by is large compared to RAND_MAX.
A much better solution than looping (which is insanely inefficient and shouldn't even be suggested) is to use a PRNG with a much larger output range. The Mersenne Twister algorithm has a maximum output of 4,294,967,295. As such doing MersenneTwister::genrand_int32() % 10 for all intents and purposes, will be equally distributed and the modulo bias effect will all but disappear.
I just wrote a code for Von Neumann's Unbiased Coin Flip Method, that should theoretically eliminate any bias in the random number generation process. More info can be found at (http://en.wikipedia.org/wiki/Fair_coin)
int unbiased_random_bit() {
int x1, x2, prev;
prev = 2;
x1 = rand() % 2;
x2 = rand() % 2;
for (;; x1 = rand() % 2, x2 = rand() % 2)
{
if (x1 ^ x2) // 01 -> 1, or 10 -> 0.
{
return x2;
}
else if (x1 & x2)
{
if (!prev) // 0011
return 1;
else
prev = 1; // 1111 -> continue, bias unresolved
}
else
{
if (prev == 1)// 1100
return 0;
else // 0000 -> continue, bias unresolved
prev = 0;
}
}
}
I want to select a number of random words from an array to make a total amount of 36 letters.
At first I tried to select a random word and add it after checking that it's not longer than the amount of free space we have. That was not efficient since the list would fill up and there would only be empty space left for a 2-3 letter word and it takes a long time to find such a short word.
So i decided to only choose six 6-letter words and I'm doing that by generating a random number and then incrementing it by 1 until we find a 6 letter word. It's pretty fast, but the words aren't really that random, often I get words that start from the same letter or only words that start with letters in sequence like a,b,c or x,y,z.
srand ( time(NULL) );
for(int i=0;i<6;i++)
{
randNumb = rand()%dictionary.size();
while(dictionary.at(randNumb).length() != 6)
{
randNumb++;
}
a << "/" << dictionary.at(randNumb) << "/";
}
I would like to choose words with different lengths but in favor of performance I'll settle with just the 6-letter words but then i would at least want them to be more randomly selected.
You should get a new random number instead of increasing the index. The way you do it, all the strings not matching your criteria "attract" more random numbers, and possibly lead to the following string to have a higher probability of being chosen.
The rand() function generates a number between 0 and RAND_MAX.
If RAND_MAX is defined as 32767, then you will not access elements in your dictionary (array?) with indices greater than that.
If you need to generate a random number greater than RAND_MAX, then think about summing the result of n calls of rand(), such that n * RAND_MAX >= dictionary.size(). The modulus of this result is then guaranteed to give an index that falls somewhere in the bounds of the entire dictionary.
Even if RAND_MAX is greater than dictionary.size(), using the % operator to select the index leads to a non-uniform distribution. The modulus will cause the early words to be selected more often than the later words (unless RAND_MAX + 1 is an integer multiple of dictionary.size()).
Consider a simple example: Assume your dictionary has 10 words, and RAND_MAX is 14. When rand() returns a value from 0 to 9, the corresponding word is chosen directly. But when rand() is 10 through 14, then one of the first five words will be chosen. So the first five words have twice the chance of being selected than the last five words.
A better way to map [0..RAND_MAX] to [0..dictionary.size()) is to use division:
assert(RAND_MAX + 1 >= dictionary.size());
randNumb = rand() * dictionary.size() / (RAND_MAX + 1);
But you have to be careful of integer overflow. If RAND_MAX * dictionary.size() is larger than you can represent in an integer, you'll need to use a larger data type. Some systems have a function like MulDiv for just this purpose. If you don't have something like MulDiv, you can convert to a floating point type and then truncate the result back to an integer:
double temp = static_cast<double>(rand()) * dictionary.size() / (RAND_MAX + 1);
randNumb = static_cast<int>(temp);
This is still an imperfect distribution, but the "hot" words will now be evenly distributed across the dictionary instead of clumping at the beginning.
The closer RAND_MAX + 1 is to an integer multiple of dictionary.size(), the better off you'll be. And if you can't be sure that it's close to an integer multiple, then you want RAND_MAX to be as large as possible relative to dictionary.size().
Since you don't have much control over RAND_MAX, you could consider tweaking dictionary.size(). For example, if you only want six-letter words, then why not strip all the others out of the dictionary?
std::vector<std::string> six_letter_words;
std::copy_if(dictionary.begin(), dictionary.end(),
std::back_inserter(six_letter_words),
[](const std::string &word){ return word.size() == 6; });
With the reduced set, we can use a more generic algorithm to select the words:
typedef std::vector<std::string> WordList;
// Returns true with the given probability, which should be 0.0 to 1.0.
bool Probably(double probability) {
return (static_cast<double>(std::rand()) / RAND_MAX) < probability;
}
// Selects n words from the dictionary using a normal distribution and
// copies them to target.
template <typename OutputIt>
OutputIt Select(int n, const WordList &dictionary, OutputIt target) {
double count = static_cast<double>(n);
for (std::size_t i = 0; count > 0.0 && i < dictionary.size(); ++i) {
if (Probably(count / (dictionary.size() - i))) {
*target++ = dictionary[i];
count -= 1.0;
}
}
return target;
}
The idea is to step through each word in the dictionary and select it with a probability of the number of words you need to pick divided by the number of words left to pick from. This works well, even if RAND_MAX is relatively small. Overall, though, it's much more computation than trying to randomly select indexes. Also note that this technique will never choose the same word more than once, where the index mapping technique could.
You call Select like this:
// Select six words from six_letter_words using a normal distribution.
WordList selected;
Select(6, six_letter_words, std::back_inserter(selected));
Also note that most implementations of rand() are pretty simplistic and may not give a good normal distribution to begin with.
Code:
int random = (rand() % 7 + 1)
if (random == 1) { } // num 1
else if (random == 2) { } // num 2
else if (random == 3 || random == 4) { } // num 3
else if (random == 5 || random == 6) { } // num 4
else if (random == 7) { } // num 5
Basically I want each of these numbers with each of these probabilities:
1: 1/7
2: 1/7
3: 2/7
4: 2/7
5: 1/7
Will this code give me proper results? I.e. if this is run infinite times, will I get the proper frequencies? Is there a less-lengthy way of doing this?
Not, it's actually slightly off, due to the way rand() works. In particular, rand returns values in the range [0,RAND_MAX]. Hypothetically, assume RAND_MAX were ten. Then rand() would give 0…10, and they'd be mapped (by modulus) to:
0 → 0
1 → 1
2 → 2
3 → 3
4 → 4
5 → 5
6 → 6
7 → 0
8 → 1
9 → 2
10 → 3
Note how 0–3 are more common than 4–6; this is bias in your random number generation. (You're adding 1 as well, but that just shifts it over).
RAND_MAX of course isn't 10, but it's probably not a multiple of 7 (minus 1), either. Most likely its a power of two. So you'll have some bias.
I suggest using the Boost Random Number Library which can give you a random number generator that yields 1–7 without bias. Look also at bames53's answer using C++11, which is the right way to do this if your code only needs to target C++11 platforms.
Just another way:
float probs[5] = {1/7.0f, 1/7.0f, 2/7.0f, 2/7.0f, 1/7.0f};
float sum = 0;
for (int i = 0; i < 5; i++)
sum += probs[i]; /* edit */
int rand_M() {
float f = (rand()*sum)/RAND_MAX; /* edit */
for (int i = 0; i < 5; i++) {
if (f <= probs[i]) return i;
f -= probs[i];
}
return 4;
}
Assuming rand() is good then your code will work with only a very small bias to the lower X numbers, where X is RAND_MAX % 7. It's much more likely that you won't get the desired odds due to the quality of the implementation of rand(). If you find that to be the case then you'll want to use an alternative random number generator.
C++11 introduces the header <random> which includes several quality RNGs. Here's an example:
#include <random>
#include <functional>
auto rand = std::bind(std::uniform_int_distribution<int>(1,7),std::mt19937());
Given this, when you call rand() you will get a number from 1 to 7 each with equal probability. (And you can choose different engines if for different quality and speed characteristics.) You can then use this to implement the if-else conditions your example currently uses with std::rand(). However <random> allows you to do even better using one of their non-uniform distributions. In this case what you want is discrete_distribution. This distribution allows you to explicitly state the weights for each value from 0 to n.
// the random number generator
auto _rand = std::bind(std::discrete_distribution<int>{1./7.,1./7.,2./7.,2./7.,1./7.},std::mt19937());
// convert results of RNG from the range [0-4] to [1-5]
auto rand = [&_rand]() { return _rand() +1; };
int toohigh = RAND_MAX - RAND_MAX%7;
int random;
do {
random = rand();
while (random >= toohigh); //should happen ~0.03% of the time
static const int results[7] = {1, 2, 3, 3, 4, 4, 5};
random = results[random%7];
This should give numbers with a distribution as even as rand can handle, and without the big if switch.
Note this does have a theoretically possible infinite loop, but the statistical odds of it staying in the loop for even are minuscule. The odds of it staying in the loop twice is quite close to the odds of winning the California Super Lotto Jackpot. Even if every person on the planet got five random numbers, it probably wouldn't stay in the loop three times. (Assuming a perfect RNG.)
rand returns pseudo-random integral number:
Notice though that this modulo operation does not generate a truly
uniformly distributed random number in the span (since in most cases
lower numbers are slightly more likely), but it is generally a good
approximation for short spans.
Now, regarding the less-lengthy way, you can use switch-case construction, or a series of conditional operators ?: (which will make your code short and unreadable:).