Related
reading from various other SO questions, when using rand() % N you may happen to modify the bias for the pseudo number you get, so you usually have to introduce some range handling.
However in all cases rand() was always mentioned, and not the newer random() or arcrandom4() functions or the native C++11 methods. What happens when you run these routines over a set? Do you get a bias like rand()?
Thanks.
The following answer does not go into as much detail as Eric Lippert's blog post on the same topic. Also, this question and its answers deal with the same topic.
Most of the bias that comes from doing rand() % N isn't from the rand() part - it's from the % N part.
Let's consider a 'good' implementation of rand() that generates all numbers from 0 to 100 (picked for simplicity) with equal probability - a uniform distribution. Next let's say that we want to use this implementation of rand() to generate random numbers between 0 and 80, so we do rand() % 80. Let's break down the possibilities of what could happen next:
rand() generates a number from 0 to 79. Any number from 0 to 79 % 80 stays the same number
rand() generates a number from 80 to 100. Any number from 80 to 100 % 80 gets converted to 0 to 20
This means that there are two ways to end up with a number from 0 and 20 but only one way to end up with a number from 21 to 79. Getting a number from 0 to 20 is more likely than getting a number from 21 to 79. This is usually not a desirable property.
Any value of N that divides evenly into the max value of rand() won't have this problem because there will be an equal number of ways to generate any value. Furthermore the bias is much smaller for small values of N than it is for values of N closer to the max value of rand().
So, what about functions other than rand()? If they return values from some fixed range and you do a mod operation, they will suffer from the same bias. If you're calling a random function that takes a range as arguments, then you don't need to do the mod operation. The function will probably handle any biases internally.
What happens when you run these routines over a set? Do you get a bias
like rand()?
The answer is: this depends on the relation between the size of range returned by the generator and the divisor in modulo operation. If divisor not evenly divides the range then distribution will be skewed. The bias ratio is in the range [ 1, 2], where 1 means no bias ( as for uniform distribution) and the bias increases with divisor. Regarding arcrandom4() this translates to skewed distribution obtained in all cases when modulo divisor is not an even divisor of 2^32. The rationale behind it is explained below.
Introduction. The bias
Imagine that we are trying to simulate uniform int distribution over interval [0, 99] with
int x = rand() % 100;
Operator % makes the probability distribution of X skewed because RAND_MAX which is maximum value for rand() can be not equal to k * 100 + 99. This results in that if you imagine all 100-length parts of 0-RAND_MAX range then you can see that last part will probably not produce a full range 0-99. Therefore you have more numbers that generates 0, 1, 2..., p but not necessary p + 1, ..., 98, 99 ( 1 more occurrence for each number in 0, 1, 2, ..., p). The inaccuracy of this approach increases with bigger divisor that not divides the range evenly and maximum bias compared to uniform distribution is equal 2.
In the following sections below we show that the bias measured as a ratio of probability of getting number from [ 0, p] to probability of number from [ p + 1, n] is equal to ( k + 1) / k and we confirm this with 2 examples.
Formula
We will show what exactly is the bias introduced by operation modulo ( operation that is applied to generator of uniform distribution in order to trim the output range). We will operate in terms of formula
x = rand() % ( n + 1)
where rand() is some generator and ( n + 1) is divisor in modulo operation. The picture below shows our standpoint:
We can see how numbers in range [ 0, n] are divided into these that repeat k + 1 times (numbers [ 0, p]) and these that repeats k times ( numbers [ p + 1, n]) in a single trial, which is "take the number from the distribution obtained by x = rand() % (n+1)". The p is defined as a remainder when dividing the maximum number ( i.e. Rand_MAX) given by the generator by the ( n + 1) which is the size of desired range:
p = ( N - 1) % ( n + 1)
N - 1 = k * ( n + 1) + p
and the k is the quotient
k = ( N - 1 - p) / ( n + 1)
In a single trial there are
( p + 1) * ( k + 1) + ( n - p) * k =
= p + 1 + k( n + 1) = N
possible outcomes. Thus the probability of receiving the element that repeats k times is k / N. Let's denote
f_0 = ( k + 1) / N, probability for each element from [ 0, p]
f_1 = k / N, probability for each element from [ p + 1, n]
Let's say that we will express the bias of sampling from this, transformed distribution over the uniform distribution as the ratio of probability of element that belongs to [ 0, p] to probability of element from the range [ p + 1, n]:
bias = f_0 / f_1 = ( k + 1) / k
So, are numbers twice as often?
No. The fact that when we look at the picture numbers repeats doesn't imply the ratio of 2. This ratio is just a special case, if range of the generator is divided into exactly 2 subranges. In general the bias ratio is( k + 1) / k and decreases asymptotically, when divisor n + 1 tends to 1, ( and k tends to N).
Examples
We now consider two simple examples (as suggested by #dyp). First we will generate 1000 * 1000 samples from a distribution given by
x = rand() % m
with generator being std::uniform_int_distribution<> dist(0, 19) and divisor m = n + 1 equal to 15 and next equal to 6.
Example 1
int x = rand() % 15; // n + 1 = 15, rand is uniform distribution over [0,19]
Test program is:
#include <iostream>
#include <random>
#include <vector>
int main()
{
std::random_device rd;
std::mt19937 mt(rd());
std::uniform_int_distribution<> dist(0, 19);
std::vector<int> v(15);
const int runs = 1000 * 1000;
for (int i = 0; i < runs; ++i)
{
++v[dist(mt) % v.size()];
}
for (int i = 0; i < v.size(); ++i)
{
std::cout << i << ": " << v[i] << "\n";
}
}
code
result:
0: 100500
1: 100016
2: 99724
3: 99871
4: 99936
5: 50008
6: 49762
7: 50023
8: 50123
9: 49963
10: 50117
11: 50049
12: 49885
13: 49760
14: 50263
We can see that in this case numbers in range [ 0, p] = [ 0, 4] appears about twice as often as the rest. This is in accordance with our bias formula
bias = f_0 / f_1 = ( k + 1) / k = 2 / 1
Example 2
int x = rand() % 6; // n + 1 = 6, rand is uniform distribution over [0,19]
Test program is:
#include <iostream>
#include <random>
#include <vector>
int main()
{
std::random_device rd;
std::mt19937 mt(rd());
std::uniform_int_distribution<> dist(0, 19);
std::vector<int> v(6);
const int runs = 1000 * 1000;
for (int i = 0; i < runs; ++i)
{
++v[dist(mt) % v.size()];
}
for (int i = 0; i < v.size(); ++i)
{
std::cout << i << ": " << v[i] << "\n";
}
}
code
result:
0: 199875
1: 199642
2: 149852
3: 149789
4: 150237
5: 150605
In this case we observe that numbers in range [ 0, p] = [ 0, 1] appears not about twice as often as the rest but in the ratio of about 20/15. In fact this is 4/3 as our bias formula in this case is
bias = f_0 / f_1 = ( k + 1) / k = 4 / 3
The picture below helps to understand this outcome.
full code
C++11 has solve this problem by adding alternative random generator engines.
The reason why using %(modulo) to constrain your random number to a range is bad has less to do with bias and more to do with the typical implementation of rand(), a linear congruential generator (LCG). Most language runtimes use LCGs for their random function; only very recently designed languages tend to differ.
An LCG is just a multiply and an add (the modulo usually being implemented via an integer’s maximum size). It should be obvious that the low bits of such a sequence follow a regular pattern – the multiply doesn’t mix higher bits into the lower bits, and the add mutates the low bits in a constant way every iteration.
By understanding the different random generators (linear_congruential_engine, mersenne_twister_engine, subtract_with_carry_engine) engines you can find the best one for your application.
there is a very good reference to the new c++ implementations in Random Engines in c++11
As said by #dpy std::uniform_int_distribution is an option given by c++ for random distributions. It treats the bias problem even if the random generator engine has . But if you set a range from 1-19 and store it in a 15 size array by using % operation the bias problem is reintroduced as discussed in many posts here.
I am a newbie in C++ and need logical help in the following task.
Given a sequence of n positive integers (n < 10^6; each given integer is less than 10^6), write a program to find the smallest positive integer, which cannot be expressed as a sum of 1, 2, or more items of the given sequence (i.e. each item could be taken 0 or 1 times). Examples: input: 2 3 4, output: 1; input: 1 2 6, output: 4
I cannot seem to construct the logic out of it, why the last output is 4 and how to implement it in C++, any help is greatly appreciated.
Here is my code so far:
#include<iostream>
using namespace std;
const int SIZE = 3;
int main()
{
//Lowest integer by default
int IntLowest = 1;
int x = 0;
//Our sequence numbers
int seq;
int sum = 0;
int buffer[SIZE];
//Loop through array inputting sequence numbers
for (int i = 0; i < SIZE; i++)
{
cout << "Input sequence number: ";
cin >> seq;
buffer[i] = seq;
sum += buffer[i];
}
int UpperBound = sum + 1;
int a = buffer[x] + buffer[x + 1];
int b = buffer[x] + buffer[x + 2];
int c = buffer[x + 1] + buffer[x + 2];
int d = buffer[x] + buffer[x + 1] + buffer[x + 2];
for (int y = IntLowest - 1; y < UpperBound; y++)
{
//How should I proceed from here?
}
return 0;
}
What the answer of Voreno suggests is in fact solving 0-1 knapsack problem (http://en.wikipedia.org/wiki/Knapsack_problem#0.2F1_Knapsack_Problem). If you follow the link you can read how it can be done without constructing all subsets of initial set (there are too much of them, 2^n). And it would work if the constraints were a bit smaller, like 10^3.
But with n = 10^6 it still requires too much time and space. But there is no need to solve knapsack problem - we just need to find first number we can't get.
The better solution would be to sort the numbers and then iterate through them once, finding for each prefix of your array a number x, such that with that prefix you can get all numbers in interval [1..x]. The minimal number that we cannot get at this point is x + 1. When you consider the next number a[i] you have two options:
a[i] <= x + 1, then you can get all numbers up to x + a[i],
a[i] > x + 1, then you cannot get x + 1 and you have your answer.
Example:
you are given numbers 1, 4, 12, 2, 3.
You sort them (and get 1, 2, 3, 4, 12), start with x = 0, consider each element and update x the following way:
1 <= x + 1, so x = 0 + 1 = 1.
2 <= x + 1, so x = 1 + 2 = 3.
3 <= x + 1, so x = 3 + 3 = 6.
4 <= x + 1, so x = 6 + 4 = 10.
12 > x + 1, so we have found the answer and it is x + 1 = 11.
(Edit: fixed off-by-one error, added example.)
I think this can be done in O(n) time and O(log2(n)) memory complexities.
Assuming that a BSR (highest set bit index) (floor(log2(x))) implementation in O(1) is used.
Algorithm:
1 create an array of (log2(MAXINT)) buckets, 20 in case of 10^6, Each bucket contains the sum and min values (init: min = 2^(i+1)-1, sum = 0). (lazy init may be used for small n)
2 one pass over the input, storing each value in the buckets[bsr(x)].
for (x : buffer) // iterate input
buckets[bsr(x)].min = min(buckets[bsr(x)].min, x)
buckets[bsr(x)].sum += x
3 Iterate over buckets, maintaining unreachable:
int unreachable = 1 // 0 is always reachable
for(b : buckets)
if (unreachable >= b.min)
unreachable += b.sum
else
break
return unreachable
This works because, assuming we are at bucket i, lets consider the two cases:
unreachable >= b.min is true: because this bucket contains values in the range [2^i...2^(i+1)-1], this implies that 2^i <= b.min. in turn, b.min <= unreachable. therefor unreachable+b.min >= 2^(i+1). this means that all values in the bucket may be added (after adding b.min all the other values are smaller) i.e. unreachable += b.sum.
unreachable >= b.min is false: this means that b.min (the smallest number the the remaining sequence) is greater than unreachable. thus we need to return unreachable.
The output of the second input is 4 because that is the smallest positive number that cannot be expressed as a sum of 1,2 or 6 if you can take each item only 0 or 1 times. I hope this can help you understand more:
You have 3 items in that list: 1,2,6
Starting from the smallest positive integer, you start checking if that integer can be the result of the sum of 1 or more numbers of the given sequence.
1 = 1+0+0
2 = 0+2+0
3 = 1+2+0
4 cannot be expressed as a result of the sum of one of the items in the list (1,2,6). Thus 4 is the smallest positive integer which cannot be expressed as a sum of the items of that given sequence.
The last output is 4 because:
1 = 1
2 = 2
1 + 2 = 3
1 + 6 = 7
2 + 6 = 8
1 + 2 + 6 = 9
Therefore, the lowest integer that cannot be represented by any combination of your inputs (1, 2, 6) is 4.
What the question is asking:
Part 1. Find the largest possible integer that can be represented by your input numbers (ie. the sum of all the numbers you are given), that gives the upper bound
UpperBound = sum(all_your_inputs) + 1
Part 2. Find all the integers you can get, by combining the different integers you are given. Ie if you are given a, b and c as integers, find:
a + b, a + c, b + c, and a + b + c
Part 2) + the list of integers, gives you all the integers you can get using your numbers.
cycle for each integer from 1 to UpperBound
for i = 1 to UpperBound
if i not = a number in the list from point 2)
i = your smallest integer
break
This is a clumsy way of doing it, but I'm sure that with some maths it's possible to find a better way?
EDIT: Improved solution
//sort your input numbers from smallest to largest
input_numbers = sort(input_numbers)
//create a list of integers that have been tried numbers
tried_ints = //empty list
for each input in input_numbers
//build combinations of sums of this input and any of the previous inputs
//add the combinations to tried_ints, if not tried before
for 1 to input
//check whether there is a gap in tried_ints
if there_is_gap
//stop the program, return the smallest integer
//the first gap number is the smallest integer
I have seen this question asked a lot but never seen a true concrete answer to it. So I am going to post one here which will hopefully help people understand why exactly there is "modulo bias" when using a random number generator, like rand() in C++.
So rand() is a pseudo-random number generator which chooses a natural number between 0 and RAND_MAX, which is a constant defined in cstdlib (see this article for a general overview on rand()).
Now what happens if you want to generate a random number between say 0 and 2? For the sake of explanation, let's say RAND_MAX is 10 and I decide to generate a random number between 0 and 2 by calling rand()%3. However, rand()%3 does not produce the numbers between 0 and 2 with equal probability!
When rand() returns 0, 3, 6, or 9, rand()%3 == 0. Therefore, P(0) = 4/11
When rand() returns 1, 4, 7, or 10, rand()%3 == 1. Therefore, P(1) = 4/11
When rand() returns 2, 5, or 8, rand()%3 == 2. Therefore, P(2) = 3/11
This does not generate the numbers between 0 and 2 with equal probability. Of course for small ranges this might not be the biggest issue but for a larger range this could skew the distribution, biasing the smaller numbers.
So when does rand()%n return a range of numbers from 0 to n-1 with equal probability? When RAND_MAX%n == n - 1. In this case, along with our earlier assumption rand() does return a number between 0 and RAND_MAX with equal probability, the modulo classes of n would also be equally distributed.
So how do we solve this problem? A crude way is to keep generating random numbers until you get a number in your desired range:
int x;
do {
x = rand();
} while (x >= n);
but that's inefficient for low values of n, since you only have a n/RAND_MAX chance of getting a value in your range, and so you'll need to perform RAND_MAX/n calls to rand() on average.
A more efficient formula approach would be to take some large range with a length divisible by n, like RAND_MAX - RAND_MAX % n, keep generating random numbers until you get one that lies in the range, and then take the modulus:
int x;
do {
x = rand();
} while (x >= (RAND_MAX - RAND_MAX % n));
x %= n;
For small values of n, this will rarely require more than one call to rand().
Works cited and further reading:
CPlusPlus Reference
Eternally Confuzzled
Keep selecting a random is a good way to remove the bias.
Update
We could make the code fast if we search for an x in range divisible by n.
// Assumptions
// rand() in [0, RAND_MAX]
// n in (0, RAND_MAX]
int x;
// Keep searching for an x in a range divisible by n
do {
x = rand();
} while (x >= RAND_MAX - (RAND_MAX % n))
x %= n;
The above loop should be very fast, say 1 iteration on average.
#user1413793 is correct about the problem. I'm not going to discuss that further, except to make one point: yes, for small values of n and large values of RAND_MAX, the modulo bias can be very small. But using a bias-inducing pattern means that you must consider the bias every time you calculate a random number and choose different patterns for different cases. And if you make the wrong choice, the bugs it introduces are subtle and almost impossible to unit test. Compared to just using the proper tool (such as arc4random_uniform), that's extra work, not less work. Doing more work and getting a worse solution is terrible engineering, especially when doing it right every time is easy on most platforms.
Unfortunately, the implementations of the solution are all incorrect or less efficient than they should be. (Each solution has various comments explaining the problems, but none of the solutions have been fixed to address them.) This is likely to confuse the casual answer-seeker, so I'm providing a known-good implementation here.
Again, the best solution is just to use arc4random_uniform on platforms that provide it, or a similar ranged solution for your platform (such as Random.nextInt on Java). It will do the right thing at no code cost to you. This is almost always the correct call to make.
If you don't have arc4random_uniform, then you can use the power of opensource to see exactly how it is implemented on top of a wider-range RNG (ar4random in this case, but a similar approach could also work on top of other RNGs).
Here is the OpenBSD implementation:
/*
* Calculate a uniformly distributed random number less than upper_bound
* avoiding "modulo bias".
*
* Uniformity is achieved by generating new random numbers until the one
* returned is outside the range [0, 2**32 % upper_bound). This
* guarantees the selected random number will be inside
* [2**32 % upper_bound, 2**32) which maps back to [0, upper_bound)
* after reduction modulo upper_bound.
*/
u_int32_t
arc4random_uniform(u_int32_t upper_bound)
{
u_int32_t r, min;
if (upper_bound < 2)
return 0;
/* 2**32 % x == (2**32 - x) % x */
min = -upper_bound % upper_bound;
/*
* This could theoretically loop forever but each retry has
* p > 0.5 (worst case, usually far better) of selecting a
* number inside the range we need, so it should rarely need
* to re-roll.
*/
for (;;) {
r = arc4random();
if (r >= min)
break;
}
return r % upper_bound;
}
It is worth noting the latest commit comment on this code for those who need to implement similar things:
Change arc4random_uniform() to calculate 2**32 % upper_bound as
-upper_bound % upper_bound. Simplifies the code and makes it the
same on both ILP32 and LP64 architectures, and also slightly faster on
LP64 architectures by using a 32-bit remainder instead of a 64-bit
remainder.
Pointed out by Jorden Verwer on tech#
ok deraadt; no objections from djm or otto
The Java implementation is also easily findable (see previous link):
public int nextInt(int n) {
if (n <= 0)
throw new IllegalArgumentException("n must be positive");
if ((n & -n) == n) // i.e., n is a power of 2
return (int)((n * (long)next(31)) >> 31);
int bits, val;
do {
bits = next(31);
val = bits % n;
} while (bits - val + (n-1) < 0);
return val;
}
Definition
Modulo Bias is the inherent bias in using modulo arithmetic to reduce an output set to a subset of the input set. In general, a bias exists whenever the mapping between the input and output set is not equally distributed, as in the case of using modulo arithmetic when the size of the output set is not a divisor of the size of the input set.
This bias is particularly hard to avoid in computing, where numbers are represented as strings of bits: 0s and 1s. Finding truly random sources of randomness is also extremely difficult, but is beyond the scope of this discussion. For the remainder of this answer, assume that there exists an unlimited source of truly random bits.
Problem Example
Let's consider simulating a die roll (0 to 5) using these random bits. There are 6 possibilities, so we need enough bits to represent the number 6, which is 3 bits. Unfortunately, 3 random bits yields 8 possible outcomes:
000 = 0, 001 = 1, 010 = 2, 011 = 3
100 = 4, 101 = 5, 110 = 6, 111 = 7
We can reduce the size of the outcome set to exactly 6 by taking the value modulo 6, however this presents the modulo bias problem: 110 yields a 0, and 111 yields a 1. This die is loaded.
Potential Solutions
Approach 0:
Rather than rely on random bits, in theory one could hire a small army to roll dice all day and record the results in a database, and then use each result only once. This is about as practical as it sounds, and more than likely would not yield truly random results anyway (pun intended).
Approach 1:
Instead of using the modulus, a naive but mathematically correct solution is to discard results that yield 110 and 111 and simply try again with 3 new bits. Unfortunately, this means that there is a 25% chance on each roll that a re-roll will be required, including each of the re-rolls themselves. This is clearly impractical for all but the most trivial of uses.
Approach 2:
Use more bits: instead of 3 bits, use 4. This yield 16 possible outcomes. Of course, re-rolling anytime the result is greater than 5 makes things worse (10/16 = 62.5%) so that alone won't help.
Notice that 2 * 6 = 12 < 16, so we can safely take any outcome less than 12 and reduce that modulo 6 to evenly distribute the outcomes. The other 4 outcomes must be discarded, and then re-rolled as in the previous approach.
Sounds good at first, but let's check the math:
4 discarded results / 16 possibilities = 25%
In this case, 1 extra bit didn't help at all!
That result is unfortunate, but let's try again with 5 bits:
32 % 6 = 2 discarded results; and
2 discarded results / 32 possibilities = 6.25%
A definite improvement, but not good enough in many practical cases. The good news is, adding more bits will never increase the chances of needing to discard and re-roll. This holds not just for dice, but in all cases.
As demonstrated however, adding an 1 extra bit may not change anything. In fact if we increase our roll to 6 bits, the probability remains 6.25%.
This begs 2 additional questions:
If we add enough bits, is there a guarantee that the probability of a discard will diminish?
How many bits are enough in the general case?
General Solution
Thankfully the answer to the first question is yes. The problem with 6 is that 2^x mod 6 flips between 2 and 4 which coincidentally are a multiple of 2 from each other, so that for an even x > 1,
[2^x mod 6] / 2^x == [2^(x+1) mod 6] / 2^(x+1)
Thus 6 is an exception rather than the rule. It is possible to find larger moduli that yield consecutive powers of 2 in the same way, but eventually this must wrap around, and the probability of a discard will be reduced.
Without offering further proof, in general using double the number
of bits required will provide a smaller, usually insignificant,
chance of a discard.
Proof of Concept
Here is an example program that uses OpenSSL's libcrypo to supply random bytes. When compiling, be sure to link to the library with -lcrypto which most everyone should have available.
#include <iostream>
#include <assert.h>
#include <limits>
#include <openssl/rand.h>
volatile uint32_t dummy;
uint64_t discardCount;
uint32_t uniformRandomUint32(uint32_t upperBound)
{
assert(RAND_status() == 1);
uint64_t discard = (std::numeric_limits<uint64_t>::max() - upperBound) % upperBound;
RAND_bytes((uint8_t*)(&randomPool), sizeof(randomPool));
while(randomPool > (std::numeric_limits<uint64_t>::max() - discard)) {
RAND_bytes((uint8_t*)(&randomPool), sizeof(randomPool));
++discardCount;
}
return randomPool % upperBound;
}
int main() {
discardCount = 0;
const uint32_t MODULUS = (1ul << 31)-1;
const uint32_t ROLLS = 10000000;
for(uint32_t i = 0; i < ROLLS; ++i) {
dummy = uniformRandomUint32(MODULUS);
}
std::cout << "Discard count = " << discardCount << std::endl;
}
I encourage playing with the MODULUS and ROLLS values to see how many re-rolls actually happen under most conditions. A sceptical person may also wish to save the computed values to file and verify the distribution appears normal.
Mark's Solution (The accepted solution) is Nearly Perfect.
int x;
do {
x = rand();
} while (x >= (RAND_MAX - RAND_MAX % n));
x %= n;
edited Mar 25 '16 at 23:16
Mark Amery 39k21170211
However, it has a caveat which discards 1 valid set of outcomes in any scenario where RAND_MAX (RM) is 1 less than a multiple of N (Where N = the Number of possible valid outcomes).
ie, When the 'count of values discarded' (D) is equal to N, then they are actually a valid set (V), not an invalid set (I).
What causes this is at some point Mark loses sight of the difference between N and Rand_Max.
N is a set who's valid members are comprised only of Positive Integers, as it contains a count of responses that would be valid. (eg: Set N = {1, 2, 3, ... n } )
Rand_max However is a set which ( as defined for our purposes ) includes any number of non-negative integers.
In it's most generic form, what is defined here as Rand Max is the Set of all valid outcomes, which could theoretically include negative numbers or non-numeric values.
Therefore Rand_Max is better defined as the set of "Possible Responses".
However N operates against the count of the values within the set of valid responses, so even as defined in our specific case, Rand_Max will be a value one less than the total number it contains.
Using Mark's Solution, Values are Discarded when: X => RM - RM % N
EG:
Ran Max Value (RM) = 255
Valid Outcome (N) = 4
When X => 252, Discarded values for X are: 252, 253, 254, 255
So, if Random Value Selected (X) = {252, 253, 254, 255}
Number of discarded Values (I) = RM % N + 1 == N
IE:
I = RM % N + 1
I = 255 % 4 + 1
I = 3 + 1
I = 4
X => ( RM - RM % N )
255 => (255 - 255 % 4)
255 => (255 - 3)
255 => (252)
Discard Returns $True
As you can see in the example above, when the value of X (the random number we get from the initial function) is 252, 253, 254, or 255 we would discard it even though these four values comprise a valid set of returned values.
IE: When the count of the values Discarded (I) = N (The number of valid outcomes) then a Valid set of return values will be discarded by the original function.
If we describe the difference between the values N and RM as D, ie:
D = (RM - N)
Then as the value of D becomes smaller, the Percentage of unneeded re-rolls due to this method increases at each natural multiplicative. (When RAND_MAX is NOT equal to a Prime Number this is of valid concern)
EG:
RM=255 , N=2 Then: D = 253, Lost percentage = 0.78125%
RM=255 , N=4 Then: D = 251, Lost percentage = 1.5625%
RM=255 , N=8 Then: D = 247, Lost percentage = 3.125%
RM=255 , N=16 Then: D = 239, Lost percentage = 6.25%
RM=255 , N=32 Then: D = 223, Lost percentage = 12.5%
RM=255 , N=64 Then: D = 191, Lost percentage = 25%
RM=255 , N= 128 Then D = 127, Lost percentage = 50%
Since the percentage of Rerolls needed increases the closer N comes to RM, this can be of valid concern at many different values depending on the constraints of the system running he code and the values being looked for.
To negate this we can make a simple amendment As shown here:
int x;
do {
x = rand();
} while (x > (RAND_MAX - ( ( ( RAND_MAX % n ) + 1 ) % n) );
x %= n;
This provides a more general version of the formula which accounts for the additional peculiarities of using modulus to define your max values.
Examples of using a small value for RAND_MAX which is a multiplicative of N.
Mark'original Version:
RAND_MAX = 3, n = 2, Values in RAND_MAX = 0,1,2,3, Valid Sets = 0,1 and 2,3.
When X >= (RAND_MAX - ( RAND_MAX % n ) )
When X >= 2 the value will be discarded, even though the set is valid.
Generalized Version 1:
RAND_MAX = 3, n = 2, Values in RAND_MAX = 0,1,2,3, Valid Sets = 0,1 and 2,3.
When X > (RAND_MAX - ( ( RAND_MAX % n ) + 1 ) % n )
When X > 3 the value would be discarded, but this is not a vlue in the set RAND_MAX so there will be no discard.
Additionally, in the case where N should be the number of values in RAND_MAX; in this case, you could set N = RAND_MAX +1, unless RAND_MAX = INT_MAX.
Loop-wise you could just use N = 1, and any value of X will be accepted, however, and put an IF statement in for your final multiplier. But perhaps you have code that may have a valid reason to return a 1 when the function is called with n = 1...
So it may be better to use 0, which would normally provide a Div 0 Error, when you wish to have n = RAND_MAX+1
Generalized Version 2:
int x;
if n != 0 {
do {
x = rand();
} while (x > (RAND_MAX - ( ( ( RAND_MAX % n ) + 1 ) % n) );
x %= n;
} else {
x = rand();
}
Both of these solutions resolve the issue with needlessly discarded valid results which will occur when RM+1 is a product of n.
The second version also covers the edge case scenario when you need n to equal the total possible set of values contained in RAND_MAX.
The modified approach in both is the same and allows for a more general solution to the need of providing valid random numbers and minimizing discarded values.
To reiterate:
The Basic General Solution which extends mark's example:
// Assumes:
// RAND_MAX is a globally defined constant, returned from the environment.
// int n; // User input, or externally defined, number of valid choices.
int x;
do {
x = rand();
} while (x > (RAND_MAX - ( ( ( RAND_MAX % n ) + 1 ) % n) ) );
x %= n;
The Extended General Solution which Allows one additional scenario of RAND_MAX+1 = n:
// Assumes:
// RAND_MAX is a globally defined constant, returned from the environment.
// int n; // User input, or externally defined, number of valid choices.
int x;
if n != 0 {
do {
x = rand();
} while (x > (RAND_MAX - ( ( ( RAND_MAX % n ) + 1 ) % n) ) );
x %= n;
} else {
x = rand();
}
In some languages ( particularly interpreted languages ) doing the calculations of the compare-operation outside of the while condition may lead to faster results as this is a one-time calculation no matter how many re-tries are required. YMMV!
// Assumes:
// RAND_MAX is a globally defined constant, returned from the environment.
// int n; // User input, or externally defined, number of valid choices.
int x; // Resulting random number
int y; // One-time calculation of the compare value for x
y = RAND_MAX - ( ( ( RAND_MAX % n ) + 1 ) % n)
if n != 0 {
do {
x = rand();
} while (x > y);
x %= n;
} else {
x = rand();
}
There are two usual complaints with the use of modulo.
one is valid for all generators. It is easier to see in a limit case. If your generator has a RAND_MAX which is 2 (that isn't compliant with the C standard) and you want only 0 or 1 as value, using modulo will generate 0 twice as often (when the generator generates 0 and 2) as it will generate 1 (when the generator generates 1). Note that this is true as soon as you don't drop values, whatever the mapping you are using from the generator values to the wanted one, one will occurs twice as often as the other.
some kind of generator have their less significant bits less random than the other, at least for some of their parameters, but sadly those parameter have other interesting characteristic (such has being able to have RAND_MAX one less than a power of 2). The problem is well known and for a long time library implementation probably avoid the problem (for instance the sample rand() implementation in the C standard use this kind of generator, but drop the 16 less significant bits), but some like to complain about that and you may have bad luck
Using something like
int alea(int n){
assert (0 < n && n <= RAND_MAX);
int partSize =
n == RAND_MAX ? 1 : 1 + (RAND_MAX-n)/(n+1);
int maxUsefull = partSize * n + (partSize-1);
int draw;
do {
draw = rand();
} while (draw > maxUsefull);
return draw/partSize;
}
to generate a random number between 0 and n will avoid both problems (and it avoids overflow with RAND_MAX == INT_MAX)
BTW, C++11 introduced standard ways to the the reduction and other generator than rand().
With a RAND_MAX value of 3 (in reality it should be much higher than that but the bias would still exist) it makes sense from these calculations that there is a bias:
1 % 2 = 1
2 % 2 = 0
3 % 2 = 1
random_between(1, 3) % 2 = more likely a 1
In this case, the % 2 is what you shouldn't do when you want a random number between 0 and 1. You could get a random number between 0 and 2 by doing % 3 though, because in this case: RAND_MAX is a multiple of 3.
Another method
There is much simpler but to add to other answers, here is my solution to get a random number between 0 and n - 1, so n different possibilities, without bias.
the number of bits (not bytes) needed to encode the number of possibilities is the number of bits of random data you'll need
encode the number from random bits
if this number is >= n, restart (no modulo).
Really random data is not easy to obtain, so why use more bits than needed.
Below is an example in Smalltalk, using a cache of bits from a pseudo-random number generator. I'm no security expert so use at your own risk.
next: n
| bitSize r from to |
n < 0 ifTrue: [^0 - (self next: 0 - n)].
n = 0 ifTrue: [^nil].
n = 1 ifTrue: [^0].
cache isNil ifTrue: [cache := OrderedCollection new].
cache size < (self randmax highBit) ifTrue: [
Security.DSSRandom default next asByteArray do: [ :byte |
(1 to: 8) do: [ :i | cache add: (byte bitAt: i)]
]
].
r := 0.
bitSize := n highBit.
to := cache size.
from := to - bitSize + 1.
(from to: to) do: [ :i |
r := r bitAt: i - from + 1 put: (cache at: i)
].
cache removeFrom: from to: to.
r >= n ifTrue: [^self next: n].
^r
Modulo reduction is a commonly seen way to make a random integer generator avoid the worst case of running forever.
When the range of possible integers is unknown, however, there is no way in general to "fix" this worst case of running forever without introducing bias. It's not just modulo reduction (rand() % n, discussed in the accepted answer) that will introduce bias this way, but also the "multiply-and-shift" reduction of Daniel Lemire, or if you stop rejecting an outcome after a set number of iterations. (To be clear, this doesn't mean there is no way to fix the bias issues present in pseudorandom generators. For example, even though modulo and other reductions are biased in general, they will have no issues with bias if the range of possible integers is a power of 2 and if the random generator produces unbiased random bits or blocks of them.)
The following answer of mine discusses the relationship between running time and bias in random generators, assuming we have a "true" random generator that can produce unbiased and independent random bits. The answer doesn't even involve the rand() function in C because it has many issues. Perhaps the most serious here is the fact that the C standard does not explicitly specify a particular distribution for the numbers returned by rand(), not even a uniform distribution.
How to generate a random integer in the range [0,n] from a stream of random bits without wasting bits?
As the accepted answer indicates, "modulo bias" has its roots in the low value of RAND_MAX. He uses an extremely small value of RAND_MAX (10) to show that if RAND_MAX were 10, then you tried to generate a number between 0 and 2 using %, the following outcomes would result:
rand() % 3 // if RAND_MAX were only 10, gives
output of rand() | rand()%3
0 | 0
1 | 1
2 | 2
3 | 0
4 | 1
5 | 2
6 | 0
7 | 1
8 | 2
9 | 0
So there are 4 outputs of 0's (4/10 chance) and only 3 outputs of 1 and 2 (3/10 chances each).
So it's biased. The lower numbers have a better chance of coming out.
But that only shows up so obviously when RAND_MAX is small. Or more specifically, when the number your are modding by is large compared to RAND_MAX.
A much better solution than looping (which is insanely inefficient and shouldn't even be suggested) is to use a PRNG with a much larger output range. The Mersenne Twister algorithm has a maximum output of 4,294,967,295. As such doing MersenneTwister::genrand_int32() % 10 for all intents and purposes, will be equally distributed and the modulo bias effect will all but disappear.
I just wrote a code for Von Neumann's Unbiased Coin Flip Method, that should theoretically eliminate any bias in the random number generation process. More info can be found at (http://en.wikipedia.org/wiki/Fair_coin)
int unbiased_random_bit() {
int x1, x2, prev;
prev = 2;
x1 = rand() % 2;
x2 = rand() % 2;
for (;; x1 = rand() % 2, x2 = rand() % 2)
{
if (x1 ^ x2) // 01 -> 1, or 10 -> 0.
{
return x2;
}
else if (x1 & x2)
{
if (!prev) // 0011
return 1;
else
prev = 1; // 1111 -> continue, bias unresolved
}
else
{
if (prev == 1)// 1100
return 0;
else // 0000 -> continue, bias unresolved
prev = 0;
}
}
}
I know there is a bit of limitations for a random number generation in C++ (can be non-uniform). How can I generate a number from 1 to 14620?
Thank you.
If you've got a c++0x environment, a close derivative of the boost lib is now standard:
#include <random>
#include <iostream>
int main()
{
std::uniform_int_distribution<> d(1, 14620);
std::mt19937 gen;
std::cout << d(gen) << '\n';
}
This will be fast, easy and high quality.
You didn't specify, but if you wanted floating point instead just sub in:
std::uniform_real_distribution<> d(1, 14620);
And if you needed a non-uniform distribution, you can build your own piece-wise constant or piece-wise linear distribution very easily.
A common approach is to use std::rand() with a modulo:
#include<cstdlib>
#include<ctime>
// ...
std::srand(std::time(0)); // needed once per program run
int r = std::rand() % 14620 + 1;
However, as #tenfour mentions in his answer, the modulo operator can disrupt the uniformity of values std::rand() returns. This is because the modulo translates the values it discards into valid values, and this translation might not be uniform. For instance, for n in [0, 10) the value n % 9 translates 9 to 0, so you can get zero by either a true zero or a 9 translated to zero. The other values have each only one chance to yield.
An alternative approach is to translate the random number from std::rand() to a floating-point value in the range [0, 1) and then translate and shift the value to within the range you desire.
int r = static_cast<double>(std::rand()) / RAND_MAX * 14620) + 1;
srand() / rand() are the functions you need, as others have answered.
The problem with % is that the result is decidedly non-uniform. To illustrate, imagine that rand() returns a range of 0-3. Here are hypothetical results of calling it 4000 times:
0 - 1000 times
1 - 1000 times
2 - 1000 times
3 - 1000 times
Now if you do the same sampling for (rand() % 3), you notice that the results would be like:
0 - 2000 times
1 - 1000 times
2 - 1000 times
Ouch! The more uniform solution is this:
int n = (int)(((((double)std::rand()) / RAND_MAX) * 14620) + 1);
Sorry for the sloppy code, but the idea is to scale it down properly to the range you want using floating point math, and convert to integer.
Use rand.
( rand() % 100 ) is in the range 0 to 99
( rand() % 100 + 1 ) is in the range 1 to 100
( rand() % 30 + 1985 ) is in the range 1985 to 2014
( rand() % 14620 + 1 ) is in the range 1 to 14620
EDIT:
As mentioned in the link, the randomizer should be seeded using srand before use. A common distinctive value to use is the result of a call to time.
As already said, you can use rand(). E.g.
int n = rand() % 14620 + 1;
does the job, but it is non-uniform.
That means some values (low values) will occur slightly more frequently. This is because rand() yields values in the range of 0 to RAND_MAX and RAND_MAX is generally not divisible by 14620. E.g. if RAND_MAX == 15000, then the number 1 would be twice as likely as the number 1000 because rand() == 0 and rand() == 14620 both yield n==1 but only rand()==999 makes n==1000 true.
However, if 14620 is much smaller than RAND_MAX, this effect is negligible. On my computer RAND_MAX is equal to 2147483647. If rand() yields uniform samples between 0 and RAND_MAX then, because 2147483647 % 14620 = 10327 and 2147483647 / 14620 = 146886, n would be between 1 and 10328 on average 146887 times while the numbers between 10329 and 14620 would occur on average 146886 times if you draw 2147483647 samples.
Not much of a difference if you ask me.
However, if RAND_MAX == 15000 it would make a difference as explained above.
In this case some earlier posts suggested to use
int n = (int)(((((double)std::rand()) / RAND_MAX) * 14620) + 1);
to make it 'more uniform'.
Note that this only changes the numbers that occur more frequently since rand() still returns 'only' RAND_MAX distinct values.
To make it really uniform, you would have to reject any integer form rand() if it is in the range between 14620*int(RAND_MAX/14620) and RAND_MAX and call rand() again.
In the example with RAND_MAX == 15000 you would reject any values of rand() between 14620 and 15000 and draw again.
For most application this is not necessary. I would worry more about the randomness of rand().
Here's a tutorial using the boost library http://www.boost.org/doc/libs/1_45_0/doc/html/boost_random/tutorial.html#boost_random.tutorial.generating_integers_in_a_range
The rand() function is not really the best Random generator, a better way would be by using CryptGenRandom().
This example should do do the trick:
#include <Windows.h>
// Random-Generator
HCRYPTPROV hProv;
INT Random() {
if (hProv == NULL) {
if (!CryptAcquireContext(&hProv, NULL, NULL, PROV_RSA_FULL, CRYPT_SILENT | CRYPT_VERIFYCONTEXT))
ExitProcess(EXIT_FAILURE);
}
int out;
CryptGenRandom(hProv, sizeof(out), (BYTE *)(&out));
return out & 0x7fffffff;
}
int main() {
int ri = Random() % 14620 + 1;
}
the modulus operator is the most important, you can apply a limit with this modulus, check this out:
// random numbers generation in C++ using builtin functions
#include <iostream>
using namespace std;
#include <iomanip>
using std::setw;
#include <cstdlib> // contains function prototype for rand
int main()
{
// loop 20 times
for ( int counter = 1; counter <= 20; counter++ ) {
// pick random number from 1 to 6 and output it
cout << setw( 10 ) << ( 1 + rand() % 6 );
// if counter divisible by 5, begin new line of output
if ( counter % 5 == 0 )
cout << endl;
}
return 0; // indicates successful termination
} // end main
I just saw this question and have no idea how to solve it. can you please provide me with algorithms , C++ codes or ideas?
This is a very simple problem. Given the value of N and K, you need to tell us the value of the binomial coefficient C(N,K). You may rest assured that K <= N and the maximum value of N is 1,000,000,000,000,000. Since the value may be very large, you need to compute the result modulo 1009.
Input
The first line of the input contains the number of test cases T, at most 1000. Each of the next T lines consists of two space separated integers N and K, where 0 <= K <= N and 1 <= N <= 1,000,000,000,000,000.
Output
For each test case, print on a new line, the value of the binomial coefficient C(N,K) modulo 1009.
Example
Input:
3
3 1
5 2
10 3
Output:
3
10
120
Notice that 1009 is a prime.
Now you can use Lucas' Theorem.
Which states:
Let p be a prime.
If n = a1a2...ar when written in base p and
if k = b1b2...br when written in base p
(pad with zeroes if required)
Then
(n choose k) modulo p = (a1 choose b1) * (a2 choose b2) * ... * (ar choose br) modulo p.
i.e. remainder of n choose k when divided by p is same as the remainder of
the product (a1 choose b1) * .... * (ar choose br) when divided by p.
Note: if bi > ai then ai choose bi is 0.
Thus your problem is reduced to finding the product modulo 1009 of at most log N/log 1009 numbers (number of digits of N in base 1009) of the form a choose b where a <= 1009 and b <= 1009.
This should make it easier even when N is close to 10^15.
Note:
For N=10^15, N choose N/2 is more than
2^(100000000000000) which is way
beyond an unsigned long long.
Also, the algorithm suggested by
Lucas' theorem is O(log N) which is
exponentially faster than trying to
compute the binomial coefficient
directly (even if you did a mod 1009
to take care of the overflow issue).
Here is some code for Binomial I had written long back, all you need to do is to modify it to do the operations modulo 1009 (there might be bugs and not necessarily recommended coding style):
class Binomial
{
public:
Binomial(int Max)
{
max = Max+1;
table = new unsigned int * [max]();
for (int i=0; i < max; i++)
{
table[i] = new unsigned int[max]();
for (int j = 0; j < max; j++)
{
table[i][j] = 0;
}
}
}
~Binomial()
{
for (int i =0; i < max; i++)
{
delete table[i];
}
delete table;
}
unsigned int Choose(unsigned int n, unsigned int k);
private:
bool Contains(unsigned int n, unsigned int k);
int max;
unsigned int **table;
};
unsigned int Binomial::Choose(unsigned int n, unsigned int k)
{
if (n < k) return 0;
if (k == 0 || n==1 ) return 1;
if (n==2 && k==1) return 2;
if (n==2 && k==2) return 1;
if (n==k) return 1;
if (Contains(n,k))
{
return table[n][k];
}
table[n][k] = Choose(n-1,k) + Choose(n-1,k-1);
return table[n][k];
}
bool Binomial::Contains(unsigned int n, unsigned int k)
{
if (table[n][k] == 0)
{
return false;
}
return true;
}
Binomial coefficient is one factorial divided by two others, although the k! term on the bottom cancels in an obvious way.
Observe that if 1009, (including multiples of it), appears more times in the numerator than the denominator, then the answer mod 1009 is 0. It can't appear more times in the denominator than the numerator (since binomial coefficients are integers), hence the only cases where you have to do anything are when it appears the same number of times in both. Don't forget to count multiples of (1009)^2 as two, and so on.
After that, I think you're just mopping up small cases (meaning small numbers of values to multiply/divide), although I'm not sure without a few tests. On the plus side 1009 is prime, so arithmetic modulo 1009 takes place in a field, which means that after casting out multiples of 1009 from both top and bottom, you can do the rest of the multiplication and division mod 1009 in any order.
Where there are non-small cases left, they will still involve multiplying together long runs of consecutive integers. This can be simplified by knowing 1008! (mod 1009). It's -1 (1008 if you prefer), since 1 ... 1008 are the p-1 non-zero elements of the prime field over p. Therefore they consist of 1, -1, and then (p-3)/2 pairs of multiplicative inverses.
So for example consider the case of C((1009^3), 200).
Imagine that the number of 1009s are equal (don't know if they are, because I haven't coded a formula to find out), so that this is a case requiring work.
On the top we have 201 ... 1008, which we'll have to calculate or look up in a precomputed table, then 1009, then 1010 ... 2017, 2018, 2019 ... 3026, 3027, etc. The ... ranges are all -1, so we just need to know how many such ranges there are.
That leaves 1009, 2018, 3027, which once we've cancelled them with 1009's from the bottom will just be 1, 2, 3, ... 1008, 1010, ..., plus some multiples of 1009^2, which again we'll cancel and leave ourselves with consecutive integers to multiply.
We can do something very similar with the bottom to compute the product mod 1009 of "1 ... 1009^3 - 200 with all the powers of 1009 divided out". That leaves us with a division in a prime field. IIRC that's tricky in principle, but 1009 is a small enough number that we can manage 1000 of them (the upper limit on the number of test cases).
Of course with k=200, there's an enormous overlap which could be cancelled more directly. That's what I meant by small cases and non-small cases: I've treated it like a non-small case, when in fact we could get away with just "brute-forcing" this one, by calculating ((1009^3-199) * ... * 1009^3) / 200!
I don't think you want to calculate C(n,k) and then reduce mod 1009. The biggest one, C(1e15,5e14) will require something like 1e16 bits ~ 1000 terabytes
Moreover executing the loop in snakiles answer 1e15 times seems like it might take a while.
What you might use is, if
n = n0 + n1*p + n2*p^2 ... + nd*p^d
m = m0 + m1*p + m2*p^2 ... + md*p^d
(where 0<=mi,ni < p)
then
C(n,m) = C(n0,m0) * C(n1,m1) *... * C(nd, nd) mod p
see, eg http://www.cecm.sfu.ca/organics/papers/granville/paper/binomial/html/binomial.html
One way would be to use pascal's triangle to build a table of all C(m,n) for 0<=m<=n<=1009.
psudo code for calculating nCk:
result = 1
for i=1 to min{K,N-K}:
result *= N-i+1
result /= i
return result
Time Complexity: O(min{K,N-K})
The loop goes from i=1 to min{K,N-K} instead of from i=1 to K, and that's ok because
C(k,n) = C(k, n-k)
And you can calculate the thing even more efficiently if you use the GammaLn function.
nCk = exp(GammaLn(n+1)-GammaLn(k+1)-GammaLn(n-k+1))
The GammaLn function is the natural logarithm of the Gamma function. I know there's an efficient algorithm to calculate the GammaLn function but that algorithm isn't trivial at all.
The following code shows how to obtain all the binomial coefficients for a given size 'n'. You could easily modify it to stop at a given k in order to determine nCk. It is computationally very efficient, it's simple to code, and works for very large n and k.
binomial_coefficient = 1
output(binomial_coefficient)
col = 0
n = 5
do while col < n
binomial_coefficient = binomial_coefficient * (n + 1 - (col + 1)) / (col + 1)
output(binomial_coefficient)
col = col + 1
loop
The output of binomial coefficients is therefore:
1
1 * (5 + 1 - (0 + 1)) / (0 + 1) = 5
5 * (5 + 1 - (1 + 1)) / (1 + 1) = 15
15 * (5 + 1 - (2 + 1)) / (2 + 1) = 15
15 * (5 + 1 - (3 + 1)) / (3 + 1) = 5
5 * (5 + 1 - (4 + 1)) / (4 + 1) = 1
I had found the formula once upon a time on Wikipedia but for some reason it's no longer there :(