Efficient random number generation with C++11 <random> - c++

I am trying to understand how the C++11 random number generation features are meant to be used. My concern is performance.
Suppose that we need to generate a series of random integers between 0..k, but k changes at every step. What is the best way to proceed?
Example:
for (int i=0; i < n; ++i) {
int k = i; // of course this is more complicated in practice
std::uniform_int_distribution<> dist(0, k);
int random_number = dist(engine);
// do something with random number
}
The distributions that the <random> header provides are very convenient. But they are opaque to the user, so I cannot easily predict how they will perform. It is not clear for example how much (if any) runtime overhead will be caused by the construction of dist above.
Instead I could have used something like
std::uniform_real_distribution<> dist(0.0, 1.0);
for (int i=0; i < n; ++i) {
int k = i; // of course this is more complicated in practice
int random_number = std::floor( (k+1)*dist(engine) );
// do something with random number
}
which avoids constructing a new object in each iteration.
Random numbers are often used in numerical simulations where performance is important. What is the best way to use <random> in these situations?
Please do no answer "profile it". Profiling is part of effective optimization, but so is a good understanding of how a library is meant to be used and the performance characteristics of that library. If the answer is that it depends on the standard library implementation, or that the only way to know is to profile it, then I would rather not use the distributions from <random> at all. Instead I can use my own implementation which will be transparent to me and much easier to optimize if/when necessary.

One thing you can do is to have a permanent distribution object so that you only create the param_type object each time like this:
template<typename Integral>
Integral randint(Integral min, Integral max)
{
using param_type =
typename std::uniform_int_distribution<Integral>::param_type;
// only create these once (per thread)
thread_local static std::mt19937 eng {std::random_device{}()};
thread_local static std::uniform_int_distribution<Integral> dist;
// presumably a param_type is cheaper than a uniform_int_distribution
return dist(eng, param_type{min, max});
}

For maximizing performance, first of all consider different PRNG, such as xorshift128+. It has been reported being more than twice as fast as mt19937 for 64-bit random numbers; see http://xorshift.di.unimi.it/. And it can be implemented with a few lines of code.
Moreover, if you don't need "perfectly balanced" uniform distribution and your k is much less than 2^64 (which likely is), I would suggest to write simply something as:
uint64_t temp = engine_64(); // generates 0 <= temp < 2^64
int random_number = temp % (k + 1); // crop temp to 0,...,k
Note, however, that integer division/modulo operations are not cheap. For example, on an Intel Haswell processor, they take 39-103 processor cycles for 64-bit numbers, which is likely much longer than calling an MT19937 or xorshift+ engine.

Related

Re-initializing random distribution

Is it reasonable to expect that a distribution from <random> re-initialized before each next number request behaves the same way as if it was initialized once? In other words, does this:
std::default_random_engine generator;
int p[10]={};
for (int i=0; i<nrolls; ++i) {
std::uniform_int_distribution<int> distribution(0,9);
int number = distribution(generator);
++p[number];
}
have the same distribution as that
std::uniform_int_distribution<int> distribution(0,9);
std::default_random_engine generator;
int p[10]={};
for (int i=0; i<nrolls; ++i) {
int number = distribution(generator);
++p[number];
}
I've checked that for uniform and normal distribution it empirically holds true. Can I expect it from every distribution in <random>?
I essentially do what your first implementation does. Construct one every time I need a distribution.
That said, yes, the behavior of the different distributions are guaranteed by the standard to behave in specific ways.
STL recommends, that since distributions are relatively cheap, don't worry about constructing one every time you need one or need a new range. He also says if you don't want to construct one every time, you can use the param member function to change the distribution range.
Microsoft Channel9 link if the above direct Youtube link dies (seek to 30 minutes in): https://channel9.msdn.com/Events/GoingNative/2013/rand-Considered-Harmful
EDIT
I was re-watching a CppCon talk from a few years ago that discusses this exact question. The result? Constructing distributions as local variables even inside loops is 4.5 times faster.

C++: How to generate random numbers while excluding numbers from a given cache

So in c++ I'm using mt19937 engine and the uniform_int_distribution in my random number generator like so:
#include <random>
#include <time.h>
int get_random(int lwr_lm, int upper_lm){
std::mt19937 mt(time(nullptr));
std::uniform_int_distribution<int> dist(lwr_lm, upper_lm);
return dist(mt);
}
What I need is to alter the above generator such that there is a cache that contains a number of integers I need to be excluded when I use the above generator over and over again.
How do I alter the above such that I can achieve this?
There are many ways to do it. A simple way would be to maintain your "excluded numbers" in a std::set and after each generation of a random number, check whether it is in the set and if it is then generate a new random number - repeat until you get a number that was not in the set, then return that.
Btw; while distributions are cheap to construct, engines are not. You don't want to re-construct your mt19937 every time the function is called, but instead create it once and then re-use it. You probably also want to use a better seed than the current time in seconds.
Are you 1) attempting to sample without replacement in the discrete interval? Or is it 2) a patchy distribution over the interval that says fairly constant?
If 1) you could use std::shuffle as per the answer here How to sample without replacement using c++ uniform_int_distribution
If 2) you could use std::discrete_distribution (element 0 corresponding to lwr_lm) and weight zero the numbers you don't want. Obviously the memory requirements are linear in upper_lm-lwr_lm so might not be practical if this is large
I would propose two similar solutions for the problem. They are based upon probabilistic structures, and provide you with the answer "potentially in cache" or "definitely not in cache". There are false positives but no false negatives.
Perfect hash function. There are many implementations, including one from GNU. Basically, run it on set of cache values, and use generated perfect hash functions to reject sampled values. You don't even need to maintain hash table, just function mapping random value to integer index. As soon as index is in the hash range, reject the number. Being perfect means you need only one call to check and result will tell you that number is in the set. There are potential collisions, so false positives are possible.
Bloom filter. Same idea, build filter with whatever bits per cache item you're willing to spare, and with quick check you either will get "possible in the cache" answer or clear negative. You could trade answer precision for memory and vice versa. False positives are possible
As mentioned by #virgesmith, in his answer, it might be better solution in function of your problem.
The method with a cache and uses it to filter future generation is inefficient for large range wiki.
Here I write a naive example with a different method, but you will be limited by your memory. You pick random number for a buffer and remove it for next iteration.
#include <random>
#include <time.h>
#include <iostream>
int get_random(int lwr_lm, int upper_lm, std::vector<int> &buff, std::mt19937 &mt){
if (buff.size() > 0) {
std::uniform_int_distribution<int> dist(0, buff.size()-1);
int tmp_index = dist(mt);
int tmp_value = buff[tmp_index];
buff.erase(buff.begin() + tmp_index);
return tmp_value;
} else {
return 0;
}
}
int main() {
// lower and upper limit for random distribution
int lower = 0;
int upper = 10;
// Random generator
std::mt19937 mt(time(nullptr));
// Buffer to filter and avoid duplication, Buffer contain all integer between lower and uper limit
std::vector<int> my_buffer(upper-lower);
std::iota(my_buffer.begin(), my_buffer.end(), lower);
for (int i = 0; i < 20; ++i) {
std::cout << get_random(lower, upper, my_buffer, mt) << std::endl;
}
return 0;
}
Edit: a cleaner solution here
It might not be the prettiest solution, but what's stopping you from maintaining that cache and checking existence before returning? It will slow down for large caches though.
#include <random>
#include <time.h>
#include <set>
std::set<int> cache;
int get_random(int lwr_lm, int upper_lm){
std::mt19937 mt(time(nullptr));
std::uniform_int_distribution<int> dist(lwr_lm, upper_lm);
auto r = dist(mt);
while(cache.find(r) != cache.end())
r = dist(mt);
return r;
}

Generating Gaussian Noise

I created a function that is suppose to generate a set of normal random numbers from 0 to 1. Although, it seems that each time I run the function the output is the same. I am not sure what is wrong.
Here is the code:
MatrixXd generateGaussianNoise(int n, int m){
MatrixXd M(n,m);
normal_distribution<double> nd(0.0, 1.0);
random_device rd;
mt19937 gen(rd());
for(int i = 0; i < n; i++){
for(int j = 0; j < m; j++){
M(i,j) = nd(gen);
}
}
return M;
}
The output when n = 4 and m = 1 is
0.414089
0.225568
0.413464
2.53933
I used the Eigen library for this, I am just wondering why each time I run it produces the same numbers.
From:
http://en.cppreference.com/w/cpp/numeric/random/random_device
std::random_device may be implemented in terms of an implementation-defined pseudo-random number engine if a non-deterministic source (e.g. a hardware device) is not available to the implementation. In this case each std::random_device object may generate the same number sequence.
Thus, I think you should look into what library stack you are actually using here, and what's known about random_device in your specific implementation.
I realize that this then might in fact be a duplicate of "Why do I get the same sequence for every run with std::random_device with mingw gcc4.8.1?".
Furthermore, it at least used to be that initializating a new mt19937 instance would be kind of expensive. Thus, you have performance reasons in addition to quality of randomness to not re-initalize both your random_device and mt19937 instance for every function call. I would go for some kind of singleton here, unless you have very clear constraints (building in a library, unclear concurrency) that would make that an unuistable choice.

Best place to initialise random generator

In my program I use a random number generator quite a lot. I believe the general rule is that you should define things as close to the place where they're "called", but does this also hold true for random number generators?
For example, in my code I have the choice between:
std::random_device rd;
std::mt19937 rng(rd());
std::uniform_int_distribution<int> uni(-2147483647, 2147483646);
lots of code
for (i = 0; i < 10000; i++)
{
variable x = uni(rng);
}
Or
lots of code
for (i = 0; i < 10000; i++)
{
std::random_device rd;
std::mt19937 rng(rd());
std::uniform_int_distribution<int> uni(-2147483647, 2147483646);
variable x = uni(rng);
}
I would say the first method is faster, but I've gotten a bit confused due to reading many threads in which it is stated to always place everything as close to the place where it's called.
In this case, it's much better to create the RNG outside your loop:
std::random_device rd;
std::mt19937 rng(rd());
std::uniform_int_distribution<int> uni(-2147483647, 2147483646);
for (i = 0; i < 10000; i++)
{
variable x = uni(rng);
}
The reason for this has little to do with performance (although it will likely perform better, too). The reason is to do with correctness:
You're initialising a new random sequence each time through the loop, and reading just one value. Instead, you should be initialising the sequence just once, and consuming many values from it. Initialise outside the loop, and consume within the loop.
On the performance side, reading from a std::random_device is much slower than taking the next value from a PRNG such as std::mt19937. Doing this just once, outside the loop, will save a lot of time. Further, the std::mt19937 PRNG has a large state (624 integers). It generates this initial state from the value passed to its constructor. Again, doing this just once will give you a performance boost.
Of course, initialising outside the loop has the advantage of also being the correct usage model for the standard RNGs.
The reason is, when you located your random generator definings on top of your code, they will become global and they will be defined automatically when you first hit the "Run" button. If you are using those variables in more than one place, probably it would be the best idea. But if you are not, you don't need it. Because in some scenarios, they might not even called. Anyway, this suggestion is for class or method usages.
However, from what I see, you are going to use that number in a for loop, which will cause your computer to run below code 1000 times.
std::random_device rd;
std::mt19937 rng(rd());
std::uniform_int_distribution<int> uni(-2147483647, 2147483646);
That is unnecessary, and useless. I beleive your first code will work better on performance side.

How to use <random> to replace rand()?

C++11 introduced the header <random> with declarations for random number engines and random distributions. That's great - time to replace those uses of rand() which is often problematic in various ways. However, it seems far from obvious how to replace
srand(n);
// ...
int r = rand();
Based on the declarations it seems a uniform distribution can be built something like this:
std::default_random_engine engine;
engine.seed(n);
std::uniform_int_distribution<> distribution;
auto rand = [&](){ return distribution(engine); }
This approach seems rather involved and is surely something I won't remember unlike the use of srand() and rand(). I'm aware of N4531 but even that still seems to be quite involved.
Is there a reasonably simple way to replace srand() and rand()?
Is there a reasonably simple way to replace srand() and rand()?
Full disclosure: I don't like rand(). It's bad, and it's very easily abused.
The C++11 random library fills in a void that has been lacking for a long, long time. The problem with high quality random libraries is that they're oftentimes hard to use. The C++11 <random> library represents a huge step forward in this regard. A few lines of code and I have a very nice generator that behaves very nicely and that easily generates random variates from many different distributions.
Given the above, my answer to you is a bit heretical. If rand() is good enough for your needs, use it. As bad as rand() is (and it is bad), removing it would represent a huge break with the C language. Just make sure that the badness of rand() truly is good enough for your needs.
C++14 didn't deprecate rand(); it only deprecated functions in the C++ library that use rand(). While C++17 might deprecate rand(), it won't delete it. That means you have several more years before rand() disappears. The odds are high that you will have retired or switched to a different language by the time the C++ committee finally does delete rand() from the C++ standard library.
I'm creating random inputs to benchmark different implementations of std::sort() using something along the lines of std::vector<int> v(size); std::generate(v.begin(), v.end(), std::rand);
You don't need a cryptographically secure PRNG for that. You don't even need Mersenne Twister. In this particular case, rand() probably is good enough for your needs.
Update
There is a nice simple replacement for rand() and srand() in the C++11 random library: std::minstd_rand.
#include <random>
#include <iostream>
int main ()
{
std:: minstd_rand simple_rand;
// Use simple_rand.seed() instead of srand():
simple_rand.seed(42);
// Use simple_rand() instead of rand():
for (int ii = 0; ii < 10; ++ii)
{
std::cout << simple_rand() << '\n';
}
}
The function std::minstd_rand::operator()() returns a std::uint_fast32_t. However, the algorithm restricts the result to between 1 and 231-2, inclusive. This means the result will always convert safely to a std::int_fast32_t (or to an int if int is at least 32 bits long).
How about randutils by Melissa O'Neill of pcg-random.org?
From the introductory blog post:
randutils::mt19937_rng rng;
std::cout << "Greetings from Office #" << rng.uniform(1,17)
<< " (where we think PI = " << rng.uniform(3.1,3.2) << ")\n\n"
<< "Our office morale is " << rng.uniform('A','D') << " grade\n";
Assuming you want the behavior of the C-style rand and srand functions, including their quirkiness, but with good random, this is the closest I could get.
#include <random>
#include <cstdlib> // RAND_MAX (might be removed soon?)
#include <climits> // INT_MAX (use as replacement?)
namespace replacement
{
constexpr int rand_max {
#ifdef RAND_MAX
RAND_MAX
#else
INT_MAX
#endif
};
namespace detail
{
inline std::default_random_engine&
get_engine() noexcept
{
// Seeding with 1 is silly, but required behavior
static thread_local auto rndeng = std::default_random_engine(1);
return rndeng;
}
inline std::uniform_int_distribution<int>&
get_distribution() noexcept
{
static thread_local auto rnddst = std::uniform_int_distribution<int> {0, rand_max};
return rnddst;
}
} // namespace detail
inline int
rand() noexcept
{
return detail::get_distribution()(detail::get_engine());
}
inline void
srand(const unsigned seed) noexcept
{
detail::get_engine().seed(seed);
detail::get_distribution().reset();
}
inline void
srand()
{
std::random_device rnddev {};
srand(rnddev());
}
} // namespace replacement
The replacement::* functions can be used exactly like their std::* counterparts from <cstdlib>. I have added a srand overload that takes no arguments and seeds the engine with a “real” random number obtained from a std::random_device. How “real” that randomness will be is of course implementation defined.
The engine and the distribution are held as thread_local static instances so they carry state across multiple calls but still allow different threads to observe predictable sequences. (It's also a performance gain because you don't need to re-construct the engine or use locks and potentially trash other people's cashes.)
I've used std::default_random_engine because you did but I don't like it very much. The Mersenne Twister engines (std::mt19937 and std::mt19937_64) produce much better “randomness” and, surprisingly, have also been observed to be faster. I don't think that any compliant program must rely on std::rand being implemented using any specific kind of pseudo random engine. (And even if it did, implementations are free to define std::default_random_engine to whatever they like so you'd have to use something like std::minstd_rand to be sure.)
Abusing the fact that engines return values directly
All engines defined in <random> has an operator()() that can be used to retrieve the next generated value, as well as advancing the internal state of the engine.
std::mt19937 rand (seed); // or an engine of your choosing
for (int i = 0; i < 10; ++i) {
unsigned int x = rand ();
std::cout << x << std::endl;
}
It shall however be noted that all engines return a value of some unsigned integral type, meaning that they can potentially overflow a signed integral (which will then lead to undefined-behavior).
If you are fine with using unsigned values everywhere you retrieve a new value, the above is an easy way to replace usage of std::srand + std::rand.
Note: Using what has been described above might lead to some values having a higher chance of being returned than others, due to the fact that the result_type of the engine not having a max value that is an even multiple of the highest value that can be stored in the destination type.
If you have not worried about this in the past — when using something like rand()%low+high — you should not worry about it now.
Note: You will need to make sure that the std::engine-type::result_type is at least as large as your desired range of values (std::mt19937::result_type is uint_fast32_t).
If you only need to seed the engine once
There is no need to first default-construct a std::default_random_engine (which is just a typedef for some engine chosen by the implementation), and later assigning a seed to it; this could be done all at once by using the appropriate constructor of the random-engine.
std::random-engine-type engine (seed);
If you however need to re-seed the engine, using std::random-engine::seed is the way to do it.
If all else fails; create a helper-function
Even if the code you have posted looks slightly complicated, you are only meant to write it once.
If you find yourself in a situation where you are tempted to just copy+paste what you have written to several places in your code it is recommended, as always when doing copy+pasting; introduce a helper-function.
Intentionally left blank, see other posts for example implementations.
You can create a simple function like this:
#include <random>
#include <iostream>
int modernRand(int n) {
std::random_device rd;
std::mt19937 gen(rd());
std::uniform_int_distribution<> dis(0, n);
return dis(gen);
}
And later use it like this:
int myRandValue = modernRand(n);
As mentioned here