Random Engine Differences - c++

The C++11 standard specifies a number of different engines for random number generation: linear_congruential_engine, mersenne_twister_engine, subtract_with_carry_engine and so on. Obviously, this is a large change from the old usage of std::rand.
Obviously, one of the major benefits of (at least some) of these engines is the massively increased period length (it's built into the name for std::mt19937).
However, the differences between the engines is less clear. What are the strengths and weaknesses of the different engines? When should one be used over the other? Is there a sensible default that should generally be preferred?

From the explanations below, a linear engine seems to be faster but less random while the Mersenne Twister has a higher complexity and randomness. Subtract-with-carry random number engine is an improvement to the linear engine and it is definitely more random. In the last reference, it is stated that Mersenne Twister has higher complexity than the Subtract-with-carry random number engine.
Linear congruential random number engine
A pseudo-random number generator engine that produces unsigned integer numbers.
This is the simplest generator engine in the standard library. Its state is a single integer value, with the following transition algorithm:
x = (ax+c) mod m
Where x is the current state value, a and c are their respective template parameters, and m is its respective template parameter if this is greater than 0, or numerics_limits<UIntType>::max() + 1, otherwise.
Its generation algorithm is a direct copy of the state value.
This makes it an extremely efficient generator in terms of processing and memory consumption, but producing numbers with varying degrees of serial correlation, depending on the specific parameters used.
The random numbers generated by linear_congruential_engine have a period of m.
Mersenne twister random number engine
A pseudo-random number generator engine that produces unsigned integer numbers in the closed interval [0,2^w-1].
The algorithm used by this engine is optimized to compute large series of numbers (such as in Monte Carlo experiments) with an almost uniform distribution in the range.
The engine has an internal state sequence of n integer elements, which is filled with a pseudo-random series generated on construction or by calling member function seed.
The internal state sequence becomes the source for n elements: When the state is advanced (for example, in order to produce a new random number), the engine alters the state sequence by twisting the current value using xor mask a on a mix of bits determined by parameter r that come from that value and from a value m elements away (see operator() for details).
The random numbers produced are tempered versions of these twisted values. The tempering is a sequence of shift and xor operations defined by parameters u, d, s, b, t, c and l applied on the selected state value (see operator()).
The random numbers generated by mersenne_twister_engine have a period equivalent to the mersenne number 2^((n-1)*w)-1.
Subtract-with-carry random number engine
A pseudo-random number generator engine that produces unsigned integer numbers.
The algorithm used by this engine is a lagged fibonacci generator, with a state sequence of r integer elements, plus one carry value.
Lagged Fibonacci generators have a maximum period of (2k - 1)*^(2M-1) if addition or subtraction is used. The initialization of LFGs is a very complex problem. The output of LFGs is very sensitive to initial conditions, and statistical defects may appear initially but also periodically in the output sequence unless extreme care is taken. Another potential problem with LFGs is that the mathematical theory behind them is incomplete, making it necessary to rely on statistical tests rather than theoretical performance.
And finally from the documentation of random:
The choice of which engine to use involves a number of tradeoffs: the linear congruential engine is moderately fast and has a very small storage requirement for state. The lagged Fibonacci generators are very fast even on processors without advanced arithmetic instruction sets, at the expense of greater state storage and sometimes less desirable spectral characteristics. The Mersenne Twister is slower and has greater state storage requirements but with the right parameters has the longest non-repeating sequence with the most desirable spectral characteristics (for a given definition of desirable).

I think that the point is that random generators have different properties, which can make them more suitable or not for a given problem.
The period length is one of the properties.
The quality of the random numbers can also be important.
The performance of the generator can also be an issue.
Depending on your need, you might take one generator or another one. E.g., if you need fast random numbers but do not really care for the quality, an LCG might be a good option. If you want better quality random numbers, the Mersenne Twister is probably a better option.
To help you making your choice, there are some standard tests and results (I definitely like the table p.29 of this paper).
EDIT: From the paper,
The LCG (LCG(***) in the paper) family are the fastest generators, but with the poorest quality.
The Mersenne Twister (MT19937) is a little bit slower, but yields better random numbers.
The substract with carry ( SWB(***), I think) are way slower, but can yield better random properties when well tuned.

As the other answers forget about ranlux, here is a small note by an AMD developer that recently ported it to OpenCL:
https://community.amd.com/thread/139236
RANLUX is also one of very few (the only one I know of actually) PRNGs that has a underlying theory explaining why it generates "random" numbers, and why they are good. Indeed, if the theory is correct (and I don't know of anyone who has disputed it), RANLUX at the highest luxury level produces completely decorrelated numbers down to the last bit, with no long-range correlations as long as we stay well below the period (10^171). Most other generators can say very little about their quality (like Mersenne Twister, KISS etc.) They must rely on passing statistical tests.
Physicists at CERN are fan of this PRNG. 'nuff said.

Some of the information in these other answers conflicts with my findings. I've run tests on Windows 8.1 using Visual Studio 2013, and consistently I've found mersenne_twister_engine to be but higher quality and significantly faster than either linear_congruential_engine or subtract_with_carry_engine. This leads me to believe, when the information in the other answers are taken into account, that the specific implementation of an engine has a significant impact on performance.
This is of great surprise to nobody, I'm sure, but it's not mentioned in the other answers where mersenne_twister_engine is said to be slower. I have no test results for other platforms and compilers, but with my configuration, mersenne_twister_engine is clearly the superior choice when considering period, quality, and speed performance. I have not profiled memory usage, so I cannot speak to the space requirement property.
Here's the code I'm using to test with (to make portable, you should only have to replace the windows.h QueryPerformanceXxx() API calls with an appropriate timing mechanism):
// compile with: cl.exe /EHsc
#include <random>
#include <iostream>
#include <windows.h>
using namespace std;
void test_lc(const int a, const int b, const int s) {
/*
typedef linear_congruential_engine<unsigned int, 48271, 0, 2147483647> minstd_rand;
*/
minstd_rand gen(1729);
uniform_int_distribution<> distr(a, b);
for (int i = 0; i < s; ++i) {
distr(gen);
}
}
void test_mt(const int a, const int b, const int s) {
/*
typedef mersenne_twister_engine<unsigned int, 32, 624, 397,
31, 0x9908b0df,
11, 0xffffffff,
7, 0x9d2c5680,
15, 0xefc60000,
18, 1812433253> mt19937;
*/
mt19937 gen(1729);
uniform_int_distribution<> distr(a, b);
for (int i = 0; i < s; ++i) {
distr(gen);
}
}
void test_swc(const int a, const int b, const int s) {
/*
typedef subtract_with_carry_engine<unsigned int, 24, 10, 24> ranlux24_base;
*/
ranlux24_base gen(1729);
uniform_int_distribution<> distr(a, b);
for (int i = 0; i < s; ++i) {
distr(gen);
}
}
int main()
{
int a_dist = 0;
int b_dist = 1000;
int samples = 100000000;
cout << "Testing with " << samples << " samples." << endl;
LARGE_INTEGER ElapsedTime;
double ElapsedSeconds = 0;
LARGE_INTEGER Frequency;
QueryPerformanceFrequency(&Frequency);
double TickInterval = 1.0 / ((double) Frequency.QuadPart);
LARGE_INTEGER StartingTime;
LARGE_INTEGER EndingTime;
QueryPerformanceCounter(&StartingTime);
test_lc(a_dist, b_dist, samples);
QueryPerformanceCounter(&EndingTime);
ElapsedTime.QuadPart = EndingTime.QuadPart - StartingTime.QuadPart;
ElapsedSeconds = ElapsedTime.QuadPart * TickInterval;
cout << "linear_congruential_engine time: " << ElapsedSeconds << endl;
QueryPerformanceCounter(&StartingTime);
test_mt(a_dist, b_dist, samples);
QueryPerformanceCounter(&EndingTime);
ElapsedTime.QuadPart = EndingTime.QuadPart - StartingTime.QuadPart;
ElapsedSeconds = ElapsedTime.QuadPart * TickInterval;
cout << " mersenne_twister_engine time: " << ElapsedSeconds << endl;
QueryPerformanceCounter(&StartingTime);
test_swc(a_dist, b_dist, samples);
QueryPerformanceCounter(&EndingTime);
ElapsedTime.QuadPart = EndingTime.QuadPart - StartingTime.QuadPart;
ElapsedSeconds = ElapsedTime.QuadPart * TickInterval;
cout << "subtract_with_carry_engine time: " << ElapsedSeconds << endl;
}
Output:
Testing with 100000000 samples.
linear_congruential_engine time: 10.0821
mersenne_twister_engine time: 6.11615
subtract_with_carry_engine time: 9.26676

I just saw this answer from Marnos and decided to test it myself. I used std::chono::high_resolution_clock to time 100000 samples 100 times to produce an average. I measured everything in std::chrono::nanoseconds and ended up with different results:
std::minstd_rand had an average of 28991658 nanoseconds
std::mt19937 had an average of 29871710 nanoseconds
ranlux48_base had an average of 29281677 nanoseconds
This is on a Windows 7 machine. Compiler is Mingw-Builds 4.8.1 64bit. This is obviously using the C++11 flag and no optimisation flags.
When I turn on -O3 optimisations, the std::minstd_rand and ranlux48_base actually run faster than what the implementation of high_precision_clock can measure; however std::mt19937 still takes 730045 nanoseconds, or 3/4 of a second.
So, as he said, it's implementation specific, but at least in GCC the average time seems to stick to what the descriptions in the accepted answer say. Mersenne Twister seems to benefit the least from optimizations, whereas the other two really just throw out the random numbers unbelieveably fast once you factor in compiler optimizations.
As an aside, I'd been using Mersenne Twister engine in my noise generation library (it doesn't precompute gradients), so I think I'll switch to one of the others to really see some speed improvements. In my case, the "true" randomness doesn't matter.
Code:
#include <iostream>
#include <chrono>
#include <random>
using namespace std;
using namespace std::chrono;
int main()
{
minstd_rand linearCongruentialEngine;
mt19937 mersenneTwister;
ranlux48_base subtractWithCarry;
uniform_real_distribution<float> distro;
int numSamples = 100000;
int repeats = 100;
long long int avgL = 0;
long long int avgM = 0;
long long int avgS = 0;
cout << "results:" << endl;
for(int j = 0; j < repeats; ++j)
{
cout << "start of sequence: " << j << endl;
auto start = high_resolution_clock::now();
for(int i = 0; i < numSamples; ++i)
distro(linearCongruentialEngine);
auto stop = high_resolution_clock::now();
auto L = duration_cast<nanoseconds>(stop-start).count();
avgL += L;
cout << "Linear Congruential:\t" << L << endl;
start = high_resolution_clock::now();
for(int i = 0; i < numSamples; ++i)
distro(mersenneTwister);
stop = high_resolution_clock::now();
auto M = duration_cast<nanoseconds>(stop-start).count();
avgM += M;
cout << "Mersenne Twister:\t" << M << endl;
start = high_resolution_clock::now();
for(int i = 0; i < numSamples; ++i)
distro(subtractWithCarry);
stop = high_resolution_clock::now();
auto S = duration_cast<nanoseconds>(stop-start).count();
avgS += S;
cout << "Subtract With Carry:\t" << S << endl;
}
cout << setprecision(10) << "\naverage:\nLinear Congruential: " << (long double)(avgL/repeats)
<< "\nMersenne Twister: " << (long double)(avgM/repeats)
<< "\nSubtract with Carry: " << (long double)(avgS/repeats) << endl;
}

Its a trade-off really. A PRNG like Mersenne Twister is better because it has extremely large period and other good statistical properties.
But a large period PRNG takes up more memory (for maintaining the internal state) and also takes more time for generating a random number (due to complex transitions and post processing).
Choose a PNRG depending on the needs of your application. When in doubt use Mersenne Twister, its the default in many tools.

In general, mersenne twister is the best (and fastest) RNG, but it requires some space (about 2.5 kilobytes). Which one suits your need depends on how many times you need to instantiate the generator object. (If you need to instantiate it only once, or a few times, then MT is the one to use. If you need to instantiate it millions of times, then perhaps something smaller.)
Some people report that MT is slower than some of the others. According to my experiments, this depends a lot on your compiler optimization settings. Most importantly the -march=native setting may make a huge difference, depending on your host architecture.
I ran a small program to test the speed of different generators, and their sizes, and got this:
std::mt19937 (2504 bytes): 1.4714 s
std::mt19937_64 (2504 bytes): 1.50923 s
std::ranlux24 (120 bytes): 16.4865 s
std::ranlux48 (120 bytes): 57.7741 s
std::minstd_rand (4 bytes): 1.04819 s
std::minstd_rand0 (4 bytes): 1.33398 s
std::knuth_b (1032 bytes): 1.42746 s

Related

Why it is appropriate to use `std::uniform_real_distribution`?

I'm trying to write Metropolis Monte Carlo simulation code.
Since the simulation will be very long, I'd like to think seriously about the performance for generating random numbers in [0, 1].
So I decided to check the performance of two methods by the following code:
#include <cfloat>
#include <chrono>
#include <iostream>
#include <random>
int main()
{
constexpr auto Ntry = 5000000;
std::mt19937 mt(123);
std::uniform_real_distribution<double> dist(0.0, std::nextafter(1.0, DBL_MAX));
double test1, test2;
// method 1
auto start1 = std::chrono::system_clock::now();
for (int i=0; i<Ntry; i++) {
test1 = dist(mt);
}
auto end1 = std::chrono::system_clock::now();
auto elapsed1 = std::chrono::duration_cast<std::chrono::microseconds>(end1-start1).count();
std::cout << elapsed1 << std::endl;
// method 2
auto start2 = std::chrono::system_clock::now();
for (int i=0; i<Ntry; i++) {
test2 = 1.0*mt() / mt.max();
}
auto end2 = std::chrono::system_clock::now();
auto elapsed2 = std::chrono::duration_cast<std::chrono::microseconds>(end2-start2).count();
std::cout << elapsed2 << std::endl;
}
Then the result is
295489 micro sec for method 1
79884 micro sec for method 2
I understand that there are many posts that recommend to use std::uniform_real_distribution.
But performance-wise, it is tempting to use the latter as this result shows.
Would you tell me what is the point of using std::uniform_real_distribution?
What is the disadvantage of using 1.0*mt() / mt.max()?
And in the current purpose, is it acceptable to use 1.0*mt() / mt.max() instead?
Edit:
I compiled this code with g++-11 test.cpp.
When I compile with -O3 flag, the result is qualitatively same (the method 1 is approx. 1.8 times slower).
I would like to discuss what is the advantage of the widely-used method.
I do concern the trend of performances, but specific performance comparison is out of my scope.
You use the standard random library because it is extremely difficult to do numerical calculations correctly and you don't want the burden of proving and maintaining your own random library.
Case in point, your random distribution is wrong. std::mt19937 produces 32-bit integers, yet you're expecting a double, which has a 53-bit significand (usually). There are values in the range [0, 1] that you will never obtain from 1.0*mt() / mt::max().
Your testing methodology is flawed. You don't use the result that you produce, so a smart optimiser may simply skip producing a result.
Would you tell me what is the point of using std::uniform_real_distribution?
The clue is in the name. It produces a uniform distribution.
Furthermore, it allows you to specify the minimum and maximum between which you want the distribution to lie.
What is the disadvantage of using 1.0*mt() / mt.max()?
You cannot specify a minimum and a maximum.
It produces a less uniform distribution.
It produces less randomness.
is it acceptable to use 1.0*mt() / mt.max() instead?
In some use cases, it could be acceptable. In some other cases, it isn't acceptable. In the rest, it won't matter.

Which machines support nondeterministic random_device?

I need to obtain data from different C++ random number generation algorithms, and for that purpose I created some programs. Some of them use pseudo-random number generators and others use random_device (nondeterministic random number generator). The following program belongs to the second group:
#include <iostream>
#include <vector>
#include <cmath>
#include <random>
using namespace std;
const int N = 5000;
const int M = 1000000;
const int VALS = 2;
const int ESP = M / VALS;
int main() {
for (int i = 0; i < N; ++i) {
random_device rd;
if (rd.entropy() == 0) {
cout << "No support for nondeterministic RNG." << endl;
break;
} else {
mt19937 gen(rd());
uniform_int_distribution<int> distrib(0, 1);
vector<int> hist(VALS, 0);
for (int j = 0; j < M; ++j) ++hist[distrib(gen)];
int Y = 0;
for (int j = 0; j < VALS; ++j) Y += abs(hist[j] - ESP);
cout << Y << endl;
}
}
}
As you can see in the code, I check for the entropy to be greater than 0. I do this because:
Unlike the other standard generators, this [random_device] is not meant to be an
engine that generates pseudo-random numbers, but a generator based on
stochastic processes to generate a sequence of uniformly distributed
random numbers. Although, certain library implementations may lack the
ability to produce such numbers and employ a random number engine to
generate pseudo-random values instead. In this case, entropy returns
zero. Source
Checking the value of the entropy allows me to abort de data obtaining if the resulting data is going to be pseudo-random (not nondeterministic). Please note that I assume that if rd.entropy() == 0 is true, then we are in pseudo-random mode.
Unfortunately, all my trials result in a file with no data because of entropy being 0. My question is: what can I do to my computer, or where can I find a machine that allows me to obtain the data?
The source you cite is misleading you. The standard says that
double entropy() const noexcept;
Returns: If the implementation employs a random number engine, returns 0.0. Otherwise, returns an entropy estimate for the random numbers returned by operator(), in the range min() to log2(max()+1).
And a better reference has some empirical observations
Notes
This function is not fully implemented in some standard libraries. For
example, LLVM libc++ always returns zero even though the device is
non-deterministic. In comparison, Microsoft Visual C++ implementation
always returns 32, and boost.random returns 10.
In practice, nearly all the main implementations (targeting general purpose computers) have non-deterministic std::random_devices. Your test has a very high false negative rate.

C++ random generator with provided (at least estimated) entropy

Using C++ standard random generator I can more or less efficiently create sequences with pre-defined distributions using language-provided tools. What about Shannon entropy? Is it possible some way to define output Shannon entropy for the provided sequence?
I tried a small experiment, generated a long enough sequence with linear distribution, and implemented a Shannon entropy calculator. Resulting value is from 0.0 (absolute order) to 8.0 (absolute chaos)
template <typename T>
double shannon_entropy(T first, T last)
{
size_t frequencies_count{};
double entropy = 0.0;
std::for_each(first, last, [&entropy, &frequencies_count] (auto item) mutable {
if (0. == item) return;
double fp_item = static_cast<double>(item);
entropy += fp_item * log2(fp_item);
++frequencies_count;
});
if (frequencies_count > 256) {
return -1.0;
}
return -entropy;
}
std::vector<uint8_t> generate_random_sequence(size_t sequence_size)
{
std::vector<uint8_t> random_sequence;
std::random_device rnd_device;
std::cout << "Random device entropy: " << rnd_device.entropy() << '\n';
std::mt19937 mersenne_engine(rnd_device());
std::uniform_int_distribution<unsigned> dist(0, 255);
auto gen = std::bind(dist, mersenne_engine);
random_sequence.resize(sequence_size);
std::generate(random_sequence.begin(), random_sequence.end(), gen);
return std::move(random_sequence);
}
std::vector<double> read_random_probabilities(size_t sequence_size)
{
std::vector<size_t> bytes_distribution(256);
std::vector<double> bytes_frequencies(256);
std::vector<uint8_t> random_sequence = generate_random_sequence(sequence_size);
size_t rnd_seq_size = random_sequence.size();
std::for_each(random_sequence.begin(), random_sequence.end(), [&](uint8_t b) mutable {
++bytes_distribution[b];
});
std::transform(bytes_distribution.begin(), bytes_distribution.end(), bytes_frequencies.begin(),
[&rnd_seq_size](size_t item) {
return static_cast<double>(item) / rnd_seq_size;
});
return std::move(bytes_frequencies);
}
int main(int argc, char* argv[]) {
size_t sequence_size = 1024 * 1024;
std::vector<double> bytes_frequencies = read_random_probabilities(sequence_size);
double entropy = shannon_entropy(bytes_frequencies.begin(), bytes_frequencies.end());
std::cout << "Sequence entropy: " << std::setprecision(16) << entropy << std::endl;
std::cout << "Min possible file size assuming max theoretical compression efficiency:\n";
std::cout << (entropy * sequence_size) << " in bits\n";
std::cout << ((entropy * sequence_size) / 8) << " in bytes\n";
return EXIT_SUCCESS;
}
First, it appears that std::random_device::entropy() hardcoded to return 32; in MSVC 2015 (which is probably 8.0 according to Shannon definition). As you can try it's not far from the truth, this example it's always close to 7.9998..., i.e. absolute chaos.
The working example is on IDEONE (by the way, their compiler hardcode entropy to 0)
One more, the main question - is it possible to create such a generator that generate linearly-distributed sequence with defined entropy, let's say 6.0 to 7.0? Could it be implemented at all, and if yes, if there are some implementations?
First, you're viewing Shannon's theory entirely wrong. His argument (as you're using it) is simply, "given the probably of x (Pr(x)), the bits required to store x is -log2 Pr(x). It has nothing to do with the probability of x. In this regard, you're viewing Pr(x) wrong. -log2 Pr(x) given a Pr(x) that should be uniformly 1/256 results in a required bitwidth of 8 bits to store. However, that's not how statistics work. Go back to thinking about Pr(x) because the bits required means nothing.
Your question is about statistics. Given an infinite sample, if-and-only-if the distribution matches the ideal histogram, as the sample size approaches infinite the probability of each sample will approach the expected frequency. I want to make it clear that you're not looking for "-log2 Pr(x) is absolute chaos when it's 8 given Pr(x) = 1/256." A uniform distribution is not chaos. In fact, it's... well, uniform. It's properties are well known, simple, and easy to predict. You're looking for, "Is the finite sample set of S meeting the criteria of a independently-distributed uniform distribution (commonly known as "Independently and Identically Distributed Data" or "i.i.d") of Pr(x) = 1/256?" This has nothing to do with Shannon's theory and goes much further back in time to the basic probability theories involving flips of a coin (in this case binomial given assumed uniformity).
Assuming for a moment that any C++11 <random> generator meets the criteria for "statistically indistinguishable from i.i.d." (which, by the way, those generators don't), you can use them to emulate i.i.d. results. If you would like a range of data that is storable within 6..7 bits (it wasn't clear, did you mean 6 or 7 bits, because hypothetically, everything in between is doable as well), simply scale the range. For example...
#include <iostream>
#include <random>
int main() {
unsigned long low = 1 << 6; // 2^6 == 64
unsigned long limit = 1 << 7; // 2^7 == 128
// Therefore, the range is 6-bits to 7-bits (or 64 + [128 - 64])
unsigned long range = limit - low;
std::random_device rd;
std::mt19937 rng(rd()); //<< Doesn't actually meet criteria for i.d.d.
std::uniform_int_distribution<unsigned long> dist(low, limit - 1); //<< Given an engine that actually produces i.i.d. data, this would produce exactly what you're looking for
for (int i = 0; i != 10; ++i) {
unsigned long y = dist(rng);
//y is known to be in set {2^6..2^7-1} and assumed to be uniform (coin flip) over {low..low + (range-1)}.
std::cout << y << std::endl;
}
return 0;
}
The problem with this is that, while the <random> distribution classes are accurate, the random number generators (presumably aside from std::random_device, but that's system-specific) are not designed to stand up to statistical tests of fitness as i.i.d. generators.
If you would like one that does, implement a CSPRNG (my go-to is Bob Jenkins' ISAAC) that has an interface meeting the requirements of the <random> class of generators (probably just covering the basic interface of std::random_device is good enough).
To test for statistically sound "no" or "we can't say no" for whether a set follows a specific model (and therefore Pr(x) is accurate and therefore Shannon's entropy function is an accurate prediction), that's a whole thing else entirely. Like I said, no generator in <random> meets these criteria (except maybe std::random_device). My advice is to do research into things like Central limit theorem, Goodness-of-fit, Birthday-spacing, et cetera.
To drive my point a bit more, under the assumptions of your question...
struct uniform_rng {
unsigned long x;
constexpr uniform_rng(unsigned long seed = 0) noexcept:
x{ seed }
{ };
unsigned long operator ()() noexcept {
unsigned long y = this->x++;
return y;
}
};
... would absolutely meet your criteria of being uniform (or as you say "absolute chaos"). Pr(x) is most certainly 1/N and the bits required to store any number of the set is -log2 Pr(1/N) which is whatever 2 to the power of the bitwidth of unsigned long is. However, it's not independently distributed. Because we know it's properties, you can "store" it's entire sequence by simply storing seed. Surprise, all PRNGs work this way. Therefore the bits required to store the entire sequence of an PRNG is -log2(1/2^bitsForSeed). As your sample grows, the bits required to store vs the bits your able to generate that sample (aka, the compression ratio) approaches a limit of 0.
I cannot comment yet, but I would like to start the discussion:
From communication/information theory, it would seem that you would require probabilistic shaping methods to achieve what you want. You should be able to feed the output of any distribution function through a shaping coder, which then should re-distribute the input to a specific target shannon entropy.
Probabilistic constellation shaping has been succesfully applied in fiber-optic communication: Wikipedia with some other links
You are not clear what you want to achieve, and there are several ways of lowering the Shannon entropy for your sequence:
Correlation between the bits, e.g. putting random_sequence through a
simple filter.
Individual bits are not fully random.
As an example below you could make the bytes less random:
std::vector<uint8_t> generate_random_sequence(size_t sequence_size,
int unit8_t cutoff=10)
{
std::vector<uint8_t> random_sequence;
std::vector<uint8_t> other_sequence;
std::random_device rnd_device;
std::cout << "Random device entropy: " << rnd_device.entropy() << '\n';
std::mt19937 mersenne_engine(rnd_device());
std::uniform_int_distribution<unsigned> dist(0, 255);
auto gen = std::bind(dist, mersenne_engine);
random_sequence.resize(sequence_size);
std::generate(random_sequence.begin(), random_sequence.end(), gen);
other_sequence.resize(sequence_size);
std::generate(other_sequence.begin(), other_sequence.end(), gen);
for(size_t j=0;j<size;++j) {
if (other_sequence[j]<=cutoff) random_sequence[j]=0; // Or j or ...
}
return std::move(random_sequence);
}
I don't think this was the answer you were looking for - so you likely need to clarify the question more.

C/C++ algorithm to produce same pseudo-random number sequences from same seed on different platforms? [duplicate]

This question already has answers here:
Consistent pseudo-random numbers across platforms
(5 answers)
Closed 9 years ago.
The title says it all, I am looking for something preferably stand-alone because I don't want to add more libraries.
Performance should be good since I need it in a tight high-performance loop. I guess that will come at a cost of the degree of randomness.
Any particular pseudo-random number generation algorithm will behave like this. The problem with rand is that it's not specified how it is implemented. Different implementations will behave in different ways and even have varying qualities.
However, C++11 provides the new <random> standard library header that contains lots of great random number generation facilities. The random number engines defined within are well-defined and, given the same seed, will always produce the same set of numbers.
For example, a popular high quality random number engine is std::mt19937, which is the Mersenne twister algorithm configured in a specific way. No matter which machine, you're on, the following will always produce the same set of real numbers between 0 and 1:
std::mt19937 engine(0); // Fixed seed of 0
std::uniform_real_distribution<> dist;
for (int i = 0; i < 100; i++) {
std::cout << dist(engine) << std::endl;
}
Here's a Mersenne Twister
Here is another another PRNG implementation in C.
You may find a collection of PRNG here.
Here's the simple classic PRNG:
#include <iostream>
using namespace std;
unsigned int PRNG()
{
// our initial starting seed is 5323
static unsigned int nSeed = 5323;
// Take the current seed and generate a new value from it
// Due to our use of large constants and overflow, it would be
// very hard for someone to predict what the next number is
// going to be from the previous one.
nSeed = (8253729 * nSeed + 2396403);
// Take the seed and return a value between 0 and 32767
return nSeed % 32767;
}
int main()
{
// Print 100 random numbers
for (int nCount=0; nCount < 100; ++nCount)
{
cout << PRNG() << "\t";
// If we've printed 5 numbers, start a new column
if ((nCount+1) % 5 == 0)
cout << endl;
}
}

Uniform random number generator in c++

I am trying to produce true random number in c++ with C++ TR1.
However, when run my program again, it produces same random numbers.The code is below.
I need true random number for each run as random as possible.
std::tr1::mt19937 eng;
std::tr1::uniform_real<double> unif(0, 1);
unif(eng);
You have to initialize the engine with a seed, otherwise the default seed is going to be used:
eng.seed(static_cast<unsigned int >(time(NULL)));
However, true randomness is something you cannot achieve on a deterministic machine without additional input. Every pseudo-random number generator is periodical in some way, which is something you wouldn't expect from a non-deterministic number. For example std::mt19937 has a period of 219937-1 iterations. True randomness is hard to achieve, as you would have to monitor something that doesn't seem deterministic (user input, atmospheric noise). See Jerry's and Handprint's answer.
If you don't want a time based seed you can use std::random_device as seen in emsr's answer. You could even use std::random_device as generator, which is the closest you'll get to true randomness with standard library methods only.
These are pseudo-random number generators. They can never produce truly random numbers. For that, you typically need special hardware (e.g., typically things like measuring noise in a thermal diode or radiation from radioactive source).
To get a difference sequences from pseudo-random generators in different runs, you typically seed the generator based on the current time.
That produces fairly predictable results though (i.e., somebody else can figure out the seed you used fairly easily. If you need to prevent that, most systems do provide some source of at least fairly random numbers. On Linux, /dev/random, and on Windows, CryptGenRandom.
Those latter tend to be fairly slow, though, so you usually want to use them as a seed, not just retrieve all your random numbers from them.
If you want true hardware random numbers then the standard library offers access to this through the random_device class:
I use it to seed another generator:
#include <random>
...
std::mt19937_64 re;
std::random_device rd;
re.seed(rd());
...
std::cout << re();
If your hardware has /dev/urandom or /dev/random then this will be used. Otherwise the implementation is free to use one of it's pseudorandom generators. On G++ mt19937 is used as a fallback.
I'm pretty sure tr1 has this as well bu as others noted I think it's best to use std C++11 utilities at this point.
Ed
This answer is a wiki. I'm working on a library and examples in .NET, feel free to add your own in any language...
Without external 'random' input (such as monitoring street noise), as a deterministic machine, a computer cannot generate truly random numbers: Random Number Generation.
Since most of us don't have the money and expertise to utilize the special equipment to provide chaotic input, there are ways to utitlize the somewhat unpredictable nature of your OS, task scheduler, process manager, and user inputs (e.g. mouse movement), to generate the improved pseudo-randomness.
Unfortunately, I do not know enough about C++ TR1 to know if it has the capability to do this.
Edit
As others have pointed out, you get different number sequences (which eventually repeat, so they aren't truly random), by seeding your RNG with different inputs. So you have two options in improving your generation:
Periodically reseed your RNG with some sort of chaotic input OR make the output of your RNG unreliable based on how your system operates.
The former can be accomplished by creating algorithms that explicitly produce seeds by examining the system environment. This may require setting up some event handlers, delegate functions, etc.
The latter can be accomplished by poor parallel computing practice: i.e. setting many RNG threads/processes to compete in an 'unsafe manner' to create each subsequent random number (or number sequence). This implicitly adds chaos from the sum total of activity on your system, because every minute event will have an impact on which thread's output ends up having being written and eventually read when a 'GetNext()' type method is called. Below is a crude proof of concept in .NET 3.5. Note two things: 1) Even though the RNG is seeded with the same number everytime, 24 identical rows are not created; 2) There is a noticeable hit on performance and obvious increase in resource consumption, which is a given when improving random number generation:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
namespace RandomParallel
{
class RandomParallel
{
static int[] _randomRepository;
static Queue<int> _randomSource = new Queue<int>();
static void Main(string[] args)
{
InitializeRepository(0, 1, 40);
FillSource();
for (int i = 0; i < 24; i++)
{
for (int j = 0; j < 40; j++)
Console.Write(GetNext() + " ");
Console.WriteLine();
}
Console.ReadLine();
}
static void InitializeRepository(int min, int max, int size)
{
_randomRepository = new int[size];
var rand = new Random(1024);
for (int i = 0; i < size; i++)
_randomRepository[i] = rand.Next(min, max + 1);
}
static void FillSource()
{
Thread[] threads = new Thread[Environment.ProcessorCount * 8];
for (int j = 0; j < threads.Length; j++)
{
threads[j] = new Thread((myNum) =>
{
int i = (int)myNum * _randomRepository.Length / threads.Length;
int max = (((int)myNum + 1) * _randomRepository.Length / threads.Length) - 1;
for (int k = i; k <= max; k++)
{
_randomSource.Enqueue(_randomRepository[k]);
}
});
threads[j].Priority = ThreadPriority.Highest;
}
for (int k = 0; k < threads.Length; k++)
threads[k].Start(k);
}
static int GetNext()
{
if (_randomSource.Count > 0)
return _randomSource.Dequeue();
else
{
FillSource();
return _randomSource.Dequeue();
}
}
}
}
As long as there is user(s) input/interaction during the generation, this technique will produce an uncrackable, non-repeating sequence of 'random' numbers. In such a scenario, knowing the initial state of the machine would be insufficient to predict the outcome.
Here's an example of seeding the engine (using C++11 instead of TR1)
#include <chrono>
#include <random>
#include <iostream>
int main() {
std::mt19937 eng(std::chrono::high_resolution_clock::now()
.time_since_epoch().count());
std::uniform_real_distribution<> unif;
std::cout << unif(eng) << '\n';
}
Seeding with the current time can be relatively predictable and is probably not something that should be done. The above at least does not limit you just to one possible seed per second, which is very predictable.
If you want to seed from something like /dev/random instead of the current time you can do:
std::random_device r;
std::seed_seq seed{r(), r(), r(), r(), r(), r(), r(), r()};
std::mt19937 eng(seed);
(This depends on your standard library implementation. For example, libc++ uses /dev/urandom by default, but in VS11 random_device is deterministic)
Of course nothing you get out of mt19937 is going to meet your requirement of a "true random number", and I suspect that you don't really need true randomness.