I am trying generate random variates by trying to generate two standard normal variates r1, r2, by using polar coordinates along with a mean and sigma value. However when I run my code, I keep getting a "-nan(ind)" as my output.
What am I doing wrong here? The code is as follows:
static double saveNormal;
static int NumNormals = 0;
static double PI = 3.1415927;
double fRand(double fMin, double fMax)
{
double f = (double)rand() / RAND_MAX;
return fMin + f * (fMax - fMin);
}
static double normal(double r, double mean, double sigma) {
double returnNormal;
if (NumNormals == 0) {
//to get next double value
double r1 = fRand(0, 20);
double r2 = fRand(0, 20);
returnNormal = sqrt(-2 * log(r1)) * cos(2 * PI*r2);
saveNormal = sqrt(-2 * log(r1)) * sin(2 * PI*r2);
}
else {
NumNormals = 0;
returnNormal = saveNormal;
}
return returnNormal*sigma + mean;
}
So, you're using the Box–Muller method to pseudo randomly sample a normal random variate. For this transform to work, r1 and r2 must be uniformly distributed independent variates in [0,1].
Instead, your r1/r2 are [0,20] supported, resulting in a negative sqrt argument when >1, this will give you nans. Replace with
double r1 = fRand(0, 1);
double r2 = fRand(0, 1);
Moreover, you should use C++11 <random> for better pseudorandom number generation; as of now, your fRand has poor quality due to rand()-to-double conversion and possible spurious correlations between adjacent calls. Moreover, your function lacks some basic error checking and badly depends on global variables and is inherently thread unsafe.
FYI, this is what a C++11 version might look like
#include <random>
#include <iostream>
int main()
{
auto engine = std::default_random_engine{ std::random_device{}() };
auto variate = std::normal_distribution<>{ /*mean*/0., /*stddev*/ 1. };
while(true) // a lot of normal samples ...
std::cout << variate(engine) << std::endl;
}
r1 can be zero, making log(r1) undefined.
furthermore, don't use rand() except when you need your numbers to look random to a human in a hurry. Use <random> instead
Related
I'm new to C++ and I'm using the boost library to generate random variables. I want to generate random variables from a negative binomial distribution.
The first parameter of boost::random::negative_binomial_distribution<int> freq_nb(r, p); has to be an integer. I want to expand that to a real value. Therefore I would like to use a poisson-gamma mixture, but I fail to.
Here's an excerpt from my code:
int nr_sim = 1000000;
double mean = 2.0;
double variance = 15.0;
double r = mean * mean / (variance - mean);
double p = mean / variance;
double beta = (1 - p) / p;
typedef boost::mt19937 RNGType;
RNGType rng(5);
boost::random::gamma_distribution<double> my_gamma(r, beta);
boost::random::poisson_distribution<int> my_poi(my_gamma(rng));
int simulated_mean = 0;
for (int i = 0; i < nr_sim; i++) {
simulated_mean += my_poi(rng);
}
double my_result = (double)simulated_mean / (double)nr_sim;
With my_result == 0.5 there is definitly something wrong. Is it my_poi(my_gamma(rng))? If so, what is the correct way to solve that problem?
I implemented simulated annealing in C++ to minimize (x-2)^2+(y-1)^2 in some range.
I'm getting varied output which is not acceptable for this type of heuristic method. It seems that the solution is converging but never quite closing in on the solution.
My code:
#include <bits/stdc++.h>
using namespace std;
double func(double x, double y)
{
return (pow(x-2, 2)+pow(y-1, 2));
}
double accept(double z, double minim, double T,double d)
{
double p = -(z - minim) / (d * T);
return pow(exp(1), p);
}
double fRand(double fMin, double fMax)
{
double f = (double)rand() / RAND_MAX;
return fMin + f * (fMax - fMin);
}
int main()
{
srand (time(NULL));
double x = fRand(-30,30);
double y = fRand(-30,30);
double xm = x, ym=y;
double tI = 100000;
double tF = 0.000001;
double a = 0.99;
double d=(1.6*(pow(10,-23)));
double T = tI;
double minim = func(x, y);
double z;
double counter=0;
while (T>tF) {
int i=1;
while(i<=30) {
x=x+fRand(-0.5,0.5);
y=y+fRand(-0.5,0.5);
z=func(x,y);
if (z<minim || (accept(z,minim,T,d)>(fRand(0,1)))) {
minim=z;
xm=x;
ym=y;
}
i=i+1;
}
counter=counter+1;
T=T*a;
}
cout<<"min: "<<minim<<" x: "<<xm<<" y: "<<ym<<endl;
return 0;
}
How can I get it to reach the solution?
There are a couple of things that I think are wrong in your implementation of the simulated annealing algorithm.
At every iteration you should look at some neighbours z of current minimum and update it if f(z) < minimum. If f(z) > minimum you can also accept the new point, but with an acceptance probability function.
The problem is that in your accept function, the parameter d is way too low - it will always return 0.0 and never trigger the condition of acceptance. Try something like 1e-5; it doesn't have to be physically correct, it only has to decrease while lowering the "temperature".
After updating the temperature in the outer loop, you should put x=xm and y=ym, before doing the inner loop or instead of searching the neigbours of the current solution you will basically randomly wander around (you aren't checking any boundaries too).
Doing so, I usually get some output like this:
min: 8.25518e-05 x: 2.0082 y: 0.996092
Hope it helped.
The following code does not seem to behave intuitively:
#include <random>
#include <iostream>
using namespace std;
int main()
{
mt19937 MyGenerator(40);
auto gauss = normal_distribution<double>(0,1);
auto linear = uniform_real_distribution<double>(0,1);
cout << gauss(MyGenerator) << endl; //line a
cout << linear(MyGenerator) << endl; //line b
cout << gauss(MyGenerator) << endl;
}
Running this code gives the output
-0.816097
0.705030
0.303032.
If now the order of lines a and b is swapped, the output changes to
0.644008
0.338080
-0.639501.
It is completely clear that the first two numbers are different now, as they are produced by different distributions. Nevertheless, why is the third number different?
In my intuition, the distribution should grab a number c = MyGenerator() which is then mapped to the random number in the specific range. The random number generator would point to the next number in the sequence of numbers after the distribution call. So, shouldn't the outcome of the third call be the same in both cases?
Another observation:
Adding a forth call to either of the distributions does in fact seem to reproduce the same numbers.
libstdc++'s implementation of normal_distribution uses the Marsaglia polar method. The interesting thing about this method is that each pass uses two random numbers from the URNG to generate two results.
That is, the first call to the distribution calls the URNG twice (possibly more times, as it uses rejection sampling, but an even number of times) and returns one result; the following call to the distribution will not call the URNG but will return the saved second result.
Here's an extract from the source code, slightly reformatted:
if (_M_saved_available)
{
_M_saved_available = false;
ret = _M_saved;
}
else
{
result_type x, y, r2;
do
{
x = result_type(2.0) * aurng() - 1.0;
y = result_type(2.0) * aurng() - 1.0;
r2 = x * x + y * y;
}
while (r2 > 1.0 || r2 == 0.0);
const result_type mult = std::sqrt(-2 * std::log(r2) / r2);
_M_saved = x * mult;
_M_saved_available = true;
ret = y * mult;
}
There's no requirement that the distribution call the underlying generator once for each value. Some distributions are best calculated by combining multiple random values.
For example, in the GNU implementation, the implemantation of the uniform distribution is
return (__aurng() * (__p.b() - __p.a())) + __p.a();
calling the generator __aurng once; while the core of the normal distribution is:
do
{
__x = result_type(2.0) * __aurng() - 1.0;
__y = result_type(2.0) * __aurng() - 1.0;
__r2 = __x * __x + __y * __y;
}
while (__r2 > 1.0 || __r2 == 0.0);
calling it at least twice.
This shouldn't be too difficult, but for some reason this sampling random numbers from a distribution is really tripping me up.
I know the best options for generating random numbers from a distribution are boost/C++11 libraries...unfortunately, I can't get this code to compile with c++0x, and anyway, I would preferably keep compatibility on a server that I'm also using, which is running gcc 4.1.2 - ancient, I know, doesn't support newer C++. Frustrations. And as always, time crunch means I need to do the best I can with a quick fix.
Taking the exponent of a random number from the box muller equations is my next option, but I'm not getting a lognormal distribution with the parameters I specify. I don't understand why this is not working.
Any help would be hugely appreciated!
void testRNG(){
int mean = 5000;
int std = 50;
ofstream out("./Output/normal_samples.out");
RunningStats normal;
for (int i=0;i<2000;++i){
double sample = randomSample(mean, std, NORMAL);//call function with box muller transformation to return a number from a normal distriubtion
out<<sample<<endl;
normal.Push(sample);//keep a running average of sampled numbers
}
cout<<"Normal Mean = "<<normal.Mean()<<endl;
cout<<"Normal Std = "<<normal.StandardDeviation()<<endl;
RunningStats lognormal;
for (int i=0;i<2000;++i){
double sample = randomSample(mean, std, LOGNORMAL);
out<<sample<<endl;
lognormal.Push(sample);
}
cout<<"Lognormal Mean = "<<lognormal.Mean()<<endl;
cout<<"Lognormal Std = "<<lognormal.StandardDeviation()<<endl;
}
The sampling functions, which I didn't write, go first to a case thing from randomSample(), and then call:
EDIT - I noticed that it actually did call a function to find the lognormal parameters. Added in.
double randNormal(double mean, double stdev) {
static long numSamples = 0;
static double Z2;
if ((numSamples++ & 1) == 0) {
double Z1, U1, U2;
do { U1 = randUniform(0, 1); } while (U1 <= 0 || U1 >= 1);
do { U2 = randUniform(0, 1); } while (U1 <= 0 || U1 >= 1);
Z1 = sqrt(-2 * log(U1)) * cos(6.28318531 * U2);
Z2 = sqrt(-2 * log(U1)) * sin(6.28318531 * U2);
return mean + stdev * Z1;
} else {
return mean + stdev * Z2;
}
}
double randLognormal(double mu, double sigma) {
return exp(randNormal(mu, sigma));
}
double randLognormalMeanStdev(double mean, double stdev) {
return randLognormal( log(mean) - 0.5 * log(1 + (stdev * stdev) / (mean * mean)) , log(1 + (stdev * stdev) / (mean * mean)));
}
So the output I get is:
Normal Mean = 4998.72 //I said 5000
Normal Std = 49.7054 //I said 50
Lognormal Mean = 4999.74
Lognormal Std = 0.492766 //this is the part that is not working
What am I missing to get the lognormal std to be what I want?
Other options would also be appreciated - maybe there is something else I am missing.
Thanks in advance!
Edit - I realized I should have made it clear that I need to sample from a lognormal distribution
I'm trying to implement a very simple 1-dimensional gradient descent algorithm. The code I have does not work at all. Basically depending on my alpha value, the end parameters will either be wildly huge (like ~70 digits), or basically zero (~ 0.000). I feel like a gradient descent should not be nearly this sensitive in alpha (I'm generating small data in [0.0,1.0], but I think the gradient itself should account for the scale of the data, no?).
Here's the code:
#include <cstdio>
#include <cstdlib>
#include <ctime>
#include <vector>
using namespace std;
double a, b;
double theta0 = 0.0, theta1 = 0.0;
double myrand() {
return double(rand()) / RAND_MAX;
}
double f(double x) {
double y = a * x + b;
y *= 0.1 * (myrand() - 0.5); // +/- 5% noise
return y;
}
double h(double x) {
return theta1 * x + theta0;
}
int main() {
srand(time(NULL));
a = myrand();
b = myrand();
printf("set parameters: a = %lf, b = %lf\n", a, b);
int N = 100;
vector<double> xs(N);
vector<double> ys(N);
for (int i = 0; i < N; ++i) {
xs[i] = myrand();
ys[i] = f(xs[i]);
}
double sensitivity = 0.008;
double d0, d1;
for (int n = 0; n < 100; ++n) {
d0 = d1 = 0.0;
for (int i = 0; i < N; ++i) {
d0 += h(xs[i]) - ys[i];
d1 += (h(xs[i]) - ys[i]) * xs[i];
}
theta0 -= sensitivity * d0;
theta1 -= sensitivity * d1;
printf("theta0: %lf, theta1: %lf\n", theta0, theta1);
}
return 0;
}
Changing the value of alpha can produce the algorithm to diverge, so that may be one of the causes of what is happening. You can check by computing the error in each iteration and see if is increasing or decreasing.
In adition, it is recommended to set randomly the values of theta at the beginning in stead of assigning them to zero.
Apart from that, you should divide by N when you update the value of theta as follows:
theta0 -= sensitivity * d0/N;
theta1 -= sensitivity * d1/N;
I had a quick look at your implementation and it looks fine to me.
The code I have does not work at all.
I wouldn't say that. It seems to behave correctly for small enough values of sensitivity, which is a value that you just have to "guess", and that is how the gradient descent is supposed to work.
I feel like a gradient descent should not be nearly this sensitive in alpha
If you struggle to visualize that, remember that you are using gradient descent to find the minimum of the cost function of linear regression, which is a quadratic function. If you plot the cost function you will see why the learning rate is so sensitive in these cases: intuitively, if the parabola is narrow, the algorithm will converge more quickly, which is good, but then the learning rate is more "sensitive" and the algorithm can easily diverge if you are not careful.