I am trying to write a short C++ routine to calculate the following function F(i,j,z) for given integers j > i (typically they lie between 0 and 100) and complex number z (bounded by |z| < 100), where L are the associated Laguerre Polynomials:
The issue is that I want this function to be callable from within a CUDA kernel (i.e. with a __device__ attribute). Standard library/Boost/etc functions are therefore out of the questions, unless they are simple enough to re-implement on my own - this especially relates to the Laguerre polynomials which exist in Boost and C++17. Regardless if I manage to wrap any standard function for Laguerre polynomials, I still have a similar pre-factor to calculate of the form (z^j/j!).
Question: How can I do a relatively simple implementation of such a function, without introducing significant numerical instability?
My idea so far is to calculate L and its pre-factor independently. The pre-factor I will calculate by first looping from 0 to j-i and calculate (z^1 * z^2/2 * ... * z^(j-1)/(j-i)!). I will then calculate the remaining factor exp(-|z|^2/2) *(j-i)! * sqrt(i!/j!) (either in a similar way, or through the Gamma-function, which is implemented in CUDA math). The idea is then to find a minimal algorithm to calculate the associated Laguerre polynomial, unless I manage to wrap an implementation from e.g. Boost or GNU C++.
Edit/side note: The expression for F actually blows up numerically for some values of i/j. It was derived wrong in the source where I got it, and the indices of the associated Laguerre polynomials should instead be L_i^(j-i). That does not invalidate the approaches suggested in the answers/comments.
I recommend finding a recurrence relation for the coefficients of the Laguerre Polynomial:
C(k+1) = g(k)C(k)
g(k) = C(k+1) / C(k)
g(k) = -z * (j - k) / ((j - i + k + 1) * (k + 1)) //Verify this yourself :)
This allows you to avoid most of factorials in computing the polynomial.
After that I would follow Severin's idea of doing the calculations in logarithms
so as to not overload the double floating point range:
log(F) = log(sqrt(i!/j!)) - |z|^2 + (j-i) * log(-z) + log(L(|z|^2))
log(L) = log((2*j - i)!) + log(sum) // where the summation is computed using the recurrence relation above
and using the fact that:
log(a!) = sum(k=1..a, log(k))
and also:
log(z) = log(|z|) + I * arg(z) for complex z
log(-z) = log(|z|) + I * arg(-z)
log(-z) = log(|z|) - I * arg(z)
for the log(sqrt(i!/j!)) part I would do (assuming that j >= i):
log(sqrt(i!/j!))
= 0.5 * (log(i!) - log(j!))
= -0.5 * sum(k==i+1..j, log(k))
I haven't tried this out so there could definitely be little mistakes here and there. This answer is more about the technique rather than a copy-paste-ready answer
Well, what you should do is to logarithm it
Assuming natural logarithm,
q = log(z^j/j!) = log(z^j) - log(j!) = j*log(z) - log(Gamma(j+1))
First term is simple, second term is standard C++ function lgamma(x) (or you could use GSL).
compute value of q and return cexp(q)
You could fold exponent in this method as well
Please refer to the following MWE:
import sympy as s
x = s.IndexedBase('x')
y = s.IndexedBase('y')
i,j,k = map(s.Idx,['i','j','k'])
a = s.exp(x[i]*y[j]*s.KroneckerDelta(i,j))
b = a.diff(x[j])
The value of b is LaTeX rendering of b. Since I am not allowed to embed images yet, here is the text form
((Derivative(KroneckerDelta(i, j), i)*Derivative(i, x[j]) + Derivative(KroneckerDelta(i, j), j)*Derivative(j, x[j]))*x[i]*y[j] + KroneckerDelta(i, j)*y[j])*exp(KroneckerDelta(i, j)*x[i]*y[j])
The key point is that there are unevaluated derivatives of KroneckerDelta with respect to indices i and j and derivatives of indices i and j with respect to x[i]. Why are these not 0?
I think, the kind of behaviour I was expecting with symbolic differentiation of indexed variables has not been fully implemented in Sympy yet. I moved on to using other tools like Maxima or Mathematica (or even pen and paper) for these calculations.
Is it possible to write these constraints in Lp solver?
I want to check whether two rectangles are overlapping or not?
Let us assume, there is one rectangle whose left bottom corner is (xk, yk) and (wi,hi) be there width and height respectively. Similarly, there is another rectangle for this left bottom corner is (xl,yl) and (wj,hj) be the width and height.
I can write for pair of (i,j) rectangles that they will not overlap such that
There does not exist any J such that
{xk < (xl+wj)^(xk+wi)>xl}^{yk<(yl+hj)^(yk+hi)>yl}
how can I write it in Lp Solver or can I use OR operator?
Thanks
A standard way to enforce two rectangles i and j do not overlap is:
x(i)+w(i) <= x(j) or
x(j)+w(j) <= x(i) or
y(i)+h(i) <= y(j) or
y(j)+h(j) <= y(i)
This can be written as a set of linear equalities:
Note that when comparing all rectangles i and j you only need to do this for i < j. The big-M constants need to be chosen with care.
I believe this is more of an algorithmic question but I also want to do this in C++.
Let me illustrate the question with an example.
Suppose I have N number of objects (not programming objects), each with different weights. And I have two vehicles to carry them. The vehicles are big enough to carry all the objects by each. These two vehicles have their own mileage and different levels of fuel in the tank. And also the mileage depends on the weight it carries.
The objective is to bring these N objects as far as possible. So I need to distribute the N objects in a certain way between the two vehicles. Note that I do not need to bring them the 'same' distance, but rather as far as possible. So example, I want the two vehicles to go 5km and 6 km, rather than one going 2km and other going 7km.
I cannot think of a theoretical closed-form calculation to determine which weights to be loaded in to each vehicle. because remember that I need to carry all the N objects which is a fixed value.
So as far as I can think, I need to try all the combinations.
Could someone advice of an efficient algorithm to try all the combinations?
For example I would have the following:
int weights[5] = {1,4,2,7,5}; // can be more values than 5
float vehicelONEMileage(int totalWeight);
float vehicleTWOMileage(int totalWeight);
How could I efficiently try all the combinations of weights[] with the two functions?
Thw two functions can be assumed as linear functions. I.e. the return value of the two mileage functions are linear functions with (different) negative slopes and (different) offsets.
So what I need to find is something like:
MAX(MIN(vehicleONEMileage(x), vehicleTWOMileage(sum(weights) - x)));
Thank you.
This should be on the cs or the math site.
Simplification: Instead of an array of objects, let's say we can distribute weight linearly.
The function we want to optimize is the minimum of both travel distances. Finding the maximum of the minimum is the same as finding the maximum of the product (Without proof. But to see this, think of the relationship between perimeter and area of rectangles. The rectangle with the biggest area given a perimeter is a square, which also happens to have the largest minimum side length).
In the following, we will scale the sum of all weights to 1. So, a distribution like (0.7, 0.3) means that 70% of all weights is loaded on vehicle 1. Let's call the load of vehicle 1 x and the load of vehicle 1-x.
Given the two linear functions f = a x + b and g = c x + d, where f is the mileage of vehicle 1 when loaded with weight x, and g the same for vehicle 2, we want to maximize
(a*x+b)*(c*(1-x)+d)
Let's ask Wolfram Alpha to do the hard work for us: www.wolframalpha.com/input/?i=derive+%28%28a*x%2Bb%29*%28c*%281-x%29%2Bd%29%29
It tells us that there is an extremum at
x_opt = (a * c + a * d - b * c) / (2 * a * c)
That's all you need to solve your problem efficiently.
The complete algorithm:
find a, b, c, d
b = vehicleONEMileage(0)
a = (vehicleONEMileage(1) - b) * sum_of_all_weights
same for c and d
calculate x_opt as above.
if x_opt < 0, load all weight onto vehicle 2
if x_opt > 1, load all weight onto vehicle 1
else, try to load tgt_load = x_opt*sum_of_all_weights onto vehicle 1, the rest onto vehicle 2.
The rest is a knapsack problem. See http://en.wikipedia.org/wiki/Knapsack_problem#0.2F1_Knapsack_Problem
How to apply this? Use the dynamic programming algorithm described there twice.
for maximizing a load up to tgt_load
for maximizing a load up to (sum_of_all_weights - tgt_load)
The first one, if loaded onto vehicle one, gives you a distribution with slightly less then expected on vehicle one.
The second one, if loaded onto vehicle two, gives you a distribution with slightly more than expected on vehicle two.
One of those is the best solution. Compare them and use the better one.
I leave the C++ part to you. ;-)
I can suggest the following solution:
The total number of combinations is 2^(number of weights). Using a bit logic we can loop through the all combinations and calculate maxDistance. Bits in the combination value show which weight goes to which vehicle.
Note that algorithm complexity is exponential and int has a limited number of bits!
float maxDistance = 0.f;
for (int combination = 0; combination < (1 << ARRAYSIZE(weights)); ++combination)
{
int weightForVehicleONE = 0;
int weightForVehicleTWO = 0;
for (int i = 0; i < ARRAYSIZE(weights); ++i)
{
if (combination & (1 << i)) // bit is set to 1 and goes to vechicleTWO
{
weightForVehicleTWO += weights[i];
}
else // bit is set to 0 and goes to vechicleONE
{
weightForVehicleONE += weights[i];
}
}
maxDistance = max(maxDistance, min(vehicelONEMileage(weightForVehicleONE), vehicleTWOMileage(weightForVehicleTWO)));
}
Is there a way to do a quick and dirty 3D distance check where the results are rough, but it is very very fast? I need to do depth sorting. I use STL sort like this:
bool sortfunc(CBox* a, CBox* b)
{
return a->Get3dDistance(Player.center,a->center) <
b->Get3dDistance(Player.center,b->center);
}
float CBox::Get3dDistance( Vec3 c1, Vec3 c2 )
{
//(Dx*Dx+Dy*Dy+Dz*Dz)^.5
float dx = c2.x - c1.x;
float dy = c2.y - c1.y;
float dz = c2.z - c1.z;
return sqrt((float)(dx * dx + dy * dy + dz * dz));
}
Is there possibly a way to do it without a square root or possibly without multiplication?
You can leave out the square root because for all positive (or really, non-negative) numbers x and y, if sqrt(x) < sqrt(y) then x < y. Since you're summing squares of real numbers, the square of every real number is non-negative, and the sum of any positive numbers is positive, the square root condition holds.
You cannot eliminate the multiplication, however, without changing the algorithm. Here's a counterexample: if x is (3, 1, 1) and y is (4, 0, 0), |x| < |y| because sqrt(1*1+1*1+3*3) < sqrt(4*4+0*0+0*0) and 1*1+1*1+3*3 < 4*4+0*0+0*0, but 1+1+3 > 4+0+0.
Since modern CPUs can compute a dot product faster than they can actually load the operands from memory, it's unlikely that you would have anything to gain by eliminating the multiply anyway (I think the newest CPUs have a special instruction that can compute a dot product every 3 cycles!).
I would not consider changing the algorithm without doing some profiling first. Your choice of algorithm will heavily depend on the size of your dataset (does it fit in cache?), how often you have to run it, and what you do with the results (collision detection? proximity? occlusion?).
What I usually do is first filter by Manhattan distance
float CBox::Within3DManhattanDistance( Vec3 c1, Vec3 c2, float distance )
{
float dx = abs(c2.x - c1.x);
float dy = abs(c2.y - c1.y);
float dz = abs(c2.z - c1.z);
if (dx > distance) return 0; // too far in x direction
if (dy > distance) return 0; // too far in y direction
if (dz > distance) return 0; // too far in z direction
return 1; // we're within the cube
}
Actually you can optimize this further if you know more about your environment. For example, in an environment where there is a ground like a flight simulator or a first person shooter game, the horizontal axis is very much larger than the vertical axis. In such an environment, if two objects are far apart they are very likely separated more by the x and y axis rather than the z axis (in a first person shooter most objects share the same z axis). So if you first compare x and y you can return early from the function and avoid doing extra calculations:
float CBox::Within3DManhattanDistance( Vec3 c1, Vec3 c2, float distance )
{
float dx = abs(c2.x - c1.x);
if (dx > distance) return 0; // too far in x direction
float dy = abs(c2.y - c1.y);
if (dy > distance) return 0; // too far in y direction
// since x and y distance are likely to be larger than
// z distance most of the time we don't need to execute
// the code below:
float dz = abs(c2.z - c1.z);
if (dz > distance) return 0; // too far in z direction
return 1; // we're within the cube
}
Sorry, I didn't realize the function is used for sorting. You can still use Manhattan distance to get a very rough first sort:
float CBox::ManhattanDistance( Vec3 c1, Vec3 c2 )
{
float dx = abs(c2.x - c1.x);
float dy = abs(c2.y - c1.y);
float dz = abs(c2.z - c1.z);
return dx+dy+dz;
}
After the rough first sort you can then take the topmost results, say the top 10 closest players, and re-sort using proper distance calculations.
Here's an equation that might help you get rid of both sqrt and multiply:
max(|dx|, |dy|, |dz|) <= distance(dx,dy,dz) <= |dx| + |dy| + |dz|
This gets you a range estimate for the distance which pins it down to within a factor of 3 (the upper and lower bounds can differ by at most 3x). You can then sort on, say, the lower number. You then need to process the array until you reach an object which is 3x farther away than the first obscuring object. You are then guaranteed to not find any object that is closer later in the array.
By the way, sorting is overkill here. A more efficient way would be to make a series of buckets with different distance estimates, say [1-3], [3-9], [9-27], .... Then put each element in a bucket. Process the buckets from smallest to largest until you reach an obscuring object. Process 1 additional bucket just to be sure.
By the way, floating point multiply is pretty fast nowadays. I'm not sure you gain much by converting it to absolute value.
I'm disappointed that the great old mathematical tricks seem to be getting lost. Here is the answer you're asking for. Source is Paul Hsieh's excellent web site: http://www.azillionmonkeys.com/qed/sqroot.html . Note that you don't care about distance; you will do fine for your sort with square of distance, which will be much faster.
In 2D, we can get a crude approximation of the distance metric without a square root with the formula:
distanceapprox (x, y) =
which will deviate from the true answer by at most about 8%. A similar derivation for 3 dimensions leads to:
distanceapprox (x, y, z) =
with a maximum error of about 16%.
However, something that should be pointed out, is that often the distance is only required for comparison purposes. For example, in the classical mandelbrot set (z←z2+c) calculation, the magnitude of a complex number is typically compared to a boundary radius length of 2. In these cases, one can simply drop the square root, by essentially squaring both sides of the comparison (since distances are always non-negative). That is to say:
√(Δx2+Δy2) < d is equivalent to Δx2+Δy2 < d2, if d ≥ 0
I should also mention that Chapter 13.2 of Richard G. Lyons's "Understanding Digital Signal Processing" has an incredible collection of 2D distance algorithms (a.k.a complex number magnitude approximations). As one example:
Max = x > y ? x : y;
Min = x < y ? x : y;
if ( Min < 0.04142135Max )
|V| = 0.99 * Max + 0.197 * Min;
else
|V| = 0.84 * Max + 0.561 * Min;
which has a maximum error of 1.0% from the actual distance. The penalty of course is that you're doing a couple branches; but even the "most accepted" answer to this question has at least three branches in it.
If you're serious about doing a super fast distance estimate to a specific precision, you could do so by writing your own simplified fsqrt() estimate using the same basic method as the compiler vendors do, but at a lower precision, by doing a fixed number of iterations. For example, you can eliminate the special case handling for extremely small or large numbers, and/or also reduce the number of Newton-Rapheson iterations. This was the key strategy underlying the so-called "Quake 3" fast inverse square root implementation -- it's the classic Newton algorithm with exactly one iteration.
Do not assume that your fsqrt() implementation is slow without benchmarking it and/or reading the sources. Most modern fsqrt() library implementations are branchless and really damned fast. Here for example is an old IBM floating point fsqrt implementation. Premature optimization is, and always will be, the root of all evil.
Note that for 2 (non-negative) distances A and B, if sqrt(A) < sqrt(B), then A < B. Create a specialized version of Get3DDistance() (GetSqrOf3DDistance()) that does not call sqrt() that would be used only for the sortfunc().
If you worry about performance, you should also take care of the way you send your arguments:
float Get3dDistance( Vec3 c1, Vec3 c2 );
implies two copies of Vec3 structure. Use references instead:
float Get3dDistance( Vec3 const & c1, Vec3 const & c2 );
You could compare squares of distances instead of the actual distances, since d2 = (x1-x2)2 + (y1-y2)2+ (z1-z2)2. It doesn't get rid of the multiplication, but it does eliminate the square root operation.
How often are the input vectors updated and how often are they sorted? Depending on your design, it might be quite efficient to extend the "Vec3" class with a pre-calculated distance and sort on that instead. Especially relevant if your implementation allows you to use vectorized operations.
Other than that, see the flipcode.com article on approximating distance functions for a discussion on yet another approach.
Depending slightly on the number of points that you are being used to compare with, what is below is pretty much guaranteed to be the get the list of points in approximate order assuming all points change at all iteration.
1) Rewrite the array into a single list of Manhattan distances with
out[ i ] = abs( posn[ i ].x - player.x ) + abs( posn[ i ].y - player.y ) + abs( posn[ i ].z - player.z );
2) Now you can use radix sort on floating point numbers to order them.
Note that in practice this is going to be a lot faster than sorting the list of 3d positions because it significantly reduces the memory bandwidth requirements in the sort operation which all of the time is going to be spend and in which unpredictable accesses and writes are going to occur. This will run on O(N) time.
If many of the points are stationary at each direction there are far faster algorithms like using KD-Trees, although implementation is quite a bit more complex and it is much harder to get good memory access patterns.
If this is simply a value for sorting, then you can swap the sqrt() for a abs(). If you need to compare distances against set values, get the square of that value.
E.g. instead of checking sqrt(...) against a, you can compare abs(...) against a*a.
You may want to consider caching the distance between the player and the object as you calculate it, and then use that in your sortfunc. This would depend upon how many times your sort function looks at each object, so you might have to profile to be sure.
What I'm getting at is that your sort function might do something like this:
compare(a,b);
compare(a,c);
compare(a,d);
and you would calculate the distance between the player and 'a' every time.
As others have mentioned, you can leave out the sqrt in this case.
If you could center your coordinates around the player, use spherical coordinates? Then you could sort by the radius.
That's a big if, though.
If your operation happens a lot, it might be worth to put it into some 3D data structure. You probably need the distance sorting to decide which object is visible, or some similar task. In order of complexity you can use:
Uniform (cubic) subdivision
Divide the used space into cells, and assign the objects to the cells. Fast access to element, neighbours are trivial, but empty cells take up a lot of space.
Quadtree
Given a threshold, divide used space recursively into four quads until less then threshold number of object is inside. Logarithmic access element if objects don't stack upon each other, neighbours are not hard to find, space efficient solution.
Octree
Same as Quadtree, but divides into 8, optimal even if objects are above each other.
Kd tree
Given some heuristic cost function, and a threshold, split space into two halves with a plane where the cost function is minimal. (Eg.: same amount of objects at each side.) Repeat recursively until threshold reached. Always logarithmic, neighbours are harder to get, space efficient (and works in all dimensions).
Using any of the above data structures, you can start from a position, and go from neighbour to neighbour to list the objects in increasing distance. You can stop at desired cut distance. You can also skip cells that cannot be seen from the camera.
For the distance check, you can do one of the above mentioned routines, but ultimately they wont scale well with increasing number of objects. These can be used to display data that takes hundreds of gigabytes of hard disc space.