In mathematics, if three matrices, 'A', 'B' and 'C' are multiplied, such that a fourth matrix 'D = A * B * C', then the order must be computed right to left. Using parentheses to clarify, the previous statement is exactly equivalent to the following; 'D = (A * (B * C))'.
You can see how this applies for the expression '{ M = A * B * C * D } == { M = (A * (B * (C * (D)))) }', which is another example.
In C, C++, I ran a test using division to check the order in which operations are done in C and C++.
There is probably a better way of doing this my compiling assembly code, however:
float a = 1.0;
float b = 2.0;
float c = 2.0;
float ans = a / b / c;
printf("answer is %f\n", ans);
This gives the output:
answer is 0.25
This suggests to me that if I were to create a class to represent a matrix, then the order of multiplication would be left to right, the reverse of what is desired, since the example with floating point operations and division is evaluated left to right since the three operations have equal precedence.
In the OpenGL Mathematics library, GLM, matrix operations are computed in the correct order, without the requirement for parentheses.
How is operator*() made to behave in this way?
Edit
Err, yeah, so it doesn't actually matter as it has been pointed out to me. In which case my question becomes "is it possible to change the order"? (Maybe I am sleepy?)
The C++ / and binary * operators are both left-to-right associative. End of story - there's no way to change that.
Matrix multiplication is associative, though, so that shouldn't affect you.
The reason is simple: matrix multiplication is associative; scalar division is not. (A / (B / C)) is not the same as ((A / B) / C). But (A * (B * C)) is the same as ((A * B) * C), for both matrix and scalar multiplication.
So the order that C++ calls operators in just doesn't matter for matrix multiplication.
Related
Example: let
M = Matrix([[1,2],[3,4]]) # and
p = Poly(x**3 + x + 1) # then
p.subs(x,M).expand()
gives the error :
TypeError: cannot add <class'sympy.matrices.immutable.ImmutableDenseMatrix'> and <class 'sympy.core.numbers.One'>
which is very plausible since the two first terms become matrices but the last term (the constant term) is not a matrix but a scalar. To remediate to this situation I changed the polynomial to
p = Poly(x**3 + x + x**0) # then
the same error persists. Am I obliged to type the expression by hand, replacing x by M? In this example the polynomial has only three terms but in reality I encounter (multivariate polynomials with) dozens of terms.
So I think the question is mainly revolving around the concept of Matrix polynomial:
(where P is a polynomial, and A is a matrix)
I think this is saying that the free term is a number, and it cannot be added with the rest which is a matrix, effectively the addition operation is undefined between those two types.
TypeError: cannot add <class'sympy.matrices.immutable.ImmutableDenseMatrix'> and <class 'sympy.core.numbers.One'>
However, this can be circumvented by defining a function that evaluates the matrix polynomial for a specific matrix. The difference here is that we're using matrix exponentiation, so we correctly compute the free term of the matrix polynomial a_0 * I where I=A^0 is the identity matrix of the required shape:
from sympy import *
x = symbols('x')
M = Matrix([[1,2],[3,4]])
p = Poly(x**3 + x + 1)
def eval_poly_matrix(P,A):
res = zeros(*A.shape)
for t in enumerate(P.all_coeffs()[::-1]):
i, a_i = t
res += a_i * (A**i)
return res
eval_poly_matrix(p,M)
Output:
In this example the polynomial has only three terms but in reality I encounter (multivariate polynomials with) dozens of terms.
The function eval_poly_matrix above can be extended to work for multivariate polynomials by using the .monoms() method to extract monomials with nonzero coefficients, like so:
from sympy import *
x,y = symbols('x y')
M = Matrix([[1,2],[3,4]])
p = poly( x**3 * y + x * y**2 + y )
def eval_poly_matrix(P,*M):
res = zeros(*M[0].shape)
for m in P.monoms():
term = eye(*M[0].shape)
for j in enumerate(m):
i,e = j
term *= M[i]**e
res += term
return res
eval_poly_matrix(p,M,eye(M.rows))
Note: Some sanity checks, edge cases handling and optimizations are possible:
The number of variables present in the polynomial relates to the number of matrices passed as parameters (the former should never be greater than the latter, and if it's lower than some logic needs to be present to handle that, I've only handled the case when the two are equal)
All matrices need to be square as per the definition of the matrix polynomial
A discussion about a multivariate version of the Horner's rule features in the comments of this question. This might be useful to minimize the number of matrix multiplications.
Handle the fact that in a Matrix polynomial x*y is different from y*x because matrix multiplication is non-commutative . Apparently poly functions in sympy do not support non-commutative variables, but you can define symbols with commutative=False and there seems to be a way forward there
About the 4th point above, there is support for Matrix expressions in SymPy, and that can help here:
from sympy import *
from sympy.matrices import MatrixSymbol
A = Matrix([[1,2],[3,4]])
B = Matrix([[2,3],[3,4]])
X = MatrixSymbol('X',2,2)
Y = MatrixSymbol('Y',2,2)
I = eye(X.rows)
p = X**2 * Y + Y * X ** 2 + X ** 3 - I
display(p)
p = p.subs({X: A, Y: B}).doit()
display(p)
Output:
For more developments on this feature follow #18555
Could someone please help me understand what this function is doing if the input is a complex number a+bi and real = a, imag = b
I have no idea what it could be computing but maybe I am missing something obvious?
double function(double real, double imag)
{
double y;
double a;
double b;
a = fabs(real);
b = fabs(imag);
if (a < b)
{
a /= b;
y = b * sqrt(a * a + 1.0);
}
else if (a > b)
{
b /= a;
y = a * sqrt(b * b + 1.0);
}
else if (b == NAN)
{
y = b;
}
else
{
y = a * sqrt(2);
}
return y;
}
The code is a defective attempt to compute the magnitude (absolute value) of the complex number passed to it without incurring needless overflow.
Consider the complex number a + b i, where a and b are the values assigned to a and b in the first few lines of the function. Its magnitude is √(a2+b2). However, if a or b is large, the floating-point calculation a*a might overflow the finite range of the floating-point format and produce infinity (∞) even though the magnitude is within the range. As a simple example, let a be 21000 and b be 0. Then the magnitude is √(22000+0) = 21000, but computing sqrt(a*a + b*b) yields infinity. (Since a*a overflowed and produced ∞, and the rest of the calculations then produce ∞ too.)
The code attempts to solve that by dividing the smaller of a and b by the larger and using a calculation that is mathematically equivalent but that does not overflow. For example, if a < b is true, it executes:
a /= b;
y = b * sqrt(a * a + 1.0);
Then a /= b produces a value less than 1, so all calculation prior to the last are safely within the floating-point finite range: a * a is less than 1, a * a + 1.0 is less than 2, and sqrt(a * a + 1.0) is less than 1.42. When we multiply by b, the final result might overflow to ∞. There are two reasons this might happen: The magnitude of a + b i might exceed the floating-point finite range, so the overflow is correct. Or rounding in the prior calculations might have caused sqrt(a * a + 1.0) to be slightly larger than the mathematical result and sufficient to cause b * sqrt(a * a + 1.0) to overflow when actual value of the magnitude is within the range. As this is not our focus, I will not analyze this case further in this answer.
Aside from that rounding issue, the first two cases are fine. However, this code is incorrect:
else if (b == NAN)
Per IEEE-754 2008 5.11 and IEEE-754 1985 5.7, a NaN is not less than, equal to, or greater than any operand, including itself. It is unordered. This means b == NAN must return false if IEEE-754 is used. C 2018 does not require IEEE-754 be used, but footnote 22 (at 5.2.4.2.2 4) says that, if IEC 60559:1989 (effectively IEEE-754) is not supported, the terms “quiet NaN” and “signaling NaN” in the C standard are intended to apply to encodings with similar behavior. And 7.12 5 tells us that NAN expands to a float representing a quiet NaN. Thus, in b == NAN, NAN should behave as an IEEE-754 NaN, and so b == NAN should yield 0, for false.
Therefore, this code governed by else if (b == NAN) is never executed:
y = b;
Instead, execution falls through to the else, which executes:
y = a * sqrt(2);
If a is a NaN, the result is a NaN, as desired. However, if a is a number and b is a NaN, this produces a number as a result when the desired result would be a NaN. Thus, the code is broken.
Consider the following piece of code showing some simple arithmetic operations
int result = 0;
result = c * (a + b) + d * (a + b) + e;
To get the result in the expression above the cpu would need to execute two integer multiplications and three integer additions. However algebraically the above expression could be simplified to the code below.
result = (c + d) * (a + b) + e
The two expressions are algebraically identical however the second expression only contains one multiplication and three additions. Is gcc (or other compilers for that matter) able to make this simple optimization on their own.
Now assuming that the compiler is intelligent enough to make this simple optimization, would it be able to optimize something more complex such as the Trapezoidal rule (used for numerical integration). Example below approximates the area under sin(x) where 0 <= x <= pi with a step size of pi/4 (small for the sake of simplicity). Please assume all literals are runtime variables.
#include <math.h>
// Please assume all literals are runtime variables. Have done it this way to
// simplify the code.
double integral = 0.5 * ((sin(0) + sin(M_PI/4) * (M_PI/4 - 0) + (sin(M_PI/4) +
sin(M_PI/2)) * (M_PI/2 - M_PI/4) + (sin(M_PI/2) + sin(3 * M_PI/4)) *
(3 * M_PI/4 - M_PI/2) + (sin(3 * M_PI/4) + sin(M_PI)) * (M_PI - 3 * M_PI/4));
Now the above function could be written like this once simplified using the trapezoidal rule. This drastically reduces the number of multiplications/divisions needed to get the same answer.
integral = 0.5 * (1 / no_steps /* 4 in th case above*/) *
(M_PI - 0 /* Upper and lower limit*/) * (sin(0) + 2 * (sin(M_PI/4) +
sin(3 * M_PI/4)) + sin(M_PI));
GCC (and most C++ compilers, for that matter) does not refactor algebraic expressions.
This is mainly because as far as GCC and general software arithmetic is concerned, the lines
double x = 0.5 * (4.6 + 6.7);
double y = 0.5 * 4.6 + 0.5 * 6.7;
assert(x == y); //Will probably fail!
Are not guaranteed to be evaluate to the exact same number. GCC can't optimize these structures without that kind of guarantee.
Furthermore, order of operations can matter a lot. For example:
int x = y;
int z = (y / 16) * 16;
assert(x == z); //Will only be true if y is a whole multiple of 16
Algebraically, those two lines should be equivalent, right? But if y is an int, what it will actually do is make x equal to "y rounded to the lower whole multiple of 16". Sometimes, that's intended behavior (Like if you're byte aligning). Other times, it's a bug. The important thing is, both are valid computer code and both can be useful depending on the circumstances, and if GCC optimized around those structures, it would prevent programmers from having agency over their code.
Yes, optimizers, gcc's included, do optimizations of this type. Not necessarily the expression that you quoted exactly, or other arbitrarily complex expressions. But a simpler expresion, (a + a) - a is likely to be optimized to a for example. Another example of possible optimization is a*a*a*a to temp = a*a; (temp)*(temp)
Whether a given compiler optimizes the expressions that you quote, can be observed by reading the output assembly code.
No, this type of optimization is not used with floating points by default (unless maybe if the optimizer can prove that no accuracy is lost). See Are floating point operations in C associative? You can let for example gcc do this with -fassociative-math option. At your own peril.
I am trying to follow an algebraic equation, and convert it to c++.
I am now stuck on:
s.dx + x
and:
(s.dy + y) /a
s, dx, dy, x, y, and a are all doubles.
Google tells me that the . in the mathematical notation means the scalar product (http://www.rapidtables.com/math/symbols/Algebra_Symbols.htm), but how can I get that from doubles? What is this algorithm in c++?
Many thanks.
Google is wrong, you are looking for the * operator:
s * dx + x
(x * dy + y) / a
You should write s*dx+x and (s*dy+y)/a.
Perhaps you (or somebody) misunderstood the . thing, perhaps because a number constant in code (immediate number like 1 or 1.0) will have a different type depending whenever you use a .. So 10000 * 10000 is an integer multiplication, while 10000.0 * 10000.0 is an floating-point (double) multiplication. But here the . isn't an operator, but is part of the floating-point syntax.
As an operator in C++, a . is used to get a member of a object (variable or method).
If I had the sum of products like z*a + z*b + z*c + ... + z*y, it would be possible to move the z factor, which is the same, out before brackets: z(a + b + c + ... y).
I'd like to know how it is possible (if it is) to do the same trick if bitwise XOR is used instead of multiplication.
z^a + z^b + ... z^y -> z^(a + b + ... + y)
Perhaps a, b, c ... should be preprocessed, such as logically negated or something else, before adding? z could change, so preprocessing, if it's needed, shouldn't depend on particular z value.
From Wikipedia:
Distributivity: with no binary function, not even with itself
So, no, unfortunately, you can't do anything like that with XOR.
To prove that a general formula does not hold you only need to prove a contradiction in a limited case.
We can reduce it to show that this does not hold:
(a^b) * c = (a^c) * (b^c)
It is trivial to show that one base case fails as such:
a = 3
b = 1
c = 1
(a^b) * c = (3^1) * 1 = 2
(a^c) * (b^c) = 2 * 0 = 0
Using the same case you can show that (a*b) ^ c = (a^c) * (b^c) and (a + b) ^ c = (a^c) + (b^c) do not hold either.
Hence, equality does not hold in a general case.
Equality can hold in special cases though, which is an entirely different subject.