Another case of "Be extra careful with `auto` in the Eigen library." - c++

EDIT: The title was
How do eigen expressions and link-time-optimization interact
But this issue has nothing to do with lto.
I just introduced and then fixed the following bug:
Eigen::Vector4d some_function(double arg) {
Eigen::Matrix4d mat;
mat << 2,2,3,4,5,6,7,8,9,10,11,2,13,14,15,16;
Eigen::Vector4d spline_constraints(a(arg),b(arg),c,d);
auto coefficients = mat.lu().solve(spline_constraints);
return coefficients;
}
This is pseudocode. In actuality both the function and the matrix are static members of a class and a(double arg) and b(double arg) are also static member functions.
I hope it is suffcient. If not, I will try to cook up a minimal working example.
The behaviour is very compiler dependent:
GCC
When I compile it into a static library and link the library elsewhere, it SEEMS to work fine, regardless of debug build or release build (i.e. with non-lto optimizations).
If I compile and link it with link time optimization, it compiles and links fine and the runtime behaves in a wrong way:
With gcc a sparse matrix of size 1350x1350, whose construction depends on the code above is claimed to be non-decomposible with the lu decomposition (without lto it doesn't claim so, although I'm not sure, the decomposition is correct in that case).
Clang
In release build with or without lto I get a segmentation fault.
The debug build works fine.
Fix
The code is fixed by with one of the following:
Eigen::Vector4d some_function() {
Eigen::Matrix4d mat;
mat << 2,2,3,4,5,6,7,8,9,10,11,2,13,14,15,16;
Eigen::Vector4d spline_constraints(a,b,c,d);
auto coefficients = mat.lu().solve(spline_constraints).eval();
return coefficients;
}
that is by adding .eval()
or
Eigen::Vector4d some_function() {
Eigen::Matrix4d mat;
mat << 2,2,3,4,5,6,7,8,9,10,11,2,13,14,15,16;
Eigen::Vector4d spline_constraints(a,b,c,d);
return mat.lu().solve(spline_constraints);
}
that is, by returning the expression itself.
My question is: What exactly happened here?
Related (would be nice to know, not really part of the question): Is there a compiler warning or other feature that could prevent me from making this mistake in the future?
Also (likewise): Is there some part of the Eigen docs, that would have warned me of this, had I found it before?
My impression from the docs was, that failing to add .eval() somewhere could result in decreased performance but not in undefined behaviour.
EDIT: I was wrong: The docs state that wrong application of eval() can lead to segfaults:
Eigen docs on eval()

Related

Netbeans 8.2 code assistance issue with unique_ptr (not shared_ptr)

Using Netbeans 8.2 on Linux and GCC 8.1, unique_ptr::operator->() gives an erroneous warning
Unable to resolve template based identifier {var} and more importantly will not autocomplete member names.
Code compiles and runs just fine, and amazingly, the autocomplete still works with no warnings if a shared_ptr is used instead. I have no idea how that is possible. Here is a problematic example, for reference
#include <iostream>
#include <memory>
struct C { int a;};
int main () {
std::unique_ptr<C> foo (new C);
std::shared_ptr<C> bar (new C);
foo->a = 10; //Displays warning, does not auto-complete the members of C
bar->a = 20; //No warning, auto-completes members of C
return 0;
}
I've tried the following to resolve this issue with no luck:
Code Assistance > Reparse Project
Code Assistance > Clean C/C++ cache and restart IDE
Manually deleting cache in ~/.cache/netbeans/8.2
Looking through View > IDE Log for anything that may help
Setting both the C and C++ compilers to C11 and C++11 in project properties
Changing the pre-processing macro __cplusplus to both 201103L and 201402L
Creating a new Netbeans project and trying the above
A large variety of permutations of the above options in different orders
Again, everything compiles and runs just fine, it's just Code Assistance that is giving me an issue. I've run out of things that I've found in other stack overflow answers. My intuition tells me that shared_ptr working and unique_ptr not working is helpful, but I don't know enough about C++ to utilize that information. Please help me, I need to get back to work...
Edit 1
This stack overflow question, although referencing Clang and the libc++ implementation, suggests that it may be an implementation issue within GCC 8.1's libstdc++ unique_ptr.
TLDR Method 1
Add a using pointer = {structName}* directive in your structure fixes code assistance, and will compile and run as intended, like so:
struct C { using pointer = C*; int a;};
TLDR Method 2
The answer referenced in my edit does in fact work for libstdc++ as well. Simply changing the return type of unique_ptr::operator->() from pointer to element_type* will fix code assistance, compile and run as expected (the same thing can be done to unique_ptr::get()). However, changing things in implementations of the standard library worries me greatly.
More Information
As someone who is relatively new to c++ and barely understands the power of template specializations, reading through unique_ptr.h was scary, but here is what I think is messing up Netbeans:
Calling unique_ptr::operator->() calls unique_ptr::get()
unique_ptr::get() calls the private implementation's (__unique_ptr_impl) pointer function __unique_ptr_impl::_M_ptr(). All these calls return the __unique_ptr_impl::pointer type.
Within the private implementation, the type pointer is defined within an even more private implementation _Ptr. The struct _Ptr has two template definitions, one that returns a raw pointer to the initial template variable of unique_ptr, and the second that seems to strip any reference off this template variable, and then find it's type named pointer. I think that this is where Netbeans messes up.
So my understanding is when you call unique_ptr<elementType, deleterType>::operator->(), it goes to __unique_ptr_impl<elementType, deleterType>, where the internal pointer type is found by stripping elementType of any references and then getting the type named dereferenced(elementType)::pointer. So by including the directive using pointer =, Netbeans gets what it wants in finding the dereferenced(elementType)::pointer type.
Including the using directive is entirely superficial, evidenced by the fact that things will compile without it, and by the following example
#include <memory>
#include <iostream>
struct A{
using pointer = A*;
double memA;
A() : memA(1) {}
};
struct B {
using pointer = A*;
double memB;
B() : memB(2) {}
};
int main() {
unique_ptr<A> pa(new A);
unique_ptr<B> pb(new B);
std::cout << pa->memA << std::endl;
std::cout << pb->memB << std::endl;
};
outputs
1
2
As it should, even though in struct B contains using pointer = A*. Netbeans actually tries to autocomplete B-> to B->memA, further evidence that Netbeans is using the logic proposed above.
This solution is contrived as all hell but at least it works (in the wildly specific context that I've used it) without making changes to implementations of stl. Who knows, I'm still confused about the convoluted typing system within unique_ptr.

Create a const Eigen (Eigen_Library) Matrix REVISITED

i have an problem with definining some constant Eigen (eigen.tuxfamily.org) Vectors in a header file, but obviously it's more a compiler problem than an Eigen-specific problem..
Defining this in a header file:
const double[] hardcodedData = {1,2,3};
const Vector3d myConstVector(hardcodedData);
works perfectly using Microsoft VC2010 via cython/distutils (which i use for testing).
Once the header file is included, i can access myConstVectory from every function/method/whatever and use it for calculations..
Using the same code with:
Microsoft (R) C/C++ Optimizing Compiler Version 17.00.50727.1 for x64
which is called by ABAQUS 6.13-2 , a finite element software,
every const Vector is initialized with zeros! Until now, i found no workaround except using something like this:
const Vector3d myConstVector()
{
const static Vector3d vec(hardcodedData);
return vec;
}
This workaround is OK, but not really what i intended to do .. Also it has some overhead.
Is there a clean solution, to get the "hardcoded" option running? Thx in advance!

Internal Compiler Error on Array Value-Initialization in VC++14 (VS2015)

I'm getting an ICE on Visual Studio 2015 CTP 6. Unfortunately, this is happening in a large project, and I can't post the whole code here, and I have been unable to reproduce the problem on a minimal sample. What I'm hoping to get is help in constructing such a sample (to submit to Microsoft) or possibly illumination regarding what's happening and/or what I'm doing wrong.
This is a mock-up of what I'm doing. (Note that the code I'm presenting here does NOT generate an ICE; I'm merely using this simple example to explain the situation.)
I have a class A which is not copyable (it has a couple of "reference" members) and doesn't have a default constructor. Another class, B holds an array of As (plain C array of A values, no references/pointers) and I'm initializing this array in the constructor of B using uniform initialization syntax. See the sample code below.
struct B;
struct A
{
int & x;
B * b;
A (B * b_, int & x_) : x (x_), b (b_) {}
A (A const &) = delete;
A & operator = (A const &) = delete;
};
struct B
{
A a [3];
int foo;
B ()
: a {{this,foo},{this,foo},{nullptr,foo}} // <-- THE CULPRIT!
, foo (2)
{ // <-- This is where the compiler says the error occurs
}
};
int main ()
{
B b;
return 0;
}
I can't use std::array because I need to construct the elements in their final place (can't copy.) I can't use std::vector because I need B to contain the As.
Note that if I don't use an array and use individual variables (e.g. A a0, a1, a2;, which I can do because the array is small and fixed in size) the ICE goes away. But this is not what I want since I'll lose ability to get to them by index, which I need. I can use a union of the loose variables over the array to solve my ICE problem and get indexing (construct using the variables, access using the array,) but I think that would result in "undefined behavior" and seems convoluted.
The obvious differences between the above sample and my actual code (aside from the scale) is that A and B are classes instead of structs, each is declared/defined in its own source/header file pair, and none of the constructors is inline. (I duplicated these and still couldn't reproduce the ICE.)
For my actual project, I've tried cleaning the built files and rebuild, to no avail. Any suggestions, etc.?
P.S. I'm not sure if my title is suitable. Any suggestions on that?!?!
UPDATE 1: This is the compiler file referenced in the C1001 fatal error message: (compiler file 'f:\dd\vctools\compiler\utc\src\p2\main.c', line 230).
UPDATE 2: Since I had forgotten to mention, the codebase compiles cleanly (and correctly) under GCC 4.9.2 in C++14 mode.
Also, I'm compiling with all optimizations disabled.
UPDATE 3: I have found out that if I rearrange the member data in B and put the array at the very end, the code compiles. I've tried several other permutations and it sometimes does compile and sometimes doesn't. I can't see any patterns regarding what other members coming before the array make the compiler go full ICE! (being UDTs or primitives, having constructors or not, POD or not, reference or pointer or value type, ...)
This means that I have sort of a solution for my problem, although my internal class layout is important to me and this application, I can tolerate the performance hit (due to cache misses resulting from putting some hot data apart from the rest) in order to get past this thing.
However, I still really like a minimal repro of the ICE to be able to submit to Microsoft. I don't want to be stuck with this for the next two years (at least!)
UPDATE 4: I have tried VS2015 RC and the ICE is still there (although the error message refers to a different internal line of code, line 247 in the same "main.c" file.)
And I have opened a bug report on Microsoft Connect.
I did report this to Microsoft, and after sharing some of my project code with them, it seems that the problem has been tracked down and fixed. They said that the fix will be included in the final VC14 release.
Thanks for the comments and pointers.

Same Program code with same compiler leads to different binaries

I have an issue with my code that has some very strange symptoms.
The code is compiled on my computer with the following versions:
a. GCC Version: 4.4.2
b. CMAKE verson: 2.8.7
c. QNX (operating system) version: 6.5.0
And the code has a segfault whilst freeing some memory and exiting from a function (not dying on any code, just on the exit from a function).
The weird things about this are:
The code does it in release mode but not debug mode:
a. The code is threaded so this indicates a race condition.
b. I cannot debug by putting it in debug mode.
The code when compiled on a workmates machine with the same versions of everything, does not have this problem.
a. The wierd things about this are that the workmates code works, but also that the binary created from compiling on his machine, which is the same, is about 6mB bigger.
Now annoyingly I cannot post the code because it is too big and also for work. But can anyone point me along a path to fixing this.
Since I am using QNX I am limited for my debug tools, I cannot use Valgrind and since it is not supported in QNX, GDB doesn't really help.
I am looking for anyone who has had a similar/same problem and what the cause was and how they fixed it.
EDIT:
Sooo... I found out what it was, but im still a bit confused about how it happened.
The culprit code was this:
Eigen::VectorXd msBb = data.modelSearcher->getMinimumBoundingBox();
where the definition for getMinimumBoundingBox is this:
Eigen::VectorXd ModelSearcher::getMinimumBoundingBox();
and it returns a VectorXd which is always initialised as VectorXd output(6, 1). So I immediately thought, right it must be because the VectorXd is not being initialised, but changing it to this:
Eigen::VectorXd msBb(6, 1); msBb = data.modelSearcher->getMinimumBoundingBox();
But this didn't work. In fact I had to fix it by changing the definition of the function to this:
void ModelSearcher::getMinimumBoundingBox(Eigen::MatrixXd& input);
and the call to this
Eigen::VectorXd msBb(6, 1); data.modelSearcher->getMinimumBoundingBox(msBb);
So now the new question:
What the hell? Why didn't the first change work but the second did, why do I have to pass by reference? Oh and the big question, how the hell didn't this break when my co-worker compiled it and I ran it? Its a straight out memory error, surely it shouldn't depend on which computer compiles it, especially since the compiler and all the other important things are the same!!??
Thanks for your help guys.
... the binary created from compiling on his machine, which is the same, is about 6mB bigger
It's worth figuring out what the difference is (even if it's just the case that his build hides, while yours exposes, a real bug):
double-check you're compiling exactly the same code (no un-committed local changes, no extra headers in the include search path, etc.)
triple-check by adding a -E switch to your gcc arguments in cmake, so it will pre-process your files with the same include path as regular compilation; diff the pre-processor output
compare output from nm or objdump or whatever you have to for your two linked executables: if some system or 3rd-party library is a different version on one box, it may show up here
compare output from ldd if it's dynamically linked, make sure they're both getting the same library versions
compare the library versions it actually gets at runtime too, if possible. Hopefully you can do one of: run pldd, compare the .so entries in /proc/pid/map, run the process under strace/dtrace/truss and compare the runtime linker activity
As for the code ... if this doesn't work:
Eigen::VectorXd ModelSearcher::getMinimumBoundingBox();
// ...
Eigen::VectorXd msBb(6, 1); msBb = data.modelSearcher->getMinimumBoundingBox();
and this does:
void ModelSearcher::getMinimumBoundingBox(Eigen::MatrixXd& input);
// ...
Eigen::VectorXd msBb(6, 1); data.modelSearcher->getMinimumBoundingBox(msBb);
you presumably have a problem with the assignment operator. If it does a shallow copy and there is dynamically-allocated memory in the vector, you'll end up with two vectors holding the same pointer, and they'll both free/delete it.
Note that if the operator isn't defined at all, the default is to do this shallow copy.
You said you have to change from:
void ModelSearcher::getMinimumBoundingBox(Eigen::MatrixXd& input);
What was it before?
If it was:
void ModelSearcher::getMinimumBoundingBox(Eigen::MatrixXd input);
and the copy constructors / assignment operators weren't implemented properly it might have caused the problem.
Please do check how they are both implemented. Here's some info that might help.

accessing element of boost sparse_matrix seems to stall program

I've got a strange bug that I'm hoping a more experience programmer might have some insight into. I'm using the boost ublas sparse matrices, specifically mapped_matrix, and there is an intermittent bug that occurs eventually, but not in the initial phases of the program. This is a large program, so I cannot post all the code but the core idea is that I call a function which belongs to a particular class:
bool MyClass::get_cell(unsigned int i, unsigned int j) const
{
return c(i,j);
}
The variable c is defined as a member of the class
boost::numeric::ublas::mapped_matrix<bool> c;
When the bug occurs, the program seems to stop (but does not crash). Debugging with Eclipse, I can see that the program enters the boost mapped_matrix code and continues several levels down into std::map, std::_Rb_tree, and std::less. Also, the program occasionally traces down to std::map, std::_Rb_tree, and std::_Select1st. While code is executing and the active line what's in memory changes in _Rb_tree, execution never seems to return in the level of std::map. The line in std::map the program is stuck on is the return of the following function.
const_iterator
find(const key_type& __x) const
{ return _M_t.find(__x); }
It seems to me that there is some element in the c matrix that the program is looking for but somehow the underlying storage mechanism has "misplaced it". However, I'm not sure why or how to fix it. That could also be completely off base.
Any help you can provide would be greatly appreciated. If I have not included the right information in this question, please let me know what I'm missing. Thank you.
Some things to try to debug the code (not necessarily permanent changes):
Change the bool to an int in the matrix type for c, to see if the matrix expects numeric types.
Change the matrix type to another with a similar interface, possibly plain old matrix.
Valgrind the app (if you're on linux) to check you're not corrupting memory.
If that fails, you could try calling get_cell every time you modify the matrix to see what might be causing the problem.
Failing that, you may have to try reduce the problem to a much smaller subset of code which you can post here.
It might help if you tell us what compiler and OS you're using.
Is this part of a multithreaded program?
I ask, because usually when I see problems in STL, it ends up being a problem with unsynchronized access.