When trying to compile more recent versions of ImageMagick (v6.8.7-2 or later, v6.8.7-1 is fine), I get a bunch of:
CCLD magick/libMagickCore-6.Q16.la
magick/.libs/magick_libMagickCore_6_Q16_la-animate.o: In function `__pthread_cleanup_routine':
/usr/include/pthread.h:581: multiple definition of `__pthread_cleanup_routine'
magick/.libs/magick_libMagickCore_6_Q16_la-accelerate.o:/usr/include/pthread.h:581: first defined here
magick/.libs/magick_libMagickCore_6_Q16_la-annotate.o: In function `__pthread_cleanup_routine':
/usr/include/pthread.h:581: multiple definition of `__pthread_cleanup_routine'
magick/.libs/magick_libMagickCore_6_Q16_la-accelerate.o:/usr/include/pthread.h:581: first defined here
magick/.libs/magick_libMagickCore_6_Q16_la-artifact.o: In function `__pthread_cleanup_routine':
/usr/include/pthread.h:581: multiple definition of `__pthread_cleanup_routine'
magick/.libs/magick_libMagickCore_6_Q16_la-accelerate.o:/usr/include/pthread.h:581: first defined here
magick/.libs/magick_libMagickCore_6_Q16_la-attribute.o: In function `__pthread_cleanup_routine':
/usr/include/pthread.h:581: multiple definition of `__pthread_cleanup_routine'
magick/.libs/magick_libMagickCore_6_Q16_la-accelerate.o:/usr/include/pthread.h:581: first defined here
... goes on for quite a bit longer, all the same.
The pertinent area of /usr/include/pthread.h (from glibc-headers 2.5-118.el5_10.2) is:
/* Function called to call the cleanup handler. As an extern inline
function the compiler is free to decide inlining the change when
needed or fall back on the copy which must exist somewhere else. */
extern __inline void
__pthread_cleanup_routine (struct __pthread_cleanup_frame *__frame)
{
if (__frame->__do_it) // <======= this is :581
__frame->__cancel_routine (__frame->__cancel_arg);
}
I've been posting on ImageMagick's forum without response.
Even if you can't say exactly what's happening, how do I start figuring out whether the issue is with ImageMagick or pthread.h? Where do I go from there?
grep pthread_cleanup_routine -r * only shows matches against the binary object files -- none of ImageMagick's source code has pthread_cleanup_routine in it. A few of the sources include "pthread.h" of course.
That's leading me to believe that this is a glibc issue, not an ImageMagick issue... but, again, previous versions of ImageMagick compile just fine. (I have diff'ed the svn sources between versions where it broke. Lots of configuration/makefile changes, but nothing sticks out to me as to why it would cause this.)
I'm on CentOS 5, kernel 2.6.18-308.24.1.el5, gcc v4.9.0, ld v2.24, glibc-headers 2.5-118.el5_10.2
I've seen a lot of people posting similar issues with other packages than ImageMagick. Hopefully others will find this useful.
Changing pthread.h, just before __pthread_cleanup_routine :
extern __inline void
to
if __STDC__VERSION__ < 199901L
extern
#endif
__inline void
Fixes the issue. Older versions of glibc had an issue when -fexceptions was used, and inline non-C99 conformance (see http://gcc.gnu.org/ml/gcc-patches/2006-11/msg01030.html.) More recent glibc's would fix the issue too, but this should be a temp fix for those who don't want to / shouldn't upgrade it.
ImageMagick svn 13539 (which later became v6.8.7-2) began using -fexceptions.
I faced this error with a newer gcc compiler (4.9.3)
The ImageMagick(6.8.9_7) configure script was checking if compiler supports gnu99 standard. If yes, the configure script sets standard to gnu99 and also enables openmp.
Inline semantics change with C standard gnu99 causing multiple definition of the extern inline function
https://gcc.gnu.org/onlinedocs/gcc-4.9.3/gcc/Inline.html#Inline.
So, I added compiler flag -fgnu89-inline to use older semantics for inline and it fixed the issue.
Related
I am using clang power tools to compile a project which is usually compiled using visual studio.
In boost's lwm_win32.hpp header (yes we are using an old version of boost and currently cannot update) I get an error reading.
function declared stdcall here was previously declared without calling convention
the line in question is:
extern "C" __declspec(dllimport) long __stdcall InterlockedExchange(long volatile *, long);
I don't get any errors or warnings for this line when compiling with visual studio. Interestingly I don't get any even if I manually change the calling convention from __stdcall to __cdecl.
Clang tells me which previous declaration it has seen. By manually inspecting this location I would say clang is right. After deciphering all preprocessor defines I would also say __cdecl is what should be seen by visual studio. However, neither the official documentation for InterlockedExchange, nor the official documentation for the intrinsic do mention a specific calling convention.
So basically I am unsure what the root of the problem is. Visual studio accepting any calling convention in the declaration? Clang not seeing the correct declaration due to some preprocessor macros set to the wrong value? Boost declaring the wrong calling convention? I must admit I am confused.
Visual Studio version is 2015 Update 3.
Clang++ version is 6.0.0 called with parameter -fms-compatibility-version=19.
EDIT
As suggested in the comments I had a look at the preprocessor output of MSVC and Clang. They looked rather identical to me. For both the line from boost expands to
extern "C" __declspec(dllimport) long __stdcall _InterlockedExchange(long volatile *, long);
Both have
#pragma intrinsic(_InterlockedExchange)
and the declarations
long __cdecl _InterlockedExchange(long volatile * _Target, long _Value);
LONG __cdecl _InterlockedExchange(LONG volatile *Target, LONG Value);
as well as several inline implementations for different overloads.
In both compilers I target 32-bit (-m32 for clang).
Do the clang power tools offer you things that you really don't want to live without?
If not (and I imagine that is a big if) then you might consider experimenting with VS 2017's support for clang. I have no experience of it personally and it's all still a bit new but what I do know is that MS are putting a lot of work in and it may well pay off in the long run.
As it is, I think you might be out on a bit of a limb. And whatever should and should not be in the header files, I would say that what MS say goes, wouldn't you?
Why are you stuck with that old version of boost? That might be a blocking issue here.
Please consider following peace of code:
// 1. Single header file. Imagine that it is some static library.
// Counter.h
#pragma once
struct Counter
{
Counter()
{
++getCount();
}
static int& getCount()
{
static int counter = 0;
return counter;
}
};
// 2. Shared library (!) :
// main_DLL.cpp
#include <iostream>
#include "counter.h"
extern "C"
{
__declspec(dllexport) // for WIN
void main_DLL()
{
Counter c;
std::cout << "main_DLL : ptr = " << &Counter::getCount()<< " value = " << Counter::getCount() << std::endl;
}
}
// 3. Executable. Shared library statically (!) linked to the executable file.
// main.cpp
#include "counter.h"
#include <iostream>
extern "C"
{
__declspec(dllimport) // for WIN
void main_DLL();
}
int main()
{
main_DLL();
Counter c;
std::cout << "main_EXE : ptr = " << &Counter::getCount() << " value = " << Counter::getCount() << std::endl;
}
Results:
Results for WIN (Win8.1 gcc 5.1.0):
main_DLL : ptr = 0x68783030 value = 1
main_EXE : ptr = 0x403080 value = 1
// conclusion: two different counters
Results for UNIX (Red Hat <I don’t remember version exactly> gcc 4.8.3):
main_DLL : ptr = 0x75693214 value = 1
main_EXE : ptr = 0x75693214 value = 2
// conclusion: the same counter addressed
Building:
Building for WIN:
g++ -c -Wall -Werror -o main_DLL.o main_DLL.cpp
g++ -shared -Wl,--out-implib=libsharedLib.a -o libsharedLib.so main_DLL.o
g++ -Wall –Werror -o simpleExample main.cpp -L./ -lsharedLib
Building for UNIX:
g++ -c -Wall -Werror -fPIC -o main_DLL.o main_DLL.cpp
g++ -shared -fPIC -o libsharedLib.so main_DLL.o
g++ -Wall –Werror -fPIC -o simpleExample main.cpp -L./ -lsharedLib
So, you see that I added –fPIC on UNIX and there is no need to create import library for UNIX, because all exports symbols are included inside shared library. On Windows I use __declspec for it.
For me, results on Windows are pretty much expected. Because shared library and executable are building separately and they should know about static variable in Counter::getCount. They should simply allocate memory for it, that’s why they have different static counters.
I did quite some analysis using tools like nm, objdump. Although I’m not a big expert in them, so I haven’t found anything suspicious. I can provide their output if needed.
Using ldd tool I can see that library linked statically in both cases.
Why I can’t see the same results on Unix for me it’s strange. Could the root cause lie in building options (–fPIC, for example), or I’m missing something?
In windows, A DLL is not exporting global and static symbols unless you add the dllexport statement, therefore, the linker doesn't even know they exists, so it allocate new instance for the static member.
In linux/unix a shared lib is exporting all the global and static symbols, so when the linker find the existence of the static member in the shared lib, it just use its address.
That is the reason for the different result.
EDIT: This is a complete rewrite of the answer. With much more details.
I think that this question deserves more elaborated answer. Especially that there are things that were not mentioned so far.
Dependency Walker
Let me start with referring to the “Dependency Walker” program.
It is a nice program (although these days a bit old-schoolish in its look & feel) that allows analyzing Windows binaries (both EXE and DLL) for symbols that they export/import and their own dependencies to other DLLs. Also it allows showing undecorated symbol names but this seems to be working only with MSVC build binaries. (And some more but that is not important here.)
Thanks to this program crucial information (for this question) can be uncovered. So I encourage you to use it during experiments.
Exporting policy on Linux vs. Windows
SHR already pointed this out but I will mention it also for completeness of the answer. And some extra details.
On Linux every symbol gets exported from a shared library by default. On the other hand on Windows you have to explicitly state which symbols to export from a shared library.
GCC seems however to provide some means of controlling exports in "Windows style". See for example Visibility entry on GCC Wiki.
Also note that there are various ways of exporting on both Linux and Windows. For example both seem to support exporting selectively by providing linker with a list of names for symbols to export. But it also seems that nowadays (on Windows at least) this isn't really used much. __declspec approach seems to be preferred.
What can be exported?
After that general introduction let's now stick to Windows case. Nowadays you export/import symbols from shared libraries by using the __declspec. Just as shown in the question. (Well maybe not exactly that - typically you use a #define to handle bi-directionality as shown in already mentioned Visibility entry on GCC Wiki.)
But the declaration can be applied not only to functions, methods and global variables. It can also be applied to types. For example you can have:
class __declspec(dllexport) Counter { /* ... */ };
Such exporting/importing means in general that all members get exported/imported.
Not so easy!
But it would be too easy, wouldn't it? The complication is that GCC and MSVC handle exporting types differently.
My notes here are based mostly on experiments (checks done using Dependency Walker) so I can be wrong or not precise enough. But I did observe differences in behavior.
In tests I used MSVC 2013 from the Express Edition with update 5. For GCC I used MinGW distro from nuwen.net, version 13.0.
MSVC, when exporting entire type, exports each and every member. Including implicitly defined members (like compiler generated copy constructor). And including inlined functions. Furthermore if inlined function has some static local variables they get exported to (!).
GCC on the other hand seems to be far more restrictive. It doesn't export implicitly defined members. Nor it doesn't export inlined members.
Exporting/Importing inline functions
If instead of exporting entire type you would explicitly export an inlined function then and only then will GCC really export it. But still it will not export static local variables in that function.
Further more if you try to import an inlined function GCC will error. With GCC you cannot define symbols that you are importing. And this happens when you import inlined (and so defined) symbol. So in fact it doesn't make any sense to export inlined functions with GCC.
MSVC allows to import inlined functions. In all cases I checked it didn't seem to actually inline the function but instead called the imported version.
Yet note that because MSVC in case of inlined function exports also its static local variables it would be possible for it to really inline the function (rather than import it) while maintaining a single copy of static local variables. For ordinary programs such behavior is mandated by the Standard (N3337, C++11), in point 7.1.2 ([dcl.fct.spec]) at $4 we can read:
(…) A static local variable in an extern inline function always refers to the same object. (…)
But a program and a shared library are actually more like two programs so they are out of scope for the Standard. Yet MSVC even in that case acts (or better to say: could act) as one would expect from a single program.
Solution
Denis Bakhvalov in a comment provided solution for his own question. The solution is to move getCount function from header to source file and export/import it.
This seems to be the only solution portable between GCC and MSVC. Or to be more precise MSVC allows more solutions to this problem but none of them will work when program is build under GCC.
The variable trick
The above is not entirely true. There is another workaround that will work consistently between GCC and MSVC.
This is to stop using static local variable. Instead make it a global variable (most likely by making it static variable in the class) and export it. This will make the trick as well.
Sadly there is no way (or I don't know any) to directly force exporting/importing static local variables. You have to change them to global variables to do that.
MSVC solutions
With MSVC you have more options.
As mentioned before exporting/importing the inlined function itself (whether directly or through type) will do the job.
Summary
As described above even consistency between GCC and MSVC on Windows only requires care. You have to limit yourself to stay in common subset of allowed solutions.
Keeping the program (source) interoperable between Linux and Windows even if with same compiler (GCC) also requires care.
Luckily there is a common subset for all three environments: GCC on Linux, GCC on Windows and MSVC on Windows. That common subset is described already by mentioned Denis' comment.
So do not inline functions that you intend to export/import. Keep them in sources. And on Windows builds (regardless of compiler) export them explicitly (otherwise you will get linker error anyway since the functions in sources of a shared library will not be available when building program).
Note that this is actually a reasonable approach on its own. Inlining function from shared library doesn't seem wise. It freezes not only the interface but also implementation (of that function). You can no longer change this function freely (and deliver new version of your shared library) since all clients would have to be rebuild since they could have inlined that function. So it is a wise approach by itself not to inline from shared library. And as a bonus it assures that your sources are multi-platform friendly.
Also do have a look into the mentioned Visibility entry on GCC Wiki. It might be reasonable to use that approach (of explicit exports) on Linux as well since it seems cleaner (from design point of view) and more efficient at runtime. While it fits well what you have to do for Windows anyway.
Using gcc 4.6.1, I get some pretty bizarre errors at link time. I've defined various objects in a namespace SpacetimeAlgebra, and the compiler claims that they're already defined in pthread.h and in std::_Vector_base<double>::_M_deallocate. The errors look like this:
build/temp.linux-x86_64-2.7/Waveforms.o: In function `~vector':
GWFrames/Code/Waveforms.cpp:4978: multiple definition of `SpacetimeAlgebra::I3'
build/temp.linux-x86_64-2.7/SpacetimeAlgebra/SpacetimeAlgebra.o:/usr/include/pthread.h:1112: first defined here
Obviously, pthread.h does not actually directly contain anything involving my objects, and certainly not in that namespace. I seriously doubt that the error is in my code, as it compiles just fine with other compilers, and this is such a ridiculous error.
It strikes me as particularly bizarre that the "first defined" references change from pthread to _M_deallocate for objects that are defined in the same place. I don't have any using commands involving SpacetimeAlgebra. Is there something else I might be doing wrong?
The compiler command and full error output are at this gist. The command was created by python's distutils. The code itself is here, in the hpp and cpp files. (These were mostly generated by Gaigen, with a few tweaks from me.)
On a related note, the compilation works without any trouble on Apple LLVM 5.1. The compiler used here is on a cluster that many people successfully use all the time for some crazy compilations, so that at least usually works.
GWFrames/Code/Waveforms.cpp:4978: multiple definition of `SpacetimeAlgebra::psI'
build/temp.linux-x86_64-2.7/SpacetimeAlgebra/SpacetimeAlgebra.o:/usr/include/pthread.h:1112: first defined here
You're reading the error message wrong, because it's confusing (and possibly buggy, though before blaming the compiler I'd check that you have debug symbols in all your .os).
The error is telling you that you've defined SpacetimeAlgebra::psI inside Waveforms.cpp and inside SpacetimeAlgebra.cpp.
Unfortunately, we cannot see Waveforms.cpp, so this cannot be verified.
I'm debugging a running program with gdb 6.6 on solaris, and noticed that sometimes gdb steps into (inline) functions, even though I issued a next command.
My development host was recently reinstalled with a slightly newer build of solaris 10, and I know for sure the auto-stepping was not present before the host was reinstalled. The code is compiled with the same options since the makefiles and all the source code is unchanged since host reinstallation.
Is there any setting/new default option which influences gdb's debugging behaviour that I can check? Does anyone know why my gdb now auto-steps? Its a pain really ...
[edit] to clarify: I did not mean the inline keyword, but rather methods/functions which are implemented in the header file. Example:
header.hpp:
class MyClass
{
public:
void someFunc() { ... does something }
}
source.cc:
{
MyClass instance;
instance.someFunc(); // doing NEXT in gdb will actually STEP into header.hpp
}
Your new version of Solaris may have included a new version of the C or C++ compiler. The new compiler may be optimizing more aggressively than it did before. Check your optimization flags. If you are using GCC, you can disable inlining with -fno-inline (note that methods that are implemented in the class in header files are inlined by default which can be disabled with -fno-default-inline). If you are using the native Solaris compiler, you will need to check its documentation.
A similar problem was reported here. In the comment, the poster mentioned changing the debug symbol to use STABS resolved the issue.
You mentioned in a comment to my answer that STABS works, but is not acceptable. Also, you mentioned that you are unable to reproduce the issue with a simple example. It will be difficult to trouble shoot this issue if you have to recompile your entire project each time to perform a test. Try to isolate the problem to a few source files in your project. See what they have in common (do they include a common header file, do they use a pragma, are the compilation options a little different from the other source fies, etc.), and try to create a small example with the same problem. This will make it easier to identify the root cause of your issue and determine how to resolve it. Without this data, we are just the blind leading the blind.
I am trying to implement backtrace functionality for a large framework, which is used for different platforms and OS'es. In some of them, it is linked against glibc, while in the other, something different (eg. uclibc) is used. backtrace() function exists only in the former.
Is there any way to tell whether glibc is used? Any #define? I was unable to find an answer in glibc manual. I know I can't have linking-time information during compilation, but I guess include files have to differ. At least backtrace have to be declared somewhere.
I would like to check it without being forced to pass explicit flags to the compiler.
Include features.h, it contains the macros you need, e.g.
#define __GNU_LIBRARY__ 6
/* Major and minor version number of the GNU C library package. Use
these macros to test for features in specific releases. */
#define __GLIBC__ 2
#define __GLIBC_MINOR__ 4
There are the #defines __GNU_LIBRARY__, __GLIBC__ and __GLIBC_MINOR__ (6, 2 and 11 on my system with glibc-2.11) in features.h.
Checking for preprocessor macros is not a good solution. uClibc and possibly other libc implementations define macros to mimic glibc (without providing all of its bloated functionality) for much the same reasons that all browsers include "Mozilla" in their User-Agent strings: broken programs that expect to see glibc and turn off lots of features if they don't see it.
Instead you should write a configure script to probe for backtrace and use it only if it's available.
Empirically, both of the following compile and run fine on GCC 6.4:
#include <stdio.h>
int main(void) {
#ifdef __GLIBC__
puts("__GLIBC__");
#endif
return 0;
}
and:
int main(void) {
#ifdef __GLIBC__
puts("__GLIBC__");
#endif
return 0;
}
but only the first produces output of course.
This must mean that __GLIBC__ comes from stdio.h which must include features.h, see also: What is the purpose of features.h header?
Therefore, strictly speaking, __GLIBC__ by itself is not a clear indication that glibc is used, since even without headers, GCC already embeds runtime objects such as crt1.o in the finale executable, and those come from glibc.
So the main missing question is: does glibc guarantee that features.h gets included by every header? I could not find a clear documentation quote. TODO.
#if defined(__GLIBC__) && !defined(__UCLIBC__) && !defined(__MUSL__)
This is getting a bit ugly and syntactically ambiguous, but useful.