Qt + C++ + C gurus, I could some advice:
I started writing a Qt6 GUI C++ app that includes some extern C code. Much to my surprise, upon launch the ui->setupUi(this) call immediately segfaulted and I found myself inside of one of the extern C functions that was imported! This is literally at the first line of my MainWindow constructor - with no calls to any of the C code even added yet.
The C function where the segfault dumped me was named "crc32". I temporarily renamed it (and all calls to it) to test my theory and sure enough the program started correctly afterwards. So... Qt appears to invoke "crc32" when doing things like adding QMenu items... due to some name collision.
I'm puzzled how to fix it. I don't want to change the C code because it's actually a git submodule for a project I do not want to maintain a fork of forever. Since it's Qt calling it from the C++ side, I can't exactly specify the namespace on the call either.
Weird sidenote: When I was doing testing this morning with Qt5 (instead of 6), I was not running into this issue. The crc32 name collision may be specific to Qt6.
Looks like you are on Linux (or similar) and encountering a violation of the One Definition Rule. When the shared lib loader tries to load symbols from a shared lib, it will first check to see if it already has one with that name loaded. If so, it skips it. Thus, if you try to load two symbols with an identical name, you'll only get one of them, and then problems can ensue.
Possible options:
Hide symbols you don't need in the library. For this, consider using GCC's visibility controls.
Separate the symbols into different executables and share data with some interprocess communication like shared mem or a socket.
Use dlopen() on one of the libs along with the RTLD_LOCAL flag so it does not override the symbol.
Add a namespace to the project (or use its build options to control that, if available -- like Boost does). Yes, this violates your wish not to modify it, but it could be done with a patch or similar that does not modify the original checkout if you switched to using a package manager like Conan instead of a git submod.
crc32 is a symbol from zlib. Qt does use either the system zlib, or a built-in version of it. If it uses the built-in version, symbols are prefixed with z_ (so z_crc32) to avoid exactly such a clash.
So, if you want to use crc32, you have the options to configure Qt to either not use zlib at all (-no-zlib), or use the built-in version (-qt-zlib).
Related
I'd like my Windows application to be able to reference an extensive set of classes and functions wrapped inside a DLL, but I need to be able to guide the application into choosing the correct version of this DLL before it's loaded. I'm familiar with using dllexport / dllimport and generating import libraries to accomplish load-time dynamic linking, but I cannot seem to find any information on the interwebs with regard to possibly finding some kind of entry point function into the import library itself, so I can, specifically, use CPUID to detect the host CPU configuration, and make a decision to load a paricular DLL based on that information. Even more specifically, I'd like to build 2 versions of a DLL, one that is built with /ARCH:AVX and takes full advantage of SSE - AVX instructions, and another that assumes nothing is available newer than SSE2.
One requirement: Either the DLL must be linked at load-time, or there needs to be a super easy way of manually binding the functions referenced from outside the DLL, and there are many, mostly wrapped inside classes.
Bonus question: Since my libraries will be cross-platform, is there an equivalent for Linux based shared objects?
I recommend that you avoid dynamic resolution of your DLL from your executable if at all possible, since it is just going to make your life hard, especially since you have a lot of exposed interfaces and they are not pure C.
Possible Workaround
Create a "chooser" process that presents the necessary UI for deciding which DLL you need, or maybe it can even determine it automatically. Let that process move whatever DLL has been decided on into the standard location (and name) that your main executable is expecting. Then have the chooser process launch your main executable; it will pick up its DLL from your standard location without having to know which version of the DLL is there. No delay loading, no wonkiness, no extra coding; very easy.
If this just isn't an option for you, then here are your starting points for delay loading DLLs. Its a much rockier road.
Windows
LoadLibrary() to get the DLL in memory: https://msdn.microsoft.com/en-us/library/windows/desktop/ms684175(v=vs.85).aspx
GetProcAddress() to get pointer to a function: https://msdn.microsoft.com/en-us/library/windows/desktop/ms683212(v=vs.85).aspx
OR possibly special delay-loaded DLL functionality using a custom helper function, although there are limitations and potential behavior changes.. never tried this myself: https://msdn.microsoft.com/en-us/library/151kt790.aspx (suggested by Igor Tandetnik and seems reasonable).
Linux
dlopen() to get the SO in memory: http://pubs.opengroup.org/onlinepubs/009695399/functions/dlopen.html
dladdr() to get pointer to a function: http://man7.org/linux/man-pages/man3/dladdr.3.html
To add to qexyn's answer, one can mimic delay loading on Linux by generating a small static stub library which would dlopen on first call to any of it's functions and then forward actual execution to shared library. Generation of such stub library can be automatically generated by custom project-specific script or Implib.so:
# Generate stub
$ implib-gen.py libxyz.so
# Link it instead of -lxyz
$ gcc myapp.c libxyz.tramp.S libxyz.init.c
I had a program written using Qt's shared library version (5.2.0) and it displayed fonts okay (presumably filling it's Database from /usr/share/fonts). (Not sure whether the Designer text is from the same Database as the programmed font handling.) But then I recompiled it under the open source static version (5.2.0), and the fonts went ugly.
I disrupted the path to the installed Qt (since the whole idea of the static version is to be independent of development directories) and sure enough it complained of not being able to access any fonts. So I concluded Qt changed their static version to no longer input /usr/share/fonts. A little research on the internet told me they instead look under ...somedir/lib/fonts.
With a little more research I found I could use env var QT_QPA_FONTDIR to set the directory of search (though I suspect it might have been reduced to a one-level search, not recursive). Not wanting to have to write a script to wrap the program and set the environment variable before calling it, I proceeded to use a putenv() in the program. I chose a place that I thought would set the variable before Qt read the environment, and since global contructors are called before main() and certainly before QApplication is instantiated, I put it in a constructor.
But, guess what, they seem to read the environment before my constructor set the QT_QPA_FONTDIR=/usr/share/fonts.
So please help!!!! How do I make MY constructor get executed first?
Some compilers offer pragmas that attempt to help you do such things, but the simple truth is that C++ offers no guarantees of the order of global construction between modules. There isn't even a guarantee that one object dependent on another in another module will be initialized second. Globals are a minefield in this respect.
If you need a portable solution, there really isn't one.
If you can guarantee your compiler will always be the same flavor, then you can use hacks like pragmas, attributes, or link order to work around the problem. For instance, gcc offers a variable attribute called init_priority, though I'm not sure you can make your priority higher than the default.
Really, it would be better to solve the problem another way.
Thank you Aiken Drum. The link order worked. I created the smallest .cpp that made a constructor to be instantiated globally calling the putenv() to set up my environment. I put it first in the .pro file, and checked that the makefile it created had it first in the link line. And, hey presto!! the Qt program now gets the font Database from the right place. Thank you so much. It's the link order that sets the constructor order (at least with GCC).
I am developing a C++ application which is required to load a dynamic library at runtime using dlopen. This library generally won't be written by me.
What method do people recommend to ensure future binary compatibility between this library and my application?
The options as I see them are:
Put the version number in the library file name, and attempt to load it (through a symbolic link) no matter what. If dlopen fails, report an error.
Maintain a second interface which returns an version number. However, if this interface changes for some reason, we run into the same problems as before.
Are there any other options?
You should define a convention about the dynamically loaded (i.e. dlopen-ed) library.
You might have the convention that the library is required to provide a const char mylib_version_str[]; symbol which gives the version of the API etc etc. Of course you could have your own preprocessor tricks to help about this.
For your inspiration, you might look at what GCC requires from its plugins (e.g. the plugin_is_GPL_compatible symbol).
If the dynamically loaded library is in C++, you might use demangling to check the signature of functions....
why not use both options? by istance few libraries already do that (Lua for example, old dll is Lua51.dll, then you have Lua52 etc. nd you can also query its version.)
A good interface can change, but not so often, why should 2 simple static methods
const char* getLibraryName();
uint32 getLibraryVersion();
change overtime?
if you/they are using a libtool to build a library/application you may recommend this way: http://www.gnu.org/software/libtool/manual/libtool.html#Versioning
I have 3 things: open source application (let's call it APP),
closed source shared library (let's call it OPENGL)
and open source plugin for OPENGL (let's call it PLUGIN)[also shared library].
OS: Linux.
There is need to share data between APP and PLUGIN,
so APP linking with PLUGIN, and when I run it,
system load it automatically.
After that APP call eglInitialize that belongs to OPENGL,
and after that this function load PLUGIN again.
And after that I have two copies of PLUGIN in the APP memory.
I know that because of PLUGIN have global data, and after debugging
I saw that there are two copies of global data.
So question how I can fix this behaviour?
I want one instance of PLUGIN, that used by APP and OPENGL.
And I can not change OPENGL library.
It obviously depends a lot on exactly what the libraries are doing, but in general some solution should be possible.
First note that normally if a shared library with the same name is loaded multiple times it will continue to use the same library. This of coruse primarily applies to loading via the standard loading/linking mechanism. If the library calls dlopen on its own it still can get the same library but it depends on the flags to dlopen. Try reading the docs on dlopen to get an understanding of how it works and how you can manipulate it.
You can also try positioning the PLUGIN earlier in the linker command so that it gets loaded first and thus might avoid a double load later one. If you must load the PLUGIN dynamically this obviously won't help. You can also check if LD_PRELOAD might resolve the linking order.
As a last resort you may have to resort to using LD_LIBARY_PATH and putting an interface library in from of the real one. This one will simply pass calls to the real one but will intercept duplicate loads and shunt them to the previous load.
This is just a general direction to consider. Your actual answer will depend highly on your code and what the other shared libraries do. Always investigate linker load ordering first, as it is the easiest to check, and then dlopen flags, before going into the other options.
I suspect that OPENGL is loading PLUGIN with the RTLD_LOCAL flag. This
is normally what you want when loading a plugin, so that multiple
plugins don't conflict.
We've had similar problems with loading code under Java: we'd load a
dozen or so different modules, and they couldn't communicate with one
another. It's possible that our solution would work for you: we wrote a
wrapper for the plugin, and told Java that the wrapper was the plugin.
That plugin then loaded each of the other shared objects, using dlopen
with RTLD_GLOBAL. This worked between plugins. I'm not sure that it
will allow the plugins to get back to the main, however (but I think it
should). And IIRC, you'll need special options when linking main for
its symbols to be available. I think Linux treats the symbols in main
as if main had been loaded with RTLD_LOCAL otherwise. (Maybe
--export-dynamic? It's been a while since I've had to do this, and I
can't remember exactly.)
I have this habit always a C++ project is compiled and the release is built up. I always open the .EXE with a hexadecimal editor (usually HxD) and have a look at the binary information.
What I hate most and try to find a solution for is the fact that somewhere in the string table, relevant (at least, from my point of view) information is offered. Maybe for other people this sounds like a schizophrenia obsession but I just don't like when my executable contains, for example, the names of all the Windows functions used in the application.
I have tried many compilers to see which of them published the least information. For example, GCC leaves all this in all of its produced final exe
libgcj_s.dll._Jv_RegisterClasses....\Data.ald.rb.Error.Data file is corrupt!
....Data for the application not found!.€.#.ř.#.0.#.€.#.°.#.p.#.p.#.p.#.p.#.
¸.#.$.#.€.#°.#.std::bad_alloc..__gnu_cxx::__concurrence_lock_error.__gnu_cxx
::__concurrence_unlock_error...std::exception.std::bad_exception...pure virt
ual method called..../../runtime/pseudo-reloc.c....VirtualQuery (addr, &b, s
ize of(b))............................/../../../gcc-4.4.1/libgcc/../gcc/conf
ig/i386/cygming-shared-data.c...0 && "Couldn't retrieve name of GCClib share
d data atom"....ret->size == sizeof(__cygming_shared) && "GCClib shared data
size mismatch".0 && "Couldn't add GCClib shared data atom".....-GCCLIBCYGMI
NG-EH-TDM1-SJLJ-GTHR-MINGW32........
Here, you can see what compiler I used, and what version. Now, a few lines below you can see a list with every Windows function I used, like CreateMainWindow, GetCurrentThreadId, etc.
I wonder if there are ways of not displaying this, or encrypting, obfuscating it.
With Visual C++ this information is not published. Instead, it is not so cross-platform as GCC, which even between two Windows systems like 7 and XP, doesn't need C++ run-time, frameworks or whatever programs compiled with VC++ need. Moreover, the VC++ executables also contain those procedures entry points to the Windows functions used in the application.
I know that even NASM, for example, saves the name of the called Windows functions, so it looks like it's a Windows issue. But maybe they can be encrypted or there's some trick to not show them.
I will have a look over the GCC source code to see where are those strings specified to be saved in the executables - maybe that instruction can be skipped or something.
Well, this is one of my last paranoia and maybe it can be treated some way. Thanks for your opinions and answers.
If you compile with -nostdlib then the GCC stuff should go away but you also lose some of the C++ support and std::*.
On Windows you can create an application that only links to LoadLibrary and GetProcAddress and at runtime it can get the rest of the functions you need (The names of the functions can be stored in encrypted form and you decrypt the string before passing it to GetProcAddress) Doing this is a lot of work and the Windows loader is probably faster at this than your code is going to be so it seems pointless to me to obfuscate the fact that you are calling simple functions like GetLastError and CreateWindow.
Windows API functions are loaded from dlls, like kernel32.dll. In order to get the loaded API function's memory address, a table of exported function names from the dll is searched. Thus the presence of these names.
You could manually load any Windows API functions you reference with LoadLibrary. The you could look up the functions' addresses with GetProcAddress and functions names stored in some obfuscated form. Alternately, you could use each function's "ordinal" -- a numeric value that identifies each function in a dll). This way, you could create a set of function pointers that you will use to call API functions.
But, to really make it clean, you would probably have to turn off linking of default libraries and replace components of the C Runtime library that are implicitly used by the compiler. Doing this is a hasslse, though.