I've long wondered when writing my C++ CMake files which is faster for the compiler:
Putting all my project's cpp files in a single shared .so library using add_library()
Using multiple libraries, one for each class / logical component.
Internet searching has not turned up any relevant articles about this curiosity so I thought I'd run a very simple experiment. I made two version of my project - one with a single library and one with two libraries. I found that, over runs, the single library version was 25% faster when compiling from scratch, and 1% faster when compiling after modifying a single line in one of the files. Granted, my experiments were not very thorough.
This is surprising to me - I've always thought having multiple libraries would speed up compiling of small changes in just one of the libraries, because the compiler wouldn't have to touch other libraries that did not depend on it.
Does anyone know a general performance rule about CMake libraries?
Can anyone explain why I see this difference?
Thanks!
Update:
I'm compiling on Ubuntu 14.04 with gcc 4.8.4
Related
I am learning more about shared libraries and also linking in general, and I seem to have confused myself.
I understand (correct me if i am wrong) that a compiled library or object file in c++ may not be compatible with another library even on the same target architecture, if the compilers are different and that this is due to name mangling and the Application Binary interface being different in OS/Compiler/Architecture etc.
This I believe is also true for C libraries although this may have more chance to work (I am not totally sure that's right)
And I believe the above applies to both static and shared/dynamic libraries
But then I get very confused, because say i make a new Visual Studio VC++ project, and it will link with say kernel32.dll or Gdi32.lib, well I have no idea what these were compiled with but my guess is it wasn't VS2017's Visual C++ Compiler so how can this be linked yet I can't link to say something compiled with MinGW or maybe even something with an earlier VS release? Surely the Windows dlls have been compiled differently?
I guess this also applies to linux libraries I am assuming the shared object files could have been compiled with a different version of gcc/g++, and I imaging yum installing a devel package would give libs that might not work?
And finally if someone has a closed source library and sell that library if the compiler and version matters does that mean that in order to sell a DLL or SO you need to compile it for more or less every compiler and version?
I have read a lot of questions and answers on here and tried to do a lot of searching but I am stuck and could really use some help in understanding this.
I am working on a project(for Windows) that is small and should be portable. I saw somewhere that MinGW gets rid of the requirement of some .dlls on the target system, so I thought I'd give it a try. To my suprise, when I tried running my program on a friend's computer, I got two errors saying that 2 different .dlls were missing. I thought MinGW used the system dlls, but it obviously didn't work. I saw an article on how to make a standalone exe with Visual Studio, but I'd prefer to use MinGW due to it's simplicity.
So my question: How to produce a standalone exe with MinGW?
Note: I am only using standard libraries, but it'd be nice to know how to include other libraries for future projects.
If you'd like an example of what I mean by a standalone exe, putty is a good example(it is an ssh client for Windows).
The two libraries you needed were most likely libgcc_s_sjlj-1.dll and libstdc++-6.dll or some variation of these, depending on how your mingw was configured when it was built. These are C and C++ standard library implementations.
If you want, you can pass extra flags to the compiler when you are linking the final executable, -static-libgcc and -static-libstdc++, which will cause it to link these libraries in statically instead of requiring you to distribute DLLs for them with your executable.
More info here: Mingw libgcc_s_sjlj-1.dll is missing
In general you can always try to statically link libraries into your executable. These special -static-libgcc flags are special flags provided only by mingw, not by other compilers, and not for other libraries, you use a different -static syntax for other libs. Static linking is fine but it gets more complicated and error prone as the dependencies get more complex. Static linking has some pitfalls associated to the order in which the libraries are linked. After a certain point, its usually simpler to make them all shared and don't do any static linking. It's up to you, it depends on how complex your project gets / if you start to have problems.
I wrote a program, that uses a shared library installed on my system. This library is seldom installed on other systems. How do I compile my program so that the library doesn't need to be installed on other systems? I have the source code for the library available. What's the best way?
The other systems of course have the same architecture and OS.
Compile it as a static library and link that into the executable.
Though the OP had solved his problem by answering a different question, there are (at least) two ways to wedge a shared library into your binary in case
there is no source code available
there is no compiler (or build-chain) available
static link does not work or it's not obvious how do it
to preserve memory layout - static link will change it and may "wake-up" hidden bugs
for "permanent link" LD_PRELOAD library into executable
The first is statifier (open source but limited to x86 and x86_64 and only object code)
The second that I know of is magic ermine (by the same developer). It is closed source, but the developer is friendly to opensource projects and ermine has the advantage of supporting more platforms as well as the ability to include all necessary data files within its virtual file system.
http://statifier.sourceforge.net/ and http://www.magicermine.com/
Let's say I have created and compiled a simple program using the MinGW 64 (g++ compiler). Running this program on my computer and looking in Process Explorer for what DLL files the program is using I find (among many others):
libgcc_s_seh-1.dll
libstdc++6.dll
libwinpthread-1.dll
These are the only ones that reside under my MinGW installation folder. The rest of the DLL files used reside under C:\Windows.
Question 1:
Are the MinGW DLL files the MinGW C++ runtime libraries (so to speak)? Do they serve the same purpose as for example msvcrXXX.dll (XXX = version of Microsoft runtime library).
Question 2:
If I want to run the application on a different computer which does not have MinGW installed, is it sufficient to include those DLL files listed above (i.e. placing them in the same folder as my executable) to have it run on the other computer (we assume the other computer is also a 64-bit Windows machine). If yes, does this mean we basically ship the MinGW C++ runtime with our executable. If no, why?
libstdc++6.dll is the C++ standard library, like you said.
libwinpthread-1.dll is for C++11 threading support. MinGW-W64 has two possible thread variants: Either use the native Windows functions like CreateThread, but C++11 stuff like std::thread won´t be available then; or include this library and use the C++11 classes (too).
Note that to switch the thread model, you´ll need to reinstall MinGW. Just removing the DLL and not using the C++11 stuff won´t work, the DLL will be required nonetheless with your current install.
libgcc_s_seh-1.dll is something about C++ exception handling.
Yes, it should be sufficient to deliver the DLLs too
(or use static linking and deliver only your program file).
For complicated projects where you're not exactly sure which DLL files need to be included to distribute your application, I made a handy dandy Bash script (for MSYS2 shells) that can tell you exactly what DLL files you need to include. It relies on the Dependency Walker binary.
#!/usr/bin/sh
depends_bin="depends.exe"
target="./build/main.exe" # Or wherever your binary is
temp_file=$(mktemp)
output="dll_list.txt"
MSYS2_ARG_CONV_EXCL="*" `cygpath -w $depends_bin` /c /oc:`cygpath -w $temp_file` `cygpath -w $target`
cat $temp_file | cut -d , -f 2 | grep mingw32 > $output
rm $temp_file
Note that this script would need to be modified slightly for use in regular MSYS (the MSYS2_ARG_CONV_EXCL and cygpath directives in particular). This script also assumes your MinGW DLL files are located in a path which contains MinGW.
You could potentially even use this script to automatically copy the DLL files in question into your build directory as part of an automatic deploy system.
You may like to add the options -static-libgcc and -static-libstdc++ to link the C and C++ standard libraries statically and thus remove the need to carry around any separate copies of those.
I used ntldd to get a list of dependencies.
https://github.com/LRN/ntldd
I'm using msys2 so i just installed it with pacman. Use that and then copy all the needed dependencies
There are several major challenges to distributing compiled software:
Compiling the code for all target processors (remember, when it comes to compiled code, you need to produce separate downloads/distributions for each type of instruction set architecture).
Ensuring that the builds are reproducible, consistent, and can be easily correlated with a specific version of the code (and versions of the dependencies).
Ensuring that the build output is self-contained and includes all of its dependencies within it (so that it is not dependent on any other installations that happen to exist on just your system).
Making sure that your code is built and distributed regularly, with updates distributed automatically so that -- in the event of security issues -- you can push out new patched versions.
For convenience and to increase reach, it is nice for non-savvy users to have a prebuilt version that they can install. However, I would recommend sharing the source code as a first step.
Most of these requirements are fairly non-trivial to hit and often require automating not only build process, but also automating the instantiation / configuration of VMs in which the build should take place. However, there are open source projects that can help... for example, check out Gitian.
In terms of bullet point #3, the key thing here is to use static linking... while this does make the binary you distribute much larger (because its dependencies are now baked into the output), it also makes your binary isolated from the version of the libraries on the system (avoiding "dependency hell").
Point #4 is very tricky, but thankfully there are also opensource tools to help here, as well such as cloudup, which provides a way to add auto-updating capability to your application distribution.
I've been trying to get SFML 2.1 working on my linux mint 15 install on my laptop, and that's when I found I should compile it from the source. So after fumbling through the tutorial on compiling SFML using cmake, I'm finally able to get some code working. But now I'm curious...what did I just do?
When I think of compiling, I think of compiling c++ code into object files, and then linking the object files into an executable. But I don't really understand what it means to compile something like SFML. When I think of a library, I think of a bunch of functions and objects that are made available to me through header files and source files created by another programmer, not necessarily something that needs to be compiled. My knowledge on the compilation and linking process is rather limited, so that might be my biggest issue at this point.
So what does it mean to "compile" a library?
EDIT: After looking at this particular question: How does the compilation/linking process work?
I noticed two bits of info that may further refine my question.
Linking: the linker takes the object files produced by the compiler
and produces either a library or an executable file.
and
The linker is what produces the final compilation output from the
object files the compiler produced. This output can be either a shared
(or dynamic) library (and while the name is similar, they haven't got
much in common with static libraries mentioned earlier) or an
executable.
So perhaps my real question is what does it mean to compile a dynamic library? What am I accomplishing by doing this?