I have a project which is very slow in linking (~ 2mins and I feel this is slow). I am aware of faster linker such as gold or lld, but I cannot change a linker.
I used a lot of C++11 templates in my code and I suspect that some template code may get instantiated repeatedly in multiple object files, but I have no clue how to find if this is true.
I wonder if there is a way to profile the whole linking stage like what we do to profile a program and try to find the bottleneck. For example, a tool that I can use to inspect how many times a symbol (unnecessarily) appear in different object files and then get discarded during linking can help me to find out which template code might be the cause. The above about repeated symbols in object files is only my speculation - I need an evidence based approach. Then based on this finding, I will think about how to improve my code to reduce linking time.
I use CMake, GNU g++ and ld as my build tools and I'm working in Linux platform.
Thanks.
One way to approach this problem is to dump defined symbols of each object file and archive included in the link with nm --demangle --defined-only --extern-only and build a mapping {symbol, definition_count}. Sort this mapping by definition_count high-to-low and print.
Related
I'm creating an application specifically for the Nintendo Wii using devKitPro. I wanted to make my application modular by offering the ability to load code passed though objcopy -O binary. My problem is, I want the modules to be able to use symbols from the main ELF that is loaded into memory. I have tried various things and I have not come up with any solutions.
I tried use -Map, as I thought that would let me use a linker map with it, but that idea failed.
I also tried compilation into a shared library, but that did not work for me, as the linker complained about "read-only segments with dynamic relocations".
I really need help with this, as I am in neither a Windows or Linux environment.
You may consider doing this using libdl which is made precisely to load symbols by name at runtime. Using libdl you can get handles to all of the functions you want to call as part of an initialization subroutine and then have them available to you when your program needs them. best of all, you dont need to have the library you are linking against at compile time so you can replace it with any elf that has those symbols defined without recompiling.
I have some huge libraries that are compiled with debug info; when linking them with some small object files I write, it still takes quite a lot of time and the generated executable contains a lot of debug information of the libraries.
So is there an option to tell gcc/clang to discard those debug information inside the library? Will it improve the link speed?
If there is no easy way, should I strip the libraries? I don't think I have the priviledge since the libraries are also used by my partners who need to use the library code for debugging purpose.
As already said in the comments, there's two ways out:
Keep a local copy of said libraries, stripped of debug info.
Link with -Wl,-s or -s, which makes the linker output a stripped executable.
I'm having some problems with a large static library (.lib) file, and am suspecting code bloat from indiscriminate use of template classes. I want to analyse the symbols in the library to confirm which are making up the bulk of the file size.
When I link my executable against this library, the resulting output is much more sensible, size-wise (about 20Mb), so the linker is obviously stripping out lots of redundant symbols. I want to find out what its removing..
I know I can use dumpbin to generate the symbols and headers, but, with the library in question being pretty large (900Mb), this dump is pretty much unusable without a utility for parsing and reporting on it.
Obviously I could write this myself, but was wondering if anyone can recommend any freeware already available for this?
Is this your own library? If so you can generate a link map that describes the layout of the code in the library, which would give you the info you need here in a more friendly form.
If you don't have source code access to do this, you could use Perl or other open-source scripting tools to crack the dumpbin output.
EDIT: you could also give LibDump a spin, it's downloadable from here. I have not used this myself.
I found one (SymbolSort) that works really well, gives me exactly what I need:
I have a project that takes about 8 seconds to link with g++ and ld.
It uses a bunch of static libraries, most of the code is c++.
I'm interested in a general list of tips on how to reduce link time.
Anything from "dont include debug symbols" to "make your code less spagetti"
I dealt with this for years at a previous job. The GNU linker simply has serious performance problems when linking large numbers of static libraries. At one point, link time was on par with compile time, which we found so strange we actually investigated this and figured it out.
You can try to merge your static libraries into a "super-object" before linking. Instead of linking like this:
$ g++ -o program program.o $STATIC_LIBS
You could try this:
$ ld -r -o libraries.o --whole-archive $STATIC_LIBS
$ g++ -o program program.o libraries.o
Note that this method gives the linker less opportunity to exclude unused object code, so your binaries may increase in size somewhat.
create a ramdisk ,compile to that and link to harddisk.
since you're using a lot of static libraries ,you can create a giant library containing all thos libraries so you end up with one libray. remove all libraries from your lib-list and add th giant one. This reduces openings of file to 1 for the libraries and may speed up reading actions.
Turn off whole program optimization (at least during development). Use p-impl to reduce dependencies.
8 seconds is pretty fast, unless you're really sure that it shouldn't take that long. I've got projects that take 5-8 minutes for a full relink since we don't do incremental linking on our release builds. Have you tried using incremental linking (if you're not using -shared, you can use -i or -r)?
How about compiling debug builds as shared libraries? That will solve the debug symbol bloat, as the import libraries are tiny with or without debug info. Maybe not the best solution, but I think this will cut down link time substantially...
You could try using clang and lld. From godot docs
You can also use Clang and LLD to build Godot. This has two upsides
compared to the default GCC + GNU ld setup:
LLD links Godot significantly faster compared to GNU ld or gold. This leads to faster iteration times.
I'm working in a Linux environment with C++, using the GCC compiler.
I'm currently working on modifying and upgrading a large pre-existing body of code. As part of this, it has been necessary to add quite a large number of small references throughout the code in a variety of places to link things together, and also to add in several new external code libraries. There is also quite a large and complex structure of Makefiles linked to a configure.ac file to handle the build process.
Upon starting the build process everything compiles without a problem, but comes back with the dreaded linker error when trying to use a newly added custom code library we've created. We have now been through a vast amount of code with a fine tooth comb looking for spelling mismatches, checking the order that all the libraries are included in the build process, and checked that the .o files created contain what we need using dumps, and all are as and where they should be. We've also tested the library separately and the problem definitely doesn't lie there.
In short, we've tried most things that you should normally do in these scenarios.
Is there a tool for C++ that can detect linker errors automatically, in a similar vein to cppcheck or splint (both of which we have run to no avail) that could help here?
Don't know your platform, but I spent sometime with linker problems in gcc till I realized that the static library (.a) linking requires specific ordering, its not the same to link gcc object.o first.a second.a than gcc object.o second.a first.a.
FWIW (not much) I try to tackle this sort of issue by using another linker as I have access to a couple of different platforms. If you can use another linker you will find either:
a) the program links, which transforms your problem from 'why doesn't it link ?' to 'what are the differences between linkers and linking ?' which isn't exactly a step forward, but sometimes a step to one side gives you a different perspective from which you can see a solution;
OR
b) it fails to link, in which case the other linker might give more useful information about why it fails.