I've noticed that there seem to be two approaches to linking: a flat and a hierarchical way.
Let me illustrate with two examples:
A Visual Studio solution can contain multiple projects with one of them being the main project. My usual approach is then to add all required libs to the main project and none to the other projects. I like this because that way I don't get "multiple definition"-linker errors. I don't know if this approach has an official name so I'll call it the flat way.
On the job our team works on a Linux-based traffic generation application. The build system uses Automake. When looking through the makefiles I noticed that each library specifies the libraries it requires (in the noinst_LIBRARIES variable). Each of these libraries can in turn specify their dependencies. This leads to a tree-like structure. So I call it the "hierarchical approach".
What are the best practices for this? It doesn't seem often discussed.
It usually comes down to static vs shared libraries.
If you use static libraries, then the main program must usually (always?) be linked against every library that it depends on, either directly or indirectly, hence a flat linking structure.
For shared libraries (or DLLs in Windows), each library bundles up its own dependencies, so the main program only needs to be linked against the libraries it depends on directly, leading to a graph structure.
Related
What if I am creating a library which is meant to be super easy to use, but its dependencies are a real pain to install? If my library depends on a bunch of libraries, is it possible to create one library which contains all of the dependencies, so my library user doesnt need to fiddle around with installing each dependency separately? I understand that people do this with full applications, distributing binaries, but it'd be nice to realize that developers are people too and sometimes just want to get something done without needing to understand everything about every library.
(I'm thinking specifically about C and C++ here.)
You have a few options:
Remove the dependencies. A lot of projects have dependencies that they do not really need.
Ship the dynamically linked libraries you depend on, with your own built executables or libraries.
Use a package manager to provide your dependencies so users don't have to build them from sources.
There is unfortunately no easy way to turn dynamically linked libraries into static ones. If you really want to try it (knowing it probably won't work out), see here: How can I convert dynamically linked application to statically one?
I use Visual Studio 2017 (but applies to any version from 2010+) and I've been trying to come up with a way to organize my Debug/Release libraries in such a way as to avoid all these linking errors we get, when mixing different versions of the Runtime libraries. My goal seems simple, conceptually, but I have not yet figured out a way to achieve all I want.
Here's what I have, and what I'd like to do:
Common Libraries:
ComLib1
ComLib2
...
Exe1:
ComLib1
ComLib2
...
Exe1Lib1
Exe1Lib2
...
Exe1
Exe2:
ComLib1
ComLib2
...
Exe2Lib1
Exe2Lib2
...
Exe2
So 2 different executable, using a set of common libraries and Exe-specific libraries.
I want to create 4 different build configurations.
Cfg1:
This would contain debugging info/non-optimized code for all libraries, including the Common Libraries.
Cfg2:
This would contain debugging info/non-optimized code for all Exe-specific libraries, but NOT for the Common Libraries.
Cfg3:
This would contain a combination of debugging info/non-optimized code libraries for some libraries, and non-debugging info/optimized libraries for the remaining ones.
Cfg4:
You guessed it. This would contain non-debugging info and optimized code for all.
My first attempt was to basically create 2 sets of binaries for each library; one compiled in Debug Mode (with /MTd /Od) and another one compiled in Release Mode with (/MT /O2). Then pick one or the other version in my various configurations. This was fine for Cfg1 & Cfg4 (since all Runtime libraries are consistent throughout), but ran into those those linking errors for Cfg2 & Cfg3.
I understand why I get these errors. I'm just not sure how one goes about resolving these things, in what I would think would be a common scenario. Maybe Cfg3 is uncommon, but I would think Cfg1,2 & 4 are.
Thanks for your inputs.
EDIT
I didn't really think I needed to add this information because I wanted to keep my question short(er). But if it can help clarify my goal, I'll add this up.
This is for a Realtime simulator. I just can't run every single library in a typical Debug configuration, as I would not be able to maintain Realtime. I seldom need to Debug the Common Libraries because they're mostly related to Server/IO tasks. The Exe libs mostly contain math/thermodynamics and is where I mostly spend my time. However, 1 Exe lib contains reactor neutronics, which involved heavy calculations. We typically treat that one as a black-box (cryptic vendor-provided code) and I almost always want to run it using Optimized code (typical Release settings).
You can not use different runtime libraries in the same process without some special considerations (e.g. using a DLL or so with no CRT object in the interface to make them entirely seperate) without either link errors or risking runtime issues if CRT objects are passed between.
You can mix most of the general optimisation options within a module with the notable exception with link time code generation that must be the same for all objects. The release runtime libraries are also generally usable for debugging as long as your own code is not optimised.
To easily switch you will want a solution configuration for each case you want (so 4). You can make one project configuration be used by multiple solution configurations if you do not want some that are duplicates but it must follow the previously mentioned limitations, and can confuse things like output directory. You can also use property sheets to share settings between multiple projects and configurations.
I've done similar using predefined macros for either the output directory path or the target filename.
For example, I use $(Platform)_$(Configuration) which expands to Win32_Debug or Win32_Release.
You can use environment variables as well. I haven't tried using preprocessor macros yet.
Search the internet for "MSDN Visual Studio predefined macros $(Platform)".
So this is how I ended up getting what I wanted.
Assuming I'm using the static Runtime libraries, I think I'll keep the typical Debug/Release (/MTd and /MT, respectively) libraries for my Common Libraries and create 3 sets of libraries for my Exe's:
Exe1Lib1Release: Typical Release Configuration
Exe1Lib1Debug: Typical Debug Configuration
Exe1Lib1DebugMT: Non-optimized code with debugging info, but using the MT Runtime libraries
Cfg1:
Will use the typical Debug libraries all around
Cfg2 & Cfg3:
Will use the typical Release libraries for the Common Libraries, and the Exe1Lib1DebugMT for the Exe's libraries
Cfg4:
Will use the typical Release libraries all around.
EDIT
Actually, Cfg2 & Cfg3 settings are more accurately represented by:
Cfg2:
Will use the typical Release libraries for the Common Libraries, and the Exe1Lib1DebugMT for the Exe's libraries
Cfg3:
Will use the typical Release libraries for the Common Libraries, and a combination of Release and Exe1Lib1DebugMT for the Exe's libraries
I've got a huge application here called HugeApp, it needs different libraries (which I've coded) and some of these libraries might need dependencies (other libs coming from the internet or ad hoc lib developped here).
I was wondering, if it was feasible and/or a good idea to hide some of these dependencies from HugeApp.
Let's say that you make a library in charge of doing the encrypted communication on a system, does the top application care and/or needs to know that there is some encryption libraries that are needed for this part (comms) of the system? It might be implementation specific... or not...
Thank you
There's no need for it to know, if you build those libraries as external DLL's then the external libraries are the only thing that care about the dependency. If you add a reference to the pre-built DLL then HugeApp doesn't need to know about the dependencies of the library (as long as they are either present in the library or the appropriate DLL or lib file is present so that your dll can make use of it). If anything your library can be another project altogether and you can include a reference to that in which case the project of your HugeApp only cares about that main reference and the other project will handle everything else.
If you enable /OPT:REF in linker optimizations it you will list which (if any) libraries that have no functions or data used by the project during link time. You can then remove them from the dependency list and link line in project settings. This will reduce the chance of removing a static library that is a dependency of another static library if any are present/used in your VS solution.
I am working on a project with SDL. I feel like a lot of my code could/should be reusable for any other projects I might want to take on.
I was going to start a library project, but I don't know how to go about linking to the SDL libraries.
Usually, I go to Project Settings and edited Mingw32 Linker to include the libraries (ming32, SDLmain, and SDL). But the linker does not show up in the settings.
So my questions:
Is there no linker settings because building a library is only a compiling operation?
And in general, is it possible to build a library on top of the existing libraries? That is, write my library using SDL functions and stucts. Or would I have to get the source code and rebuild entirely with my code included?
Also, advice on shared vs. static in this instance?
(Also, any advice for where to go about learning more about compilers and linkers and such. I got through data structures in school but no farther)
As an introduction, you have to distinguish very well static library from dynamic ones, they are completely different beasts... said that, to your questions:
Is there no linker settings because building a library is only a compiling operation?
I guess you are creating a static library in this case. A static library is simply the collection in one single object file of all the individual object files (i.e., the .o files produced by the compiler) that make up your source tree. No more, no less.
With a static library you don't need to specify which are the dependencies, since it is understood that it is when compiling the final executable that your library will be linked with all the other libraries that it depends upon. Therefore it is only at that time (final executable build) that any missing symbol will be detected, and all other libraries must be available.
A shared library (also dynamic library), is an executable file that embeds all the static libraries that it depends upon. It can also have external dependencies with other shared library, which would not be embedded.
And in general, is it possible to build a library on top of the existing libraries? That is, write my library using SDL functions and stucts. Or would I have to get the source code and rebuild entirely with my code included?
It is perfectly possible, both for static and dynamic libraries.
Also, advice on shared vs. static in this instance?
In this instance it is not possible to advice, because you don't specify enough information.
Look at this: When to use dynamic vs. static libraries, and this, to have a sort of guideline.
(Also, any advice for where to go about learning more about compilers and linkers and such. I got through data structures in school but no farther)
I think that the two links above give you plenty of information. If you want to further go into details, you could start from this wikipedia article and browse from there.
The library you are building can have external dependencies. That means that you can link it with SDL or any other external libraries that you like.
I think this page explains all your other questions: DLL Creation in MingW
I'm using a couple of external libraries and I'd rather not have to include all their source and header files in my main source directory or in my project file. One option would be to compile the libraries as lib files and link them like that. However I'm not sure the defines get evaluated before or after the lib file gets created (which one is it?). If it's before then obviously I can't just pack them because they might not work properly on different compilers or systems.
So if I can't pack the libraries as lib files, is there any way for me to link in the c or cpp source files? Probably not, since they would have to be compiled first, but maybe I'm wrong.
Edit: Here's a follow-up question, based on answers. Do you think it'd be too much of a hassle to have a makefile that creates the lib files? I'd still rather not add the sources to my project or in my source directory.
Library is a binary file, so all defines obviously already in.
Just to make order, defines are evaluated as 1st stage of compilation process - the step is called preprocess. At this stage, for each cpp files created one file containing all #include'ed in it files recursively and all macros are evaluated.
Any way 3rd party should not depend on your compilation flags with one exception - release/build lib. Only in this case you need 2 versions of 3rd lib.
As regarding to question if to compile 3rd party libs once or each time while compiling your code it depends. If you are doing it only for itself than do what looks an easies way for you, but if we're talking about development team and the project to be maintain for a long time, than more things are to be considered.
SO we're talking about some solid solution for a team and we want to compile library several times.
In this case I personally strive to compile 3rd part library once and use it many times. This reduces compilation times for each build for each developers, which means faster development.
Nice, but where you hold these libs. I like phisycal separation - 3rd party library and my code not in same tree. This can avoid some not intentional errors. A good build system, and most of time it's mandatory, should be re-buildable. This means that if you checkout your code after year, you can compile and receive exactly same binaries.
Once I used some external read only tree on my machine. This tree was managed only by me.
To make my sources re-buildable, each next version of 3rd party library put in direcoty containing it's version and my source tree was updated to point to this point. If you build on several machines, than the read only tree should be visible on all these machines.
Additioanal solution is to check if your SCM tool (I suppose you use one) gives you some ability to combine several sub-tries from repository in one checkhout. For each 3rd party library there's one sub-tree. This way 3rd party libraries are available on all machines your build. I currenly use these method on subversion - it's called svn:external. On CVS AFAIK it's called cvs modules. Additional advantage that the libraries are managed by source control system, so you can track all changes done to 3rd party libs.
defines get evaluated even before compiling. They are dealt with by the pre-processor, that prepares the code for the compiler to use. So yes, they are evaluated before the libraries are created.
You can't link against source code. You can only link with object files, static libraries, or dynamic libraries (shared object files/DLLs).
Using dynamic linking can be a good option, especially if the externals are large and/or you'll be using them in many executables.