This question may be partly duplicate, e.g. this question, but is more about what if any are better solutions. Since this question ended up rather long, I marked specific questions with "+Q+" in bold italic.
I have the situation that I wrote a small library B that depends on some other huge project A split into many libraries A1, A2, ..., An, some of which my library depends on and some on which it doesn't. It was a bit of a pain to do the proper linking. The library is starting to be used by others and I would like to avoid everyone having to go through this awful linking process, i.e. I want to compile all the external libraries of A into my B. Assume A is completely external, i.e. that I have no way to recompile A (in this case I do, but it is complicated and I would like to know options for the case that I don't).
I imagine this must be a very standard thing to do, I have used other popular libraries and never did I have to link all the other libraries that they transitively depend on .. ? So I started looking for solutions, and while I found working ones, most solutions seem like a big mess and I wonder if this is really done in practice or if there is some idiomatic way for this.
To avoid more eventual headaches if I ever need a different case, I want to consider all combinations of static/shared libraries, i.e.
A & B are static
A is static, B is shared
A is shared, B is static
A & B are shared
To give some MWE of the code setup (the variables LIB1_ROOT and LIB2_ROOT in the CMakeLists.txt files are A/ and B/ respectively):
A/include/lib1.hh
struct Lib1 { void run() const; };
A/src/lib1.cc
#include <iostream>
#include <lib1.hh>
void Lib1::run() const { std::cout << "Hello from lib1\n"; }
A/CMakeLists.txt
cmake_minimum_required(VERSION 3.14)
project(A)
include_directories(include)
add_library(lib1 src/lib1.cc)
install(TARGETS lib1 DESTINATION "${CMAKE_CURRENT_SOURCE_DIR}/lib")
B/include/lib2.hh
class Lib2 {
class Implementation;
Implementation* impl;
public:
Lib2();
~Lib2();
void run() const;
};
B/src/lib2.cc
#include <iostream>
#include <lib1.hh>
#include <lib2.hh>
class Lib2::Implementation {
const Lib1 m_lib1{};
public:
void run() const { std::cout << "using lib1 from lib2: "; m_lib1.run(); }
};
Lib2::Lib2() : impl{new Implementation} {}
Lib2::~Lib2() { delete impl; };
void Lib2::run() const { impl->run(); }
App/src/app.cc
#include <lib2.hh>
int main() { Lib2 l; l.run(); }
App/CMakeLists.cc
cmake_minimum_required(VERSION 3.14)
project(App)
include_directories(include "${LIB2_ROOT}/include")
find_library(LIB2 lib2 "${LIB2_ROOT}/lib")
add_executable(app src/main.cc)
target_link_libraries(app "${LIB2}")
install(TARGETS app DESTINATION "${CMAKE_CURRENT_SOURCE_DIR}/bin")
I used the pImpl pattern for B since what is the point of hiding link dependencies when I then make the user of my library dig out all the headers anyway.
Finally B/CMakeLists.txt (for my library) depends on the cases I mentioned above:
A & B static
B/CMakeLists.txt
cmake_minimum_required(VERSION 3.14)
project(B)
include_directories(include "${LIB1_ROOT}/include")
find_library(LIB1 lib1 "${LIB1_ROOT}/lib")
add_library(lib2_dependent src/lib2.cc)
add_custom_target(lib2 ALL
COMMAND ar -x "${LIB1}"
COMMAND ar -x "$<TARGET_FILE:lib2_dependent>"
COMMAND ar -qcs "${CMAKE_STATIC_LIBRARY_PREFIX}lib2${CMAKE_STATIC_LIBRARY_SUFFIX}" *.o
COMMAND rm *.o
DEPENDS lib2_dependent
WORKING_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/lib"
)
This solution is from the question I linked at the start. There was also an in my opinion much better version in this answer using
ar -M <<EOM
CREATE lib2.a
ADDLIB lib1.a
ADDLIB lib2_dependent.a
SAVE
END
EOM
but I could not get to work the here-document inside CMakeLists.txt ... ? There was an additional answer providing what I think was a CMake function to do this, but it was a huge block of code which I found a bit ridiculous for something that should be simple / standard practice / integrated into CMake ?
The custom_target solution I wrote here also works, but as mentioned in other answers as well, it unpacks object files which lie around and have to be removed again, for each library I want to compile this way.
And still in both cases, I can only wonder what is the point of using CMake then if I have to use ar manually anyhow. +Q+ Is there no better / CMake-integrated way to "compile-in" transitive dependencies / combine static libraries?
A static, B shared
As far as I found in this case, I'm out of luck if I cannot recomple A and it is not compiled as position independent code. The next best thing to make it work was to do just that by adding set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fPIC") to A/CMakeLists.txt. Again, there seemed to be better alternatives: set(CMAKE_POSITION_INDEPENDENT_CODE ON) or set_property(TARGET lib1 PROPERTY POSITION_INDEPENDENT_CODE ON) from here, which were complete ignored in my case .. (compiling with VERBOSE=1, there was no -fPIC flag to be seen anywhere and B did not compile)
The B/CMakeLists.txt was easy in this case
cmake_minimum_required(VERSION 3.14)
project(B)
include_directories(include "${LIB1_ROOT}/include")
find_library(LIB1 lib1 "${LIB1_ROOT}/lib")
add_library(lib2 SHARED src/lib2.cc)
target_link_libraries(lib2 "${LIB1}")
install(TARGETS lib2 DESTINATION "${CMAKE_CURRENT_SOURCE_DIR}/lib")
and while I don't find anything wrong with this solution, according to answers I found I should need to set additional flags like set(CMAKE_SHARED_LINKER_FLAGS "-Wl,--export-all-symbols") in order for the symbols in the static library to be found. However the above works just fine, compiles, and App runs without issues? +Q+ Am I doing anything wrong here? Or is it maybe due to some update to CMake since these older answers?
A shared, B static or both shared
According to what I found here, this is basically impossible, because shared libraries are "final" in some sense. I find this very strange, surely there are many libraries that do not require a project that uses them to link every single dependency of that library that happens to be a shared library? +Q+ Are there really no options in these cases?
Yep, you're doing wrong :)
The CMake-way is to use packages. Once you've made the LibA package, you just do find_package(LibA) in your B/CMakeLists.txt and also the generated LibBConfig.cmake (the package config file, so your clients would need just find_package(libB) in their App/CMakeLists.txt) ought to find (that is questionable IMHO, but currently its CMake way) its dependency the same way (using or not the CMake's helper find_dependency).
The whole process gets much simpler that way:
you have/give the full control on how to build (including what version, library type static/dynamic, &etc) and where to install the libA on your developer's host and customer's machine (so dependent projects could find it)
same for libB and App
you and any customer of your library(ies) use the well known CMake way to find dependencies and have full control over this process
and the most important let CMake do complicated things instead of you that keeps your build logic much simpler in CMakeLists.txt of all involved projects
Related
I have two C++ modules: A and B, where B links to a set of static libs lib*.a (I use * here to mean a set of lib files) and A links to B.
I have CMakeLists.txt for B:
add_library(B STATIC B.cpp)
target_include_directories(B PUBLIC ${CMAKE_CURRENT_SOURCE_DIR} "/path/to/headers/directory/for/lib*.a")
link_directories("Path/to/directory/contains/lib*.a")
target_link_libraries(B PRIVATE lib*)
and CMakeLists.txt for A:
add_library(A STATIC A.cpp)
target_include_directories(A PUBLIC ${CMAKE_CURRENT_SOURCE_DIR} "/path/to/headers/for/libB.a") # compiler outputs libB.a when target name is B
link_directories("Path/to/directory/contains/libB.a")
target_link_libraries(A PRIVATE B)
Everything works fine, until I try to make A a shared lib, then I got error from the linker saying cannot find -l*. I believe the reason is that when I set A to be shared, the compiler looks for shared lib, which is not available.
The thing I don't understand:
B is already a static lib, why the linker needs lib*?
why compiler looks for shared lib when I set A to be shared? I thought even a shared A can be linked to static libs
I do want to note that in A.cpp, I have #include B.hh at the top, and B.hh further includes the headers for lib*.a, that is why I have target_include_directories(B PUBLIC...)
Your initial setup was already not working, but you did not see the error until switching to a shared library.
The reason for this is that all your target_link_libraries dependencies on A and B were not actually processed at all so long as both were static libraries. This has to do with what a static library is: Just a bunch of object files packed together into a single file. A static library is not linked, hence any build dependencies that concern the linker (such as linking in additional static libraries as dependencies) will not get resolved. The build system will keep track of them, but if they are missing, you will not notice unless you actually pass those libraries to something that is actually being linked - such as an executable or, in your case now, a shared library.
Since it can be a bit embarassing to have invisible unresolved dependencies like this, idiomatic CMake will have you model things differently. You never pull in static dependencies directly, instead you use an approach like the following: At the lowest layer, you have a find_library call that finds any external pre-built dependencies on the system (the lib* in your example). You then wrap that up into an imported target so that you never have to touch the raw library path again. To smoothen things out, you can hide all of this inside a find package script, so that your code only ever sees a find_package call that provides a lib* target and don't worry about the internals.
You then model all of your library interdepencies through target dependencies only. Such an approach has several advantages that make it more robust: If CMake can't find the lib*, it will tell you right away when the find_package call fail, instead of waiting until you're trying to link like it does now. By only specifying target-based dependencies, you ensure that the build is actually consuming the libraries that it was configured with (which is very much not ensured with the link_directories based approach you're having now).
I'm currently transforming my C++ project from simply using a Makefile to using CMake. As I'm unfamiliar with CMake, I try and avoid learning "old" CMake and stick to "modern CMake" best practices, e.g. described here.
However, I could not figure out how to include a simple, header-only, third party library without getting bothered with warnings about that library.
Say I have a project structure like this:
CMakeLists.txt
src/
CMakeLists.txt
main.cpp
third-party-lib/
foo/
foo.h
Say main.cpp uses foo: #include <foo.h>
The problem is, I use a bunch of warning flags and foo.h introduces some warnings that I don't care about, as I'm not the author and it's nothing serious.
In a classic Makefile I would just have written -isystem third-party-lib/foo somewhere appropriate and be done.
In CMake, I achieved this by a include_directories(SYSTEM PRIVATE third-party-lib/foo) in the top-level CMakeLists.txt.
Now, according to the link above, I should keep my hands off include_directories() to avoid hidden dependencies and I think that this is a good idea. I'd rather specify for each target that actually needs foo to include that library and treat it as 'don't warn me here'. As I understand it, the approach would be:
find foo target in top-level CMakeLists.txt
target_link_libraries(main PRIVATE foo) or so in src/CMakeLists.txt
How would I achieve that, i.e. what is the command for finding, and where to specify the SYSTEM property? Or any other suggestions for how to approach this?
To add a header-only library in modern CMake, you can use target_include_directories with SYSTEM INTERFACE. You can place this in your top-level CMake file, before processing the src/CMakeLists.txt file:
add_library(foo INTERFACE)
target_include_directories(foo SYSTEM INTERFACE
${CMAKE_SOURCE_DIR}/third-party-lib/foo
)
Your friend is target_include_directories. It works like include_directories but on a per-target basis.
add_executable(foo foo.c)
target_include_directories(foo SYSTEM
PRIVATE sys_inc
)
PROBLEM: I would like to ignore 'dll not found' errors and continue code execution that does not rely on that library.
I understand this is asking alot, but there's no definitive guide on shared libraries and the mess of conversations out there are confusing. I have gathered that there are three types of libraries that CMake is aware of:
Static: Linked into executable on compile.
Shared: UNIX: loadable whenever, WINDOWS: requires a symbols STATIC library to be linked on compile that maps out the DLL.
Module: Runtime loaded shared library, of which I'm still in the process of understanding.
I'm using the CMake 'GenerateExportHeader' to make my life easier in producing shared libraries. This works. I can even version those libraries to change them on the fly. What I'm having trouble figuring out, however, is how to check and possibly ignore the error if the dll is not found:
"The code execution cannot proceed because xxx.dll was not found. Reinstalling the program may fix this problem."
I have tried putting try/catch blocks in the include/object creation. Google foo didn't find anything.
I have looked into "MODULE" type shared library, but I was hoping to avoid the mess that is '__declspec' and loadable libraries.
CMake's 'GenerateExportHeader' is very very nice, and if what I'm asking for isn't possible without manually exporting symbols that's fine, but I thought I might ask the gurus out there.
So.... The code:
CMakeLists.txt:
...
set(BUILD_SHARED_LIBS ON)
set(BUILD_TYPE SHARED)
set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON)
include(GenerateExportHeader)
...
# Create the library.
add_library(${NAME} ${BUILD_TYPE}
${SOURCES}
)
# Visual Studio Source Tree and filters.
source_group(TREE ${CMAKE_CURRENT_SOURCE_DIR} FILES ${SOURCES})
set_property(TARGET ${NAME} PROPERTY FOLDER "${NAME}")
if(BUILD_SHARED_LIBS)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
GENERATE_EXPORT_HEADER(${NAME}
BASE_NAME ${NAME}
EXPORT_MACRO_NAME ${NAME}_EXPORTS
EXPORT_FILE_NAME ${NAME}_EXPORTS.h
STATIC_DEFINE SHARED_EXPORTS_BUILT_AS_STATIC)
endif(BUILD_SHARED_LIBS)
target_include_directories(${NAME} PUBLIC
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/public>
$<INSTALL_INTERFACE:public>
PRIVATE private
)
# Link to other libraries.
target_link_libraries(${NAME}
${EXAMPLE_LIBRARY}
)
# 'make install' to correct locations.
install(TARGETS ${NAME} EXPORT ${NAME}Config
ARCHIVE DESTINATION lib
LIBRARY DESTINATION lib
RUNTIME DESTINATION bin) # For windows.
install(DIRECTORY public/ DESTINATION include)
# 'public'/ project, 'include' for install.
# Importable from the install directory.
# Configs in per-project dir.
install (EXPORT ${NAME}Config DESTINATION share/${NAME}/cmake)
# Importable from the build directory.
export(TARGETS ${NAME} FILE ${NAME}Config.cmake)
# Library Testing/Future unit tests.
set(TESTING_PRIVATE
Tests/Main.cpp
)
set(TESTING_PUBLIC
)
set(TESTING_COLLECTIVE
${TESTING_PRIVATE}
${TESTING_PUBLIC}
)
add_executable(${NAME}Develop
${TESTING_COLLECTIVE}
)
set_property(TARGET ${NAME}Develop PROPERTY FOLDER "${NAME}")
target_include_directories(${NAME}Develop
PUBLIC ${CMAKE_BINARY_DIR}/exports
)
if(NOT BUILD_MODULAR_LIBS)
target_link_libraries(${NAME}Develop
${NAME}
)
endif(NOT BUILD_MODULAR_LIBS)
My code does not have anything specific in it, so for the purposes of this question I simplified a test case.
library.h:
#pragma once
namespace Library
{
class Library
{
public:
void testing();
};
} // namespace Library
Library.cpp:
#include <iostream>
#include "Library.h"
namespace Library
{
void Library::testing()
{
std::cout << "Library API: Hello, world!.... VERSIONED!" << std::endl;
}
} // namespace Library
Test/Main.cpp:
#include <iostream>
#include <conio.h>
// Tried different error handling, I think I need preprocessors here for this... google didn't help.
// I feel that I should be referencing the DLL, not the header
// and I couldn't find information in wikis or CMake documentation.
#include "Library.h"
int main(int argc, char const *argv[])
{
// The same applies for these two lines.
Library::Library test;
test.testing();
std::cout << "Press any key to exit..." << std::endl;
_getch();
return 0;
}
WHAT I HAVE TRIED:
Putting try/catch blocks around code that references the library. EDIT: Yes I know this is bad and probably would never work, but I didn't know if something internal might be calling a library load.
Compiling a version of my program with and without the option libraries. (Honestly... this is bad... very bad... preprocessor directives can solve this, but this means each person were they to want a tailored version without certain libraries would have to build themselves rather than just dropping the dlls they'd like in there, which is what I would like to do.)
Manual '__declspec' declarations to do this manually, which works but what I would like (if possible) to avoid. There's enough information out there that I was able to piece this together, and am willing to do, but the idea really is to write the code WITH classes just like the executable or a library in what (in c++ terms) would be called plain english.
That last point is really what I'm trying to accomplish. I'd like to be able to write c++ code, and make it readable to other developers, without having to manage some of the more complicated features including platform specific shared library loading.
I would like to build an executable from static libraries (i. e. .a-files) only. This is possible, because the main() function is contained in one of these libraries.
The add_executable() function requires me to provide at least one source file. But this is not what I want to do.
There is no way to do it without a hack. You need at least one *.c or *.cpp file.
What I do is make a dummy null.cpp file (zero bytes) and use that. You can also use /dev/null but that only works on Linux.
file(WRITE null.cpp "")
add_executable(tester
null.cpp
)
target_link_libraries(tester
-Wl,--whole-archive
libtest1
libtest2
libtest3
libtest4
-Wl,--no-whole-archive
gtest_main
)
There are mainly two reasons why a source file is enforced by CMake:
To determine the LINKER_LANGUAGE from the file ending(s)
Not all compilers do support an object/library only link step (for details see below)
And if you move the main() function to library please keep the following in mind: Why does the order in which libraries are linked sometimes cause errors in GCC?
So if you build the libraries with CMake in the same project, I would recommend to change your libraries (at least the one containing your main() function) to an object library:
cmake_minimum_required(VERSION 2.8.8)
project(NoSourceForExe)
file(WRITE main.cc "int main() { return 0; }")
add_library(MyLibrary OBJECT main.cc)
add_executable(MyExecutable $<TARGET_OBJECTS:MyLibrary>)
The add_library() documentation lists a warning here:
Some native build systems may not like targets that have only object files, so consider adding at least one real source file to any target that references $<TARGET_OBJECTS:objlib>.
But those are rare and listed in Tests/ObjectLibrary/CMakeLists.txt:
# VS 6 and 7 generators do not add objects as sources so we need a
# dummy object to convince the IDE to build the targets below.
...
# Xcode does not seem to support targets without sources.
Not knowing which host OS(s) you are targeting, you may just give it a try.
References
CMake Object Lib containing main
CMake/Tutorials/Object Library
I have a lot of libraries in my project, and a LOT of individual applications. A few of the my libraries have dependency libraries, some of them external ones, and I'd like a way for the application CMakeList.txt files to be simpler. I'm hoping to use macros to simplify.
Below is a much reduced test case. For example, in my project, including one of my libraries requires also include_directories, link_directories, and target_link_libraries for ImageMagick, pugixml, jsoncpp, liboauthcpp, etc... And, some of these third party libraries require compiler flags. My project's version of the _LIB() macro below will be much longer...
Question: Is there a way I can have the _LIB() macro below automatically add something to the target_link_libraries that invoke the macro?
I'm not sure how to do that because target_link_libraries argument 1 is the target name, which changes per application.
~/codeTest/CMakeLists.txt
cmake_minimum_required(VERSION 2.6)
project(codeTest)
macro(_LIB)
include_directories(~/codeTest/lib)
link_directories(~/codeTest/lib)
endmacro()
add_subdirectory(lib)
add_subdirectory(app)
~/codeTest/lib/CMakeLists.txt
include_directories(~/codeTest/lib)
add_library(lib lib.cpp)
~/codeTest/lib/lib.h
#ifndef __LIB__
#define __LIB__
namespace LIB {
unsigned long libFunc(unsigned long inValue);
}
#endif
~/codeTest/lib/lib.cpp
#include <lib.h>
namespace LIB {
unsigned long libFunc(unsigned long inValue) {
return inValue+1;
}
}
~/codeTest/app/CMakeLists.txt
_LIB()
add_executable(app app.cpp)
target_link_libraries(app lib)
~/codeTest/app/app.cpp
#include <lib.h>
using namespace LIB;
int main() {
unsigned long x = 1;
unsigned long y = libFunc(x);
}
With the growth of your project's complexity, you will soon find out that multiple {link,include}_directories directives, gathered in a macro, is inflexible solution, and they should be specified explicitly.
I recommend you to take into consideration an idea of modules (/usr/share/cmake/Modules/): for each of your external dependencies use a Find*.cmake file that can be found in cmake modules directory (see above) or in the package's one (if its authors have written an according module). Or you can write it yourself.
That modules usually define a number of variables, (e.g., Boost_INCLUDE_DIRS, Boost_LIBRARY_DIRS), that you will use only in those subprojects, that really need ones.
So, for each subproject you specify include_.../link_... directives explicitly referencing variables defined in modules or in your other subprojects (for internal dependencies).
Moreover, macros and functions in cmake are really very unpredictable for users, that expect them to behave like traditional functions in well-known programming languages (C, Java, Ruby, ...) when you start adding arguments to them; read about variable caching, scoping, and conversions between strings and lists.
Consider encoding the usage requirements into the targets themselves instead of using macros.
This would be 'modern cmake':
http://www.cmake.org/cmake/help/git-next/manual/cmake-buildsystem.7.html
http://www.kdab.com/modern-cmake-with-qt-and-boost/
http://www.steveire.com/WhatWhyHowCMake.pdf
http://www.cmake.org/cmake/help/v3.0/manual/cmake-packages.7.html
I might recommend creating an INTERFACE library, if you can group some of your dependencies into 1 target this way.
For example,
add_library(ImageMagick ...)
#...etc gather all library targets however is conviennt
add_library(CoreDependencies INTERFACE)
target_link_libraries(CoreDependencies PUBLIC ImageMagick ...)
Then in your app(s) you can just
target_link_libraries(MyApp PRIVATE CoreDependencies)
and this will bring in everything you need. You could create several subsets of libraries if you really have so many different use cases.
Of note, this approach requires CMake 3.
https://cmake.org/cmake/help/v3.0/manual/cmake-buildsystem.7.html#interface-libraries