I have a lot of libraries in my project, and a LOT of individual applications. A few of the my libraries have dependency libraries, some of them external ones, and I'd like a way for the application CMakeList.txt files to be simpler. I'm hoping to use macros to simplify.
Below is a much reduced test case. For example, in my project, including one of my libraries requires also include_directories, link_directories, and target_link_libraries for ImageMagick, pugixml, jsoncpp, liboauthcpp, etc... And, some of these third party libraries require compiler flags. My project's version of the _LIB() macro below will be much longer...
Question: Is there a way I can have the _LIB() macro below automatically add something to the target_link_libraries that invoke the macro?
I'm not sure how to do that because target_link_libraries argument 1 is the target name, which changes per application.
~/codeTest/CMakeLists.txt
cmake_minimum_required(VERSION 2.6)
project(codeTest)
macro(_LIB)
include_directories(~/codeTest/lib)
link_directories(~/codeTest/lib)
endmacro()
add_subdirectory(lib)
add_subdirectory(app)
~/codeTest/lib/CMakeLists.txt
include_directories(~/codeTest/lib)
add_library(lib lib.cpp)
~/codeTest/lib/lib.h
#ifndef __LIB__
#define __LIB__
namespace LIB {
unsigned long libFunc(unsigned long inValue);
}
#endif
~/codeTest/lib/lib.cpp
#include <lib.h>
namespace LIB {
unsigned long libFunc(unsigned long inValue) {
return inValue+1;
}
}
~/codeTest/app/CMakeLists.txt
_LIB()
add_executable(app app.cpp)
target_link_libraries(app lib)
~/codeTest/app/app.cpp
#include <lib.h>
using namespace LIB;
int main() {
unsigned long x = 1;
unsigned long y = libFunc(x);
}
With the growth of your project's complexity, you will soon find out that multiple {link,include}_directories directives, gathered in a macro, is inflexible solution, and they should be specified explicitly.
I recommend you to take into consideration an idea of modules (/usr/share/cmake/Modules/): for each of your external dependencies use a Find*.cmake file that can be found in cmake modules directory (see above) or in the package's one (if its authors have written an according module). Or you can write it yourself.
That modules usually define a number of variables, (e.g., Boost_INCLUDE_DIRS, Boost_LIBRARY_DIRS), that you will use only in those subprojects, that really need ones.
So, for each subproject you specify include_.../link_... directives explicitly referencing variables defined in modules or in your other subprojects (for internal dependencies).
Moreover, macros and functions in cmake are really very unpredictable for users, that expect them to behave like traditional functions in well-known programming languages (C, Java, Ruby, ...) when you start adding arguments to them; read about variable caching, scoping, and conversions between strings and lists.
Consider encoding the usage requirements into the targets themselves instead of using macros.
This would be 'modern cmake':
http://www.cmake.org/cmake/help/git-next/manual/cmake-buildsystem.7.html
http://www.kdab.com/modern-cmake-with-qt-and-boost/
http://www.steveire.com/WhatWhyHowCMake.pdf
http://www.cmake.org/cmake/help/v3.0/manual/cmake-packages.7.html
I might recommend creating an INTERFACE library, if you can group some of your dependencies into 1 target this way.
For example,
add_library(ImageMagick ...)
#...etc gather all library targets however is conviennt
add_library(CoreDependencies INTERFACE)
target_link_libraries(CoreDependencies PUBLIC ImageMagick ...)
Then in your app(s) you can just
target_link_libraries(MyApp PRIVATE CoreDependencies)
and this will bring in everything you need. You could create several subsets of libraries if you really have so many different use cases.
Of note, this approach requires CMake 3.
https://cmake.org/cmake/help/v3.0/manual/cmake-buildsystem.7.html#interface-libraries
Related
The target_include_directories documentation states the following:
Include directories usage requirements commonly differ between the build-tree and the install-tree. The BUILD_INTERFACE and INSTALL_INTERFACE generator expressions can be used to describe separate usage requirements based on the usage location.
I would like to make my CMake library installable, and I'm not sure if that means I should use these generator expressions. Currently, my src/CMakeLists.txt contains the following:
target_include_directories(mylib PRIVATE ${CMAKE_SOURCE_DIR}/include/mylib)
target_include_directories(mylib INTERFACE ${CMAKE_SOURCE_DIR}/include)
This allows me to #include my library's public headers without specifying the parent directory while still requiring that users of my library do so themselves. My understanding is that this is standard practice for preventing header filename collisions.
That being said, I don't believe this code will work in its current form for installing the library. What is typically the preferred way to go about fixing this?
I have a package, libtorch, that depends on libraries that use the keyword slots for some function (e.g. in Aten). In the meantime, the project that I am using is based on Qt, which use slots as a special keyword (macro), which then interferes with torchlib.
The solution to avoid such interference is to add the QT_NO_KEYWORDS definition when importing the external library that enters in conflict with Qt: Turning the no_keyword (Qt option) ON in a cmake file to work with boost::signals
It has then to be done only on the code that depends on libtorch and not the rest, so I tried several ways to add it, like CMake Add and Remove a Macro definition to compile Shared Library/Executable and Add target properties to an existing imported library in CMake, but I can't get it to work.
target_compile_definitions seems to be available after cmake 3.11 on "IMPORTED targets"(on cmake terminology).
So why the following cmakelist extract doesn't work? (in that case, the normal QT slots code are not properly recognized as a macro):
find_package (Torch REQUIRED PATHS ${Torch_DIR})
add_executable( ${PROJECT_NAME} ${header_files} ${source_files} ${hpp_files} ${moc_list} ${generated_ui_files} ${generated_qrc_files} )
target_link_libraries(${PROJECT_NAME} ${Qt_LINK_LIBRARIES})
#add_definitions(-DQT_NO_KEYWORDS)
target_link_libraries(${PROJECT_NAME} ${TORCH_LIBRARIES})
#remove_definitions(-DQT_NO_KEYWORDS)
target_compile_definitions(torch INTERFACE QT_NO_KEYWORDS)
Possible Solutions
Option 1:
Use QT_NO_KEYWORDS and replace slots and other keywords with their equivalent uppercase macros, e.g. Q_SLOTS.
Option 2:
Don't use QT_NO_KEYWORDS and wrap the parts of your C++ code with conflicting keywords like this:
#undef slots
#include <torch/torch.h>
#define slots Q_SLOTS
Explanations
The INTERFACE keyword in target_compile_definitions() tells CMake that all targets that depend on the torch library need the QT_NO_KEYWORDS macro defined as well. This is why all sources of your executable will be compiled with -DAT_NO_KEYWORDS and the slots keyword will not be defined.
I suppose you tried to fix your problem with the commented out add_definitions()/remove_definitions() calls. Several reason why those don't work as expected:
They only affect compilation not linking. Putting them around target_link_libraries() has no effect.
They manipulate the compile flags of all targets created in the current scope (current CMakeLists.txt), no matter if they were created before or after the command gets invoked, see the docs. This means calling add_definitions() and subsequently remove_definitions() with the same arguments will result in no flags being added at all no matter at which point in your CMakeLists.txt you call them.
You should also note that commands operating on the directory level (add_*/remove_*) are considered "old style" CMake. Modern CMake aims to only operate on the target level (target_* commands). Some more "Modern CMake" suggestions:
Use target_sources() instead of variables like header_files, source_files, etc. to add sources to your targets.
Always use target_link_libraries() with a PRIVATE, PUBLIC or INTERFACE keyword. Excerpt from Craig Scotts Professional CMake book:
The target_link_libraries() command also has a few other forms, some of which have been part of CMake from well before version 2.8.11. [...] Their use is generally discouraged for new projects. The full form shown previously with PRIVATE, PUBLIC and INTERFACE sections should be preferred, as it expresses the nature of dependencies with more accuracy. [...] The above form [(without PRIVATE/PUBLIC/INTERFACE)] is generally equivalent to the items being defined as PUBLIC, but in certain situations, they may instead be treated as PRIVATE. In particular, if a project defines a chain of library dependencies with a mix of both old and new forms of the command, the old-style form will generally be treated as PRIVATE.
This question may be partly duplicate, e.g. this question, but is more about what if any are better solutions. Since this question ended up rather long, I marked specific questions with "+Q+" in bold italic.
I have the situation that I wrote a small library B that depends on some other huge project A split into many libraries A1, A2, ..., An, some of which my library depends on and some on which it doesn't. It was a bit of a pain to do the proper linking. The library is starting to be used by others and I would like to avoid everyone having to go through this awful linking process, i.e. I want to compile all the external libraries of A into my B. Assume A is completely external, i.e. that I have no way to recompile A (in this case I do, but it is complicated and I would like to know options for the case that I don't).
I imagine this must be a very standard thing to do, I have used other popular libraries and never did I have to link all the other libraries that they transitively depend on .. ? So I started looking for solutions, and while I found working ones, most solutions seem like a big mess and I wonder if this is really done in practice or if there is some idiomatic way for this.
To avoid more eventual headaches if I ever need a different case, I want to consider all combinations of static/shared libraries, i.e.
A & B are static
A is static, B is shared
A is shared, B is static
A & B are shared
To give some MWE of the code setup (the variables LIB1_ROOT and LIB2_ROOT in the CMakeLists.txt files are A/ and B/ respectively):
A/include/lib1.hh
struct Lib1 { void run() const; };
A/src/lib1.cc
#include <iostream>
#include <lib1.hh>
void Lib1::run() const { std::cout << "Hello from lib1\n"; }
A/CMakeLists.txt
cmake_minimum_required(VERSION 3.14)
project(A)
include_directories(include)
add_library(lib1 src/lib1.cc)
install(TARGETS lib1 DESTINATION "${CMAKE_CURRENT_SOURCE_DIR}/lib")
B/include/lib2.hh
class Lib2 {
class Implementation;
Implementation* impl;
public:
Lib2();
~Lib2();
void run() const;
};
B/src/lib2.cc
#include <iostream>
#include <lib1.hh>
#include <lib2.hh>
class Lib2::Implementation {
const Lib1 m_lib1{};
public:
void run() const { std::cout << "using lib1 from lib2: "; m_lib1.run(); }
};
Lib2::Lib2() : impl{new Implementation} {}
Lib2::~Lib2() { delete impl; };
void Lib2::run() const { impl->run(); }
App/src/app.cc
#include <lib2.hh>
int main() { Lib2 l; l.run(); }
App/CMakeLists.cc
cmake_minimum_required(VERSION 3.14)
project(App)
include_directories(include "${LIB2_ROOT}/include")
find_library(LIB2 lib2 "${LIB2_ROOT}/lib")
add_executable(app src/main.cc)
target_link_libraries(app "${LIB2}")
install(TARGETS app DESTINATION "${CMAKE_CURRENT_SOURCE_DIR}/bin")
I used the pImpl pattern for B since what is the point of hiding link dependencies when I then make the user of my library dig out all the headers anyway.
Finally B/CMakeLists.txt (for my library) depends on the cases I mentioned above:
A & B static
B/CMakeLists.txt
cmake_minimum_required(VERSION 3.14)
project(B)
include_directories(include "${LIB1_ROOT}/include")
find_library(LIB1 lib1 "${LIB1_ROOT}/lib")
add_library(lib2_dependent src/lib2.cc)
add_custom_target(lib2 ALL
COMMAND ar -x "${LIB1}"
COMMAND ar -x "$<TARGET_FILE:lib2_dependent>"
COMMAND ar -qcs "${CMAKE_STATIC_LIBRARY_PREFIX}lib2${CMAKE_STATIC_LIBRARY_SUFFIX}" *.o
COMMAND rm *.o
DEPENDS lib2_dependent
WORKING_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/lib"
)
This solution is from the question I linked at the start. There was also an in my opinion much better version in this answer using
ar -M <<EOM
CREATE lib2.a
ADDLIB lib1.a
ADDLIB lib2_dependent.a
SAVE
END
EOM
but I could not get to work the here-document inside CMakeLists.txt ... ? There was an additional answer providing what I think was a CMake function to do this, but it was a huge block of code which I found a bit ridiculous for something that should be simple / standard practice / integrated into CMake ?
The custom_target solution I wrote here also works, but as mentioned in other answers as well, it unpacks object files which lie around and have to be removed again, for each library I want to compile this way.
And still in both cases, I can only wonder what is the point of using CMake then if I have to use ar manually anyhow. +Q+ Is there no better / CMake-integrated way to "compile-in" transitive dependencies / combine static libraries?
A static, B shared
As far as I found in this case, I'm out of luck if I cannot recomple A and it is not compiled as position independent code. The next best thing to make it work was to do just that by adding set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fPIC") to A/CMakeLists.txt. Again, there seemed to be better alternatives: set(CMAKE_POSITION_INDEPENDENT_CODE ON) or set_property(TARGET lib1 PROPERTY POSITION_INDEPENDENT_CODE ON) from here, which were complete ignored in my case .. (compiling with VERBOSE=1, there was no -fPIC flag to be seen anywhere and B did not compile)
The B/CMakeLists.txt was easy in this case
cmake_minimum_required(VERSION 3.14)
project(B)
include_directories(include "${LIB1_ROOT}/include")
find_library(LIB1 lib1 "${LIB1_ROOT}/lib")
add_library(lib2 SHARED src/lib2.cc)
target_link_libraries(lib2 "${LIB1}")
install(TARGETS lib2 DESTINATION "${CMAKE_CURRENT_SOURCE_DIR}/lib")
and while I don't find anything wrong with this solution, according to answers I found I should need to set additional flags like set(CMAKE_SHARED_LINKER_FLAGS "-Wl,--export-all-symbols") in order for the symbols in the static library to be found. However the above works just fine, compiles, and App runs without issues? +Q+ Am I doing anything wrong here? Or is it maybe due to some update to CMake since these older answers?
A shared, B static or both shared
According to what I found here, this is basically impossible, because shared libraries are "final" in some sense. I find this very strange, surely there are many libraries that do not require a project that uses them to link every single dependency of that library that happens to be a shared library? +Q+ Are there really no options in these cases?
Yep, you're doing wrong :)
The CMake-way is to use packages. Once you've made the LibA package, you just do find_package(LibA) in your B/CMakeLists.txt and also the generated LibBConfig.cmake (the package config file, so your clients would need just find_package(libB) in their App/CMakeLists.txt) ought to find (that is questionable IMHO, but currently its CMake way) its dependency the same way (using or not the CMake's helper find_dependency).
The whole process gets much simpler that way:
you have/give the full control on how to build (including what version, library type static/dynamic, &etc) and where to install the libA on your developer's host and customer's machine (so dependent projects could find it)
same for libB and App
you and any customer of your library(ies) use the well known CMake way to find dependencies and have full control over this process
and the most important let CMake do complicated things instead of you that keeps your build logic much simpler in CMakeLists.txt of all involved projects
I'm currently transforming my C++ project from simply using a Makefile to using CMake. As I'm unfamiliar with CMake, I try and avoid learning "old" CMake and stick to "modern CMake" best practices, e.g. described here.
However, I could not figure out how to include a simple, header-only, third party library without getting bothered with warnings about that library.
Say I have a project structure like this:
CMakeLists.txt
src/
CMakeLists.txt
main.cpp
third-party-lib/
foo/
foo.h
Say main.cpp uses foo: #include <foo.h>
The problem is, I use a bunch of warning flags and foo.h introduces some warnings that I don't care about, as I'm not the author and it's nothing serious.
In a classic Makefile I would just have written -isystem third-party-lib/foo somewhere appropriate and be done.
In CMake, I achieved this by a include_directories(SYSTEM PRIVATE third-party-lib/foo) in the top-level CMakeLists.txt.
Now, according to the link above, I should keep my hands off include_directories() to avoid hidden dependencies and I think that this is a good idea. I'd rather specify for each target that actually needs foo to include that library and treat it as 'don't warn me here'. As I understand it, the approach would be:
find foo target in top-level CMakeLists.txt
target_link_libraries(main PRIVATE foo) or so in src/CMakeLists.txt
How would I achieve that, i.e. what is the command for finding, and where to specify the SYSTEM property? Or any other suggestions for how to approach this?
To add a header-only library in modern CMake, you can use target_include_directories with SYSTEM INTERFACE. You can place this in your top-level CMake file, before processing the src/CMakeLists.txt file:
add_library(foo INTERFACE)
target_include_directories(foo SYSTEM INTERFACE
${CMAKE_SOURCE_DIR}/third-party-lib/foo
)
Your friend is target_include_directories. It works like include_directories but on a per-target basis.
add_executable(foo foo.c)
target_include_directories(foo SYSTEM
PRIVATE sys_inc
)
I would like to build an executable from static libraries (i. e. .a-files) only. This is possible, because the main() function is contained in one of these libraries.
The add_executable() function requires me to provide at least one source file. But this is not what I want to do.
There is no way to do it without a hack. You need at least one *.c or *.cpp file.
What I do is make a dummy null.cpp file (zero bytes) and use that. You can also use /dev/null but that only works on Linux.
file(WRITE null.cpp "")
add_executable(tester
null.cpp
)
target_link_libraries(tester
-Wl,--whole-archive
libtest1
libtest2
libtest3
libtest4
-Wl,--no-whole-archive
gtest_main
)
There are mainly two reasons why a source file is enforced by CMake:
To determine the LINKER_LANGUAGE from the file ending(s)
Not all compilers do support an object/library only link step (for details see below)
And if you move the main() function to library please keep the following in mind: Why does the order in which libraries are linked sometimes cause errors in GCC?
So if you build the libraries with CMake in the same project, I would recommend to change your libraries (at least the one containing your main() function) to an object library:
cmake_minimum_required(VERSION 2.8.8)
project(NoSourceForExe)
file(WRITE main.cc "int main() { return 0; }")
add_library(MyLibrary OBJECT main.cc)
add_executable(MyExecutable $<TARGET_OBJECTS:MyLibrary>)
The add_library() documentation lists a warning here:
Some native build systems may not like targets that have only object files, so consider adding at least one real source file to any target that references $<TARGET_OBJECTS:objlib>.
But those are rare and listed in Tests/ObjectLibrary/CMakeLists.txt:
# VS 6 and 7 generators do not add objects as sources so we need a
# dummy object to convince the IDE to build the targets below.
...
# Xcode does not seem to support targets without sources.
Not knowing which host OS(s) you are targeting, you may just give it a try.
References
CMake Object Lib containing main
CMake/Tutorials/Object Library