I am trying to compile two executables foo and bar with cmake generating "Unix makefiles" under cygwin. Both executables are built from a single cmake project, so I have only a single CmakeLists.txt file.
The dependency between foo and bar is generated by an object file named depend.o, which is generated by the linker when linking foo.
I am able to tell cmake, that bar has an external dependency to a generated object file. I am also able to tell cmake that bar depends on foo, to get proper build order, but still the build fails when parallel builds are enabled.
This is what I did:
add_executable(bar .....)
add_executable(foo .....)
....
#Tell the linker we have an additional external library.
set_property(bar APPEND PROPERTY LINK_LIBRARIES "${CMAKE_CURRENT_BINARY_DIR}/depend.o")
#Tell cmake depend.o is a generated object file.
SET_SOURCE_FILES_PROPERTIES("${CMAKE_CURRENT_BINARY_DIR}/depend.o" PROPERTIES EXTERNAL_OBJECT true GENERATED true)
#And finally create dependency between the foo and bar
add_dependencies(bar foo)
So the question is: what trick is needed to get parallel builds work or how to properly define the dependency between foo and bar?
Edit:
My question is not a duplicate of CMake: reuse object files built for a lib into another lib target. That question is about how to use the same object file in multiple libraries which can be done with "object libraries". I am after solving a dependency problem between two executables.
Related
In a large CMake/C++ project our major library target Foo is being added. It has many statically linked dependencies that are provided as absolute-paths to target_link_libraries(). Now I'm writing the CMake to export this library once built, including the generated CMake so a library consumer can simply use CMake find_package(). However the generated FooTargets.cmake is embeding the absolute-paths of the link-libraries instead of just the library names.
Additionally, I've already created an add_library(BarCommon INTERFACE), to wrap up the referenced 3rd-party static libs separately and that seems to be working and the client-application consumed it. That accounts for 90% of the libraries also mentioned by FOO. So really FOO's exported INTERFACE_LINK_LIBRARIES should be about 3 file names instead of 40 absolute-pathnames.
I'm confused, why is the target_link_libraries(FOO PRIVATE ${libs} is defining the exported 'INTERFACE_LINK_LIBRARIES' property?
I would like that generated INTERFACE_LINK_LIBRARIES property to be only library filenames, and optimally have finer control over which filenames end up in it. How do I control this?
Code:
Inside a large CMake Macro for generating targets we have this:
-
I call it to create library/target FOO.
_TARGET_ARGS_LIB_DEPENDS will resolve to ABSOLUTE FILE PATHNAMES for about 40 statically linked library dependencies.
add_library(${TargetName} STATIC ${AllSources} ${_TARGET_ARGS_OBJ_FILES})
<snip>
target_link_libraries(${TargetName}
PRIVATE
$<${GCC}:-Wl,--start-group>
${_TARGET_ARGS_LIB_DEPENDS}
$<${GCC}:-Wl,--end-group>
)
where that longish function returns I have just added these blocks:
target_link_directories(FOO
INTERFACE
$<BUILD_INTERFACE:${CMAKE_CURRENT_BINARY_DIR}>
$<INSTALL_INTERFACE:foo/lib>
)
target_include_directories(FOO
INTERFACE
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}>
$<INSTALL_INTERFACE:foo/include>
)
install(TARGETS Foo
EXPORT
FooTargets
DESTINATION
foo/lib
)
There is some more 'install' boilerplate at the end of my CMakeLists.txt
After I run CMAKE, build the project and run the 'INSTALL' target CMake generated from Visual Studio I can examine generated file: local_install\Release\Windows\Foo\lib\cmake\Foo\FooTargets.cmake shows:
set_target_properties(Nexidia.Workbench.StaticLib.StaticLib PROPERTIES
INTERFACE_INCLUDE_DIRECTORIES "${_IMPORT_PREFIX}/Foo/include"
INTERFACE_LINK_DIRECTORIES "${_IMPORT_PREFIX}/Foo/lib"
INTERFACE_LINK_LIBRARIES "\$<LINK_ONLY:\$<0:-Wl,--start-group>>;C:/full/path/to/libA.lib;C:/full/path/to/libB.lib \$<LINK_ONLY:\$<0:-Wl,--end-group>>"
)
(with the full paths to about 40 libraries).
PS. I'm reading Professional CMake: A Practical Guide, so if someone could cite the chapter/sub-section that answers this it might help.
This is because your library is static and so it cannot be correctly linked to consumers without including its dependencies on consumers' link lines. The same thing happens with the shared dependencies of shared libraries. Static dependencies of shared libraries are fully "absorbed" into the shared library and so they don't appear.
CMake is aware of all this, and so to respect the other semantics of PRIVATE, it inserts the $<LINK_ONLY:...> generator expression. This ensures that the libraries are used for linking only and that any associated INTERFACE_* properties are not propagated.
In other words, CMake is behaving correctly here.
You should resolve this by creating and linking to targets for libB and friends that you can either export as part of your main build or import into both your main build and via find_dependency in your distributed package, along with the libraries themselves, of course.
The best advice I can give you is to always link to targets in CMake, never to paths or raw library flags.
I've tried to build a rather large shared library on Windows with cmake + ninja + msvc, that is composed of multiple static libs from subfolders. So a root CMakeLists.txt looks like:
project (sharedlib CXX)
include(${CMAKE_CURRENT_LIST_DIR}/staticlib1/CMakeLists.txt)
include(${CMAKE_CURRENT_LIST_DIR}/staticlib2/CMakeLists.txt)
add_library(sharedlib SHARED)
target_link_libraries(sharedlib
staticlib1
staticlib2
)
set_target_properties(wux PROPERTIES LINK_FLAGS "/WHOLEARCHIVE")
Where the CMakeLists.txt in sub-folders staticlib1 and staticlib2 both look something like:
add_library(staticlib1 STATIC)
target_sources(staticlib1 PUBLIC
${CMAKE_CURRENT_LIST_DIR}/sourceA.cpp
${CMAKE_CURRENT_LIST_DIR}/sourceB.cpp
)
target_include_directories(staticlib1 PUBLIC
${CMAKE_CURRENT_LIST_DIR}/inc
)
target_compile_options(staticlib1 PUBLIC
/flag1
/flag2
)
When I run cmake --build both staticlib1 and staticlib2 get built no problem. Cool. But when the linker tries to build sharedlib, the cmake-generated rsp file has:
CMakeFiles\sharedlib.dir\staticlib1\sourceA.cpp.obj
CMakeFiles\sharedlib.dir\staticlib1\sourceB.cpp.obj
CMakeFiles\sharedlib.dir\staticlib2\sourceC.cpp.obj
CMakeFiles\sharedlib.dir\staticlib2\sourceD.cpp.obj
staticlib1.lib
staticlib2.lib
So I get linker errors because symbols are defined twice. How do I get cmake to stop adding both the objects and the final libs to the linker rsp?
Technically I don't need the static libs after the fact. However, I cannot simply switch to using OBJECT libraries. The actual project has ~250 static libs, comprising ~3500 object files. The linker dies (out of memory) with just a fraction of the object files.
I must build up the static libs first and then link just them into the shared lib after the fact, to get around the memory limitations of the linker. Our current build scripts follow this pattern just fine, so I know it works. I just need cmake to follow the same pattern.
This is the effect of target_sources command: with PUBLIC keyword it adds sources both for the library (static) and for anyone who links that library.
You should instead use PRIVATE keyword, or, better, add sources in the add_library call itself:
add_library(staticlib1 STATIC
${CMAKE_CURRENT_LIST_DIR}/sourceA.cpp
${CMAKE_CURRENT_LIST_DIR}/sourceB.cpp
)
Background
I have a project that uses other smaller projects. These projects themselves are made of other projects. A lot of this is legacy or has other managerial-fiat reasons for being arranged as is, so rolling everything into a single project is not an option. Some libraries are pre-compiled on remote shares.
I have 2 main subprojects that are giving me a headache:
Project Foo is an executable and library that links several static subprojects (foo_subproject_1, foo_subproject_n). These subprojects are further linked against static libraries in remote locations (some_lib, some_other_lib). Project Foo's executable compiles, links, and runs correctly
Project Bar is an executable that links several other projects, including libFoo. Linking fails with "undefined reference to " foo_subproject functions
As far as I can tell, the two projects are arranged similarly with their linkage instructions. Looking on SO, I discover that linking static libraries against static libraries shouldn't work, but then I'm confused as to how Project Foo is compiled successfully.
gcc and g++ 4.9.2 are the compilers (extern "C" problems of having some parts in C and some in C++ have already been checked for)
Question
I have misunderstood something about either or both how CMake add_subdirectory works, or about how the linker works. Can someone please explain how Project Foo works successfully, and Project Bar (doesn't) works as expected?
Update I looked closer at foo_lib.a and foo_runtime.
I should have determined that something was off to start with, because foo_runtime is nearly 100MB in size, and foo_lib is only 10KB.
nm reveals that foo_lib.a references a few dozen symbols, most of which are undefined. foo_runtime meanwhile references everything.
Equally confusing is that foo_subproject_1.a is similarly mostly undefined. Again, this is what I expect to see; but I don't understand how foo_runtime can be built from this?
I'm still unclear as to why some_library -> subproject -> foo_runtime is successful, but some_library -> subproject -> foo_lib -> bar isn't. At this stage of my investigations, I am expecting both commands to fail.
Project Foo is arranged (using CMake) thusly:
cmake_minimum_required(VERSION 2.6)
project(foo)
set(FOO_SRCS
# source and headers for main foo project
)
# Project Foo libraries are subdirectories within this project
add_subdirectory(subs/foo_subproject_1)
add_subdirectory(subs/foo_subproject_2)
# Runtime executable
add_executable(foo_runtime main.c ${FOO_SRCS})
target_link_libraries(foo_runtime foo_subproject_1 foo_subproject_2)
# Library version (static library)
add_library(foo_lib STATIC ${FOO_SRCS})
target_link_libraries(foo_lib foo_subproject_1 foo_subproject_2)
Project Foo's subdirectories loosely have the following architecture:
cmake_minimum_required(VERSION 2.6)
project(foo_subproject_<n>)
set(FOO_SUBPROJECT_<N>_SRCS
# source and headers for subproject
)
# foo_subproject's remote libraries are all static
add_library(some_lib STATIC IMPORTED)
set_target_properties(some_lib PROPERTIES IMPORTED_LOCATION /path/to/libsome_lib.a)
add_library(some_other_lib STATIC IMPORTED)
set_target_properties(some_other_lib PROPERTIES IMPORTED_LOCATION /path/to/libsome_other_lib.a)
include_directories(/paths/to/libs/include/)
# Static library for foo_subproject_N, links against static libs above
add_library(foo_subproject_<N> STATIC ${FOO_SUBPROJECT_<N>_SRCS})
target_link_libraries(foo_subproject_<N> some_library some_other_library)
Project Bar is arranged thusly:
cmake_minimum_required(VERSION 2.6)
project(bar)
set(BAR_SRCS
# source and headers for main bar project
)
# Project Bar libraries are remote from Bar's perspective
add_library(foo_lib STATIC IMPORTED)
set_target_properties(foo_lib PROPERTIES IMPORTED_LOCATION /path/to/foo/libfoo_lib.a)
include_directories(/path/to/foo/include/)
# Runtime executable
add_executable(bar main.c ${BAR_SRCS} foo_lib)
Project Bar fails to link (compiles ok) with multiple errors of the form:
bar_frobulator.cpp:123: undefined reference to 'foo_subproject_1_init_frobulation'
where foo_subproject_1_init_frobulation lives in foo_subproject_1
Can someone please explain how Project Foo works successfully, and Project Bar (doesn't) works as expected?
In short: Creating STATIC library doesn't involve linking step!
Details
In the Foo project you have an executable foo_runtime, which "works" because it is linked with proper libraries (e.g. with library foo_subproject_1 which defines foo_subproject_1_init_frobulation symbol).
An executable bar from Bar project doesn't perform that linking, so it fails. The line
target_link_libraries(bar foo_lib)
links with foo_lib, but this library doesn't defines the needed symbol foo_subproject_1_init_frobulation.
Note, that the line
target_link_libraries(foo_lib foo_subproject_1 foo_subproject_2)
in the Foo project doesn't perform actual linking: in general, building a static library doesn't involve linking step.
Given line just propagates include directories (and other compile-features) from foo_subproject_* libraries to the foo_lib one.
How to make it work
Because static library foo_lib doesn't track its dependency, you need to link bar with a library, which knows that. E.g., make foo_lib shared, or combine foo_subproject_* libraries into archive library, as suggested by the referenced question How to combine several C/C++ libraries into one?.
Alternatively, you may build Foo subproject within Bar one and, instead of creation of IMPORTED foo_lib target, use "normal" foo_lib target, created within Foo project. In that case, line
target_link_libraries(bar foo_lib)
would mean for CMake to (actually) link bar with foo_subproject_* libraries, because those libraries are "linked" (in CMake sense) into foo_lib. Again, the last "linking" has a meaning only for CMake: the file foo_lib.a doesn't aware about needing of foo_subproject_* libraries.
Tsyvarev's answer described well-enough what I was actually doing (rather than what I thought I was doing) to get me to look up the right things to answer the root question that I had ("Why does foo_runtime work but bar_runtime doesn't, when both link against static libraries linked against static libraries?")
From the CMake documentation for target_link_libraries:
Library dependencies are transitive by default with this signature. When this target is linked into another target then the libraries linked to this target will appear on the link line for the other target too.
target_link_libraries does not cause linking in the static libraries foo_subproject_1 and foo_subproject_2 (static libraries don't invoke the linker). What is does is makes the list of required libraries available for anything that tries to link with foo_subproject_1 or foo_subproject_2
So, effectively, my foo_runtime and foo_lib target_link_libraries command is, as far as CMake is concerned:
target_link_libraries(foo_runtime foo_subproject_1 some_library some_other_library foo_subproject_2 some_library some_other_library)
target_link_libraries(foo_lib foo_subproject_1 some_library some_other_library foo_subproject_2 some_library some_other_library)
foo_lib, being static, doesn't invoke the linker. foo_runtime, being an executable, does.
bar is a totally different project, so doesn't get to take advantage of the transitive dependencies (and linking fails, because I'm missing all the symbols from the lower-level libraries).
This behaviour of target_link_libraries wasn't expected, as such the behaviour of the project as a whole was somewhat misleading (since it looked like I was bundling a bunch of libraries at the subproject level, and then bundling those at the upper level. In reality, I had just told the upper level what all the libraries it needed were).
To summerise:
Q "Why does foo_runtime work but bar_runtime doesn't, when both link against static libraries linked against static libraries?"
A "foo_runtime is not linking against static libraries linked against static libraries. foo_runtime is linked against more static libraries than you originally thought it was linking against.
bar_runtime doesn't work because your foo_lib is basically empty"
I would like to build an executable from static libraries (i. e. .a-files) only. This is possible, because the main() function is contained in one of these libraries.
The add_executable() function requires me to provide at least one source file. But this is not what I want to do.
There is no way to do it without a hack. You need at least one *.c or *.cpp file.
What I do is make a dummy null.cpp file (zero bytes) and use that. You can also use /dev/null but that only works on Linux.
file(WRITE null.cpp "")
add_executable(tester
null.cpp
)
target_link_libraries(tester
-Wl,--whole-archive
libtest1
libtest2
libtest3
libtest4
-Wl,--no-whole-archive
gtest_main
)
There are mainly two reasons why a source file is enforced by CMake:
To determine the LINKER_LANGUAGE from the file ending(s)
Not all compilers do support an object/library only link step (for details see below)
And if you move the main() function to library please keep the following in mind: Why does the order in which libraries are linked sometimes cause errors in GCC?
So if you build the libraries with CMake in the same project, I would recommend to change your libraries (at least the one containing your main() function) to an object library:
cmake_minimum_required(VERSION 2.8.8)
project(NoSourceForExe)
file(WRITE main.cc "int main() { return 0; }")
add_library(MyLibrary OBJECT main.cc)
add_executable(MyExecutable $<TARGET_OBJECTS:MyLibrary>)
The add_library() documentation lists a warning here:
Some native build systems may not like targets that have only object files, so consider adding at least one real source file to any target that references $<TARGET_OBJECTS:objlib>.
But those are rare and listed in Tests/ObjectLibrary/CMakeLists.txt:
# VS 6 and 7 generators do not add objects as sources so we need a
# dummy object to convince the IDE to build the targets below.
...
# Xcode does not seem to support targets without sources.
Not knowing which host OS(s) you are targeting, you may just give it a try.
References
CMake Object Lib containing main
CMake/Tutorials/Object Library
I have a project using 3 CMakeLists.txt:
CMakeLists.txt C is my executable and depends on
CMakeLists.txt B which is a static lib and that depends on
CMakeLists.txt A which is also a static lib and depends on an external lib
In CMakeLists.txt C I specify my dependency against B using using target_link_libraries() and I do the same for the dependency of B against A. In CMakeLists.txt A I specify the dependency against the external lib.
I would expect this to work but C actually complains at link-time and I can only get it to work by specifying a dependency in C against the external lib.
Note that the external lib is dynamic (a .so file).
This looks weird to me, no? Anyone understands what is going on?
Thanks,
Antoine.
That should work. I bet there's a bug in the CMakeLists.txt.
View real dependencies
Check it with:
cmake .. --graphviz=deps.dot
xdot deps.dot
It will show a pretty picture of the dependency tree that cmake sees.
If you don't have xdot, export it to a png:
dot -Tpng deps.dot -o deps.png
firefox deps.png
Library not found ?
Another possibility is that the external library can't actually be found. Use find_library rather than just putting the library name:
find_library(FAIL failingmadly)
if (NOT FAIL)
message(FATAL_ERROR "Couldn't find the failingmadly library")
endif()
target_link_libraries(my_lib_a ${FAIL})
Position independent code?
Another possibility when linking static libs with dependencies on shared libs is the PIC complaints. You could add this in cmake before compiling anything:
add_definitions(-fPIC)
Good luck.