I am trying to "duplicate" a target in my CMake file without maintaining 2 targets and all it's dependencies.
For example I have a main target MyBigLibrary
add_library(MyBigLibrary STATIC "")
target_compile_definitions(MyBigLibrary PRIVATE definitions..)
target_include_directories(MyBigLibrary PUBLIC public_directories..)
target_include_directories(MyBigLibrary PRIVATE private_directories..)
target_link_libraries(MyBigLibrary INTERFACE libraries..)
...
...
target_sources(MyBigLibrary ..source files..)
What I am trying to achieve is to have a identical copy of MyBigLibrary target (e.g. MyBigLibraryModified) which then I can feed to external script via add_custom_command.
I know there is way to have 2 targets, but then you have maintain 2 targets and all of it's dependencies.
Is there a way to have a phony target, e.g MyBigLibraryModified which is built only MyBigLibrary, and inherits INTERFACE flags as a dependencies?
When I wanted to create two different libraries containing the same compiled objects but with different features, I used the OBJECT library type containing all the source files, then I created two different libraries that listed the OBJECT target in their target_link_libraries.
The OBJECT library in CMake is still not fully correct, even after years of waiting for it to be enhanced, but it works mostly OK in most situations these days.
See the documentation for more info.
Comments under question:
... Are you trying strip symbols from release version of your target? –
Marek R
#MarekR that is correct. – pureofpure
So you are doing that wrong. You do not need (and you should not have) separate target for stripped version of your library/application.
For stripped version build proves is just a bit different.
One way to do it is just do:
$ cmake --install . --prefix PathWithResultsWithSybols .....
$ cmake --install . --prefix PathWithStripedResults --strip ....
See cmake doc
Related
I am using cmake to create my static libraries with something along the lines of
add_library(library library.cpp)
install(TARGETS library DESTINATION lib)
which creates liblibrary.a which is what I want. However I would like to bundle that with a library, let's say vendor/proprietary.a by doing something custom like
tmp=$(mktemp -d)
cd $tmp
ar -x $<TARGET_FILE:library>
ar -x vendor/proprietary.a
ar -qc $<TARGET_FILE:library> *
rm -rf $tmp
Can I do that with cmake without it forgetting that the target library is actually a library (eg by using add_custom_command/add_custom_target).
This is, disappointingly, quite hard. We managed to do it on the Halide team, but only because it was a hard requirement from a corporate client. To other readers without such constraints, I say this: here be dragons. Use CMake's usual targets and dependencies and let it put all the static libraries on the end-product's link line.
To OP, I say, try this:
First, create a CMakeLists.txt in your vendor directory with the following content:
# 0. Convenience variable
set(proprietary_lib "${CMAKE_CURRENT_SOURCE_DIR}/proprietary.a")
# 1. Get list of objects inside static lib
execute_process(COMMAND "${CMAKE_AR}" -t "${proprietary_lib}"
WORKING_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}"
OUTPUT_VARIABLE proprietary_objects)
string(STRIP "${proprietary_objects}" proprietary_objects)
string(REPLACE "\n" ";" proprietary_objects "${proprietary_objects}")
# 2. Attach configure dependency to the static lib
set_property(DIRECTORY . APPEND PROPERTY
CMAKE_CONFIGURE_DEPENDS proprietary.a)
# 3. Extract the lib at build time
add_custom_command(
OUTPUT ${proprietary_objects}
COMMAND "${CMAKE_AR}" -x "${proprietary_lib}"
DEPENDS "${proprietary_lib}")
# 4. Get absolute paths to the extracted objects
list(TRANSFORM proprietary_objects
PREPEND "${CMAKE_CURRENT_BINARY_DIR}/")
# 5. Attach the objects to a driver target so the
# custom command doesn't race
add_custom_target(proprietary.extract DEPENDS ${proprietary_objects})
# 6. Add a target to encapsulate this
add_library(proprietary OBJECT IMPORTED GLOBAL)
set_target_properties(proprietary PROPERTIES
IMPORTED_OBJECTS "${proprietary_objects}")
# TODO: add usage requirements
# target_include_directories(proprietary INTERFACE ...)
# 7. Force proprietary to run completely after extraction
add_dependencies(proprietary proprietary.extract)
There's a lot going on here, but ultimately the steps are straightforward and the complications are with explaining the dependencies to CMake. Also, this comes with the caveat that it is Linux-only (or at least GNU-ar-compatible archiver only). It is possible to do something similar for MSVC, but it would be too much for this answer.
So first we ask the archiver which objects are in the library and we lightly process its one-object-per-line output into a CMake list. That's step 1 above.
Step 2 tells CMake that if the timestamp on proprietary.a is ever updated, then it will need to re-run CMake (and thereby get a new list of objects).
Step 3 creates a custom command which will, at build time, run the archiver tool to extract the objects into the vendor build directory.
Step 4 turns the (relative) list of objects into a list of absolute paths to those objects after the custom command runs. This is for the benefit of add_custom_target which expects absolute paths (or rather, does weird things with relative paths if certain policies are enabled).
Step 5 creates a custom target to drive the archive extraction.
Step 6 creates an imported object library to encapsulate the extracted library. It has to be global because imported targets are directory-scoped by default and this is an abuse of the imported-library feature. You can add additional usage requirements here.
Finally, step 7 puts a dependency to the driver target on the object library.
This can then be used transparently. Here's an example:
cmake_minimum_required(VERSION 3.16)
project(example)
add_subdirectory(vendor)
add_library(library library.cpp)
target_link_libraries(library PRIVATE proprietary)
Let's say I have 2 different conanfile.py in a project and I'm calling conan install two times to install their dependencies. I'm having trouble while adding them to cmake.
If I use basic setup
include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
conan_basic_setup()
it only includes latest one. Is it possible to include multiple conanbuildinfo.cmake files ?
If you have 2 completely separate projects, you can have 2 different conanfiles and put the generated files in different folders:
$ conan install conanfile1.py --install-folder=folder1
$ conan install conanfile2.py --install-folder=folder2
Then in your first project:
include(<...>/folder1/conanbuildinfo.cmake)
conan_basic_setup()
And in your second project:
include(<...>/folder2/conanbuildinfo.cmake)
conan_basic_setup()
You would need to define some consistent convention to locate the generated files for each project.
Note, however, that if the different modules are intended to use together, like linked together lately, if you don't use the same dependencies and same versions, you will probably get linking or runtime errors in your global application. If the modules are related and you want to use the same versions of the dependencies, then you definitely want to use just 1 conanfile with all dependencies defined in it.
Note that there are different ways to define the specific dependencies that you want, even if you use only 1 conanfile:
You can use the TARGETS of the cmake generator:
include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
conan_basic_setup(TARGETS)```
add_library(mylib1 ... <sources>)
target_link_libraries(mylib1 PUBLIC CONAN_PKG::Dep1 CONAN_PKG::Dep2)
add_library(mylib2 ... <sources>)
target_link_libraries(mylib2 PUBLIC CONAN_PKG::Dep3 CONAN_PKG::Dep4)
The cmake_find_package generators also generate one findXXXX.cmake file for each package in the dependency graph. You can use the find_package(XXXX) and later the results, specifying different dependencies. The cmake_find_package_multi generator is recommended.
I have a c++ cmake project. In this project I build (among other) one example, where I need to use another project, call it Foo. This Foo project does not offer a cmake build system. Instead, it has a pre-made Makefile.custom.in. In order to build an executable that uses Foo's features, one needs to copy this makefile in his project, and modify it (typically setting the SOURCES variable and a few compiler flags). Basically, this Makefile ends up having the sources for your executable and also all the source files for the Foo project. You will not end up using Foo as a library.
Now, this is a design I don't like, but for the sake of the question, let's say we stick with it.
To create my example inside my cmake build I added a custom target:
CONFIGURE_FILE( ${CMAKE_CURRENT_SOURCE_DIR}/Makefile.custom.in Makefile.custom)
ADD_CUSTOM_TARGET(my_target COMMAND $(MAKE) -f Makefile.custom
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR})
This works. I can specify some variables to cmake, which get resolved in the call to CONFIGURE_FILE, and I end up with a working Makefile.custom. Then, invoking make my_target from the build directory, I can build the executable. I can even add it to the all target (to save me the effort of typing make my_target) with
SET_TARGET_PROPERTIES(my_target PROPERTIES EXCLUDE_FROM_ALL FALSE)
Sweet. However, cmake appears to assign a single job to the custom target, slowing down my compilation time (the Foo source folder contains a couple dozens cpp files). On top of that, the make clean target does not forward to the custom makefile. I end up having to add another target:
ADD_CUSTOM_TARGET(really-clean COMMAND "$(MAKE)" clean
COMMAND "$(MAKE)" -f Makefile.custom clean
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR})
which, unlike my_target with all, I can't include in the clean target (can I?).
Now, I know that a cleaner solution would be to have the Foo project be built as an external project, and then link to it. However, I've been 'recommended' to use their Makefile.custom.in makefile, modifying the few lines I need (adding my sources, specifying compiler flags, and few other minor modifications). So, regardless of how neat and clean this design pattern is, my questions are:
is there a way to tell cmake that make should use more than 1 job when making the target my_target?
is there a cleaner way to include a pre-existing makefile in a cmake project? Note that I don't want (can't?) use Foo as a library (and link against it). I want (need?) to compile it together with my executable using a makefile not generated by cmake (well, cmake can help a bit, through CONFIGURE_FILE, by resolving some variables, but that's it).
Note: I am aware of ExternalProject (as suggested also in this answer), but I think it's not exactly what I need here (since it would build Foo and then use it as a library). Also, both my project and Foo are written exclusively in C++ (not sure this matter at all).
I hope the question makes sense (regardless of how ugly/annoying/unsatisfactory the resulting design would be).
Edit: I am using cmake version 3.5.2
First, since you define your own target, you can assign more cores to the build process for the target my_target, directly inside your CMakeLists.txt.
You can include the Cmake module ProcessCount to determine the number of cores in your machine and then use this for a parallel build.
include(ProcessorCount)
ProcessorCount(N)
if(NOT N EQUAL 0)
# given that cores != 0 you could modify
# math(EXPR N "${N}+1") # modify (increment/decrement) N at your will, in this case, just incrementing N by one
set(JOBS_IN_PARALLEL -j${N})
endif(NOT N EQUAL 0)
and when you define your custom target have something like the following:
ADD_CUSTOM_TARGET(my_target
COMMAND ${CMAKE_MAKE_PROGRAM} ${JOBS_IN_PARALLEL} -f Makefile.custom
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR})
by the way, I don't think there's the need for you to include also CMAKE_BUILD_TOOL among the COMMANDs in your target.
I believe that instead of modifying the lines as above, you could call
make -j8 my_target
and it might start 8 jobs (just an example) without modifying the CMakeLists.txt, but I cannot guarantee this works having defined the COMMAND the way you have, just try if that's enough.
For the second point, I cannot think right now of a "cleaner" way.
I would like to have the following structure A -> B -> C, where:
C is boilerplate code, wrappers for third-party libraries, very
basic code etc.
B is the common classes, functions and data
structures specific to the project's domain.
A is the project itself.
I would like to make it easy to reuse C or B(+C) in future in my other projects. In addition, I have the following requirements:
As all three projects are in-progress, I would like to have an ability to build C, C+B and C+B+A in one shot.
I would prefer the static linkage over dynamic, so that C and C+B would be static libraries, and C+B+A would be the executable
I would like to keep cmake lists and config files simple and clean. Examples which I found in the official wiki and over the internet are pretty big and monstrous.
It would be great if it won't require changing more than a couple of lines if I'd change the locations of A, B or C in the filesystem.
All these three components are using google-test, but I'm not sure if it is important for the project layout.
I am pretty new to cmake and I don't even understand is it better to write XXXConfig.cmake or FindXXX.cmake files. Also, I am not sure, how should I pass relative paths from subcomponent to the parent component using X_INCLUDE_DIRS.
First I have to admit that I agree with #Tsyvarev. Your CMake environment should fit to your processes/workflow and should take project sizes and team structure into account. Or generally speaking the environment CMake will be used in. And this tends to be - in a positive way - very alive.
So this part of your question is difficult to answer and I'll concentrate on the technical part:
CMake has to know the location of the dependencies - relative or absolute - by
having a monolithic source tree (the one you don't want anymore)
CMake share library with multiple executables
CMake: How to setup Source, Library and CMakeLists.txt dependencies?
a common directory location for includes/libraries/binaries
Custom Directory for CMake Library Output
cmake install not installing libraries on windows
getting the paths via config files/variable definitions
How can I get cmake to find my alternative boost installation?
How to add_custom_command() for the CMake build process itself?
using registration in or installation from a database provided on the host
Making cmake library accessible by other cmake packages automatically
cmake wont run build_command in ExternalProject_Add correctly
To keep your CMake files as simple as possible I would recommend to group your CMake code into separate dedicated files:
Prefer toolchain files over if(SomeCompiler) statements
Move common/repeating code parts as function() bodies into a shared CMake include file
Move complex non-target specific code parts into their own (CMake) script files
Example Code
Since you have specifically asked for the find_package() variant, taking Use CMake-enabled libraries in your CMake project and the things listed above:
MyCommonCode.cmake
cmake_policy(SET CMP0022 NEW)
function(my_export_target _target _include_dir)
file(
WRITE "${CMAKE_CURRENT_BINARY_DIR}/${_target}Config.cmake"
"
include(\"\$\{CMAKE_CURRENT_LIST_DIR\}/${_target}Targets.cmake\")
set_property(
TARGET ${_target}
APPEND PROPERTY
INTERFACE_INCLUDE_DIRECTORIES \"${_include_dir}\"
)
"
)
export(
TARGETS ${_target}
FILE "${CMAKE_CURRENT_BINARY_DIR}/${_target}Targets.cmake"
EXPORT_LINK_INTERFACE_LIBRARIES
)
export(PACKAGE ${_target})
endfunction(my_export_target)
C/CMakeLists.txt
include(MyCommonCode.cmake)
...
my_export_target(C "${CMAKE_CURRENT_SOURCE_DIR}/include")
B/CMakeLists.txt
include(MyCommonCode.cmake)
find_package(C REQUIRED)
...
target_link_libraries(B C)
my_export_target(B "${CMAKE_CURRENT_SOURCE_DIR}/include")
A/CMakeLists.txt
include(MyCommonCode.cmake)
find_package(B REQUIRED)
...
target_link_libraries(A B)
This keeps all 3 build environments separate, only sharing the relatively static MyCommonCode.cmake file. So in this approach I have so far not covered your first point, but would recommend the use of a external script to chain/trigger your build steps for A/B/C.
I need to build two 3rd party shared libraries, so their .so files will be reused by other projects. However, after build one of these libraries contains hardcoded path to another. This path is invalid on other machines and causes linker warnings. How can I prevent the full path from being embedded in the resulting .so files?
Details:
First library source: ~/dev/A
Second library source: ~/dev/B
Both of them have configure script to generate make files. Library B depends on A. So, first I build A:
$ ~/dev/A/configure --prefix=~/dev/A-install
$ make && make install
Then I build B:
$ ~/dev/B/configure --prefix=~/dev/B-install --with-A=~/dev/A-install
$ make && make install
Then I want to upload the contents of ~/dev/A-install and ~/dev/B-install to our file server, so other teams and build machines can use the binaries. But they get linker warnings when they try to use B:
/usr/bin/ld: warning: libA.so.2, needed by /.../deps/B/lib/libB.so, not found (try using -rpath or -rpath-link)
When I run ldd libB.so it gives:
...
libA.so.2 => /home/alex/dev/A-install/lib/libA.so.2
Obviously this path exists only on my machine and cannot be found on other machines.
How can I remove full hardcoded path from libB.so?
Thanks.
You have to use --prefix value that will be valid in the runtime environment for both packages!
Than you override prefix or DESTDIR (prefix replaces the prefix, DESTDIR is prepended to it, but works more reliably) on the make command-line when installing. Like:
~/dev/A$ ./configure
~/dev/A$ make
~/dev/A$ make install prefix=~/dev/A-install
~/dev/B$ ./configure --with-A=~/dev/A-install
~/dev/B$ make
~/dev/B$ make install prefix=~/dev/B-install
or (which is preferred and is how all package-building tools use it):
~/dev/A$ ./configure
~/dev/A$ make
~/dev/A$ make install DESTDIR=~/dev/A-install
~/dev/B$ ./configure --with-A=~/dev/A-install/usr/local
~/dev/B$ make
~/dev/B$ make install prefix=~/dev/B-install
because this way you are installing to ~/dev/A-install/$prefix, so with default prefix ~/dev/A-install/usr/local. The advantage of this later option is, that if you redefine some specific installation paths without refering to prefix (say --sysconfdir=/etc), DESTDIR will still get prepended to it, while it won't be affected by prefix.
-Wl,-rpath,~/deps/A/lib:~/deps/B/lib:~/dev/MyApp/bin
This linker option is responsible for saving the path inside the library. You need somehow to remove this.
See with ./configure --help if there's some option that could influence this. Another option is to edit manually the makefile and remove this option.
== edit2 ==
One more thing
-L~/deps/A/lib -L~/deps/B/lib ~/deps/A/lib/libA.so ~/deps/B/lib/libB.so
If libA.so and libB.so don't have SONAME, linking them like "~/deps/A/lib/libA.so" will also cause saving the path. Soname is set using -Wl,-soname,<soname> linker option when building shared library.
If soname is set in the shared library, linking it using "~/deps/A/lib/libA.so" form is ok.
Like Jan mentioned in the comments, the better way is using "-Llibrary/path -llibrary_name" without rpath.
-L~/deps/A/lib -L~/deps/B/lib -lA -lB
When I run ldd libB.so it gives:
libA.so.2 => /home/alex/dev/A-install/lib/libA.so.2
The low-level solution to this problem is to use the option "-soname=libA.so" when you link the libA.so library. By having SONAME defined for a shared object, the linker will not embed absolute paths when linking against that shared object.
The OP is using "configure", so this isn't an easy solution to implement unless he is willing to go into the bowels of the Makefile generated by the configure script.
Shared libraries and executables have a list of directories to look for shared libraries, in addition to the list in the operating system's configuration. RPATH is used to add shared library search paths, used at runtime.
If you want a relative path to be used in RPATH, there is a special syntax that most Linux/UNIX (but not AIX) systems support - $ORIGIN or ${ORIGIN}.
$ORIGIN will expand at runtime to the directory where the binary resides - either the library or executable.
So if you put executable binaries in prefix/bin/ and shared libraries in prefix/lib/ you can add an entry to RPATH like ${ORIGIN}/../lib and this will expand at runtime to prefix/bin/../lib/
It's often a little trick to get the syntax correct in a Makefile, because you have to escape the $ in ORIGIN so it will not be expanded. It's typical to do this in a Makefile:
g++ -Wl,-rpath=\$${ORIGIN}/../lib
Both Make and the shell will want to look in your environment for a variable called ORIGIN - so it needs to be double-escaped.
I just got caught out thinking I had the same problem.
None of the above answers helped me.
Whenever I tried
ldd libB.so
I would get in the output:
libA.so.1 => /home/me/localpath/lib/libA.so.1.0
and so I thought I had a hardcoded path. Turns out that I had forgotten I had LD_LIBRARY_PATH set for local testing. Clearing LD_LIBRARY_PATH meant that ldd found the correct installed library in /usr/lib/
Perhaps using the -rpath and -soname options to ld could help (assuming a binutils or binutils.gold package for ld on a recent Linux system)?