I want to integrate a header-only C++ library in my Autotools project. Since the library uses Autoconf and Automake, I use AC_CONFIG_SUBDIRS in configure.ac and added the library dir to the SUBDIRS = line in Makefile.am.
My question is: how do I prevent the header library from being installed by make install? I'm building a single binary, so my users don't need these headers.
I'd prefer not to tamper with the library, so I can fetch upgrade by just untarring the new version.
Here is an idea.
Move all the third-party libraries you do not want to see installed into a subdirectory called noinst/. So for instance if you want to ship your project with something like Boost, unpack it into the directory noinst/boost/. Use AC_CONFIG_SUBDIRS([noinst/boost]). Inside noinst/Makefile.am, do something like this:
SUBDIRS = boost
# Override Automake's installation targets with the command ":" that does nothing.
install:; #:
install-exec:; #:
install-data:; #:
uninstall:; #:
The effect is that whenever some of the recursive "make install*" or "make uninstall" commands are run from the top-level directory, the recursion will stop in noinst/ and not visit its subdirectories. Other recursive commands (like "make", "make clean" or "make dist") will still recurse into the subdirectories.
You could of course override install: and friends directly into the third-party package, and avoid the extra noinst/ directory. But if you are like me, you don't want to tamper with third-party packages to ease their update.
Also a nice property of the above setup is that if someone goes into noinst/boost/ and decide to run make install, it will work. It just does not occur by default when they install your package.
just came across a similar problem and found the solution in the automake manual:
noinst_HEADERS would be the right variable to use in a directory containing only headers and no associated library or program
Andreas
Don't use SUBDIRS then. The following hack may work:
all-local:
${MAKE} -C thatlib all
Of course it would be best if the library remained in its own directory outside of your project, and you just point to it via CFLAGS/LIBS flags.
Related
I have been working on a project which uses rplidar_sdk and in the beginning, I was facing this problem:
How can I link locally installed SDK's static library in my C++ project?
Basically, the SDK generates the library in its local directory, and in its Makefile, it does not have install rules. I mean I can run make but after that, if I run sudo make install then it gives make: *** No rule to make target 'install'. Stop. error.
So, with the help of this & this answer, I was able to build my local project. So far so good.
However, the main problem is that I have to hard-code the RPLidar SDK path in CMakeLists.txt of my repo. Now, whenever someone else in my team starts working on that repo (which is quite obvious) then he/she has to update the CMakeLists.txt first. This is not a good idea/practice!
To fix this, I updated the Makefile of RPLidar SDK as follow:
.
.
.
RPLIDAR_RELEASE_LIB := $(HOME_TREE)/output/Linux/Release/librplidar_sdk.a
install: $(RPLIDAR_RELEASE_LIB)
install -d $(DESTDIR)/usr/local/lib/rplidar/Release/
install -m 644 $(RPLIDAR_RELEASE_LIB) $(DESTDIR)/usr/local/lib/rplidar/Release/
RPLIDAR_DEBUG_LIB := $(HOME_TREE)/output/Linux/Debug/librplidar_sdk.a
install: $(RPLIDAR_DEBUG_LIB)
install -d $(DESTDIR)/usr/local/lib/rplidar/Debug/
install -m 644 $(RPLIDAR_DEBUG_LIB) $(DESTDIR)/usr/local/lib/rplidar/Debug/
RPLIDAR_HEADERS := $(HOME_TREE)/sdk/include
install: $(RPLIDAR_HEADERS)
install -d $(DESTDIR)/usr/local/include/rplidar/
cp -r $(RPLIDAR_HEADERS)/* $(DESTDIR)/usr/local/include/rplidar/
RPLIDAR_HEADERS_HAL := $(HOME_TREE)/sdk/src/hal
install: $(RPLIDAR_HEADERS_HAL)
install -d $(DESTDIR)/usr/local/include/rplidar/
cp -r $(RPLIDAR_HEADERS_HAL) $(DESTDIR)/usr/local/include/rplidar/
Due to this update, now, I can run sudo make install which basically copies the header files of RPLidar SDK from the local directory to /usr/local/rplidar/ directory. It also copies the lib file to /usr/local/lib/rplidar/<Debug> or <Release>/ directory.
Now, in my local project, I updated the CMakeLists.txt to as follow:
cmake_minimum_required(VERSION 3.1.0 FATAL_ERROR)
project(<project_name>)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED TRUE)
SET(CMAKE_CXX_FLAGS -pthread)
include_directories(include)
add_executable(${PROJECT_NAME} src/main.cpp src/another_src_file.cpp)
find_package(rplidar REQUIRED)
include_directories(${rplidar_INCLUDE_DIRS})
link_directories(${rplidar_LIBRARY_DIRS})
target_link_libraries(${PROJECT_NAME} ${rplidar_LIBRARY})
However, upon running cmake .. command, I'm getting this error:
.
.
.
CMake Error at CMakeLists.txt:12 (find_package):
By not providing "Findrplidar.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "rplidar", but
CMake did not find one.
Could not find a package configuration file provided by "rplidar" with any
of the following names:
rplidarConfig.cmake
rplidar-config.cmake
Add the installation prefix of "rplidar" to CMAKE_PREFIX_PATH or set
"rplidar_DIR" to a directory containing one of the above files. If
"rplidar" provides a separate development package or SDK, be sure it has
been installed.
-- Configuring incomplete, errors occurred!
As far as I know, RPLidar SDK does not have rplidarConfig.cmake or rplidar-config.cmake file.
How can I fix this error?
Rants from my soul:
It sucks when you have to use any library foo when the author fails to provide a foo-config.cmake for you to use easily by invoking find_package(foo). It's absolutely outrageous when a reasonably modern project still uses hand written Makefiles as its build system. I myself is stuck with a much worse constructed SDK than yours right now.
Short answer:
Since the author of the SDK fails to provide a config file to support your cmake usage, if you still insists on invoking find_package on the library (and you should!), you are required to write your own Module file to clean up their mess. (Yeah, you are doing the work for the library authors).
To truly achieve cross platform usage, you should write a Findrplidar.cmake module file to find the libraries for you.
To write a reasonable module file, you would most likely use API find_path for header files and find_library for libs. You should check out its docs and try using them, and maybe Google a few tutorials.
Here is my version of Findglog.cmake for your reference. (glog authors have updated their code and supports Config mode. Unfortunately, Ubuntu build doesn't use it, so I still have to write my own file)
find_path(glog_INCLUDE_DIR glog/logging.h)
message(STATUS "glog header found at: ${glog_INCLUDE_DIR}")
find_library(glog_LIB glog)
message(STATUS "libglog found at: ${glog_LIB}")
mark_as_advanced(glog_INCLUDE_DIR glog_LIB)
include(FindPackageHandleStandardArgs)
find_package_handle_standard_args(glog REQUIRED_VARS
glog_INCLUDE_DIR
glog_LIB
)
if(glog_FOUND AND NOT TARGET glog::glog)
add_library(glog::glog SHARED IMPORTED)
set_target_properties(glog::glog PROPERTIES
IMPORTED_LINK_INTERFACE_LANGUAGES "CXX"
IMPORTED_LOCATION "${glog_LIB}"
INTERFACE_INCLUDE_DIRECTORIES
"${glog_INCLUDE_DIR}"
)
endif()
And you can use it like this:
find_package(glog)
target_link_libraries(main PRIVATE glog::glog)
Long answer:
The history of developers using cmake is an absolute nightmare. The internet is filled with bad practice/examples of how not to use cmake in your project, including the old official cmake tutorial (which still might be there). Mostly because no one really gives a **** (If I can build my project, who cares if it's cross platform). Another valid reason is that cmake documentations are really daunting to beginners.
This is why I am writing my own answer here, lest you get misguided by Googling elsewhere.
The nightmare is no more. The wait has ended. "The Messiah" of cmake (source) is come. He bringeth hope to asm/C/C++/CUDA projects written in 2020 and on. Here is The Word.
The link above points to the only way how cmake projects should be written and truly achieve cross platform once and for all. Note the material is not easy at all to follow for beginners. I myself spent an entire week to fully grasp what was covered in The Word, when I had become somewhat familiar with cmake concepts at the time (but lost in my old sinful ways).
The so-called "long answer" is actually shorter. It's just a pointer to the real answer. Good luck reading the Word. Embrace the Word, for anything against that is pure heresy.
Response of comment 1-5:
Good questions. A lot of those can be obtained from the Word. But the word is better digested when you become more familiar with CMake. Let me answer them in decreasing of relevance to your problem at hand.
For the ease of discussion, I'll just use libfoo as an example.
Let's say you always wants to use libfoo like this:
find_package(foo)
target_link_libraries(your_exe ... foo::foo)
Pretend foo is installed at the following location:
- /home/dev/libfoo-dev/
- include
- foo
- foo.h
- bar.h
- ...
- lib
- libfoo.so
- share
- foo/foo-config.cmake # This may or may not exist. See discussion.
Q: Only one .h file. Why?
A: Because in the case of libfoo (also true for glog), only one search of header location is necessary. Just like the example from libfoo,
where foo/foo.h and foo/bar.h are at the same location. So their output of find_path would be the same: /home/dev/libfoo-dev/include.
Q: Why I'm getting NOTFOUND for my headers and libs?
A: The function find_path and find_library only search locations specify in the documentations. By default they search system locations,
like /usr/include and /usr/lib respectively. Refer to the official docs for details on system locations. In the case of libfoo, however,
they reside in /home/dev/libfoo-dev. So you must specify these locations in cmake variable CMAKE_PREFIX_PATH. It's a ; seperated string.
One would do cmake -D CMAKE_PREFIX_PATH="/home/dev/libfoo-dev;/more/path/for/other/libs/;...;/even/more/path" .... on the command line.
One very important note: unlike Unix command find, find_path will only search specific paths inside /home/dev/libfoo-dev, not all the way down:
include (usually also include/{arch} where {arch} is sth like x86_64-linux-gnu for x86 Linux) for find_path; lib variant for find_library,
respectively. Unusual locations would require passing in more arguments, which is uncommon and most likely unnecessary.
For this very reason, for libfoo, calling find_path(... foo.h ...) is undesired. One would want find_path(... foo/foo.h ...). Refer to the docs
for more details. You can also try out yourself.
Also for this reason, it is desirable to organize libraries in the usual bin include lib share quad on Unix-like systems. I'm not familiar with Windows.
Q: Debug & Release
A: There are several options. The easiest one might be:
Prepare rplidar debug and release build in two different folders, /path/to/debug & /path/to/release for instance
Passing to Debug & Release build respectively (cmake -D CMAKE_PREFIX_PATH="/path/to/debugORrelease" ....)
There are definitely others ways, but perhaps requires special care in your Findrplidar.cmake script (maybe some if statements).
Q: Why glog::glog rather than glog?
A: It's just modern cmake practice, with small benefits. Not important right now. Refer to the Word if you are interested.
Q: You mentioned that you are writing rplidarConfig.cmake. Instead you should rename the file to Findrplidar.cmake.
A: CMake philosophy is as such:
Library authors should write foo-config.cmake or fooConfig.cmake
When they fail to provide one, it sucks. And according to the Messiah, it should be reported as a bug.
In this case, you as library user, should write Findfoo.cmake by guessing how to describe the dependencies for libfoo. For simple libraries, this is not so bad. For complex ones, like Boost, this sucks!
A few side note on this topic:
Note how Findfoo.cmake is written by library users, from guessing.
This is insane! Users shouldn't do this. This is the authors' fault, to put their users in this uncomfortable situation.
A foo-config.cmake file is extremely easy to write for authors of libfoo, IF they follow the Word exactly.
Extremely easy for the authors in the sense that: cmake can take care of everything. It will generate scripts automatically for the authors to use in their foo-config.cmake file.
Guaranteed to be cross-platform and easy to use by the users, thanks to cmake.
However, the reality sucks. Now you have to write Findfoo.cmake
Q: Why only find_package & target_link_libraries?
A: This is what the Word says. It's therefore good practice. Why the Word says so is something you have to find out yourself.
It's not possible for me to explain the gist of the Word in this answer, nor would it be convincing to you. I would just say the following:
It's very easy to write spaghetti CMakeLists that are next to impossible to maintain. The spirit of the Word helps you avoid that by
forcing you to carefully think about:
library structure: public vs private headers, for example. This makes you think about what to include in your headers and public APIs.
build specification: what is necessary to build a library you write (what to include; what to link)
usage requirement: what is necessary for others to use a library you write (what to include; what to link)
dependencies: what is the relationship of the library you write & its dependencies
Maybe more
If you think about it, these aspects are crucial to writing a cross-platform and maintainable library.
include_directories, link_directories and add_definitions are all very bad practice
(according to lots of sources, including the official documentations of these APIs). Bad practice tends to obscure the aspects above,
and causes problems later on when everything gets integrate together as a whole. E.g. include_directories will add -I to compiler for every
target written in the directory of that CMakeLists.txt. Read this sentence a few times and Spock will tell you it's illogical.
Don't worry. It's okay for now to use them when you are not familiar with the Word (Why else would this be in the last section). Once you know the Word, refactor your CMakeLists when you have time. Bad practice might cause problem later on, when your project becomes more complex. (In my professional experience, 5 very small groups of people is enough to cause a nightmare. By nightmare I mean hard code everything in CMakeLists; Create a git branch for every single different platform/device/environment; Fixing a bug meaning to cherry-pick one commit to each branch. I've been there before knowing the Word.)
The practice of the Word very well utilize the philosophy of modern CMake, to encapsulate build specifications and usage requirements inside
CMake targets. So when target_link_libraries is called, these properties gets propagated properly.
I am confused on the right way to get an external library integrated into my own Cmake project (This external project needs to be built along with my project, it's not installed separately, so we can't use find_library, or so I think)
Let's assume we have a project structure like this (simplified for this post):
my_proj/
--CMakeLists.txt
--src/
+---CMakeLists.txt
+---my_server.cpp
That is, we have a master CMakeLists.txt that basically sits at root and invokes CMakeLists for sub directories. Obviously, in this example, because its simplified, I'm not showing all the other files/directories.
I now want to include another C++ GitHub project in my build, which happens to be this C++ bycrypt implementation: https://github.com/trusch/libbcrypt
My goal:
While building my_server.cpp via its make process, I'd like to include the header files for bcrypt and link with its library.
What I've done so far:
- I added a git module for this external library at my project root:
[submodule "third_party/bcrypt"]
path = third_party/bcrypt
url = https://github.com/trusch/libbcrypt
So now, when I checkout my project and do a submodule update, it pulls down bcrypt to ${PROJ_ROOT}/third_party
Next up, I added this to my ROOT CMakeLists.txt
# Process subdirectories
add_subdirectory(third_party/bcrypt)
add_subdirectory(src/)
Great. I know see when I invoke cmake from root, it builds bcrypt inside third_party. And then it builds my src/ directory. The reason I do this is I assume this is the best way to make sure the bcrypt library is ready before my src directory is built.
Questions:
a) Now how do I correctly get the include header path and the library location of this built library into the CMakeLists.txt file inside src/ ? Should I be hardcoding #include "../third_party/bcrypt/include/bcrypt/bcrypt.h" into my_server.cpp and -L ../third_party/libcrypt.so into src/CMakeLists.txt or is there a better way? This is what I've done today and it works, but it looks odd
I have, in src/CMakeLists.txt
set(BCRYPT_LIB,"../third_party/bcrypt/libbcrypt.so")
target_link_libraries(my app ${MY_OTHERLIBS} ${BCRYPT_LIB})
b) Is my approach of relying on sequence of add_directory correct?
Thank you.
The best approach depends on what the bcrypt CMake files are providing you, but it sounds like you want to use find_package, rather than hard-coding the paths. Check out this answer, but there are a few different configurations for find_package: MODULE and CONFIG mode.
If bcrypt builds, and one of the following files gets created for you:
FindBcrypt.cmake
bcrypt-config.cmake
BcryptConfig.cmake
that might give you an idea for which find_package configuration to use. I suggest you check out the documentation for find_package, and look closely at how the search procedure is set up to determine how CMake is searching for bcrypt.
I do already ask a quiet similar question but in fact I now change my mind.
Id like like to compile proftpd and add a copy of the library it uses to the choosen installation directory.
Let's say I define a prefix in my compilation like:
/usr/local/proftpd
Under this directory I would like to find and use those directories only :
./lib
./usr/lib
./usr/bin
./usr/.....
./etc
./var/log/proftpd
./bin
./sbin
./and others I will not put the whole list
So the idea is after I have all libraries and config file in my main directory I could tar it and send it on another server with the same OS and without installing all the dependencies of protfpd I could use it.
I know it does sound like a windows installer not using shared library but that's in fact exactly what I'm trying to accomplish.
So far I have manage to compile it on AIX using this command line:
./configure --with-modules=mod_tls:mod_sql:mod_sql_mysql:mod_sql_passwd:mod_sftp:mod_sftp_sql --without-getopt --enable-openssl --with-includes=/opt/freeware/include:/opt/freeware/include/mysql/mysql/:/home/poney2/src_proftpd/libmath_header/ --with-libraries=/opt/freeware/lib:/opt/freeware/lib/mysql/mysql/:/home/poney2/src_proftpd/libmath_lib --prefix=/home/poney/proftpd_bin --exec-prefix=/home/poney/proftpd_bin/proftpd
Before trying to ask me why I'm doing so, it's because I have to compile proftpd on IBM AIX with almost all modules and this is not available on the IBM rpm binary repositories.
The use of this LDFLAG
LDFLAGS="-Wl,-blibpath:/a/new/lib/path"
where /a/new/lib/path contains all your library does work with Xlc and Gcc compiler.
I'm currently working to upgrade a set of c++ binaries that each use their own set of Makefiles to something more modern based off of Autotools. However I can't figure out how to include a third party library (eg. the Oracle Instant Client) into the build/packaging process.
Is this something really simple that I've missed?
Edit to add more detail
My current build environment looks like the following:
/src
/lib
/libfoo
... source and header files
Makefile
/oci #Oracle Instant Client
... header and shared libraries
Makefile
/bin
/bar
... source and header files
Makefile
Makefile
/build
/bin
/lib
build.sh
Today the top level build.sh does the following steps:
Runs each lib's Makefile and copies the output to /build/lib
Runs each binary's Makefile and copied the output to /build/bin
Each Makefile has a set of hardcoded paths to the various sibling directories. Needless to say this has become a nightmare to maintain. I have started testing out autotools but where I am stuck is figuring out the equivalent to copying /src/lib/oci/*.so to /build/lib for compile time linking and bundling into a distribution.
I figured out how to make this happen.
First I switched to a non recursive make.
Next I made the following changes to configure.am as per this page http://www.openismus.com/documents/linux/using_libraries/using_libraries
AC_ARG_WITH([oci-include-path],
[AS_HELP_STRING([--with-oci-include-path],
[location of the oci headers, defaults to lib/oci])],
[OCI_CFLAGS="-$withval"],
[OCI_CFLAGS="-Ilib/oci"])
AC_SUBST([OCI_CFLAGS])
AC_ARG_WITH([oci-lib-path],
[AS_HELP_STRING([--with-oci-lib-path],
[location of the oci libraries, defaults to lib/oci])],
[OCI_LIBS="-L$withval -lclntsh -lnnz11"],
[OCI_LIBS='-L./lib/oci -lclntsh -lnnz11'])
AC_SUBST([OCI_LIBS])
In the Makefile.am you then use the following lines (assuming a binary named foo)
foo_CPPFLAGS = $(OCI_CFLAGS)
foo_LDADD = libnavycommon.la $(OCI_LIBS)
ocidir = $(libdir)
oci_DATA = lib/oci/libclntsh.so.11.1 \
lib/oci/libnnz11.so \
lib/oci/libocci.so.11.1 \
lib/oci/libociicus.so \
lib/oci/libocijdbc11.so
The autotools are not a package management system, and attempting to put that type of functionality in is a bad idea. Rather than incorporating the third party library into your distribution, you should simply have the configure script check for its existence and abort if the required library is not available. The onus is on the user to satisfy the dependency. You can then release a binary package that will allow the user to use the package management system to simplify dependency resolution.
I have been struggling with this for quite a while, and my adventures with cmake have only resulted in hackish solutions that I am pretty sure are not correct.
I created a library that consists of several files, as follows:
-libfolder
-codepart1folder
-CMakeLists.txt
-codepart1.cpp
-codepart1.hpp
-codepart2folder
-codepart3folder
-lib.cpp
-lib.hpp
-CMakeLists.txt
I wrote a CMakeLists file to compile the library (after some experimentation), and I can generate a lib.a file. Now I would like to include this code as a library in other projects, and access it through the interface in lib.hpp. What is the best way to do this, in terms of directory structure, and what I need to put into CMakeLists.txt in my root project?
My current attempt has been to add -libfolder as a subfolder to my current project, and add the commands:
include_directories(${PROJECT_SOURCE_DIR}/libfolder)
link_directories(${PROJECT_BINARY_DIR}/libfolder)
add_subdirectory(libfolder)
target_link_libraries(project lib)
When I run make, the library compiles fine, but when project.cpp compiles, it complains that it cannot find codepart1.hpp (which is included in lib.hpp, included from project.cpp).
I suspect that this is the wrong way about doing this, but I cannot wade through the CMake documentation and find a good tutorial on setting up projects like this. Please help, CMake gurus!
The clean way to import one CMake project into another is via the find_package command. The package declaration is done by using the export command. An advantage of using find_package is that it eliminates the need to hard-code paths to the package's files.
Regarding the missing hpp file, you didn't include codepart1folder, so it's not on the include path.
Ok, so after consulting a coworker of mine who is a CMake guru, it seems CMake does not have support for what I am trying to do, leaving one with 3 options:
Add all of the dependencies to the parent projects CMakeLists.txt - not very clean, but it will get the thing to work. You'll have to do this for every project you add the code to, and go back and fix things if your library changes.
clean up your library headers. This is done through some compiler hackery. The idea is to forward-declare every class, and use only pointers or boost::shared_ptr, and then include the dependencies only in the cpp file. That way you can build the cpp file using all the findpackage stuff, and you get the bonus of being able to use the lib by only including the header and linking to the library.
Look into build systems. Having portable code and fast code compilation with complex dependencies is not a solved problem! From my investigations it turned out to be quite complicated. I ended up adopting my coworkers build system which he created himself in cmake, using things he picked up from Google.
Looking at your post you don't seem to add 'codepart1folder' to the includes anywhere. How are you including codepart1.hpp as:
#include <codepart1.hpp>
#include "codepart1folder/codepart1.hpp"
I don't think there is a standard accepted way to structure cmake projects. I've looked at a bunch of cmake repos and they tend to have differences. Personally I do the following:
-project
CMakeLists.txt
-build
-cmake
OptionalCmakeModule.cmake
-src
-Main
Main.cpp
Main.hpp
-DataStructs
SomeTree.hpp
SomeObject.hpp
-Debug
Debug.hpp
-UI
Window.hpp
Window.cpp
Basically that dumps all the source code into 1 directory, then you perform an out of source build with: 'mkdir build && cd build && cmake .. && make' in the projects root folder.
If you have separate libs as part of your project, then you might want a separate libs directory with another subfolder for your specific lib.
I have some of my repos on: https://github.com/dcbishop/ if you want to look at the CMakeLists.txt files.
The main problems with my project structure are that I use the FILE_GLOB which is apparently the 'wrong' way to do things (if you add files after running 'cmake ..' then they won't be picked up hen you do a 'make'). I haven't figured out what the 'right' way to do it is (from what I can see it involves keeping a separate list of files) I also only use 1 CMakeLists.txt file.
Some projects also choose to separate their cpp and hpp files into separate directories. So you would have an include and src folders (at least for the hpp files that are intended to be used externally). I think that would mainly be for projects that are mainly large libraries. Would also make installing header files much easier.
You are probably missing
include_directories(${PROJECT_SOURCE_DIR}/libfolder/codepart1folder)
In such a case you might want to set( CMAKE_INCLUDE_CURRENT_DIR on) to add all folders to the include directory path variable.
Check cmake's output on the command line whether the correct include folders are set or not. Additionally you can always use message() as "print debugging" for cmake variables.
In case of include directories however you need to read the directory property to see what is actually in the include directories.
get_property(inc_dirs DIRECTORY PROPERTY INCLUDE_DIRECTORIES)
message("inc_dirs = ${inc_dirs}")
I hope this helps you figuring out what is missing.
Edit
I just saw your comment about added codepart1folder in the libfolder. It is only available in the libfolder's include_directory path and not propagated to the root folder.
Since the include codepart1.hpp is present in the lib.hpp however you need to have it also available in the project path otherwise you will get missing declaration errors when you build your project.