Short version (TL;DR)
Using CMake with the PkgConfig module I try to print all the include directories for a library I need for my project, let's call it thelibrary. I do this by printing the value of the THELIBRARY_INCLUDE_DIRS variable. The list of include directories is printed, but one directory is missing.
This really surprised it, as if I type the following on a terminal:
pkg-config thelibrary --cflags
then all the include directories are printed, including the one missing from THELIBRARY_INCLUDE_DIRS.
The same happens if I try to print the value of THELIBRARY_CFLAGS.
How is this possible? Where is the mistake? (Hoping it is a mistake of mine)
Details
Context
I am working on a C++ Cmake project which is compiled correctly under Mac OS but not under Linux (Ubuntu), and when investigating on the possible reasons I found the issue below.
Problem
I included the PkgConfig module in file CMakeLists.txt by writing:
find_package(PkgConfig REQUIRED)
After that, in order to use the library, I set the PKG_CONFIG_PATH environment variable by writing:
set(ENV{PKG_CONFIG_PATH} "/usr/local/opt/thelibrary/pkgconfig:$ENV{PKG_CONFIG_PATH}")
Finally, I put this instruction to search the library.
pkg_check_modules(THELIBRARY thelibrary)
Finally, in order to debug, I have tried to print the list of included directories by writing:
MESSAGE( STATUS "THELIBRARY DIRS: " ${THELIBRARY_INCLUDE_DIRS} )
The problem is, it worked, in fact it printed a list of include directories, but the problem is that the most important one is missing. In fact, the output is something like:
/opt/auxlib1/include/opt/auxlib2/include ...
where auxlib1, auxlib2 etc. are auxiliary include directories I need to compile.
But I also expected the dir /opt/thelibrary/include in output, which is printed if I use the pkg-config command on the terminal as said at the beginning. How is it possible that only that directory is not printed?
Thanks to the suggestion by Tsyvarev (deleting CMakeCache.txt), I understood what the cause was, and since this may happen to other people, I hope this explanation helps.
The reason why the directory /opt/thelibrary/include wasn't appearing in THELIBRARY_INCLUDE_DIRS is that file CMakeCache.txt contained information about a previous version of the library, whose header files were installed in a directory with a slightly different name, which was removed when that previous version was uninstalled.
Since that directory does not exist anymore, then it doesn't appear in THELIBRARY_INCLUDE_DIRS, and the new directory does not appear simply because the file CmakeCache.txt was still pointing to the previous version.
Solution: delete CmakeCache.txt and re-run cmake.
Related
I have recently installed CMake in order to write code to make use of Libbitcoin in C++ but I am having a hard time, I was trying to build the example code on GitHub here. And it haters been going terribly. I can't manage to link the library right in CMake, here is my code. I read and people were saying that I should try Autoconf but I have no idea how to even start that as I know nothing about Autoconf. I have CMake 3.16, and installed Libbitcoin with brew but alias were made in /usr/local/include for the library, I am on Mac OS X 10.15. The CMake runs fine but when running "make", it responds with:
Scanning dependencies of target CreateAddr
main.cxx:1:10: fatal error: bitcoin/bitcoin.hpp: No such file or directory
1 | #include <bitcoin/bitcoin.hpp>
| ^~~~~~~~~~~~~~~~~~~~~
Here is my CMake text:
Please all help is appreciated I am beyond lost.
It is hard to be sure without knowing the specifics of your installation, but it appears that your include directory paths may be overlapping with what is specified for the header in main.cxx. The include_directories() call tells the compiler to include headers from this directory:
/usr/local/include/bitcoin
Then, in main.cxx, you're including the file with bitcoin/bitcoin.hpp. Combining these suggests the file is located here:
/usr/local/include/bitcoin/bitcoin/bitcoin.hpp
The error states the header could not be found, so perhaps you meant to locate it here:
/usr/local/include/bitcoin/bitcoin.hpp
In that case, just remove the relative directory path from the main.cxx file, like this:
#include <bitcoin.hpp>
Also, you want to link to your libbitcoin library correctly. Using link_directories() is not recommended. Instead, you can specify the full path to your libbitcoin library directly in the call to target_link_libraries(). The library may not be located in /usr/local/include/bitcoin. With these changes, the last few lines in your CMake would look something more like this:
include_directories(/usr/local/include/bitcoin)
add_executable(CreateAddr main.cxx)
target_link_libraries(CreateAddr PUBLIC /your/path/to/libs/libbitcoin.so)
I added the xgboost library as a git submodule of my project and I'm trying to add it to cmake as a subdirectory. Unfortunately it's not working. A simple hello world project with the following CMakeLists.txt replicates the error that I'm getting.
cmake_minimum_required(VERSION 3.2)
project(foo)
add_subdirectory(xgboost)
add_executable(${PROJECT_NAME} foo.cpp)
target_link_libraries(${PROJECT_NAME} xgboost)
After building the library there is nothing in the xgboost/lib directory so I get the following error.
clang: error: no such file or directory:
'/Users/.../myproject/xgboost/lib/libxgboost.dylib'
I think that the problem is generated in their CMakeLists file since they have two different targets. Maybe cmake is choosing the wrong target but I'm not familiar enough with cmake to figure it out. The following code is from xgboost's CMakeLists.
# Executable
add_executable(runxgboost $<TARGET_OBJECTS:objxgboost> src/cli_main.cc)
set_target_properties(runxgboost PROPERTIES
OUTPUT_NAME xgboost
)
set_output_directory(runxgboost ${PROJECT_SOURCE_DIR})
target_link_libraries(runxgboost ${LINK_LIBRARIES})
# Shared library
add_library(xgboost SHARED $<TARGET_OBJECTS:objxgboost>)
target_link_libraries(xgboost ${LINK_LIBRARIES})
set_output_directory(xgboost ${PROJECT_SOURCE_DIR}/lib)
#Ensure these two targets do not build simultaneously, as they produce outputs with conflicting names
add_dependencies(xgboost runxgboost)
My questions in order of importance are:
Is there any way to fix it without modifying xgboost's CMakeLists.txt file?
Is it reasonable to try to add xgboost to my project as a git submodule?
Is there any reason cmake is not instructing to build the library?
Note: There were several edits to this question since I tried to narrow down the problem and to provide more information.
(I would love to ask for few things beforehand in the comment section, but I have too low reputation to do so, so I will just give it a shot ;))
I have few suspects, and one of them is ${CMAKE_SOURCE_DIR} of the submodule's root CMakeLists.txt. Although the paths are set properly when you run that CMakeLists.txt alone, cmake gets confused the moment you add it as your subdirectory. Have you looked into another directories for your output binaries?
First I would suggest testing this hypothesis, and then I would suggest writing similar, but separate CMakeLists.txt file for xgboost library, and then substitute it in the project temporarily. Unfortunately the CMakeLists.txt filename is hardcoded and there is no possibility to have two files of that kind in one directory; so it seems that the answer to 1) is, that you rather have to change the file.
For the 2): as long as it does not require huge additional logic in your CMakeLists.txt, it makes sense. Other viable option is to create an install target, which you can use to install your xgboost library locally (using CMAKE_INSTALL_PREFIX(doc) variable), and then add the installation path to your CMAKE_LIBRARY_PATH(doc).
Whenever I wanted to include a directory that was located outside of my project with Clion I would use the -I somedir flag. This time however, what I want to do is to have a hierarchy like this:
/project
CMakeLists.txt
/src
/Graph
Graph.h
Graph.cpp
/Dijkstra
Dijkstra.h
Dijstra.cpp
I want my code in a /src directory. And not only that, but also, for example, inside the file Dijkstra.h I want to include the Graph.h like this: #include "Graph/Graph.h and not like this: #include "../Graph/Graph.h.
If I only add an -I src flag, then if I am inside the Dijkstra.h file and I wanted to include Graph.h, I would have to write #include "../Graph/Graph.h, which is not what I want.
So I tried to also add INCLUDE_DIRECTORIES(src). That fixed the problem above, however when tried to compiled, I got a linker error undefined reference to....
So I tried adding the files one by one like this:
set(SOURCE_FILES
src/Dijkstra/Dijkstra.h
src/Dijkstra/Dijkstra.cpp
src/Graph/Graph.h
src/Graph/Graph.cpp)
add_executable(someprojectname ${SOURCE_FILES})
and that brought back the previous problem, where I had to include the files like this: #include "../Graph/Graph.h".
How can I do this properly to get the behavior I want ?
Command INCLUDE_DIRECTORIES doesn't add any source file for compile!
Instead, this command defines directories for search header files.
You need to list all source files in add_executable() call in any case:
include_directories(src)
set(SOURCE_FILES
src/Dijkstra/Dijkstra.cpp
src/Graph/Graph.cpp)
add_executable(someprojectname ${SOURCE_FILES})
UPDATE: #Tsyvarev's answer is correct. I've edited this answer to remove the incorrect part and keep the comments relating to target_include_directories(), but it should be viewed as additional to Tsyvarev's answer.
INCLUDE_DIRECTORIES(src) will make the src directory get added as a search path to all targets defined from that point on. It does not add sources to any targets. The search path will be relative to the current source directory and CMake will adjust it as appropriate when descending into subdirectories via add_subdirectory(). While this is fine if that's what you want, as the project gets bigger and more complicated, you may find you would prefer to apply the include path settings to just some targets. For that, use target_include_directories() instead:
target_include_directories(someprojectname "${CMAKE_CURRENT_SOURCE_DIR}/src")
This will have the same effect, but it restricts the use of the added include path to just the someprojectname target. If you later define some other target which doesn't need the include path, it won't be added. This can help prevent situations like unexpected files being picked up if you have deep directory hierarchies and you re-use directory names in different places, for example).
The target_include_directories() command has additional benefits when applied to library targets because CMake has the ability to carry the include path through to anything you link against that library too. Doesn't sound like much, but for large projects which define and link many libraries, it can be a huge help. There are other target-specific commands which have similar benefits too. This article may give you a bit of a feel for what is possible (disclaimer: I wrote the article). It is more focused on target_sources(), but the discussion around carrying dependencies through to other targets may be useful.
I have been struggling with this for quite a while, and my adventures with cmake have only resulted in hackish solutions that I am pretty sure are not correct.
I created a library that consists of several files, as follows:
-libfolder
-codepart1folder
-CMakeLists.txt
-codepart1.cpp
-codepart1.hpp
-codepart2folder
-codepart3folder
-lib.cpp
-lib.hpp
-CMakeLists.txt
I wrote a CMakeLists file to compile the library (after some experimentation), and I can generate a lib.a file. Now I would like to include this code as a library in other projects, and access it through the interface in lib.hpp. What is the best way to do this, in terms of directory structure, and what I need to put into CMakeLists.txt in my root project?
My current attempt has been to add -libfolder as a subfolder to my current project, and add the commands:
include_directories(${PROJECT_SOURCE_DIR}/libfolder)
link_directories(${PROJECT_BINARY_DIR}/libfolder)
add_subdirectory(libfolder)
target_link_libraries(project lib)
When I run make, the library compiles fine, but when project.cpp compiles, it complains that it cannot find codepart1.hpp (which is included in lib.hpp, included from project.cpp).
I suspect that this is the wrong way about doing this, but I cannot wade through the CMake documentation and find a good tutorial on setting up projects like this. Please help, CMake gurus!
The clean way to import one CMake project into another is via the find_package command. The package declaration is done by using the export command. An advantage of using find_package is that it eliminates the need to hard-code paths to the package's files.
Regarding the missing hpp file, you didn't include codepart1folder, so it's not on the include path.
Ok, so after consulting a coworker of mine who is a CMake guru, it seems CMake does not have support for what I am trying to do, leaving one with 3 options:
Add all of the dependencies to the parent projects CMakeLists.txt - not very clean, but it will get the thing to work. You'll have to do this for every project you add the code to, and go back and fix things if your library changes.
clean up your library headers. This is done through some compiler hackery. The idea is to forward-declare every class, and use only pointers or boost::shared_ptr, and then include the dependencies only in the cpp file. That way you can build the cpp file using all the findpackage stuff, and you get the bonus of being able to use the lib by only including the header and linking to the library.
Look into build systems. Having portable code and fast code compilation with complex dependencies is not a solved problem! From my investigations it turned out to be quite complicated. I ended up adopting my coworkers build system which he created himself in cmake, using things he picked up from Google.
Looking at your post you don't seem to add 'codepart1folder' to the includes anywhere. How are you including codepart1.hpp as:
#include <codepart1.hpp>
#include "codepart1folder/codepart1.hpp"
I don't think there is a standard accepted way to structure cmake projects. I've looked at a bunch of cmake repos and they tend to have differences. Personally I do the following:
-project
CMakeLists.txt
-build
-cmake
OptionalCmakeModule.cmake
-src
-Main
Main.cpp
Main.hpp
-DataStructs
SomeTree.hpp
SomeObject.hpp
-Debug
Debug.hpp
-UI
Window.hpp
Window.cpp
Basically that dumps all the source code into 1 directory, then you perform an out of source build with: 'mkdir build && cd build && cmake .. && make' in the projects root folder.
If you have separate libs as part of your project, then you might want a separate libs directory with another subfolder for your specific lib.
I have some of my repos on: https://github.com/dcbishop/ if you want to look at the CMakeLists.txt files.
The main problems with my project structure are that I use the FILE_GLOB which is apparently the 'wrong' way to do things (if you add files after running 'cmake ..' then they won't be picked up hen you do a 'make'). I haven't figured out what the 'right' way to do it is (from what I can see it involves keeping a separate list of files) I also only use 1 CMakeLists.txt file.
Some projects also choose to separate their cpp and hpp files into separate directories. So you would have an include and src folders (at least for the hpp files that are intended to be used externally). I think that would mainly be for projects that are mainly large libraries. Would also make installing header files much easier.
You are probably missing
include_directories(${PROJECT_SOURCE_DIR}/libfolder/codepart1folder)
In such a case you might want to set( CMAKE_INCLUDE_CURRENT_DIR on) to add all folders to the include directory path variable.
Check cmake's output on the command line whether the correct include folders are set or not. Additionally you can always use message() as "print debugging" for cmake variables.
In case of include directories however you need to read the directory property to see what is actually in the include directories.
get_property(inc_dirs DIRECTORY PROPERTY INCLUDE_DIRECTORIES)
message("inc_dirs = ${inc_dirs}")
I hope this helps you figuring out what is missing.
Edit
I just saw your comment about added codepart1folder in the libfolder. It is only available in the libfolder's include_directory path and not propagated to the root folder.
Since the include codepart1.hpp is present in the lib.hpp however you need to have it also available in the project path otherwise you will get missing declaration errors when you build your project.
I am using CMake on a small C++ project and so far it works great... with one twist :x
When I change a header file, it typically requires recompiling a number of sources files (those which include it, directly or indirectly), however it seems that cmake only detects some of the source files to be recompiled, leading to a corrupted state. I can work around this by wiping out the project and rebuilding from scratch, but this circumvents the goal of using a make utility: only recompiling what is needed.
Therefore, I suppose I am doing something wrong.
My project is very simply organized:
a top directory where all resources sit, the main CMakeLists.txt sits there
a "include" directory where all public headers lies (in various subdirectories)
a "src" directory where all the subdirectories for sources files are, the src CMakeLists.txt sits there
a CMakeLists.txt per subdirectory in the "src" directory
The main directory has:
cmake_minimum_required(VERSION 2.8)
project(FOO)
set(EXECUTABLE_OUTPUT_PATH ${CMAKE_BINARY_DIR}/bin)
# Compiler Options
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -g -std=c++0x -Wall -Wextra -Werror")
include_directories($(FOO_SOURCE_DIR)/include)
add_subdirectory(src)
The "src" directory:
add_subdirectory(sub1)
add_subdirectory(sub2)
add_subdirectory(sub3)
add_subdirectory(sub4)
add_executable(foo main.cpp)
target_link_libraries(foo sub1 sub2 sub3 sub4)
Where sub4 depends on sub3 which depends on sub2 which depends on sub1
And an example of a subdirectory (sub3):
set(SUB3_SRCS
File1.cpp
File2.cpp
File3.cpp
File4.cpp
File5.cpp
File6.cpp
)
add_library(sub3 ${SUB3_SRCS})
target_link_libraries(sub3 sub1 sub2)
I'd be glad if anyone could point my mistake to me, searching here or on CMake didn't yield anything so I guess it's very easy or should work out of the box...
(for reference, I am using cmake version 2.8.2 on MSYS)
EDIT:
Thanks to Bill's suggestion I have checked the depend.make file generated by CMake, and it is indeed lacking (severely). Here is an example:
src/sub3/CMakeFiles/sub3.dir/File1.cpp.obj: ../src/sub3/File1.cpp
Yep, that's all, none of the includes were referenced at all :x
You should look at the depend.make files in your binary tree. It will be in CMakeFiles/target.dir/depend.make. Try to find one of those that is missing a .h file that you think it should have. Then create a bug report for cmake or email the cmake mailing list.
I just hit the same issue. After changing paths in include_directories() from absolute to relative it added appropriate dependencies.
Looks like CMake tries to guess which headers are system and which are project related. I suspect that directories that starts with / passed as -isystem /some/path and thus are not presented in generated dependencies.
If you can't replace ${FOO_SOURCE_DIR} with relative path you can try to calculate relative path using appropriate CMake functions. I.e.:
file(RELATIVE_PATH FOO_SOURCE_REL_DIR
${CMAKE_CURRENT_SOURCE_DIR}
${FOO_SOURCE_DIR}/.)
include_directories(${FOO_SOURCE_REL_DIR}/include)
Did you run cmake before or after adding the includes to your cpp files?
I ran into this problem and re-running cmake fixed it. I'd added the include post-cmake.
Apparently cmake removes system include paths from the dependency trees (thank you #ony for this hint). This probably makes sense most of the time, but sometimes cmake doesn't know what the compiler thinks is a system path or not. We are using a custom gcc build that ignores /usr/include, but cmake thinks it doesn't ignore it. To force cmake to make /usr/include a dependency that is NOT optimized away, try this trick: prepend /. to the path.
I am trying to make all of the library dependencies use the cmake dependency feature, including certain "3rd" party libraries that are not always installed by default on Linux or even available. For example, Z-lib compression.
The following interface target worked fine if the Z lib includes were not in /usr/include, but would break if they were.
find_package(ZLIB REQUIRED)
message(status "found zlib ${ZLIB_LIBRARIES}")
message(status "found zlib includes ${ZLIB_INCLUDE_DIRS}")
target_link_libraries(zlib_target INTERFACE ${ZLIB_LIBRARIES})
target_include_directories(zlib_target INTERFACE ${ZLIB_INCLUDE_DIRS})
I changed the last line to
target_include_directories(zlib_target INTERFACE /.${ZLIB_INCLUDE_DIRS})
and it worked. Now targets that depended on zlib_target would automatically get -I/./usr/include during compilation.
Make sure that the include statement for the missing header is placed before the first program instruction. In my case cmake depend.make was not including the header file, because it was following the first program instruction.