I searched for the best way to do this, but I was unable to find a clear answer.
Was anyone able to build a tensorflow-serving client using cmake?
I am having difficulties with generating CPP files from proto, since they are needed for prediction service. Those proto files also include proto files from tensorflow.
so far I have come up with this:
project(serving C CXX)
find_package(Protobuf REQUIRED)
file(GLOB_RECURSE proto_files RELATIVE ${serving_SOURCE_DIR}/tensorflow/
"${serving_SOURCE_DIR}/tensorflow/*.proto")
set(PROTOBUF_GENERATE_CPP_APPEND_PATH OFF)
include_directories(${PROTOBUF_INCLUDE_DIRS})
include_directories( ${CMAKE_CURRENT_BINARY_DIR} )
PROTOBUF_GENERATE_CPP(PROTO_SRCS PROTO_HDRS ${proto_files})
add_library(tf_protos ${PROTO_SRCS} ${PROTO_HDRS})
target_link_libraries(tf_protos PUBLIC ${PROTOBUF_LIBRARIES})
Cmake builds successfully but the make command gives me an error:
No rule to make target '../tensorflow/tools/proto_text/test.proto', needed by 'tensorflow/tools/proto_text/test.pb.cc'. Stop.
To overcome the problem of .proto includes not being found I used command
set(PROTOBUF_GENERATE_CPP_APPEND_PATH OFF)
which was explained here: https://groups.google.com/forum/#!topic/protobuf/eow2fNDUHvc
My current folder structure is
serving/
CmakeLists.txt
tensorflow/
tensorflow_serving/
apis/
Folder apis contains .proto files that are needed in the client implementation and they include .proto files from the folder tensorflow.
Is this even the right way to go?
Any help/advice would be much appreciated.
I was able to get it to work in the layout you have where the CMakeLists.txt file is placed in the same level as the serving repository here. You'll need to install Tensorflow too though (using tensorflow_cc).
However, you probably don't want to muck with a fork of the official tensorflow/serving repository so I went a step further and moved the CMakeLists.txt out so you can just submodule the official repository. I made an example here
The gist is that the protobuf CMake submodule expects proto files to be laid out in the same directory from which it's called. I made some modifications to the submodules to let us call it from a level above serving and to ensure it invokes the compiler with the include paths in the right order to support the nested structure of the proto files in serving/tensorflow_serving/apis/* (and placing it accordingly in the specified build directory)
Hopefully someone else with better knowhow can make this better!
These worked for me.
https://github.com/wardsng/inception_cmake
https://github.com/FloopCZ/tensorflow_cc
You can choose a private install directory instead of the default, e.g. /usr/local/...
cmake -DCMAKE_INSTALL_PREFIX= ..
Related
I am a newbie with CMake please bear with me. I have a library (libvpop) which I created in c++ using some Boost components (system and date_time). I can link to it without a problem in windows but on Ubuntu, I am getting an error that implies the path to the boost include files cannot be found. Here is the simple CMakeLists.txt file.
cmake_minimum_required(VERSION 3.0.0)
set (Boost_DEBUG 1)
project(vpoplibuser)
find_package(fmt CONFIG REQUIRED)
find_package(Boost CONFIG REQUIRED system )
find_package(Boost CONFIG REQUIRED date_time)
add_executable(vpoplibuser vpoplibuser.cpp vpoplib.h)
find_library(VPLIB libvpop HINTS ~/projects/vpoplibuser/ )
message(STATUS "VPLib include dir: ${VPLIB}")
target_include_directories(vpoplibuser PUBLIC ${PROJECT_SOURCE_DIR} )
target_link_libraries(vpoplibuser PUBLIC ${VPLIB})
target_link_libraries(vpoplibuser PRIVATE fmt::fmt)
target_link_libraries(vpoplibuser PRIVATE Boost::system Boost::date_time)
When I run CMake, I get message:
CMake Error in CMakeLists.txt
Imported target "Boost::system" includes non-existent path "/include"
in its INTERFACE_INCLUDE_DIRECTORIES. Possible reasons include:
The path was deleted, renamed, or moved to another location.
An install or uninstall procedure did not complete successfully.
The installation package was faulty and references files it does not provide.
I have removed and reinstalled Boost. My Boost libraries are at /lib/x86_64-linux-gnu. I cannot figure out exactly where CMake is searching for the boost include file. When I inspect the _BOOST_INCLUDEDIR variable in boost_header-1.71.0/boost_headers-config.cmake it tells me _BOOST_INCLUDEDIR is "/include". I have read something about the PATH variable being an issue so I added /usr to the beginning of my PATH (there is a folder /usr/include/boost which has the boost .hpp files so I was making an assumption that is what CMake is looking for). I have been stuck on this for a couple of days so I would appreciate any advice from the expert community.
I have found a work around thanks to this article: https://github.com/VowpalWabbit/vowpal_wabbit/issues/3003
Something in the Boost cmake process is causing boost to look for the include files at /include when they are really at /usr/include. I created a symbolic link for /include to point to /usr/include and this allowed cmake to find everything. I have not solved the root cause but can move forward with this approach.
I tries to use FOREACH to generated several pb files. And make two list names PROTO_SRCS & PROTO_HDRS like below.
I can use it in the main CMakeLists. Like add_executable(a SHARED ${PROTO_SRCS} main.cpp).
But I can not use this param in subdirectories to make a library. when I type "cmake .." in main CMakelists build dir. It shown that "Cannot find source file: a.pb.cc".
main CMakeLists.txt
add_library(xxx SHARED ${PROTO_SRCS})
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/src/back back)
in src/back CMakeLists.txt
add_executable(yyy ${PROTO_SRCS})
and I can use message to show ${PROTO_SRCS} in subdir so it pass into successfully.
Please help me to point out the problem. Thx a lot
The issue is that in CMake versions older than 3.20 the GENERATED property of source files is only visible in the directory where it is set. Thus, when you add the protobuf-generated source files to a target defined in a different directory, CMake will no longer know that these are files generated during the build. Consequently, CMake will try to locate these files at configuration time, when they obviously do not exist yet.
Unfortunately, at the time of writing there is only a release candidate for CMake 3.20 and no official release yet. So depending on whether you need to coordinate with other coworkers or whether you're working on this project on your own it might not be feasible to use the release candidate.
If you can't use it, the alternative is to create an object library via add_library(protobuf_objs OBJECT ${PROTO_SRCS}) in the directory where you generate the files and to use target_sources(xxx PRIVATE $<TARGET_OBJECTS:protobuf_objs>) and target_sources(yyy PRIVATE $<TARGET_OBJECTS:protobuf_objs>) instead of adding the ${PROTO_SRCS} as source files to these targets directly.
I have a directory with c++ source and header files. I want to create a CMakeLists.txt to build this as a library for use in other CMake projects that include it as a sub directory.
Structure:
example/
foo.h
foo.cpp
CMakeLists.txt
The problem I run into is CMake doesn't seem to put foo.h anywhere, so getting the parent CMake to know how to find the header file is beguiling me.
Here's my current CMakeLists.txt:
cmake_minimum_required(VERSION 3.8.2)
project(example)
set (CMAKE_CXX_STANDARD 11)
# add library target foo
add_library(foo STATIC foo.cpp)
# tell cmake where to find headers for it
target_include_directories(foo PUBLIC .)
# sad attempt to get it to output the header
set_target_properties(foo PROPERTIES PUBLIC_HEADER foo.h)
I DON'T want to have to do install. The idea here is that the library would be used by other CMake projects, not by the entire system.
Ideally, the foo.h would show up next to libfoo.a in the build directory.
I've tried calling it a "FRAMEWORK", no luck; that only makes is a macOs framework.
I believe I can jury rig this, but methinks there's a best practice out there.
Open to an answer that says "here's a better way", too...
UPDATE
It might help to clarify how I think I want to pull this project into another. I've seen other projects use something like this:
add_subdirectory(<path_to_foo>/foo foo_build)
which causes the foo build to happen in a subdirectory. This allows me to refer to the library using 'foo_build', which is nice and clean. However, I still have to point at the original include directory to get the .h file, which makes me feel like I'm missing something.
It seems like cmake would have a clean solution for this.
I am fairly new to CMake but what I think you want is a 'add_custom_command'.
add_custom_command(TARGET foo.a POST_BUILD COMMAND copy foo.h ${CMAKE_LIBRARY_OUTPUT_DIRECTORY})
That might work.
What you are looking for is the following structure:
example/
- CMakeLists.txt
- src/
- main.c
- sub/
- foo/
- CMakeLists.txt
- src/
- foo/
- foo.c
- foo.h
Your CMakeLists will look like the following
example/CMakeLists.txt
# use modern target-based cmake features
cmake_minimum_required (VERSION 3.0)
# projectname
project (ff1_selfcheck)
add_subdirectory (sub/foo)
# executable to create
add_executable(${PROJECT_NAME}
src/main.c
)
# link libraries
target_link_libraries(${PROJECT_NAME}
PRIVATE
foo # imported target
)
example/sub/foo/CMakeLists.txt
# use modern target-based cmake features
cmake_minimum_required (VERSION 3.0)
# projectname
project (foo)
# executable to create
add_library(${PROJECT_NAME}
src/foo.c
)
# directories where to search for header files
target_include_directories(${PROJECT_NAME}
PUBLIC
source # the headerfiles in source are the includes
)
By using the project name foo in target_link_libraries(...) you refer to the foo library target
Furthermore, by using the PUBLIC keyword in the foo library, your headers (your include directory) is automatically propagated to every CMake project that adds this library via add_subdirectory(...).
Therefore you don't need to copy your headers! CMake >= 2.8.12 is beautiful, isn't it?
If you really want to copy files via CMake, the following would work:
file(COPY srcDir
DESTINATION dstDir
FILES_MATCHING
PATTERN .h
)
Take a look here: https://cmake.org/cmake/help/v3.2/command/file.html
As a general rule for CMake, sources are kept in the source directory and binaries and other generated files are within the build directory. So you wish is not very CMake-ish.
CMake would put headers and libraries according to your wishes when you install the project. Then you can specify what to copy where.
As you don't want to install this module, the best way is to create a package by providing a CMake config file for your project. This means that your project Foo would generate a file FooConfig.cmake which contains the paths to includes and libraries. The other CMake project would use find_package(Foo) to look for the file. By adding a hint to Foo_DIR you can make CMake find your project in a non-standard directory.
Further reading:
CMake documentation about packages
About how to use your library
Note, that configure_file is unrelated to what you wish, the confusing name has historic reasons. You can use this command, but per se it is unrelated.
UPDATE: after the update, I think that you want to use an external project. Behaves like an internal library, but pretty separated. See https://cmake.org/cmake/help/latest/module/ExternalProject.html
you should use generator expression for your "foo" include directory:
target_include_directories(foo PUBLIC
$<BUILD_INTERFACE:${PROJECT_SOURCE_DIR})
And since you don't want install rules not need to also add a $<INSTALL_INTERFACE:include>...
BTW you should don't care to copy the include file in the build directory (supposing you are building out of the source).
ps: if you also generate headers files simply add $<BUILD_INTERFACE:${PROJECT_BINARY_DIR}>
I added the xgboost library as a git submodule of my project and I'm trying to add it to cmake as a subdirectory. Unfortunately it's not working. A simple hello world project with the following CMakeLists.txt replicates the error that I'm getting.
cmake_minimum_required(VERSION 3.2)
project(foo)
add_subdirectory(xgboost)
add_executable(${PROJECT_NAME} foo.cpp)
target_link_libraries(${PROJECT_NAME} xgboost)
After building the library there is nothing in the xgboost/lib directory so I get the following error.
clang: error: no such file or directory:
'/Users/.../myproject/xgboost/lib/libxgboost.dylib'
I think that the problem is generated in their CMakeLists file since they have two different targets. Maybe cmake is choosing the wrong target but I'm not familiar enough with cmake to figure it out. The following code is from xgboost's CMakeLists.
# Executable
add_executable(runxgboost $<TARGET_OBJECTS:objxgboost> src/cli_main.cc)
set_target_properties(runxgboost PROPERTIES
OUTPUT_NAME xgboost
)
set_output_directory(runxgboost ${PROJECT_SOURCE_DIR})
target_link_libraries(runxgboost ${LINK_LIBRARIES})
# Shared library
add_library(xgboost SHARED $<TARGET_OBJECTS:objxgboost>)
target_link_libraries(xgboost ${LINK_LIBRARIES})
set_output_directory(xgboost ${PROJECT_SOURCE_DIR}/lib)
#Ensure these two targets do not build simultaneously, as they produce outputs with conflicting names
add_dependencies(xgboost runxgboost)
My questions in order of importance are:
Is there any way to fix it without modifying xgboost's CMakeLists.txt file?
Is it reasonable to try to add xgboost to my project as a git submodule?
Is there any reason cmake is not instructing to build the library?
Note: There were several edits to this question since I tried to narrow down the problem and to provide more information.
(I would love to ask for few things beforehand in the comment section, but I have too low reputation to do so, so I will just give it a shot ;))
I have few suspects, and one of them is ${CMAKE_SOURCE_DIR} of the submodule's root CMakeLists.txt. Although the paths are set properly when you run that CMakeLists.txt alone, cmake gets confused the moment you add it as your subdirectory. Have you looked into another directories for your output binaries?
First I would suggest testing this hypothesis, and then I would suggest writing similar, but separate CMakeLists.txt file for xgboost library, and then substitute it in the project temporarily. Unfortunately the CMakeLists.txt filename is hardcoded and there is no possibility to have two files of that kind in one directory; so it seems that the answer to 1) is, that you rather have to change the file.
For the 2): as long as it does not require huge additional logic in your CMakeLists.txt, it makes sense. Other viable option is to create an install target, which you can use to install your xgboost library locally (using CMAKE_INSTALL_PREFIX(doc) variable), and then add the installation path to your CMAKE_LIBRARY_PATH(doc).
I try to learn c++ with cmake. I have done a project, and cmake runs, it compiles, it works, that is nice, ok. Now I started a new project in which I want to use some classes of the first one. I have tried to read some sourcecode and I have understand that I need to make a module that I can read from both the application. So this will be my repository:
/
/cmake
/modules/Network
/software/sw1
/software/sw2
both the projects sw1 and sw2 depends on the module Netowrk. In the folder cmake there has to be the FindNetwork.cmake file, and in sw1, sw2 and Network there has to be the CMakeList.txt .
But.. I messed up with include_directories and other cmake obscurities..
Can someone point me out to a nice overview how to easily organize a repository with softwares that depend on common modules?
This link will provide some examples for you.
In terms of how you are looking at the project/infrastructure then it's best to not confuse things too much. So here is a couple of points to get you started (I hope)
In c++ a module is a library (so refer to your Network module as a
library)
To include a library you need to link it and also make the
header files available.
In cmake this is two commands target_link_libraries and include_directories respectively.
With that in mind the project structure could be
/Network/include (api here)
/Network/src
/sw1/src
/sw2/src
with an example base CmakeLists.txt file for you: (place in root dir of project)
cmake_minimum_required(VERSION 2.7) // your choice
project(myproject) // change name
add_subdirectory(Network)
add_subdirectory(sw1)
add_subdirectory(sw2)
in the Network Directory you would have this
add_library(Network net1.cc net2.cc etc.)
In the sw1 dir
include_dirs(${MYPROJECT_SOURCE_DIR}/Network/include)
link_directories(${MYPROJECT_BINARY_DIR}/Network)
add_executable (sw1prog sw1.cc sw11.cc etc.)
target_link_libraries (sw1prog Network)
In the sw2 dir
include_dirs(${MYPROJECT_SOURCE_DIR}/Network/include)
link_directories(${MYPROJECT_BINARY_DIR}/Network)
add_executable (sw2prog sw2.cc sw21.cc etc.)
target_link_libraries (sw2prog Network)
This is a very simplified version of what you may require, it removes the need for a FindXXModule.cmake file to be created and refers to the library you create implicitly. I think this is the best mechanism for you, if you did want to create a FindXXModule.cmake then I would suggest it's when you actually install your libs to the machine and wish others to be able to find it, either that or have a mechanism for multiple projects to link to each other libraries.
I hope this is a little useful, please bear in mind the cmake site has some examples and cmake --help is your friend.