How can I link a system library statically in mostly static mode (linkstatic=1)? I tried to use "-Wl,-Bstatic -lboost_thread -Wl,-Bdynamic" or "-Wl,-Bstatic", "-lboost_thread", "-Wl,-Bdynamic", but none of them worked. I don't want to hard code the path of libboost_thread.a in the system.
cc_binary(
name = "main",
srcs = [
"main.cpp",
],
linkopts = [
"-lboost_thread",
],
)
And boost_thread library is linked as a dynamic library.
ldd bazel-bin/main
linux-vdso.so.1
libboost_thread.so.1.54.0 => /usr/lib/x86_64-linux-gnu/libboost_thread.so.1.54.0
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
...
In your WORKSPACE file define an external repository...
new_local_repository(
name = "boost_thread",
path = "/usr/lib/x86_64-linux-gnu",
build_file = "boost_thread.BUILD"
)
Create a boost_thread.BUILD file
cc_library(
name = "lib",
srcs = ["libboost_thread.a"],
visibility = ["//visibility:public"],
)
Then in your cc_binary rule add
deps = ["#boost_thread//:lib",],
and throw in a
linkstatic = 1
to be on the safe side.
Based on the answer in this question, Telling gcc directly to link a library statically, "-l:libboost_thread.a" will link the system library statically, without hard coding the path of libboost_thread.a in the system.
cc_binary(
name = "main",
srcs = [
"main.cpp",
],
linkopts = [
"-l:libboost_thread.a",
"-l:libboost_system.a",
],
)
Related
I am working on extending a project using bazel to build. However, one of my thrid_party dependency relying on a dynamic linked QT library. And I had a hard time linking it.
My project base is envpool, and I am using procgen as my third-party dependency. However, procgen relies on a series of QT library.
My approachs so far:
In the WorkSpace file, I specify the local directory of the qt library. (I am working on a new EC2 instance of Ubuntu 20.04 LTS on amazon cloud, and install qt5 and other base tools)
// download the github project
maybe(
http_archive,
name = "procgen",
sha256 = "8d443b7b8fba44ef051b182e9a87abfa4e05292568e476ca1e5f08f9666a1b72",
strip_prefix = "procgen-0.10.7/procgen/src/",
urls = [
"https://github.com/openai/procgen/archive/refs/tags/0.10.7.zip"
]
build_file = "//third_party/procgen:procgen.BUILD",
)
new_local_repository(
name = "qt",
path = "/usr/include/x86_64-linux-gnu/qt5", // I check indeed the header files are there
build_file = "BUILD.qt"
)
And the BUILD.qt file is
cc_library(
name = "qt_core",
hdrs = glob(["QtCore/**"]),
includes = ["."],
linkopts = [
"-lQt5Core",
],
visibility = ["//visibility:public"],
)
cc_library(
name = "qt_widgets",
hdrs = glob(["QtWidgets/**"]),
includes = ["."],
deps = [":qt_core"],
linkopts = [
"-lQt5Widgets",
],
visibility = ["//visibility:public"],
)
cc_library(
name = "qt_gui",
hdrs = glob(["QtGui/**"]),
includes = ["."],
deps = [":qt_core"],
linkopts = [
"-lQt5Gui",
],
visibility = ["//visibility:public"],
)
And the BUILD file for the procgen is
package(default_visibility = ["//visibility:public"])
cc_library(
name = "procgen",
srcs = glob(["*.cpp", "games/*.cpp"]),
hdrs = glob(["*.h"]),
deps = [
"#qt//:qt_widgets",
"#qt//:qt_gui",
"#qt//:qt_core",
]
)
However, when I use Bazel to build the project, it gives back the error that
Use --sandbox_debug to see verbose messages from the sandbox
In file included from external/procgen/games/starpilot.cpp:2:
external/procgen/games/../assetgen.h:10:10: fatal error: QColor: No such file or directory
10 | #include <QColor>
| ^~~~~~~~
compilation terminated.
I know I probably mess up the path or header file's include somewhere. For example, I am basically follow this post to include the QT library in the project, but I notice that guys use the full path "#include <qt/QTWidgets/xxxx>" instead. But I cannot change the code for "include" in the procgen.
This is the first time I use bazel, and on such a big project. Really appreciate if someone could help me out.
I put my current whole project package at here for reproducibility. You can use short-cut "sudo make bazel-build" to build it.
Best,
YJ
I consider this a fairly fundamental requirement for a build system, but somehow it is not that straightforward - I've found 2 pieces of information, but without any luck yet:
The so-called official doc: https://docs.bazel.build/versions/main/cpp-use-cases.html#adding-dependencies-on-precompiled-libraries, it is really not much information
The cc_import reference:https://docs.bazel.build/versions/main/be/c-cpp.html#cc_import, which claimed for "allows users to import precompiled C/C++ libraries"
But for both cases, I don't see a way to specify the path/to/precompiled/lib, then how suppose it should work? I tried to ln the library folder into the Bazel workspace, but also got no luck.
Would really appreciate if you could shed lights on how to use an external dependency in Bazel, or point me to a real, working example.
Take a look at https://github.com/justbuchanan/bazel_rules_qt to see how a precompiled Qt5 version can be used with Bazel. The problem with precompiled libs is that you need them for every OS, compiler(-settings), platform. If you only target a specific OS + host compiler and target platform then precompiled libs are fine.
Think also about bazelizing your third-party library. I did this for instance for OpenEXR and oneTBB. I also hacked together some scripting to convert Visual Studio Project files to Bazel BUILD files.
Here is an example of how to use a precompiled OpenCV version on Windows:
WORKSPACE.bazel:
workspace(name = "OpenCVDemo")
load("#bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
# OpenCV
http_archive(
name = "opencv",
build_file = "//third_party:opencv.BUILD",
strip_prefix = "opencv/build",
# Release
url = "https://github.com/opencv/opencv/releases/download/4.3.0/opencv-4.3.0-dldt-2020.2-vc16-avx2.zip",
)
opencv.BUILD:
package(default_visibility = ["//visibility:public"])
MAIN_MODULES = [
"core",
"imgproc",
"imgcodecs",
"videoio",
"highgui",
"video",
"calib3d",
"features2d",
"objdetect",
"dnn",
"ml",
"flann",
"photo",
"stitching",
"gapi",
]
# https://stackoverflow.com/questions/56108940/how-to-specify-the-compiler-flags-to-be-used-in-opt-compilation-mode-by-my-own
config_setting(
name = "fastbuild_mode",
values = {"compilation_mode": "fastbuild"},
)
config_setting(
name = "dbg_mode",
values = {"compilation_mode": "dbg"},
)
cc_import(
name = "tbb",
shared_library = select({
":fastbuild_mode": "bin/tbb.dll",
":dbg_mode": "bin/tbb_debug.dll",
"//conditions:default": "bin/tbb.dll",
}),
)
[
(
cc_import(
name = module,
interface_library = select({
":fastbuild_mode": "lib/opencv_{}430.lib".format(module),
":dbg_mode": "lib/opencv_{}430d.lib".format(module),
"//conditions:default": "lib/opencv_{}430.lib".format(module),
}),
shared_library = select({
":fastbuild_mode": "bin/opencv_{}430.dll".format(module),
":dbg_mode": "bin/opencv_{}430d.dll".format(module),
"//conditions:default": "bin/opencv_{}430.dll".format(module),
}),
)
)
for module in MAIN_MODULES
]
cc_library(
name = "opencv",
hdrs = [
"include/opencv2/calib3d.hpp",
"include/opencv2/calib3d/calib3d.hpp",
"include/opencv2/calib3d/calib3d_c.h",
"include/opencv2/core.hpp",
"include/opencv2/core/hal/interface.h",
"include/opencv2/cvconfig.h",
"include/opencv2/dnn.hpp",
"include/opencv2/features2d.hpp",
"include/opencv2/flann.hpp",
"include/opencv2/flann/config.h",
"include/opencv2/flann/defines.h",
"include/opencv2/flann/miniflann.hpp",
"include/opencv2/highgui.hpp",
"include/opencv2/highgui/highgui.hpp",
"include/opencv2/highgui/highgui_c.h",
"include/opencv2/imgcodecs.hpp",
"include/opencv2/imgproc.hpp",
"include/opencv2/ml.hpp",
"include/opencv2/ml/ml.inl.hpp",
"include/opencv2/objdetect.hpp",
"include/opencv2/opencv.hpp",
"include/opencv2/opencv_modules.hpp",
"include/opencv2/photo.hpp",
"include/opencv2/stitching.hpp",
"include/opencv2/video.hpp",
"include/opencv2/video/background_segm.hpp",
"include/opencv2/video/tracking.hpp",
"include/opencv2/videoio.hpp",
"include/opencv2/videoio/videoio_c.h",
],
includes = ["include"],
deps = MAIN_MODULES + [
"tbb",
],
)
BUILD.bazel:
cc_binary(
name = "OpenCVDemo",
srcs = ["main.cpp"],
deps = ["#opencv"],
)
Should work similar for other libraries
Given a C++ application, built with Bazel, that depends on an external, system provided shared library:
cc_binary(
name = 'app',
srcs = ['app.cpp'],
linkstatic = False,
deps = ['#my_system//:system_lib'],
)
The WORKSPACE and BUILD.my_system files:
new_local_repository(
name = 'my_system',
build_file = 'BUILD.my_system',
path = '/usr/lib/my_system/',
)
cc_import(
name = 'system_lib',
shared_library = 'system_lib.so',
visibility = ['//visibility:public'],
)
This builds, but first copies the system provided lib to the cache, and links to that:
$ ldd bazel-bin/app/app
system_lib.so => /home/erenon/bazel/proj/bazel-bin/app/../_solib_k8/_U#my_system_S_S_Csystem_Ulib___Uexternal_Smy_system/system_lib.so
[...]
If I move app to an identical system that has /usr/lib/my_system/system_lib.so, it breaks, as it misses the cache. I'd like to package app in a way that it directly links to the original .so, without the intermediate cache copy or name mangling, i.e: I'd like to achieve:
$ ldd bazel-bin/app/app
system_lib.so => /usr/lib/my_system/system_lib.so
[...]
I tried cc_import.system_provided, but that appears only work for Windows lib/dlls.
Wrap your library with another cc_library.
I can't explain why it works, but it does.
cc_library(
name = "libsystem_lib",
srcs = [ ":system_lib" ],
hdrs = ...
)
Depend on this one instead.
I will base my question on stage 2 from the bazel tutorial for c++.
Normally this example will create hello-world linked statically with libhello-greet.a. However I would like to create hello-world linked dynamically with libhello-greet.so.
Therefore I found some kind of workaround by using this BUILD file:
cc_binary(
name = "libhello-greet.so",
srcs = ["hello-greet.cc", "hello-greet.h"],
linkshared = 1,
)
cc_import(
name = "libhello-greet",
shared_library = "libhello-greet.so",
hdrs = ["hello-greet.h"],
)
cc_binary(
name = "hello-world",
srcs = ["hello-world.cc"],
deps = [
":libhello-greet",
],
)
but this doesn't feel like the best solution. Is there a better way to create and link with a shared library?
If you specify the linkstatic-flag in the binary, it will link all libraries either as static or as shared libraries. But I do not know how to link only certain libraries as shared libraries.
cc_library(
name = "hello-greet",
srcs = ["hello_greet.cc"],
hdrs = ["hello_greet.h"],
)
cc_binary(
name = "hello-world",
srcs = ["main.cc"],
deps = [
":hello-greet",
],
linkstatic=False,
)
Folks,
I am trying to link .h and static libraries into my tensorflow program. My headers are in
/usr/local/include/lcm
And static/shared libraries (.so, etc.) in
/usr/local/lib
But Bazel complains they don't exist, or that it cannot find them. Here is my code on my BUILD file:
package(default_visibility = ["//tensorflow:internal"])
licenses(["notice"]) # Apache 2.0
exports_files(["LICENSE"])
# LCM shared libraries path
cc_library(
name = "lcm_lib",
srcs = glob([
"*.so",
]),
copts = ["-L/usr/local/lib"],
linkopts = ["-pthread", "-shared"],
visibility = ["//visibility:public"],
)
# LCM header libraries path
cc_library(
name = "lcm_headers",
hdrs = glob([
"include/**/*.h",
]),
copts = ["-L/usr/local/include"],
linkopts = ["-pthread"],
visibility = ["//visibility:public"],
)
cc_binary(
name = "myProject",
srcs = [
"main.cc",
],
linkopts = ["-lm"],
deps = [
"//tensorflow/cc:cc_ops",
"//tensorflow/core:framework_internal",
"//tensorflow/core:tensorflow",
],
)
filegroup(
name = "all_files",
srcs = glob(
["**/*"],
exclude = [
"**/METADATA",
"**/OWNERS",
"bin/**",
"gen/**",
],
),
visibility = ["//tensorflow:__subpackages__"],
)
If I remove LCM related code (from both BUILD and main.cc), then my program builds and runs. When I include LCM, then I get errors saying that lcm::LCM::~LCM() is undefined, and that it cannot find the liblcm.so.
Now, my non-tensorflow code (or the majority of my project) is running cmake, and LCM and the rest of the libraries I need (openCV, etc.) can be found. I use commands in my CMakeList.txt like:
# search path for LCM header files
set(LCM_IncludeSearchPaths
/usr/include/
/usr/local/include/
/opt/local/include
)
# search path for LCM static/dynamic libraries
set(LCM_LibrarySearchPaths
/usr/lib/
/usr/local/lib/
/opt/local/lib/
)
find_path(LCM_INCLUDE_DIR
NAMES lcm/lcm.h
HINTS ${LCM_IncludeSearchPaths}
)
FIND_LIBRARY(LCM_LIBS
NAMES lcm
HINTS ${LCM_LibrarySearchPaths}
PATH_SUFFIXES lib
)
And it all works. But it does not work for tensorflow and Bazel
Here are my build and WORKSPACE files, located in the same directory:
This is my touched WORKSPACE file:
# LCM static libraries
new_local_repository(
name = "lcm_libs",
# pkg-config --variable=libdir x11
path = "/usr/local/lib",
build_file_content = """
cc_library(
name = "liblcm",
srcs = ["liblcm.so"],
visibility = ["//visibility:public"],
)
""",
)
# LCM header files
new_local_repository(
name = "lcm_headers",
# pkg-config --variable=libdir x11
path = "/usr/local/include",
build_file_content = """
cc_library(
name = "lcm",
hdrs = glob([
"lcm/*.h", "lcm/*.hpp",
]),
visibility = ["//visibility:public"],
)
""",
)
# bind to a name to avoid using the "actual" format
#bind(
# name = "liblcm",
# actual = "#lcm_libs//:liblcm",
#)
#bind(
# name = "lcm",
# actual = "#lcm_headers//:lcm",
#)
#
And my BUILD:
# Description:
# Tensorflow C++ inference example for labeling images.
package(default_visibility = ["//tensorflow:internal"])
licenses(["notice"]) # Apache 2.0
exports_files(["LICENSE"])
cc_binary(
name = "facialFatigue",
srcs = [
"main.cc",
],
linkopts = ["-lm"],
deps = [
"//tensorflow/cc:cc_ops",
"//tensorflow/core:framework_internal",
"//tensorflow/core:tensorflow",
"#lcm_libs//:liblcm",
"#lcm_headers//:lcm",
],
)
filegroup(
name = "all_files",
srcs = glob(
["**/*"],
exclude = [
"**/METADATA",
"**/OWNERS",
"bin/**",
"gen/**",
],
),
visibility = ["//tensorflow:__subpackages__"],
)
Sorry for the long question :-(
Bazel executes action (in your case C++ compile action) in a sandbox, to ensure the hermeticity. This is needed to be correct when inputs of the action change. Therefore you have to tell Bazel about all the inputs, including the system ones.
But of course you can depend on system libraries, take a look at local_repository rule. You might also find this example in bazel-discuss# thread helpful.