Given a C++ application, built with Bazel, that depends on an external, system provided shared library:
cc_binary(
name = 'app',
srcs = ['app.cpp'],
linkstatic = False,
deps = ['#my_system//:system_lib'],
)
The WORKSPACE and BUILD.my_system files:
new_local_repository(
name = 'my_system',
build_file = 'BUILD.my_system',
path = '/usr/lib/my_system/',
)
cc_import(
name = 'system_lib',
shared_library = 'system_lib.so',
visibility = ['//visibility:public'],
)
This builds, but first copies the system provided lib to the cache, and links to that:
$ ldd bazel-bin/app/app
system_lib.so => /home/erenon/bazel/proj/bazel-bin/app/../_solib_k8/_U#my_system_S_S_Csystem_Ulib___Uexternal_Smy_system/system_lib.so
[...]
If I move app to an identical system that has /usr/lib/my_system/system_lib.so, it breaks, as it misses the cache. I'd like to package app in a way that it directly links to the original .so, without the intermediate cache copy or name mangling, i.e: I'd like to achieve:
$ ldd bazel-bin/app/app
system_lib.so => /usr/lib/my_system/system_lib.so
[...]
I tried cc_import.system_provided, but that appears only work for Windows lib/dlls.
Wrap your library with another cc_library.
I can't explain why it works, but it does.
cc_library(
name = "libsystem_lib",
srcs = [ ":system_lib" ],
hdrs = ...
)
Depend on this one instead.
Related
I am working on extending a project using bazel to build. However, one of my thrid_party dependency relying on a dynamic linked QT library. And I had a hard time linking it.
My project base is envpool, and I am using procgen as my third-party dependency. However, procgen relies on a series of QT library.
My approachs so far:
In the WorkSpace file, I specify the local directory of the qt library. (I am working on a new EC2 instance of Ubuntu 20.04 LTS on amazon cloud, and install qt5 and other base tools)
// download the github project
maybe(
http_archive,
name = "procgen",
sha256 = "8d443b7b8fba44ef051b182e9a87abfa4e05292568e476ca1e5f08f9666a1b72",
strip_prefix = "procgen-0.10.7/procgen/src/",
urls = [
"https://github.com/openai/procgen/archive/refs/tags/0.10.7.zip"
]
build_file = "//third_party/procgen:procgen.BUILD",
)
new_local_repository(
name = "qt",
path = "/usr/include/x86_64-linux-gnu/qt5", // I check indeed the header files are there
build_file = "BUILD.qt"
)
And the BUILD.qt file is
cc_library(
name = "qt_core",
hdrs = glob(["QtCore/**"]),
includes = ["."],
linkopts = [
"-lQt5Core",
],
visibility = ["//visibility:public"],
)
cc_library(
name = "qt_widgets",
hdrs = glob(["QtWidgets/**"]),
includes = ["."],
deps = [":qt_core"],
linkopts = [
"-lQt5Widgets",
],
visibility = ["//visibility:public"],
)
cc_library(
name = "qt_gui",
hdrs = glob(["QtGui/**"]),
includes = ["."],
deps = [":qt_core"],
linkopts = [
"-lQt5Gui",
],
visibility = ["//visibility:public"],
)
And the BUILD file for the procgen is
package(default_visibility = ["//visibility:public"])
cc_library(
name = "procgen",
srcs = glob(["*.cpp", "games/*.cpp"]),
hdrs = glob(["*.h"]),
deps = [
"#qt//:qt_widgets",
"#qt//:qt_gui",
"#qt//:qt_core",
]
)
However, when I use Bazel to build the project, it gives back the error that
Use --sandbox_debug to see verbose messages from the sandbox
In file included from external/procgen/games/starpilot.cpp:2:
external/procgen/games/../assetgen.h:10:10: fatal error: QColor: No such file or directory
10 | #include <QColor>
| ^~~~~~~~
compilation terminated.
I know I probably mess up the path or header file's include somewhere. For example, I am basically follow this post to include the QT library in the project, but I notice that guys use the full path "#include <qt/QTWidgets/xxxx>" instead. But I cannot change the code for "include" in the procgen.
This is the first time I use bazel, and on such a big project. Really appreciate if someone could help me out.
I put my current whole project package at here for reproducibility. You can use short-cut "sudo make bazel-build" to build it.
Best,
YJ
I have a project for a plugin in C/C++ where I have different implementations and I would like to be able to select which implementation to use when building.
WORKSPACE
plugin/
BUILD
plugin.c
plugin.h
plugin_impl1/
BUILD
...
plugin_impl2/
BUILD
...
plugin_impl1/BUILD:
cc_library(
name = "plugin_impl1"
srcs = [...]
hdrs = [...]
)
plugin_impl2/BUILD:
cc_library(
name = "plugin_impl2"
srcs = [...]
hdrs = [...]
)
I want to know how to write plugin/BUILD so that I could write something like:
bazel build //plugin --which plugin_impl2
The executable name should always be the same so having multiple rules with different target names (for plugin) is not an option.
I consider this a fairly fundamental requirement for a build system, but somehow it is not that straightforward - I've found 2 pieces of information, but without any luck yet:
The so-called official doc: https://docs.bazel.build/versions/main/cpp-use-cases.html#adding-dependencies-on-precompiled-libraries, it is really not much information
The cc_import reference:https://docs.bazel.build/versions/main/be/c-cpp.html#cc_import, which claimed for "allows users to import precompiled C/C++ libraries"
But for both cases, I don't see a way to specify the path/to/precompiled/lib, then how suppose it should work? I tried to ln the library folder into the Bazel workspace, but also got no luck.
Would really appreciate if you could shed lights on how to use an external dependency in Bazel, or point me to a real, working example.
Take a look at https://github.com/justbuchanan/bazel_rules_qt to see how a precompiled Qt5 version can be used with Bazel. The problem with precompiled libs is that you need them for every OS, compiler(-settings), platform. If you only target a specific OS + host compiler and target platform then precompiled libs are fine.
Think also about bazelizing your third-party library. I did this for instance for OpenEXR and oneTBB. I also hacked together some scripting to convert Visual Studio Project files to Bazel BUILD files.
Here is an example of how to use a precompiled OpenCV version on Windows:
WORKSPACE.bazel:
workspace(name = "OpenCVDemo")
load("#bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
# OpenCV
http_archive(
name = "opencv",
build_file = "//third_party:opencv.BUILD",
strip_prefix = "opencv/build",
# Release
url = "https://github.com/opencv/opencv/releases/download/4.3.0/opencv-4.3.0-dldt-2020.2-vc16-avx2.zip",
)
opencv.BUILD:
package(default_visibility = ["//visibility:public"])
MAIN_MODULES = [
"core",
"imgproc",
"imgcodecs",
"videoio",
"highgui",
"video",
"calib3d",
"features2d",
"objdetect",
"dnn",
"ml",
"flann",
"photo",
"stitching",
"gapi",
]
# https://stackoverflow.com/questions/56108940/how-to-specify-the-compiler-flags-to-be-used-in-opt-compilation-mode-by-my-own
config_setting(
name = "fastbuild_mode",
values = {"compilation_mode": "fastbuild"},
)
config_setting(
name = "dbg_mode",
values = {"compilation_mode": "dbg"},
)
cc_import(
name = "tbb",
shared_library = select({
":fastbuild_mode": "bin/tbb.dll",
":dbg_mode": "bin/tbb_debug.dll",
"//conditions:default": "bin/tbb.dll",
}),
)
[
(
cc_import(
name = module,
interface_library = select({
":fastbuild_mode": "lib/opencv_{}430.lib".format(module),
":dbg_mode": "lib/opencv_{}430d.lib".format(module),
"//conditions:default": "lib/opencv_{}430.lib".format(module),
}),
shared_library = select({
":fastbuild_mode": "bin/opencv_{}430.dll".format(module),
":dbg_mode": "bin/opencv_{}430d.dll".format(module),
"//conditions:default": "bin/opencv_{}430.dll".format(module),
}),
)
)
for module in MAIN_MODULES
]
cc_library(
name = "opencv",
hdrs = [
"include/opencv2/calib3d.hpp",
"include/opencv2/calib3d/calib3d.hpp",
"include/opencv2/calib3d/calib3d_c.h",
"include/opencv2/core.hpp",
"include/opencv2/core/hal/interface.h",
"include/opencv2/cvconfig.h",
"include/opencv2/dnn.hpp",
"include/opencv2/features2d.hpp",
"include/opencv2/flann.hpp",
"include/opencv2/flann/config.h",
"include/opencv2/flann/defines.h",
"include/opencv2/flann/miniflann.hpp",
"include/opencv2/highgui.hpp",
"include/opencv2/highgui/highgui.hpp",
"include/opencv2/highgui/highgui_c.h",
"include/opencv2/imgcodecs.hpp",
"include/opencv2/imgproc.hpp",
"include/opencv2/ml.hpp",
"include/opencv2/ml/ml.inl.hpp",
"include/opencv2/objdetect.hpp",
"include/opencv2/opencv.hpp",
"include/opencv2/opencv_modules.hpp",
"include/opencv2/photo.hpp",
"include/opencv2/stitching.hpp",
"include/opencv2/video.hpp",
"include/opencv2/video/background_segm.hpp",
"include/opencv2/video/tracking.hpp",
"include/opencv2/videoio.hpp",
"include/opencv2/videoio/videoio_c.h",
],
includes = ["include"],
deps = MAIN_MODULES + [
"tbb",
],
)
BUILD.bazel:
cc_binary(
name = "OpenCVDemo",
srcs = ["main.cpp"],
deps = ["#opencv"],
)
Should work similar for other libraries
I will base my question on stage 2 from the bazel tutorial for c++.
Normally this example will create hello-world linked statically with libhello-greet.a. However I would like to create hello-world linked dynamically with libhello-greet.so.
Therefore I found some kind of workaround by using this BUILD file:
cc_binary(
name = "libhello-greet.so",
srcs = ["hello-greet.cc", "hello-greet.h"],
linkshared = 1,
)
cc_import(
name = "libhello-greet",
shared_library = "libhello-greet.so",
hdrs = ["hello-greet.h"],
)
cc_binary(
name = "hello-world",
srcs = ["hello-world.cc"],
deps = [
":libhello-greet",
],
)
but this doesn't feel like the best solution. Is there a better way to create and link with a shared library?
If you specify the linkstatic-flag in the binary, it will link all libraries either as static or as shared libraries. But I do not know how to link only certain libraries as shared libraries.
cc_library(
name = "hello-greet",
srcs = ["hello_greet.cc"],
hdrs = ["hello_greet.h"],
)
cc_binary(
name = "hello-world",
srcs = ["main.cc"],
deps = [
":hello-greet",
],
linkstatic=False,
)
I am playing with LLVM and I hit an issue when trying to use JIT. I was able to build a compiler, it can be compiled, linked and it runs correctly (it compiles my toy programs). However, when I am trying to use build a JIT, it fails.
dyld: Symbol not found: __ZN4llvm11RuntimeDyld13MemoryManager6anchorEv
Referenced from: /Users/gruszczy/Projects/shwifty/./bazel-bin/_solib_darwin//liblibjit.so
Expected in: flat namespace
in /Users/gruszczy/Projects/shwifty/./bazel-bin/_solib_darwin//liblibjit.so
Abort trap: 6
I use Bazel to build everything, these are my build rules:
new_local_repository(
name = "llvm",
path = "/opt/local/libexec/llvm-4.0",
build_file= "llvm.BUILD")
cc_library(
name = "main",
srcs = glob(["lib/*.a"]),
hdrs = glob(["include/**/*.*"]),
visibility = ["//visibility:public"],
copts = ["-Iexternal/llvm/include"],
)
I use JIT in tests (I generate IR in the test then jit it, then run the method to see if it worked).
cc_library(
name = "jit",
srcs = ["jit.cc"],
hdrs = ["jit.h"],
deps = [
":ast",
":common",
"#llvm//:main"
],
copts = GENERAL_COPTS)
cc_test(
name = "codegen_test",
srcs = ["codegen_test.cc"],
deps = [
":ast",
":jit",
":lexer",
":parser",
":codegen",
"#gtest//:main",
"#llvm//:main"
],
copts = TEST_COPTS,
data = [":examples"],
size = "small"
)
Any suggestions what I might be missing?
The source of confusion is that Bazel by default links binaries statically, but tests dynamically. This makes the test-code-refactor loop faster, because changes to the test code only trigger the rebuild of the test, not the whole application. It can be disabled by setting linkstatic = 1 on codegen_test target.
As to why the symbols are not present in codegen_test when built as a shared library, that's much harder question and would need more project-specific information. But a possible solution might be to mark targets producing VMRuntimeDyld.a and VMMCJit.a as alwayslink = 1.
For the completeness, here's the link to an issue you reported on bazel.