I'm working on a project, which depends on a third party project, let's say project E, which is a precompiled libraries. I've set a new repository_rule and invoked it in the workspace.bzl, and in the BUILD of the project E, the corresponding library rule has been set like following:
cc_library(
name = "e_lib",
srcs = [
"#e//:e_lib1",
"#e//:e_lib2",
],
hdrs = ["#e//:e_headers"],
visibility = ["//visibility:public"],
)
And the target e_lib1, e_lib2 and e_headers are defined in the file e.BUILD, and it is correctly symblo linked to the downloaded contents folder in the last part of implementation function of repository_rule.
The e.BUILD is something like:
filegroup(
name = "e_headers",
srcs = glob(["include/*"]),
visibility = ["//visibility:public"],
)
filegroup(
name = "e_lib1",
srcs = if_darwin(["lib/lib1.dylib"])
+ if_linux_x86_64(["lib/lib1.so"]),
visibility = ["//visibility:public"],
)
filegroup(
name = "e_lib2",
srcs = if_darwin(["lib/lib2.dylib"])
+ if_linux_x86_64(["lib/lib2.so"]),
visibility = ["//visibility:public"],
)
Let's say there is a header baz.h in the directory include of the project E.But during the building phase, an error, baz.h file not found, occurred when bazel tried to compile a file core.cc. The pseudocodes are something like:
// core.cc
#include "core.h"
...
// core.h
#include "im.h"
...
// im.h
#include "baz.h"
// This file doesn't have the corresponding
// .cc file since all the implementations
// are put in this header.
The dependency of these files is core.cc -> core.h -> im.h -> baz.h, and the building rule of core.cc is:
cc_library(
name = "core",
srcs = [
"core.cc",
],
hdrs = "core.h",
deps = [
"//third_party/E:e_lib"],
alwayslink = 1,
)
So is there something wrong with my BUILD file? I guess that maybe the core.cc doesn't directly depend on the e_lib, so the deps of it is actually useless. Should I build a cc_library for the im.h?
Related
I am trying to include a header file from a third party package to a C++ code in my project. This is how I structured it:
main_repo/
- third_party/some_lib/
- BUILD
- header_file.h
- extensions/my_project/
- BUILD
- my_app.cpp
- my_app.hpp
main_repo/extensions/my_project/BUILD
cc_library(
name = "my_app",
srcs = ["my_app.cpp"],
hdrs = ["my_app.hpp"],
visibility = ["//visibility:public"],
deps = [
"//third_party/some_lib:some_lib",
],
)
main_repo/third_party/some_lib/BUILD
cc_library(
name = "some_lib",
hdrs = glob(["*.h"]),
includes = ["."],
visibility = ["//visibility:public"],
)
main_repo/extensions/my_project/my_app.hpp
// some code here
#include "header_file.h"
// some code here
I see
header_file.h: No such file or directory
error when I try building it. Do you see a problem with my approach?
I am working on extending a project using bazel to build. However, one of my thrid_party dependency relying on a dynamic linked QT library. And I had a hard time linking it.
My project base is envpool, and I am using procgen as my third-party dependency. However, procgen relies on a series of QT library.
My approachs so far:
In the WorkSpace file, I specify the local directory of the qt library. (I am working on a new EC2 instance of Ubuntu 20.04 LTS on amazon cloud, and install qt5 and other base tools)
// download the github project
maybe(
http_archive,
name = "procgen",
sha256 = "8d443b7b8fba44ef051b182e9a87abfa4e05292568e476ca1e5f08f9666a1b72",
strip_prefix = "procgen-0.10.7/procgen/src/",
urls = [
"https://github.com/openai/procgen/archive/refs/tags/0.10.7.zip"
]
build_file = "//third_party/procgen:procgen.BUILD",
)
new_local_repository(
name = "qt",
path = "/usr/include/x86_64-linux-gnu/qt5", // I check indeed the header files are there
build_file = "BUILD.qt"
)
And the BUILD.qt file is
cc_library(
name = "qt_core",
hdrs = glob(["QtCore/**"]),
includes = ["."],
linkopts = [
"-lQt5Core",
],
visibility = ["//visibility:public"],
)
cc_library(
name = "qt_widgets",
hdrs = glob(["QtWidgets/**"]),
includes = ["."],
deps = [":qt_core"],
linkopts = [
"-lQt5Widgets",
],
visibility = ["//visibility:public"],
)
cc_library(
name = "qt_gui",
hdrs = glob(["QtGui/**"]),
includes = ["."],
deps = [":qt_core"],
linkopts = [
"-lQt5Gui",
],
visibility = ["//visibility:public"],
)
And the BUILD file for the procgen is
package(default_visibility = ["//visibility:public"])
cc_library(
name = "procgen",
srcs = glob(["*.cpp", "games/*.cpp"]),
hdrs = glob(["*.h"]),
deps = [
"#qt//:qt_widgets",
"#qt//:qt_gui",
"#qt//:qt_core",
]
)
However, when I use Bazel to build the project, it gives back the error that
Use --sandbox_debug to see verbose messages from the sandbox
In file included from external/procgen/games/starpilot.cpp:2:
external/procgen/games/../assetgen.h:10:10: fatal error: QColor: No such file or directory
10 | #include <QColor>
| ^~~~~~~~
compilation terminated.
I know I probably mess up the path or header file's include somewhere. For example, I am basically follow this post to include the QT library in the project, but I notice that guys use the full path "#include <qt/QTWidgets/xxxx>" instead. But I cannot change the code for "include" in the procgen.
This is the first time I use bazel, and on such a big project. Really appreciate if someone could help me out.
I put my current whole project package at here for reproducibility. You can use short-cut "sudo make bazel-build" to build it.
Best,
YJ
I have two implementations of an interface in cpp and I want to choose one of the implementation each time while building (Bazel Build) the project and the generated executable application should be of same name each time.
My BUILD file looks like:
cc_library(
name = "printme1",
srcs = ["Firstimplementation.cpp"],
hdrs = ["source.h"],
)
cc_library(
name = "printme2",
srcs = ["Secondimplementation.cpp"],
hdrs = ["source.h"],
)
cc_binary(
name = "call",
srcs = ["main.cpp"],
deps = [
":printme1",
],
)
cc_binary(
name = "call2",
srcs = ["main.cpp"],
deps = [
":printme2",
],
)
With this BUILD file I am able to get two different results but the generated executable application is of different name. I am building it as follows:
$ bazel build call ---> gives me execuatable with name call in Gen-bin folder
$ bazel build call2 ---> gives me execuatable with name call2 in Gen-bin folder
What should I do to have same name in the end and I can call include every time my own choice implementation.
Thanks in advance.
You can use select together with a config_setting.
Should be something like this:
config_setting(
name = "printme1",
values = {
"define": "p1",
},
)
config_setting(
name = "printme2",
values = {
"define": "p2",
},
)
cc_library(
name = "printme",
srcs = select({":printme1" : ["Firstimplementation.cpp"],
":printme2" : ["Secondimplementation.cpp"]}),
hdrs = ["source.h"],
)
cc_binary(
name = "call",
srcs = ["main.cpp"],
deps = [
":printme",
])
In this case you need to define, when building your app:
bazel build --define p1 :call
Background
I am on Windows building with Bazel using cl to compile C++.
Subset of files:
third_party/icu/source/common/unicode/schriter.h
third_party/icu/source/common/unicode/utypes.h
third_party/icu/source/common/unicode/stringpiece.h
third_party/icu/source/common/stringpiece.cpp
third_party/icu/BUILD
a/a.cc
a/a.h
a/BUILD
b/cpp/src/strings/stringpiece.h
b/cpp/src/util/uri_utils.h
b/BUILD
schriter.h has #include "unicode/utypes.h".
uri_utils.h and b/cpp/src/strings/stringpiece.h both have class StringPiece. third_party/icu/source/common/unicode/stringpiece.h has class U_COMMON_API StringPiece : public UMemory
a.cc refers to StringPiece and has these includes:
#include "b/cpp/util/uri_utils.h"
#include "strings/stringpiece.h"
#include "third_party/icu/source/common/unicode/schriter.h"
a/BUILD:
cc_library(
name = "a",
srcs = ["a.cc"],
hdrs = ["a.h"],
deps = [
"//third_party/icu:common",
"//b:sdk_strings",
],
)
b/BUILD:
cc_library(
name = "sdk_strings",
srcs = [
"cpp/util/uri_utils.cc",
"cpp/src/strings/stringpiece.cc"
],
hdrs = [
"cpp/util/uri_utils.h",
"cpp/src/strings/stringpiece.h",
],
includes = ["cpp/src"],
)
third_party/icu/BUILD:
cc_library(
name = "common",
srcs = [
"source/common/stringpiece.cpp",
"source/stubdata/stubdata.c",
],
hdrs = glob(["**/*.h"]),
)
Problem
As is, building third_party/icu:common fails with:
third_party/icu/source/stubdata/stubdata.c(20): fatal error C1083: Cannot open include file: 'unicode/utypes.h': No such file or directory
If I add copts = ["/Ithird_party/icu/source/common",], to third_party/icu/BUILD, then icu:common builds but target a fails with:
third_party/icu/source/common/unicode/schriter.h(21): fatal error C1083: Cannot open include file: 'unicode/utypes.h': No such file or directory
If instead I add includes = ["source/common",],, then icu:common builds but target a fails with:
a/a.cc(168): error C2872: 'StringPiece': ambiguous symbol
b/cpp/util/uri_utils.h(24): note: could be 'StringPiece'
third_party\icu\source\common\unicode/stringpiece.h(52): note: or 'icu_54::StringPiece'
The source compiles fine using cmake, so I shouldn't need to change the source. How do I change the BUILD files to make this build correctly? How do I let everything in icu access the headers in unicode, but not expose unicode/stringpiece.h to targets that depend on icu?
You should be able to add the namespace (icu::StringPiece, I'm guessing?) to resolve the C2872 error.
For limiting visibility, check out the documentation:
For cc_library rules, headers in hdrs comprise the public interface of the library and can be directly included both from the files in hdrs and srcs of the library itself as well as from files in hdrs and srcs of cc_* rules that list the library in their deps. Headers in srcs must only be directly included from the files in hdrs and srcs of the library itself.
This means that hdrs define transitively visible headers and srcs is for "private" headers.
However, as the docs points out further down, this cannot always be perfectly enforced:
Unfortunately Bazel currently cannot distinguish between direct and transitive inclusions, so it cannot detect error cases where a file illegally includes a header directly that is only allowed to be included transitively. For example, Bazel would not complain if in the example above foo.cc directly includes baz.h. This would be illegal, because foo does not directly depend on baz.
So, it should prevent all but the most determined user to put it in the srcs of a private target with a copts = ['-Ithird_party/icu/source/common'] option.
Folks,
I am trying to link .h and static libraries into my tensorflow program. My headers are in
/usr/local/include/lcm
And static/shared libraries (.so, etc.) in
/usr/local/lib
But Bazel complains they don't exist, or that it cannot find them. Here is my code on my BUILD file:
package(default_visibility = ["//tensorflow:internal"])
licenses(["notice"]) # Apache 2.0
exports_files(["LICENSE"])
# LCM shared libraries path
cc_library(
name = "lcm_lib",
srcs = glob([
"*.so",
]),
copts = ["-L/usr/local/lib"],
linkopts = ["-pthread", "-shared"],
visibility = ["//visibility:public"],
)
# LCM header libraries path
cc_library(
name = "lcm_headers",
hdrs = glob([
"include/**/*.h",
]),
copts = ["-L/usr/local/include"],
linkopts = ["-pthread"],
visibility = ["//visibility:public"],
)
cc_binary(
name = "myProject",
srcs = [
"main.cc",
],
linkopts = ["-lm"],
deps = [
"//tensorflow/cc:cc_ops",
"//tensorflow/core:framework_internal",
"//tensorflow/core:tensorflow",
],
)
filegroup(
name = "all_files",
srcs = glob(
["**/*"],
exclude = [
"**/METADATA",
"**/OWNERS",
"bin/**",
"gen/**",
],
),
visibility = ["//tensorflow:__subpackages__"],
)
If I remove LCM related code (from both BUILD and main.cc), then my program builds and runs. When I include LCM, then I get errors saying that lcm::LCM::~LCM() is undefined, and that it cannot find the liblcm.so.
Now, my non-tensorflow code (or the majority of my project) is running cmake, and LCM and the rest of the libraries I need (openCV, etc.) can be found. I use commands in my CMakeList.txt like:
# search path for LCM header files
set(LCM_IncludeSearchPaths
/usr/include/
/usr/local/include/
/opt/local/include
)
# search path for LCM static/dynamic libraries
set(LCM_LibrarySearchPaths
/usr/lib/
/usr/local/lib/
/opt/local/lib/
)
find_path(LCM_INCLUDE_DIR
NAMES lcm/lcm.h
HINTS ${LCM_IncludeSearchPaths}
)
FIND_LIBRARY(LCM_LIBS
NAMES lcm
HINTS ${LCM_LibrarySearchPaths}
PATH_SUFFIXES lib
)
And it all works. But it does not work for tensorflow and Bazel
Here are my build and WORKSPACE files, located in the same directory:
This is my touched WORKSPACE file:
# LCM static libraries
new_local_repository(
name = "lcm_libs",
# pkg-config --variable=libdir x11
path = "/usr/local/lib",
build_file_content = """
cc_library(
name = "liblcm",
srcs = ["liblcm.so"],
visibility = ["//visibility:public"],
)
""",
)
# LCM header files
new_local_repository(
name = "lcm_headers",
# pkg-config --variable=libdir x11
path = "/usr/local/include",
build_file_content = """
cc_library(
name = "lcm",
hdrs = glob([
"lcm/*.h", "lcm/*.hpp",
]),
visibility = ["//visibility:public"],
)
""",
)
# bind to a name to avoid using the "actual" format
#bind(
# name = "liblcm",
# actual = "#lcm_libs//:liblcm",
#)
#bind(
# name = "lcm",
# actual = "#lcm_headers//:lcm",
#)
#
And my BUILD:
# Description:
# Tensorflow C++ inference example for labeling images.
package(default_visibility = ["//tensorflow:internal"])
licenses(["notice"]) # Apache 2.0
exports_files(["LICENSE"])
cc_binary(
name = "facialFatigue",
srcs = [
"main.cc",
],
linkopts = ["-lm"],
deps = [
"//tensorflow/cc:cc_ops",
"//tensorflow/core:framework_internal",
"//tensorflow/core:tensorflow",
"#lcm_libs//:liblcm",
"#lcm_headers//:lcm",
],
)
filegroup(
name = "all_files",
srcs = glob(
["**/*"],
exclude = [
"**/METADATA",
"**/OWNERS",
"bin/**",
"gen/**",
],
),
visibility = ["//tensorflow:__subpackages__"],
)
Sorry for the long question :-(
Bazel executes action (in your case C++ compile action) in a sandbox, to ensure the hermeticity. This is needed to be correct when inputs of the action change. Therefore you have to tell Bazel about all the inputs, including the system ones.
But of course you can depend on system libraries, take a look at local_repository rule. You might also find this example in bazel-discuss# thread helpful.