Tensorflow: dynamic linking with shared library - c++

I'm doing a shared library which uses Tensorflow. For now I placed it in Tensorflow's source tree as subproject with the following BUILD file:
cc_binary(
name = "recognizer.so",
srcs = glob(["recognizer.cpp"]),
linkshared = 1,
deps = [
"//tensorflow:сore"
],
)
Everything links together but I end up with a shared library about 94 megabytes in size and not depended on libtensorflow_cc.so. Actually there is even no such binary as libtensorflow_cc.so built.
There is a target //tensorflow:libtensorflow_cc.so . It is declared as cc_binary which means (according to bazel) I cannot depend on it. Moreover this target is actually non-public, which means that I can build it but not refer to it from another subproject. At least with bazel.
So, is there any way to do such a simple thing?

I cannot comment on why libtensorflow.so or libtensorflow_cc.so are :internal. But there is a trick you can do in Bazel to be able to depend on a shared library created by cc_binary: declare it as a source of a cc rule.
cc_binary(
name = "liba.so",
srcs = [ "a.cc" ],
linkshared = 1
)
cc_binary(
name = "main",
srcs = [ "main.cc", "liba.so" ],
)
Now this is highly unsupported :) In fact, we are going to change how we handle shared libraries in the following months, so I can almost promise you it will break. You can subscribe to https://github.com/bazelbuild/bazel/issues/1920 or follow bazel-dev# to be updated.

Related

Bazel cc_library with no srcs doesn't compile on its own

I have a cc_library that is header-only. Whenever I try to compile such library by itself, it won't actually compile anything. I purposely put some errors to try to get such errors on compilation, but bazel doesn't actually compile anything. Here's a small example.
// test.h
This should not compile fdsafdsafdsa
int foo() { return 1; }
# BUILD
cc_library(
name = 'test',
hdrs = ['test.h']
)
// bazel build :test
INFO: Analyzed target //:test (2 packages loaded, 3 targets configured).
INFO: Found 1 target...
Target //:test up-to-date (nothing to build)
INFO: Elapsed time: 0.083s, Critical Path: 0.00s
INFO: 0 processes.
INFO: Build completed successfully, 1 total action
Is this behavior intended?
I also ran the same experiment but splitting the .h and .cc files, and in that case, I got the error when I compiled.
cc_library (other rules as well incl. pkg_tar for instance) does not have to have any sources. This is also valid:
cc_library(
name = "empty",
srcs = [],
)
And it is actually quite useful too. You may have configurable attributes such as deps (or srcs) where actual content is only applicable for certain conditions:
cc_binary(
name = "mybinary",
srcs = ["main.c"],
deps = select({
":platform1": [":some_plat1_only_lib"],
":platform2": [":empty"], # as defined in the above snippet
}),
)
Or (since above you could have just as well used [] for :platform2 deps) where you have a larger tree and you expect developers to just depend on //somelib:somelib, you could use this empty library through an alias to give them a single label without having to worry about all the platform specific details and how providing certain function is handled where:
# somelib/BUILD:
alias(
name = "somelib",
actual = select({
":platform1": [":some_plat1_only_lib"],
":platform2": [":empty"], # as defined in the above snippet
}),
visibility = ["//visibility:public"], # set appropriately
)
And mybinary or any other target could now say:
cc_binary(
name = "mybinary",
srcs = ["main.c"],
deps = ["//somelib"],
)
And of course, as the other answer here states, there are header only libraries.
Also in the example you've used in your question. (bazel or not) you would normally not (and it would not be very useful either) compile a header file on its own. You would only use its content and only then see the compiler fails as you attempt to build a source the header is #included from. That is for bazel build to fail, another target would have to depend on test and #include "test.h".
A header only library means that you don't need to build anything. Simply include the headers you need in your program and use them.

How to link a library build using make rule in bazel

I have built a lib.so using make rule in bazel. How do I link this external lib.so to a regular cc_library rule. I tried adding it in deps, but the guide suggests that deps can have cc_library or objc_library targets.
Also, do I need to pass any specific linking options, and how can I read more about them?
In the BUILD file, create a cc_library target that imports the built lib.so for other cc_library targets to depend on:
cc_library(
name = "lib",
srcs = ["lib.so"],
linkopts = ["...", "..."],
)
See the documentation on C++ use cases for more information.

Using Bazel with local boost installation

I would just like to know if someone has tried doing this?
I am currently using nelhage/rules_boost for my boost dependencies(just to make some things compile for the meantime), but since the code I'm working with is only 100% compatible with 1.55 I cannot use his rules for long.
I could also try adapting his code to work with boost 1.55, but I think it would make it a lot easier if I just make Bazel depend on an installation of boost since I am also working with containers.
I usually use boost as pre-built external dependency with Bazel. I just reference the local installation in my WORKSPACE file and then create a BUILD file for it, e.g.:
# WORKSPACE file
new_local_repository(
name = "boost",
path = "/your/path/to/boost",
build_file = "third_party/boost.BUILD",
)
In the BUILD file you can choose to split headers and libs into separate rules or combine them together. In the following example I keep all the headers as a rule and separate libraries into different rules:
# third_party/boost.BUILD
cc_library(
name = "boost-headers",
hdrs = glob(["include/boost/**"]),
visibility = ["//visibility:public"],
includes = ['include'],
)
cc_library(
name = "boost-atomic",
srcs = ["lib/libboost_atomic.a"],
visibility = ["//visibility:public"],
)
cc_library(
name = "boost-chrono",
srcs = ["lib/libboost_chrono.a"],
visibility = ["//visibility:public"],
)
...
Then in my binary/library I pick-up the dependencies:
cc_binary(
name = 'main',
srcs = ['main.cc'],
deps = [
'#boost//:boost-headers',
'#boost//:boost-regex',
]
)
This should also work is you have boost installed into /usr/include / /usr/lib, but I haven't tried to be honest.
Hope this helps.

Can I provide a relative deps path in my bazel target?

When I specify build rules in bazel, my dependencies are either full paths (from the root of the repo), or just the target name (since its in the same directory):
cc_binary(
name = "program",
srcs = ["main.cpp"],
deps = ["//a/full/path/to/the/library:lib",
"foo"]
)
Assume I'm writing a build rule from directory "the".
I was hoping to do something like this:
cc_binary(
name = "program",
srcs = ["main.cpp"],
deps = ["library:lib",
"foo"]
)
This does not seem to be possible. Is there some kind of way, where I can specify the target deeper starting from the location of the BUILD file?
You cannot.
Relative labels cannot be used to refer to targets in other packages;
the repository identifier and package name must always be specified in this case.
From Bazel labels documentation

How to use select to properly detect whether I am building C++ code in Windows or Linux?

I am writing a sample C++ project that uses Bazel to serve as an example idiom for other collaborators to follow.
Here is the repository: https://github.com/thinlizzy/bazelexample
I am interested to know if I am doing it 'right', more specifically about this file: https://github.com/thinlizzy/bazelexample/blob/38cc07931e58ff5a888dd6a83456970f76d7e5b3/demo/BUILD
when regarding to pick particular implementations.
cc_library(
name = "demo",
srcs = ["demo.cpp"],
deps = [
"//example:frontend",
],
)
cc_binary(
name = "main_win",
deps = [
":demo",
"//example:impl_win",
],
)
cc_binary(
name = "main_linux",
deps = [
":demo",
"//example:impl_linux",
],
)
Is this following a correct/expected idiom for Bazel projects? I am doing this way already for other projects, by concentrating all the platform-specific dependencies in separate targets and then the binaries just depend on them.
Someone in bazel-discuss list told me to use select, instead, but my attempts failed to 'detect' the operating system. I'm sure I did something wrong, but the lack of info and examples don't tell me much how to use it properly.
#bazel_tools contains predefined platform conditions:
$ bazel query #bazel_tools//src/conditions:all
#bazel_tools//src/conditions:windows_msys
#bazel_tools//src/conditions:windows_msvc
#bazel_tools//src/conditions:windows
#bazel_tools//src/conditions:remote
#bazel_tools//src/conditions:host_windows_msys
#bazel_tools//src/conditions:host_windows_msvc
#bazel_tools//src/conditions:host_windows
#bazel_tools//src/conditions:freebsd
#bazel_tools//src/conditions:darwin_x86_64
#bazel_tools//src/conditions:darwin
You can use them directly in the BUILD file:
cc_library(
name = "impl",
srcs = ["Implementation.cpp"] + select({
"#bazel_tools//src/conditions:windows": ["ImplementationWin.cpp"],
"#bazel_tools//src/conditions:darwin": ["ImplementationMacOS.cpp"],
"//conditions:default": ["ImplementationLinux.cpp"],
}),
# .. same for hdrs and data
)
cc_binary(
name = "demo",
deps = [":impl"],
)
See the documentation for select for details on the syntax.
Add a .bazelrc to your project. Add the lines build:vs2019 --cxxopt=/std:c++14 and build:gcc --cxxopt=-std=c++14. Build your code bazel build --config=msvc //... or bazel build --config=gcc //....
#Vertexwahn's answer caused some confusion on my end, so I hope this answer helps clarify a bit. While his answer does not directly tie into the question, it may be of use to others trying to build on entirely different platforms without file specific inclusions.
Here is a link to where I answered that particular question: How do I specify portable build configurations for different operating systems for Bazel?