In the code base I am working with we use the oracle instant client library as a third party dependency in Bazel as follows:
cc_library(
name = "instant_client_basiclite",
srcs = glob(["*.so*"]),
visibility = ["//visibility:public"],
)
The library looks as this:
$ bazel query 'deps(#instant_client_basiclite//:instant_client_basiclite)'
#instant_client_basiclite//:instant_client_basiclite
#instant_client_basiclite//:liboramysql.so
#instant_client_basiclite//:libociicus.so
#instant_client_basiclite//:libocci.so.21.1
#instant_client_basiclite//:libocci.so
#instant_client_basiclite//:libnnz21.so
#instant_client_basiclite//:libclntshcore.so
...
It works as far as linking is concerned, but it seems that the path to the library is still needed because otherwise I get a run time error (oracle error 1804). The error can be solved by setting any of the environment variables ORACLE_HOME or LD_LIBRARY_PATH. In fact for the IBM WebSphere MQ there is the same need (character encoding table files need to be found).
ldd on a binary points to .../bazel-bin/app/../../../_solib_k8/_U#instant_Uclient_Ubasiclite_S_S_Cinstant_Uclient_Ubasiclite___U/libocci.so.21.1
How can I set those needed path variables so that bazel test, bazel run and Bazel container image rules work?
One possibility is to add the following command line option:
--test_env=ORACLE_HOME="$(bazel info output_base)/external/instant_client_basiclite"
It is a pity that it cannot be put in .bazelrc.
Here is my meson.build:
project('Adventum', 'cpp', version : '0.1.0', license : '', default_options : ['cpp_std=c++2a'])
glfw = subproject('glfw').get_variable('glfw_dep')
vulkan = dependency('vulkan')
sources = ['src/main.cpp', 'src/render/window.cpp']
exe = executable('Adventum', sources, include_directories : 'src', dependencies : [glfw, vulkan])
and here is my 'subproject" (wrap):
[wrap-git]
url = https://github.com/henry4k/glfw-meson.git
revision = head
I'm using the GLFW library but I have to pull from a third-party fork that supports meson. This all works fine, and meson successfully downloads and builds the GLFW fork, but even though meson should clearly know that my project depends on the GLFW library it doesn't actually place the GLFW binaries alongside my projects executable, so it builds the dependency but my program cannot run because it doesn't find the necessary binaries.
Is there something I must specify in my meson.build for it to automatically place the binaries next to my executeable?
if you do not specify a directory name or a [provide] section with dependency_names, the file name of the .wrap will determine how the subproject is called.
So, in case your wrap file was called glfw-meson.wrap your subproject could be referenced with glfw-meson only.
Better practice is to specify directory:
[wrap-git]
directory = glfw
url = https://github.com/henry4k/glfw-meson.git
revision = head
or dependency_names:
[wrap-git]
url = https://github.com/henry4k/glfw-meson.git
revision = head
[provide]
dependency_names = glfw
Then, you should be able to call subproject('glfw')
I would like to create a shared c++ library with Bazel using a soname.
With cmake I could set properties like:
set_target_properties(my-library
PROPERTIES
SOVERSION 3
VERSION 3.2.0
)
which would then generate
libmy-library.so -> libmy-library.so.3
libmy-library.so.3 -> libmy-library.so.3.2.0
libmy-library.so.3.2.0
However in bazel documentation I cannot find anything that would allow me to do so easily. I know that I could define the soname and version directly and pass some linkopts in the build file:
cc_binary(
name = "libmy-library.so.3.2.0",
srcs = ["my-library.cpp", "my-library.h"],
linkshared = 1,
linkopts = ["-Wl,-soname,libmy-library.so.3"],
)
which does produce libmy-library.so.3.2.0 with the correct soname, but not the .so file so it would require a whole lot of hacks around to:
create libmy-library.so.3 symlink
create libmy-library.so symlink
create some import rules such that I can build binaries that link with this library.
This does not feel like the right way. What would be the right way to solve such problem?
I'm compiling my program using Bazel, and I have a dependency on Intel TBB.
Intel TBB only provide dynamic libraries (no static) for good reasons (if you are curious: ctrl+f static here).
In my bazel WORKSPACE I defined that rule :
new_local_repository(
name = "inteltbb",
path = "./third_party/intel_tbb",
build_file = "./third_party/inteltbb.BUILD",
)
and in my "inteltbb.BUILD" I have:
cc_library(
name = "dynamic_lib",
srcs = ["build/macos_intel64_clang_cc8.1.0_os10.12.5_debug/libtbb_debug.dylib"],
hdrs = glob(["include/**/*.h"]),
visibility = ["//visibility:public"],
strip_include_prefix = "include/"
)
Then in my final program (under the cc_binary rule) I have:
deps = [
"#inteltbb//:dynamic_lib", [...]
It compile properly, find the headers successfully but at runtime it crashes saying:
____Running command line: bazel-bin/build-game
dyld: Library not loaded: #rpath/libtbb_debug.dylib
Referenced from: /private/var/tmp/_bazel_dmabin/526b91f44cfc47d856222c6b20765cc8/execroot/__main__/bazel-out/darwin_x86_64-fastbuild/bin/build-game
Reason: image not found
I check under the "bazel-bin" folder (where the executable is symlink for runtime execution: bazel-bin/build-game.runfiles/main/) and I do have:
the symlink for the executable (build-game)
a folder called (brace yourself): _solib_darwin_x86_64/_U#inteltbb_S_S_Cdynamic_Ulib___Uexternal_Sinteltbb_Sbuild_Smacos_Uintel64_U
clang_Ucc8.1.0_Uos10.12.5_Udebug that contain my libtbb_debug.dylib for intel tbb.
Also when I run: otool -l build-game | grep LC_RPATH -A2 the result is:
cmd LC_RPATH
cmdsize 152
path $ORIGIN/_solib_darwin_x86_64/_U#inteltbb_S_S_Cdynamic_Ulib___Uexternal_Sinteltbb_Sbuild_Smacos_Uintel64_Uclang_Ucc8.1.0_Uos10.12.5_Udebug (offset 12)
I don't understand why my executable doesn't find my dylib. I can't find anything wrong about the otool output but I'm not a mac expert (at all).
Any idea is welcome.
[edit]
If I edit the executable using otool to replace the path to the dylib like that:
#executable_path/_solib_darwin_x86_64/_U#inteltbb_S_S_Cdynamic_Ulib___Uexternal_Sinteltbb_Slib/libtbb_debug.dylib
Then it works fine. I doesn't feel right that bazel force me to do that at every compilation :/
[edit 2]
A few people asked me the line I used to modify the path of the lib in the executable.
First run:
otool -L build-game
To spot the path of the lib you want to change. In my case it was: #rpath/libtbb_debug.dylib.
After that run:
install_name_tool -change #rpath/libtbb_debug.dylib #executable_path/_solib_darwin_x86_64/_U#inteltbb_S_S_Cdynamic_Ulib___Uexternal_Sinteltbb_Slib/libtbb_debug.dylib build-game
to change the path. Note that you can use #executable_path to make it relative to the binary path or you can put an absolute path, up to you.
I think this is already a known issue, and it was just fixed. Sorry for the trouble.
[edit: updated after submitting the fix]
I have a hierarchical build system based on SCons. I have a root SConstruct that calls into a SConscript that builds a shared library and then into a different SConscript that builds an executable that depends on the shared library.
So here's my question: my understanding of shared libraries on linux is that when you want to do the final ld link for the executable that will be using the shared lib, the shared lib has to be included on the executable's ld command line as a source to reference it (unless it's in a standard location in which case the -l option works).
So here's something like what my SCons files look like:
=== rootdir/SConstruct
env=DefaultEnvironment()
shared_lib = SConscript('foolib/SConscript')
env.Append( LIBS=[shared_lib] )
executable = SConscript('barexec/SConscript')
=== rootdir/foolib/SConscript
env=DefaultEnvironment()
env.Append(CPPPATH=Glob('inc'))
penv = env.Clone()
penv.Append(CPPPATH=Glob('internal/inc'))
lib = penv.SharedLibrary( 'foo', source=['foo.c', 'morefoo.c']
Return("lib")
=== rootdir/barexec/SConscript
env=DefaultEnvironment()
exe = env.Program( 'bar', source=['main.c', 'bar.c', 'rod.c'] )
Return("exe")
So the hitch here is this line:
env.Append( LIBS=[shared_lib] )
This would be a great way to add generated libraries to the command line for any other libs that need them, EXCEPT that because SCons is doing a two-pass run through the SConscripts (first to generate it's dependency tree, then to do the work), rootdir/foolib/libfoo.so winds up on the command line for ALL products, EVEN libfoo.so itself:
gcc -g -Wall -Werror -o libfoo.so foo.o morefoo.o libfoo.so
So how is this best done with SCons? For now I've resorted to this hack:
=== rootdir/SConstruct
env=DefaultEnvironment()
shared_lib = SConscript('foolib/SConscript')
env['shared_lib'] = shared_lib
executable = SConscript('barexec/SConscript')
...
=== rootdir/barexec/SConscript
env=DefaultEnvironment()
exe = env.Program( 'bar', source=['main.c', 'bar.c', 'rod.c'] + env['shared_lib'] )
Return("exe")
Is there a more SCons-y way of doing this?
You should allow the shared libraries to be found by the build.
Look for the LIBPATH and RPATH variables in the SCons documentation; these are the "Scons-y" way to set up search paths so that any generated -l options find libraries properly.
Having mentioned the above, here's what you should see gcc do based on the setup of SCons (and if it doesn't, you may have to do it manually).
The -l option always finds shared libraries provided that you also give the compiler the location of the library. There are two times this is needed: at compile time (-L option) and at runtime (-rpath generated linker option).
The LIBPATH SCons setup should generate something that looks like -L/some/directory/path for the compile-time search path.
The RPATH SCons setup should generate a linker option to embed a search path; e.g. -Wl,-rpath -Wl,\$ORIGIN/../lib would embed a search path that searches relative to the executable so that executables placed in bin search in the parallel lib directory of the installation.
Here is a better way to organize your SConsctruct/SConscript files. Usually with Hierarchical builds you should share the env with the rest of the sub-directories. Notice that I cloned the main env in the barexec directory as well, so that the foolib is only used to link that binary.
=== rootdir/SConstruct
import os
env=DefaultEnvironment()
subdirs = [
'foolib',
'barexec'
]
# The exports attribute allows you to pass variables to the subdir SConscripts
for dir in subdirs:
SConscript( os.path.join(dir, 'SConscript'), exports = ['env'])
=== rootdir/foolib/SConscript
# inports the env created in the root SConstruct
#
# Any changes made to 'env' here will be reflected in
# the root/SConstruct and in the barexec/SConscript
#
Import('env')
# Adding this 'inc' dir to the include path for all users of this 'env'
env.Append(CPPPATH=Glob('inc'))
penv = env.Clone()
# Adding this include only for targets built with penv
penv.Append(CPPPATH=Glob('internal/inc'))
penv.SharedLibrary( 'foo', source=['foo.c', 'morefoo.c'])
=== rootdir/barexec/SConscript
Import('env')
clonedEnv = env.Clone()
# The foo lib will only be used for targets compiled with the clonedEnv env
# Notice that specifying '#' in a path means relative to the root SConstruct
# for each [item] in LIBS, you will get -llib on the compilation line
# for each [item] in LIBPATH, you will get -Lpath on the compilation line
clonedEnv.Append(LIBS=['foo'], LIBPATH=['#foolib'])
clonedEnv.Program( 'bar', source=['main.c', 'bar.c', 'rod.c'] )
Additional to Brady decision i use static/global variables to store targets name and path. It's allow me more control over build.
# site_scons/project.py
class Project:
APP1_NAME = "app1_name"
APP2_NAME = "app2_name"
MYLIB1_NAME = "mylib1_name"
# etc
APP_PATH = "#build/$BuildMode/bin" # BuildMode - commonly in my projects debug or release, `#` - root of project dir
LIB_PATH = "#build/$BuildMode/lib"
#staticmethod
def appPath(name) :
return os.path.join(APP_PATH, name)
#staticmethod
def libPath(name) :
return os.path.join(LIB_PATH, name)
Define targets:
from project import Project
...
env.SharedLibrary(Project.libPath(Project.MYLIB1_NAME), source=['foo.c', 'morefoo.c'])
Application:
from project import Project
...
env.Append(LIBPATH = [Project.LIB_PATH])
env.Append(LIBS = [Project.MYLIB1_NAME])
env.Program(Project.appPath(Project.MYAPP1_NAME), source=[...])
In my projects it works fine, scons automatically find depends of library without any additional commands. And if i want to change name of library i just change my Project class.
One issue that Brady's answer doesn't address is how to get correct library paths when building out-of-source using variant dirs. Here's a very similar approach that builds two different variants:
SConstruct
# Common environment for all build modes.
common = Environment(CCFLAGS=["-Wall"], CPPPATH=["#foolib/inc"])
# Build-mode specific environments.
debug = common.Clone()
debug.Append(CCFLAGS=["-O0"])
release = common.Clone()
release.Append(CCFLAGS=["-O"], CPPDEFINES=["NDEBUG"])
# Run all builds.
SConscript("SConscript", exports={"env": debug}, variant_dir="debug")
SConscript("SConscript", exports={"env": release}, variant_dir="release")
The # in the value for CPPPATH makes the include path relative to the project root instead of the variant dir.
SConscript
Import("env")
subdirs=["barexec", "foolib"]
senv = env.Clone(FOOLIBDIR=Dir("foolib"))
SConscript(dirs=subdirs, exports={"env": senv})
This root-level SConscript is required to build the subdirectories in each variant_dir.
By using the function Dir() when setting FOOLIBDIR, the library's variant build directory is resolved relative to this file rather than where it's used.
foolib/SConscript
Import("env")
penv = env.Clone()
penv.Append(CPPPATH=["internal/inc"])
penv.SharedLibrary("foo", source=["foo.c", "morefoo.c"])
It's important to clone the environment before making any changes to avoid affecting other directories.
barexec/SConscript
Import("env")
clonedEnv = env.Clone()
clonedEnv.Append(LIBPATH=["$FOOLIBDIR"], LIBS=["foo"])
clonedEnv.Program("bar", source=["main.c", "bar.c", "rod.c"])
The library's variant build dir is added to LIBPATH so both SCons and the linker can find the correct library.
Adding "foo" to LIBS informs SCons that barexec depends on foolib which must be built first, and adds the library to the linker command line.
$FOOLIBDIR should only be added to LIBPATH when "foo" is also added to LIBS – if not, barexec might be built before foolib, resulting in linker errors because the specified library path does not (yet) exist.