I would like to create a shared c++ library with Bazel using a soname.
With cmake I could set properties like:
set_target_properties(my-library
PROPERTIES
SOVERSION 3
VERSION 3.2.0
)
which would then generate
libmy-library.so -> libmy-library.so.3
libmy-library.so.3 -> libmy-library.so.3.2.0
libmy-library.so.3.2.0
However in bazel documentation I cannot find anything that would allow me to do so easily. I know that I could define the soname and version directly and pass some linkopts in the build file:
cc_binary(
name = "libmy-library.so.3.2.0",
srcs = ["my-library.cpp", "my-library.h"],
linkshared = 1,
linkopts = ["-Wl,-soname,libmy-library.so.3"],
)
which does produce libmy-library.so.3.2.0 with the correct soname, but not the .so file so it would require a whole lot of hacks around to:
create libmy-library.so.3 symlink
create libmy-library.so symlink
create some import rules such that I can build binaries that link with this library.
This does not feel like the right way. What would be the right way to solve such problem?
Related
I would like to package lets say a binary with the pkg_tar command. But I would also automatically like it to include all deps to that binary, for example all .so files from other Bezel targets that are referenced with deps. Is it possible?
pkg_tar(
name = "example",
srcs = ["//myprogram"], # This only packages myprogram target
mode = "0644",
)
Currently, this feature isn't officially supported. You have three basic options:
explicitly enumrate all deps
use one of "hacks" from https://github.com/bazelbuild/bazel/issues/1920
use undocumented include_runfiles = True feature https://github.com/bazelbuild/rules_pkg/issues/145
Setting the (undocumented) include_runfiles = True will include the shared object and any other runfiles of all the transitive dependencies.
Assume that a prec-compiled dependency is supplied by a vendor:
$ ls /opt/vendor/protobuf/lib
libprotobuf.so.3 -> libprotobuf.so.3.0.0
libprotobuf.so.3.0.0
To use this with Bazel, the following target can be created:
cc_import(
name = "protobuf",
shared_library = "lib/libprotobuf.3.0.0",
)
This way, a bazel-built application can link to the library, however, it fails to start:
error while loading shared libraries: libprotobuf.so.3: cannot open shared object file: no such file or directory
The root cause is that the actual so file has a custom SONAME field:
objdump -x libprotobuf.so.3.0.0|grep SONAME
SONAME libprotobuf.so.3
The loader will look for libprotobuf.so.3 (instead of 3.0.0), but will not find it in the sandbox, as we never told bazel about the symlink. The symlink is relative, specifying it in the cc_import target will yield to a similar error.
Is it possible to create a runnable binary with bazel that links to such a shared library that is supposed to be found via a symlink?
Setting the RPATH can be a workaround. The cc_import need to be wrapped by a cc_library:
cc_library(
name = "protobuf",
deps = [":protobuf_impl"],
linkopts = ["-Wl,-rpath=/opt/vendor/protobuf/lib"],
)
cc_import(
name = "protobuf_impl",
shared_library = "lib/libprotobuf.3.0.0",
)
This will make the binary run, but assumes that "/opt/vendor/protobuf/lib" is present on every system (incl. remote execution), and the loader during runtime still escapes the sandbox. A clearer solution would be nice.
Summary
I am currently trying to build a header only C++ (17) library for iOS. This library should output a .framework which should be consumable in a standard iOS application. The library also (of course) has other dependencies which do have source files.
Background
The existing setup uses Bazel, and up until this point has been using Tulsi for integration, but now this needs to change to versioned library releases (unless Tulsi can generate an iOS .framework complete with header files which I have not seen yet - please educate me if it exists).
Using initially ios_framework from the build_bazel_rules_apple, until it was made clear in the documentation that this should only be used within a Bazel ios target (i.e. using the ios_application) and that this can't then be taken out and used separately within another project.
I then turned towards ios_static_framework (also seen in build_bazel_rules_apple) which seems to be the correct thing, however I can never get it to maintain the header structure.
Currently
Bazel
Using ios_static_framework requires (apparently) first wrapping the normal cc_library with an objc_library, which can then be put in the deps of the ios_static_framework i.e.
# BUILD example (assume all rules are loaded and WORKSPACE is setup correctly)
cc_library(
name = "somelib",
deps = [
"//otherlib0", # A sub part of the lib I am trying to build
"//otherlib1" # Another sub part of the lib I am trying to build
],
hdrs = glob(["**/*.h"]),
visibility=["//visibility:public"]
)
objc_library(
name = "somelibwrapperobjc",
deps = [
"//:somelib"
],
hdrs = glob(["**/*.h"]),
visibility=["//visibility:public"]
)
ios_static_framework(
name = "somelibidealoutput",
families = ["iphone"],
hdrs = glob(["**/*.h"]),
umbrella_header = "libheader.h",
deps = [
"//:somelibwrapperobjc"
]
)
It appears to also build without the cc_library step to the same affect and the output is a directory with a somelibidealoutput.framework and the header file libheader.h but everything else is gone.
So this ^^ doesn't work.
CMake
So then I tried with CMake from scratch.
cmake_minimum_required(VERSION 3.1)
project(myproject C CXX)
include_directories(${PROJECT_SOURCE_DIR}/projectincludesadditional)
set(HEADER_FILES
myheader1.hpp
onelayer/myheader2.hpp
additional/header/myheader3.hpp
...
)
I then attempting the next part in two different ways. Since CMake header only libraries are simple I tried:
add_library(mylibrary INTERFACE)
target_include_directories(mylibrary INTERFACE ${HEADER_FILES})
Which could then be linked to the actual library, as well as the other alternative:
add_library(myconsuminglibrary SHARED
${HEADER_FILES})
The framework can then be built using the following:
set_target_properties(myconsuminglibrary PROPERTIES
FRAMEWORK TRUE
FRAMEWORK_VERSION CXX
LINKER_LANGUAGE CXX
MACOSX_FRAMEWORK_IDENTIFIER com.something.idontcare
MACOSX_FRAMEWORK_INFO_PLIST info.plist
VERSION 16.4.0
SOVERSION 1.0.0
PUBLIC_HEADER "myuselessumbrellaheaderthatdoesntpullanyotherheadersin.hpp"
XCODE_ATTRIBUTE_CODE_SIGN_IDENTITY "iPhone Developer"
)
But the headers don't copy across! only the specified public umbrella header does (and even if there's an include in there it won't copy that across since it's not lexing the file.
I followed a possible solution here (on cmake) in the suggested comments but it seems to just copy one layer deep and again removes the file structure.
Should I just be using a simple install and assuming thats the best way for these headers to then be used in a framework? Essentially copy them and hope for the best?
Xcode
Finally, I tried to make a traditional xcode framework starting from scratch in xcode.
I managed to get everything to build as expected (with minimal changes), but this required path changes to the xcode project etc.
However, it flattens the hierarchy of the headers and therefore all relative header paths once again become useless in the consuming application/library.
Question
Can a header only C++ library be compiled into a dynamic/static library for iOS?
If so, which build system & which config changes are needed to my CMake/Bazel build system?
Any other suggestions??
I have built a lib.so using make rule in bazel. How do I link this external lib.so to a regular cc_library rule. I tried adding it in deps, but the guide suggests that deps can have cc_library or objc_library targets.
Also, do I need to pass any specific linking options, and how can I read more about them?
In the BUILD file, create a cc_library target that imports the built lib.so for other cc_library targets to depend on:
cc_library(
name = "lib",
srcs = ["lib.so"],
linkopts = ["...", "..."],
)
See the documentation on C++ use cases for more information.
I have a hierarchical build system based on SCons. I have a root SConstruct that calls into a SConscript that builds a shared library and then into a different SConscript that builds an executable that depends on the shared library.
So here's my question: my understanding of shared libraries on linux is that when you want to do the final ld link for the executable that will be using the shared lib, the shared lib has to be included on the executable's ld command line as a source to reference it (unless it's in a standard location in which case the -l option works).
So here's something like what my SCons files look like:
=== rootdir/SConstruct
env=DefaultEnvironment()
shared_lib = SConscript('foolib/SConscript')
env.Append( LIBS=[shared_lib] )
executable = SConscript('barexec/SConscript')
=== rootdir/foolib/SConscript
env=DefaultEnvironment()
env.Append(CPPPATH=Glob('inc'))
penv = env.Clone()
penv.Append(CPPPATH=Glob('internal/inc'))
lib = penv.SharedLibrary( 'foo', source=['foo.c', 'morefoo.c']
Return("lib")
=== rootdir/barexec/SConscript
env=DefaultEnvironment()
exe = env.Program( 'bar', source=['main.c', 'bar.c', 'rod.c'] )
Return("exe")
So the hitch here is this line:
env.Append( LIBS=[shared_lib] )
This would be a great way to add generated libraries to the command line for any other libs that need them, EXCEPT that because SCons is doing a two-pass run through the SConscripts (first to generate it's dependency tree, then to do the work), rootdir/foolib/libfoo.so winds up on the command line for ALL products, EVEN libfoo.so itself:
gcc -g -Wall -Werror -o libfoo.so foo.o morefoo.o libfoo.so
So how is this best done with SCons? For now I've resorted to this hack:
=== rootdir/SConstruct
env=DefaultEnvironment()
shared_lib = SConscript('foolib/SConscript')
env['shared_lib'] = shared_lib
executable = SConscript('barexec/SConscript')
...
=== rootdir/barexec/SConscript
env=DefaultEnvironment()
exe = env.Program( 'bar', source=['main.c', 'bar.c', 'rod.c'] + env['shared_lib'] )
Return("exe")
Is there a more SCons-y way of doing this?
You should allow the shared libraries to be found by the build.
Look for the LIBPATH and RPATH variables in the SCons documentation; these are the "Scons-y" way to set up search paths so that any generated -l options find libraries properly.
Having mentioned the above, here's what you should see gcc do based on the setup of SCons (and if it doesn't, you may have to do it manually).
The -l option always finds shared libraries provided that you also give the compiler the location of the library. There are two times this is needed: at compile time (-L option) and at runtime (-rpath generated linker option).
The LIBPATH SCons setup should generate something that looks like -L/some/directory/path for the compile-time search path.
The RPATH SCons setup should generate a linker option to embed a search path; e.g. -Wl,-rpath -Wl,\$ORIGIN/../lib would embed a search path that searches relative to the executable so that executables placed in bin search in the parallel lib directory of the installation.
Here is a better way to organize your SConsctruct/SConscript files. Usually with Hierarchical builds you should share the env with the rest of the sub-directories. Notice that I cloned the main env in the barexec directory as well, so that the foolib is only used to link that binary.
=== rootdir/SConstruct
import os
env=DefaultEnvironment()
subdirs = [
'foolib',
'barexec'
]
# The exports attribute allows you to pass variables to the subdir SConscripts
for dir in subdirs:
SConscript( os.path.join(dir, 'SConscript'), exports = ['env'])
=== rootdir/foolib/SConscript
# inports the env created in the root SConstruct
#
# Any changes made to 'env' here will be reflected in
# the root/SConstruct and in the barexec/SConscript
#
Import('env')
# Adding this 'inc' dir to the include path for all users of this 'env'
env.Append(CPPPATH=Glob('inc'))
penv = env.Clone()
# Adding this include only for targets built with penv
penv.Append(CPPPATH=Glob('internal/inc'))
penv.SharedLibrary( 'foo', source=['foo.c', 'morefoo.c'])
=== rootdir/barexec/SConscript
Import('env')
clonedEnv = env.Clone()
# The foo lib will only be used for targets compiled with the clonedEnv env
# Notice that specifying '#' in a path means relative to the root SConstruct
# for each [item] in LIBS, you will get -llib on the compilation line
# for each [item] in LIBPATH, you will get -Lpath on the compilation line
clonedEnv.Append(LIBS=['foo'], LIBPATH=['#foolib'])
clonedEnv.Program( 'bar', source=['main.c', 'bar.c', 'rod.c'] )
Additional to Brady decision i use static/global variables to store targets name and path. It's allow me more control over build.
# site_scons/project.py
class Project:
APP1_NAME = "app1_name"
APP2_NAME = "app2_name"
MYLIB1_NAME = "mylib1_name"
# etc
APP_PATH = "#build/$BuildMode/bin" # BuildMode - commonly in my projects debug or release, `#` - root of project dir
LIB_PATH = "#build/$BuildMode/lib"
#staticmethod
def appPath(name) :
return os.path.join(APP_PATH, name)
#staticmethod
def libPath(name) :
return os.path.join(LIB_PATH, name)
Define targets:
from project import Project
...
env.SharedLibrary(Project.libPath(Project.MYLIB1_NAME), source=['foo.c', 'morefoo.c'])
Application:
from project import Project
...
env.Append(LIBPATH = [Project.LIB_PATH])
env.Append(LIBS = [Project.MYLIB1_NAME])
env.Program(Project.appPath(Project.MYAPP1_NAME), source=[...])
In my projects it works fine, scons automatically find depends of library without any additional commands. And if i want to change name of library i just change my Project class.
One issue that Brady's answer doesn't address is how to get correct library paths when building out-of-source using variant dirs. Here's a very similar approach that builds two different variants:
SConstruct
# Common environment for all build modes.
common = Environment(CCFLAGS=["-Wall"], CPPPATH=["#foolib/inc"])
# Build-mode specific environments.
debug = common.Clone()
debug.Append(CCFLAGS=["-O0"])
release = common.Clone()
release.Append(CCFLAGS=["-O"], CPPDEFINES=["NDEBUG"])
# Run all builds.
SConscript("SConscript", exports={"env": debug}, variant_dir="debug")
SConscript("SConscript", exports={"env": release}, variant_dir="release")
The # in the value for CPPPATH makes the include path relative to the project root instead of the variant dir.
SConscript
Import("env")
subdirs=["barexec", "foolib"]
senv = env.Clone(FOOLIBDIR=Dir("foolib"))
SConscript(dirs=subdirs, exports={"env": senv})
This root-level SConscript is required to build the subdirectories in each variant_dir.
By using the function Dir() when setting FOOLIBDIR, the library's variant build directory is resolved relative to this file rather than where it's used.
foolib/SConscript
Import("env")
penv = env.Clone()
penv.Append(CPPPATH=["internal/inc"])
penv.SharedLibrary("foo", source=["foo.c", "morefoo.c"])
It's important to clone the environment before making any changes to avoid affecting other directories.
barexec/SConscript
Import("env")
clonedEnv = env.Clone()
clonedEnv.Append(LIBPATH=["$FOOLIBDIR"], LIBS=["foo"])
clonedEnv.Program("bar", source=["main.c", "bar.c", "rod.c"])
The library's variant build dir is added to LIBPATH so both SCons and the linker can find the correct library.
Adding "foo" to LIBS informs SCons that barexec depends on foolib which must be built first, and adds the library to the linker command line.
$FOOLIBDIR should only be added to LIBPATH when "foo" is also added to LIBS – if not, barexec might be built before foolib, resulting in linker errors because the specified library path does not (yet) exist.