Assuming I have a cc_binary() rule like this:
cc_binary(
name = "App",
srcs = [
"app.cpp",
],
)
This will produce App.exe somewhere in bazel-bin. I need to write a genrule that can read App.exe and produce another version of it. How can I do that?
Edit: this is my current attempt for a genrule, but it forces the recompilation of :hello_main with a different configuration and I end up getting errors like "error C2955: 'std::pair': use of class template requires template argument list"
genrule(
name = "tbds",
srcs = [
":data",
],
outs = ["tbds.exe"],
exec_tools = ["#libpackfiles//FilePacker"],
tools = [":hello_main"],
cmd = "cp $(location :hello_main) $(OUTS) && $(location #libpackfiles//FilePacker) $(OUTS) -files $(SRCS) -append",
executable = true,
)
cc_binary(
name = "hello_main",
srcs = ["hello_main.cc"],
deps = [
":hello",
],
)
genrule(
name = "foo",
outs = ["out.txt"],
cmd = "du -sh $(location :hello_main)` > $#",
tools = [":hello_main"],
visibility = ["//visibility:public"],
)
will create a out.txt file with an output of a du -sh command. You can add an another tool to a tools attribute to run some transforming script on your binary
Related
my project A is relying on brpc:https://github.com/apache/incubator-brpc
projectA.WORKSPACE:
git_repository(
name = "brpc",
# remote = "https://github.com/apache/incubator-brpc.git",
remote = "git#github.com:apache/incubator-brpc.git",
commit = "c4de79975ea54684634d1e52d4691f96d134d34a",
shallow_since = "1671716289 +0800"
)
projectA.classB BUILD
load("#rules_cc//cc:defs.bzl", "cc_library")
cc_library(
name = "client",
srcs = glob([
"*.cpp",
]),
hdrs = glob([
"*.h",
]),
deps = [ "#brpc//:brpc"
],
visibility = ["//visibility:public"],
)
thrid_party/brpc BUILD
load("#rules_cc//cc:defs.bzl", "cc_library")
cc_library(
name = "brpc",
srcs = glob([
"src/brpc/*.cpp",
"src/bthread/*.cpp",
"src/brpc/*.h",
]),
deps = [
],
includes = [
".",
],
include_prefix = "brpc",
strip_include_prefix = "brpc",
visibility = ["//visibility:public"],
)
bazel build msg:
INFO: Invocation ID: 9aae1a04-1225-4f62-9780-d334df616f9a
WARNING: errors encountered while analyzing target '//client:client': it will not be built
INFO: Analysis succeeded for only 1 of 2 top-level targets
INFO: Analyzed 2 targets (19 packages loaded, 56 targets configured).
INFO: Found 1 target...
Target //third_party/brpc:brpc up-to-date (nothing to build)
ERROR: command succeeded, but not all targets were analyzed
INFO: Elapsed time: 0.262s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
FAILED: Build did NOT complete successfully
still can not find *.h in brpc
figure out which bazel config is wrong?
I assume the above issue because there might not be visibility of the header (.h) file to the sandbox and it gets fail. Try adding the files as mentioned here. Please reach if that did not resolve your issue.
I have created a Bazel project (C++) and have tried to include the gtkmm library in my project.
I get the below error, that header files are missing, e.g.:
Showing Recent Messages
../mediapipe/external/gtkmm/gtkmm/accelerator.cc:18:10: fatal error: 'glibmm/utility.h' file not found
#include <glibmm/utility.h>
^~~~~~~~~~~~~~~~~~
1 error generated.
Error in child process '/usr/bin/xcrun'. 1
Aspect #tulsi//:tulsi/tulsi_aspects.bzl%tulsi_outputs_aspect of //mediapipe/examples/motionize/motionize:motionize up-to-date:
bazel-bin/mediapipe/examples/motionize/motionize/motionize.tulsiouts
/private/var/tmp/_bazel_{user}/5a416b3c18d355cbf1dfbcf0102ffc0b/external/gtkmm/BUILD.bazel:7:11: Compiling gtkmm/accelerator.cc failed: (Aborted): wrapped_clang_pp failed: error executing command
Elapsed time: 10.459s, Critical Path: 0.45s
12 processes: 12 internal.
<*> Running Bazel completed in 10826.492 ms
Build did NOT complete successfully
The includes in the library do not seem to find the header files in general, not just this one. I have tried different setups and also a local repository instead of http_archive and I still get this kind of errors
Note: I am using Tulsi to create the Bazel xcode project
I have added the following in my WORKSPACE file:
http_archive(
name = "gtkmm",
strip_prefix = "gtkmm-4.8.0/gtk",
urls = ["https://github.com/GNOME/gtkmm/archive/refs/tags/4.8.0.zip"],
build_file = "#//third_party:gtkmm.BUILD",
)
I also added the gtkmm.BUILD file which looks like this:
package(default_visibility = ["//visibility:public"])
licenses(["notice"]) # BSD license
exports_files(["LICENSE"])
cc_library(
name = "main",
srcs = glob(
["**/*.cc"],
),
hdrs = glob([
"*.h",
"gtkmm/**/object_p.h",
"**/*.hg",
"**/*.h",
"**/selectiondata_private.h"
]),
includes = glob([
"*.h",
"gtkmm/**/object_p.h",
"**/*.hg",
"**/*.h",
"**/selectiondata_private.h"
]),
copts = ["-Iexternal/gtkmm-4.8.0/gtk"],
linkopts = ["-pthread"],
visibility = ["//visibility:public"],
)
Finally my BUILD file:
cc_binary(
name="motionize",
srcs=["main.cpp"],
copts = [
"-Iexternal/gtkmm-4.8.0/gtk/gtkmm",
"-Iexternal/gtkmm-4.8.0/gtk/",
"-Iexternal/gtkmm-4.8.0/gtk/src",
"-Iexternal/gtkmm-4.8.0/gtk/gtkmm/private",
],
deps=[
"//mediapipe/framework:calculator_framework",
"//mediapipe/calculators/core:pass_through_calculator",
"//mediapipe/calculators/image:image_cropping_calculator",
"//mediapipe/calculators/image:scale_image_calculator",
"//mediapipe/framework/formats:image_frame",
"//mediapipe/framework/formats:image_frame_opencv",
"//mediapipe/framework/port:opencv_core",
"//mediapipe/framework/port:opencv_highgui",
"//mediapipe/framework/port:opencv_imgproc",
"//mediapipe/framework/port:opencv_imgcodecs",
"//mediapipe/framework/port:opencv_video",
"//mediapipe/framework/port:parse_text_proto",
"//mediapipe/framework/port:status",
"#gtkmm//:main",
],
)
I have a variable declared in .bzl file example: VERSION_NUMBER = "00".
I want to override this variable from command line when i build different version of the project.
example: bazel build target --sunbversion_number= "99"
I want to change this variable because it is invoked in some function to create the name of the output paths.
exammple: for version "00" the outputfile will be: name_00.extension
and for version "99" the outputfile will be: name_99.extension
This is my example:
in .bzl file i declared:
SUBVERSION_NUMBER = "99"
and a fucntion that return a name of the file regarding to the SUBVERSION_NUMBER
def get_name(SUBVERSION_NUMBER):
return "test-"+SUBVERSION_NUMBER
OUTPUT_NAME = get_name("99")
Then my genrule():
genrule(name = "test",
srcs = [srcs],
outs = [OUTPUT_NAME+".tek"],
cmd = "cmd to generate the file" )
when i build this rule i get the output file test-99.tek
what i want is when i run bazel build test --//version=01 or any other suggested solution, i want to get output test-01.tek
Thanks
There is no way to get values from the command line into a bzl file like that, but there are a few options, depending on exactly what you want to do.
One way is to use "stamping" to set the version, and then put the versioned file in a zip file with a known name, e.g. using a genrule. Build stamping is usually used for embedding a version number and other information into the output files themselves, but can be used here too. A zip file is necessary because the output file name has to be known at load time (i.e. before any configuration data from the command line is processed at analysis time).
Something like this:
# out.txt is the original unversioned file
genrule(
name = "gen_out",
outs = ["out.txt"],
cmd = "echo foo > $#",
)
genrule(
name = "gen_versioned_out",
outs = ["out_versioned.zip"],
srcs = [":out.txt"],
tools = ["#bazel_tools//tools/zip:zipper"],
stamp = True,
cmd = """
# bazel-out/stable-status.txt is created when stamp = True
# Value of BUILD_EMBED_LABEL key comes from --embed_label on the command line
version="$$(grep BUILD_EMBED_LABEL bazel-out/stable-status.txt | cut -d ' ' -f 2)"
# Set a reasonable default if --embed_label was not specified
if [ -z "$$version" ]; then version="0"; fi
file="$$(basename $<)"
name="$${file%%.*}"
ext="$${file#*.}"
versioned_file="$${name}_$${version}.$${ext}"
# zipper allows specifying the name of the file in the zip directly, unlike the
# regular zip tool.
# c = create
# $# = output file from "outs"
# $< = input file from "srcs"
# $$versioned_file=$< = "put this file in to the archive with this name"
$(location #bazel_tools//tools/zip:zipper) c "$#" "$$versioned_file=$<"
""",
)
Then:
$ bazel build out_versioned.zip --embed_label=99
INFO: Analyzed target //:out_versioned.zip (7 packages loaded, 19 targets configured).
INFO: Found 1 target...
Target //:out_versioned.zip up-to-date:
bazel-bin/out_versioned.zip
INFO: Elapsed time: 0.340s, Critical Path: 0.10s
INFO: 3 processes: 1 internal, 2 linux-sandbox.
INFO: Build completed successfully, 3 total actions
$ unzip -l bazel-bin/out_versioned.zip
Archive: bazel-bin/out_versioned.zip
Length Date Time Name
--------- ---------- ----- ----
4 2010-01-01 00:00 out_99.txt
--------- -------
4 1 file
So that results in a zip file with a known name containing the versioned file.
Another approach is to use "User-defined build settings":
https://docs.bazel.build/versions/master/skylark/config.html#user-defined-build-settings
Something like this:
defs.bzl:
def _version_file_impl(ctx):
version = ctx.attr._version[VersionProvider].version
name, extension = ctx.file.file.basename.rsplit(".", 1)
versioned_file = ctx.actions.declare_file(
"%s_%s.%s" % (name, version, extension))
copy_args = ctx.actions.args()
copy_args.add_all([ctx.file.file, versioned_file])
ctx.actions.run_shell(
inputs = [ctx.file.file],
outputs = [versioned_file],
command = 'cp "$1" "$2"',
arguments = [copy_args])
return DefaultInfo(files = depset([versioned_file]))
version_file = rule(
implementation = _version_file_impl,
attrs = {
"file": attr.label(mandatory = True, allow_single_file = True),
"_version": attr.label(default = "//:version"),
}
)
VersionProvider = provider(fields = ["version"])
def _version_flag_impl(ctx):
return VersionProvider(version = ctx.build_setting_value)
version_flag = rule(
implementation = _version_flag_impl,
build_setting = config.int(flag = True),
)
BUILD:
load(":defs.bzl", "version_flag", "version_file")
version_flag(
name = "version",
build_setting_default = 0,
)
genrule(
name = "gen_out",
outs = ["out.txt"],
cmd = "echo foo > $#",
)
version_file(
name = "versioned_out",
file = ":out.txt",
)
Then:
$ bazel build :versioned_out --//:version=99
INFO: Analyzed target //:versioned_out (5 packages loaded, 10 targets configured).
INFO: Found 1 target...
Target //:versioned_out up-to-date:
bazel-bin/out_99.txt
INFO: Elapsed time: 0.322s, Critical Path: 0.06s
INFO: 3 processes: 1 internal, 2 linux-sandbox.
INFO: Build completed successfully, 3 total actions
So that results in a file with the version in its name. There's no label to refer to the versioned file itself though, so bazel build :out_99.txt nor srcs = [":out_99.txt"] will work, you have to go through the versioned_out target.
Update:
Here's a version that can version multiple outputs:
defs.bzl:
def _versioned_files_impl(ctx):
version = ctx.attr._version[VersionProvider].version
versioned_files = []
for f in ctx.attr.src.files.to_list():
name, extension = f.basename.rsplit(".", 1)
versioned_file = ctx.actions.declare_file(
"%s_%s.%s" % (name, version, extension))
versioned_files.append(versioned_file)
copy_args = ctx.actions.args()
copy_args.add_all([f, versioned_file])
ctx.actions.run_shell(
inputs = [f],
outputs = [versioned_file],
command = 'cp "$1" "$2"',
arguments = [copy_args])
return DefaultInfo(files = depset(versioned_files))
versioned_files = rule(
implementation = _versioned_files_impl,
attrs = {
"src": attr.label(mandatory = True),
"_version": attr.label(default = "//:version"),
}
)
VersionProvider = provider(fields = ["version"])
def _version_flag_impl(ctx):
return VersionProvider(version = ctx.build_setting_value)
version_flag = rule(
implementation = _version_flag_impl,
build_setting = config.int(flag = True),
)
BUILD:
load(":defs.bzl", "version_flag", "versioned_files")
version_flag(
name = "version",
build_setting_default = 0,
)
genrule(
name = "gen_out",
outs = ["foo.txt", "bar.txt", "baz.txt"],
cmd = """
echo foo > $(location foo.txt)
echo bar > $(location bar.txt)
echo baz > $(location baz.txt)
""",
)
versioned_files(
name = "versioned_files",
src = ":gen_out",
)
usage:
$ bazel build versioned_files --//:version=123
INFO: Analyzed target //:versioned_files (5 packages loaded, 9 targets configured).
INFO: Found 1 target...
Target //:versioned_files up-to-date:
bazel-bin/foo_123.txt
bazel-bin/bar_123.txt
bazel-bin/baz_123.txt
INFO: Elapsed time: 0.491s, Critical Path: 0.06s
INFO: 5 processes: 1 internal, 4 linux-sandbox.
INFO: Build completed successfully, 5 total actions
Update:
An example of putting the version in a cc target's define:
BUILD:
cc_binary(
name = "main",
srcs = ["main.cc"],
defines = ["VERSION=\\\"$(VERSION)\\\""],
)
main.cc:
#include <iostream>
#ifndef VERSION
#define VERSION "0.0.0"
#endif
int main() {
std::cout << "version: " << VERSION << std::endl;
return 0;
}
build and run:
$ bazel run main --define=VERSION=1.2.3
INFO: Analyzed target //:main (15 packages loaded, 52 targets configured).
INFO: Found 1 target...
Target //:main up-to-date:
bazel-bin/main
INFO: Elapsed time: 0.524s, Critical Path: 0.26s
INFO: 6 processes: 4 internal, 2 linux-sandbox.
INFO: Build completed successfully, 6 total actions
INFO: Build completed successfully, 6 total actions
version: 1.2.3
Combining the above methods, you would have to specify both --//:version=1.2.3 and --define=VERSION=1.2.3 on the command line. There's a way to have only --//:version, but it would require another Starlark rule like versioned_files which either
generates a file with the version in it that goes in the data attribute and the program reads at runtime, or
a Starlark rule which generates a C++ file with the version in it, which then gets put in the sources of a cc_library, which the rest of your program can depend on and use at compile time.
These approaches will probably require refactoring your program.
I want to add a third-party C library to tensorflow so I can use it in one of the examples. There doesn't seem to be any examples to follow so any assistance would be appreciated.
Here's my work using event2 as the third-party C library.
I've created an 'ln -s' in tensorflow/third_party to provide the event2/ headers:
ls -al ~/code/tensorflow/third_party/event2
lrwxr-xr-x 1 XXXX staff 25 May 17 16:03 ~/code/tensorflow/third_party/event2 -> /usr/local/include/event2
/usr/local/include> ls event2
BUILD bufferevent_struct.h event_compat.h listener.h thread.h
buffer.h dns.h event_struct.h rpc.h util.h
buffer_compat.h dns_compat.h http.h rpc_compat.h visibility.h
bufferevent.h dns_struct.h http_compat.h rpc_struct.h
bufferevent_compat.h event-config.h http_struct.h tag.h
bufferevent_ssl.h event.h keyvalq_struct.h tag_compat.h
third_party/event2/BUILD:
licenses(["notice"])
cc_library(
name = "event2",
srcs = glob( [ "*.h" ] ),
visibility = [ "//visibility:public" ],
)
In tensorflow/examples/label_image/BUILD, I added the reference to libevent and my test files that use the events2 library:
cc_binary(
name = "label_image",
srcs = [
"main.cc",
"my_new_file_using_events.c",
"my_new_file_using_events.h",
],
linkopts = ["-lm", ],
copts = [ "-Ithird_party", ],
deps = [
"//tensorflow/cc:cc_ops",
"//tensorflow/core:framework_internal",
"//tensorflow/core:tensorflow",
"//third_party/event2:event2",
],
)
It compiles fine but when I run the binary I get the following errors:
dyld: lazy symbol binding failed: Symbol not found: _event_base_new
Referenced from: /Users/XXXX/code/tensorflow/bazel-bin/tensorflow/examples/label_image/label_image
Expected in: flat namespace
dyld: Symbol not found: _event_base_new
Referenced from: /Users/XXXX/code/tensorflow/bazel-bin/tensorflow/examples/label_image/label_image
Expected in: flat namespace
[1] 41395 trace trap bazel-bin/tensorflow/examples/label_image/label_image
The libevent.a, libevent.dylib and other libevent* libs are in /usr/local/lib. according to nm, event_base undefined.
nm -f label_image | grep event_base
U _event_base_dispatch
U _event_base_new
How do I resolve this linkage error? Thanks.
Aren't we missing event2 sources there? Also, I think you want to put .h into hdrs attribute.
cc_library(
name = "event2",
hdrs = glob( [ "*.h" ] ),
srcs = glob( [ "*.cpp" ] ),
visibility = [ "//visibility:public" ],
)
Does this help?
I created a filegroup rule to have in one label all the .proto files of a library.
filegroup(
name = "protos_all_src",
srcs = glob(
["**/*.proto"],
exclude = [
"protobuf/worker.proto",
"protobuf/worker_service.proto",
"protobuf/master.proto",
"protobuf/master_service.proto",
],
)
)
tf_proto_library(
name = "protos_all",
srcs = ":protos_all_src",
),
...
)
but strangely with this format this other rule fails:
cc_library(
name = "lib_internal",
srcs = glob(
[
"lib/**/*.h",
"lib/**/*.cc",
"platform/*.h",
"platform/*.cc",
] + tf_additional_lib_srcs(),
exclude = [
"**/*test*",
"platform/**/cuda.h",
"platform/**/stream_executor.h",
],
),
hdrs = [
"lib/core/blocking_counter.h",
"lib/core/refcount.h",
"lib/gtl/edit_distance.h",
"lib/gtl/int_type.h",
"lib/gtl/iterator_range.h",
"lib/gtl/manual_constructor.h",
"lib/gtl/top_n.h",
"lib/io/iterator.h",
"lib/io/match.h",
"lib/jpeg/jpeg_handle.h",
"lib/png/png_io.h",
"lib/random/random.h",
"lib/random/random_distributions.h",
"lib/random/weighted_picker.h",
"lib/strings/ordered_code.h",
"lib/strings/proto_text_util.h",
"lib/strings/regexp.h",
"lib/strings/scanner.h",
"lib/wav/wav_io.h",
"platform/demangle.h",
"platform/denormal.h",
"platform/platform.h",
"platform/tensor_coding.h",
"platform/tracing.h",
],
copts = tf_copts(),
linkopts = ["-ldl"],
deps = [
":protos_all_cc",
"//tensorflow/core/platform/default/build_config:platformlib",
"//third_party/eigen3",
],
)
note the protos_all_cc rule as a dep.
if I revert back to
tf_proto_library(
name = "protos_all",
srcs = glob(
["**/*.proto"],
exclude = [
"protobuf/worker.proto",
"protobuf/worker_service.proto",
"protobuf/master.proto",
"protobuf/master_service.proto",
],
),
everything works fine.
I would expect that the first and the second format would be completely equal. What I am missing ?
EDIT:
The tf_proto_library is defined here:
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/platform/default/build_config.bzl#L26
and the cc_proto_library is defined here:
https://github.com/google/protobuf/blob/master/protobuf.bzl#L109
bazel version:
Build label: 0.2.3-homebrew
Build target: bazel-out/local-fastbuild/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue May 17 15:07:52 2016 (1463497672)
Build timestamp: 1463497672
Build timestamp as int: 1463497672
Could you paste your actual code and the error message you're getting? It looks like there are several problems and I'm not sure which are copy-paste errors and which are code.
I'd guess the issue you're having is that you cannot list files (nor filegroups) as deps in a cc_library rule, and ":protos_all_cc" is a filegroup. deps must be other cc_libraries (it's "things this rule should link together"). But it's very hard to tell with out the error message nor definition of protos_all_cc.
The srcs attribute should be a list, so your definition of protos_all should be
tf_proto_library(
name = "protos_all",
srcs = [":protos_all_src"], # note the list here
...
)
Maybe this is just a typo in the question, and the real problem is something else?
I get weird Permission denied problems when I try your filegroup approach.