I am performing a simple c++ compile of my project using Bazel 5.3.0. from a Docker image running in a GITLab Ci Job. I am able to compile successfully on my host by running:
docker exec -it pdi-container /bin/bash -c "bazel build --config-cpp_config //main:verify_tool"
However, when I compile this project from GitLab I receive the below error from GITLab:
bazel --output_base output build --genrule_strategy=standalone --spawn_strategy=standalone --
verbose_failures --sandbox_debug --config=cpp_config //main:verify_tool
Loading:
Loading: 0 packages loaded
Analyzing: target //main:verify_tool (0 packages loaded, 0 targets configured)
INFO: Analyzed target //main:verify_tool (0 packages loaded, 6 targets configured).
INFO: Found 1 target...
[0 / 3] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[3 / 8] Compiling main/pm.cpp; 1s local ... (4 actions running)
[3 / 8] Compiling main/pm.cpp; 11s local ... (4 actions running)
ERROR: /builds/avsw/programs/new_shepard/PDI/pdi-verify-tool/main/BUILD:3:10: Compiling
main/process_verify_data_helper.cpp failed: (Exit 1): g++ failed: error executing command
(cd /builds/avsw/programs/new_shepard/PDI/pdi-verify-tool/output/execroot/__main__ && \
exec env - \
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
PWD=/proc/self/cwd \
/usr/bin/g++ -MD -MF bazel-out/64-fastbuild/bin/main/_objs/verify_tool/process_verify_data_helper.d '-frandom-seed=bazel-out/64- fastbuild/bin/main/_objs/verify_tool/process_verify_data_helper.o' -iquote . -iquote bazel-out/64-fastbuild/bin -iquote external/bazel_tools -iquote bazel-out/64-fastbuild/bin/external/bazel_tools -g -Wall -Werror '-std=c++17' - I/usr/include/jinja2cpp/include -I/usr/local/include/boost -c main/process_verify_data_helper.cpp -o bazel-out/64-fastbuild/bin/main/_objs/verify_tool/process_verify_data_helper.o)# Configuration: 757e432bd578388925ee038e81d419bbf7bedee5ead3ef18301b28a7f1139389 # Execution platform: #local_config_platform//:host g++: fatal error: Killed signal terminated program cc1plus compilation terminated.Target //main:verify_tool failed to build
The only difference is I am building the project from GITLab CI. Any help would be appreciated.
Related
I'm installing grpc via vcpkg on Ubuntu 16.04 inside docker.
./vcpkg install vcpkg:x64-linux
on the stdout:
-- Building x64-linux-dbg
CMake Error at scripts/cmake/vcpkg_execute_build_process.cmake:129 (message):
Command failed: /home/techstar/vcpkg/downloads/tools/cmake-3.24.0-linux/cmake-3.24.0-linux-x86_64/bin/cmake --build . --config Debug --target install -- -v -j13
Working Directory: /home/techstar/vcpkg/buildtrees/grpc/x64-linux-dbg
See logs for more information:
/home/techstar/vcpkg/buildtrees/grpc/install-x64-linux-dbg-out.log
Call Stack (most recent call first):
installed/x64-linux/share/vcpkg-cmake/vcpkg_cmake_build.cmake:74 (vcpkg_execute_build_process)
installed/x64-linux/share/vcpkg-cmake/vcpkg_cmake_install.cmake:16 (vcpkg_cmake_build)
ports/grpc/portfile.cmake:69 (vcpkg_cmake_install)
scripts/ports.cmake:147 (include)
error: building grpc:x64-linux failed with: BUILD_FAILED
error: Please ensure you're using the latest port files with `git pull` and `vcpkg update`.
Then check for known issues at:
https://github.com/microsoft/vcpkg/issues?q=is%3Aissue+is%3Aopen+in%3Atitle+grpc
You can submit a new issue at:
https://github.com/microsoft/vcpkg/issues/new?template=report-package-build-failure.md&title=[grpc]+Build+error
Include '[grpc] Build error' in your bug report title, the following version information in your bug description, and attach any relevant failure logs from above.
vcpkg-tool version: 2022-09-20-522aa94e9d261c7d7b2f079bf2591ca62df5c714
vcpkg-scripts version: fe57571 2022-10-05 (13 hours ago)
Inside the log:
FAILED: CMakeFiles/grpc.dir/src/core/lib/slice/percent_encoding.cc.o
/usr/bin/c++ -DCARES_STATICLIB -I/home/techstar/vcpkg/buildtrees/grpc/src/v1.49.0-b3763f9139.clean/include -I/home/techstar/vcpkg/buildtrees/grpc/src/v1.49.0-b3763f9139.clean -I/home/techstar/vcpkg/buildtrees/grpc/src/v1.49.0-b3763f9139.clean/third_party/address_sorting/include -I/home/techstar/vcpkg/buildtrees/grpc/src/v1.49.0-b3763f9139.clean/src/core/ext/upb-generated -I/home/techstar/vcpkg/buildtrees/grpc/src/v1.49.0-b3763f9139.clean/src/core/ext/upbdefs-generated -I/home/techstar/vcpkg/buildtrees/grpc/src/v1.49.0-b3763f9139.clean/third_party/xxhash -isystem /home/techstar/vcpkg/installed/x64-linux/include -fPIC -g -fPIC -pthread -std=c++14 -MD -MT CMakeFiles/grpc.dir/src/core/lib/slice/percent_encoding.cc.o -MF CMakeFiles/grpc.dir/src/core/lib/slice/percent_encoding.cc.o.d -o CMakeFiles/grpc.dir/src/core/lib/slice/percent_encoding.cc.o -c /home/techstar/vcpkg/buildtrees/grpc/src/v1.49.0-b3763f9139.clean/src/core/lib/slice/percent_encoding.cc
/home/techstar/vcpkg/buildtrees/grpc/src/v1.49.0-b3763f9139.clean/src/core/lib/slice/percent_encoding.cc:48:20: in constexpr expansion of ‘((grpc_core::{anonymous}::UrlTable*)(& grpc_core::{anonymous}::g_url_table))->grpc_core::{anonymous}::UrlTable::UrlTable()’
/home/techstar/vcpkg/buildtrees/grpc/src/v1.49.0-b3763f9139.clean/src/core/lib/slice/percent_encoding.cc:38:41: in constexpr expansion of ‘((grpc_core::{anonymous}::UrlTable*)this)->grpc_core::{anonymous}::UrlTable::<anonymous>.grpc_core::BitSet<kTotalBits, kUnitBits>::set<256ul, 64ul>(i)’
/home/techstar/vcpkg/buildtrees/grpc/src/v1.49.0-b3763f9139.clean/src/core/lib/slice/percent_encoding.cc:48:20: internal compiler error: in fold_binary_loc, at fold-const.c:9921
constexpr UrlTable g_url_table;
^
Please submit a full bug report,
with preprocessed source if appropriate.
Am I missing some dependenciy or is this a bug? Dose anyone have any Idea?
I have trained a keras machine learning model in python tensorflow 1.14 and my goal is to use that trained model to aid in performing fluid simulation calculations in C++. For this reason, I would like to use the tensorflow C++ API to run the previously trained model. I have been following the tutorial found here: http://www.bitbionic.com/2017/08/18/run-your-keras-models-in-c-tensorflow/ (however using r1.14 instead of r1.1 - but note I have tried with r1.1 as well). I am using Python 2.7, GCC 9.2.1, Bazel 0.24.1 - this is what is tested by Google here: https://www.tensorflow.org/install/source with the exception of GCC (4.8 by google).
The steps I have taken are as follows:
Clone tensorflow and checkout r1.14
Configure with default settings:
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
You have bazel 0.24.1 installed.
Please specify the location of python. [Default is /usr/bin/python]:
Found possible Python library paths:
/usr/local/lib/python2.7/dist-packages
/usr/lib/python2.7/dist-packages
Please input the desired Python library path to use. Default is [/usr/local/lib/python2.7/dist-packages]
Do you wish to build TensorFlow with XLA JIT support? [Y/n]:
XLA JIT support will be enabled for TensorFlow.
Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]:
No OpenCL SYCL support will be enabled for TensorFlow.
Do you wish to build TensorFlow with ROCm support? [y/N]:
No ROCm support will be enabled for TensorFlow.
Do you wish to build TensorFlow with CUDA support? [y/N]:
No CUDA support will be enabled for TensorFlow.
Do you wish to download a fresh release of clang? (Experimental) [y/N]:
Clang will not be downloaded.
Do you wish to build TensorFlow with MPI support? [y/N]:
No MPI support will be enabled for TensorFlow.
Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native -Wno-sign-compare]:
Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]:
Not configuring the WORKSPACE for Android builds.
Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details.
--config=mkl # Build with MKL support.
--config=monolithic # Config for mostly static monolithic build.
--config=gdr # Build with GDR support.
--config=verbs # Build with libverbs support.
--config=ngraph # Build with Intel nGraph support.
--config=numa # Build with NUMA support.
--config=dynamic_kernels # (Experimental) Build kernels into separate shared objects.
Preconfigured Bazel build configs to DISABLE default on features:
--config=noaws # Disable AWS S3 filesystem support.
--config=nogcp # Disable GCP support.
--config=nohdfs # Disable HDFS support.
--config=noignite # Disable Apache Ignite support.
--config=nokafka # Disable Apache Kafka support.
--config=nonccl # Disable NVIDIA NCCL support.
Configuration finished
I was recieving a build error (shown below), so I patched this release to fix an error I was receiving with what is described here: https://github.com/clearlinux/distribution/issues/1151 (not included in tutorial)
ERROR: /home/mark/.cache/bazel/_bazel_mark/0de26f992c566e8f868a9b2c0c3da3e9/external/grpc/BUILD:507:1: C++ compilation of rule '#grpc//:gpr_base' failed (Exit 1): gcc failed: error executing command
(cd /home/mark/.cache/bazel/_bazel_mark/0de26f992c566e8f868a9b2c0c3da3e9/execroot/org_tensorflow && \
exec env - \
PATH=/home/mark/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/local/cuda/bin:/snap/bin \
PWD=/proc/self/cwd \
PYTHON_BIN_PATH=/usr/bin/python \
PYTHON_LIB_PATH=/usr/local/lib/python2.7/dist-packages \
TF_CONFIGURE_IOS=0 \
TF_DOWNLOAD_CLANG=0 \
TF_NEED_CUDA=0 \
TF_NEED_OPENCL_SYCL=0 \
TF_NEED_ROCM=0 \
/usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections -fdata-sections '-std=c++0x' -MD -MF bazel-out/k8-opt/bin/external/grpc/_objs/gpr_base/log_linux.pic.d '-frandom-seed=bazel-out/k8-opt/bin/external/grpc/_objs/gpr_base/log_linux.pic.o' -fPIC '-DGRPC_ARES=0' -iquote external/grpc -iquote bazel-out/k8-opt/genfiles/external/grpc -iquote bazel-out/k8-opt/bin/external/grpc -isystem external/grpc/include -isystem bazel-out/k8-opt/genfiles/external/grpc/include -isystem bazel-out/k8-opt/bin/external/grpc/include -mavx '-mfpmath=both' -msse4.2 -fno-canonical-system-headers -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -c external/grpc/src/core/lib/gpr/log_linux.cc -o bazel-out/k8-opt/bin/external/grpc/_objs/gpr_base/log_linux.pic.o)
Execution platform: #bazel_tools//platforms:host_platform
external/grpc/src/core/lib/gpr/log_linux.cc:43:13: error: ambiguating new declaration of 'long int gettid()'
43 | static long gettid(void) { return syscall(__NR_gettid); }
| ^~~~~~
In file included from /usr/include/unistd.h:1170,
from external/grpc/src/core/lib/gpr/log_linux.cc:41:
/usr/include/x86_64-linux-gnu/bits/unistd_ext.h:34:16: note: old declaration '__pid_t gettid()'
34 | extern __pid_t gettid (void) __THROW;
| ^~~~~~
external/grpc/src/core/lib/gpr/log_linux.cc:43:13: warning: 'long int gettid()' defined but not used [-Wunused-function]
43 | static long gettid(void) { return syscall(__NR_gettid); }
| ^~~~~~
Target //tensorflow:libtensorflow_cc.so failed to build
INFO: Elapsed time: 513.481s, Critical Path: 35.74s
INFO: 1907 processes: 1907 local.
FAILED: Build did NOT complete successfully
The patch application: downloaded https://github.com/clearlinux-pkgs/tensorflow/blob/master/Add-grpc-fix-for-gettid.patch and then from tensorflow root
>> git am Add-grpc-fix-for-gettid.patch
Bazel build process then completes with no error and creation of .so file. I would show output here but it's very long - if needed I can paste it somewhere.
bazel build --jobs=4 --verbose_failures -c opt --copt=-mavx --copt=-mfpmath=both --copt=-msse4.2 //tensorflow:libtensorflow_cc.so
Made a project with the main.cpp file in the tutorial (found here: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/label_image/main.cc) and a CMakeLists file:
cmake_minimum_required(VERSION 3.16)
project(TensorflowTesting)
set(CMAKE_CXX_STANDARD 14)
#link to tensorflow headers
include_directories(SYSTEM /home/mark/Research/tensorflow_test)
include_directories(SYSTEM /home/mark/Research/tensorflow_test/bazel-tensorflow_test/external/eigen_archive)
include_directories(SYSTEM /home/mark/Research/tensorflow_test/bazel-tensorflow_test/external/protobuf/src)
include_directories(SYSTEM /home/mark/Research/tensorflow_test/bazel-tensorflow_test/bazel-genfiles)
#link to shared object
link_directories(/home/mark/Research/tensorflow_test/bazel-bin/tensorflow) #have to link to the .a and .so files
add_executable(TensorflowTesting main.cpp)
target_link_libraries(TensorflowTesting tensorflow_cc)
When I try to build this, it gives me the error:
In file included from /home/mark/Research/tensorflow_test/tensorflow/core/framework/tensor.h:22,
from /home/mark/Research/tensorflow_test/tensorflow/cc/framework/ops.h:21,
from /home/mark/Research/tensorflow_test/tensorflow/cc/ops/const_op.h:19,
from /home/mark/CLionProjects/TensorflowTesting/main.cpp:38:
/home/mark/Research/tensorflow_test/tensorflow/core/framework/allocator.h:24:10: fatal error: absl/strings/string_view.h: No such file or directory
24 | #include "absl/strings/string_view.h"
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[3]: *** [CMakeFiles/TensorflowTesting.dir/build.make:63: CMakeFiles/TensorflowTesting.dir/main.cpp.o] Error 1
make[2]: *** [CMakeFiles/Makefile2:76: CMakeFiles/TensorflowTesting.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/TensorflowTesting.dir/rule] Error 2
make: *** [Makefile:118: TensorflowTesting] Error 2
And I was able to get around this error by the suggestion here: https://github.com/tensorflow/tensorflow/issues/22007 (added in root of tf) - However this seems wrong that I need to manually include this library.
git clone https://github.com/abseil/abseil-cpp.git
ln -s abseil-cpp/absl ./absl
Now when I try to build this, I get an error:
In file included from /home/mark/Research/tensorflow_test/tensorflow/core/framework/tensor.h:23,
from /home/mark/Research/tensorflow_test/tensorflow/cc/framework/ops.h:21,
from /home/mark/Research/tensorflow_test/tensorflow/cc/ops/const_op.h:19,
from /home/mark/CLionProjects/TensorflowTesting/main.cpp:38:
/home/mark/Research/tensorflow_test/tensorflow/core/framework/tensor_shape.h:22:10: fatal error: tensorflow/core/framework/types.pb.h: No such file or directory
22 | #include "tensorflow/core/framework/types.pb.h"
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[3]: *** [CMakeFiles/TensorflowTesting.dir/build.make:63: CMakeFiles/TensorflowTesting.dir/main.cpp.o] Error 1
make[2]: *** [CMakeFiles/Makefile2:76: CMakeFiles/TensorflowTesting.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/TensorflowTesting.dir/rule] Error 2
make: *** [Makefile:118: TensorflowTesting] Error 2
I have tried many suggestions to fix this. Including also trying to run tensorflow/contrib/makefile/build_all_linux.sh, but when I do, I recieve the error:
remote: Total 73632 (delta 33), reused 36 (delta 12), pack-reused 73529
Receiving objects: 100% (73632/73632), 60.29 MiB | 12.97 MiB/s, done.
Resolving deltas: 100% (50705/50705), done.
Note: checking out '5902e759108d14ee8e6b0b07653dac2f4e70ac73'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b <new-branch-name>
HEAD is now at 5902e7591 Add missing declaration in protobuf_headers
Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark'
Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest'
Cloning into '/home/mark/Research/tensorflow_test/tensorflow/contrib/makefile/downloads/protobuf/third_party/benchmark'...
Cloning into '/home/mark/Research/tensorflow_test/tensorflow/contrib/makefile/downloads/protobuf/third_party/googletest'...
Submodule path 'third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8'
Submodule path 'third_party/googletest': checked out 'c3bb0ee2a63279a803aaad956b9b26d74bf9e6e2'
cat: third_party/eigen3/gebp_neon.patch: No such file or directory
download_dependencies.sh completed successfully.
g++ -M -std=c++11 -DNSYNC_USE_CPP11_TIMEPOINT -DNSYNC_ATOMIC_CPP11 -I../../platform/c++11.futex -I../../platform/c++11 -I../../platform/gcc -I../../platform/posix -pthread -I../../public -I../../internal ../../internal/*.c ../../testing/*.c ../../platform/linux/src/nsync_semaphore_futex.c ../../platform/c++11/src/per_thread_waiter.cc ../../platform/c++11/src/yield.cc ../../platform/c++11/src/time_rep_timespec.cc ../../platform/c++11/src/nsync_panic.cc \
../../platform/c++11/src/start_thread.cc > dependfile
g++ -DNSYNC_USE_CPP11_TIMEPOINT -DNSYNC_ATOMIC_CPP11 -I../../platform/c++11.futex -I../../platform/c++11 -I../../platform/gcc -I../../platform/posix -pthread -I../../public -I../../internal -O -std=c++11 -Werror -Wall -Wextra -pedantic -c ../../internal/common.c
g++ -DNSYNC_USE_CPP11_TIMEPOINT -DNSYNC_ATOMIC_CPP11 -I../../platform/c++11.futex -I../../platform/c++11 -I../../platform/gcc -I../../platform/posix -pthread -I../../public -I../../internal -O -std=c++11 -Werror -Wall -Wextra -pedantic -c ../../internal/counter.c
../../internal/counter.c: In function ‘nsync::nsync_counter_s_* nsync::nsync_counter_new(uint32_t)’:
../../internal/counter.c:39:28: error: ‘void* memset(void*, int, size_t)’ clearing an object of type ‘struct nsync::nsync_counter_s_’ with no trivial copy-assignment; use value-initialization instead [-Werror=class-memaccess]
39 | memset (c, 0, sizeof (*c));
| ^
../../internal/counter.c:29:8: note: ‘struct nsync::nsync_counter_s_’ declared here
29 | struct nsync_counter_s_ {
| ^~~~~~~~~~~~~~~~
cc1plus: all warnings being treated as errors
make: *** [../../platform/posix/make.common:72: counter.o] Error 1
I'm not sure where to go from here and I was hoping to get a suggestion. I would be happy to follow another tutorial if one is suggested here as this one may be out of date. I appreciate any help!
After frustration with the above, I took a different route and ended up using https://github.com/FloopCZ/tensorflow_cc to solve the problem. I had to change versions of tensorflow to v1.15.0. Following the readme and by manually disabling CUDA support in CMake, I was able to compile and install TensorFlow to be used by CMake without bazel. I'm pretty sure I was close with my previous methodology but it was beginning to get messy and the current solution seems more practical for me.
I've been following the instructions for OSX here. I've installed bazel, cloned the sample and tried to build it with bazel build //main:hello-world. But here where it got wrong.
INFO: Analysed target //main:hello-world (0 packages loaded). INFO:
Found 1 target... ERROR:
/Users/myuser/work/temp/bazel-sample/examples/cpp-tutorial/stage1/main/BUILD:1:1:
Linking of rule '//main:hello-world' failed (Exit 1) ccache: error:
Failed to create temporary file for
/Users/myuser/.ccache/6/stats.tmp: Operation not
permitted Target //main:hello-world failed to build Use
--verbose_failures to see the command lines of failed build steps. INFO: Elapsed time: 0.295s, Critical Path: 0.07s INFO: 0 processes.
FAILED: Build did NOT complete successfully
Seems like there's an issue with ccache which I have installed. After running same command with --verbose_failures
INFO: Analysed target //main:hello-world (0 packages loaded). INFO:
Found 1 target... ERROR:
/Users/myuser/work/temp/bazel-sample/examples/cpp-tutorial/stage1/main/BUILD:1:1:
Linking of rule '//main:hello-world' failed (Exit 1): cc_wrapper.sh
failed: error executing command (cd
/private/var/tmp/_bazel_myuser/550c05da61518b4bbbb0ffdfd033154f/execroot/__main__
&& \ exec env - \
APPLE_SDK_PLATFORM='' \
APPLE_SDK_VERSION_OVERRIDE='' \
PATH=/Users/myuser/Library/Android/sdk/platform-tools:/Users/myuser/Library/Android/sdk/tools:/Users/myuser/Library/Android/sdk/build-tools/25.0.0:/Users/myuser/Library/Android/ndk/android-ndk-r10c:/Applications/CMake.app/Contents/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/Users/myuser/bin
\
XCODE_VERSION_OVERRIDE=9.3.0 \ external/local_config_cc/cc_wrapper.sh -fobjc-link-runtime -Wl,-S -o
bazel-out/darwin-fastbuild/bin/main/hello-world
bazel-out/darwin-fastbuild/bin/main/_objs/hello-world/main/hello-world.o
-headerpad_max_install_names -lc++ -no-canonical-prefixes)
Use --sandbox_debug to see verbose messages from the sandbox ccache:
error: Failed to create temporary file for
/Users/myuser/.ccache/0/stats.tmp: Operation not permitted Target
//main:hello-world failed to build INFO: Elapsed time: 0.294s,
Critical Path: 0.07s INFO: 0 processes. FAILED: Build did NOT complete
successfully
I figured out that there's an issue with sandbox. Adding --sandbox_debug prints:
INFO: Analysed target //main:hello-world (0 packages loaded).
INFO: Found 1 target...
ERROR: /Users/myuser/work/temp/bazel-sample/examples/cpp-tutorial/stage1/main/BUILD:1:1: Linking of rule '//main:hello-world' failed (Exit 1): sandbox-exec failed: error executing command
(cd /private/var/tmp/_bazel_myuser/550c05da61518b4bbbb0ffdfd033154f/execroot/__main__ && \
exec env - \
APPLE_SDK_PLATFORM='' \
APPLE_SDK_VERSION_OVERRIDE='' \
DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer \
PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/myuser/bin \
SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk \
TMPDIR=/var/folders/x3/r5tf7wfx3_q0xnyx95s3r5400000gq/T/ \
XCODE_VERSION_OVERRIDE=9.3.0 \
/usr/bin/sandbox-exec -f /private/var/tmp/_bazel_myuser/550c05da61518b4bbbb0ffdfd033154f/sandbox/darwin-sandbox/1/sandbox.sb /private/var/tmp/_bazel_myuser/550c05da61518b4bbbb0ffdfd033154f/execroot/__main__/_bin/process-wrapper '--timeout=0' '--kill_delay=15' external/local_config_cc/cc_wrapper.sh -fobjc-link-runtime -Wl,-S -o bazel-out/darwin-fastbuild/bin/main/hello-world bazel-out/darwin-fastbuild/bin/main/_objs/hello-world/main/hello-world.o -headerpad_max_install_names -lc++ -no-canonical-prefixes)
ccache: error: Failed to create temporary file for /Users/myuser/.ccache/b/stats.tmp: Operation not permitted
Target //main:hello-world failed to build
INFO: Elapsed time: 0.268s, Critical Path: 0.06s
INFO: 0 processes.
FAILED: Build did NOT complete successfully
the content of the sandbox file sandbox.sb is:
(version 1)
(debug deny)
(allow default)
(deny file-write*)
(allow file-write*
(subpath "/dev")
(subpath "/Users/myuser/Library/Logs")
(subpath "/Users/myuser/Library/Developer")
(subpath "/private/var/folders/x3/r5tf7wfx3_q0xnyx95s3r5400000gq/T")
(subpath "/private/var/tmp/_bazel_myuser/550c05da61518b4bbbb0ffdfd033154f/sandbox/darwin-sandbox/1/execroot/__main__")
(subpath "/private/var/tmp")
(subpath "/private/var/folders/x3/r5tf7wfx3_q0xnyx95s3r5400000gq/C")
(subpath "/private/tmp")
)
It is apparently missing the permissions to write to the /Users/myuser/.ccache directory. I have tried disabling ccache with export CCACHE_DISABLE=1 but nothing changes. How can I fix it without removing ccache?
Is setting an absolute path to the compiler and passing it to CC environment variable an option? You might need to use BAZEL_USE_CPP_ONLY_TOOLCHAIN=1 if you have Xcode installed.
export CC=/usr/bin/clang
export CXX=/usr/bin/clang++
<your bazel command>
can replace clang with gcc
I'm trying to build tensorflow on Ubuntu 14.04 LTS with python 2.7, no GPU. when I run the following command on terminal followed from this tutorial:
bazel build tensorflow/examples/image_retraining:retrain
it says failed to build with log:
ERROR: /home/yuan/tensorflow_source/tensorflow/tensorflow/python/BUILD:1826:1: Linking of rule '//tensorflow/python:_pywrap_tensorflow.so' failed: gcc failed: error executing command
(cd /home/yuan/.cache/bazel/_bazel_yuan/e5b8b6538ba16bbae5b1e0f5c26b7a12/execroot/tensorflow && \
exec env - \
/usr/bin/gcc -shared -o bazel-out/local-fastbuild/bin/tensorflow/python/_pywrap_tensorflow.so -Wl,--version-script tensorflow/tf_version_script.lds -pthread -Wl,-no-as-needed -B/usr/bin -B/usr/bin -pass-exit-codes '-Wl,--build-id=md5' '-Wl,--hash-style=gnu' -Wl,-S -Wl,#bazel-out/local-fastbuild/bin/tensorflow/python/_pywrap_tensorflow.so-2.params): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.
bazel-out/local-fastbuild/bin/tensorflow/core/libversion_lib.a(version_info.pic.o): In function `tf_git_version()':
version_info.cc:(.text+0x0): multiple definition of `tf_git_version()'
bazel-out/local-fastbuild/bin/tensorflow/core/libframework_internal.lo(version_info.pic.o):version_info.cc:(.text+0x0): first defined here
bazel-out/local-fastbuild/bin/tensorflow/core/libversion_lib.a(version_info.pic.o): In function `tf_compiler_version()':
version_info.cc:(.text+0xd): multiple definition of `tf_compiler_version()'
bazel-out/local-fastbuild/bin/tensorflow/core/libframework_internal.lo(version_info.pic.o):version_info.cc:(.text+0xd): first defined here
collect2: error: ld returned 1 exit status
Target //tensorflow/examples/image_retraining:retrain failed to build
INFO: Elapsed time: 52.885s, Critical Path: 34.63s
Here's my build environment:
OS: Ubuntu 14.04 LTS 64 bit with RAM 12Gib
gcc version: 4.8.4
python version: 2.7.6
bazel version: 0.3.2
git version of tensorflow souce: v0.11.0rc0-1541-g3737ac3
Does anyone have idea to fix this error? Thanks a lot!
#David thanks for your help, it seems my ./configure result is not as expected.
I finally build successful by the following steps
sync tensorflow source to HEAD
run ./configure again, if have any error, refer to this.
run the build command again
bazel build tensorflow/examples/image_retraining:retrain
And my build result will have no error.
My goal is to make some new function to pnacl toolchain. Before I doing so, I think I should build the pnacl toolchain successfully first.
[11/04 Update]
Finally it works. Some package(bison, byacc, flex) I did't install and caused this problem.
[11/03 Update]
I use ubuntu-14.04.1-desktop-amd64 as my system
Also, ubuntu-12.04.5-desktop-amd64 is now testing [11/03 updated]
I've followed the step of the document all the way. Everything goes just fine until I run the toolchain_build_pnacl script
So I add the new error message here :
if I run the script using gcc to compile
toolchain_build/toolchain_build_pnacl.py --gcc --verbose --sync --clobber
--install toolchain/linux_x86/pnacl_newlib
then the following error message is
gcc -DHAVE_CONFIG_H -I. -I../../../src/binutils/gold
-I../../../src/binutils/gold -I../../../src/binutils/gold/../include - I../../../src/binutils/gold/../elfcpp -DLOCALEDIR="\"/share/locale\""
-DBINDIR="\"/bin\"" -DTOOLBINDIR="\"/arm-pc-nacl/bin\"" -DTOOLLIBDIR="\"/arm-pc-nacl/lib\"" -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -frandom-seed=yyscript.o -Wno-extended-offsetof -Wno-absolute-value -Wno-unused-function -Wno-unused-const-variable -Wno-unneeded-internal-declaration -Wno-unused-private-field -Wno-format-security -MT yyscript.o -MD -MP -MF .deps/yyscript.Tpo -c -o yyscript.o yyscript.c yyscript.c:1:1: error: return type defaults to ‘int’ [-Werror=return-type] yyscript.c:1:1: error: function
declaration isn’t a prototype [-Werror=strict-prototypes] cc1: error:
unrecognized command line option "-Wno-unused-private-field" [-Werror]
cc1: error: unrecognized command line option
"-Wno-unneeded-internal-declaration" [-Werror] cc1: error:
unrecognized command line option "-Wno-unused-const-variable"
[-Werror] cc1: error: unrecognized command line option
"-Wno-absolute-value" [-Werror] cc1: error: unrecognized command line
option "-Wno-extended-offsetof" [-Werror] cc1: all warnings being
treated as errors make[4]: *** [yyscript.o] Error 1
if I run it with default clang to compile
toolchain_build/toolchain_build_pnacl.py --verbose --clobber
--install toolchain/linux_x86/pnacl_newlib
then following error message is generated
/home/albaforia/SVN/nativeclient/third_party/llvm-build/Release+Asserts/bin/clang
-DHAVE_CONFIG_H -I. -I../../../src/binutils/gold -I../../../src/binutils/gold -I../../../src/binutils/gold/../include -I../../../src/binutils/gold/../elfcpp -DLOCALEDIR="\"/share/locale\"" -DBINDIR="\"/bin\"" -DTOOLBINDIR="\"/arm-pc-nacl/bin\"" -DTOOLLIBDIR="\"/arm-pc-nacl/lib\"" -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -frandom-seed=yyscript.o -Wno-extended-offsetof -Wno-absolute-value -Wno-unused-function -Wno-unused-const-variable -Wno-unneeded-internal-declaration -Wno-unused-private-field -Wno-format-security -MT yyscript.o -MD -MP -MF .deps/yyscript.Tpo -c -o yyscript.o yyscript.c yyscript.c:1:1: error: type specifier missing, defaults to 'int' [-Werror,-Wimplicit-int]
main() { return 0;
} ^~~~ 1 error generated. make[4]: *** [yyscript.o] Error 1
By default the PNaCl toolchain builds with a downloaded binary of clang because Google's automated builders run Ubuntu 12.04, and the gcc on those systems cannot build recent versions of LLVM. So there are 2 options:
If you used Chromium's depot_tools to get the Native Client sources, you can easily get a copy of the same compiler the builders use: from the parent directory of native_client run
tools/clang/scripts/update.sh which will download and install clang in the directory where toolchain_build_pnacl.py expects it. Once it is installed, it will be kept up to date automatically by Native Clients DEPS hooks if you use gclient from depot_tools.
Otherwise you can use the system gcc to compile the PNaCl toolchain using the --gcc option. This should definitely work on Ubuntu 14.04 as that the system we develop PNaCl on.
As an aside I've started another document that has a few more details on the build process (even if its focus isn't exactly what you are interested in) and of course there is the -h flag of toolchain_build_pnacl.py and its sources.
Could you post more information on what the error is? You probably need to scroll up a bit (the errors are interleaved with other parts of the build because it builds in parallel). To help debug this you could also run toolchain_build/toolchain_build_pnacl.py on specific targets and reduce the amount of building that goes on, e.g. you could do:
toolchain_build/toolchain_build_pnacl.py llvm_x86_64_linux --verbose --install toolchain/linux_x86/pnacl_newlib
FYI, when you re-run the toolchain_build/toolchain_build_pnacl.py script you don't need to re-specify --sync and --clobber.
You can also run the make command manually as you debug the issue. In any case we'd like to fix the issue you're running into!
To use system compiler instead of compiler from third_party/llvm_build/... you may want to pass the --gcc option to the build script. Though, it's still unlikely that you'll succeed in building toolchain under Ubuntu 14.04.
I'm using Ubuntu 14.04 and in order to build pnacl toolchain I have set up Ubuntu 12.04 chroot environment. Inside it I can successfully build the toolchain using the following commands:
# get sources for NaCl itself
gclient config http://src.chromium.org/native_client/trunk/src/native_client#13992 --name native_client
gclient sync -j16
cd native_client
# sync sources for binutils, llvm, etc.
toolchain_build/toolchain_build_pnacl.py --verbose --sync-only
# build and install toolchain
toolchain_build/toolchain_build_pnacl.py --verbose --gcc --install install_dir
PS: I mount /proc, /dev, /dev/pts, /sys, /run, /run/shm, /tmp into my chroot environment and everything runs smoothly. Without mounting some of these directories some build scripts may fail, though I'm not sure about it.