Issue building Tensorflow using Bazel (Windows) => to use Tensorflow C++ API - c++

Question

My initial goal was simply to use Tensorflow's C++ API to read a ".pb" frozen model and make predictions with it. Was using Keras to prototype it, managed to freeze the model and try it with Tensorflow on Python ; everything goes smoothly.
So i basically JUST need the C++ API to work.
I'm using an i5-8400T computer, no GPU, 8GB ram. All versions of bazel and tensorflow were downloaded today (27/08/2019).
First thing I did was to follow tensorflow's guide of course. Ran the little bazel run -c opt //tensorflow/cc/example:example, and that... several times.
First time, needed to downgrade my bazel version to 0.26.1 to make tensorflow's configure.py happy.
Then I had the same problem as a few people had apparently, which is an error fetching io_bazel_rules_docker, which then stops the rest of the build of course. (what follows is what i got as an output here)
bazel : INFO: Options provided by the client:
Au caractère Ligne:1 : 1
+ bazel run -c opt //tensorflow/cc/example:example > error.txt 2>&1
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (INFO: Options provided by the client::String) [], RemoteException
+ FullyQualifiedErrorId : NativeCommandError
Inherited 'common' options: --isatty=0 --terminal_columns=80
INFO: Options provided by the client:
Inherited 'build' options: --python_path=C:/Users/IMEDSYS/AppData/Local/Programs/Python/Python37/python.exe
INFO: Reading rc options for 'run' from c:\users\imedsys\bin\tensorflow\.bazelrc:
Inherited 'build' options: --apple_platform_type=macos --define framework_shared_object=true --define
open_source_build=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true
--spawn_strategy=standalone --strategy=Genrule=standalone -c opt --cxxopt=-std=c++14 --host_cxxopt=-std=c++14
--announce_rc --define=grpc_no_ares=true --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib
--define=INCLUDEDIR=$(PREFIX)/include
INFO: Reading rc options for 'run' from c:\users\imedsys\bin\tensorflow\.tf_configure.bazelrc:
Inherited 'build' options: --action_env
PYTHON_BIN_PATH=C:/Users/IMEDSYS/AppData/Local/Programs/Python/Python37/python.exe --action_env
PYTHON_LIB_PATH=C:/Users/IMEDSYS/AppData/Local/Programs/Python/Python37/lib/site-packages
--python_path=C:/Users/IMEDSYS/AppData/Local/Programs/Python/Python37/python.exe --config monolithic --copt=-w
--host_copt=-w --copt=-DWIN32_LEAN_AND_MEAN --host_copt=-DWIN32_LEAN_AND_MEAN --copt=-DNOGDI --host_copt=-DNOGDI
--verbose_failures --distinct_host_configuration=false --action_env TF_CONFIGURE_IOS=0
INFO: Found applicable config definition build:monolithic in file c:\users\imedsys\bin\tensorflow\.bazelrc: --define
framework_shared_object=false
Loading:
Loading: 0 packages loaded
INFO: Call stack for the definition of repository 'io_bazel_rules_docker' which is a git_repository (rule definition
at C:/users/imedsys/_bazel_imedsys/wmdtvioi/external/bazel_tools/tools/build_defs/repo/git.bzl:252:18):
- C:/users/imedsys/_bazel_imedsys/wmdtvioi/external/bazel_toolchains/repositories/repositories.bzl:37:9
- C:/users/imedsys/bin/tensorflow/WORKSPACE:35:1
ERROR: An error occurred during the fetch of repository 'io_bazel_rules_docker':
Traceback (most recent call last):
File "C:/users/imedsys/_bazel_imedsys/wmdtvioi/external/bazel_tools/tools/build_defs/repo/git.bzl", line 234
_clone_or_update(ctx)
File "C:/users/imedsys/_bazel_imedsys/wmdtvioi/external/bazel_tools/tools/build_defs/repo/git.bzl", line 74, in
_clone_or_update
fail(("error cloning %s:\n%s" % (ctx....)))
error cloning io_bazel_rules_docker:
fatal: not a git repository (or any parent up to mount point /mnt)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
/bin/bash: line 1: cd: C:/users/imedsys/_bazel_imedsys/wmdtvioi/external: No such file or directory
+ cd C:/users/imedsys/_bazel_imedsys/wmdtvioi/external
/bin/bash: line 3: cd: C:/users/imedsys/_bazel_imedsys/wmdtvioi/external: No such file or directory
+ git -C C:/users/imedsys/_bazel_imedsys/wmdtvioi/external/io_bazel_rules_docker reset --hard
251f6a68b439744094faff800cd029798edf9faa
fatal: cannot change to 'C:/users/imedsys/_bazel_imedsys/wmdtvioi/external/io_bazel_rules_docker': No such file or
directory
+ git -C C:/users/imedsys/_bazel_imedsys/wmdtvioi/external/io_bazel_rules_docker fetch '' origin
251f6a68b439744094faff800cd029798edf9faa:251f6a68b439744094faff800cd029798edf9faa
fatal: cannot change to 'C:/users/imedsys/_bazel_imedsys/wmdtvioi/external/io_bazel_rules_docker': No such file or
directory
+ git -C C:/users/imedsys/_bazel_imedsys/wmdtvioi/external/io_bazel_rules_docker fetch origin
251f6a68b439744094faff800cd029798edf9faa:251f6a68b439744094faff800cd029798edf9faa
fatal: cannot change to 'C:/users/imedsys/_bazel_imedsys/wmdtvioi/external/io_bazel_rules_docker': No such file or
directory
ERROR: ... *idem*
ERROR: ... *idem*
INFO: Elapsed time: 2.346s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
ERROR: Build failed. Not running target
FAILED: Build did NOT complete successfully (0 packages loaded)
***
I so tried applying what this issue was offering, which involves going to the rules_docker repository and replace targets with the latest ones.
The thing is even after this you get the same kind of errors with the protobuf targets, which I get hash errors for if i apply the same strategy.
I then tried to do it with CMake as many were saying last year that bazel building on windows was not yet usable. I used two similar guides, one of them here, and so tried to build with
cmake .. -A x64 -DCMAKE_BUILD_TYPE=Release -DSWIG_EXECUTABLE=C:\swigwin-3.0.12\swig.exe -DPYTHON_EXECUTABLE=C:\Anaconda3\envs\tensorflow\python.exe -DPYTHON_LIBRARIES=C:\Anaconda3\envs\tensorflow\libs\python35.lib
at first, and then with
`cmake .. -A x64 -DCMAKE_BUILD_TYPE=Release -DSWIG_EXECUTABLE=C:\swigwin-3.0.12\swig.exe -DPYTHON_EXECUTABLE=C:\Anaconda3\envs\tensorflow\python.exe -DPYTHON_LIBRARIES=C:\Anaconda3\envs\tensorflow\libs\python35.lib -Dtensorflow_BUILD_PYTHON_BINDINGS=OFF -Dtensorflow_ENABLE_GRPC_SUPPORT=OFF -Dtensorflow_BUILD_SHARED_LIB=ON`
and i get those errors :
cmake : CMake Warning at CMakeLists.txt:9 (message):
Au caractère Ligne:1 : 1
+ cmake .. -A x64 -DCMAKE_BUILD_TYPE=Release -DSWIG_EXECUTABLE=C:\Users ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (CMake Warning a...xt:9 (message)::String) [], RemoteException
+ FullyQualifiedErrorId : NativeCommandError
Your current cmake generator is set to use 32 bit toolset architecture.
This may cause "compiler out of heap space" errors when building. Consider
using the flag -Thost=x64 when running cmake. Ignore this if you are on
CMake GUI.
-- Selecting Windows SDK version to target Windows 10.0.17763.
-- C:/Users/IMEDSYS/bin/tensorflow/tensorflow/contrib/cmake/build/abseil_cpp/src/abseil_cpp_build
-- Configuring done
CMake Error at tf_core_ops.cmake:77 (add_library):
Cannot find source file:
C:/Users/IMEDSYS/bin/tensorflow/tensorflow/contrib/factorization/ops/clustering_ops.cc
Tried extensions .c .C .c++ .cc .cpp .cxx .cu .m .M .mm .h .hh .h++ .hm
.hpp .hxx .in .txx
Call Stack (most recent call first):
tf_core_ops.cmake:95 (GENERATE_CONTRIB_OP_LIBRARY)
CMakeLists.txt:594 (include)
CMake Error at tf_core_ops.cmake:77 (add_library):
Cannot find source file:
C:/Users/IMEDSYS/bin/tensorflow/tensorflow/contrib/rnn/ops/gru_ops.cc
Tried extensions .c .C .c++ .cc .cpp .cxx .cu .m .M .mm .h .hh .h++ .hm
.hpp .hxx .in .txx
Call Stack (most recent call first):
tf_core_ops.cmake:107 (GENERATE_CONTRIB_OP_LIBRARY)
CMakeLists.txt:594 (include)
CMake Error at tf_core_ops.cmake:77 (add_library):
Cannot find source file:
C:/Users/IMEDSYS/bin/tensorflow/tensorflow/contrib/rnn/ops/lstm_ops.cc
Tried extensions .c .C .c++ .cc .cpp .cxx .cu .m .M .mm .h .hh .h++ .hm
.hpp .hxx .in .txx
Call Stack (most recent call first):
tf_core_ops.cmake:108 (GENERATE_CONTRIB_OP_LIBRARY)
CMakeLists.txt:594 (include)
CMake Error at tf_core_kernels.cmake:205 (add_library):
Cannot find source file:
C:/Users/IMEDSYS/bin/tensorflow/tensorflow/contrib/factorization/kernels/clustering_ops.cc
Tried extensions .c .C .c++ .cc .cpp .cxx .cu .m .M .mm .h .hh .h++ .hm
.hpp .hxx .in .txx
Call Stack (most recent call first):
CMakeLists.txt:596 (include)
CMake Error at tf_core_ops.cmake:77 (add_library):
No SOURCES given to target: tf_contrib_factorization_clustering_ops
Call Stack (most recent call first):
tf_core_ops.cmake:95 (GENERATE_CONTRIB_OP_LIBRARY)
CMakeLists.txt:594 (include)
CMake Error at tf_core_ops.cmake:77 (add_library):
No SOURCES given to target: tf_contrib_rnn_gru_ops
Call Stack (most recent call first):
tf_core_ops.cmake:107 (GENERATE_CONTRIB_OP_LIBRARY)
CMakeLists.txt:594 (include)
CMake Error at tf_core_ops.cmake:77 (add_library):
No SOURCES given to target: tf_contrib_rnn_lstm_ops
Call Stack (most recent call first):
tf_core_ops.cmake:108 (GENERATE_CONTRIB_OP_LIBRARY)
CMakeLists.txt:594 (include)
CMake Error at tf_core_ops.cmake:77 (add_library):
No SOURCES given to target: tf_contrib_tpu_ops
Call Stack (most recent call first):
tf_core_ops.cmake:115 (GENERATE_CONTRIB_OP_LIBRARY)
CMakeLists.txt:594 (include)
CMake Error at tf_core_kernels.cmake:205 (add_library):
No SOURCES given to target: tf_core_kernels
Call Stack (most recent call first):
CMakeLists.txt:596 (include)
CMake Generate step failed. Build files cannot be regenerated correctly.
This time i feel like it's simply that CMake looks for files that are either depreciated, either relocated.
SO,
my questions is rather simple :
does someone has 2 version numbers of both bazel and tensorflow that actually work together (because it seems that it does work for some people => I consider trying voodoo)?
does someone has a very straightforward solution to use Tensorflow's c_api or C++ API ?
does someone knows another way to make a prediction with either Keras or Tensorflow outputs (.hdf5 or .pb) from a C++ built software, without doing as disgusting as "system("python predict.py")"?
Thank you in advance for your quick answers! :)

Answer 1

I had done similar work building older Tensorflow version (not 2.) for VS 2017 using Cmake.
Cuda 9 . Tensorflow 1.14 VS 2017 .