I am trying to build TensorFlow 2.4.1 C++ API on Windows 10 and I am having issues.
What I've done so far:
Download TensorFlow Source from the official repo
https://github.com/tensorflow/tensorflow
and switched to the official v2.4.1 tag
Download and install Python 3.6.8 x64
Created a virtual env with python 3.6.8
Created and installed requirements.txt based on the data here
Download and install CUDA 11.0 and cuDNN v8.0.4.30 (for CUDA 11)
Downloaded and installed msys2 and set it's location in PATH
Download and install bazel (3.1.0) as it is the most recent entry here
Then I run the configuration process with python configure.py
I configure for C++ build (tensorflow_cc) with GPU support
Here I had some problem that apparently during the config process windows-style backslashes are accepted as a valid input, but then when you actually run bazel compilation they cause problems, so I reran my configuration to provide linux-style backslashes. Thus CUDA and cuDNN were successfully detected and compilation started.
The full contents of my .tf_configure.bazelrc are below
build --action_env PYTHON_BIN_PATH="D:/code/sdk/tensorflow/venv/Scripts/python.exe"
build --action_env PYTHON_LIB_PATH="D:/code/sdk/tensorflow/venv/lib/site-packages"
build --python_path="D:/code/sdk/tensorflow/venv/Scripts/python.exe"
build --config=xla
build --action_env TF_CUDA_VERSION="11.0"
build --action_env TF_CUDNN_VERSION="8.0.4"
build --action_env TF_CUDA_PATHS="C:/Program Files/NVIDIA GPU Computing
Toolkit/CUDA/v11.0,D:/code/sdk/cudnn-11.0-windows-x64-v8.0.4.30/cuda"
build --action_env CUDA_TOOLKIT_PATH="C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.0"
build --action_env TF_CUDA_COMPUTE_CAPABILITIES="5.0"
build --config=cuda
build:opt --copt=/arch:AVX
build:opt --host_copt=/arch:AVX
build:opt --define with_default_optimizations=true
build --define=override_eigen_strong_inline=true
test --flaky_test_attempts=3
test --test_size_filters=small,medium
test:v1 --test_tag_filters=-benchmark-test,-no_oss,-no_windows,-no_windows_gpu,-no_gpu,-oss_serial
test:v1 --build_tag_filters=-benchmark-test,-no_oss,-no_windows,-no_windows_gpu,-no_gpu
test:v2 --test_tag_filters=-benchmark-test,-no_oss,-no_windows,-no_windows_gpu,-no_gpu,-oss_serial,-
v1only
test:v2 --build_tag_filters=-benchmark-test,-no_oss,-no_windows,-no_windows_gpu,-no_gpu,-v1only
build --action_env TF_CONFIGURE_IOS="0"
About 20 minutes into the compilation however it failed with the following error:
ERROR: D:/code/sdk/tensorflow/tensorflow/stream_executor/cuda/BUILD:366:1: C++ compilation of rule '//tensorflow/stream_executor/cuda:cudnn_stub' failed (Exit 2): python.exe failed: error executing command
And the reason the command fails to execute is
bazel-out/x64_windows-opt/bin/external/local_config_cuda/cuda/_virtual_includes/cudnn_header\third_party/gpus/cudnn/cudnn.h(61): fatal error C1083: Cannot open include file: 'cudnn_ops_infer.h': No such file or directory
And here I am wondering what is going on? I already provided a path to cuDNN, but apparently bazel doesn't really know about it, even though it previously acknowledged that the path I have provided is correct. Am I missing some environment variable that I need to set to instruct where cuDNN is?
Has anyone built TF C++ v2.4.1 on Windows? There is so little information online, even the official page says nothing about Windows builds. It's only Linux and Mac...
As I was running out of ideas I decided to go take a look at the Bazel build scripts for CUDA
In <REPO>\tensorflow\third_party\gpus\cuda_configure.bzl I saw that cuDNN path is read from env variable CUDNN_INSTALL_PATH
and if not present it will default to /usr/local/include?
Anyway, tried set CUDNN_INSTALL_PATH=D:/code/sdk/cudnn-11.0-windows-x64-v8.0.4.30/cuda and WOOHOO It compiled!
(Pro Tip: Set the env var without any quotes and with linux-style slashes...)
Related
I'm trying to build QT from source code in Windows.
I installed QT binary (5.12.8) to get qmake, the bin path is added to system environment.
I downloaded QT source code (qtbase only) tag v5.12.8 and unzip to d:\github\qtbase
I opened the vs native x64 cmd shell and run: qmake qtbase.pro
The output is like:
Support enabled for:
Using pkg-config ....................... no
udev ................................... no
Using system zlib ...................... no
.
.
.
Qt is now configured for building. Just run 'nmake'.
Once everything is built, you must run 'nmake install'.
Qt will be installed into 'C:\Qt\Qt5.12.8\5.12.8\msvc2017_64'.
then I run nmake but I had this error:
Project ERROR: Could not find feature system-zlib.
NMAKE : fatal error U1077: 'cd' : return code '0x3'
Stop.
From the config log above, system zlib is not used, I assume QT's internal zlib will be used.
I have checked my folder, \src\3rdparty\zlib\src is existed and zlib.h is there.
I'd like to know how to fix this error or disable zlib.
#AlanBirtles Thanks for the hint, I followed the official instructions and compiled again, it works.
In the official instruction, configure.bat is used instead of my direct use of qmake.
There is one more difference in my second try: I use the src code in C:\Qt\Qt5.12.8\5.12.8\Src, not the code from github in d:\github\qtbase.
The code in C:\Qt is downloaded during my installation of binary QT, because I checked the source code item in the installation.
Hello i need help about building artefact for Java. I get an error when running this command for build:
./configure --with-vpnc-script=~/Downloads/vpnc-script --with-java=/Library/Java/JavaVirtualMachines/jdk-14.0.1.jdk/Contents/Home --disable-nls.
This is the error that i get:
checking jni.h usability... no
configure: error: unable to compile JNI test program
I need your help please. I'm using release version 8.08 and building it on mac. This is the official gitlab repository
Here is the content of config.log generated
Thanks
It seems that configure script expects you to pass the path to the JDK's include directory, not the JDK itself.
This should work:
--with-java=/Library/Java/JavaVirtualMachines/jdk-14.0.1.jdk/Contents/Home/include
I have never used qt before, but I got a task to build an already existing application for Windows and Mac using the newest version of qt. What I have done so far is:
Installed Visual Studio Professional 2017.
Installed the latest Qt 5 Open Source.
Note: when installing the QT 5 I installed all of the components under QT/ QT 5.10.1 which were: qt 5.10.1 components and under QT/Tools I installed QT Creator 4.6.0 CDB Debugger Support and MinGW 5.3.0 : qt 5 Tools
I cloned the branch from GitHub that I wanted to build to C:/Projects
Then in Git Bash I cd to the project and tried to run qmake. The first time I had this problem:
"bash: qmake: command not found"
After searching into the problem I found out that I haven't added "Qt/5.10.1/msvc2017_64/bin" to PATH and I added it. Now when I echo $PATH I can see the directory added.
Now after bash recognises qmake and I try to run it it says:
Project ERROR: Cannot run target compiler 'cl'. Output:
===================
Maybe you forgot to setup the environment?
I have no idea how to fix the problem due to the fact that I have never used QT to build anything before. Can you help me?
In the read me file for the application I am trying to build it says:
Step-by-step instructions
1. Clone this branch
2. cd into the project
3. Run qmake - this wil generate Makefiles for all of the project's modules
4.Run make (Use the -j4 of -j flags - this greatly speeds up the build process - see the manual page for make for more info). You can also use the -s flag to silence the output - this also wins you a few seconds.
I am a bit puzzled by the instructions, it does not say to run qmake -project anywhere, but there is a qt project file in the directory that I cd to. Does this mean that I don't have to run qmake -project?
Also I have no idea how to run make by using the -j4 and -j flags and how to run flags in general. Could someone help me?
I believe that for the Cannot Run Compiler 'cl' issue I have some kind of a problem with the set up. Did I install too many components or is it to do with the VS setup?
I highly recommend you to install Qt from the Qt installer, and to use Qt Creator.
However, if you want to use it from source, you first need to get a compiler (ex: MinGW). Then you need to execute the instructions, but you need to use a MinGW (or whatever compiler you use) command prompt, which will automatically set the environment variables needed.
Note that this is Windows specific. On mac and linux, the compiler's environment variables are already set in the default command prompt (if there is a compiler, obviously).
I need to use the C++ API of TensorFlow. Therefore it seems necessary to build TensorFlow from sources, so I do this according to https://www.tensorflow.org/install/install_sources. I want to have the version with GPU support. I run the ./configure with mostly default options.
However when running
bazel build --config=opt --config=cuda
//tensorflow/tools/pip_package:build_pip_package
the following error occurs:
Cuda Configuration Error: cuDNN version detected from
/usr/lib/x86_64-linux-gnu/include/cudnn.h (7.0.3) does not match
TF_CUDNN_VERSION (6)
I had previously CuDNN 7 installled, but then installed cudnn 6 for use with tensorflow. What can I do?
Could you try disable CuDNN 7 from your environment varaities, and try to put the file in cudnn6 folder to the relative folder in your CUDA folder.
This works for me, hope it helps.
I've been strugling for two weeks to create an environment for building a gstreamer plugin on windows (needed for a songbird addon).
I've installed MSYS, MinGW and Cygwin, then installed GStreamer OSSBuild, and I also downloaded the sources for Songbird, which come with their own precompiled version of gstreamer.
I was unable to run gst-inspect (or any other gstreamer applications) from the songbird sources and I figured I will settle for OSSBuild (as I was able to run gst-inspect from the compiled OSSBuild).
When following the instructions for building a GST plugin (found here) through, cygwin will not recognize the OSSBuild and the build fails when running autogen, with the following error:
checking for GST... no
configure: error:
You need to install or upgrade the GStreamer development
packages on your system. On debian-based systems these are
libgstreamer0.10-dev and libgstreamer-plugins-base0.10-dev.
on RPM-based systems gstreamer0.10-devel, libgstreamer0.10-devel
or similar. The minimum version required is 0.10.16.
configure failed
I could also not use MSYS or MinGW as they are unable to run autogen at all.
I understand that cygwin should have it's own gstreamer development packages but I couldn't find how to install them.
My question: How do I install the gstreamer packages in cygwin or how do I build using cygwin with the OSSBuild dependencies?
In short, how do I get an environment where I can build a gstreamer plugin under windows?
you can install precompiled gstreamer packages for cygwin at cygwinports. there you will find installation instructions and a list of available packages. you should not need to build them from source.
Configure's most likely going to look for libtool (.la) or pkg-config (.pc) files. Since OSSBuild is built using MSVC, you're not likely to get those files so configure can pick them up. However, you can manually create them and set them in a location to be picked up by the script. I do know that OSSBuild does have as its goal to eventually provide Visual Studio-compatible C/C++ project templates for GStreamer plugins as well as libtool and pkg-config files, but they're not available just yet.
If you checkout the OSSBuild source and can follow MSVC property files, you can see how the plugins are setup and configured.