How to pass CMAKE_GENERATOR to a CMake external project? - c++

I am trying to compile the TensorFlow CMake build as an external project on Windows. This requires the 64-bit toolchain to not run out of memory during compilation. However, even though the CMAKE_GENERATOR for the top-level project is Visual Studio 14 2015 Win64, and this is passed to the external project via the CMAKE_GENERATOR variable, the 32 bit toolchain is used to compile the external project.
What I observe: When the build is started, a 64 bit MSBuild process spawns to drive the main project as expected. This can be seen in the task manager. When building the TensorFlow external project however, an MSBuild (32 bit) process start for the external project. The compiler later fails with the error: c1060: compiler is out of heap space.
The CMake code for the TensorFlow external project is
ExternalProject_Add(TensorFlow
GIT_REPOSITORY ${TensorFlow_REPOSITORY}
SOURCE_DIR ${TensorFlow_SOURCE_DIR}
SOURCE_SUBDIR tensorflow/contrib/cmake
BINARY_DIR ${TensorFlow_BINARY_DIR}
CMAKE_GENERATOR ${CMAKE_GENERATOR}
CMAKE_ARGS
--no-warn-unused-cli
-DBUILD_SHARED_LIBS:BOOL=${BUILD_SHARED_LIBS}
-DSWIG_EXECUTABLE:FILEPATH=${SWIG_EXECUTABLE}
-DPYTHON_EXEUTABLE:FILEPATH=${PYTHON_EXECUTABLE}
-DPYTHON_LIBRARIES:LIST=${PYTHON_LIBRARIES}
-Dtensorflow_WIN_CPU_SIMD_OPTIONS=/arch:AVX
UPDATE_COMMAND ""
DEPENDS ${TensorFlow_DEPENDENCIES}
)
The output from the TensorFlow configuration stage is
Creating directories for 'TensorFlow'
Performing download step (git clone) for 'TensorFlow'
Cloning into 'TensorFlow'...
Checking out files: 100% (7280/7280), done.
Already on 'master'
Your branch is up-to-date with 'origin/master'.
No update step for 'TensorFlow'
No patch step for 'TensorFlow'
Performing configure step for 'TensorFlow'
Not searching for unused variables given on the command line.
-- The C compiler identification is MSVC 19.0.24215.1
-- The CXX compiler identification is MSVC 19.0.24215.1
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe -- wor
ks
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test COMPILER_OPT_ARCH_NATIVE_SUPPORTED
-- Performing Test COMPILER_OPT_ARCH_NATIVE_SUPPORTED - Failed
-- Performing Test COMPILER_OPT_WIN_CPU_SIMD_SUPPORTED
-- Performing Test COMPILER_OPT_WIN_CPU_SIMD_SUPPORTED - Success
-- Found PythonInterp: C:/Users/kasper/Anaconda3/python.exe (found version "3.5.2")
-- Found PythonLibs: C:/Users/kasper/Anaconda3/libs/python35.lib (found version "3.5.2")
-- Found SWIG: C:/Development/build/t/SWIG/swig.exe (found version "3.0.10")
-- Configuring done
-- Generating done
-- Build files have been written to: C:/Development/build/t/TensorFlow-build
Then adding message(STATUS "CMAKE_GENERATOR: ${CMAKE_GENERATOR") to the top-level TensorFlow CMakeLists.txt and running cmake . in the TensorFlow-build directory gives
-- CMAKE_GENERATOR: Visual Studio 14 2015 Win64
-- Configuring done
-- Generating done
The value of CMAKE_GENERATOR in the TensorFlow external project is Visual Studio 14 2015 Win64. I also tried to add -G ${CMAKE_GENERATOR} to CMAKE_ARGS but to no avail. Task manager still shows MSBuild (32 bit) and Microsoft Compiler/Linker Driver (32 bit) when the external project starts.
The code can be found here: https://github.com/kaspermarstal/TensorFlowImageFilter.
What am I missing?

Related

CMake cant find instance of Visual Studio 15 2017 generator, but works fine with Visual Studio 16 2019

Similar to CMake: Visual Studio 15 2017 could not find any instance of Visual Studio but in my case I've got my main installations version of MSVC generator working. It's the 2017 generator that doesn't seem to be working for me. Here is my installation setup:
My understanding is that this should work, I should have the appropriate tools.
When I run cmake -G "Visual Studio 16 2019" ../
I get
-- Selecting Windows SDK version 10.0.18362.0 to target Windows 10.0.18363.
-- The C compiler identification is MSVC 19.28.29337.0
-- The CXX compiler identification is MSVC 19.28.29337.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Professional/VC/Tools/MSVC/14.28.29333/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Professional/VC/Tools/MSVC/14.28.29333/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: C:/.../build
Looks good so far. But when I run cmake -G "Visual Studio 15 2017" ../
I get
CMake Error at CMakeLists.txt:2 (project):
Generator
Visual Studio 15 2017
could not find any instance of Visual Studio.
-- Configuring incomplete, errors occurred!
See also "C:/.../build/CMakeFiles/CMakeOutput.log".
I'm currently use this version of cmake:
cmake -version
cmake version 3.20.0-rc2
CMake suite maintained and supported by Kitware (kitware.com/cmake).
so I should have all the latest methods of using cmake.
I know I could manually configure the generators, but that sticks with the cmake cache and gets deleted constantly, and I was under the impression CMake could handle multiple visual studio instances with out such manual tuning, and that I should be able to use environment variables to point to each one at the very least, though that also doesn't appear to be working.
If you want to generate targeting the Visual Studio 2017 platform toolset (v141), but using Visual Studio 2019 as the generator, you'd use the following CMake generate command
cmake -G "Visual Studio 16 2019" -T v141 ..
I think this is a case of unfortunate naming. Installing the "VS 2017 C++ x64/x86 build tools" only installs the compiler, linker and standard libraries and not actually the entire "Visual Studio Build Tools 2017" which include MSBuild. But you need the latter to use the "Visual Studio 15 2017" generator. You can find the installer for the 2017 build tools at https://aka.ms/vs/15/release/vs_buildtools.exe

CMake build Tensorflow C++ on Windows 10 Error : Test COMPILER_OPT_ARCH_NATIVE_SUPPORTED - failed

I want to build Tensorflow C++ on Windows 10 using CMake. However I cannot pass the COMPILER_OPT_ARCH_NATIVE_SUPPORTED test, which resulting in failed building attempts. Here is my open issue on tf github repo: https://github.com/tensorflow/tensorflow/issues/24076, which has received no response thus why I am asking on stackoverflow and hopefully someone here can enlighten me. I am running out of ideas now. I have tried Python 3.5 and 3.6, visual studio 2017 and 2015. All with no success.
System information
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10
TensorFlow installed from (source or binary): source
TensorFlow version: the latest version
Python version: 3.6
Bazel version (if compiling from source): I am using CMake
GCC/Compiler version (if compiling from source): gcc 6.30
CUDA/cuDNN version: Installing only the CPU version
I am also using Microsoft Visual Studio Community 2017 Version 15.9.3, btw
Describe the problem
I am following the "Step by step Windows Build" from https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/cmake. However, at step 3, I can't seem to pass Performing Test COMPILER_OPT_ARCH_NATIVE_SUPPORTED, which resulting failed building process.
Provide the exact sequence of commands / steps that you executed before running into the problem
PS C:\Users\bw\tensorflow\tensorflow\contrib\cmake\build> cmake .. -A x64 -Thost=x64 -DCMAKE_BUILD_TYPE=Release `
>> -DSWIG_EXECUTABLE='C:\Program Files\swigwin-3.0.12\swig.exe' `
>> -DPYTHON_EXECUTABLE='C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\python.exe' `
>> -DPYTHON_LIBRARIES='C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\libs\python36.lib'
-- Building for: Visual Studio 15 2017
CMake Warning at CMakeLists.txt:9 (message):
Your current cmake generator is set to use 32 bit toolset architecture.
This may cause "compiler out of heap space" errors when building. Consider
using the flag -Thost=x64 when running cmake.
-- The C compiler identification is MSVC 19.16.27024.1
-- The CXX compiler identification is MSVC 19.16.27024.1
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/bin/Hostx64/x64/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/bin/Hostx64/x64/cl.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/bin/Hostx64/x64/cl.exe
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/bin/Hostx64/x64/cl.exe -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test COMPILER_OPT_ARCH_NATIVE_SUPPORTED
-- Performing Test COMPILER_OPT_ARCH_NATIVE_SUPPORTED - Failed
-- Performing Test MSVC_OPENMP_SUPPORT
-- Performing Test MSVC_OPENMP_SUPPORT - Success
-- Found PythonInterp: C:/Program Files (x86)/Microsoft Visual Studio/Shared/Python36_64/python.exe (found version "3.6.7")
-- Found PythonLibs: optimized;C:/Program Files (x86)/Microsoft Visual Studio/Shared/Python36_64/libs/python36.lib;debug;C:/Program Files (x86)/Microsoft Visual Studio/Shared/Python36_64/libs/python36_d.lib (found version "3.6.7")
-- Found SWIG: C:/Program Files/swigwin-3.0.12/swig.exe (found version "3.0.12")
CMake Error at tf_python.cmake:811 (string):
string sub-command REPLACE requires at least four arguments.
Call Stack (most recent call first):
CMakeLists.txt:583 (include)
CMake Error at tf_python.cmake:812 (string):
string sub-command REPLACE requires at least four arguments.
Call Stack (most recent call first):
CMakeLists.txt:583 (include)
CMake Error at tf_python.cmake:813 (string):
string sub-command REPLACE requires at least four arguments.
Call Stack (most recent call first):
CMakeLists.txt:583 (include)
-- Configuring incomplete, errors occurred!
See also "C:/Users/bw/tensorflow/tensorflow/contrib/cmake/build/CMakeFiles/CMakeOutput.log".
See also "C:/Users/bw/tensorflow/tensorflow/contrib/cmake/build/CMakeFiles/CMakeError.log".
Any other info / logs
CMakeOutput.log: https://www.dropbox.com/s/7fweyunxdbmxa1k/CMakeOutput.log?dl=0
CMakeError.log: https://www.dropbox.com/s/tucx0tl6346kdpd/CMakeError.log?dl=0
The first line of the guide states:
CMAKE build is deprecated for TensorFlow. Please use bazel to build TF for all platforms. For details, see the TensorFlow install guide.
Apparently you need to use a release version that is known to build correctly via CMAKE on Windows. Probably should switch to the recommended build system if you want to build the latest code.
Looking at the CMakeLists.txt file for this project shows that Performing Test COMPILER_OPT_ARCH_NATIVE_SUPPORTED - Failed is the step where it checks if the compiler accepts the -march=native flag. It doesn't, see this fix https://github.com/tensorflow/tensorflow/issues/8724#issuecomment-289326917.
The actual errors are due to a scripting issue in tf_python.cmake that cannot properly parse one of the source files. It seems to me that the issues are unrelated.

CMake with Boost library Windows 10 Library not found correctly

Like many others i have problems using boost libraries with windows. On Ubuntu 16.04 it works all very well with libboost-all-dev but on windows i have many problems.
I try to build a cryptonote application which i can compile completely without any errors under linux. But i need also windows binaries so i did the steps to install the following tools:
Visual Studio 2013 (vc120)
CMake 3.10.1
Python 3.6.4
Boost 1.58
For boost i did the following steps:
Installing boost from binary
Run bootstrap.bat
Run b2 install
Run bjam install "--with-some-libs"
Nothing works. The cryptonote throws me an error that some but not all libraries could not be found.
I tried to set the BOOST_ROOT, BOOST_LIBRARY_DIRS, BOOST_INCLUDE_DIRS -> Nothing.
What i did wrong? The error i get from cmake is this:
-- Building for: Visual Studio 12 2013
-- The C compiler identification is MSVC 18.0.31101.0
-- The CXX compiler identification is MSVC 18.0.31101.0
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio 12.0/VC/bin/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio 12.0/VC/bin/cl.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio 12.0/VC/bin/cl.exe
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio 12.0/VC/bin/cl.exe -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - not found
-- Found Threads: TRUE
CMake Error at C:/Program Files/CMake/share/cmake-3.10/Modules/FindBoost.cmake:1928 (message):
Unable to find the requested Boost libraries.
Boost version: 1.58.0
Boost include path: C:/local/boost_1_58_0
Could not find the following static Boost libraries:
boost_system
boost_filesystem
boost_thread
boost_date_time
boost_chrono
boost_regex
boost_serialization
boost_program_options
Some (but not all) of the required Boost libraries were found. You may
need to install these additional Boost libraries. Alternatively, set
BOOST_LIBRARYDIR to the directory containing Boost libraries or BOOST_ROOT
to the location of Boost.
Call Stack (most recent call first):
CMakeLists.txt:113 (find_package)
-- WARNING: Git was not found!
-- Found PythonInterp: C:/Users/chris/AppData/Local/Programs/Python/Python36-32/python.exe (found version "3.6.4")
CMake Warning in CMakeLists.txt:
CMAKE_SKIP_INSTALL_RULES was enabled even though installation rules have
been specified
-- Configuring incomplete, errors occurred!
See also "C:/Users/chris/Documents/GitHub/cryptonote/build/CMakeFiles/CMakeOutput.log".
See also "C:/Users/chris/Documents/GitHub/cryptonote/build/CMakeFiles/CMakeError.log".
Maybe someone can help to fix this issue ?
Greetings
Ok finally i've found a solution after reading many, many and many more pages on the internet.
The solution is really simple. After downloading the boost package and installing it (or building from source) you just need the following two parts:
Open your CMD (cmd.exe)
switch to your installation directory of boost (example C:\local\boost_1_58_0)
Just run the following commands:
bootstrap.bat
b2 link=static runtime-link=static release stage
This will create all librarys static and makes it useable for cmake.
This solution works for me and made me able to build the needed package.
Thanks for all helpers
Or you could just tell boost which compiler to use by adding
toolset=msvc-12.0
(for MS 2013 v12, etc.)
to the b2 command line :-)

AMQP-CPP RabbitMQ Build Integrating to CPP Project

Hi I currently try to integrate RabbitMQ via AMQPCPP into my VisualStudio Project. I am restricted to use a Windows PC which is a pain in the ass for the installation.
I think I was able to build the Project using CMAKE But I received some Error doing it . Now I am trying to include the build into the VS2017 Project. In the Readme https://github.com/CopernicaMarketingSoftware/AMQP-CPP this is done using
#include <amqpcpp.h>
But this header is not found in my build. I am not used to CPP so I hope I havent done some rudimental error. Thanks in advance!
Steps to build AMQP-CPP using CMake in Windows - Visual Studio 2017
Open command prompt and switch to AMQP-CPP git/source path.
1) >mkdir build
2) >cd build
3) >cmake -G "Visual Studio 15 2017 Win64" ..\
-- Selecting Windows SDK version 10.0.15063.0 to target Windows 6.1.7601.
-- The C compiler identification is MSVC 19.10.25019.0
-- The CXX compiler identification is MSVC 19.10.25019.0
-- Check for working C compiler: E:/Program Files (x86)/Microsoft Visual Studio/2017/Professional/VC/Tools/MSVC/14.10.25017/bin/HostX86/x64/cl.exe
-- Check for working C compiler: E:/Program Files (x86)/Microsoft Visual Studio/2017/Professional/VC/Tools/MSVC/14.10.25017/bin/HostX86/x64/cl.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: E:/Program Files (x86)/Microsoft Visual Studio/2017/Professional/VC/Tools/MSVC/14.10.25017/bin/HostX86/x64/cl.exe
-- Check for working CXX compiler: E:/Program Files (x86)/Microsoft Visual Studio/2017/Professional/VC/Tools/MSVC/14.10.25017/bin/HostX86/x64/cl.exe -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: D:/Projects/AMQP-CPP/build
4) >cmake --build . --target ALL_BUILD --config RELEASE --clean-first -- /m
Build succeeded.
"D:\Projects\AMQP-CPP\build\ALL_BUILD.vcxproj" (default target) (1) -> "D:\Projects\AMQP-CPP\build\amqpcpp.vcxproj" (default target) (3) -> (ClCompile target) ->
D:\Projects\AMQP-CPP\src\deferredreceiver.cpp(48): warning C424
4: 'argument': conversion from 'uint64_t' to 'uint32_t', possible loss o
f data [D:\Projects\AMQP-CPP\build\amqpcpp.vcxproj]
1 Warning(s)
0 Error(s)
Time Elapsed 00:00:44.16
How to include amqpcpp in your project
Edit your project CMakeLists.txt file for adding amqpcpp include path and library path for linking.
include_directories(../include)
link_directories(../build/bin/Release)
Include and link directory path may differ based on actual location in your machine.

Cmake Visual Studio 2015 Identification Unknown

So basically i am trying to use Cmake to build a project i am working on, the project uses GLEW (as shared library). GLEW links to both glu32.lib and opengl32.lib. They are added to the CMakelists.txt of glew on the target_link_libraries call like:
target_link_libraries(glew PUBLIC opengl32 glu32)
When i try to generate code for using the viusal studio compiler (2015 Community). I get problems with CMake adding the link libraries to the Visual studio project with -lopengl32 and -lglue32. This is wrong since visual studio cant handle this and gives met the following error when trying to compile:
LINK : warning LNK4044: unrecognized option '/lglu32.lib';
LINK : warning LNK4044: unrecognized option '/lopengl32.lib';
The output of running the Visual Studio Generator is:
C:\Project\build>cmake .. -G "Visual Studio 14 2015 Win64"
-- The C compiler identification is unknown
-- The CXX compiler identification is unknown
-- Check for working C compiler using: Visual Studio 14 2015 Win64
-- Check for working C compiler using: Visual Studio 14 2015 Win64 -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler using: Visual Studio 14 2015 Win64
-- Check for working CXX compiler using: Visual Studio 14 2015 Win64 -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Found OpenGL: opengl32
-- Configuring done
-- Generating done
-- Build files have been written to: C:/Project/build
The following two lines:
-- The C compiler identification is unknown
-- The CXX compiler identification is unknown
seem quite odd to me since it is able to run the Visual Studio compiler. I read something about this being the cause of the invalid -l addition to the linker.
Is there anyone that had this problem and knows how to solve it?
Thanks in advance
CMakeError.txt:
Compilation of the C compiler identification source "CMakeCCompilerId.c" did not produce an executable in "C:Project/build/CMakeFiles/3.5.2/CompilerIdC".
Compilation of the CXX compiler identification source "CMakeCXXCompilerId.cpp" did not produce an executable in "C:/Project/build/CMakeFiles/3.5.2/CompilerIdCXX".
After spending some time searching i fixed it! The actual path of the project i was trying to build contained a SPACE in its path. Appearantly CMake can't handle this.