Code parsing not working with CUDA, Clion and CMake - c++

I have a project divided in modules, here is a dummy example:
root
CMakeLists.txt
modules
utils
CMakeLists.txt
src
util_file.cpp
cuda
CMakeLists.txt
src
cuda_file.cu
If I edit the cuda_file.cu with CLion, all the symbols are unresolved (even the includes from standard library) by CLion. All the code completion/creation features are then of course gone (among other things). The problem seems to be that whenever you create a library or an executable with only CUDA files, Clion becomes stupid and doesn't parse or resolve anything anymore.
There is two workarounds I've found but they are not friendly or "clean" to use :
add an empty .cpp file to the directory and add it to the add_library() CMake line.
switch to another library or executable target that has .cpp files (like utils in my dummy example). But then when you want to compile or execute you have to switch again to cuda target (or some subtarget like test_cuda for test units) and then switch back again to continue coding or debugging, etc...
Here is the CMakeLists.txt from the cuda module with the workaround:
cmake_minimum_required(VERSION 3.5)
message(STATUS "Configuring module cuda")
# Build module static library
FILE(GLOB CUDA_SRCS
${CMAKE_CURRENT_SOURCE_DIR}/src/*.cpp)
FILE(GLOB CUDA_CU_SRCS
${CMAKE_CURRENT_SOURCE_DIR}/src/*.cu)
FILE(GLOB CUDA_CU_HDRS
${CMAKE_CURRENT_SOURCE_DIR}/include/*.cuh)
cuda_compile(cuda_objs ${CUDA_CU_SRCS} ${CUDA_CU_HDRS})
add_library(cuda STATIC ${CUDA_SRCS} ${cuda_objs})
# because only .cu files, help cmake detect C++ language
set_target_properties(cuda PROPERTIES LINKER_LANGUAGE CXX)
Is there a way to avoid CLion derping when resolving links to other headers and libraries ?
I've already added .cu and .cuh files as C/C++ code in CLion options and tried using JETBRAINS_IDE define option as explained in another similar post, but those two problems are not the same.

It seems like without the intervention of Jetbrains to add official CUDA support, the most I could get out of the combo CLion + CMake + CUDA was achieved by:
adding .cu and .cuh as C++ files in CLion. This allows Clion to recognize cuda code as C++ code and color it correctly.
adding an empty dummy .cpp file to the cuda source directory if it is only filled by .cu files (one of my "dirty" hacks from my question). I could not find better. This allows Clion to not completely derp. A simple thing like recognizing cstdio doesn't work without this "hack" and CLion is basically an enhanced notepad.
using when possible CMake 3.8+ that officially supports CUDA as a language and use the new "cuda aware" add_library() instead of the old macro defined cuda_add_library(). This can avoid problems in the future in case of deprecation.
in the CMakeLists of the cuda module (or main CMakeLists if only one), include the path to the include directory of cuda to allow Clion to "see" the cuda headers. CLion can then propose to you to include them so that CLion resolves correctly CUDA API calls like cudaMalloc() or cudaFree(). This is only needed for CLion as the CUDA compiler doesn't need this includes to compile properly (cuda.h, cuda_runtime.h, ...).
use this answer to create a "clion helper" header file, so that it doesn't derp on symbols like __device__ or __global__.
I think that if Jetbrains starts adding support for CUDA, not also will it remove the need to add this dummy file, but it will probably also resolve all other things listed.
Here is the link to the nvidia blog with examples about the official cuda language support in CMake and new "cuda aware" add_library() : https://devblogs.nvidia.com/parallelforall/building-cuda-applications-cmake/

As of the version 2020.1 of CLion, CUDA projects are now officially supported
https://blog.jetbrains.com/clion/2020/04/clion-2020-1-cuda-clang-embedded/

Related

How to include source of external library for IntelliSense in a CMake project?

I am working on a C++ application in Visual Studio, using the Visual Studio's CMake project template.
To build my application I only need the header and the external library and I can link it like this:
# CMakeList.txt : CMake project for myapp, include source and define project specific logic here.
cmake_minimum_required (VERSION 3.8)
# Add source to this project's executable.
add_executable (myapp "myapp.cpp" "myapp.h")
# Add TIFF library
set(TIFF_INCLUDE_DIR "C:\\libs\\tiff\\out\\install\\x64-Debug\\include")
set(TIFF_LIBRARY "C:\\libs\\tiff\\out\\install\\x64-Debug\\lib\\tiffd.lib")
find_package(TIFF REQUIRED)
target_link_libraries(myapp PRIVATE TIFF::TIFF)
So far, so good. I can use the tiff library to open, read, write tiff files, etc., and IntelliSense is catching up with declarations in the header files. But, I would like IntelliSense to also be aware of the full source code of the TIFF library. For example, if I am in myapp.cpp and I ctrl+click on TIFFOpen it opens the header corresponding to TIFFOpen, but when I ctrl+click TIFFOpen in the header file, it doesn't go to the corresponding source file, which is the normal behaviour for source files in myapp. This is understandable, since I never told Visual Studio where to find the source files of the external library.
CMake doesn't need to know where the source files of the external libraries are, since it won't build the external library, therefore I guess I don't/shouldn't change anything in CMakeLists.txt
One option (I haven't tried yet, but I'm fairly sure it would work), would be to just include the entire tiff library as a sub-project of myapp. I do have some problems with this solution though:
The external library is not conceptually an integral part of the project, and I don't plan to modify the external library. This is more of a principle issue.
Simply having it as a subfolder in my project makes it a risk of changing something I didn't intend to change.
I don't want to rebuild the external library when I do a rebuild all. I know Visual Studio / CMake is smart enough to figure out that nothing changed and doesn't rebuild, but I would rather have Visual Studio / CMake not even try.
The way I see it, I have to set the directory with the source files somewhere in Visual Studio settings, but still related to the project. My best guess is that the .vs/ProjectSettings.json is the file I need to edit somehow, but honestly, I have no clue.
Alternatively, maybe I could write some command in CMakeLists.txt that doesn't do anything, but triggers the IntelliSense to look in the folder with source files. Again, I have no clue how should I go about this.
In a nutshell, I want IntelliSense to see all source files of an external library, the same way it sees the source files of myapp, without including all source files of the external library as a sub-project of myapp. How should I go about it, if even possible?
If relevant, I use Visual Studio 2019 Community and the CMake it comes with it (3.15).
Regarding your last comment, writing code in comment section is inconvenient so I'll just post it here although not an answer.
libtiff-4.pc is for pkg-config, not cmake, and find_package() can't deal with it directly on Windows and would take some work if you really want to. It might be easier to just write everything manually. Remember to set the tiff.lib and tiffd.lib according to your configuration. You can use CMAKE_BUILD_TYPE variable and if() command such as:
# set build type to release if not specified
if(NOT CMAKE_BUILD_TYPE OR CMAKE_BUILD_TYPE STREQUAL "")
set(CMAKE_BUILD_TYPE "Release" CACHE STRING "Build Type" FORCE)
endif()
# switch lib to link according to build type
if(CMAKE_BUILD_TYPE STREQUAL "Release")
# for release
set(TIFF_LIBRARY "<your_path_to_installed_lib>/tiff.lib")
else()
# for debug. If you have other configs like relwithdebinfo you can add more
set(TIFF_LIBRARY "<your_path_to_installed_lib>/tiffd.lib")
endif()
Also, remove find_package() and use target_link_libraries() and target_inlude_directories():
set(TIFF_INCLUDE_DIR "<your_path_to_installed_headers>/include")
target_link_libraries(myapp PRIVATE ${TIFF_LIBRARY})
target_include_directories(myapp PRIVATE ${TIFF_INCLUDE_DIR})
You can also skip setting TIFF_LIBRARY and TIFF_INCLUDE_DIR and pass the string directly if you like.
I use Visual Studio a lot and it's my favorite IDE. But package management with cmake on Windows is not as smooth as Linux. Always remember to set environment variables after compiling and installing external libraries.
Normally find_package() will look for a system environment variable named <libname>_DIR (for example TIFF_DIR, which is not found in your case), which is used to store path to installed lib, then look for <libname>Config.cmake and <libname>ConfigVersion.cmake in that folder (and would fail for TIFF since it doesn't have them).
It also searches other places, check https://cmake.org/cmake/help/v3.15/command/find_package.html?highlight=find_package for details.
So for those libs with cmake exported files, add a variable with correct name and value. And that's only for compiling.
If you want your application to run after compiling, you also need to add the path of installed lib's binaries (usually *.dll) to system's Path variable. In your case you should find something like tiff.dll and tiffd.dll after compiling and installation, add that folder to Path and you are good to go.
Actually, showing the source in the IDE is easy enough.
Just add the source with add_library(tiff_for_ide EXCLUDE_FROM_ALL ${SOURCES}) but don't link with it in the main program. You'll want to use a different target name for the library. For this you'd need the source in your project directly or as a submodule (if using Git, something else if available by you VCS).
Other options are using ExternalModule_Add; or FetchContent_MakeAvailable with the add_library as above, to avoid adding third party deps into the repository directly.
Now, just cross your fingers and hope that the IDE is not intelligent enough to restrict searching for sources that are linked against the target which compiles the current file. Or that it's intelligent enough to detect this situation but fallback to searching the whole project files anyway when it's linking against a binary.

Including directories for CLion inspections while using custom toolchain

I'm using CLion 2018.2 to write C/C++ code for a custom compiler toolchain which are not supported natively by CLion. I currently compile with make on the Terminal instead of building from within the IDE.
I have a custom include directory with header files which are not resolved/found by CLion since they are not part of the project. However, I want to get code inspection features for them. The headers are e.g. located at C:\devkitPro\wups\include.
I decided to use the include_directories() CMake command to improve CLion's ability to resolved code:
include_directories("C:\\devkitPro\\wups\\include")
Then I also modified the CMake include path:
set(CMAKE_INCLUDE_PATH "C:\\devkitPro\\wups\\include")
And also decided to link against the lib directory:
link_directories("C:\\devkitPro\\wups\\lib")
After doing all that, the headers still aren't resolved in CLion (but it still compiles using make of course). How can the header resolution be done with CLion or is it not possible, yet?
Depending on the configured toolchain in CLion, CMake expects a Windows or a WSL-style path. Inspections will work with the include_directories directive, e.g.
# Add extra include directories
if (WIN32) # When using a Windows compilation toolchain
set(WUT "/c/devkitPro/wut/include")
set(WUPS "/c/devkitPro/wups/include")
else () # When using WSL as toolchain
set(WUT "/mnt/c/devkitPro/wut/include")
set(WUPS "/mnt/c/devkitPro/wups/include")
endif ()
include_directories(${WUT})
include_directories(${WUPS})
A more detailed written tutorial can be found in this pull request.

How to improve my script and copy program resources on build?

So I got into a problem where I needed to use linux for a while instead of windows, and figured linux doesn't have Visual Studio. I then also realized that I made my project Visual Studio only, which I don't want.
So I looked up some CMake tutorials and try'ed creating some examples that could be loaded in both Visual Studio and CodeBlocks. When I got that to work, I went and code a CMake script for my actual program by piecing together what I learned and what I found in tutorials.
See here:
cmake_minimum_required(VERSION 2.8.9)
add_subdirectory(libraries/glfw)
project (OpenGLEngine3D)
include_directories(libraries/glfw/include)
include_directories(libraries/glm)
include_directories(libraries/glad/include)
include_directories(libraries/whereami/src)
include_directories(libraries/stb)
file(GLOB SOURCES "src/*.cpp" "src/*.h" "libraries/whereami/src/whereami.c" "libraries/glad/src/glad.c")
add_executable(OpenGLEngine3D ${SOURCES})
target_link_libraries(OpenGLEngine3D glfw)
Its dirty(I think) but it works.
So now my first question is, how to improve my CMake script? What is redundant or done in a poor way?
Now my program also requires some resources(like shaders, textures) which I have stored in a directory along with my CMake script, libraries and c++ files.
So my second question is, how would I tell CMake to the ide/compiler to copy the files in a certain directory to the program build directory(where the compiled binaries are) after compiling?(And have it only do it when the files aren't there ofc.)
Thanks!
I recently came across this article, which might be help you with CMake.
Here are a couple suggestions from me:
You're safe to use newer version of CMake. Currently, it's 3.11. There's no point in sticking to old versions.
Consider listing all your sources in a variable (using set command) instead of using file(GLOB). file(GLOB) will be evaluated only once, when generating build files. If you add new sources, you'll have to re-generate it manually, so it doesn't help too much, and is harder to debug (and really, debugging more complex CMake projects can be painful).
Avoid using include_directories (and link_libraries). Prefer using target_include_directories which works per-target. It allows to express dependencies between targets (when you have more than one - which might happen as your project grows)
You might consider using find_package(GLFW), instead of including GLFW in your project. Edit: actually, CMake doesn't came with find module for GLFW, but you can use an external module for this (e. g. this one) as described here.
Edit:
Example below illustrates the idea behind point 2), assuming that sources are placed in "src/" directory on the same level as CMakeLists.txt file:
set(engine_sources
"src/header_1.hpp"
"src/source_1.cpp"
# and so on for reset of files...
)
add_executable(OpenGLEngine3D ${engine_sources})
As for the second question: that's what file(COPY) command is for. Alternatively, you could just leave assets in source directory, and set working directory in IDE.
Side note: if you choose the second options, there appears to be a way to set this from CMake for Visual Studio, by setting VS_DEBUGGER_WORKING_DIRECTORY property:
set_target_properties(OpenGLEngine3D PROPERTIES VS_DEBUGGER_WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR})
(I haven't checked if this works as expected, as I rarely use VS. I remember looking for something like that in the past, and just recently came across it.)

Clion as an IDE without compiling

Im trying to use the CLion IDE for Cpp development where actual compilation is happening on a remote sever, Although all the headers files are available, the compiling on local environment would fail since the required libraries are only available in the remote server . Is there a way to use CLion just as an IDE with code completion option only. I've used NetBeans like this where it only provides code completion and i just cant find a way to do this with CLion
If I understand the question correctly, yes, it is possible.
I'm using Clion to write code for microcontroller applications where the actual compilation is done on the command line, not from within the IDE.
I feel like I'm stating the obvious, but you can just not click the "Build" button. You can point your include path to where your header files reside and Clion will give you code completion, as it only needs the headers for that.
Since Clion reads the CMakeLists.txt to figure out your project configuration, you'd need that and set your include path there.
cmake_minimum_required(VERSION 2.8.4)
project(my_cool_project)
set(INCLUDE_DIRECTORIES "/path/to/your/headers")
set(SOURCE_FILES src/main.cpp
src/foo.cpp
src/foo.h
src/bar.cpp
src/bar.h)
add_executable(my_cool_project ${SOURCE_FILES})
As I said, Clion uses the CMakeLists.txt as the project configuration, so look into setting up this file if you're not familiar with it already and you'll be set.

Building CUDA object files using cmake

I got the following setup. I'm going to extend a framework written in C++ using MPI and other Stuff using CUDA. The Project uses cmake for building. I would like to avoid using a library for my extensions and build object files from my cuda sources. Afterwards I would like to link these object object files and some other files compiled with other compilers.
Does anyone have a clue on hwo to achieve that?
I had a look at http://code.google.com/p/cudpp/wiki/BuildingCUDPPwithCMake for getting an overview on how to use CUDA with cmake but this solution uses a library as well.
It is possible to compile object files with the CUDA support that comes with newer versions of cmake. You use the cuda_compile command. See below.
# CMakeLists.txt for G4CU project
project(test-cuda-thrust-gdb)
# required cmake version
cmake_minimum_required(VERSION 2.8)
# packages
find_package(CUDA)
# nvcc flags
set(CUDA_NVCC_FLAGS ${CUDA_NVCC_FLAGS};-gencode arch=compute_20,code=sm_20)
cuda_compile(HELPER_O helper.cu)
cuda_compile(DRIVER_O driver.cu OPTIONS -G)
cuda_add_executable(driver ${HELPER_O} ${DRIVER_O})
If you need more information have a look at the FindCUDA.cmake file.