CMake, C++ and Jenkins/Continuous integration - c++

I have a CMake sample project that I would like to build on Jenkins running on Ubuntu 15.10. I have installed:
https://wiki.jenkins-ci.org/display/JENKINS/CMake+Plugin
And created two build steps:
Run CMake to generate makefiles
Run make all from the build dir
It works fine:
[build] $ cmake -G "Unix Makefiles" -D CMAKE_BUILD_TYPE=Debug /var/lib/jenkins/workspace/cmake-test/cmake-gtest/src
-- Configuring done
-- Generating done
-- Build files have been written to: /var/lib/jenkins/workspace/cmake-test/cmake-gtest/build
[build] $ /usr/bin/make
[ 4%] Built target libfoo
[ 9%] Built target libbar
[ 14%] Built target myApp
[ 52%] Built target gmock
[ 90%] Built target gtest
[100%] Built target testfoo
[cmake-test] $ /bin/sh -xe /tmp/hudson1792271459427590561.sh
+ cd cmake-gtest/build
+ make all
[ 4%] Built target libfoo
[ 9%] Built target libbar
[ 14%] Built target myApp
[ 52%] Built target gmock
[ 90%] Built target gtest
[100%] Built target testfoo
Finished: SUCCESS
But is this the recommended approach for using CMake in a CI/Jenkins setup?
Currently, my CMake/Jenkins build will on each push; 1) generate the makefiles, 2) build the project.
I am a bit worried that the first step 1) generate makefiles would eat up build time and it does not really seem optimal to do this step on each push. Especially since I would not expect to change CMakeLists.txt files that often, but when they change newly generated files should, of course, be used.
Is the above approach common practices that I just have to get used to or have I missed something?

Yes, you can do it in one step.
For example, in my Jenkins environment, when building the Ubuntu jobs I use the CMake plugin for the whole compilation, as it allows multiple build tool invocations.
My screenshot at the bottom of this post is of a CMake step in a job (I use Ninja instead of Unix Makefiles, but the effect is the same).
I use two build invocations:
Blank - equivalent to calling ninja or make in a shell,
Install - equivalent to calling DESTDIR=. ninja install.
If I wanted to build additional targets from the makefiles, I could just add extra invocations to this step.
Note that in your screenshot you have a blank invocation in the configuration. This will already be calling make, and as confirmed by your log output, you are in fact compiling your project twice, because of your manual call to make all in the following step.
You can remove your shell step and your project will still build.
Regarding your question on best practices and regenerating CMake, I refer you to this article on Jenkins Best Practices where it states:
To ensure a build can be reproducible, the build must be a clean build, which is built fully from Source Code Control. This practice also implies that all code including third-party jars, build scripts, release notes, etc. must be checked into Source Code Control.
Note that I also check "Clean Build" in my CMake step, so that the entire CMake workspace is wiped out and the project is generated from scratch for every build. This ensures there are no issues caused by stale cache variables, etc.
Screenshot of a CMake step in one of my jobs:

I am not even sure you need this plugin. AFAIR the plugin webpage it is only useful if you want Jenkins to use a specific CMake version, or you want a GUI for which CMake variables you might want to set. I just execute CMake from the command line.
For your second part of the question, If you change CMakeLists.txt, the Makefile will automatically rerun CMake, so strictly speaking it is not necessary to run CMake each time. But on the other side, CMake configuring is quite fast and most likely takes less time than compiling.

Related

How to install a cpp library using cmake on Windows x64?

I'm using CLion with MinGW-GCC on the Windows-x64 platform - This is the background of the problem.
I was trying to install gtest before. But a lot of confusion arose in the middle.
First time I ran those commands(in googletest-release-1.12.1\) according to the instructions of googletest-release-1.12.1\googletest\README.md:
mkdir build
cd build
cmake ..
But I got error messages like:
CMake Error at CMakeLists.txt:51 (project):
Failed to run MSBuild command:
C:/Windows/Microsoft.NET/Framework/v4.0.30319/MSBuild.exe
to get the value of VCTargetsPath:
Then I changed my last command to
cmake -G "MinGW Makefiles" ..
because I use make provided by MinGW. I don't know whether it's right but, it ran properly.
then I called
make
make install
make ran smoothly. But when I ran make install, I got these messages:
Install the project...
-- Install configuration: ""
-- Installing: C:/Program Files (x86)/googletest-distribution/include
CMake Error at googlemock/cmake_install.cmake:41 (file):
file INSTALL cannot make directory "C:/Program Files
(x86)/googletest-distribution/include": No such file or directory.
Call Stack (most recent call first):
cmake_install.cmake:42 (include)
make: *** [Makefile:109: install] Error 1
I have no idea at all this time. So I changed my way. According to this answer, I copied the whole library into my project and edited CMakeLists.txt like this:
cmake_minimum_required(VERSION 3.23)
project(gtest_study)
set(CMAKE_CXX_STANDARD 20)
add_subdirectory(googletest-release-1.12.1)
include_directories(googletest-release-1.12.1/googletest/include)
include_directories(googletest-release-1.12.1/googlemock/include)
add_executable(gtest_study main.cpp)
target_link_libraries(gtest_study gtest gtest_main)
target_link_libraries(gtest_study gmock gmock_main)
So my questions are:
Is there any difference between the two which build it using make and cmake metioned firstly, and just use commands like include_directories and target_link_libraries in CMakeLists.txt? (maybe like .h and .dll file? Or just completely the same? I don't know)
When I use make install to install a library on Windows, what should I do in particular? Specify some directory (I don't know which one) or what?
Although in my system environment I use MinGW-makefile, in CLion which the libraries are eventually used, I use ninja as the generator for CMake (it just comes with CLion, not installed for the system). Do I have to specify it and how? (-G "Ninja"doesn't work in my native env)
The difference between
cmake ..
and
cmake -G "MinGW Makefiles" ..
Is the choice of generator: The former uses the default generator, the latter uses the generator you specified. (cmake --help should put a * next to the default generator.)
Based on the error message I assume this is a visual studio generator and you may not be able to run that one properly from within a MinGW terminal.
In the latter case the default install directory seems to be based on the target OS (Windows) but does not seem to incorporate the fact that you're running from a MinGW terminal where the default install path (C:/Program Files (x86)/googletest-distribution) is not valid.
You could try to fix this by providing it during cmake configuration (passing -D 'CMAKE_INSTALL_PREFIX=/c/Program Files (x86)/googletest-distribution' before the source dir) or by providing the install directory during the installation.
The following process should allow you to install the lib. I'm using my preferred way of building here, i.e. not using build system dependent commands, but using cmake to run the build/install commands. I assume the working directory to be the root directory of the gtest sources:
cmake -G "MinGW Makefiles" -S . -B build
cmake --build build
cmake --install build --prefix '/c/Program Files (x86)/googletest-distribution'
The last command needs to be run with admin privileges, the first 2 I don't recommend running as admin. You could instead install to a directory where you do have the permissions to create directories even without admin privileges.
The difference between using the process described above and using add_subdirectory is that the former results in a installation on the system which can be used via find_package and the google test libs won't be rebuilt for every project where you do this.
...
project(gtest_study)
...
# you may need to pass the install location via -D CMAKE_PREFIX_PATH=<install_location> during configuration for this to work
find_package(GTest REQUIRED)
target_link_libraries(gtest_study PRIVATE GTest::gtest_main GTest::gmock)
The latter builds the google test project as part of your own project build and for every project where you use this approach a seperate version of the google test libs is built. Note: there should be no need to specify the include dirs yourself, since this kind of information is attached to the cmake target and gets applied to the linking target automatically:
#include_directories(googletest-release-1.12.1/googletest/include)
#include_directories(googletest-release-1.12.1/googlemock/include)
add_executable(gtest_study main.cpp)
target_link_libraries(gtest_study PRIVATE gtest_main gmock)
As for 3.: The CMake generator used for building GTest should be independent of the generator of the project using it. The thing that's important is that the compilers used by the build systems are compatible. I cannot go into detail about this, since I've never used CLion and therefore have too little knowlege about the compilers used by it. (Personally I'm working with Visual Studio on Windows.)

LLVM build only selected tools or targets

I am new to c++ & am referring to the llvm project https://llvm.org/docs/GettingStarted.html. There is a guide saying
If you are space-constrained, you can build only selected tools or only selected targets. The Release build requires considerably less space.
But I am unable to find the exact way to do it.
Currently, I am running the following to build all the tools
mkdir build
cmake ../llvm
make
This would create many tools such as llvm-addr2line, llvm-ar, llvm-dwarfdump etc.
How could I build it such that it only gives me 1 tool, eg llvm-dwarfdump (eg https://llvm.org/docs/CommandGuide/llvm-dwarfdump.html)?
Is there an easy way to know the code that makes up these command.
Thank you
CMake generates a target for every add_library, add_executable, and add_custom_target command. These targets can be built separately by calling make <target> or with the build tool independent CMake abstraction cmake --build <builddir> --target <targetname> --config Release. To find out which targets have been defined by LLVM you need to inspect all CMakeLists.txt and *.cmake files.

Unit testing by CTest of Cross-compiled Project

I am developing hobby OS which is cross-compiled to other architecture than my development PC and is run in QEMU emulator.
I am trying to introduce unit testing of the source files I develop for my kernel but to let them run on my development machine rather than in QEMU on my target.
In order to cross-compile my sources, I use the toolchain file which uses my custom GCC toolchain. As coming from CMake principle of toolchain file usage, the toolchain file is set up prior to project(...) definition in my root CMakeLists.txt file.
My question is how to build my test executables for my dev machine (x86_64) using the built-in GCC while cross-compiling my kernel binary for target platform... I do not know how to set CMAKE_CXX_COMPILER / CMAKE_C_COMPILER cmake variables specifically for each use case (testing, target build)
To have a look at my particular project, please feel free to navigate here:https://gitlab.com/revolta/revolta
I would like to add test/ to my project root including selected sources from source/... and manage it somehow from my root CMakeLists.txt
Thanks in advance for any concept ideas and help! Cheers Martin
The rule is - there is one compiler per configuration. Do not try to make to use two compilers per configuration. Instead run cmake two times and configure it twice,. separately for x86 testing and separately for releasing to target build.
So write a small script (you have configure.sh anyway) (I usually write a makefile with all PHONY targets) that would run and build the project twice for two configurations:
# ./compile_and_test_your_project.sh
# build for target host
cmake -DCMAKE_TOOLCHAIN_FILE=the_toolchain -S. -B_build/crosscompiled
cmake --build _build/crosscompiled --target the_main_project_target
( cd _build/crosscompiled && ctest )
# build for native host
cmake -DCMAKE_C_FLAGS="-fsanitize=undefined -ggdb3 -O0" -S. -B_build/native
cmake --build _build/native --target only_testing_targets
( cd _build/native && ctest )
Do not set CMAKE_CROSSCOMPILING_EMULATOR inside cmake config. I advise to try to keep cmake configuration platform agnostic as much as you can and pass platform specific parts using arguments to cofiguration cmake. Such way is scalable - you may use a different toolchain and different environment with ease, or try different compiler options. Inside cmake you can see if you are crosscompiling with just if (CMAKE_CROSSCOMPILING). Also see CMAKE_CROSSCOMPILING_EMULATOR.
ps. My makefile from one of my projects that sets different CMAKE_CROSSCOMPILING_EMULATOR depending on make target.

CMake: target "test" doesn't build tests, target "all" does

I created a command-line portable script-based industrialized agnostic build system useful to quickly build several dependent projects while not having to rely on a particular IDE or a build factory. It's agnostic, because it's not based on a single build engine. I founded the first version using cmake, because my projects are mostly C++, but that'll evolve (to include gradle for example or whatever). That's to say, I'm not centered on CMake, it's merely a tool to a goal, easy portable C++ project building. I had BJam in mind formerly and would have kept it if there had been more documentation.
As a result though, I'm very dependent on CMake to perform the build and unit tests. As of today, I realized that tests are built under the 'all' target and run under the 'test' target.
With CMake 2- (and -here for example- a Unix Makefiles generator):
make all # Build project AND tests
make test # Run tests
With CMake 3+ and any generator:
cmake --build . --target all # Build project AND tests
cmake --build . --target test # Run tests
I'd like to know if someone would know a way to split the 'build project' phase apart from the 'build tests' phase (also because it feels more natural in my build system to join test building and running tests than the other way around).
Important precision: I don't want to bootstrap the project with one vision or another (by flipping BUILD_TESTING). The idea would be to have 3 stages like:
cmake --build . --target <project> # 1. Build project only
cmake --build . --target <build_tests> # 2. Build tests
cmake --build . --target <run_tests> # 3. Run tests
If I choose not to run tests, I could go straight from phase 1 above to installing, but running phase 3 would trigger the previous dependent phases.
Any clue? (If not, I suspect I'll have to ask CMake developers directly...)
Thanks in advance. Regards.
Assuming you choose build_tests target for build test's executables and run_tests target for run them:
Defining targets:
add_custom_target(build_tests)
# 'run_tests' triggers 'ctest'.
add_custom_target(run_tests COMMAND ${CMAKE_CTEST_COMMAND})
# 'run_tests' implies 'build_tests'
add_dependencies(run_tests build_tests)
Creating test's executables, so they won't be built by default but with build_tests target.
# Do not build 'test1' by default
add_executable(test1 EXCLUDE_FROM_ALL ...)
# 'build_tests' implies (among other things) building 'test1'
add_dependencies(build_tests test1)

KDE app does not install after 'make install'

I have just started having a go at developing a small KDE app as a test for myself and I have had some problems fathoming how to use cmake which I have largely overcome. Currently my app builds ok (i.e. cmake && make run without errors). However running 'make install' gives the following error:
kemra102#kaon:~/projects/kquickformat/build> make install
[ 0%] Built target kquickformat_automoc
Scanning dependencies of target kquickformat
[ 50%] Building CXX object src/CMakeFiles/kquickformat.dir/kquickformat_automoc.o
Linking CXX executable kquickformat
[100%] Built target kquickformat
Linking CXX executable CMakeFiles/CMakeRelink.dir/kquickformat
Install the project...
-- Install configuration: ""
-- Installing: /usr/local/bin/kquickformat
CMake Error at src/cmake_install.cmake:38 (FILE):
file INSTALL cannot copy file
"/home/kemra102/projects/kquickformat/build/src/CMakeFiles/CMakeRelink.dir/kquickformat"
to "/usr/local/bin/kquickformat".
Call Stack (most recent call first):
cmake_install.cmake:37 (INCLUDE)
make: *** [install] Error 1
Any ideas as nothing seems obvious to myself?
To install to default locations you usually need root privileges. So either use sudo make install, or rerun cmake specifying an installation prefix where you have write permission. For example, I use:
cmake -DCMAKE_INSTALL_PREFIX=$HOME/install/
If you do that, remember also to set all Qt and KDE related environment variables, otherwise the application you installed will not be working properly or at all. I usually call a shell function that does something like:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/install/lib
export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:$HOME/install/lib/pkgconfig
export KDEDIR=$HOME/install
export KDEDIRS=$KDEDIR
export XDG_DATA_DIRS=$XDG_DATA_DIRS:$HOME/install
export QT_PLUGIN_PATH=$QT_PLUGIN_PATH:$HOME/install/plugins
Your user account does not (at least, should not) have write access to /usr/local/bin. Try sudo make install.