CMake file for integrated Visual Studio unit testing - c++

Visual Studio 2017 has integrated C++ unit testing (native, google test, ctest, etc.). How can I create a CMakeLists.txt file that will create a project like this that will use the integrated IDE testing, for example using either google test or the native microsoft unit testing framework? All of Microsoft's examples unfortunately just create the project in Visual Studio, rather than starting from a CMake file.
Any help is appreciated, thanks!

Mikewho,
I setup a small example using Google Test project that works with integrated IDE testing.
Create an empty directory and save these two files:
CMakeLists.txt
cmake_minimum_required(VERSION 3.0)
project(test_me)
# GTest
enable_testing()
find_package(GTest REQUIRED)
include_directories(${GTEST_INCLUDE_DIRS})
# Unit Tests
# Add test cpp file
add_executable( runUnitTests tests.cpp)
# Link test executable against gtest & gtest_main
target_link_libraries(runUnitTests ${GTEST_BOTH_LIBRARIES})
add_test( runUnitTests runUnitTests )
tests.cpp
#include <gtest/gtest.h>
TEST(ABC, TEST1) {
EXPECT_EQ(true, true);
}
The in a command prompt type
mkdir build
cd build
cmake .. "-DCMAKE_TOOLCHAIN_FILE=C:/dev/vcpkg/scripts/buildsystems/vcpkg.cmake"
Note: I had vcpkg install gtest
C:\dev\vcpkg>vcpkg.exe install gtest
Make sure you have this installed in Visual Studio 2017
In Tools > Options > Test Adapter for Google Test set the regex to .exe
Build the solution and press Run all in the Test Explorer
The first time it runs it will find the test case
[12/3/2018 8:38:41 AM Informational] ------ Run test started ------
[12/3/2018 8:38:42 AM Warning] Could not locate debug symbols for 'C:\dev\cpptests\GoogleTest\build\Debug\runUnitTests.exe'. To make use of '--list_content' discovery, ensure that debug symbols are available or make use of '<ForceListContent>' via a .runsettings file.
[12/3/2018 8:38:42 AM Informational] Test Adapter for Google Test: Test execution starting...
**[12/3/2018 8:38:42 AM Informational] Found 1 tests in executable** C:\dev\cpptests\GoogleTest\build\Debug\runUnitTests.exe
[12/3/2018 8:38:42 AM Informational] Running 1 tests...
[12/3/2018 8:38:42 AM Informational] Google Test execution completed, overall duration: 00:00:00.2390446
[12/3/2018 8:38:42 AM Informational] ========== Run test finished: 1 run (0:00:01.2668844) ==========
I hope this helps?

Although this is a late reply, but it's the latest answer for any reference.
For Microsoft Unit Testing Framework, the following simplified files work with me:
CMakeLists.txt
set(UNIT_TEST_TARGET_NAME my_lib)
add_library(${UNIT_TEST_TARGET_NAME} SHARED)
target_sources(${UNIT_TEST_TARGET_NAME} PRIVATE
tests.cpp
tests.h
)
target_link_libraries(${UNIT_TEST_TARGET_NAME}
PRIVATE MSUnitTestFramework::MSUnitTestFramework)
enable_testing()
find_program(VSTest_EXECUTABLE NAME vstest.console.exe REQUIRED)
list(PREPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_LIST_DIR}/cmake")
find_package(MSUnitTestFramework REQUIRED)
add_test(NAME ${UNIT_TEST_TARGET_NAME}
COMMAND "${VSTest_EXECUTABLE}" "${CMAKE_ARCHIVE_OUTPUT_DIRECTORY}/${UNIT_TEST_TARGET_NAME}.dll"
)
/cmake/FindMSUnitTestFramework.cmake
add_library(MSUnitTestFramework::MSUnitTestFramework SHARED IMPORTED)
set_target_properties(MSUnitTestFramework::MSUnitTestFramework PROPERTIES
INTERFACE_INCLUDE_DIRECTORIES "$ENV{VCInstallDir}Auxiliary/VS/UnitTest/include"
IMPORTED_IMPLIB "$ENV{VCInstallDir}Auxiliary/VS/UnitTest/lib/x86/Microsoft.VisualStudio.TestTools.CppUnitTestFramework.lib"
)
set(MSUnitTestFramework_FOUND TRUE)
tests.cpp
#include "CppUnitTest.h"
using namespace Microsoft::VisualStudio::CppUnitTestFramework;
namespace my_unit_test
{
TEST_CLASS(my_tests)
{
public:
TEST_METHOD(test1)
.
.
You can Run All Tests and so on from MSVC.
BTW, instead of add_test for the whole DLL, you can use it to add test by test.

Related

Visual Studio 2019 Test Explorer did not find c++ google tests

I want the c++ unit test written in google test displayed in the VS 2019 test explorer.
The tests are setup correctly and can be executed. The results are shown in the VS debug console/commandline-like window. No error messages besides the test dependent messages are shown.
I want start the tests form the test explorer and want to create test play lists.
I installed the google test adapter provided by the VS Installer. I followed guidelines and the suggested troubleshooting at TestAdapterForGoogleTest.
Does another way exist to get the google test to be dispalyed in the test explorer?
What are other known incompatibilities with google test and the VS test explorer?
I had the same problem as you. I'm having a main CMakeLists.txt, and another two CMakeLists.txt files in the subdirectories: one for the static library I'm testing and one for the test project itself.
For making sure the tests appear in the Test Explorer I had to move enable_testing() from the test subdirectory into the main CMakeLists.txt.
option(MY_PROJECT_TESTS "Build unit tests" ON)
if(MY_PROJECT_TESTS)
enable_testing()
add_subdirectory("test")
endif()
Then in the test subdirectory, I'm setting up the GoogleTest environment, and adding test the following way:
set(GTEST_DIR "googletest/googletest" CACHE PATH "gtest directory")
include(GoogleTest)
set(gtest_force_shared_crt OFF CACHE BOOL "" FORCE)
add_subdirectory("googletest")
project(My_project_test)
if (WIN32)
add_library(qtpcre STATIC IMPORTED)
set_target_properties(qtpcre PROPERTIES
IMPORTED_LOCATION_DEBUG ${QT5_DIR}/lib/qtpcre2d.lib
IMPORTED_LOCATION_RELEASE ${QT5_DIR}/lib/qtpcre2.lib
)
endif()
set(CommonTestLib
Qt5::Core
My_project
gtest_main
)
if (WIN32)
list(APPEND CommonTestLib
Ws2_32.lib
version.lib
Netapi32.lib
Userenv.lib
Crypt32.lib
Winmm.lib
qtpcre
)
endif()
add_executable (My_project_test test_main.cpp test_cases.cpp)
target_precompile_headers(My_project_test REUSE_FROM My_project)
target_link_libraries(My_project_test ${CommonTestLib})
gtest_add_tests(TARGET My_project_test EXTRA_ARGS --arg1 "${CMAKE_CURRENT_SOURCE_DIR}/data")
The very last line is important. Instead of gtest_add_tests, you can also use add_test. It needs different parameters but that works too when your goal is showing test cases in VS2019's Test Explorer.
The reason the above solution helped:
When you add enable_testing() to your top-level CMakeLists.txt file, it will generate a top-level CTestTestfile.cmake file in your build directory. This is needed by the Test Explorer to roll up all the test cases generated during the build process. If you've got a certain CMake hierarchy within your code structure you should have a similar one for CTest.
My top-level CTestTestfile.cmake file content:
# CMake generated Testfile for
# Source directory: C:/Projects/myproject
# Build directory: C:/Projects/myproject/out/build/x86-Debug
#
# This file includes the relevant testing commands required for
# testing this directory and lists subdirectories to be tested as well.
subdirs("test")
The lower level CTestTestfile.cmake file content:
# CMake generated Testfile for
# Source directory: C:/Projects/MyProject/test
# Build directory: C:/Projects/MyProject/out/build/x86-Debug/test
#
# This file includes the relevant testing commands required for
# testing this directory and lists subdirectories to be tested as well.
add_test(Environment.TestCommandLineArgument "C:/Projects/MyProject/out/build/x86-Debug/test/MyProject_test.exe" "--gtest_filter=Environment.TestCommandLineArgument" "--arg1" "C:/Projects/MyProject/test/data/")
set_tests_properties(Environment.TestCommandLineArgument PROPERTIES _BACKTRACE_TRIPLES "C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.19/Modules/GoogleTest.cmake;380;add_test;C:/Projects/MyProject/test/CMakeLists.txt;38;gtest_add_tests;C:/Projects/MyProject/test/CMakeLists.txt;0;")
add_test(MyProjectExampleCreatorDevice.TestCreateExampleImage "C:/Projects/MyProject/out/build/x86-Debug/test/MyProject_test.exe" "--gtest_filter=MyProjectExampleCreatorDevice.TestCreateExampleImage" "--arg1" "C:/Projects/MyProject/test/data/")
set_tests_properties(MyProjectExampleCreatorDevice.TestCreateExampleImage PROPERTIES _BACKTRACE_TRIPLES "C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.19/Modules/GoogleTest.cmake;380;add_test;C:/Projects/MyProject/test/CMakeLists.txt;38;gtest_add_tests;C:/Projects/MyProject/test/CMakeLists.txt;0;")
...
for me upgrading from 1.8.1.3 to 1.8.1.4 using nuget broke my project.
It switched
Microsoft.googletest.v140.windesktop.msvcstl.static.rt-dyn.1.8.1.3
to
Microsoft.googletest.v140.windesktop.msvcstl.dyn.rt-dyn.1.8.1.3\build\native\Microsoft.googletest.v140.windesktop.msvcstl.dyn.rt-dyn.targets"
revert back to the previous worked for me. (test explorer is now back) I think just making sure you use the static version as opposed to the dynamic should work as well.
There are multiple extensions that seem like they should be used.
If you follow the advice of https://learn.microsoft.com/en-us/visualstudio/test/how-to-use-google-test-for-cpp?view=vs-2022
you use the TestAdapterForGoogleTest from microsoft ? ()
There is another from Google. (Google test adapter). I have a project that used old versions of gtest. I installed this extension and things stopped working.
uninstall it and restart. rebuild and things seemed to work again.

google test with visual studio 2019 and cmake

I am trying to setup google test with visual studio 2019 and cmake.
This is my CMakeFileLists.txt content:
cmake_minimum_required(VERSION 3.0)
project(test_me)
# GTest
enable_testing()
find_package(GTest REQUIRED)
include_directories(${GTEST_INCLUDE_DIRS})
# Unit Tests
# Add test cpp file
add_executable( runUnitTests tests.cpp)
# Link test executable against gtest & gtest_main
target_link_libraries(runUnitTests ${GTEST_BOTH_LIBRARIES})
add_test( runUnitTests runUnitTests )
My tests.cpp file looks like this:
#include <gtest/gtest.h>
TEST(ABC, TEST1) {
EXPECT_EQ(true, true);
}
TEST(ABC, TEST2) {
ASSERT_TRUE(2 == 2);
}
this minimal example is from another stackoverflow question:
CMake file for integrated Visual Studio unit testing
Thats what I get after I have build the application:
a single Test with the name runUnitTests. However in the picture of the answer from the question above I would expect to see the name of each test function. Something like this:
runUnitTests
- ABC
- TEST1
- TEST2
I have tested it with a new visual studio solution and added a google unit test project. Pasting the test functions into this projects results into this picture:
So this works fine. It must have something to do with the open a local folder method which I am using to handle my cmake project.
Here is a possible answer to your question:
cmake_minimum_required(VERSION 3.10)
project(test_me)
enable_testing()
find_package(GTest REQUIRED)
include(GoogleTest)
add_executable(runUnitTests tests.cpp)
target_link_libraries(runUnitTests GTest::GTest GTest::Main)
gtest_discover_tests(runUnitTests)
The command that discovers the full list of tests in your test binary is gtest_discover_tests(), and it is part of the GoogleTest CMake module (you can see the docs locally with cmake --help-module GoogleTest). This module has been introduced with CMake 3.9, but the command gtest_discover_tests() was only added in CMake 3.10. You should note, however, that if your test binary has many test cases, this will slow things down significantly.

CMake header only dependency

I am having some trouble running a simple test with catch2 using CMake. Since catch is header only I got the impression that it is easily dropped into any project so i thought I'd just include it like a normal header file.
Project structure:
-build
-external
-Catch2
catch2.hpp
CMakeLists.txt
tester.cpp
CMakeLists.txt:
cmake_minimum_required(VERSION 3.12)
project(Test VERSION 1.0.0)
include_directories(external)
enable_testing()
add_executable(tester tester.cpp)
add_test(Tester tester)
tester.cpp:
#define CATCH_CONFIG_MAIN
#include "Catch2\catch.hpp"
TEST_CASE( "1 is 1" ) {
REQUIRE( 1 == 1 );
}
Output:
0% tests passed, 1 tests failed out of 1
Total Test time (real) = 0.03 sec
The following tests FAILED:
1 - Tester (Exit code 0xc0000139
)
Errors while running CTest
Obviously the test should pass, but it does not. Since I am a beginner both in the realm of CMake and catch2 I am having a hard time to pinpoint the issue. What I can say for certain is that catch.hpp is found and there is no linker errors, it just return some error code.
I looked around and found this:
CMake Header from library not found when implementing in catch test case
But it has no answers, and the author does not seem to have the same problem anyway.
This is how I build and run the tests (standing in the build directory):
cmake .. -G "MinGW Makefiles" && mingw32-make && ctest
Any help is appretiated :)
Alright, i removed catch and just played around with mingw and apparently i get the same error just by using std::string. Someone said that it has to do with missing DLL files. I ran dependency walker on the executable and indeed a bunch of DLLs was missing. I didn't now what to do or where to get these so i ditched mingw and tried the cygwin approach.
But using cmake with cygwin i did not find any compatable generators for my development environment (windows).
I then switched to generating a visual studio project instead (which i was avoiding from the start because i did not want to develop in an IDE). But i found out that i can use msbuild to build the executable from the generated visual studio project and it works like a charm, with catch.

CMake: How to specify directory where ctest should look for executables?

I wanted to integrate ctest to a c++/c project. I use google tests to write unit tests.
Relevant part of my CMakeLists.txt looks like this:
...
####### CREATING EXE #######
add_executable(test_exe main.cpp test.cpp)
target_link_libraries(test_exe GTest::GTest GTest::Main)
set_target_properties (test_exe PROPERTIES RUNTIME_OUTPUT_DIRECTORY ${UNIT_TEST_BIN_OUTPUT_DIR})
add_test(test_exe test_exe)
As you can see i specified the output directory of my executable (UNIT_TEST_BIN_OUTPUT_DIR).
The executable works fine on its own when I use the terminal:
cd <UNIT_TEST_BIN_OUTPUT_DIR>
./test_exe
I want to use ctest to execute my tests. So I go to the "ctest folder" generated by cmake. Here I want to use ctest to execute all test added by "add_test" in cmake.
user#user:~/<dir to cmake>/cmake/unit_tests$ ctest
Test project /<dir to cmake>/cmake/unit_tests
Start 1: test_exe
Could not find executable test_exe
Looked in the following places:
test_exe
test_exe
Release/test_exe
Release/test_exe
Debug/test_exe
Debug/test_exe
MinSizeRel/test_exe
MinSizeRel/test_exe
RelWithDebInfo/test_exe
RelWithDebInfo/test_exe
Deployment/test_exe
Deployment/test_exe
Development/test_exe
Development/test_exe
Unable to find executable: test_exe
1/1 Test #1: test_exe ......***Not Run 0.00 sec
0% tests passed, 1 tests failed out of 1
Total Test time (real) = 0.00 sec
The following tests FAILED:
1 - test_exe (Not Run)
Errors while running CTest
If I put the "test_exe" in one of the shown paths it works fine. But I don't want them to be there.
My Question:
Is there a way to tell ctest it should look in UNIT_TEST_BIN_OUTPUT_DIR in order to find the executable?
Documentation for add_test specifies WORKING_DIRECTORY option for long form of the command. Value of this option is used as a directory in which test operates:
add_test(NAME test_exe COMMAND test_exe WORKING_DIRECTORY ${UNIT_TEST_BIN_OUTPUT_DIR})
If you just want the test to find the executable, it is sufficient to use
add_test(NAME test_exe COMMAND test_exe)
This is a long form of add_test command. In this form, CMake checks whether COMMAND is a target name, and, if it is so, replaces it with an absolute path to the executable corresponded to that target. Such way the test can be run from any directory.
Note, that automatic replacement of the target doesn't work for a short form of add_test which you use.
Using CMake 3.20 and greater, you can tell CTest which directory contains your tests by using a CLI option:
ctest --test-dir /path/to/your/tests
This is a less-invasive solution for existing tests, for which you don't want to modify the CMake files.
In our projects we always specify the path when we call add_test(), e.g.:
add_test( ${filename} ${CMAKE_CURRENT_BINARY_DIR}/${filename} )

How to execute all gtests for a cmake project with qtcreator?

(As a follow up to this question)
My cmake file looks like this:
include(CTest)
add_subdirectory(/usr/src/gtest gtest)
include_directories(${GTEST_INCLUDE_DIR})
add_executable(TestA TestA.cpp)
target_link_libraries(TestA gtest)
add_test(NAME AllTestsInA COMMAND TestA)
add_executable(TestB TestB.cpp)
target_link_libraries(TestB gtest)
add_test(NAME AllTestsInB COMMAND TestB)
I am able to run single tests from within qtcreator and all tests from command line using ctest --output-on-failure, but how can I execute all tests from within qtcreator?
qtcreator version: 2.7.0
You should have a target called just test. If you build this, it executes all the tests.
For non-makefile IDEs (like Visual Studio) the equivalent target is called RUN_TESTS.
Using QtCreator 2.4.1, try adding a custom build step:
Projects -> "Build Steps" -> [Add Build Step].
[x] Enable this step and use "ctest" as the command.