CMake: set path to MPI headers and binaries manually - c++

I am developing an MPI application which requires to be run with a specific implementation of MPI (let's call it MPIvA). On my workstation, another implementation of MPI (let's call it MPIvB) is installed.
My application is built using CMake and the find_library(MPI) obviously points to MPIvB. It compiles and runs without hassle.
I compiled MPIvA on my workstation. How can I make CMake use these headers and binaries?

CMake comes with a FindMPI module, that does all the heavy lifting for you.
In your CMakeLists.txt, instead of calling find_library(MPI), use find_package like so:
#### MPI
find_package(MPI REQUIRED)
if (MPI_FOUND)
include_directories(SYSTEM ${MPI_INCLUDE_PATH})
else (MPI_FOUND)
message(SEND_ERROR "This application cannot compile without MPI")
endif (MPI_FOUND)
Then wherever you link your application, link against the ${MPI_LIBRARIES}:
target_link_libraries(example-app ${MPI_LIBRARIES})
Now cmake will automatically find a MPI implementation in your system. If you have multiple different MPI versions, and want to specify which one to compile with, you can set the MPI_C_COMPILER and MPI_CXX_COMPILER variables to the corresponding mpicc and mpicxx compiler wrappers. The CMake module will then use those to figure out all the required compiler and linker flags itself.
Example:
cmake -DMPI_C_COMPILER=/usr/share/mvapich/bin/mpicc your-project-dir
To make sure cmake is using the correct MPI, start in a new empty build directory.
More information on the FindMPI module here: https://cmake.org/cmake/help/v3.0/module/FindMPI.html

In order to compile against MPIvA on your workstation, you'll need CMake to find those headers and binaries first. The following link describes how CMake's find_library's search order works: https://cmake.org/cmake/help/v3.0/command/find_library.html
I'd suggest adding MPIvA to the CMAKE_LIBRARY_PATH. See the top answer to the following question for an example:
How do I instruct CMake to look for libraries installed by MacPorts?

Well, this is not a new post but it may be useful to others in the future.
Even though the previous answers can also work, I think that the a better approach is to use the CMAKE_PREFIX_PATH option, ie sth like
CMAKE_PREFIX_PATH=~/my-libs-install-path ccmake ..
Now, some other comments:
All MPI libraries should be interchangeable, ie code that uses the MPI specification should compile and run with any MPI implementation (eg MPIvA and MPIvB which I am assuming are intelmpi,openmpi or mpich).
Regarding this answer (my reputation doesn't allow me to comment so I reply here):
find_package(MPI REQUIRED)
if (MPI_FOUND)
include_directories(SYSTEM ${MPI_INCLUDE_PATH})
else (MPI_FOUND)
message(SEND_ERROR "This application cannot compile without MPI")
endif (MPI_FOUND)
even though the functionality would be mostly as expected, the following is functionally equivalent:
find_package(MPI REQUIRED)
include_directories(SYSTEM ${MPI_INCLUDE_PATH})
The REQUIRED part in the findpackage(MPI REQUIRED) command will cause the command to fail if MPI is not found, so the if(MPI_FOUND) part is useless and the else clause will actually never be executed.
Finally, regarding the include_directories command, in my tests was not required if you are using targets, eg
target_link_libraries(my_app PUBLIC MPI::C)
is enough.

the built in cmake will interfere with your desired library more often than not, especially if you are using different clusters as usually default compilers are not the latest version you might need, the work around this is to have your own FindMYMPI.cmake and take the control.
Otherwise command line option or changing it with the ccmake GUI is also a possibility as provided in the above answers.

In this specific case, I succeeded using the following environment variable:
export MPI_HOME=/your/path/to/prefix
Please set it before using cmake, or empty the build directory.

Related

Is it ok for CMake to use c++, rather than mpicxx, to compile my code?

I'm trying to build an executable from C++ source code which uses MPI, on a GNU/Linux Devuan Chimaera system. Now, I'm an MPI/OpenMP newbie, I'm just trying to adapt this code, which isn't mine, to be built with CMake - when before it had a Makefile. My build succeeds, but I'm seeing segfaults, so I want to make sure my problem isn't with the build phase, which bugs me.
My CMakeLists.txt has:
find_package(OpenMP REQUIRED)
find_package(MPI REQUIRED)
and my system has OpenMPI 4.1.1 installed, which is found. I do this for my target:
target_link_libraries(my_executable PRIVATE MPI::MPI_CXX OpenMP::OpenMP_CXX)
but nothing else which indicates its expecting to be compiled by mpicxx.
... and indeed, when I configure (with CMake 3.22.1) and then build, the usual c++ executable gets invoked to compile (and then link) the my_target executable.
Questions:
Can source code which originally was getting compiled with mpicxx be compiled with "just" a C++ compiler, with the appropriate includes?
Assuming there's any merit to using mpicxx for compilation - how do I get CMake to use it for my target?
Edit: It's been suggested to me to try using mpirun to run my program. With it, I get no segmentation faults, consistently; it's only when I run directly that I see them.
Can source code which originally was getting compiled with mpicxx be compiled with "just" a C++ compiler, with the appropriate includes?
Yes, and you're doing it correctly.
The MPI systems I've used (Cori, Perlmutter, Stampede) have all provided implementations that work correctly with CMake's MPI support. However, it's possible that a sufficiently poorly administered system will break this.
Assuming there's any merit to using mpicxx for compilation - how do I get CMake to use it for my target?
This is a toolchain setting... set CMAKE_CXX_COMPILER to /path/to/mpicxx either at the command line, in a preset, or in a toolchain file.

CMake target_include_directories and target_compile_definitions alternative

CMake 2.8.11 introduces target_include_directories and target_compile_definitions. However some of the devices I am working one has 2.8.9. And I don't want to compile and build latest CMake there. So what is the alternative that I can use conditionally for those devices ?
How to check whether this cmake have target_include_directories or not from inside the CMakeLists.txt ?
There is no easy way to get this functionality on earlier CMake versions, as the respective target properties that they are based on (INTERFACE_INCLUDE_DIRECTORIES and INTERFACE_COMPILE_DEFINITIONS) are missing.
This is crucial as CMake uses these target properties to automatically propagate public and interface fields to dependent targets. Emulating this behavior from CMake code is rather difficult. For an in-house project we built a system that does just this a couple of years back before CMake supported it. This basically requires writing your own replacement functions for add_library and add_executable that handle the propagation in the background through user defined properties. All targets in the system then must use the replacement functions instead of the original CMake ones for the system to work. I would strongly recommend reconsidering your CMake upgrade policies before going for this option though.
The quickest way to check whether you can use these functions is by doing a cmake_minimum_required (which you anyway should always do for all projects). If you want to make use of the commands, simply require at least 2.8.11. If you cannot make that requirement, use of the functions is out of the question in the first place and you will need to come up with an alternative. In which case it does not make much sense to check for their existence to begin with.
ComicSansMS is correct about emulation not being possible prior to 2.8.11.
You can use
if (COMMAND target_include_directories)
endif()
or
if (COMMAND target_include_directories AND COMMAND target_compile_definitions)
endif()
as your test of whether the commands are available.
How to check whether this cmake have target_include_directories or not from inside the CMakeLists.txt ?
if (NOT CMAKE_VERSION VERSION_LESS 2.8.12)
target_include_directories(...)
else()
# Not available
endif()

Can the output from CMake's FindBoost feature be silenced?

CMake's FindBoost feature, called using find_package(Boost ... ), creates a lot of output that looks like this:
-- Boost version: 1.57.0
-- Found the following Boost libraries:
-- system
# ... other libraries ...
Can this output be silenced?
Setting set(Boost_DEBUG OFF) seems to have no effect.
(For those curious about why this might be desirable: our build system is very tightly controlled; we do not intend external developers to be able to build our code in arbitrary environments (and in fact we hard-code the paths to our compilers, libraries, etc in our CMake files). So there's not really any value to seeing this output when building, and it can make it easier for developers to miss information that's actually important.)
Use QUIET option for find_package(Boost...) call. It will supress "Found the following Boost libraries:" message.

How should I setup a C++ project on github so that it is portable?

I would like to start a C++ project on GitHub such that I will be able to clone/build on different operating systems (for example, OSX and unix). There will be lots of dependencies that need to be installed (for example curl, libxml2, etc), and much as possible I'd like to avoid having the user need to manually install the deps on the system. Is it possible to do this?
It depends on how much you want to bite off.
The short answer is let CMake do the work. It will generate projects for whatever platform and they should be usable. You don't need to distribute binaries for this, assuming they are readily available to you (and to them, by extension).
Here is an example that sets up sqlite, boost, and eigen that I used for one of my projects.
cmake_minimum_required(VERSION 2.8)
set(CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/Modules)
# boost
find_package(Boost 1.42.0 REQUIRED )
include_directories(${Boost_INCLUDE_DIRS})
link_directories(${Boost_LIBRARY_DIRS})
find_package(Eigen REQUIRED)
include_directories(${EIGEN_INCLUDE_DIRS})
find_package(Sqlite3 REQUIRED)
include_directories(${SQLITE3_INCLUDE_DIR})
set(CMAKE_CXX_FLAGS "-std=c++0x")
include_directories(.)
link_libraries(
${Boost_LIBRARIES}
${SQLITE3_LIBRARIES}
)
add_executable(igd_sqlite
main.cpp
)
You'd take this and generate visual studio projects, makefiles, etc. Then build the project as you normally would.
CMake supports lots of libraries out of the box, though sometimes you do have to google for less popular ones and add them to your project.
I use it for my day-to-day work, even when I don't need cross platform.
When I do actually want to distribute binaries, I usually setup an external folder with the binary files.
Here is an example:
https://github.com/tomisarobot/curl_race
This works great if you don't have a lot of external dependencies and if they aren't huge. When that isn't the case, then I'd recommend putting each plattform in different repositories. Your source should be in its own too. You can use subprojects if you want, though that isn't strictly necessary. External deps dont change that often, so its not always worth the overhead. Usually I just write a bash script to document where to get everything. This script is usually necessary for a distribution build anyway.
Things like the maven-nar-plugin exist, though I am unaware of its maturity. If you're just creating all the binaries to distribute with your source anyway, then maybe it isn't that attractive. I don't see a lot of talk about it, so I assume adoption is low. Having seen what maven does for Java, it should be more popular.
Few thoughts
1) Tools
When possible, include the tool sources in the repository so that one can build them as a first-time-only step
When not possible, clearly specify what minimum version of what tool is required so that the user can install them herself
Possibly check if the dependency requirements are satisfied by running a script
2) Language
Compile with strict language compliance. For example
g++ -std=c++11 -pedantic

Using CMake with multiple compilers for the same language

It seems like CMake is fairly entrenched in its view that there should be one, and only one, CMAKE_CXX_COMPILER for all C++ source files. I can't find a way to override this on a per-target basis. This makes a mix of host-and-cross compiling in a single CMakeLists.txt very difficult with the built-in CMake facilities.
So, my question is: what's the best way to use multiple compilers for the same language (i.e. C++)?
It's impossible to do this with CMake.
CMake only keeps one set of compiler properties which is shared by all targets in a CMakeLists.txt file. If you want to use two compilers, you need to run CMake twice. This is even true for e.g. building 32bit and 64bit binaries from the same compiler toolchain.
The quick-and-dirty way around this is using custom commands. But then you end up with what are basically glorified shell-scripts, which is probably not what you want.
The clean solution is: Don't put them in the same CMakeLists.txt! You can't link between different architectures anyway, so there is no need for them to be in the same file. You may reduce redundancies by refactoring common parts of the CMake scripts into separate files and include() them.
The main disadvantage here is that you lose the ability to build with a single command, but you can solve that by writing a wrapper in your favorite scripting language that takes care of calling the different CMake-makefiles.
You might want to look at ExternalProject:
http://www.kitware.com/media/html/BuildingExternalProjectsWithCMake2.8.html
Not impossible as the top answer suggests. I have the same problem as OP. I have some sources for cross compiling for a raspberry pi pico, and then some unit tests that I am running on my host system.
To make this work, I'm using the very shameful "set" to override the compiler in the CMakeLists.txt for my test folder. Works great.
if(DEFINED ENV{HOST_CXX_COMPILER})
set(CMAKE_CXX_COMPILER $ENV{HOST_CXX_COMPILER})
else()
set(CMAKE_CXX_COMPILER "g++")
endif()
set(CMAKE_CXX_FLAGS "")
The cmake devs/community seems very against using set to change the compiler since for some reason. They assume that you need to use one compiler for the entire project which is an incorrect assumption for embedded systems projects.
My solution above works, and fits the philosophy I think. Users can still change their chosen compiler via environment variables, if it's not set then I do assume g++. set only changes variables for the current scope, so this doesn't affect the rest of the project.
To extend #Bill Hoffman's answer:
Build your project as a super-build, by using some kind of template like the one here https://github.com/Sarcasm/cmake-superbuild
which will configure both the dependencies and your project as an ExternalProject (standalone cmake configure/build/install environment).