I'm working on c++ project on a linux machine and it uses several boost libraries. I've installed them on my system using vcpkg and build it using the toolchain provided by vcpkg. My question is:
How do I define the dependencies so that they automatically install on a different system, if they were to build it?
Conan has a way of doing it by defining the dependencies in conanfile.txt. How do I do the same with vcpkg?
Edit1: I've found autovcpkg which does the job I'm looking to do but can the same be done natively inside cmakelists.txt or by vcpkg itself?
If you have vcpkg as a submodule for your project, define a manifest for the libraries you want vcpkg to build, and are using the vcpkg CMake toolchain - then you will get everything you want.
Adding vcpkg as a submodule means that your users don't need to install it themselves, the CMake toolchain will install it on your behalf. It also means that you can fix the package versions
Using a manifest file is how you programmatically tell vcpkg which packages to get and build during a CMake configuration phase
Using a CMake toolchain file is the only way to tie this into your project's build system
$ git clone .../my_project
$ cd ./my_project
$ git submodule update --init
$ mkdir ../build
$ cd ../build
$ cmake ../my_project
-- Running vcpkg install
-- Running vcpkg install - done
...
I've found autovcpkg which does the job I'm looking to do but can the same be done natively inside cmakelists.txt or by vcpkg itself?
You can write a vcpkg port for your library or executable by providing a CONTROL and portfile.cmake file. In the CONTROL file you define all the dependencies and possible features while the portfile contains the build instruction. You can use vcpkg create <myport> <url> <filename> to create the CONTROL and portfile.cmake from a template which can be customized to your needs.
Together with a port-overlay this port can also be used by others without being merged into vcpkg/master
Related
I'm pulling in Boost using CPM to build another dependency of my project.
CPMAddPackage(
NAME Boost
VERSION 1.77.0
GITHUB_REPOSITORY "boostorg/boost"
GIT_TAG "boost-1.77.0"
)
The issue I'm having is exposing the header files to my dependency. I dumped all of the cmake variables, and there are many variables like boost_accumulators_SOURCE_DIR, boost_algorithm_SOURCE_DIR, boost_any_SOURCE_DIR, boost_asio_SOURCE_DIR, etc. My dependency depends on many of these libraries, and it's really tedious to list them all as include directories:
target_include_directories(
nghttp2_asio
PRIVATE "${boost_system_SOURCE_DIR}/include"
PRIVATE "${boost_config_SOURCE_DIR}/include"
PRIVATE "${boost_asio_SOURCE_DIR}/include"
PRIVATE "${boost_throw_exception_SOURCE_DIR}/include"
PRIVATE "${boost_assert_SOURCE_DIR}/include"
PRIVATE "${BoostAlign_SOURCE_DIR}/include"
PRIVATE "${boost_date_time_SOURCE_DIR}/include"
)
Is there a better way to do this?
CPM is just a thin layer around FetchContent, which in turn downloads your dependency into your build folder and then attempts to add_subdirectory it, adding it to your main build.
I think this is a bad idea for a lot of reasons...
Boost is one of the most commonly packaged C++ libraries, period. Integrating your project into an existing environment (like a Linux distro, or another package manager like Conan or Vcpkg) is going to be difficult if not impossible (without patching, I mean) since it will surely want your project to use the curated build of Boost.
Adding any third party CMake code to your own build is signing up for headaches... have you noticed how hard it is to write correct CMake code? What if Boost clobbers your cache variables or defines targets that conflict with yours?
CMake has built-in and standard support for locating many versions of Boost, including the newer versions that provide first-party CMake config modules.
Boost now provides first-party CMake config modules.
I don't think any build using Boost should be any more complex than this:
cmake_minimum_required(VERSION 3.22)
project(boost-usage-example)
find_package(Boost 1.77 REQUIRED system date_time)
add_executable(nghttp2_asio main.cpp ...)
target_link_libraries(
nghttp2_asio
PRIVATE
Boost::boost # all header-only libs: config asio throw_exception assert align
Boost::date_time
Boost::system
)
See the documentation here: https://cmake.org/cmake/help/latest/module/FindBoost.html
You only list as components (after REQUIRED) the non-header-only libraries. Those are also the ones that require special addition via target_link_libraries. Link to Boost::boost to get all the header-only modules.
This will work no matter what package manager you're using.
If you want to use vcpkg, create a file called vcpkg.json in your project root with the following contents:
{
"name": "boost-usage-example",
"version-string": "0.1.0-dev",
"dependencies": [
"boost-system",
"boost-config",
"boost-asio",
"boost-throw-exception",
"boost-assert",
"boost-align",
"boost-date-time"
]
}
You can also depend on just boost and it will acquire all boost modules, not just the ones you need. Then acquire vcpkg:
$ git clone https://github.com/microsoft/vcpkg.git
$ ./vcpkg/bootstrap-vcpkg.sh
Then build with:
$ cmake -S . -B build -DCMAKE_TOOLCHAIN_FILE=$PWD/vcpkg/scripts/buildsystems/vcpkg.cmake -DCMAKE_BUILD_TYPE=Release
$ cmake --build build
Done!
Suppose on the other hand that you wanted to use Conan. No big deal, just install Conan in a pip virtual environment:
$ python3 -m venv venv
$ . venv/bin/activate
$ python3 -m pip install -U pip setuptools wheel
$ python3 -m pip install conan
Then create a conanfile.txt with the following contents:
[requires]
boost/1.77.0
[generators]
cmake_paths
Install the dependencies:
$ mkdir build && pushd build && conan install .. && popd
And then build, using Conan's generated toolchain file:
$ cmake -S . -B build -DCMAKE_TOOLCHAIN_FILE=$PWD/build/conan_paths.cmake -DCMAKE_BUILD_TYPE=Release
$ cmake --build build
In fact, you can provide both a vcpkg.json and a conanfile.txt and your users will be free to use either one or neither and rely on their system package manager or a package manager you don't know about. In any case it will just work and you free yourself of a mountain of maintenance burdens.
For a project which uses MQTT, I always had to compile the QtMqtt module from source, because it wasn't included in the prebuilt windows release and couldn't be chosen for installation either. In Qt5 that was pretty easy: Download source code from official git (https://code.qt.io/cgit/qt/qtmqtt.git/), open the .pro file in QtCreator and compile the project. For installation, I just moved the .dll files to my Qt install directory.
Now in Qt6, the build process was switched from qmake to cmake, so I cannot simply load the project in QtCreator, but have to compile it manually using CMake, which I find really unintuitive and prone to errors. From what I understand, I cannot compile single modules on their own from now on, but have to get the whole Qt 6.2.0 source code instead (i.e. via installer option) and then use a --target option on CMake to build only specific module. So here is what I did so far:
Get Qt source code from installer (installed to Qt/6.2.0/Src)
Create a command line environment according to this manual
Open cmd environment, navigate to build folder (i.e. Qt/6.2.0/build)
Configure build using command: ..\Src\configure -prefix Qt\6.2.0\build
Build with cmake using command cmake --build . --target qtmqtt
Install using command cmake --install .
What happens, is that the configuration works, and the build supposedly too, but installation fails with something like:
CMake Error at qtbase/src/3rdparty/libpng/cmake_install.cmake:41 (file):
file INSTALL cannot find
"F:/DEV/prog/Qt/6.2.0/build/qtbase/mkspecs/modules/qt_ext_libpng.pri": File
exists.
Call Stack (most recent call first):
qtbase/src/3rdparty/cmake_install.cmake:42 (include)
qtbase/src/cmake_install.cmake:42 (include)
qtbase/cmake_install.cmake:244 (include)
cmake_install.cmake:42 (include)
The folder Qt/6.2.0/build then only consists of .cmake files but nothing which seems to be usable .dll files for me. I just don't understand how to properly set up everything with cmake. Why would they make it so complicated after all now, as it was fairly easy to compile modules with qmake in Qt5?
Having Qt 6.4.2 installed in ${HOME}/Qt/6.4.2/ using the Qt Installer, this is how I did it on Linux.
Detailed steps:
# Create a work directory
mkdir ~/temporal && cd ~/temporal
# Clone the repository
git clone https://github.com/qt/qtmqtt.git
# Switch to the repository
cd qtmqtt
# Checkout the branch corresponding to the target kit
git checkout 6.4.2
# Create a directory to build the module cleanly
mkdir build && cd build
# Use the qt-configure-module tool
~/Qt/6.4.2/gcc_64/bin/qt-configure-module ..
# Build it here
~/Qt/Tools/CMake/bin/cmake --build .
# Install the module in the correct location
~/Qt/Tools/CMake/bin/cmake --install . --verbose
Testing on a clean project:
Edit the project's CMakeLists.txt file
(1) Add Mqtt to the relevant find_package macro like this:
find_package(QT NAMES Qt6 COMPONENTS Widgets Network Sql Mqtt REQUIRED)
(2) Add Qt${QT_VERSION_MAJOR}::Mqtt to a target_link_libraries line like this:
target_link_libraries(MyCleanProject PRIVATE Qt${QT_VERSION_MAJOR}::Widgets Qt${QT_VERSION_MAJOR}::Network Qt${QT_VERSION_MAJOR}::Mqtt Qt${QT_VERSION_MAJOR}::Sql)
(3) Run CMake
Try actually using the module
Add relevant includes in the project source like this:
#include <QtMqtt/QtMqtt>
// and then try to use QtMqtt classes
QMqttClient client;
Rebuild the project. Should compile and link cleanly.
Addendum
It's desirable to have proper environment variables set while building Qt projects from the CLI. My preference is to have a script in my user directory with all the relevant variables.
My Qt_environment.sh:
export QT_VERSION="6.4.2"
export QT_INSTALL_DIR="${HOME}/Qt"
export CMAKE_BIN_DIR="${QT_INSTALL_DIR}/Tools/CMake/bin"
export QMAKE_BIN_DIR="${QT_INSTALL_DIR}/${QT_VERSION}/gcc_64/bin"
export CMAKE_PREFIX_PATH="${QT_INSTALL_DIR}/${QT_VERSION}/gcc_64/"
export NINJA_DIR="${QT_INSTALL_DIR}/Tools/Ninja"
export PATH="${PATH}:${CMAKE_BIN_DIR}:${QMAKE_BIN_DIR}:${NINJA_DIR}"
One can choose to load manually from the command line before building a project like this
source $HOME/Qt_environment.sh
# do some Qt stuff here, cmake, ninja, etc
or simply have it loaded from .bashrc for every new bash shell. The same can be accomplished for any other shell.
# $HOME/.bashrc
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi
# ...
# rest of the .bashrc file ommited for brevity
# ...
# Relevant part here - add these two lines at the end
# Load Qt environment variables
source "$HOME/Qt_environment.sh"
From what I understand, I cannot compile single modules on their own
from now on, but have to get the whole Qt 6.2.0 source code instead
(i.e. via installer option)
That's not correct. Given a pre-built Qt, you should be still able to just build the qmqtt package from source by running e.g.
<QTDIR>\bin\qt-configure-module <mqtt_src_dir>
cmake --build .
try to replace cmake --install . with cmake --install qtmqtt
I am trying to build a program on MacOS and that program requires Eigen version <= 3.3.7 but mine has 3.3.8 with brew install eigen
I went to Eigen website but 3.3.7 source is no longer available anymore.
Any workaround?
Thanks a lot!
Updates: thanks a lot but I am still very confused with how to proceed. Below is the information of the installation details and directory info.
The directory of the program source code that I want to build look like:
program
cmake
Cmakelist.txt
build
the program has the following instructions:
Create the build directory in the source tree root
mkdir build
Configure cmake, from the build directory, passing the Shogun source root as an argument. It is recommended to use any of CMake GUIs (e.g. replace cmake .. with ccmake ..), in particular, if you feel unsure about possible parameters and configurations. Note that all cmake options read as -DOPTION=VALUE.
cd build
cmake [options] ..
Compile
make
Install (prepend sudo if installing system-wide), and you are done.
make install
Sometimes you might need to clean up your build (e.g. in case of some major changes). First, try
make clean
Then I have downloaded eigen3.3.7, where the INSTALL file is as following. How should I proceed?
Method 1. Installing without using CMake
****************************************
You can use right away the headers in the Eigen/ subdirectory. In order
to install, just copy this Eigen/ subdirectory to your favorite location.
If you also want the unsupported features, copy the unsupported/
subdirectory too.
Method 2. Installing using CMake
********************************
Let's call this directory 'source_dir' (where this INSTALL file is).
Before starting, create another directory which we will call 'build_dir'.
Do:
cd build_dir
cmake source_dir
make install
You can install Eigen 3.3.7 by compiling the source code(available here).
I've been trying to build and install gRPC with cmake. Building the project went relatively smoothly with ninja after some confusion related to acquiring nuget packages and updating the git submodules.
I am having trouble installing gRPC though. After reading the cmake file I discovered that you need to manually set the gRPC_INSTALL cache variable to ON in order for cmake to generate an install target. After doing this I can invoke the install target and the libraries and headers and cmake config files are emplaced. But the cmake targets file is missing, and it isn't even being generated. The config file is simple, all it does is call the targets file:
include(${CMAKE_CURRENT_LIST_DIR}/gRPCTargets.cmake)
But it doesn't look like the CMakeLists file in gRPC is even attempting to generate this file, unless I am missing something? You can't even build the example cpp project in the gRPC repo with cmake because when it tries to find the gRPC package, the config file fails to find gRPCTargets.cmake. So what is the proper way to build, install, and link to gRPC with cmake? I am on Windows but that shouldn't matter with cmake.
on OSX:
run these command:
cd grpc
mkdir -p cmake/build
cd cmake/build
cmake -DgRPC_INSTALL=ON -DgRPC_BUILD_TESTS=OFF -DgRPC_PROTOBUF_PROVIDER=package -DgRPC_ZLIB_PROVIDER=package -DgRPC_CARES_PROVIDER=package -DgRPC_SSL_PROVIDER=package -DCMAKE_BUILD_TYPE=Release -DOPENSSL_ROOT_DIR=/usr/local/Cellar/openssl/1.0.2r -DOPENSSL_LIBRARIES=/usr/local/Cellar/openssl/1.0.2r/lib ../..
make install
I just download poppler to Linux system,and I want to incorporate it in my app to parse pdf file.
(My goal is to convert pdf file to plain text.)
How can I do this?
Poppler's git tree includes a useless INSTALL doc that just tells you to run ./configure, but they don't include automake/autoconf auto-generated files (including configure) in git. (Probably they do include them in tarball source releases.)
I just built poppler from git source (on Ubuntu 15.04) like so:
git clone --depth 50 --no-single-branch git://git.freedesktop.org/git/poppler/poppler
cmake -G 'Unix Makefiles' # other -G options are to generate project files for various IDEs
# look at the output. If it didn't find some libraries,
# install them with your package manager and re-run cmake
make -j4
# optionally:
sudo make install
It appears that they maintain an autoconf/automake build setup, so you can use that OR cmake to create a Makefile.
If you just want to see if the latest git poppler works better than the distro package, you don't need to sudo make install, you can just run utils/pdftotext or whatever right from the source directory. It apparently tells the linker to embed the build path into the binary, as a library search path, so running /usr/local/src/poppler/utils/pdftotext works, and finds /usr/local/src/poppler/libpoppler.so.52.
If the latest poppler does work better than the distro-packaged poppler, you should install it to /usr/local/bin with sudo make install. When you upgrade to the next version of your distro, check your /usr/local. Often the new distro version will be newer than when you built it from source, so you should just remove your version from /usr/local/{bin,share,lib,man,include}. (Or make uninstall in the source dir, if supported).
Their website explains it very clearly :
Poppler is available from git. To clone the repository use the following command:
git clone git://git.freedesktop.org/git/poppler/poppler
Once you download the source code, read the INSTALL file where it says :
cd to the directory containing the package's source code and type
./configure to configure the package for your system.
Type `make' to compile the package.
Type `make install' to install the programs and any data files and
documentation.
Since some time has passed and it seems there was some uncertainty, I also took a look.
At the end of 2021, their homepage says
We run continuous integration via the gitlab CI
I checked out their .gitlab-ci.yml which has many build tasks. It would seem these days we build libpoppler like this:
git clone git://git.freedesktop.org/git/poppler/test test.repo
mkdir -p build && cd build
cmake -DTESTDATADIR=`pwd`/../test.repo -G Ninja ..
ninja