METIS: undefined reference to `METIS_WPartGraphRecursive' - c++

I have to compile a software that depends on METIS library, but whose CMake thing was written by disabling all the places where METIS was needed. Now, I have to re-enable that code again and thus the code depends now on METIS.
I installed metis-5.1.0 from source, and I wrote a CMake module to find it (actually I used this one). I modify the CMakeLists.txt accordingly, basically adding the following lines
find_package(METIS REQUIRED)
if (METIS_FOUND)
include_directories(SYSTEM ${METIS_INCLUDE_PATH})
else (METIS_FOUND)
message (SEND_ERROR "This application cannot compile without METIS")
endif (METIS_FOUND)
and, at the end,
target_link_libraries(<my_executable> ${METIS_LIBRARIES})
After cmake, it seems everything is fine, because cmake prints:
-- Found METIS: /usr/local/include
-- METIS libraries /usr/local/lib/libmetis.a
...
-- Configuring done
-- Generating done
-- Build files have been written to: <mylocation>
However, after I run make, I get
undefined reference to `METIS_WPartGraphKway'
undefined reference to `METIS_WPartGraphRecursive'
How can I solve?
EDIT:
As an additional information, when I compile with make VERBOSE=1, the linker seems to look for the right library, as it includes /usr/local/lib/libmetis.a, which corresponds to the location that cmake was specifying and which also exists. Moreover, when I look into the library with nm /usr/local/lib/libmetis.a, I see:
00000000000001c0 T METIS_WPartGraphKway
00000000000009c0 T METIS_WPartGraphRecursive
P.S.: In Ubuntu 17.04, I have tried with metis-5.1.0, metis-4.0.3 and also installing with sudo apt-get install libmetis-dev. In the latter case I find libmetis.so instead of libmetis.a, but, also in this case, this is correctly recognized by cmake, correctly looked for by the linker, but at the end I get the same error.
I also tried sudo apt-get install libmetis-dev, installing from source metis-5.1.0 and metis-3.0.6 on Ubuntu 14.04 and I had the same problem.
This seems a hard-to-solve error, as many people ask the same question in this page of the METIS formum.
I have also tried to add -lmetis at the end of the compilation command, as suggested here
As an additional information, I did a grep WPartGraphKway in the METIS source folder. In version 5.1.0 there is no such string. In version 4.0.3 there is.

I solved doing this:
I install metis-3.0
I run cmake for my software
I run make VERBOSE=1
I copy the last gcc command being printed
I paste it, I add at the end of the command -lmetis and execute the command
Note that, if I repeat the same operation with metis-5.1.0, it does not work and I have the same error that I wrote in the first post.

Related

CMake include_directories isn't using one directory, but using all others around it

I'm trying to use conda to manage several dependencies for a c++ library I'm building (I was previously using git submodules in my project's external/ dir which worked fine, but for a number of reasons I wanted to migrate to conda).
The libraries I need to use are tbb, zstd, and cspice. I've verified they are all installed correctly by conda, and are in my environment (named vira_env). The headers are very clearly in ~/mamba_envs/vira_env/include/.
However when I add include_directories(${CMAKE_INSTALL_PREFIX}/include) to my root CMakeLists.txt and run cmake using -DCMAKE_INSTALL_PREFIX=${CONDA_PREFIX}, it fails to find any of the headers.
Running make VERBOSE=1 reveals that g++ has no -I argument to include the ~/mamba_envs/vira_env/include/ directory. Which makes sense, as it is failing to find the headers that are there.
However, if I use include_directories(${CMAKE_INSTALL_PREFIX}) and rerun make VERBOSE=1, or it clearly shows that g++ has a -I /home/cgnam/mamba_envs/vira_env/.
Similarly, if I use include_directories(${CMAKE_INSTALL_PREFIX}/include/oneapi) it shows g++ has a -I /home/cgnam/mamba_envs/vira_env/include/oneapi. (oneapi is the directory where the tbb headers are placed).
So include_directories() will happily use either the conda environment root, or even one of the directories inside the conda environment's include/ directory... but it will just completely ignore the include/ directory itself.
EDIT
As discovered in one of my comments below, printing CMAKE_CXX_IMPLICIT_INCLUDE_DIRECTORIES shows /home/cgnam/mamba_envs/vira_env/include. Which explains why my include_directories() is not having any effect on the g++ call by make, since it believes that the compiler should be looking in that directory already.
make VERBOSE=1is showing that the compiler being used is: /home/cgnam/mamba_envs/vira_env/bin/x86_64-conda-linux-gnu-c++. However using which g++ is showing ~/mamba_envs/vira_env/bin/g++. Further, which ld returns: ~/mamba_envs/vira_env/bin/ld.
If I use ld --verbose | grep SEARCH_DIR, it does not show /home/cgnam/mamba_envs/vira_env/include. Neither does ~/mamba_envs/vira_env/bin/x86_64-conda-linux-gnu-ld --verbose | grep SEARCH_DIR.

Hand built clang cannot find implicitly linked static library in Xcode default toolchain

As part of a research project I'm trying to use clang 6.0.1 with Xcode 9.4.1. I've built and installed clang in a custom location (/opt/llvm-6_0_1/clang). I wrote a simple xcplugin compiler specification to integrate my clang version with Xcode.
Now I can open projects in Xcode, select my proxy compiler and use it to build instead of Apple's default clang.
There were some minor additions that I had to make to the xcplugin's xcspec file to get this to work that probably won't be interesting to most people, so I won't provide the details here unless asked.
This all works with most of the projects I've played with, but I'm running into an odd problem where an implicitly linked static library cannot be found by my copy of clang. Specifically I get this error:
ld: file not found: /opt/llvm-6_0_1/clang/Toolchains/LLVM6.0.1.xctoolchain/usr/lib/arc/libarclite_macosx.a
clang-6.0: error: linker command failed with exit code 1 (use -v to see invocation)
Note that the libarclite_macosx.a file is not explicitly included by the Xcode project. I figured it must be implicitly included, perhaps because this project enables ARC?
After pouring over the Xcode generated link command line (it's complex) I decided to look at the MyProject__dependency_info.dat file, which is passed in via the -dependency_info option. Apparently this data file (the path is defined as env var LD_DEPENDENCY_INFO_FILE) is created during the linking process, not as an input to the linker. Perhaps it exists because of a hack workaround using symlinks that I used to get a link to work (described at the end).
In any case the format appears to be binary, but I was able to see a text reference to libarclite_macosx.a in the file:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/arc/libarclite_macosx.a
After enabling the -Xlinker -v option I could see that my built clang was not searching the default toolchain lib or arc paths so I added them:
-L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib
-L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/arc
Now I can see the search paths in the verbose output, but clang still cannot find the library:
Library search paths:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/arc
I've tried adding the paths to the frameworks search paths. I also tried defining the various link path env vars. Nothing has worked.
To try to get a sense of what clang is actually doing, I used fs_usage while getting the link error:
sudo fs_usage -e -w -f filesys | grep "lib/arc"
14:11:00.461965 stat64 [ 2] /opt/llvm-6_0_1/clang/Toolchains/LLVM6.0.1.xctoolchain/usr/lib/arc>>>>>>>>>>>>>>>>>>>>>> 0.000006 ld.1421614
14:11:00.461968 stat64 [ 2] /opt/llvm-6_0_1/clang/Toolchains/LLVM6.0.1.xctoolchain/usr/lib/arc>>>>>>>>>>>>>>>>>>>>>> 0.000002 ld.1421614
Clearly clang really wants to look for this file in the installed location, not the location indicated in the -dependency_info, nor in the search paths that I'm providing.
At this stage the only way I can get a build to work is to add a symlink to Xcode's "arc" directory to my installed clang lib directory. That "works", but is fragile and nasty.
Any thoughts as to how how I can get clang find the static library where it actually lives?

cmake : failing to link against google profiler (gperftools)

I am on ubuntu 16.04.
Here what I did:
installed gperftools :
sudo apt-get install google-perftools libgoogle-perftools-dev
downloaded FindGperftools.cmake from: https://github.com/vast-io/vast/blob/master/cmake/FindGperftools.cmake
renamed FindGperftools.cmake to GperftoolsConfig.cmake and placed it in a cmake folder in my package
added to CMakelists.txt:
set (Gperftools_DIR "${CMAKE_CURRENT_LIST_DIR}/cmake/")
find_package(Gperftools REQUIRED)
in same CMakelists.txt, link my executable:
target_link_libraries(my_executable ${GPERFTOOLS_PROFILER})
in a terminal, export the CPUPROFILE environment variable:
export CPUPROFILE=/my_path/prof.out
in the same terminal, run the executable:
./my_executable
There is no error message, but the log file /my_path/prof.out is not created.
If I run "ldd" on "my_executable", it does not show any linkage against profiler (ldd ./my_executable | grep profil does not result in anything).
Yet, when looking at files in the build folder, the compiler seems to do the linkage (-lprofile is there).
Anything I may have forgotten?
Note: not sure it is relevant, but I use catkin.
This looks like ubuntu's (and non-standard) linker feature to not link libraries which symbols are directly not used. Try adding -Wl,-no-as-needed to your LDFLAGS (and make sure it is passed before -lprofiler).

/usr/bin/ld: cannot find -lboost_system-mt

I recently upgraded from boost 1.40 to 1.45 by removing the previous boost directory completely, downloading the 1.45 sources and rebuilding the libraries I wanted. I then installed the libs using bjam install.
Nothing else has changed on my machine, yet, now when I am building my C++ program, I get the following link error:
/usr/bin/ld: cannot find -lboost_system-mt
I searched and the file really does not exist. It seems that the mt libraries are no longer part of the library - or am I missing something?
How may I resolve this?
Well, I solved this error on ubuntu 12.04 (x86_64) by the good old scattergun approach
Installing openvrml with the error "cannot find -lboost_filesystem-mt" after make.
libboost-all-dev. installs 54 different packages. One of 'em must've done the trick, runs fine.
This version probably doesn't bring multi-threading enabled by default.
Try passing -lboost_system instead of -lboost_system-mt
Edit:
Also it's good to check if the new libs are really inside /usr/local/lib. You should look for /usr/local/lib/libboost_system.so since you did not requested the libs to be built with multi-threading. If the file is there, then your $PATH (environment variable) could be missing /usr/local/lib, and you should update the compilation command so the compiler knows where to find them:
-L/usr/local/lib -lboost_system-mt
I had a strange encounter with this, too. My solution was a bid odd - but since it worked for me and I didn't read about it anywhere else, here it is.
In my case lboost_python3 was missing.
Hence, i loaded all 54 packages like #Kilgore Trout suggested:
sudo apt-get install libboost-all-dev
Unfortunately, when i looked into the /usr/lib - folder only certain packages were available there. However, when I searched /usr/lib I got more results - the missing files were all in the /usr/lib/arm-linux-gnueabihf - folder.
I simply copy-pasted all libboost-related files into the /usr/lib-folder et voilĂ  - the next time I tried to build anything with the lboost_python3.so , everything worked.
It seems some paths got mixed up or something like this.
Hope this helps you or somebody else.
I have saucy:
$ dpkg -S /usr/lib/libboost_system-mt.so
libboost-system1.49-dev: /usr/lib/libboost_system-mt.so
thus, you can do:
sudo apt-get install libboost-system1.49-dev
Are your sure the /usr/lib/libboost_system-mt.so sym-link point to the right file:
$ realpath /usr/lib/libboost_system-mt.so
Otherwise you have to install the project or use yours distribution package management. For Debian/Ubuntu it would be apt-get install libboost-system1.45-dev -- but this package does not exists while writing this.
Fixed this thanks to #KilgoreTrout's and #user3191035, so here's my notes: I'm on Ubuntu Natty 11.04; my usual state was:
$ dpkg -S libboost_filesystem
libboost-filesystem1.42.0: /usr/lib/libboost_filesystem.so.1.42.0
Then I installed:
sudo apt-get install libboost-all-dev # ton of packages
... and after that, I get this:
$ dpkg -S libboost_filesystem
libboost-filesystem1.42-dev: /usr/lib/libboost_filesystem-mt.a
libboost-filesystem1.42.0: /usr/lib/libboost_filesystem.so.1.42.0
libboost-filesystem1.42-dev: /usr/lib/libboost_filesystem.so
libboost-filesystem1.42-dev: /usr/lib/libboost_filesystem-mt.so
libboost-filesystem1.42-dev: /usr/lib/libboost_filesystem.a
So, that is where the libboost_filesystem-mt.so is in this OS...
change libboost_thread-mt to libboost_thread, first find the address of libboost_thread.so and libboost_thread.a then make softlinks to these files in the same address, so it should be
ln -s /...libboostSourceFiles.../libboost_thread.so /..RequestTOmtFiles.../libboost_thread-mt.so
it works for other libboost -mt files too, like serialization , iostreams, programoptions

How to Use CCache with CMake?

I would like to do the following: If CCache is present in PATH, use "ccache g++" for compilation, else use g++. I tried writing a small my-cmake script containing
CC="ccache gcc" CXX="ccache g++" cmake $*
but it does not seem to work (running make still does not use ccache; I checked this using CMAKE_VERBOSE_MAKEFILE on).
Update:
As per this link I tried changing my script to
cmake -D CMAKE_CXX_COMPILER="ccache" -D CMAKE_CXX_COMPILER_ARG1="g++" -D CMAKE_C_COMPILER="ccache" -D CMAKE_C_COMPILER_ARG1="gcc" $*
but cmake bails out complaining that a test failed on using the compiler ccache (which can be expected).
As of CMAKE 3.4 you can do:
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache
It is now possible to specify ccache as a launcher for compile commands and link commands (since cmake 2.8.0). That works for Makefile and Ninja generator. To do this, just set the following properties :
find_program(CCACHE_FOUND ccache)
if(CCACHE_FOUND)
set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE ccache)
set_property(GLOBAL PROPERTY RULE_LAUNCH_LINK ccache) # Less useful to do it for linking, see edit2
endif(CCACHE_FOUND)
It is also possible to set these properties only for specific directories or targets.
For Ninja, this is possible since version 3.4.
For XCode, Craig Scott gives a workaround in his answer.
Edit : Thanks to uprego and Lekensteyn's comment, I edited the answer to check if ccache is available before using it as launcher and for which generators is it possible to use a compile launcher.
Edit2: #Emilio Cobos recommended to avoid doing that for the linking part as ccache doesn't improve linking speed and can mess with other types of cache like sccache
I personally have /usr/lib/ccache in my $PATH. This directory contains loads of symlinks for every possible name the compiler could be called from (like gcc and gcc-4.3), all pointing to ccache.
And I didn't even create the symlinks. That directory comes pre-filled when I install ccache on Debian.
From CMake 3.1, it is possible to use ccache with the Xcode generator and Ninja is supported from CMake 3.4 onwards. Ninja will honour RULE_LAUNCH_COMPILE just like the Unix Makefiles generator (so #Babcool's answer gets you there for Ninja too), but getting ccache working for the Xcode generator takes a little more work. The following article explains the method in detail, focussing on a general implementation which works for all three CMake generators and making no assumptions about setting up ccache symlinks or the underlying compiler used (it still lets CMake decide the compiler):
https://crascit.com/2016/04/09/using-ccache-with-cmake/
The general gist of the article is as follows. The start of your CMakeLists.txt file should be set up something like this:
cmake_minimum_required(VERSION 2.8)
find_program(CCACHE_PROGRAM ccache)
if(CCACHE_PROGRAM)
# Support Unix Makefiles and Ninja
set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE "${CCACHE_PROGRAM}")
endif()
project(SomeProject)
get_property(RULE_LAUNCH_COMPILE GLOBAL PROPERTY RULE_LAUNCH_COMPILE)
if(RULE_LAUNCH_COMPILE AND CMAKE_GENERATOR STREQUAL "Xcode")
# Set up wrapper scripts
configure_file(launch-c.in launch-c)
configure_file(launch-cxx.in launch-cxx)
execute_process(COMMAND chmod a+rx
"${CMAKE_BINARY_DIR}/launch-c"
"${CMAKE_BINARY_DIR}/launch-cxx")
# Set Xcode project attributes to route compilation through our scripts
set(CMAKE_XCODE_ATTRIBUTE_CC "${CMAKE_BINARY_DIR}/launch-c")
set(CMAKE_XCODE_ATTRIBUTE_CXX "${CMAKE_BINARY_DIR}/launch-cxx")
set(CMAKE_XCODE_ATTRIBUTE_LD "${CMAKE_BINARY_DIR}/launch-c")
set(CMAKE_XCODE_ATTRIBUTE_LDPLUSPLUS "${CMAKE_BINARY_DIR}/launch-cxx")
endif()
The two script template files launch-c.in and launch-cxx.in look like this (they should be in the same directory as the CMakeLists.txt file):
launch-c.in:
#!/bin/sh
export CCACHE_CPP2=true
exec "${RULE_LAUNCH_COMPILE}" "${CMAKE_C_COMPILER}" "$#"
launch-cxx.in:
#!/bin/sh
export CCACHE_CPP2=true
exec "${RULE_LAUNCH_COMPILE}" "${CMAKE_CXX_COMPILER}" "$#"
The above uses RULE_LAUNCH_COMPILE alone for Unix Makefiles and Ninja, but for the Xcode generator it relies on help from CMake's CMAKE_XCODE_ATTRIBUTE_... variables support. The setting of the CC and CXX user-defined Xcode attributes to control the compiler command and LD and LDPLUSPLUS for the linker command is not, as far as I can tell, a documented feature of Xcode projects, but it does seem to work. If anyone can confirm it is officially supported by Apple, I'll update the linked article and this answer accordingly.
I didn't like to set a symlink from g++ to ccache. And CXX="ccache g++" didn't work for me as some cmake test case wanted to have just the compiler program without attributes.
So I used a small bash script instead:
#!/bin/bash
ccache g++ "$#"
and saved it as an executable in /usr/bin/ccache-g++.
Then C configured cmake to use /usr/bin/ccache-g++ as C++ compiler.
This way it passes the cmake test cases and I feel more comfortable than having symlinks that I might forget about in 2 or 3 weeks and then maybe wonder if something doesn't work...
I verified the following works (source: this link):
CC="gcc" CXX="g++" cmake -D CMAKE_CXX_COMPILER="ccache" -D CMAKE_CXX_COMPILER_ARG1="g++" -D CMAKE_C_COMPILER="ccache" -D CMAKE_C_COMPILER_ARG1="gcc" $*
Update: I later realized that even this does not work. Strangely it works every alternate time (the other times cmake complains).
Let me add one important item that was not mentioned here before.
While bootstrapping a minimalistic build system from the ubuntu:18.04 docker image, I've found that order of installation makes a difference.
In my case ccache worked fine when calling gcc, but failed to catch invocations of the same compiler by the other names: cc and c++.
To fully install ccache, you need to make sure all compilers are installed first, or add a call to update-ccache symlinks to be safe.
sudo /usr/sbin/update-ccache-symlinks
export PATH="/usr/lib/ccache/:$PATH"```
... and then (due to updated symlinks) also calls to cc and c++ get caught!
In my opinion the best way is to symlink gcc,g++ to ccache, but if you would like to use within cmake, try this:
export CC="ccache gcc" CXX="ccache g++" cmake ...
Here are 2 methods I think are clean/robust, and also don't pollute your CMake code.
1.) Set environment variables
This method is nice since you don't have to individually set it up for each CMake project. The con is you may not want ccache for each CMake project.
# Requires CMake 3.17 (https://cmake.org/cmake/help/latest/envvar/CMAKE_LANG_COMPILER_LAUNCHER.html)
export CMAKE_CXX_COMPILER_LAUNCHER=/usr/bin/ccache
export CMAKE_C_COMPILER_LAUNCHER=/usr/bin/ccache
2.) Pass in cache variables during project configuration
Con a bit annoying to do for each project. This can be negated by your IDE though.
# Requires CMake 3.4
$ cmake ... -D CMAKE_CXX_COMPILER_LAUNCHER=/usr/bin/ccache \
-D CMAKE_C_COMPILER_LAUNCHER=/usr/bin/ccache
NOTE: It isn't really necessary to specify the full path.
If ccache is in your path you can just specify ccache instead.
export CMAKE_CXX_COMPILER_LAUNCHER=ccache
export CMAKE_C_COMPILER_LAUNCHER=ccache
It is extending #Nicolas answer.
Add following line to your cmake file:
list(PREPEND CMAKE_PROGRAM_PATH /usr/lib/ccache)
Or add it as argument to cmake configuration step:
cmake -DCMAKE_PROGRAM_PATH=/usr/lib/ccache