I noticed that the cmake generation step in my project was taking a long time and so I ran cmake through strace to figure out the root cause. I found that cmake was trying to find internal shared libraries in the wrong locations repeatedly, which was causing a lot of unnecessary file system lookups. Here is a simple project that illustrates the problem:
$ ll -R
.:
total 20K
drwxr-xr-x 2 mark mark 4.0K May 25 16:22 alpha
drwxr-xr-x 2 mark mark 4.0K May 25 16:22 beta
drwxr-xr-x 2 mark mark 4.0K May 25 16:42 build
-rw-r--r-- 1 mark mark 185 May 25 16:20 CMakeLists.txt
-rw-r--r-- 1 mark mark 0 May 25 16:16 dummy.cc
drwxr-xr-x 2 mark mark 4.0K May 25 16:22 gamma
./alpha:
total 4.0K
-rw-r--r-- 1 mark mark 0 May 25 16:16 alpha.cc
-rw-r--r-- 1 mark mark 69 May 25 16:20 CMakeLists.txt
./beta:
total 4.0K
-rw-r--r-- 1 mark mark 0 May 25 16:16 beta.cc
-rw-r--r-- 1 mark mark 67 May 25 16:18 CMakeLists.txt
./build:
total 0
./gamma:
total 4.0K
-rw-r--r-- 1 mark mark 35 May 25 16:19 CMakeLists.txt
-rw-r--r-- 1 mark mark 0 May 25 16:16 gamma.cc
alpha.cc, beta.cc, gamma.cc and dummy.cc are all empty cc files. Here are the contents of all the CMakLists.txt files:
Top level CMakeLists.txt
$ cat CMakeLists.txt
cmake_minimum_required(VERSION 2.8)
add_subdirectory(alpha)
add_subdirectory(beta)
add_subdirectory(gamma)
add_executable(dummy_exec dummy.cc)
target_link_libraries(dummy_exec alpha)
alpha/CMakeLists.txt
$ cat alpha/CMakeLists.txt
add_library(alpha SHARED alpha.cc)
target_link_libraries(alpha beta)
beta/CMakeLists.txt
$ cat beta/CMakeLists.txt
add_library(beta SHARED beta.cc)
target_link_libraries(beta gamma)
gamma/CMakeLists.txt
$ cat gamma/CMakeLists.txt
add_library(gamma SHARED gamma.cc)
I invoke cmake (through strace) as follows:
$ cd build/
$ strace -f -o /tmp/s.out cmake ..
-- The C compiler identification is GNU 4.8.4
-- The CXX compiler identification is GNU 4.8.4
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Configuring done
-- Generating done
-- Build files have been written to: /home/mark/Downloads/cmake/build
When I inspect the strace output, I see it trying to access libalpha in invalid locations (such as under the beta/ and gamma/ sub directories):
31430 access("/home/mark/Downloads/cmake/build/beta/libalpha.so", R_OK) = -1 ENOENT (No such file or directory)
31430 access("/home/mark/Downloads/cmake/build/gamma/libalpha.so", R_OK) = -1 ENOENT (No such file or directory)
Similarly, it is trying to access libbeta and libgamma in invalid locations:
31430 access("/home/mark/Downloads/cmake/build/alpha/libbeta.so", R_OK) = -1 ENOENT (No such file or directory)
31430 access("/home/mark/Downloads/cmake/build/gamma/libbeta.so", R_OK) = -1 ENOENT (No such file or directory)
31430 access("/home/mark/Downloads/cmake/build/alpha/libgamma.so", R_OK) = -1 ENOENT (No such file or directory)
31430 access("/home/mark/Downloads/cmake/build/beta/libgamma.so", R_OK) = -1 ENOENT (No such file or directory)
For a large project with many shared libraries and many dependencies, these invalid lookups add up and seem to be causing big delays at the makefile generation step. Any idea on why this is happening and how I could prevent cmake from searching in these invalid paths ?
Related
According to this answer, gcc and g++ have some functional differences. As confirmed by this script, the commands point to the exact same binary, effectively making them duplicates. Why is that so?
$ uname
Darwin
$ md5 `which cc c++ gcc g++ clang clang++`
fac4668657765c8dfe89d8995acfb5a2 /usr/bin/cc
fac4668657765c8dfe89d8995acfb5a2 /usr/bin/c++
fac4668657765c8dfe89d8995acfb5a2 /usr/bin/gcc
fac4668657765c8dfe89d8995acfb5a2 /usr/bin/g++
fac4668657765c8dfe89d8995acfb5a2 /usr/bin/clang
fac4668657765c8dfe89d8995acfb5a2 /usr/bin/clang++
The executable can determine the name it was called with by inspecting the first (or zeroth) command line argument passed to it. By convention it is the name of the executable and is passed by whatever program is invoking the compiler (typically e.g. a shell).
Although it is the same executable, it can then take different actions based on whether or not that value is gcc or g++.
Also, the files you are seeing are unlikely to be duplicate files. They are most likely just (soft or hard) links to the same file.
For the part that clang/clang++ and gcc/g++ seem to be the same, although they are completely different compilers, that is an Apple quirk. They link gcc and g++ to clang and clang++ for some reason, but in reality both refer to Apple clang, which is also different from upstream clang. It often causes confusion (at least for me).
the commands point to the exact same binary, effectively making them duplicates
Yes, they're intended to be the same, and are actually the same single file on disk, given the inode number of them is the same
$ which cc c++ gcc g++ clang clang++ | xargs ls -li
1152921500312779808 -rwxr-xr-x 76 root wheel 167120 May 10 04:30 /usr/bin/c++
1152921500312779808 -rwxr-xr-x 76 root wheel 167120 May 10 04:30 /usr/bin/cc
1152921500312779808 -rwxr-xr-x 76 root wheel 167120 May 10 04:30 /usr/bin/clang
1152921500312779808 -rwxr-xr-x 76 root wheel 167120 May 10 04:30 /usr/bin/clang++
1152921500312779808 -rwxr-xr-x 76 root wheel 167120 May 10 04:30 /usr/bin/g++
1152921500312779808 -rwxr-xr-x 76 root wheel 167120 May 10 04:30 /usr/bin/gcc
On a typical *nix system symlinks are usually used instead of hard links like that
$ ls -al /usr/bin | grep 'vim'
lrwxr-xr-x 1 root wheel 3 May 10 04:30 ex -> vim
lrwxr-xr-x 1 root wheel 3 May 10 04:30 rview -> vim
lrwxr-xr-x 1 root wheel 3 May 10 04:30 rvim -> vim
lrwxr-xr-x 1 root wheel 3 May 10 04:30 vi -> vim
lrwxr-xr-x 1 root wheel 3 May 10 04:30 view -> vim
-rwxr-xr-x 1 root wheel 5056496 May 10 04:30 vim
lrwxr-xr-x 1 root wheel 3 May 10 04:30 vimdiff -> vim
-rwxr-xr-x 1 root wheel 2154 May 10 04:30 vimtutor
That said, in any case the command can be determined easily, regardless of the full executable file, a hard link or a symlink, by checking the command executed which is argv[0] in a typical C or C++ program, or $0 in bash. This is extremely common and one notable usage of it is in BusyBox where almost all POSIX utilities are in a single busybox binary and anything you run like ls, mv, test, rm... will eventually run busybox
But why is gcc and clang the same binary? It's another weird thing in macOS because Apple stopped using/including gcc for more than a decade ago due to licensing issues. They lie to others by telling "clang is gcc" and the only way you can know it is by running gcc --version
OS X: installed gcc links to clang
What is the difference between c++ and g++ on Mac OS X (CommandLineTools)?
How do I make sure that my default C/C++ compiler is GCC
OS X 10.9 gcc links to clang
I started working with c++ a few weeks ago, and I started a new project to start learning more. I'm encountering an issue with an external dependency. I'm trying to use a library called:
libbgp, and I installed it base on their documentation.
Here is my code:
https://gist.github.com/amb1s1/9b2c72294da0ec9416810c8686d3adce
Error:
/usr/bin/ld: /tmp/ccsdO32O.o: in function `ticker(libbgp::BgpFsm&)':
ambgp.cpp:(.text+0xa7): undefined reference to `libbgp::BgpFsm::tick()'
collect2: error: ld returned 1 exit status
I'm not sure if there is anything else that I have to do after installing the lib for the library to be accessible in my source code.
Update
I ran it with the -lbgp flag and when running it, i get the following error:
g++ -lbgp ambgp.cpp -o ambgp
Error:
./ambgp: error while loading shared libraries: libbgp.so.0: cannot open shared object file: No such file or directory
My Lib:
ls -l /usr/local/lib/
-rw-r--r-- 1 root root 10875880 Jan 18 16:56 libbgp.a
-rwxr-xr-x 1 root root 924 Jan 18 16:56 libbgp.la
lrwxrwxrwx 1 root root 15 Jan 18 16:56 libbgp.so -> libbgp.so.0.0.0
lrwxrwxrwx 1 root root 15 Jan 18 16:56 libbgp.so.0 -> libbgp.so.0.0.0
-rwxr-xr-x 1 root root 4291128 Jan 18 16:56 libbgp.so.0.0.0
drwxrwsr-x 3 root staff 4096 Dec 16 19:27 python3.7
echo $LD_LIBRARY_PATH
:/usr/local/lib
Before running the executable, you have to tell it where to find the libgdp.so file, if it is not stored in a standard place such as /usr/lib or /lib. Following should help you:
$ export LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:/usr/local/lib"
$ ./ambgp
If you do not want to export the LD_LIBRARY_PATH each time you start the shell manually, add the line to your /home/<user>/.bashrc file.
Additionally, i think the -lbgp flag should go after the source file in the compiler command (g++ ambgp.cpp -lbgp -o ambgp)
TL;DR :
ld states it cannot find libbgp.so.0, and you wrote in the comments that you found libbgp.so without a trailing .0. So, creating a symlink to the library could help too:
$ sudo ln -s /usr/local/lib/libbgp.so /usr/local/lib/libbgp.so.0
For linking, you need a library file without a trailing .0, but for loading, the library name must have a trailing .0.
The last thing to try is to directly specify the library location to the linker with -Wl,-rpath=/usr/local/lib (as you worked out already !)
I'm new to cmake, and I'm only using it to install opencv on my ubuntu linux.
Here's the command I ran: cmake -DCMAKE_BUILD_TYPE=Release DCMAKE_INSTALL_PREFIX=/home/jinha/OCV/source
Then it returns the error:
FATAL: In-source builds are not allowed. You should create separate directory for build files.
My current directory, ~/OCV/build/opencv, does contain the CMakefiles.txt file, so that's not the problem. I tried to change the directory in my command, but they all raise the same error. I saw the other answers on this issue, so I erased CMakeFiles directory and CMakeCache.txt file every time before I ran the command, but none of them worked.
Thanks.
It wants you to create a separate build directory (anywhere), and run cmake there. For example:
mkdir my_build_dir
cd my_build_dir
rm ../CMakeCache.txt
cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/home/jinha/OCV/source
Note the .. in this example telling cmake where to look for the source.
In case you didn't remove CMakeCache.txt before building again, it will still show this error.
So, please remember to delete CMakeCache.txt first before running cmake.
After you have success downloaded and unzipped OpenCV sources from sources you need create simple command-file install.sh. For example, your working dir will be /home/user/myopencv
So /home/user/myopencv/install.sh will be contain next code:
#!/bin/bash
rm CMakeCache.txt
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=RELEASE -DCMAKE_INSTALL_PREFIX=/usr/local
make
make install
make clean
Next
chmod 777 install.sh
./install.sh
And after the all you will get those executable files:
root#cartman:/usr/local/bin# ls -las | grep opencv
32 -rwxr-xr-x 1 root root 29888 апр 20 18:10 opencv_annotation
244 -rwxr-xr-x 1 root root 247608 апр 20 18:10 opencv_createsamples
244 -rwxr-xr-x 1 root root 247504 апр 20 18:10 opencv_haartraining
20 -rwxr-xr-x 1 root root 18600 апр 20 18:10 opencv_performance
288 -rwxr-xr-x 1 root root 294592 апр 20 18:10 opencv_traincascade
16 -rwxr-xr-x 1 root root 14288 апр 20 18:10 opencv_version
60 -rwxr-xr-x 1 root root 61040 апр 20 18:10 opencv_visualisation
Enjoy!
I have a project that I compile with the intel toolchain. All the .f90 files are compiled to corresponding .o files in the bin directory. When I use the codecov utility I get an HTML with the list of covered files. However that list is missing a bunch of files which should be eligible for coverage. Some of the modules contained in those files are even used during the runs.
What could be a reason for these files to be missing from the coverage report?
EDIT
Running the binary with a debugger I noticed something which might be a clue.
If I try to set a breakpoint in the files that are not covered gdb throws an error:
No line xxx in file "disturb_coms.f90".
However disturb_coms.f90 is in the bin folder:
manfredo#cave05:build$ ll bin-dbg-A/disturb*
-rw-r--r-- 1 manfredo staff 15K Jun 29 10:41 bin-dbg-A/disturb_coms.f90
-rw-r--r-- 1 manfredo staff 3.9K Jun 29 10:41 bin-dbg-A/disturb_coms.mod
-rw-r--r-- 1 manfredo staff 4.9K Jun 29 10:41 bin-dbg-A/disturb_coms.o
-rw-r--r-- 1 manfredo staff 221K Jun 29 10:43 bin-dbg-A/disturbance.f90
-rw-r--r-- 1 manfredo staff 4.1M Jun 29 10:43 bin-dbg-A/disturbance.o
-rw-r--r-- 1 manfredo staff 3.2M Jun 29 10:43 bin-dbg-A/disturbance_utils.mod
and was compiled with:
ifort -c -FR -CB -O0 -check -g -prof-gen=srcpos -debug full -debug-parameters all -fpe0 -traceback -ftrapuv -fp-stack-check -implicitnone -assume byterecl -warn unused -warn uncalled -warn usage -gen-interfaces -I../../src/include -I/usr/local/hdf5_mio/include disturb_coms.f90
Furthermore a file called disturbance.f90 contains the following lines:
module disturbance_utils
contains
subroutine apply_disturbances(cgrid)
...
use disturb_coms , only : treefall_disturbance_rate ! ! intent(in)
...
cpoly%disturbance_rates(2,2,isi) = treefall_disturbance_rate
...
end subroutine apply_disturbances
end module disturbance_utils
where disturb_coms is used, and the file disturbance is actually covered.
EDIT
By adding a write instruction inside disturb_coms.f90 and trying to compile I get the following error:
disturb_coms.f90(44): error #6274: This statement must not appear in the specification part of a module.
write(*,*) "try"
^
I don't have much experience with Fortran but it looks to me as if that module is a kind of c-equivalent header file. Would there still be a possibility to cover that file as well? Like just checking that the definitions contained there are used elsewhere?
I am trying to compile the latest Boost c++ libraries for Centos. I 've used bjam install and it has placed the libraries in /usr/lib and /usr/lib64.
The problem is I need the -mt variants for a specific application to run. I cannot understand in the documentation how to create the multithreading variants. :(
Please give me a hint!
Thanks!
-mt is just distribution specific extension.
either edit your config file or create symbolic link to libboost_thread
andrey#localhost:~$ ls -l /usr/lib/libboost_thread*
-rw-r--r-- 1 root root 174308 2010-01-25 10:36 /usr/lib/libboost_thread.a
lrwxrwxrwx 1 root root 41 2009-11-04 10:10 /usr/lib/libboost_thread-gcc41-mt-1_34_1.so.1.34.1 -> libboost_thread-gcc42-mt-1_34_1.so.1.34.1
-rw-r--r-- 1 root root 49912 2008-11-01 02:55 /usr/lib/libboost_thread-gcc42-mt-1_34_1.so.1.34.1
lrwxrwxrwx 1 root root 17 2010-01-27 18:32 /usr/lib/libboost_thread-mt.a -> libboost_thread.a
lrwxrwxrwx 1 root root 25 2010-01-27 18:32 /usr/lib/libboost_thread-mt.so -> libboost_thread.so.1.40.0
lrwxrwxrwx 1 root root 25 2010-01-27 18:32 /usr/lib/libboost_thread.so -> libboost_thread.so.1.40.0
-rw-r--r-- 1 root root 89392 2010-01-25 10:36 /usr/lib/libboost_thread.so.1.40.0
You can build all variations of the boost binary libraries using the --build-type=complete option. For example:
bjam --build-type=complete stage
This will put all library files into <your boost dir>/stage/lib/