Cannot link to existing libopenblas.so.0 DLL - c++

I need to use the libpca in C++, which depends on Armadillo, which in turn requires OpenBlas and lapack.
I got some problems in installing/configuring these dependencies.
I am using Red Hat 4.8.5-16 and wish to write code and call the package in /my/dir/program/. I do NOT have sudo privilege.
All those packages are downloaded and unzipped in /my/dir/
Here is the steps I have followed:
1. to install OpenBlas
/my/dir/OpenBLAS-0.2.20$ make
/my/dir/OpenBLAS-0.2.20$ make PREFIX=/my/dir/program/
<Step 1 is successful>
2. to install lapack
/my/dir/lapack-3.8.0$ mv make.inc.example make.inc
/my/dir/lapack-3.8.0$ make
<Step 2 is successful>
3. to install Armadillo
/my/dir/armadillo-8.500.0$ cmake . -DCMAKE_INSTALL_PREFIX:PATH=/my/dir/pgrogram/
/my/dir/armadillo-8.500.0$ make
/my/dir/armadillo-8.500.0$ make install
<Step 3 is successful>
4. to compile the example code
/my/dir/armadillo-8.500.0$ cp examples/example1.cpp /my/dir/program
/my/dir/program$ export LD_LIBRARY_PATH=/opt/intel/mkl/lib/intel64/:../OpenBLAS-0.2.20/:../armadillo-8.500.0/:./lib/:./lib64/:$LD_LIBRARY_PATH ;
/my/dir/program$ g++ example1.cpp -o example1 -O2 -I./include -L./lib -L./lib64 -L../lapack-3.8.0 -L../OpenBLAS-0.2.20 -larmadillo -lopenblas -llapack ;
<There is no warning or error in compiling>
5. to run the example code
/my/dir/program$ export LD_LIBRARY_PATH=/opt/intel/mkl/lib/intel64/:../OpenBLAS-0.2.20/:../armadillo-8.500.0/:./lib/:./lib64/:$LD_LIBRARY_PATH ;
/my/dir/program$ ./example1
It returns
./example1: error while loading shared libraries: libopenblas.so.0: cannot
open shared object file: No such file or directory
Actually the libopenblas.so.0 dll is located in ./lib/, and I've already added the DLL path.
Since I do not have root privilege, I cannot yum install openblas.x86_64 openblas-devel.x86_64 as suggested by this post.
What should I do?

For g++ / ld to find libmkl_rt.so is one thing.
Running ./example1 is a new situation : The "location information" must be repeated →
export LD_LIBRARY_PATH=/opt/intel.. .. ...:$LD_LIBRARY_PATH && ./example1
Please be aware that the export command is temporary : Valid for the current shell session only. When you exit the shell, the command is cancelled.

Related

How to install c++ program into conda environment from source

I would like to compile and then use the scientific project BASS, which is distributed as c++ source code on github. I've set up a conda environment bass to hold everything related to BASS, and I'd like to compile BASS into this environment (so that if I delete the environment, it's cleanly removed).
I don't know if I should be using conda-build or make to do this. There is a Makefile distributed with the project, but I think it might have an error.
My latest try is the following: (the source files are in code/ and gsl seems to be a dependency):
conda create -n bass
conda activate bass
conda install make
conda install gsl
cd code/
make
I get the following:
gcc -c main.cpp -I../Libs/GSL/include/ -I./
main.cpp:19:10: fatal error: 'gsl/gsl_statistics.h' file not found
#include <gsl/gsl_statistics.h>
^~~~~~~~~~~~~~~~~~~~~~
1 error generated.
make: *** [Makefile:11: main.o] Error 1
My questions:
should I be doing this with make?
do I need gcc/g++/cxx-compiler installed in my conda env?
is the Makefile correct where it says "-I../Libs/GSL/include/"
Thank you!
Here's a very basic Conda recipe for the package. Make a folder (say recipe) and put the following two files in it:
meta.yaml
package:
name: bass
version: 0.1 # this is totally arbitrary (there are no versions)
source:
git_url: https://github.com/judyboon/BASS.git
requirements:
build:
- {{ compiler('cxx') }}
host:
- gsl
run:
- gsl
about:
home: https://github.com/judyboon/BASS
license: GPL-3.0-only
license_file: COPYING
license_family: GPL
build.sh
#!/bin/bash
## change to source dir
cd ${SRC_DIR}/code
## compile
${CXX} -c main.cpp *.cpp -I./ ${CXXFLAGS}
## link
${CXX} *.o -o BASS -lgsl -lgslcblas -lm ${LDFLAGS}
## install
mkdir -p $PREFIX/bin
cp BASS ${PREFIX}/bin/BASS
You can then build this with conda build ., run from in that directory.
Installing
After successful build, you can create an environment with this package installed using
conda create -n foo --use-local bass
Since I'm on an Intel architecture, and this tool uses CBLAS, I would further specify to use MKL:
conda create -n foo --use-local bass blas=*=*mkl
Now if you activate the environment foo (that's an arbitrary name), the software will be on PATH, under the binary BASS.
Additional Notes
I verified this works on osx-64 platform with the conda-forge channel prioritized.
The Makefile accompanying the code isn't very generic. Here we just have it compile all .cpp files and subsequently link all .o files.
There aren't any releases on the repository, so the versioning in the meta.yaml is arbitrary.
The build.sh defines the final output binary as BASS - that could be changed, and differs from the original Makefile which outputs the binary as main.
Based on what you write, the correct way is to build a conda recipe. By writing a recipe the conda knows what to delete, when you delete the environment.
The recipe is a text file (the meta.yaml file), where you write some metadata describing the package, define dependencies and write a short build script that executes the make file and installs the binaries to correct location. Defining the compilation environment may not be trivial and you should follow the documentation for the package you are trying to install. If there are problems with the package code (that includes the makefile) that's something conda can't help you with.
See the documentation on how to write the recipe:
https://docs.conda.io/projects/conda-build/en/latest/concepts/recipe.html
When you've got the recipe complete, then you would run the conda build and conda install -c [path to the build] [the package name] (I'm writing this because it took me ages to realize how to correctly install a local package.)

Tensorflow Op: how to include libtensorflow_framework.so?

I followed the instructions of this tutorial:
https://www.tensorflow.org/extend/adding_an_op#implement_the_gradient_in_python.
There is this comment provided: g++ -std=c++11 -shared zero_out.cc -o zero_out.so -fPIC -I$TF_INC -I$TF_INC/external/nsync/public -L$TF_LIB -ltensorflow_framework -O2
But the linker cannot find -ltensorflow_framework (it should be a tensorflow_frameowork.so file!?)
After some research, I found following links:
https://github.com/tensorflow/tensorflow/issues/1569
https://github.com/eaplatanios/tensorflow_scala/issues/26 --> I downloaded the .jar and linked it via -l/pathto/tensorflow_framework.so, still the fatal error: tensorflow/core/framework/op_kernel.h: No such file or directory is not found.
https://github.com/tensorflow/tensorflow/issues/1270 last comment does not work and so does not help me.
I tried to search for sudo find /usr/. -name "tensorflow_framework.so" recursively but I could not find anything. Tensorflow is installed for sure via anaconda and I also cloned and compiled the repository from source.
How to find a way to include the -ltensorflow_framework?
One answer, I have found:
I have installed my python via anaconda2 and I always tried to find out TF_INC and TF_LIB when I activated my repository source activate <env>. and the could not found any ~/anaconda2/envs/tensorflow/lib/python2.7/site-packages/tensorflow
*.so files
This time I went out every python environment with the shell command source deactivate and I typed the following command
python -c 'import tensorflow as tf; print(tf.sysconfig.get_lib())'
Now, I got a different path: ~/anaconda2/lib/python2.7/site-packages/tensorflow, where the lib libtensorflow_framework.so is located.
In my case, the file libtensorflow_framework.so.1 existed inside my TF_LIB directory instead of libtensorflow_framework.so. In order to solve this issue, I had to create a symbolic link as follows:
ln -s libtensorflow_framework.so.1 libtensorflow_framework.so
Source: Tensorflow NotFoundError: libtensorflow_framework.so: cannot open shared file or directory
tensorflow_framework is not used before Tensorflow 1.4.1
When you call python from the shell make sure you are calling the right one:
TF_LIB = $(shell python -c 'import tensorflow; print(tensorflow.sysconfig.get_lib())')
or
TF_LIB = $(shell python3 -c 'import tensorflow; print(tensorflow.sysconfig.get_lib())')
To be more clear:
Get the path from python -c 'import tensorflow as tf; print(tf.sysconfig.get_lib())', and there is a libtensorflow_framework.so.1 inside the directory. Say /home/.../lib/python3.7/site-packages/tensorflow_core/libtensorflow_framework.so.1
Run ln -s /home/.../lib/python3.7/site-packages/tensorflow_core/libtensorflow_framework.so.1 /home/.../lib/python3.7/site-packages/tensorflow_core/libtensorflow_framework.so

Can't use static lib of mongo-cxx-driver on Linux

So I follow the official tutorial for the installation : https://mongodb.github.io/mongo-cxx-driver/mongocxx-v3/installation/
Neverless, I can't use the produced libraries as static.
So I managed to compile the C version of the driver as described, I've enabled the flag --enable-static=yes with the ./configure before doing make && sudo make install and I got the libmongoc-1.0.a and the libbson-1.0.a which are static. So this far, everything it's alright.
Then I have done the cxx version of the driver, except that there is no configuration file as in the C version. So I've juste done a
cmake -DCMAKE_BUILD_TYPE=Release -DBSONCXX_POLY_USE_BOOST=1 -DCMAKE_INSTALL_PREFIX=/usr/local
from the build folder, followed by a make && sudo make install
So I got the libmongocxx.a and the libbsoncxx.a, but when I try to compile with them, I can't run the binary because I got the following error :
error while loading shared libraries: libmongocxx.so._noabi: cannot open shared object file: No such file or directory
So I understand that is because there is some symbols missing and then I need to use the shared library to run the binary but I don't want this to happend, I want the symbols within the binary that I can run it without any LD_PRELOAD.
Any suggestions ?
I had the same issue in an Ubuntu 16.04 and I run a apt-get update & apt-get upgrade and the problem was solved.
It seems that there were some update to the compiler and some libraries that prevent some test from reaching the shared libraries.
I have a similar question, and solved, now I compiled and run my binary with static libs successfully.
I write my build script using newlisp, but the static link options are very helpful, I paste it here.
c++ /to/your/path/site/code/back_end/builder/object/files1.cc.o ... /to/your/path/site/code/back_end/builder/object/files10.cc.o -o bin/site -static-libgcc -static-libstdc++ -L/usr/lib -lpthread -l:libmongocxx.a -l:libbsoncxx.a -l:libmongoc-1.0.a -l:libbson-1.0.a -lrt -lssl -lcrypto -lsasl2 -l:libboost_log.a -l:libboost_log_setup.a -l:libboost_system.a -l:libboost_thread.a -l:libboost_filesystem.a -lcppcms -lbooster -lcurl -ljsoncpp

NTL header file not found

I have downloaded and installed the NTL library on my Ubuntu. I'm currently using gedit to write my program and having included this ZZ.h header in my program. This is how i compile my program in the terminal: - g++ keygen.cpp -o keygen -I ../include -L ../lib -lntl -lm.
I'm pretty sure this line is correct but for some unknown reason, i get the following error:
KeyGen.cpp:9:20: error: NTL/ZZ.h: No such file or directory
KeyGen.cpp:15: error: expected constructor, destructor, or type conversion before ‘int’
The solution seems pretty straightforward to me: which is to add the NTL library directly to my program folder. I did just that, but still i get the same error.
If you don't need the latest (6.0.0) version of NTL you may do as follows in your Ubuntu:
user#host:~$ sudo apt-get install libntl-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
libntl-5.4.2
The following NEW packages will be installed:
libntl-5.4.2 libntl-dev
0 upgraded, 2 newly installed, 0 to remove and 112 not upgraded.
Need to get 2,035 kB of archives.
After this operation, 7,016 kB of additional disk space will be used.
Do you want to continue [Y/n]? y
Get:1 http://ftp.de.debian.org/debian/ squeeze/main libntl-5.4.2 amd64 5.4.2-4 [767 kB]
Get:2 http://ftp.de.debian.org/debian/ squeeze/main libntl-dev amd64 5.4.2-4 [1,268 kB]
Fetched 2,035 kB in 2s (1,017 kB/s)
Selecting previously deselected package libntl-5.4.2.
(Reading database ... 59184 files and directories currently installed.)
Unpacking libntl-5.4.2 (from .../libntl-5.4.2_5.4.2-4_amd64.deb) ...
Selecting previously deselected package libntl-dev.
Unpacking libntl-dev (from .../libntl-dev_5.4.2-4_amd64.deb) ...
Can not write log, openpty() failed (/dev/pts not mounted?)
Setting up libntl-5.4.2 (5.4.2-4) ...
Setting up libntl-dev (5.4.2-4) ..
user#host:~$
after that the complete compiled NTL library with all development headers is installed in your system and you may compile your program with it without any additional -I<path>.
If you need a newer version that your distro has (check http://packages.ubuntu.com/en/source/trusty/ntl) you may try to build the library package yourself.
The problem with your attempt to compile and output an executable file appears to be compiler's inability to link the necessary library after obtaining an object .o file.
Many people often test the point of fault by separating the two stages by first compiling g++ -c then by linking the libraries for an executable g++ -o. Although -Wall switch does not always work, trying it to provide you with as much information as possible during compilation can also be helpful.
Check this webpage. As for using different switches to link libraries try this webpage.
I'm not certain if it was a typo; but I wonder if the space between the switch and directory:-I ../include and -L ../libwas the problem.
You said in comments:
Icreated a folder called 'include' within the .cpp folder and included the NTL library in that folder already
But your compilation command says:
g++ keygen.cpp -o keygen -I ../include -L ../lib -lntl -lm.
It seems to me, you meant:
g++ keygen.cpp -o keygen -I ./include -L ../lib -lntl -lm.
# ^^^^^^^^^
since .. goes up one directory.

Boost librairies not found but compilation is ok

I am trying to use filesystem from boost in c++
It seems the compilation is ok when using
"c++ -c Analyse.c -o Analyse.o -g -W -Wall -L/usr/local/lib -lboost_filesystem -lboost_system"
However I have the following error when trying to execute my code :
"error while loading shared libraries: libboost_filesystem.so.1.54.0: cannot open shared object file: No such file or directory", a find / -iname "libboost_system.so.1.54.0
I had some issues to install boost (I first installed the 1.49 and after that 1.54) so I was wondering if there could be any conflict between the 2 version ?
P.S : btw a "find / -iname "libboost_system.so.1.54.0" gave me the following
/usr/include/boost/boost_1_54_0/bin.v2/libs/system/build/gcc-4.7/release/threading-multi/libboost_system.so.1.54.0
/usr/local/lib/libboost_system.so.1.54.0
Try to add the directory before execution. For example:
LD_LIBRARY_PATH="/usr/local/lib/" ./Analyse.o
I encountered this issue very recently, after a fresh installation of boost. In my case, the solution was to simply run
sudo ldconfig
The explanation is that the system keeps a cache of the installed shared libraries (located in /usr/lib, /lib, /usr/local/lib). When the libraries are changed, or new ones are added, the cache is not updated until ldconfig is run. More details can be found in the ldconfig manual.