I'm new to Linux, so I try to setup a virtual machine on Oracle VirtualBox to work with Ubuntu 14.04 -32bit version. I downloaded BLAS-3.7.0 package from http://www.netlib.org/blas/ and install it by first compiling all the .f file with gfortran:
gfortran -c *.f
then creating a shared library with all the compiled file:
sudo gfortran -shared -fPIC -o libblas.so *.o
When I checked the shared library with
nm -u libblas.so
It returned a list of undefined reference like those:
U cabsf##GLIBC_2.1
U cabs##GLIBC_2.1
w __cxa_finalize
U _gfortran_stop_string##GFORTRAN_1.0
U _gfortran_string_len_trim##GFORTRAN_1.0
U _gfortran_st_write_done##GFORTRAN_1.0
U _gfortran_st_write##GFORTRAN_1.0
U _gfortran_transfer_character_write##GFORTRAN_1.4
U _gfortran_transfer_integer_write##GFORTRAN_1.4
w __gmon_start__
w _ITM_deregisterTMCloneTable
w _ITM_registerTMCloneTable
w _Jv_RegisterClasses
U sqrtf##GLIBC_2.0
U sqrt##GLIBC_2.0
I then tried to add the math library of glibc and the gfortran library like some suggestion with -lm and -lgfortran:
sudo gfortran -L/usr/lib/i386-linux-gnu -L/usr/lib/gcc/i686-linux-gnu/4.8 -shared -fPIC -o libblas.so *.o -lm -lgfortran
but the same error returns.
I would be very grateful if somebody can help me point out the problem of my shared library. I need the BLAS and LAPACK package to install GPAW simulation library (https://wiki.fysik.dtu.dk/gpaw/) but GPAW always sends back the error message due to the undefined references while linking with BLAS and LAPACK shared library.
Edit: *********
For GPAW-1.2.0, I install it using static library of BLAS-3.7.0, lapack-3.7.0 and libxc-3.0.0 by modifying the extra_link_args in the setup.py file:
extra_link_args ['/usr/local/lib/xc/lib/libxc.a', '/usr/local/lib/BLAS-3.7.0/libblas.a', '/usr/local/lib/lapack-3.7.0/liblapack.a']
My configuration.log file after installing GPAW look like this:
libraries []
library_dirs []
include_dirs ['/usr/local/lib/xc/include', '/usr/lib/python2.7/dist-packages/numpy/core/include']
define_macros [('NPY_NO_DEPRECATED_API', 7)]
extra_link_args ['/usr/local/lib/xc/lib/libxc.a', '/usr/local/lib/BLAS- 3.7.0/libblas.a', '/usr/local/lib/lapack-3.7.0/liblapack.a']
extra_compile_args ['-Wall', '-std=c99']
runtime_library_dirs []
extra_objects []
I also add the library path and include path into LIBRARY_PATH and C_INCLUDE_PATH:
echo $C_INCLUDE_PATH
/usr/local/lib/xc/include/
echo $LIBRARY_PATH
/usr/local/lib/BLAS-3.7.0:/usr/local/lib/lapack-3.7.0:/usr/local/lib/xc/lib
I then installed it with:
sudo python setup.py install >&log.txt
when I try to run a test script in gpaw/test folder, or simply just typing "gpaw info" in the terminal window, it returned this:
OpenBLAS : Your OS does not support AVX instructions. OpenBLAS is using Nehalem kernels as a fallback, which may give poorer performance.
Traceback (most recent call last):
File "/usr/local/bin/gpaw", line 2, in <module>
from gpaw.cli.main import main
File "/usr/local/lib/python2.7/dist-packages/gpaw/__init__.py", line 239, in <module>
from gpaw.calculator import GPAW
File "/usr/local/lib/python2.7/dist-packages/gpaw/calculator.py", line 11, in <module>
import gpaw.mpi as mpi
File "/usr/local/lib/python2.7/dist-packages/gpaw/mpi/__init__.py", line 16, in <module>
from gpaw.utilities import is_contiguous
File "/usr/local/lib/python2.7/dist-packages/gpaw/utilities/__init__.py", line 13, in <module>
import _gpaw
ImportError: /usr/local/lib/python2.7/dist-packages/_gpaw.so: undefined symbol: dnrm2_
I guess the openblas warning come from the fact that I installed numpy on my machine through
sudo apt-get install python-numpy
but I did not select openblas in my install configuration.
Related
I followed the instructions of this tutorial:
https://www.tensorflow.org/extend/adding_an_op#implement_the_gradient_in_python.
There is this comment provided: g++ -std=c++11 -shared zero_out.cc -o zero_out.so -fPIC -I$TF_INC -I$TF_INC/external/nsync/public -L$TF_LIB -ltensorflow_framework -O2
But the linker cannot find -ltensorflow_framework (it should be a tensorflow_frameowork.so file!?)
After some research, I found following links:
https://github.com/tensorflow/tensorflow/issues/1569
https://github.com/eaplatanios/tensorflow_scala/issues/26 --> I downloaded the .jar and linked it via -l/pathto/tensorflow_framework.so, still the fatal error: tensorflow/core/framework/op_kernel.h: No such file or directory is not found.
https://github.com/tensorflow/tensorflow/issues/1270 last comment does not work and so does not help me.
I tried to search for sudo find /usr/. -name "tensorflow_framework.so" recursively but I could not find anything. Tensorflow is installed for sure via anaconda and I also cloned and compiled the repository from source.
How to find a way to include the -ltensorflow_framework?
One answer, I have found:
I have installed my python via anaconda2 and I always tried to find out TF_INC and TF_LIB when I activated my repository source activate <env>. and the could not found any ~/anaconda2/envs/tensorflow/lib/python2.7/site-packages/tensorflow
*.so files
This time I went out every python environment with the shell command source deactivate and I typed the following command
python -c 'import tensorflow as tf; print(tf.sysconfig.get_lib())'
Now, I got a different path: ~/anaconda2/lib/python2.7/site-packages/tensorflow, where the lib libtensorflow_framework.so is located.
In my case, the file libtensorflow_framework.so.1 existed inside my TF_LIB directory instead of libtensorflow_framework.so. In order to solve this issue, I had to create a symbolic link as follows:
ln -s libtensorflow_framework.so.1 libtensorflow_framework.so
Source: Tensorflow NotFoundError: libtensorflow_framework.so: cannot open shared file or directory
tensorflow_framework is not used before Tensorflow 1.4.1
When you call python from the shell make sure you are calling the right one:
TF_LIB = $(shell python -c 'import tensorflow; print(tensorflow.sysconfig.get_lib())')
or
TF_LIB = $(shell python3 -c 'import tensorflow; print(tensorflow.sysconfig.get_lib())')
To be more clear:
Get the path from python -c 'import tensorflow as tf; print(tf.sysconfig.get_lib())', and there is a libtensorflow_framework.so.1 inside the directory. Say /home/.../lib/python3.7/site-packages/tensorflow_core/libtensorflow_framework.so.1
Run ln -s /home/.../lib/python3.7/site-packages/tensorflow_core/libtensorflow_framework.so.1 /home/.../lib/python3.7/site-packages/tensorflow_core/libtensorflow_framework.so
I'm trying to create a helloWorld module for Python 3 in C++ using the boost::python library.
Here is a CmakeList.txt:
set(Python_ADDITIONAL_VERSIONS 3.4)
find_package( PythonLibs 3.4 REQUIRED )
include_directories( ${PYTHON_INCLUDE_DIRS} )
find_package( Boost 1.56.0 EXACT COMPONENTS python3 REQUIRED )
include_directories( ${Boost_INCLUDE_DIR} )
# Define the wrapper library that wraps our library
add_library( hello SHARED main.cpp )
target_link_libraries( hello ${Boost_LIBRARIES} ${PythonLibs_LIBRARIES} )
# don't prepend wrapper library name with lib
set_target_properties( hello PROPERTIES PREFIX "" OUTPUT_NAME hello)
main.cpp
#include <boost/python.hpp>
char const* greet( )
{
return "Hello world";
}
BOOST_PYTHON_MODULE(mymodule)
{
using namespace boost::python;
def( "greet", greet );
}
I installed boost libraries from the source described here, but it does not allow me to use the boost-python3 library (have an error in Cmake). For this purpose I used
./bootstrap.sh --with-python-version=3.4 --prefix=/usr/local
instead of
./bootstrap.sh --prefix=/usr/local
to explicitly specify version of python;
As an output, we get a shared library hello.so. All seems to be ok. But...
When I try to import the library to python script script.py with content:
import hello
in terminal using command ...$ python3 script.py
I receive an error
Traceback (most recent call last):
File "script.py", line 1, in <module>
import hello
ImportError: /usr/local/lib/libboost_python3.so.1.56.0: undefined symbol: PyClass_Type
The question is: How to make the boost library compatible with Python 3? There are no problems with python2. But I need python3.
I also saw the page when the same error happens but it didn't help me.
My software:
boost version 1.56.0
Python 3.4
cmake version 2.8.12.2
gcc 4.8.2
OS: Ubuntu 14.04 LTS, 64 bit
As noted in this answer:
PyClass_Type is is part of the Python 2 C API and not part of the Python 3 C API. Hence, the Boost.Python library was likely built against Python 2. However, it is being loaded by a Python 3 interpreter, where the PyClass_Type is not available.
The exact procedure used to produce libboost_python3.so is not presented, so I can only speculate a non clean build, such as building Boost.Python with Python2, then reconfiguring bootstrap with Python3, and then building Boost.Python with the Python2 object files. Regardless, verify a clean build of Boost.Python with Python3.
$ ./bootstrap.sh --with-python=/usr/bin/python2
...
Detecting Python version... 2.7
$ ./b2 --with-python --buildid=2 # produces libboost_python-2.so
$ ./bootstrap.sh --with-python=/usr/bin/python3 --with-python-root=/usr
...
Detecting Python version... 3.3
$ ./b2 --with-python --buildid=3noclean # produces libboost_python-3noclean.so
$ ./b2 --with-python --clean
$ ./b2 --with-python --buildid=3 # produces libboost_python-3.so
$ nm -D stage/lib/libboost_python-2.so | grep PyClass_Type
U PyClass_Type
$ nm -D stage/lib/libboost_python-3noclean.so | grep PyClass_Type
U PyClass_Type
$ nm -D stage/lib/libboost_python-3.so | grep PyClass_Type
As expected, libboost_python-2.so references the PyClass_Type symbol. Additionally, the libboost_python-3noclean.so contains a reference to PyClass_Type as it was built with libboost_python-2.so's object files. With a clean build, libboost_python-3.so should not contain a reference to PyClass_Type.
Having some issues, now I have read the following:
hello world python extension in c++ using boost?
I have tried installing boost onto my desktop, and, done as the posts suggested in terms of linking. I have the following code:
#include <boost/python.hpp>
#include <Python.h>
using namespace boost::python;
Now I have tried linking with the following:
g++ testing.cpp -I /usr/include/python2.7/pyconfig.h -L /usr/include/python2.7/Python.h
-lpython2.7
And I have tried the following as well:
g++ testing.cpp -I /home/username/python/include/ -L /usr/include/python2.7/Python.h -lpython2.7
I keep getting the following error:
/usr/include/boost/python/detail/wrap_python.hpp:50:23: fatal error: pyconfig.h: No such
file or directory
# include <pyconfig.h>
I don't know where I am going wrong. I do have boost.python installed, there's just a problem linking?
I just had the same error, the problem is g++ can't find pyconfig.h(shocking, I know). For me this file is located in /usr/include/python2.7/pyconfig.h so appending -I /usr/include/python2.7/ should fix it, alternatively you can add the directory to your path with:
export CPLUS_INCLUDE_PATH="$CPLUS_INCLUDE_PATH:/usr/include/python2.7/"
You can also add this to your .bashrc and it will be added whenever you start your shell next(you will have to reopen your terminal to realize the changes).
You can find your own python include path by using find /usr/include -name pyconfig.h, in my case this returns:
/usr/include/python2.7/pyconfig.h
/usr/include/i386-linux-gnu/python2.7/pyconfig.h
There two possible causes for this symptom: 1. you don't have python-dev installed. 2. you have python-dev installed and your include path is incorrectly configured, which the above posting provide a solution. In my case, I was installing boost, and it is looking for the pyconfig.h header file that is missing in my ubuntu:
The solution is
apt-get install python-dev
In other linux flavors, you have to figure out how to install python header.
If you have a .c file (hello.c) and you want to build an libhello.so library, try:
find /usr/include -name pyconfig.h
[out]:
/usr/include/python2.7/pyconfig.h
/usr/include/x86_64-linux-gnu/python2.7/pyconfig.h
then use the output and do:
gcc -shared -o libhello.so -fPIC hello.c -I /usr/include/python2.7/
If you're converting from cython's .pyx to .so, try this python module, it will automatically build the .so file given the .pyx file:
def pythonizing_cython(pyxfile):
import os
# Creates ssetup_pyx.py file.
setup_py = "\n".join(["from distutils.core import setup",
"from Cython.Build import cythonize",
"setup(ext_modules = cythonize('"+\
pyxfile+".pyx'))"])
with open('setup_pyx.py', 'w') as fout:
fout.write(setup_py)
# Compiles the .c file from .pyx file.
os.system('python setup_pyx.py build_ext --inplace')
# Finds the pyconfig.h file.
pyconfig = os.popen('find /usr/include -name pyconfig.h'\
).readline().rpartition('/')[0]
# Builds the .so file.
cmd = " ".join(["gcc -shared -o", pyxfile+".so",
"-fPIC", pyxfile+".c",
"-I", pyconfig])
os.system(cmd)
# Removing temporary .c and setup_pyx.py files.
os.remove('setup_pyx.py')
os.remove(pyxfile+'.c')
I had a similar experience when building boost for centos7. I was not able to find pyconfig.h on my system only pyconfig-64.h.
After searching around I found that you need to install python-devel to get pyconfig.h
For CentOS do this: yum install python-devel. Then try again.
In my case, I had to create a soft link in my dir /usr/include/
ln -s python3.5m python3.5
the probleme was that i was using python 3.5 but only the python3.5m directory was existing so it wasn't able to find the pyconfig.h file.
In case you have multiple Python installations, the sysconfig module can report the location of pyconfig.h for a given installation.
$ /path/to/python3 -c 'import sysconfig; print(sysconfig.get_config_h_filename())'
/path/to/pyconfig.h
Sorry for such a general title, but i'm not quite sure what exactly i'm missing or what i'm doing wrong. My aim is to build a python extension using boost.python under cygwin and avoiding boost.build tools, that is using make instead of bjam. The latter way was working for me quite good, but now i want to do it this way. I solved many issues by googling and looking for similar topics and that helped me to figure out some tricks and to move forward. Yet at the last step there seems to be some problem. I'll try to describe all my steps in some details in hope that this post may be useful to others in the future and also to better describe the setup.
Because i was not quite sure about the original (from various cygwin repositories) installations of both python and boost i decided to install them from scratch in my home directory, so here is what i do:
first install python. I'll skip the details for this, it is more-or-less straightforward.Important for the latter description is just the path:
/home/Alexey_2/Soft/python2.6 - this is PYTHONPATH, also included in PATH
working on boost:
a) unpack boost source into
/home/Alexey_2/Soft/boost_1_50_0 - this is BOOST_ROOT
b) making bjam. first go into directory:
/home/Alexey_2/Soft/boost_1_50_0/tools/build/v2
next, invoke bootstrap.sh this eventually creates b2 and bjam executables in this directory. In .bash_profile add this directory to PATH, so we can call bjam. Here and after each future edits of .bash_profile i restart cygwin to make the changes
come to the effect
c) still in the
/home/Alexey_2/Soft/boost_1_50_0/tools/build/v2
directory - edit user-config.jam, to let bjam know which python to use. So in my case i only add one line:
using python : 2.6 : /home/Alexey_2/Soft/python2.6/bin/python2.6 : /home/Alexey_2/Soft/python2.6/include/python2.6 : /home/Alexey_2/Soft/python2.6/bin ;
in the lib-path (last entry) i put /home/Alexey_2/Soft/python2.6/bin because it contains libpython2.6.dll
d) ok. now we can make boost-python libraries. go to BOOST_ROOT directory and execute command
bjam --with-python toolset=gcc link=shared
this creates necessary libraries (cygboost_python.dll and libboost_python.dll.a) and places them into
/home/Alexey_2/Soft/boost_1_50_0/stage/lib
building python extension.
here is my simple test program (actually part of the example code)
// file xyz.cpp
#include <boost/python.hpp>
#include <iostream>
#include <iomanip>
using namespace std;
using namespace boost::python;
class Hello
{
std::string _msg;
public:
Hello(std::string msg){_msg = msg;}
void set(std::string msg) { this->_msg = msg; }
std::string greet() { return _msg; }
};
BOOST_PYTHON_MODULE(xyz)
{
class_<Hello>("Hello", init<std::string>())
.def("greet", &Hello::greet)
.def("set", &Hello::set)
;
}
And here is the Makefile:
FLAGS= -fno-for-scope -O2 -fPIC
CPP=c++
# BOOST v 1.50.0
p1=/home/Alexey_2/Soft/boost_1_50_0
pl1=/home/Alexey_2/Soft/boost_1_50_0/stage/lib
# PYTHON v 2.6
p2=/home/Alexey_2/Soft/python2.6/include/python2.6
pl2=/home/Alexey_2/Soft/python2.6/bin
I=-I${p1} -I${p2}
L=-L${pl1} -lboost_python -L${pl2} -lpython2.6
all: xyz.so
xyz.o: xyz.cpp
${CPP} ${FLAGS} ${I} -c xyz.cpp
xyz.so: xyz.o
${CPP} ${FLAGS} -shared -o xyz.so xyz.o ${L}
clean:
rm *.o
rm xyz.so
Some comments:
library paths are set and i compile against proper libraries (see more: compile some code with boost.python by mingw in win7-64bit ).
The link above explains why it is important to configure user-config.jam - i did it in step 1c.
To avoid possible problems (as mentioned in above link and also in Cannot link boost.python with mingw (although for mingw) ) with boost.python libraries liked statically i use
link=shared
as the argument to bjam (see 1d)
As explained here: MinGW + Boost: undefined reference to `WSAStartup#8' the libraries with which one wants to compile something should be listed after object files, that is why we have:
${CPP} ${FLAGS} -shared -o xyz.so xyz.o ${L}
and not
${CPP} ${FLAGS} -shared ${L} -o xyz.so xyz.o
And here is a piece of my .bash_profile (eventually), where i define environmental variables:
# Python
export PATH=/home/Alexey_2/Soft/python2.6/bin:$PATH
export PYTHONPATH=/home/Alexey_2/Soft/python2.6
export LD_LIBRARY_PATH=/home/Alexey_2/Soft/python2.6/lib:/home/Alexey_2/Soft/python2.6/bin:$LD_LIBRARY_PATH
# Boost
export BOOST_ROOT=/home/Alexey_2/Soft/boost_1_50_0
export LD_LIBRARY_PATH=/home/Alexey_2/Soft/boost_1_50_0/stage/lib:$LD_LIBRARY_PATH
export PATH=/home/Alexey_2/Soft/boost_1_50_0/stage/lib:$PATH
# bjam
export PATH=/home/Alexey_2/Soft/boost_1_50_0/tools/build/v2:$PATH
Finally, to the problem. With the above setup i was able to successfully build the python extension object file:
xyz.so
However, when i test it by simple script:
# this is a test.py script
import xyz
the ImportError comes:
$ python test.py
Traceback (most recent call last):
File "test.py", line 1, in <module>
import xyz
ImportError: No module named xyz
It has been noted that the reason for such a problem may be the wrong python executable being used, but this is not the case:
$ which python
/home/Alexey_2/Soft/python2.6/bin/python
what is expected (note python is a symbolic link to python2.6 from that directory)
Here is one more useful piece of information i have:
$ ldd xyz.so
ntdll.dll => /cygdrive/c/Windows/SysWOW64/ntdll.dll (0x76fa0000)
kernel32.dll => /cygdrive/c/Windows/syswow64/kernel32.dll (0x76430000)
KERNELBASE.dll => /cygdrive/c/Windows/syswow64/KERNELBASE.dll (0x748e0000)
cygboost_python.dll => /home/Alexey_2/Soft/boost_1_50_0/stage/lib/cygboost_python.dll (0x70cc0000)
cygwin1.dll => /usr/bin/cygwin1.dll (0x61000000)
cyggcc_s-1.dll => /usr/bin/cyggcc_s-1.dll (0x6ff90000)
cygstdc++-6.dll => /usr/bin/cygstdc++-6.dll (0x6fa90000)
libpython2.6.dll => /home/Alexey_2/Soft/python2.6/bin/libpython2.6.dll (0x67ec0000)
??? => ??? (0x410000)
I'm wondering what
??? => ??? (0x410000)
could possibly mean. May be this what i'm missing. But what is that?
Any comments and suggestions (not only about the last question) are very much appreciated.
EDIT:
Following suggestion of (twsansbury) examining the python module search path with -vv option:
python -vv test.py
gives
# trying /home/Alexey_2/Programming/test/xyz.dll
# trying /home/Alexey_2/Programming/test/xyzmodule.dll
# trying /home/Alexey_2/Programming/test/xyz.py
# trying /home/Alexey_2/Programming/test/xyz.pyc
...
# trying /home/Alexey_2/Soft/python2.6/lib/python2.6/site-packages/xyz.dll
# trying /home/Alexey_2/Soft/python2.6/lib/python2.6/site-packages/xyzmodule.dll
# trying /home/Alexey_2/Soft/python2.6/lib/python2.6/site-packages/xyz.py
# trying /home/Alexey_2/Soft/python2.6/lib/python2.6/site-packages/xyz.pyc
Traceback (most recent call last):
File "test.py", line 1, in <module>
import xyz
ImportError: No module named xyz
The first directory is wherefrom i call python to run the script. The main conclusion is that cygwin python is looking for modules (libraries) with the standard Windows extension - dll (among other 3 types), not the .so as i originally expected from the Linux-emulation-style of cygwin. So changing the following lines in the previous Makefile to:
all: xyz.dll
xyz.o: xyz.cpp
${CPP} ${FLAGS} ${I} -c xyz.cpp
xyz.dll: xyz.o
${CPP} ${FLAGS} -shared -o xyz.dll xyz.o ${L}
clean:
rm *.o
rm xyz.dll
produces xyz.dll, which can successfully be loaded:
python -vv test.py
now gives:
Python 2.6.8 (unknown, Mar 21 2013, 17:13:04)
[GCC 4.5.3] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
# trying /home/Alexey_2/Programming/test/xyz.dll
dlopen("/home/Alexey_2/Programming/test/xyz.dll", 2);
import xyz # dynamically loaded from /home/Alexey_2/Programming/test/xyz.dll
ThatImportError is generally not Boost.Python related. Rather, it normally indicates that xyz is not in the Python Module Search Path.
To debug this, consider running python with -vv arguments. This will cause python to print a message for each file that is checked for when trying to import xyz. Regardless, the build process look correct, so the problem is likely the result of either the file extension or the module is not in the search path.
I am unsure as to how Cygwin will interact with Python's run-time loading behavior. However:
On Windows, python extensions have a .pyd extension.
On Linux, python extensions have a .so extension.
Additionally, verify that the xyz library is located in one of the following:
The directory containing the test.py script (or the current directory).
A directory listed in the PYTHONPATH environment variable.
The installation-dependent default directory.
If the unresolved library shown in ldd causes errors, it will generally manifest as an ImportError with a message indicating undefined references.
How does boost.python deal with Python 3? Is it Python 2 only?
Newer versions of Boost should work fine with Python V3.x. This support has been added quite some time ago, I believe after a successful Google Summer of Code project back in 2009.
The way to use Python V3 with Boost is to properly configure the build system by adding for instance:
using python : 3.1 : /your_python31_root ;
to your user-config.jam file.
libboostpython needs to be built with python3 in order to do this. This doesn't work with boost 1.58 (which comes with Ubuntu 16.04), so make sure you download the latest boost distribution. I just did this with boost_1_64_0.
As mentioned above, find the file "user-config.jam" in you boost code distribution, and copy it to $HOME.
cp /path/to/boost_1_64_0/tools/build/example/user-config.jam $HOME
Then edit the python line (the last line) so that is says:
using python : 3.5 : /usr/bin/python3 : /usr/include/python3.5m : /usr/lib ;
This is correct for Ubuntu 16.04. You can use pkg-config to find the correct include directory.
user#computer > pkg-config --cflags python3
-I/usr/include/python3.5m -I/usr/include/x86_64-linux-gnu/python3.5m
And you only need the first include directory.
Then build boost from scratch. (Sorry.) I install it to /usr/local
cd /path/to/boost_1_64_0
./bootstrap.sh --prefix=/usr/local
./b2
sudo ./b2 install
Now jump into the python example directory, and build the tutorial
cd /path/to/boost_1_64_0/libs/python/example/tutorial
bjam
This will not build correctly if you have a system install of boost, because, under the hood, bjam is linking to libboostpython using the g++ parameter "-lboost". But, on Ubuntu 16.04, this will just go and find "/usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.58.0", and then the python bindings will fail to load. In fact, you'll get his error:
ImportError: /usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.58.0: undefined symbol: PyClass_Type
If you want to see the g++ commands that bjam is using, do this:
user#computer > bjam -d2 -a | grep g++
g++ -ftemplate-depth-128 -O0 -fno-inline -Wall -g -fPIC -I/usr/include/python3.5m -c -o "hello.o" "hello.cpp"
g++ -o hello_ext.so -Wl,-h -Wl,hello_ext.so -shared -Wl,--start-group hello.o -Wl,-Bstatic -Wl,-Bdynamic -lboost_python -ldl -lpthread -lutil -Wl,--end-group
Here we see the problem, you need "-L/usr/includ/lib" just before "-lboost_python". So execute this to link the shared library correctly:
g++ -o hello_ext.so -Wl,-h -Wl,hello_ext.so -shared -Wl,--start-group hello.o -Wl,-Bstatic -Wl,-Bdynamic -L/usr/local/lib -lboost_python -ldl -lpthread -lutil -Wl,--end-group
You may need to rerun ldconfig (or reboot)
sudo ldconfig
And you are finally ready to go:
user#computer > python3
Python 3.5.2 (default, Nov 17 2016, 17:05:23)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import hello_ext
>>> hello_ext.greet()
'hello, world'
>>> exit()
Yes this question is super old, but I had to do something that wasn't specified in any of the answers here (though it was built off some of the suggestions), so I'll quickly jot down my entire process:
Download boost_X_Y_Z.tar.bz2 (I used boost 1.68.0)
tar --bzip2 -xf boost_1_68_0.tar.bz2 (where you want folder to be temporarily)
cd boost_1_68_0
./bootstrap.sh --with-python-version=3.6 --prefix=/usr/local
./b2
sudo ./bjam install
cp tools/build/example/user-config.jam $HOME, then modify the contents of this file to say using python : 3.6 : /usr/bin/python3 : /usr/include/python3.6m : /usr/lib ; (or whatever folders are appropriate for your environment)
Given this C++ source file BoostPythonHelloWorld.cpp:
#include <boost/python.hpp>
char const* say_hi()
{
return "Hi!";
}
BOOST_PYTHON_MODULE(BoostPythonHelloWorld)
{
boost::python::def("say_hi", say_hi);
}
And this Python script BoostPythonHelloWorld.py:
import BoostPythonHelloWorld
print(BoostPythonHelloWorld.say_hi())
It can be compiled and ran as such:
gcc -c -fPIC -I/path/to/boost_1_68_0 -I/usr/include/python3.6 /other_path/to/BoostPythonHelloWorld.cpp
gcc -shared -Wall -Werror -Wl,--export-dynamic BoostPythonHelloWorld.o -L/path/to/boost_1_68_0/stage/lib -lboost_python36 -o BoostPythonHelloWorld.so
python3 BoostPythonHelloWorld.py
The part that was different for me was -Wl,--export-dynamic BoostPythonHelloWorld.o, I had not seen that anywhere else, and I was getting a Python error concerning an undefined symbol until I added that.
Hope this helps someone down the line!
If you get "error: No best alternative for /python_for_extension" be sure to have
using python : 3.4 : C:\\Python34 : C:\\Python34\\include : C:\\Python34\\libs ;
only in user-config.jam in your home path and nowhere else.
Use double backslashes when compiling under windows with mingw (toolset=gcc) or MSVC (toolset=msvc).
Compile with cmd, not msys, and if you also have python 2.7 installed remove that from PATH in that shell.
First do
bootstrap.bat gcc/msvc
assuming you have the gcc/msvc tools available via PATH (/ for the alternatives, but use only one, or leave away)
Afterward you can also do
booststrap.sh --with-bjam=b2
in msys to generate a project-config.jam, but need to edit it to remove the "using python" and "/usr",..
Then the following
b2 variant=debug/shared link=static/shared toolset=gcc/msvc > b2.log
With static the python quickstart examples did not work for me, although I would prefer to do without the boost_python dll.
I did not try on linux, but it should be more straightforward there.
You can even specify the python distribution via
./bootstrap.sh --with-python=<path to your python binary>
e.g.
./bootstrap.sh --with-python=python3
for your system's python3 or
./bootstrap.sh --with-python=$VIRTUAL_ENV/bin/python
for the python version of your currently active virtual env python.
Refer this to know how to build boost with python. It shows the way to build with python2 with Visual Studio 10.0 (2010). But I go through the same procedure for a project that I am currently working on and it works fine with python 3.5 and Visual Studio 14.1 (2017).
If you get this error when building your python boost project, just add BOOST_ALL_NO_LIB value to Preprocessor Definitions (inside C\C++ > preprocessor tab) in your project properties.
And also, do not forget to add boost .dll files location to your system path.
When the path to Python contains spaces, you will be in for quite a ride. After a whole lot of trial and error, I finally managed to get something that works. Behold my user-config.jam (which has to be in my home directory for bjam to find it):
import toolset : using ;
using python : 3.6
: \"C:\\Program\ Files\ (x86)\\Microsoft\ Visual\ Studio\\Shared\\Python36_64\\python.exe\"
: C:\\Program\ Files\ (x86)\\Microsoft\ Visual\ Studio\\Shared\\Python36_64\\include
: C:\\Program\ Files\ (x86)\\Microsoft\ Visual\ Studio\\Shared\\Python36_64\\libs
;
The inconsistent quoting is intended and seems to be required. With this I can build boost-python and use it as Boost::python36 in my CMakeLists.txt. Still, one issue remains: I have to link to python manually viz
target_link_libraries(MyTarget
Boost::boost Boost::python36
"C:/Program Files (x86)/Microsoft Visual Studio/Shared/Python36_64/libs/python36.lib")
target_include_directories(MyTarget PRIVATE
"C:/Program Files (x86)/Microsoft Visual Studio/Shared/Python36_64/include")
In my case adding "Using Python : 3 etc." into user-config.jam in my home directory didn't work. I had to add the line into project-config.jam instead, which resides in the root directory of unpacked boost.
Specifically the line was:
using python : 3.9 : /usr/bin/python3 : /usr/include/python3.9 : /usr/lib ;
and the version of boost was 1_78_0