I am currently trying to deploy a project using openmp. I have the flag '-fopenmp' on Travis.
How could I fix that ?
In local I just brew install libopenmp which solved the issue. But not on Travis, what are the options ?
Using cython I got the following ".travis.yml"
os: linux
dist: xenial
language: python
python:
- "3.7"
cache: pip
addons:
apt:
packages:
- patchelf
matrix:
include:
- os: osx
# No version of Python is available via virtualenv on OS X workers, see https://github.com/travis-ci/travis-ci/issues/2312
language: generic
env: TOXENV=py37
fast_finish: true
before_install:
brew install libomp
install:
- pip install --upgrade "pip < 19.1" -r CI/requirements.txt
- python setup.py develop
script:
- pytest
Travis fails while executing :
clang -fno-strict-aliasing -fno-common -dynamic -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/include -I/usr/local/opt/openssl#1.1/include -I/usr/local/opt/sqlite/include -I/usr/local/Cellar/python#2/2.7.17/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c cpt/alphabet.c -o build/temp.macosx-10.13-x86_64-2.7/cpt/alphabet.o -fopenmp
While in local it compiles with python 3.7, how could I fix that aswell ?
On apple's llvm, -fopenmp is not supported. One should use brew's llvm.
The following ables to link openmp:
- brew install llvm libomp
- export CPP=/usr/local/opt/llvm/bin/clang;
For reference, the issue where there are all the commands: https://github.com/bluesheeptoken/CPT/issues/68#issuecomment-563342866
Related
When am trying to install the memsql python library getting below error
I have tried the below options already:
sudo pip install memsql
sudo pip install memsql --global-option=build_ext --global-option="-I/usr/local/opt/openssl/include" --global-option="-L/usr/local/opt/openssl/lib"
Installing collected packages: MySQL-python, memsql
Running setup.py install for MySQL-python ... error
Complete output from command /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python -u -c "import setuptools, tokenize;__file__='/private/tmp/pip-install-JDxMyP/MySQL-python/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" build_ext -I/usr/local/opt/openssl/include -L/usr/local/opt/openssl/lib install --record /private/tmp/pip-record-azfFw0/install-record.txt --single-version-externally-managed --compile:
running build_ext
building '_mysql' extension
creating build
creating build/temp.macosx-10.14-intel-2.7
cc -fno-strict-aliasing -fno-common -dynamic -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -Dversion_info=(1,2,5,'final',1) -D__version__=1.2.5 -I/usr/local/Cellar/mysql/8.0.15/include/mysql -I/usr/local/opt/openssl/include -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c _mysql.c -o build/temp.macosx-10.14-intel-2.7/_mysql.o
_mysql.c:44:10: fatal error: 'my_config.h' file not found
#include "my_config.h"
^~~~~~~~~~~~~
1 error generated.
error: command 'cc' failed with exit status 1}
In order to install memsql-python you need to ensure that you have the dependencies already available. Specifically, the dependency that seems to be missing from your environment is libmysqlclient-dev or an equivalent package.
Note - you are overriding the include paths to point at your openssl installation which may be causing this issue due to how the compiler is searching for development headers.
According to the project's readme, use the following steps to get started quickly on Ubuntu:
sudo apt-get update
sudo apt-get install -y mysql-client python-dev libmysqlclient-dev python-pip
sudo pip install memsql
Use the following steps to get started with RHEL based distributions such as Amazon Linux or Centos:
sudo yum update
sudo yum install -y gcc mysql-devel
sudo pip install memsql
-- edit to address question regarding installation on Mac OSX
On a Mac it appears that you can install the mysql-client package from brew which in theory should provide the libmysqlclient-dev package. Once available try installing the memsql-python package without overriding linker flags since memsql-python should build fine without openssl. If you need openssl support then you will need to ensure that both the openssl and libmysqlclient-dev shared objects are available to the linker at compilation time.
This procedure worked for me:
brew remove mysql
brew install mysql#5.7
brew link --force mysql#5.7
I read a post and adjusted the procedure to link things in the /usr/local/bin instead of adding a new route to the PATH variable.
Reference: MacOS: "pip install MySQL-python" returning an error: "_mysql.c:44:10: fatal error: 'my_config.h' file not found"
I am trying to create a one to one real-time messaging system using Django. Upon searching I found out about django-channels. So i created a virtual env with python 3.5 and latest Django but when I do pip install channels I get :
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.5m -I/home/kshitij/Documents/projects/personal/maker/venv/include/python3.5m -c src/twisted/test/raiser.c -o build/temp.linux-x86_64-3.5/src/twisted/test/raiser.o
#include "Python.h"
^
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
probably because of missing python headers so i did : sudo apt-get install python3-dev but it still shows the same error. What should I do?
pip version: 18
python version: 3.5
django version: 2.1
I tried doing sudo apt-get install python3-dev but getting:
ImportError: cannot import name '_BACKCOMPAT_MAGIC_NUMBER'
Verify your pip is not pip and its pip3.
You may also try installing sudo apt install python3.X-dev.
For you, it's sudo apt install python3.5-dev
Try installing the Linux compiler and python3-dev:
sudo apt install gcc
sudo apt install python3-dev
pip install channels
The last error may indicates you have an issue caused by many python versions installed on your system (see this answer). You should make sure you don't have incomplete or corrupted python installation on your system (did you already compiled Python by yourself?). In addition, make sure the package python3 is correctly installed or sudo apt-get install python3-dev cannot succeed.
In addition, you may try to locate Python.h on your system. Make sure each Python.h on your system correspond to a valid python setup (you may have one for python 2 and one more for python 3).
I'm just getting started with using travis-CI, so I apologize if this is a silly or obvious question.
Following the instructions here:
I wrote the following travis.yml
language: cpp
dist: trusty
matrix:
include:
- os: linux
compiler: gcc
addons:
apt:
sources:
- ubuntu-toolchain-r-test
packages:
- g++-7
env:
- MATRIX_EVAL="CC=gcc-7 && CXX=g++-7"
- os: linux
compiler: clang
addons:
apt:
sources:
- llvm-toolchain-trusty-5.0
packages:
- clang-5.0
env:
- MATRIX_EVAL="CC=clang-5.0 && CXX=clang++-5.0"
before_install:
- eval "${MATRIX_EVAL}"
script:
- mkdir build
- cd build
- cmake -DCMAKE_VERBOSE_MAKEFILE=ON ..
- cmake --build .
- ctest
Which causes the following error in the clang build:
/home/travis/build/path_to_project/./include/abulafia/support/type_traits.h:20:12:
error: no member named 'decay_t' in namespace 'std'; did you mean
'decay'?
/usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/type_traits:1725:11:
note: 'decay' declared here
When compiling with the following command:
cd /home/travis/build/path_to_project/build/tests/char_set && /usr/bin/clang++-5.0 -Wall -pedantic -Wextra -std=c++17 -I/home/travis/build/path_to_project/./include -I/home/travis/build/path_to_project/googletest/googletest/include -o CMakeFiles/char_set_tests.dir/test_any.cpp.o -c
Which tells me it's loading gcc's libraries. Is there something I'm not understanding right here?
Thanks!
Yes, this is a well-known issue with the travis-ci build environment. It is compiling and linking against the default ubuntu-trusty libstdc++, which is the gcc 4-series stdlib and not even C++11 conforming.
See an issue I opened a long time ago.
If you need a C++14 libstdc++ with travis-ci, you should use docker and make a more recent ubuntu image. That's the best workaround AFAIK.
This can be fixed by installing g++7 alongside clang, to upgrade the standard library. The relevant matrix entry becomes:
matrix:
include:
- os: linux
addons:
apt:
sources:
- llvm-toolchain-trusty-5.0
- ubuntu-toolchain-r-test
packages:
- clang-5.0
- g++-7
env: MATRIX_EVAL="CC=clang-5.0 && CXX=clang++-5.0"
Replacing this into the OP's yaml should do the trick. Note: compiler: clang was in excess – its effects get overridden by the eval "${MATRIX_EVAL}" trick.
I understand some dependencies need to be satisfied before installing cartopy and basemap. I sorted out most of them using pip (like numpy etc.).
I found out there are some cartopy issues with Proj.4 and some basemap issues with GEOS.
I thought the python bindings to the Java project Proj.4 were enough and I am a bit lost with the C++ library GEOS (I think GDAL is enough for GEOS, I've linked it to the python world this way: pip install --global-option=build_ext --global-option="-I/usr/include/gdal" GDAL==`gdal-config --version).
I would prefer avoiding all those things like anaconda, canopy, etc.
If possible I would like to use only pip on both Ubuntu (apt-get, only if pip is not enough) and Mac OSX (homebrew e.g. brew install <some_package>, only if pip is not enough).
cartopy gets stuck on a version of Proj.4 that is too old. The pip output says: Proj4 version 4.8.0 is installed, but cartopy requires at least version 4.9.0..
I was assuming this pip install of the python bindings was enough, but it does not solve the problem:
$ pip show pyproj
Name: pyproj
Version: 1.9.5.1
Summary: Python interface to PROJ.4 library
Home-page: https://github.com/jswhit/pyproj
Author: Jeff Whitaker
Author-email: jeffrey.s.whitaker#noaa.gov
License: OSI Approved
Location: /usr/local/lib/python2.7/dist-packages
Requires:
I don't get why the python bindings to Proj.4 are not enough, even if those bindings are the latest version available using pip.
basemap is stuck in different ways:
Using pip2 install basemap somehow I get Could not find a version that satisfies the requirement basemap (from versions: )
No matching distribution found for basemap
But then from the basemap documentation I found that library is not linked to the pip repositories for some reason and needs to be installed from the source code. So I've run: pip2 install https://github.com/matplotlib/basemap/archive/v1.0.7rel.tar.gz and I believe that a GEOS compilation of the C++ code is triggered and after a while it stops with at the bottom this:
In file included from src/_geoslib.c:255:0:
/usr/include/geos_c.h:151:22: note: expected ‘GEOSMessageHandler’ but argument is of type ‘void (*)(char *, char *)’
extern void GEOS_DLL initGEOS(GEOSMessageHandler notice_function,
^
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/src/_geoslib.o -lgeos_c -lgeos -o build/lib.linux-x86_64-2.7/_geoslib.so
/usr/bin/ld: cannot find -lgeos
collect2: error: ld returned 1 exit status
/usr/bin/ld: cannot find -lgeos
collect2: error: ld returned 1 exit status
error: Command "x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/src/_geoslib.o -lgeos_c -lgeos -o build/lib.linux-x86_64-2.7/_geoslib.so" failed with exit status 1
----------------------------------------
Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-QVrKRr-build/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-EgAOPT-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-QVrKRr-build/
I am not sure what I am doing wrong:
how to provide the correct Proj.4 stuff to cartopy? Actually I would prefer finding an older version of cartopy that is fine with the existing version of Proj.4 (which I am assuming comes from the pip install bindings)
how to get basemap finding GEOS from the GDAL links I've setup with pip? If that's not enough, then what should I do more? Perhaps some apt-get of a GEOS pakcage that avoids triggering the GEOS compilation inside the basemap installation?
I am a bit confused in this no mans land between Java binaries, C++ binaries and bindings to python packages.
As you noticed cartopy requires Proj.4 lib in version >=4.9. On Ubuntu (newer than 16.04) you can install it through sudo apt-get install libproj-dev.
For older Ubuntu, e.g. 14.04, you could simply download newest package from 16.04 (xenial libproj-dev) and install it manually. Notice that it depends on libproj9 in version 4.9. Remember to remove older version before you update them with
sudo apt-get purge libproj-dev libproj9
Once you done that, open and download packages from links for appropriate architecture (from your log I understand that's amd64). Double click downloaded files in order librproj9 and libproj-dev or use
sudo dpkg -i <path_to_file>.
When I sudo pip install pyquery, sudo pip install lxml, and sudo pip install cython, I get very similar output with the same error that says:
x86_64-linux-gnu-gcc: error: unrecognized command line option ‘-fstack-protector-strong’
Here's the complete pip output for sudo pip install pyquery:
Requirement already satisfied (use --upgrade to upgrade): pyquery in /usr/local/lib/python2.7/dist-packages
Downloading/unpacking lxml>=2.1 (from pyquery)
Running setup.py egg_info for package lxml
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'
warnings.warn(msg)
Building lxml version 3.4.1.
Building without Cython.
Using build configuration of libxslt 1.1.28
Requirement already satisfied (use --upgrade to upgrade): cssselect in /usr/local/lib/python2.7/dist-packages/cssselect-0.9.1-py2.7.egg (from pyquery)
Installing collected packages: lxml
Running setup.py install for lxml
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'
warnings.warn(msg)
Building lxml version 3.4.1.
Building without Cython.
Using build configuration of libxslt 1.1.28
building 'lxml.etree' extension
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/libxml2 -I/root/build/lxml/src/lxml/includes -I/usr/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o -w
x86_64-linux-gnu-gcc: error: unrecognized command line option ‘-fstack-protector-strong’
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
Complete output from command /usr/bin/python -c "import setuptools;__file__='/root/build/lxml/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externally-managed --record /tmp/pip-gg4SuG-record/install-record.txt:
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'
warnings.warn(msg)
Building lxml version 3.4.1.
Building without Cython.
Using build configuration of libxslt 1.1.28
running install
running build
running build_py
copying src/lxml/includes/lxml-version.h -> build/lib.linux-x86_64-2.7/lxml/includes
running build_ext
building 'lxml.etree' extension
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/libxml2 -I/root/build/lxml/src/lxml/includes -I/usr/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o -w
x86_64-linux-gnu-gcc: error: unrecognized command line option ‘-fstack-protector-strong’
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
Command /usr/bin/python -c "import setuptools;__file__='/root/build/lxml/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externally-managed --record /tmp/pip-gg4SuG-record/install-record.txt failed with error code 1 in /root/build/lxml
Storing complete log in /root/.pip/pip.log
The complete log from /root/.pip/pip.log is here: http://pastebin.com/5R4VZDu8
I've looked at this, this, this, and this for help, but I haven't been able to solve the issue yet.
I've already installed libxml2-dev, libxslt1-dev, and python-dev. I'm running Debian 7.0 x64 on a DigitalOcean droplet.
I'm just trying to install pyquery. Can someone please help me out?
Thanks
I ran into this problem when I attempted to upgrade my pandas version to 0.15.2
If you install gcc-4.9 you are likely to still have an older version of gcc on your system (gcc-4.7 in my case).
I can think of 3 ways to solve this issue:
a) symlink /usr/bin/x86_64-linux-gnu-gcc to
/usr/bin/x86_64-linux-gnu-gcc-4.9
if you want to be more organized about this use update-alternatives, see https://askubuntu.com/questions/26498/choose-gcc-and-g-version
b) figure out how to manually specify which compiler pip uses and set it in some sort of .conf file - I haven't examined where this file lives or if there are CLI options for pip that accomplish the equivalent. In principle, creating/editing a /usr/lib/pythonX.Y/distutils/distutils.cfg should do it. I ran into problems when I tried to use this approach.
c) Edit /usr/lib/python2.7/plat-x86_64-linux-gnu/_sysconfigdata_nd.py to reflect your updated compiler
How to use MinGW's gcc compiler when installing Python package using Pip?
https://docs.python.org/2/install/#distutils-configuration-files
I went with the quick and dirty solution (a) to force everything to work
root#localhost:/home/user1$ rm /usr/bin/x86_64-linux-gnu-gcc
root#localhost:/home/user1$ ln -s /usr/bin/gcc-4.9 /usr/bin/x86_64-linux-gnu-gcc
root#localhost:/home/user1$ pip install pandas --upgrade
. . . pandas compiles with gcc-4.9 here . . .
move things back to how they were
root#localhost:/home/user1$ rm /usr/bin/x86_64-linux-gnu-gcc
root#localhost:/home/user1$ ln -s /usr/bin/gcc-4.7 /usr/bin/x86_64-linux-gnu-gcc
One possible solution would be to use GCC 4.9 or higher, which does support the -fstack-protector-strong flag.
Instead of upgrading to GCC 4.9, I was trying to find where the flag was defined and remove it. On Debian Wheezy, I found it in /usr/lib/python2.7/plat-x86_64-linux-gnu/_sysconfigdata_nd.py which belongs to the libpython2.7-minimal package. My solution was to replace all occurrences of -fstack-protector-strong with -fstack-protector within this file. Then pip install executed the correct build command.