Travis CI install boost from source log limitation error - c++

It is for a while that I am working on a project and I want to build it on travis ci. My project has a dependency to boost-geometry, so I tried the following travis.yml
language: cpp
dist: Xenial
sudo: true
matrix:
include:
- os: linux
env:
- PYTHON=3.6
- CXX=g++-5
- CC=gcc-5
- CXXFLAGS = "$CXXFLAGS -std=c++14"
# - os: linux
# env: CONDA=3.6 CXX=g++ CC=gcc CXXFLAGS = "$CXXFLAGS -std=c++14"
# Boost
addons:
apt:
update: true
packages:
- libboost-all-dev
- g++-5
before_install:
- sudo add-apt-repository -y ppa:ubuntu-toolchain-r/test
- sudo apt-get update -qq
- sudo apt-get install gcc-5 -y
- sudo apt-get install g++-5 -y
- sudo apt-get install libboost-all-dev -y
- sudo apt-get install python-pip python-dev build-essential
- sudo apt-get install python3-setuptools
- sudo apt-get install python3-pip
- sudo pip install --user --upgrade pip virtualenv
- sudo pip install --user setuptools
- sudo virtualenv -p python3 venv
- source venv/bin/activate
- sudo pip install pybind11
- sudo python3 -m pip install pybind11
- sudo python3 -m pip install --upgrade pip
- cd src
- ls
- cd ../
install:
- ls $TRAVIS_BUILD_DIR
- sudo python3 setup.py install
- sudo python3 setup.py sdist
- sudo pip install --verbose dist/*.tar.gz
script:
- python tests/test.py
In this file it gives me a error. which indicates that this file is not installed as part of boost geometry library.
could not find boost/geometry/core/assert.hpp
So I decided to build boost from source. I used the following file
language: cpp
dist: Xenial
sudo: true
matrix:
include:
- os: linux
env:
- PYTHON=3.6
- CXX=g++-5
- CC=gcc-5
- CXXFLAGS = "$CXXFLAGS -std=c++14"
# - os: linux
# env: CONDA=3.6 CXX=g++ CC=gcc CXXFLAGS = "$CXXFLAGS -std=c++14"
# Boost
addons:
apt:
update: true
sources:
-boost-latest
packages:
- libboost-all-dev
- g++-5
before_install:
- sudo add-apt-repository -y ppa:ubuntu-toolchain-r/test
- sudo apt-get update -qq
- sudo apt-get install gcc-5 -y
- sudo apt-get install g++-5 -y
- sudo apt-get install libboost-all-dev -y
- sudo apt-get install python-pip python-dev build-essential
- sudo apt-get install python3-setuptools
- sudo apt-get install python3-pip
- wget https://sourceforge.net/projects/boost/files/boost/1.70.0/boost_1_70_0.tar.gz
- tar -xzvf boost_1_70_0.tar.gz
- cd boost_1_70_0
- sudo ./bootstrap.sh --prefix=/usr/local --with-libraries=all
- sudo ./b2 install
- sudo /bin/bash -c 'echo "/usr/local/lib" > /etc/ld.so.conf.d/boost.conf'
- sudo ldconfig
- cd ../../
- sudo pip install --user --upgrade pip virtualenv
- sudo pip install --user setuptools
- sudo virtualenv -p python3 venv
- source venv/bin/activate
- sudo pip install pybind11
- sudo python3 -m pip install pybind11
- sudo python3 -m pip install --upgrade pip
- cd src
- ls
- cd ../
# Inform user that Boost 1.55 was successfully installed
# echo "Boost 1.55 was successfully installed."
install:
#- |
# if [ -n "$PYTHON" ]; then
# sudo apt-get install -qq g++-6
# sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-6 90
- ls $TRAVIS_BUILD_DIR
- sudo python3 setup.py install
- sudo python3 setup.py sdist
- sudo pip install --verbose dist/*.tar.gz
# elif [ -n "$CONDA" ]; then
# conda build conda.recipe
# conda install --use-local pydggrid
# fi
script:
- python tests/test.py
It starts to run but as boost writes logs travis raises the following error
The job exceeded the maximum log length, and has been terminated.
Well, what can I do to fix this issue? I looked for a flag to avoid boost generating that amount of logs but could not find any. I also tried to find a why to use apt-get to install boost-geometry but did not get any chance to find proper package. Can some one help me with this issue?
thanks

well I ended up with this method
- wget https://sourceforge.net/projects/boost/files/boost/1.70.0/boost_1_70_0.tar.gz -P $HOME
- cd $HOME
- tar -xzvf boost_1_70_0.tar.gz >/dev/null
and as boost is header only library I did not compile it, I just added its directory in include section of my make script

Related

How to fail codebuild if testing coverage is below a threshold?

I am new to AWS codeBuild and code pipeline, what I am wondering is if there is a way I can fail a pipeline or codeBuild project if the coverage of my test cases is below a certain threshold, i.e 80%?
I have an angular project configured with Karma & Jasmine.
Upon research I have found that I can use this command aws codebuild stop-build --id ${CODEBUILD_BUILD_ID}, not sure if this will help me in any way?
Buildspec.yml
version: 0.2
phases:
install:
commands:
# Install the Angular CLI
- npm install -g #angular/cli
# Install puppeteer as a dev dependency
- npm i -D puppeteer
- npm i -D #angular-devkit/build-angular
# Print out missing libs
- echo "Missing Libs" || ldd ./node_modules/puppeteer/.local-chromium/linux-549031/chrome-linux/chrome | grep not
# Upgrade apt
- apt-get upgrade
# Update libs
- apt-get update
# Install apt-transport-https
- apt-get install -y apt-transport-https
# Use apt to install the Chrome dependencies
- apt-get install -y libxcursor1
- apt-get install -y libgtk-3-dev
- apt-get install -y libxss1
- apt-get install -y libasound2
- apt-get install -y libnspr4
- apt-get install -y libnss3
- apt-get install -y libx11-xcb1
# Print out missing libs
- echo "Missing Libs" || ldd ./node_modules/puppeteer/.local-chromium/linux-549031/chrome-linux/chrome | grep not
# Install project dependencies
- npm install
# - usermod -g 0 -o codebuild-user
build:
# run-as: codebuild-user
commands:
# Run headless Chrome tests
- ng test --browsers=ChromeHeadlessNoSandbox --watch=false --code-coverage
- printenv
I wish to condition the ng tests output here to fail the pipeline.
- aws codebuild stop-build --id ${CODEBUILD_BUILD_ID}
artifacts:
files: imagedefinitions.json

xmlrpc.py Connection refused error while using supervisor in docker

hello guys im writing a docker file and compose with ubuntu 20.04 and try to install supervisor inside it
docker file :
...
FROM ubuntu:20.04
WORKDIR /src/app
ENV BACKENDENVIRONMENT 0
COPY gsigner .
COPY docker_requirements.txt ./
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update
RUN apt install -y python-is-python3
RUN apt-get install -y python3.9
RUN apt-get install python3-pip -y
RUN apt-get install gcc musl-dev python3-dev libffi-dev openssl libssl-
dev cargo -y
RUN apt install -y postgresql postgresql-contrib
RUN apt-get update && apt-get install -y postgresql-server-dev-all gcc
python3-dev musl-dev
RUN pip install --upgrade pip setuptools wheel \
&&pip install -r docker_requirements.txt
RUN mkdir /run/gsigner
RUN apt-get install -y supervisor
COPY backend_supervisord.conf /etc/supervisor/conf.d/
dockerfile updated
docker compose :
version: "3.9"
services:
gsigner:
build: .
command: bash -c "python manage.py migrate && supervisorctl reread && supervisorctl reload&&supervisorctl start daphne"
ports:
- 8000:8000
volumes:
- static:/var/static/gsigner/
- media:/var/media/gsigner/
- gsigner:/src/app/
- log:/var/log/gsigner/
volumes:
static:
media:
log:
gsigner:
dockercompose updated
daphne is my program name in my supervisor conf file
my supervisor conf file :
[supervisord]
[supervisorctl]
[program:daphne]
command=daphne gsigner.asgi:application
directory=/src/app/gsigner/
user=root
autostart=true
autorestart=true
i really did not realize what is happening here
and this is the err msg :
error:error: <class 'ConnectionRefusedError'>, [Errno 111] Connection refused: file: /usr/lib/python3/dist-packages/supervisor/xmlrpc.py line: 560

Opencv works with python2.7 but not with python3.5.2

i installed opencv as follows for both python2 and python3. However, it's only work with python2 .
when l try with python2: import cv2 it works but not with python 3.
here is the script of the commands
https://github.com/milq/milq/blob/master/scripts/bash/install-opencv.sh
sudo apt-get -y update
sudo apt-get -y upgrade
sudo apt-get -y dist-upgrade
sudo apt-get -y autoremove
# INSTALL THE DEPENDENCIES
# Build tools:
sudo apt-get install -y build-essential cmake
# GUI (if you want to use GTK instead of Qt, replace 'qt5-default' with 'libgtkglext1-dev' and remove '-DWITH_QT=ON' option in CMake):
sudo apt-get install -y qt5-default libvtk6-dev
# Media I/O:
sudo apt-get install -y zlib1g-dev libjpeg-dev libwebp-dev libpng-dev libtiff5-dev libjasper-dev libopenexr-dev libgdal-dev
# Video I/O:
sudo apt-get install -y libdc1394-22-dev libavcodec-dev libavformat-dev libswscale-dev libtheora-dev libvorbis-dev libxvidcore-dev libx264-dev yasm libopencore-amrnb-dev libopencore-amrwb-dev libv4l-dev libxine2-dev
# Parallelism and linear algebra libraries:
sudo apt-get install -y libtbb-dev libeigen3-dev
# Python:
sudo apt-get install -y python-dev python-tk python-numpy python3-dev python3-tk python3-numpy
# Java:
sudo apt-get install -y ant default-jdk
# Documentation:
sudo apt-get install -y doxygen
# INSTALL THE LIBRARY (YOU CAN CHANGE '3.2.0' FOR THE LAST STABLE VERSION)
sudo apt-get install -y unzip wget
wget https://github.com/opencv/opencv/archive/3.2.0.zip
unzip 3.2.0.zip
rm 3.2.0.zip
mv opencv-3.2.0 OpenCV
cd OpenCV
mkdir build
cd build
cmake -DWITH_QT=ON -DWITH_OPENGL=ON -DFORCE_VTK=ON -DWITH_TBB=ON -DWITH_GDAL=ON -DWITH_XINE=ON -DBUILD_EXAMPLES=ON ..
make -j4
sudo make install
sudo ldconfig
What's wrong with python3

How to install openCV 2.4.13 for Python 2.7 on Ubuntu 16.04?

I have tried a lot of online posts to install opencv but they are not working for Ubuntu 16.04. May anyone please give me the steps to install openCV 2.4.13 on it?
There's a script I had made a while back, for installing the latest version of OpenCV (4.2 as of the last update to this answer)
Here's the link to it
https://github.com/rsnk96/Ubuntu-Setup-Scripts/blob/master/Build-OpenCV.sh
NOTE: For Ubuntu 16.10+, there are some minor dependency changes you will have to make. Have a look at the comments for the same.
Since you do not want to install opencv contrib, and you specifically want opencv 2.4.13, I have modified the script below. I would suggest you go through the script and understand what is happening before you execute it
sudo apt-get update
sudo apt-get upgrade -y
sudo apt-get install build-essential -y
sudo apt-get install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev -y
sudo apt-get install python3-numpy python3-pip python3-scipy python3-matplotlib python-dev python-matplotlib python-numpy python-scipy -y
sudo apt-get install python-pip python-tk libqt4-dev libqt4-opengl-dev libeigen3-dev yasm libfaac-dev libopencore-amrnb-dev libopencore-amrwb-dev libtheora-dev libvorbis-dev libxvidcore-dev libx264-dev sphinx-common texlive-latex-extra libv4l-dev libdc1394-22-dev libavcodec-dev libavformat-dev libswscale-dev default-jdk ant -y
echo "GUI and openGL extensions"
sudo apt-get install qt4-default libqt4-opengl-dev libvtk5-qt4-dev libgtk2.0-dev libgtkglext1 libgtkglext1-dev -y
echo "image manipulation libraries"
sudo apt-get install libpng3 pngtools libpng12-dev libpng12-0 libpng++-dev -y
sudo apt-get install libjpeg-dev libjpeg9 libjpeg9-dbg libjpeg-progs libtiff5-dev libtiff5 libtiffxx5 libtiff-tools libjasper-dev libjasper1 libjasper-runtime zlib1g zlib1g-dbg zlib1g-dev -y
echo "video manipulation libraries"
sudo apt-get install libavformat-dev libavutil-ffmpeg54 libavutil-dev libxine2-dev libxine2 libswscale-dev libswscale-ffmpeg3 libdc1394-22 libdc1394-22-dev libdc1394-utils -y
echo "codecs"
sudo apt-get install libavcodec-dev -y
sudo apt-get install libfaac-dev libmp3lame-dev -y
sudo apt-get install libopencore-amrnb-dev libopencore-amrwb-dev -y
sudo apt-get install libtheora-dev libvorbis-dev libxvidcore-dev -y
sudo apt-get install ffmpeg x264 libx264-dev -y
sudo apt-get install libv4l-0 libv4l v4l-utils -y
echo "multiproccessing library"
sudo apt-get install libtbb-dev -y
echo "finally download and install opencv"
mkdir opencv
cd opencv
wget "https://github.com/opencv/opencv/archive/2.4.13.2.zip"
unzip opencv-2.4.13.2.zip
cd opencv-2.4.13.2
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=RELEASE \
-DCMAKE_INSTALL_PREFIX=/usr/local \
-DINSTALL_C_EXAMPLES=ON \
-DINSTALL_PYTHON_EXAMPLES=ON \
-DBUILD_EXAMPLES=ON \
-DBUILD_opencv_cvv=OFF \
-DBUILD_NEW_PYTHON_SUPPORT=ON \
-DWITH_TBB=ON \
-DWITH_V4L=ON \
-DWITH_QT=ON \
-DWITH_OPENGL=ON \
-DWITH_VTK=ON ..
echo "making and installing"
make -j8
sudo make install
echo "finishing off installation"
sudo /bin/bash -c 'echo "/usr/local/lib" > /etc/ld.so.conf.d/opencv.conf'
sudo ldconfig
echo "Congratulations! You have just installed OpenCV. And that's all, folks! :P"
P.S. Create a script file out of this and then just execute the script file, rather than copy-pasting it line by line into the terminal. You can do that by copying all of it, placing it in a file with the extension .sh, and then simply running that .sh file from the terminal using $ ./filename.sh
According to this source, from Ubuntu 16.04, you can now just do this to install OpenCV for Python2.7:
pip install opencv-python
or for python3:
pip3 install opencv-python
For more information, see here.
I tested this on my machine and it works, great stuff this :).
This is much easier than all other methods I have come across thus far.
sudo apt-get install build-essential cmake git pkg-config
sudo apt-get install libjpeg8-dev libtiff4-dev libjasper-dev libpng12-dev
sudo apt-get install libgtk2.0-dev
sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev
sudo apt-get install libatlas-base-dev gfortran
sudo apt-get install python2.7-dev
sudo pip install numpy
sudo apt-get install python-opencv
Then you can have a try:
$ python
Python 2.7.6 (default, Oct 26 2016, 20:30:19)
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv
>>> import cv2
If you have more than one opencv installation on your machine, now I tell you the most direct, effective and easy way to choose opencv version in python.
You can type the following command to check the current opencv version in python
import cv2
cv2.__version__
opencv path in python is set in system variable PYTHONPATH, you can echo this variable with echo $PYTHONPATH
If I want to use opencv3.x in python, then using vi or gedit open .bashrc in the home folder, add this content to the end
export PYTHONPATH=/home/ismart/ipa2/devel/lib/python2.7/dist-packages:/home/ismart/catkin_ws/install_isolated/lib/python2.7/dist-packages:/usr/local/opencv3.x/lib/python2.7/dist-packages
Replace the path with yours

how to install scrapy on ubuntu?

I know that intall the scrapy should install the w3lib first,so I install the w3lib firstly,but when I import the scrapy in python ide,the program is crashed.
the error:
creating Twisted.egg-info
writing requirements to Twisted.egg-info\requires.txt
writing Twisted.egg-info\PKG-INFO
writing top-level names to Twisted.egg-info\top_level.txt
writing dependency_links to Twisted.egg-info\dependency_links.txt
writing manifest file 'Twisted.egg-info\SOURCES.txt'
warning: manifest_maker: standard file '-c' not found
reading manifest file 'Twisted.egg-info\SOURCES.txt'
writing manifest file 'Twisted.egg-info\SOURCES.txt'
copying twisted\internet\_sigchld.c -> build\lib.win-amd64-2.7\twisted\internet
creating build\lib.win-amd64-2.7\twisted\internet\iocpreactor\iocpsupport
copying twisted\internet/iocpreactor/iocpsupport\iocpsupport.c -> build\lib.win-amd64-2.7\twisted\internet/iocpreactor/i
ocpsupport
copying twisted\internet/iocpreactor/iocpsupport\winsock_pointers.c -> build\lib.win-amd64-2.7\twisted\internet/iocpreac
tor/iocpsupport
copying twisted\python\_epoll.c -> build\lib.win-amd64-2.7\twisted\python
copying twisted\python\_initgroups.c -> build\lib.win-amd64-2.7\twisted\python
copying twisted\python\sendmsg.c -> build\lib.win-amd64-2.7\twisted\python
copying twisted\runner\portmap.c -> build\lib.win-amd64-2.7\twisted\runner
copying twisted\test\raiser.c -> build\lib.win-amd64-2.7\twisted\test
running build_ext
What's wrong?
This is how I installed scrapy on ubuntu:
sudo apt-get update
sudo apt-get install python-pip build-essential python-dev libxslt-dev libxml2-dev
sudo -H pip install Scrapy
scrapy version
The important thing that solved my issues was sudo -H pip install Scrapy specifically the -H flag.
I also exited out of the terminal and started a new terminal to ensure the all the environment variables were set correctly
Make sure you had installed the Twisted, pyOpenSSL and pycrypto.
These are my steps to install scrapy on ubuntu.
1.install gcc and lxml:
sudo apt-get install python-dev
sudo apt-get install libevent-dev
sudo apt-get install libxml2 libxml2-dev
apt-get install libxml2-dev libxslt-dev
apt-get install python-lxml
2.install twisted:
sudo apt-get install python-twisted python-libxml2 python-simplejson
sudo apt-get install build-essential libssl-dev libffi-dev python-dev
3.install pyOpenSSL:
wget http://pypi.python.org/packages/source/p/pyOpenSSL/pyOpenSSL-0.13.tar.gz
tar -zxvf pyOpenSSL-0.13.tar.gz
cd pyOpenSSL-0.13
sudo python setup.py install
4.install pycrypto
wget http://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.5.tar.gz
tar -zxvf pycrypto-2.5.tar.gz
cd pycrypto-2.5
sudo python setup.py install
5.install easy_install:(if you don't have easy_install)
wget http://peak.telecommunity.com/dist/ez_setup.py
python ez_setup.py
6.install w3lib
sudo easy_install -U w3lib
7.install scrapy
sudo easy_install Scrapy
If you wanna know much,please goto my blog.
First install system dependencies
sudo apt-get install -y \
python-dev python-pip python-setuptools \
libffi-dev libxml2-dev libxslt1-dev \
libtiff5-dev libjpeg62-turbo-dev zlib1g-dev libfreetype6-dev \
liblcms2-dev libwebp-dev tcl8.5-dev tk8.5-dev python-tk
Then add followings in your requirements.txt
lxml
pyOpenSSL
Scrapy
Pillow
And finally pip install -r requirements.txt
You can look around gist.github.com as well to resolve latest dependencies issues. I'm using docker to setup scrapy deps in a separate container.
I've created one for mine needs here