I have ansible (v2.0.0.2) and python (v2.7.6) and i'm running the 'maven_artifact' module.
As a direct ansible command, it works fine
ansible localhost -m maven_artifact -a "group_id=commons-collections artifact_id=commons-collections dest=/tmp/commons-collections-latest.jar" -vvvv
but when i do the same via a playbook
- name: download via maven
maven_artifact: group_id=junit artifact_id=junit dest=/tmp/junit-latest.jar
it fails with this error
fatal: [test01vm1]: FAILED! => {"changed": false, "failed": true, "invocation": {"module_name": "maven_artifact"},
"module_stderr": "",
"module_stdout": "\r\nTraceback (most recent call last):\r\n
File \"/home/admin123/.ansible/tmp/ansible-tmp-1454675562.75-201853614879442/maven_artifact\",
line 25, in <module>\r\n from lxml import etree\r\nImportError:
No module named lxml\r\n", "msg": "MODULE FAILURE", "parsed": false}
I believe might be related to the python lxml module, and i found these existing tickets
http://stackoverflow.com/questions/13355984/get-errors-when-import-lxml-etree-to-python
http://stackoverflow.com/questions/4598229/installing-lxml-module-in-python
I'm wondering might someone have a workaround for this?
EDIT - Add python path details
I ran this command to see what path's are on the python home
14:55:11#pcZBook-15:/usr/local/etc$ python -c 'import sys; print(":".join(sys.path))'
The list of folders is
:/opt/stack/keystone
:/opt/stack/glance
:/opt/stack/cinder
:/opt/stack/nova
:/opt/stack/horizon
:/usr/lib/python2.7
:/usr/lib/python2.7/plat-x86_64-linux-gnu
:/usr/lib/python2.7/lib-tk
:/usr/lib/python2.7/lib-old
:/usr/lib/python2.7/lib-dynload
:/usr/local/lib/python2.7/dist-packages
:/usr/lib/python2.7/dist-packages
:/usr/lib/python2.7/dist-packages/PILcompat
:/usr/lib/python2.7/dist-packages/gtk-2.0
:/usr/lib/pymodules/python2.7
:/usr/lib/python2.7/dist-packages/ubuntu-sso-client
:/usr/lib/python2.7/dist-packages/wx-2.8-gtk2-unicode
It seems that ansible was trying to execute the 'maven_artifact' command on the remote target host, which didn't have the required python libraries.
In my case I only wanted to run the command on the local 'ansible_host' so I just added the 'local_action' prefix and the command runs.
- name: download via maven
local_action: maven_artifact group_id=junit artifact_id=junit dest=/tmp/junit-latest.jar
What I usually do is to call:
- name: Install PIP
apt: name=python-pip state=present
- name: Install lxml
pip: name=lxml
Just do
sudo apt-get install python-lxml
You need to install lxml in [test01vm1].
pip install lxml
Verify that target machine has python lxml module, and ansible selects the python interpreter where you have installed the lxml module.
[ec2-user#i-05f345345aas6b bin]$ pip list | grep xml
lxml 3.2.1
And then double check that ansible picks that python version by adding:
ansible_python_interpreter=/usr/bin/python2
See https://docs.ansible.com/ansible/latest/reference_appendices/interpreter_discovery.html
I have observed that ansible 2.12 ansible picks python3 interpreter by default, if no python interpreter is defined in target machine. So double check with
pip3 list | grep lxml
that lxml module is also available for python3
Related
I am trying to run ansible playbook via jenkins groovy scripts but keep getting error: boto3 is required. I have already installed boto3:
pip list boto | grep boto
boto3 1.20.3
botocore 1.23.3
I have inventory as:
[localhost]
localhost ansible_connection=local ansible_python_interpreter=/usr/local/bin/python
Python:
which python
/usr/bin/python
pip:
which pip
/home/john/.local/bin/pip
boto:
find $HOME/.local -name 'boto3' -type d/home/john/.local/lib/python3.6/site-packages/boto3
versions:
pip --version
pip 21.3.1 from /home/john/.local/lib/python3.6/site-packages/pip (python 3.6)
python --version
Python 3.6.9
Ansible:
which ansible
/usr/bin/ansible
Playbook in sh file:
ansible-playbook -c local \
-e ansible_python_interpreter=$(which python) \
-i localhost, \
-e env="'${ENV}'" \
-e image="'${IMAGE_NAME}'" \
-e version="'${BUILD_NUMBER}'" \
infra/test.ansible.yaml
What else did I miss to configure?
Finally after days of struggle fixed my problem using the steps below:
Created virtual environment for python, boto and ansible
Edit the ansible inventory file to point the interpreter to python instead of /usr/bin/python
sudo pip install virtualenv
sudo pip install boto botocore
source ansible_vEnv/bin/activate
Set the following in ansible inventory:
[localhost]
localhost ansible_python_interpreter=python
Gave the playbook command as ansible-playbook -c local dir/test.yaml
Note: make sure you use boto in yaml file and not boto3:
- hosts: localhost
gather_facts: no
tasks:
- name:
pip:
name: boto // here
state: present
Pointing the interpreter to python will actually pickup the python from our isolated environment i.e. virtual environment we created in step 2.
Also, I did installed ansible as root using:
sudo su -
Uninstalled the existing ansible that was installed using apt-get
Installed ansible using pip install ansible
Set the path in Jenkins as /usr/bin for ansible plugin in global tool configuration
I am trying to install django using pipenv but failing to do so.
pipenv install django
Creating a virtualenv for this project...
Pipfile: /home/djangoTry/Pipfile
Using /usr/bin/python3.9 (3.9.5) to create virtualenv...
⠴ Creating virtual environment...RuntimeError: failed to query /usr/bin/python3.9 with code 1 err: 'Traceback (most recent call last):\n File "/home/.local/lib/python3.6/site-packages/virtualenv/discovery/py_info.py", line 16, in <module>\n from distutils import dist\nImportError: cannot import name \'dist\' from \'distutils\' (/usr/lib/python3.9/distutils/__init__.py)\n'
✘ Failed creating virtual environment
[pipenv.exceptions.VirtualenvCreationException]:
Failed to create virtual environment.
Other info
python --version
Python 2.7.17
python3 --version
Python 3.6.9
python3.9 --version
Python 3.9.4
pip3 --version
pip 21.1 from /home/aman/.local/lib/python3.6/site-packages/pip (python 3.6)
Please help.Thanks
The problem isn't with installing Django, is on creating your venv, I don't create on this way so I will pass the way that i create
Step 1 - Install pip
Step 2 - Install virtaulEnv
check here how to install both
after installed, run python3 -m venv venvthis wil create an venv on your current directory. Initiate them with . venv/bin/activateand so run pip install django. That should work
Try pipenv install --python 3.9
How to enable python 2.7 library like GDAL in Google App Engine standard? Currently there are linux python-modules in lib-folder in app engine, but when trying to run the code through endpoints, app engine gives internal server error: ImportError: No module named _gdal. I'm using pygdal version 2.2.3.3. Should the libgdal (demanded for pygdal)be installed also on app engine, and if so, how to do it? I installed GDAL locally into lib folder (using ubuntu bash on windows10) following these instructions using this syntax: sudo pip install --target lib --requirement requirements.txt --ignore-installed as it says here. Please help!
From What compiler can I use to build GDAL/OGR?
GDAL/OGR is written in ANSI C and C++. It can be compiled with all modern C/C++ compilers.
Which means it's incompatible with the (1st generation/python 2.7) standard environment Pure Python sandbox requirement:
All code for the Python runtime environment must be pure Python, and
not include any C extensions or other code that must be compiled.
You may want to look at the flexible environment instead. Probably with a custom runtime, see Up-to-date pip with AppEngine Python flex env?
Google App Engine's standard environment for Python27 only supports a specific set of third-party libraries that use C-extensions, listed here. pygdal is not in the list.
You may want to look into the Python3 standard runtime, though it is in beta. It allows you to install arbitrary third-party libraries.
Modifying this links' answer I managed to get GDAL working in App Engine Flexible.
My dockerfile:
FROM gcr.io/google-appengine/python
RUN apt-get update && apt-get -y install libproj-dev libgdal-dev
RUN export CPLUS_INCLUDE_PATH=/usr/include/gdal
RUN export C_INCLUDE_PATH=/usr/include/gdal
RUN gdal-config --version
# Create a virtualenv for dependencies. This isolates these packages from
# system-level packages.
RUN virtualenv /env -p python2.7
# Setting these environment variables are the same as running
# source /env/bin/activate.
ENV VIRTUAL_ENV /env
ENV PATH /env/bin:$PATH
# Copy the application's requirements.txt and run pip to install all
# dependencies into the virtualenv.
ADD requirements.txt requirements.txt
RUN pip install -r requirements.txt
# Add the application source code.
ADD . /app
CMD gunicorn -t 120 -b :$PORT main:app
My app.yaml-file:
runtime: custom
env: flex
entrypoint: gunicorn -t 120 -b :$PORT main:app
endpoints_api_service:
name: xxxxx.com
rollout_strategy: managed
beta_settings:
cloud_sql_instances: project:europe-north1:dbinstance
runtime_config:
python_version: 2
requirements.text-file:
pygdal==1.11.3.3
I'm following this for django manage.py module
http://docs.ansible.com/django_manage_module.html
for e.g. one of my tasks looks like -
- name: Django migrate
django_manage: command=migrate
app_path={{app_path}}
settings={{django_settings}}
tags:
- django
this works perfectly fine with python2(default in ubuntu) but when I try with python3-django project it throws error
failed: [123.456.200.000] => (item=school) => {"cmd": "python manage.py makemigrations --noinput school --settings=myproj.settings.production", "failed": true, "item": "school", "path": "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games", "state": "absent", "syspath": ["/home/ubuntu/.ansible/tmp/ansible-tmp-1432039779.41-30449122707918", "/usr/lib/python2.7", "/usr/lib/python2.7/plat-x86_64-linux-gnu", "/usr/lib/python2.7/lib-tk", "/usr/lib/python2.7/lib-old", "/usr/lib/python2.7/lib-dynload", "/usr/local/lib/python2.7/dist-packages", "/usr/lib/python2.7/dist-packages"]}
msg:
:stderr: Traceback (most recent call last):
File "manage.py", line 8, in <module>
from django.core.management import execute_from_command_line
ImportError: No module named django.core.management
from this error it seems Ansible bydefault uses Python2. can we change this to python3 or anyother workaround?
PS: pip freeze ensure that django 1.8 has installed (for python3 using pip3)
Suggestions:
when I run ubuntu#ubuntu:/srv/myproj$ python3 manage.py migrate it works fine. so I'm thinking of passing command directly
something like
- name: Django migrate
command: python3 manage.py migrate
tags:
- django
but how do I pass the project path or manage.py file's path, there is only an option to pass settings, something like --settings=myproject.settings.main.
can we do by passing direct command?
From Ansible website http://docs.ansible.com/intro_installation.html
Python 3 is a slightly different language than Python 2 and most Python programs (including Ansible) are not switching over yet. However, some Linux distributions (Gentoo, Arch) may not have a Python 2.X interpreter installed by default. On those systems, you should install one, and set the ‘ansible_python_interpreter’ variable in inventory (see Inventory) to point at your 2.X Python. Distributions like Red Hat Enterprise Linux, CentOS, Fedora, and Ubuntu all have a 2.X interpreter installed by default and this does not apply to those distributions. This is also true of nearly all Unix systems. If you need to bootstrap these remote systems by installing Python 2.X, using the ‘raw’ module will be able to do it remotely.
If you edit the shebang in the Django manage.py file to be #!/usr/bin/env python3 then you can ensure that python 3 will always be used to run your Django app.
Tried successfully with Ansible 2.3.0 and Django 1.10.5. YMMV
Ansible is using python to run the django command: https://github.com/ansible/ansible-modules-core/blob/devel/web_infrastructure/django_manage.py#L237
Your only solution is thus to override the executable that will be run, for instance by changing your PATH:
- file: src=/usr/bin/python3 dest=/home/user/.local/bin/python state=link
- name: Django migrate
django_manage: command=migrate
app_path={{app_path}}
settings={{django_settings}}
environment:
- PATH: "/home/user/.local/bin/:/bin:/usr/bin:/usr/local/bin"
I have intalled gunicorn,but gunicorn command not found:
# pip3.4 install gunicorn
Requirement already satisfied (use --upgrade to upgrade): gunicorn in /usr/local/python3.4/lib/python3.4/site-packages
# gunicorn
-bash: gunicorn: command not found
what is the problem,is gunicorn install path not be recognized by system?
I faced the same issue and it turns out I had to add gunicorn binary path to Linux PATH variable. You can start by echoing $PATH to see all binary path listed on the system. Then find out where gunicorn is installed. For my case I was using python virtual environment and pyenv which helps manage several python versions and dependencies separately.
(venv3.6) dave#daverig (develop)✗ % pip show gunicorn
Name: gunicorn
Version: 19.7.1
Summary: WSGI HTTP Server for UNIX
Home-page: http://gunicorn.org
Author: Benoit Chesneau
Author-email: benoitc#e-engura.com
License: MIT
Location: /home/dave/.pyenv/versions/3.6.2/envs/venv3.6/lib/python3.6/site-packages
Notice gunicorn is installed in /home/dave/.pyenv/versions/3.6.2/envs/venv3.6/lib/python3.6/site-packages and the corresponding path for the binaries for this particular python version is at /home/dave/.pyenv/versions/3.6.2/envs/venv3.6/bin. So I had to add that to Linux path via ~/.profile file like so;
export PATH=$PATH:$HOME/.pyenv/versions/3.6.2/envs/venv3.6/bin then ofcourse you want to refresh this using source ~/.profile or restart your terminal. Once I was able to do this, gunicorn binary was now available on my console;
(venv3.6) dave#daverig (develop)✗ % gunicorn --version
gunicorn (version 19.7.1)
I had the same problem on Debian.
It turns out that on Debian the documentation advises to install gunicorn via apt:
$ sudo apt install gunicorn
i just created a file named gunicorn and type these codes below which is the same as my development server , and included it into system path,such as /usr/bin
#!/usr/local/bin/python3.4
#-*- coding: utf-8 -*-
import re
import sys
from gunicorn.app.wsgiapp import run
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$','',sys.argv[0])
sys.exit(run())
in this way, it solved my problem,but still confused me,why gunicorn command not generated and included into system path automatically?and why my development server did ,both the same OS (centos 6.5 x64)
Installing gunicorn from source saved me after 2 hours trying!
pip3 install git+https://github.com/benoitc/gunicorn.git
If you installed python3 from source compiled, you should export your python3 PATH:
export PATH=$PATH:/usr/local/python3/bin
Are you use python3.4-venv?
if true
Delete env folder
Just reinstall: python3.4-venv. Ex for ubuntu:
apt install python3.4-venv
Exec: python3.4 -m venv env
source env/bin/activate
reinstall gunicorn by : pip3 install gunicorn or install from requirements.txt by pip3 install -r requirements.txt
go to terminal and change directory to environment and then type the below command.
pip install gunicorn
#Enjoy1