My Codebuild project that it creates AMI by packer by ansible provisioner.
This packer settings success in my local environment and Amazon linux2 ec2 environment. However, when I use AWS Codebuild with aws/codebuild/amazonlinux2-x86_64-standard:1.0 image and it fails.
I already tried this settings remote_tmp = /tmp or remote_tmp = /tmp/.ansible-${USER}/tmp but did not work.
Authentication or permission failure, did not have permissions on the remote directory
version: 0.2
phases:
install:
runtime-versions:
python: 3.7
pre_build:
commands:
- python --version
- pip --version
- curl -qL -o packer.zip https://releases.hashicorp.com/packer/1.4.3/packer_1.4.3_linux_amd64.zip && unzip packer.zip
- ./packer version
- pip install --user ansible==2.8.5
- ansible --version
- echo 'Validate packer json'
- ./packer validate packer.json
build:
commands:
- ./packer build -color=false packer.json | tee build.log
{
"builders": [{
"type": "amazon-ebs",
"region": "ap-northeast-1",
"ami_regions": "ap-northeast-1",
"source_ami": "ami-0ff21806645c5e492",
"instance_type": "t2.micro",
"ssh_username": "ec2-user",
"ami_name": "packer-quick-start {{timestamp}}",
"ami_description": "created by packer at {{timestamp}}",
"ebs_optimized": false,
"tags": {
"OS_Version": "Amazon Linux AMI 2018.03",
"timestamp": "{{timestamp}}",
"isotime": "{{isotime \"2006-01-02 03:04:05\"}}"
},
"disable_stop_instance": false
}],
"provisioners": [
{
"type" : "ansible",
"extra_arguments": [
"-vvv"
],
"playbook_file" : "ansible/main.yaml"
}
]
}
==> amazon-ebs: Prevalidating AMI Name: packer-quick-start 1569943272
amazon-ebs: Found Image ID: ami-0ff21806645c5e492
==> amazon-ebs: Creating temporary keypair: packer_5d936ee8-541f-5c9a-6955-9672526afc1a
==> amazon-ebs: Creating temporary security group for this instance: packer_5d936ef1-6546-d9d0-60ff-2dc4c011036f
==> amazon-ebs: Authorizing access to port 22 from [0.0.0.0/0] in the temporary security groups...
==> amazon-ebs: Launching a source AWS instance...
==> amazon-ebs: Adding tags to source instance
amazon-ebs: Adding tag: "Name": "Packer Builder"
amazon-ebs: Instance ID: i-04b00db56a8b3b6d0
==> amazon-ebs: Waiting for instance (i-04b00db56a8b3b6d0) to become ready...
==> amazon-ebs: Using ssh communicator to connect: 3.112.61.8
==> amazon-ebs: Waiting for SSH to become available...
==> amazon-ebs: Connected to SSH!
==> amazon-ebs: Provisioning with Ansible...
==> amazon-ebs: Executing Ansible: ansible-playbook --extra-vars packer_build_name=amazon-ebs packer_builder_type=amazon-ebs -o IdentitiesOnly=yes -i /tmp/packer-provisioner-ansible244097143 /codebuild/output/src965785042/src/github.com/repoUsername/reponame/ansible/main.yaml -e ansible_ssh_private_key_file=/tmp/ansible-key242793848 -vvv
amazon-ebs: ansible-playbook 2.8.5
amazon-ebs: config file = /codebuild/output/src965785042/src/github.com/repoUsername/reponame/ansible.cfg
amazon-ebs: configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
amazon-ebs: ansible python module location = /root/.local/lib/python3.7/site-packages/ansible
amazon-ebs: executable location = /root/.local/bin/ansible-playbook
amazon-ebs: python version = 3.7.4 (default, Sep 20 2019, 22:55:10) [GCC 7.3.1 20180303 (Red Hat 7.3.1-5)]
amazon-ebs: Using /codebuild/output/src965785042/src/github.com/repoUsername/reponame/ansible.cfg as config file
amazon-ebs: host_list declined parsing /tmp/packer-provisioner-ansible244097143 as it did not pass it's verify_file() method
amazon-ebs: script declined parsing /tmp/packer-provisioner-ansible244097143 as it did not pass it's verify_file() method
amazon-ebs: auto declined parsing /tmp/packer-provisioner-ansible244097143 as it did not pass it's verify_file() method
amazon-ebs: Parsed /tmp/packer-provisioner-ansible244097143 inventory source with ini plugin
amazon-ebs:
amazon-ebs: PLAYBOOK: main.yaml ************************************************************
amazon-ebs: 1 plays in /codebuild/output/src965785042/src/github.com/repoUsername/reponame/ansible/main.yaml
amazon-ebs:
amazon-ebs: PLAY [all] *********************************************************************
amazon-ebs: META: ran handlers
amazon-ebs:
amazon-ebs: TASK [be sure httpd is installed] **********************************************
amazon-ebs: task path: /codebuild/output/src965785042/src/github.com/repoUsername/reponame/ansible/main.yaml:6
amazon-ebs: <127.0.0.1> ESTABLISH SSH CONNECTION FOR USER: root
amazon-ebs: <127.0.0.1> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o Port=35595 -o 'IdentityFile="/tmp/ansible-key242793848"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/root/.ansible/cp/02aaab1733 127.0.0.1 '/bin/sh -c '"'"'echo ~root && sleep 0'"'"''
amazon-ebs: <127.0.0.1> (0, b'/root\n', b"Warning: Permanently added '[127.0.0.1]:35595' (RSA) to the list of known hosts.\r\n")
amazon-ebs: <127.0.0.1> ESTABLISH SSH CONNECTION FOR USER: root
amazon-ebs: <127.0.0.1> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o Port=35595 -o 'IdentityFile="/tmp/ansible-key242793848"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/root/.ansible/cp/02aaab1733 127.0.0.1 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /root/.ansible/tmp/ansible-tmp-1569943320.4544108-49329379039882 `" && echo ansible-tmp-1569943320.4544108-49329379039882="` echo /root/.ansible/tmp/ansible-tmp-1569943320.4544108-49329379039882 `" ) && sleep 0'"'"''
amazon-ebs: <127.0.0.1> (1, b'', b'mkdir: cannot create directory \xe2\x80\x98/root\xe2\x80\x99: Permission denied\n')
amazon-ebs: <127.0.0.1> Failed to connect to the host via ssh: mkdir: cannot create directory ‘/root’: Permission denied
amazon-ebs: fatal: [default]: UNREACHABLE! => {
amazon-ebs: "changed": false,
amazon-ebs: "msg": "Authentication or permission failure. In some cases, you may have been able to authenticate and did not have permissions on the target directory. Consider changing the remote tmp path in ansible.cfg to a path rooted in \"/tmp\". Failed command was: ( umask 77 && mkdir -p \"` echo /root/.ansible/tmp/ansible-tmp-1569943320.4544108-49329379039882 `\" && echo ansible-tmp-1569943320.4544108-49329379039882=\"` echo /root/.ansible/tmp/ansible-tmp-1569943320.4544108-49329379039882 `\" ), exited with result 1",
amazon-ebs: "unreachable": true
amazon-ebs: }
amazon-ebs:
amazon-ebs: PLAY RECAP *********************************************************************
amazon-ebs: default : ok=0 changed=0 unreachable=1 failed=0 skipped=0 rescued=0 ignored=0
amazon-ebs:
==> amazon-ebs: Terminating the source AWS instance...
==> amazon-ebs: Cleaning up any extra volumes...
==> amazon-ebs: No volumes to clean up, skipping
==> amazon-ebs: Deleting temporary security group...
==> amazon-ebs: Deleting temporary keypair...
I know it fails because it tried to mkdir /root and Permission denied.
But don't know why it tried to mkdir /root. How can I change this behavior?
I solved and it was super simple cause.
Because AWS Codebuild builds by the root user, ansible makes a connection by the root user. I just wrote like this and solved it.
"provisioners": [
{
"type" : "ansible",
"user": "ec2-user",
"playbook_file" : "ansible/main.yaml"
}
]
My ansible file is simple for testing.
---
- hosts: all
become: yes
gather_facts: no
tasks:
- name: be sure httpd is installed
yum: name=httpd state=installed
- name: be sure httpd is running and enabled
service: name=httpd state=started enabled=yes
Related
I am trying to deploy dockerized react web app to EC2 but I am still getting an error when configuring the instance. Already search but did not find anything.
Deploying using command:
ansible-playbook -vvvvv ansible/ec2_deploy.yml --user ubuntu
Docker which I am running ansible in:
FROM node:10.23.0-alpine3.9
COPY . .
ENV PYTHONUNBUFFERED=1
RUN apk add --update --no-cache python3 && ln -sf python3 /usr/bin/python
RUN python3 -m ensurepip
RUN pip3 install --no-cache --upgrade pip setuptools
RUN apk add --update ansible
RUN apk add
RUN pip install boto
RUN chmod 777 get_vault_pass.sh
ENTRYPOINT [ "/bin/sh" ]
Ansible deployment:
- name: Deploy to EC2
hosts: localhost
connection: local
tasks:
- name: Launch EC2 instance
ec2:
instance_type: t2.micro
image: ami-0885b1f6bd170450c
region: us-east-1
key_name: eshop-key-pair
vpc_subnet_id: subnet-cafc34fb
assign_public_ip: yes
wait: yes
count: 1
group: eshop
aws_access_key: 'key'
aws_secret_key: 'key2'
security_token: 'token'
register: ec2
- name: Add instance host to group
add_host: hostname={{ item.public_dns_name }} groupname=launched
with_items: '{{ec2.instances}}'
- name: Wait for SSH connection
wait_for: host={{ item.public_dns_name }} port=22 delay=60 timeout=600 state=started
with_items: '{{ec2.instances}}'
- name: Configure EC2
hosts: launched
connection: ssh
tasks:
- name: Install docker
apt:
name: docker.io
state: present
update_cache: yes
become: yes
- service:
name: docker
state: started
enabled: yes
become: yes
- name: Get project files from GIT
git:
repo: 'https://github.com/romanzdk/4IT572_ZS_2020_circleci.git'
dest: ./app
- name: Build docker with eshop
shell: cd app && docker build -t myeshop:latest .
become: yes
- name: Run docker with eshop
shell: docker run -p 80:3000 myeshop
async: 90
poll: 15
become: yes
- wait_for: delay=60 timeout=600
port: 80
Stack trace:
PLAY [Configure EC2] ***********************************************************************************************************************
TASK [Gathering Facts] *********************************************************************************************************************
task path: /ansible/ec2_deploy.yml:30
<ec2-100-25-28-7.compute-1.amazonaws.com> ESTABLISH SSH CONNECTION FOR USER: None
<ec2-100-25-28-7.compute-1.amazonaws.com> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<ec2-100-25-28-7.compute-1.amazonaws.com> SSH: ansible_password/ansible_ssh_pass not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<ec2-100-25-28-7.compute-1.amazonaws.com> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<ec2-100-25-28-7.compute-1.amazonaws.com> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/root/.ansible/cp/aaee2dc684)
<ec2-100-25-28-7.compute-1.amazonaws.com> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/root/.ansible/cp/aaee2dc684 ec2-100-25-28-7.compute-1.amazonaws.com '/bin/sh -c '"'"'echo ~ && sleep 0'"'"''
The full traceback is:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/ansible/executor/task_executor.py", line 140, in run
res = self._execute()
File "/usr/lib/python3.6/site-packages/ansible/executor/task_executor.py", line 612, in _execute
result = self._handler.run(task_vars=variables)
File "/usr/lib/python3.6/site-packages/ansible/plugins/action/normal.py", line 46, in run
result = merge_hash(result, self._execute_module(task_vars=task_vars, wrap_async=wrap_async))
File "/usr/lib/python3.6/site-packages/ansible/plugins/action/__init__.py", line 745, in _execute_module
self._make_tmp_path()
File "/usr/lib/python3.6/site-packages/ansible/plugins/action/__init__.py", line 294, in _make_tmp_path
tmpdir = self._remote_expand_user(tmpdir, sudoable=False)
File "/usr/lib/python3.6/site-packages/ansible/plugins/action/__init__.py", line 613, in _remote_expand_user
data = self._low_level_execute_command(cmd, sudoable=False)
File "/usr/lib/python3.6/site-packages/ansible/plugins/action/__init__.py", line 980, in _low_level_execute_command
rc, stdout, stderr = self._connection.exec_command(cmd, in_data=in_data, sudoable=sudoable)
File "/usr/lib/python3.6/site-packages/ansible/plugins/connection/ssh.py", line 1145, in exec_command
(returncode, stdout, stderr) = self._run(cmd, in_data, sudoable=sudoable)
File "/usr/lib/python3.6/site-packages/ansible/plugins/connection/ssh.py", line 392, in wrapped
return_tuple = func(self, *args, **kwargs)
File "/usr/lib/python3.6/site-packages/ansible/plugins/connection/ssh.py", line 1035, in _run
return self._bare_run(cmd, in_data, sudoable=sudoable, checkrc=checkrc)
File "/usr/lib/python3.6/site-packages/ansible/plugins/connection/ssh.py", line 790, in _bare_run
p = subprocess.Popen(cmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
File "/usr/lib/python3.6/subprocess.py", line 729, in __init__
restore_signals, start_new_session)
File "/usr/lib/python3.6/subprocess.py", line 1364, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: b'ssh': b'ssh'
fatal: [ec2-52-73-248-179.compute-1.amazonaws.com]: FAILED! => {
"msg": "Unexpected failure during module execution.",
"stdout": ""
}
Any idea what is wrong? I already spent an ages on this...
here is some more text as I am asked to add more details because of the long code, lol
chepner's comment is spot on - your docker image doesn't have ssh installed. Try
apk add openssh-client
and the error should be solved.
I want to make AMI file from packer and ansible.
I have tried many configuration, but I have still a problem of connection to the instance.
Here is my packer conf:
{
"variables": {
"aws_access_key": "{{env `AWS_ACCESS_KEY_ID`}}",
"aws_secret_key": "{{env `AWS_SECRET_ACCESS_KEY`}}",
"region": "us-east-1"
},
"builders": [
{
"type": "amazon-ebs",
"access_key": "{{ user `aws_access_key` }}",
"secret_key": "{{ user `aws_secret_key` }}",
"region": "{{ user `region` }}",
"instance_type": "t2.micro",
"source_ami_filter": {
"filters": {
"virtualization-type": "hvm",
"name": "*Windows_Server-2012-R2*English-64Bit-Base*",
"root-device-type": "ebs"
},
"most_recent": true,
"owners": "amazon"
},
"ami_name": "packer-demo-{{timestamp}}",
"user_data_file": "userdata/windows-aws.txt",
"communicator": "winrm",
"winrm_username": "Administrator"
}],
"provisioners": [{
"type": "powershell",
"inline": [
"dir c:\\"
]
},
{
"type": "ansible",
"playbook_file": "./win-playbook.yml",
"extra_arguments": [
"--connection", "packer", "-vvv",
"--extra-vars", "ansible_shell_type=powershell ansible_shell_executable=None"
]
}]
}
The User data script is activating winrm on the AWS instance:
<powershell>
winrm quickconfig -q
winrm set winrm/config/winrs '#{MaxMemoryPerShellMB="300"}'
winrm set winrm/config '#{MaxTimeoutms="1800000"}'
winrm set winrm/config/service '#{AllowUnencrypted="true"}'
winrm set winrm/config/service/auth '#{Basic="true"}'
netsh advfirewall firewall add rule name="WinRM 5985" protocol=TCP dir=in localport=5985 action=allow
netsh advfirewall firewall add rule name="WinRM 5986" protocol=TCP dir=in localport=5986 action=allow
net stop winrm
sc config winrm start=auto
net start winrm
Set-ExecutionPolicy -ExecutionPolicy Bypass -Scope LocalMachine
</powershell>
Here is win-playbook.yml file:
---
- hosts: all
tasks:
- win_ping:
I do have the packer.py installed in the ~/.ansible/plugins/connection_plugins/ directory and configured in ~/.ansible.cfg:
root#ip-172-31-30-11:~/demo# grep connection_plugins /etc/ansible/ansible.cfg
connection_plugins = /root/.ansible/plugins/connection_plugins
root#ip-172-31-30-11:~/demo# ll /root/.ansible/plugins/connection_plugins
total 16
drwx------ 2 root root 4096 May 2 16:58 ./
drwx------ 4 root root 4096 May 2 17:11 ../
-rwx--x--x 1 root root 511 May 2 16:53 packer.py*
and then this is output error:
==> amazon-ebs: Provisioning with Ansible...
==> amazon-ebs: Executing Ansible: ansible-playbook --extra-vars packer_build_name=amazon-ebs packer_builder_type=amazon-ebs -i /tmp/packer-provisioner-ansible962278842 /root/demo/win-playbook.yml -e ansible_ssh_private_key_file=/tmp/ansible-key842946567 --connection packer -vvv --extra-vars ansible_shell_type=powershell ansible_shell_executable=None
amazon-ebs: ansible-playbook 2.5.2
amazon-ebs: config file = /etc/ansible/ansible.cfg
amazon-ebs: configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
amazon-ebs: ansible python module location = /usr/lib/python2.7/dist-packages/ansible
amazon-ebs: executable location = /usr/bin/ansible-playbook
amazon-ebs: python version = 2.7.12 (default, Dec 4 2017, 14:50:18) [GCC 5.4.0 20160609]
amazon-ebs: Using /etc/ansible/ansible.cfg as config file
amazon-ebs: Parsed /tmp/packer-provisioner-ansible962278842 inventory source with ini plugin
amazon-ebs:
amazon-ebs: PLAYBOOK: win-playbook.yml *****************************************************
amazon-ebs: 1 plays in /root/demo/win-playbook.yml
amazon-ebs:
amazon-ebs: PLAY [all] *********************************************************************
amazon-ebs:
amazon-ebs: TASK [Gathering Facts] *********************************************************
amazon-ebs: task path: /root/demo/win-playbook.yml:2
amazon-ebs: Using module file /usr/lib/python2.7/dist-packages/ansible/modules/windows/setup.ps1
amazon-ebs: <127.0.0.1> ESTABLISH SSH CONNECTION FOR USER: root
amazon-ebs: The full traceback is:
amazon-ebs: Traceback (most recent call last):
amazon-ebs: File "/usr/lib/python2.7/dist-packages/ansible/executor/task_executor.py", line 138, in run
amazon-ebs: res = self._execute()
amazon-ebs: File "/usr/lib/python2.7/dist-packages/ansible/executor/task_executor.py", line 558, in _execute
amazon-ebs: result = self._handler.run(task_vars=variables)
amazon-ebs: File "/usr/lib/python2.7/dist-packages/ansible/plugins/action/normal.py", line 46, in run
amazon-ebs: result = merge_hash(result, self._execute_module(task_vars=task_vars, wrap_async=wrap_async))
amazon-ebs: File "/usr/lib/python2.7/dist-packages/ansible/plugins/action/__init__.py", line 705, in _execute_module
amazon-ebs: self._make_tmp_path()
amazon-ebs: File "/usr/lib/python2.7/dist-packages/ansible/plugins/action/__init__.py", line 251, in _make_tmp_path
amazon-ebs: result = self._low_level_execute_command(cmd, sudoable=False)
amazon-ebs: File "/usr/lib/python2.7/dist-packages/ansible/plugins/action/__init__.py", line 902, in _low_level_execute_command
amazon-ebs: rc, stdout, stderr = self._connection.exec_command(cmd, in_data=in_data, sudoable=sudoable)
amazon-ebs: File "/usr/lib/python2.7/dist-packages/ansible/plugins/connection/ssh.py", line 976, in exec_command
amazon-ebs: use_tty = self.get_option('use_tty')
amazon-ebs: File "/usr/lib/python2.7/dist-packages/ansible/plugins/__init__.py", line 58, in get_option
amazon-ebs: option_value = C.config.get_config_value(option, plugin_type=get_plugin_class(self), plugin_name=self._load_name, variables=hostvars)
amazon-ebs: File "/usr/lib/python2.7/dist-packages/ansible/config/manager.py", line 284, in get_config_value
amazon-ebs: value, _drop = self.get_config_value_and_origin(config, cfile=cfile, plugin_type=plugin_type, plugin_name=plugin_name, keys=keys, variables=variables)
amazon-ebs: File "/usr/lib/python2.7/dist-packages/ansible/config/manager.py", line 304, in get_config_value_and_origin
amazon-ebs: defs = self._plugins[plugin_type][plugin_name]
amazon-ebs: KeyError: 'connection'
amazon-ebs: fatal: [default]: FAILED! => {
amazon-ebs: "msg": "Unexpected failure during module execution.",
amazon-ebs: "stdout": ""
amazon-ebs: }
amazon-ebs: to retry, use: --limit #/root/demo/win-playbook.retry
amazon-ebs:
amazon-ebs: PLAY RECAP *********************************************************************
amazon-ebs: default : ok=0 changed=0 unreachable=0 failed=1
packer version: 1.2.3
ansible version: 2.5.2
It looks like this issue is common for Ansible 2.5.x and Packer. Adarobin commented on the packer issue https://github.com/hashicorp/packer/issues/5845. We ran into the same issue, tested the solution and it worked for us.
I was hitting the KeyError: 'connection' issue with Ansible 2.5 on
Packer 1.2.2 with the AWS builder and I think I have discovered the
issue. It looks like Ansible now requires plugins to have a
documentation string. I copied the documentation string from the SSH
connection plugin (since that is what the packer plugin is based on)
made a few changes and my packer.py now looks like this.
https://gist.github.com/adarobin/2f02b8b993936233e15d76f6cddb9e00
I have set up a list of ec2 instances to which I can log into using a ssh.cfg file that performs proxying & agent forwarding via the publicly accessible NAT instance as follows:
ssh -F ssh.cfg admin#private_instance_ip
ssh.cfg has the following structure:
Host 10.40.*
ProxyCommand ssh -W %h:%p ec2-user#nat_instance_ip -o StrictHostKeyChecking=no
Host *
ControlMaster auto
ControlPath ~/.ssh/mux-%r#%h:%p
ControlPersist 60m
ForwardAgent yes
My network is of course 10.40.*.
I want to test ansible connectivity to such hosts (defined in inventoryfile) as follows:
ansible -i inventoryfile -m ping all
Is there a way to do this?
edit: an example of how the ansible ping fails:
ansible -i inventoryfile -m ping 10.40.187.22
10.40.187.22 | UNREACHABLE! => {
"changed": false,
"msg": "Failed to connect to the host via ssh: Permission denied (publickey).\r\n",
"unreachable": true
}
However:
ssh -F ssh.cfg admin#10.40.187.22
Last login: Fri Oct 27 07:36:05 2017 from private_ip_of_NAT
admin#ip-10.40.187.22:~$ exit
Here is the ansible.cfg
[defaults]
nocows = 1
callback_whitelist = profile_tasks
host_key_checking = False
retry_files_enabled = False
gathering = explicit
forks=50
vault_password_file = .vault
[ssh_connection]
ssh_args = -F ssh.cfg
pipelining = True
edit2: when specifying user in the ping command, it still gives me an error:
ansible -i inventoryfile -m ping 10.40.187.22 -u admin
10.40.187.22 | UNREACHABLE! => {
"changed": false,
"msg": "Failed to connect to the host via ssh: Permission denied (publickey).\r\n",
"unreachable": true
}
It turns out that private IPs won't work in ansible ping when the later is proxied via e.g. a public nat instance.
The inventory file is dynamic and based on aws tags, i.e.
[ec2_hosts:children]
tag_Name_my_srv1
tag_Name_my_srv2
so the following command succeeds:
ansible -i inventoryfile -m ping tag_Name_my_srv1 -u admin
10.40.187.22 | SUCCESS => {
"changed": false,
"failed": false,
"ping": "pong"
}
I have a launch script (user data) that runs on startup in aws with an ubuntu 16.04 image, and the issue I'm having is that when it gets to the part where it runs an ansible playbook the playbook fails saying this basic error message Could not get lock /var/lib/dpkg/lock. Now when I log in and try to run the ansible script manually it works, but if I run it from the aws user data, it fails with the error.
This is the full error
TASK [rabbitmq : install packages (Ubuntu default repo is used)] ***************
task path: /etc/ansible/roles/rabbitmq/tasks/main.yml:50
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: root
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo $HOME/.ansible/tmp/ansible-tmp-1480352390.01-116502531862586 `" && echo ansible-tmp-1480352390.01-116502531862586="` echo $HOME/.ansible/tmp/ansible-tmp-1480352390.01-116502531862586 `" ) && sleep 0'
<localhost> PUT /tmp/tmpGHaVRP TO /.ansible/tmp/ansible-tmp-1480352390.01-116502531862586/apt
<localhost> EXEC /bin/sh -c 'chmod u+x /.ansible/tmp/ansible-tmp-1480352390.01-116502531862586/ /.ansible/tmp/ansible-tmp-1480352390.01-116502531862586/apt && sleep 0'
<localhost> EXEC /bin/sh -c 'LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 /usr/bin/python /.ansible/tmp/ansible-tmp-1480352390.01-116502531862586/apt; rm -rf "/.ansible/tmp/ansible-tmp-1480352390.01-116502531862586/" > /dev/null 2>&1 && sleep 0'
fatal: [localhost]: FAILED! => {"cache_update_time": 0, "cache_updated":
false, "changed": false, "failed": true, "invocation": {"module_args":
{"allow_unauthenticated": false, "autoremove": false, "cache_valid_time":
null, "deb": null, "default_release": null, "dpkg_options": "force-
confdef,force-confold", "force": false, "install_recommends": null, "name":
"rabbitmq-server", "only_upgrade": false, "package": ["rabbitmq-server"],
"purge": false, "state": "present", "update_cache": false, "upgrade": null},
"module_name": "apt"}, "msg": "'/usr/bin/apt-get -y -o \"Dpkg::Options::=--
force-confdef\" -o \"Dpkg::Options::=--force-confold\" install
'rabbitmq-server'' failed: E: Could not get lock /var/lib/dpkg/lock - open
(11: Resource temporarily unavailable)\nE: Unable to lock the administration
directory (/var/lib/dpkg/), is another process using it?\n", "stderr": "E: Could
not get lock /var/lib/dpkg/lock - open (11: Resource temporarily
unavailable)\nE: Unable to lock the administration directory (/var/lib/dpkg/),
is another process using it?\n", "stdout": "", "stdout_lines": []}
I ran into the same lock issue. I found that ubuntu was installing some packages on first boot which cloud-init did not wait for.
I use the following script to check that the lock file is available for at least 15 seconds prior to trying to install anything.
#!/bin/bash
i="0"
while [ $i -lt 15 ]
do
if [ $(fuser /var/lib/dpkg/lock) ]; then
i="0"
fi
sleep 1
i=$[$i+1]
done
The reason I prefer this vs sleep 5m because in an autoscale group the instance may be removed before it's even provisioned.
I'm new to Ansible, Ansible Tower, and AWS Cloud Formation and am trying to have Ansible Tower deploy an EC2 Container Service using a Cloud Formation template. I try to run the deploy job and am running into this error below.
TASK [create/update stack] *****************************************************
task path: /var/lib/awx/projects/_6__api/tasks/create_stack.yml:2
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: awx
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo $HOME/.ansible/tmp/ansible-tmp-1470427494.79-207756006727790 `" && echo ansible-tmp-1470427494.79-207756006727790="` echo $HOME/.ansible/tmp/ansible-tmp-1470427494.79-207756006727790 `" ) && sleep 0'
<127.0.0.1> PUT /tmp/tmpgAsKKv TO /var/lib/awx/.ansible/tmp/ansible-tmp-1470427494.79-207756006727790/cloudformation
<127.0.0.1> EXEC /bin/sh -c 'sudo -H -S -n -u root /bin/sh -c '"'"'echo BECOME-SUCCESS-coqlkeqywlqhagfixtfpfotjgknremaw; LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 AWS_DEFAULT_REGION=us-west-2 LC_MESSAGES=en_US.UTF-8 /usr/bin/python /var/lib/awx/.ansible/tmp/ansible-tmp-1470427494.79-207756006727790/cloudformation; rm -rf "/var/lib/awx/.ansible/tmp/ansible-tmp-1470427494.79-207756006727790/" > /dev/null 2>&1'"'"' && sleep 0'
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "invocation": {"module_name": "cloudformation"}, "module_stderr": "/bin/sh: /usr/bin/sudo: Permission denied\n", "module_stdout": "", "msg": "MODULE FAILURE", "parsed": false}
This is the create/update task:
---
- name: create/update stack
cloudformation:
stack_name: my-stack
state: present
template: templates/stack.yml
template_format: yaml
template_parameters:
VpcId: "{{ vpc_id }}"
SubnetId: "{{ subnet_id }}"
KeyPair: "{{ ec2_keypair }}"
DbUsername: "{{ db_username }}"
DbPassword: "{{ db_password }}"
InstanceCount: "{{ instance_count | default(1) }}"
tags:
Environment: test
register: cf_stack
- debug: msg={{ cf_stack }}
when: debug is defined
The playbook that Ansible Tower executes is a site.yml file:
---
- name: Deployment Playbook
hosts: localhost
connection: local
gather_facts: no
environment:
AWS_DEFAULT_REGION: "{{ lookup('env', 'AWS_DEFAULT_REGION') | default('us-west-2', true) }}"
tasks:
- include: tasks/create_stack.yml
- include: tasks/deploy_app.yml
This is what my playbook folder structure looks like:
/deploy
/group_vars
all
/library
aws_ecs_service.py
aws_ecs_task.py
aws_ecs_taskdefinition.py
/tasks
stack.yml
/templates
site.yml
I'm basing everything really on Justin Menga's pluralsight course "Continuous Delivery using Docker and Ansible", but he uses Jenkins, not Ansible Tower, which is probably why the disconnect. Anyway, hopefully that is enough information, let me know if I should also provide the stack.yml file. The files under the library directory are Menga's customized modules from his video course.
Thanks for reading all this and for any potential help! This is a link to his deploy playbook repository that I closely modeled everything after, https://github.com/jmenga/todobackend-deploy. Things that I took out are the DB RDS stuff.
If you look at the two last lines of the error message you can see that it is attempting to escalate privileges but failing:
<127.0.0.1> EXEC /bin/sh -c 'sudo -H -S -n -u root /bin/sh -c '"'"'echo BECOME-SUCCESS-coqlkeqywlqhagfixtfpfotjgknremaw; LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 AWS_DEFAULT_REGION=us-west-2 LC_MESSAGES=en_US.UTF-8 /usr/bin/python /var/lib/awx/.ansible/tmp/ansible-tmp-1470427494.79-207756006727790/cloudformation; rm -rf "/var/lib/awx/.ansible/tmp/ansible-tmp-1470427494.79-207756006727790/" > /dev/null 2>&1'"'"' && sleep 0'
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "invocation": {"module_name": "cloudformation"}, "module_stderr": "/bin/sh: /usr/bin/sudo: Permission denied\n", "module_stdout": "", "msg": "MODULE FAILURE", "parsed": false}
As this is a local task it is attempting to switch to the root user on the box that Ansible Tower is running on and the user presumably (and for good reason) doesn't have the privileges to do this.
With normal Ansible you can avoid this by not specifying the --become or -b flags on the command line or by specifying become: false in the task/play definition.
As you pointed out in the comments, with Ansible Tower it's a case of unticking the "Enable Privilege Escalation" option in the job template.