I am trying to spin up an ec2 instance inside a VPC on a private subnet. Every time I run kitchen test, I am able to spin up the instance with the right security groups and in the right subnet range. When test-kitchen is trying to SSH on to the instance, it is asking for password. However, when I manually try to ssh (ssh <private_ip> -i <path_to_ssh_key> -l ubuntu) on to the machine I succeed without being prompted for a password.
The following is my .kitchen.yml file
---
driver:
name: ec2
aws_ssh_key_id: id-spanning
security_group_ids: ['sg-9....5']
region: us-east-1
availability_zone: us-east-1a
require_chef_omnibus: true
subnet_id: subnet-5...0
associate_public_ip: false
instance_type: m3.medium
interface: private
transport:
ssh_key: ~/.ssh/id-spanning.pem
connection_timeout: 10
connection_retries: 5
username: ubuntu
provisioner:
name: chef_solo
platforms:
- name: Ubuntu-14.04
driver:
image_id: ami-8821cae0
suites:
- name: default
run_list:
attributes:
I have the aws credentials in place on the environment variables. The following is my output.
kitchen test
-----> Starting Kitchen (v1.4.0)
-----> Cleaning up any prior instances of <default-Ubuntu-1404>
-----> Destroying <default-Ubuntu-1404>...
EC2 instance <i-16f468c6> destroyed.
Finished destroying <default-Ubuntu-1404> (0m1.90s).
-----> Testing <default-Ubuntu-1404>
-----> Creating <default-Ubuntu-1404>...
Creating <>...
If you are not using an account that qualifies under the AWS
free-tier, you may be charged to run these suites. The charge
should be minimal, but neither Test Kitchen nor its maintainers
are responsible for your incurred costs.
Instance <i-8fad345f> requested.
EC2 instance <i-8fad345f> created.
Waited 0/300s for instance <i-8fad345f> to become ready.
Waited 5/300s for instance <i-8fad345f> to become ready.
Waited 10/300s for instance <i-8fad345f> to become ready.
Waited 15/300s for instance <i-8fad345f> to become ready.
Waited 20/300s for instance <i-8fad345f> to become ready.
Waited 25/300s for instance <i-8fad345f> to become ready.
EC2 instance <i-8fad345f> ready.
Password:
I tried several times and haven't had any luck on bypassing the password to allow test-kitchen to ssh on to the instance. The following is my kitchen diagnose output.
---
timestamp: 2015-05-26 15:34:29 UTC
kitchen_version: 1.4.0
instances:
default-Ubuntu-1404:
platform:
os_type: unix
shell_type: bourne
state_file:
hostname: ''
server_id: i-1.....6
driver:
associate_public_ip: false
availability_zone: us-east-1a
aws_access_key_id:
aws_secret_access_key:
aws_session_token:
aws_ssh_key_id: id-spanning
block_device_mappings:
ebs_optimized: false
flavor_id:
iam_profile_name:
image_id: ami-8821cae0
instance_type: m3.medium
interface: private
kitchen_root: "/Users/jonnas2/Desktop/apache101"
log_level: :info
name: ec2
price:
private_ip_address:
region: us-east-1
retryable_sleep: 5
retryable_tries: 60
security_group_ids:
- sg-9....5
shared_credentials_profile:
subnet_id: subnet-5....0
tags:
created-by: test-kitchen
test_base_path: "/Users/jonnas2/Desktop/apache101/test/integration"
user_data:
username:
provisioner:
attributes: {}
chef_metadata_url:
chef_omnibus_install_options:
chef_omnibus_root: "/opt/chef"
chef_omnibus_url: https://www.chef.io/chef/install.sh
chef_solo_path: "/opt/chef/bin/chef-solo"
clients_path:
cookbook_files_glob: README.*,metadata {json,rb},attributes/**/*,definitions/**/*,files/**/*,libraries/**/*,providers/**/*,recipes/**/*,resources/**/*,templates/**/*
data_bags_path:
data_path:
encrypted_data_bag_secret_key_path:
environments_path:
http_proxy:
https_proxy:
kitchen_root: "/Users/jonnas2/Desktop/apache101"
log_file:
log_level: :info
name: chef_solo
nodes_path:
require_chef_omnibus: true
roles_path:
root_path: "/tmp/kitchen"
run_list: []
solo_rb: {}
sudo: true
sudo_command: sudo -E
test_base_path: "/Users/jonnas2/Desktop/apache101/test/integration"
transport:
compression: zlib
compression_level: 6
connection_retries: 5
connection_retry_sleep: 1
connection_timeout: 10
keepalive: true
keepalive_interval: 60
kitchen_root: "/Users/jonnas2/Desktop/apache101"
log_level: :info
max_wait_until_ready: 600
name: ssh
port: 22
ssh_key: "/Users/jonnas2/.ssh/id-spanning.pem"
test_base_path: "/Users/jonnas2/Desktop/apache101/test/integration"
username: ubuntu
verifier:
busser_bin: "/tmp/verifier/bin/busser"
http_proxy:
https_proxy:
kitchen_root: "/Users/jonnas2/Desktop/apache101"
log_level: :info
name: busser
root_path: "/tmp/verifier"
ruby_bindir: "/opt/chef/embedded/bin"
sudo: true
sudo_command: sudo -E
suite_name: default
test_base_path: "/Users/jonnas2/Desktop/apache101/test/integration"
version: busser
versions used:
test-kitchen 1.4.0
kitchen-ec2 0.9.0
Any help would be greatly appreciated. Thanks.
This issue was resolved by test-kitchen 1.4.1. A fix was merged (https://github.com/test-kitchen/test-kitchen/pull/704]) into core test-kitchen which disables password auth if an ssh_key is configured.
Related
I am trying to provide initial configuration and software installation to a newly created AWS EC2 instance by using Ansible. If I run my playbooks independently it works just as I want. However, if I try to automate it into a single playbook by using two imports, it doesn't work (probably because the dynamic inventory can't get the newly created IP address?)...
Running together:
[WARNING]: Could not match supplied host pattern, ignoring:
aws_region_eu_central_1
PLAY [variables from dynamic inventory] ****************************************
skipping: no hosts matched
Running separately:
TASK [Gathering Facts] *********************************************************
[WARNING]: Platform linux on host XX.XX.XX.XX is using the discovered Python
interpreter at /usr/bin/python, but future installation of another Python
interpreter could change the meaning of that path. See https://docs.ansible.com
/ansible/2.10/reference_appendices/interpreter_discovery.html for more
information.
ok: [XX.XX.XX.XX]
This is my main playbook:
- import_playbook: server-setup.yml
- import_playbook: server-configuration.yml
server-setup.yml:
---
# variables from dynamic inventory
- name: variables from dynamic inventory
remote_user: ec2-user
hosts: localhost
roles:
- ec2-instance
server-configuration.yml:
---
# variables from dynamic inventory
- name: variables from dynamic inventory
remote_user: ec2-user
become: true
become_method: sudo
become_user: root
ignore_unreachable: true
hosts: aws_region_eu_central_1
gather_facts: false
pre_tasks:
- pause:
minutes: 5
roles:
- { role: epel, sudo: true }
- { role: nodejs, sudo: true }
This is my ansible.cfg file:
[defaults]
inventory = test_aws_ec2.yaml
private_key_file = master-key.pem
enable_plugins = aws_ec2
host_key_checking = False
pipelining = True
log_path = ansible.log
roles_path = /roles
forks = 1000
and finally my hosts.ini:
[local]
localhost ansible_python_interpreter=/usr/local/bin/python3
I have used Ansible to create 1 AWS EC2 instance using the examples in the Ansible ec2 documentation. I can successfully create the instance with a tag. Then I temporarily add it to my local inventory group using add_host.
After doing this, I am having trouble when I try to configure the newly created instance. In my Ansible play, I would like to specify the instance by its tag name. eg. hosts: <tag_name_here>, but I am getting an error.
Here is what I have done so far:
My directory layout is
inventory/
staging/
hosts
group_vars/
all/
all.yml
site.yml
My inventory/staging/hosts file is
[local]
localhost ansible_connection=local ansible_python_interpreter=/home/localuser/ansible_ec2/.venv/bin/python
My inventory/staging/group_vars/all/all.yml file is
---
ami_image: xxxxx
subnet_id: xxxx
region: xxxxx
launched_tag: tag_Name_NginxDemo
Here is my Ansible playbook site.yml
- name: Launch instance
hosts: localhost
gather_facts: no
tasks:
- ec2:
key_name: key-nginx
group: web_sg
instance_type: t2.micro
image: "{{ ami_image }}"
wait: true
region: "{{ region }}"
vpc_subnet_id: "{{ subnet_id }}"
assign_public_ip: yes
instance_tags:
Name: NginxDemo
exact_count: 1
count_tag:
Name: NginxDemo
exact_count: 1
register: ec2
- name: Add EC2 instance to inventory group
add_host:
hostname: "{{ item.public_ip }}"
groupname: tag_Name_NginxDemo
ansible_user: centos_user
ansible_become: yes
with_items: "{{ ec2.instances }}"
- name: Configure EC2 instance in launched group
hosts: tag_Name_NginxDemo
become: True
gather_facts: no
tasks:
- ping:
I run this playbook with
$ cd /home/localuser/ansible_ec2
$ source .venv/bin/activate
$ ansible-playbook -i inventory/staging site.yml -vvv`
and this creates the EC2 instance - the 1st play works correctly. However, the 2nd play gives the following error
TASK [.....] ******************************************************************
The authenticity of host 'xx.xxx.xxx.xx (xx.xxx.xxx.xx)' can't be established.
ECDSA key fingerprint is XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.
Are you sure you want to continue connecting (yes/no)? yes
fatal: [xx.xxx.xxx.xx]: FAILED! => {"changed": false, "module_stderr":
"Shared connection to xx.xxx.xxx.xx closed.\r\n", "module_stdout": "/bin/sh:
1: /usr/bin/python: not found\r\n", "msg": "MODULE FAILURE", "rc": 127}
I followed the instructions from
this SO question to create the task with add_hosts
here to set gather_facts: False, but this still does not allow the play to run correctly.
How can I target the EC2 host using the tag name?
EDIT:
Additional info
This is the only playbook I have run to this point. I see this message requires Python but I cannot install Python on the instance as I cannot connect to it in my play Configure EC2 instance in launched group...if I could make that connection, then I could install Python (if this is the problem). Though, I'm not sure how to connect to the instance.
EDIT 2:
Here is my Python info on the localhost where I am running Ansible
I am running Ansible inside a Python venv.
Here is my python inside the venv
$ python --version
Python 2.7.15rc1
$ which python
~/ansible_ec2/.venv/bin/python
Here are my details about Ansible that I installed inside the Python venv
ansible 2.6.2
config file = /home/localuser/ansible_ec2/ansible.cfg
configured module search path = [u'/home/localuser/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /home/localuser/ansible_ec2/.venv/local/lib/python2.7/site-packages/ansible
executable location = /home/localuser/ansible_ec2/.venv/bin/ansible
python version = 2.7.15rc1 (default, xxxx, xxxx) [GCC 7.3.0]
Ok, so after a lot of searching, I found 1 possible workaround here. Basically, this workaround uses the lineinfile module and adds the new EC2 instance details to the hosts file permanently....not just for the in-memory plays following the add_host task. I followed this suggestion very closely and this approach worked for me. I did not need to use the add_host module.
EDIT:
The line I added in the lineinfile module was
- name: Add EC2 instance to inventory group
- lineinfile: line="{{ item.public_ip }} ansible_python_interpreter=/usr/bin/python3" insertafter=EOF dest=./inventory/staging/hosts
with_items: "{{ ec2.instances }}"
The objective is to spin up multiple instances which can be achieved using count but I have been give specific range of private IP addresses, and want to assign them to the instances.
Below is my present playbook,
---
- name: Provision an EC2 Instance
hosts: local
connection: local
gather_facts: False
tags: provisioning
# Necessary Variables for creating/provisioning the EC2 Instance
vars:
instance_type: t2.micro
security_group: default # Change the security group name here
image: ami-a9d276c9 # Change the AMI, from which you want to launch the server
region: us-west-2 # Change the Region
keypair: ansible # Change the keypair name
ip_addresses:
- 172.31.1.117/32
- 172.31.1.118/32
count: 2
tasks:
- name: Launch the new EC2 Instance
local_action: ec2
group={{ security_group }}
instance_type={{ instance_type}}
image={{ image }}
wait=true
region={{ region }}
keypair={{ keypair }}
count={{count}}
vpc_subnet_id=subnet-xxxxxxx
# private_ip={{private_ip}}
with_items: ip_addresses
register: ec2
- name: Wait for SSH to come up
local_action: wait_for
host={{ item.public_ip }}
port=22
state=started
with_items: ec2.instances
- name: Add tag to Instance(s)
local_action: ec2_tag resource={{ item.id }} region={{ region }} state=present
with_items: ec2.instances
args:
tags:
Name: ansible
- name: Update system
apt: update_cache=yes
- name: Install Git
apt:
name: git
state: present
- name: Install Python2.7
apt:
name: python=2.7
state: present
- name: Install Java
apt:
name: openjdk-8-jdk
state: present
Which is although bringing up the instances but not assigning the IP addresses intended to be assigned. and I'm getting following warning
PLAY [Provision an EC2 Instance] ***********************************************
TASK [Launch the new EC2 Instance] *********************************************
changed: [localhost -> localhost] => (item=172.31.1.117/32)
changed: [localhost -> localhost] => (item=172.31.1.118/32)
[DEPRECATION WARNING]: Skipping task due to undefined attribute, in the future this will be a fatal error.. This feature will be removed in a future release. Deprecation warnings can be
disabled by setting deprecation_warnings=False in ansible.cfg.
Please suggest me the best possible way to achieve this.
You are giving count=2, so 2 instances will be launched
Your IP addresses are wrong, you are giving a CIDR instead of IP
You are not using the IP address anywhere in your code when launching the instances
How to fix?
ip_addresses:
- 172.31.1.117
- 172.31.1.118
Don't specify count in ec2 module
Loop through the list of ipaddresses (there are 2 of them)
Make sure you use the IP by referencing {item}
Like this:
private_ip={{item}}
I followed these 3 guides:
http://docs.ansible.com/ansible/guide_aws.html
http://docs.ansible.com/ansible/ec2_module.html
https://gist.github.com/tristanfisher/e5a306144a637dc739e7
and I wrote this Ansible play
---
- hosts: localhost
connection: local
gather_facts: false
tasks:
- include_vars: aws_credentials.yml
- name: Creating EC2 Ubuntu instance
ec2:
instance_type: t1.micro
image: ami-86e0ffe7
region: us-west-2
key_name: my-aws-key
zone: us-west-2a
vpc_subnet_id: subnet-04199d61
group_id: sg-cf6736aa
assign_public_ip: yes
count: 1
wait: true
volumes:
- device_name: /dev/sda1
volume_type: gp2
volume_size: 10
instance_tags:
Name: ansible-test
Project: test
Ansible: manageable
register: ec2
then I run ansible-playbook create-ec2.yml -v --private-key ~/.ssh/my-key --vault-password-file ~/.password/to_ansible_vault
and I was getting this message
PLAY [localhost] ***************************************************************
TASK [include_vars] ************************************************************
ok: [localhost] => {"ansible_facts": {"ec2_access_key": "decrypted_acces_key_XXXXX", "ec2_secret_key": "decrypted_secret_key_XXXXX"}, "changed": false}
TASK [Creating EC2 Ubuntu instance] ********************************************
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "No handler was ready to authenticate. 1 handlers were checked. ['HmacAuthV4Handler'] Check your credentials"}
NO MORE HOSTS LEFT *************************************************************
[WARNING]: Could not create retry file 'create-ec2.retry'. [Errno 2] No such file or directory: ''
PLAY RECAP *********************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=1
when I ran ansible-vault view aws_credentials.yml --vault-password-file ~/.password/to_ansible_vault I got readable content of encrypted aws_credentials.yml,
something like this :
---
ec2_access_key: "XXXXX"
ec2_secret_key: "XXXXX"
Also when I used plain aws_credentials.yml, it doesn't work. Only when I export my credentials, it works without any failure.
Could somebody help me, how can I write playbook for creating ec2 instance with credentials stored in encrypted file?
I think you should supply your keys directly to ec2 module in this case.
Try this:
- name: Creating EC2 Ubuntu instance
ec2:
aws_access_key: "{{ ec2_access_key }}"
aws_secret_key: "{{ ec2_secret_key }}"
instance_type: t1.micro
image: ami-86e0ffe7
region: us-west-2
...
The code suggests that it only checks module's arguments and environment variables, not host variables.
Also you can export your AWS API keys to OS environment variables, like a:
export AWS_ACCESS_KEY=XXXXXXX
In that case in Ansible scenario you need to set:
- name: Creating EC2 Ubuntu instance
ec2:
aws_access_key: "{{ lookup('env', 'AWS_ACCESS_KEY') }}"
aws_secret_key: "{{ lookup('env', 'AWS_SECRET_KEY') }}"
instance_type: t1.micro
image: ami-86e0ffe7
region: us-west-2
I am attempting to install Apache on an EC2 instance through Ansible. My playbook looks like this:
# Configure and deploy Apache
- hosts: localhost
connection: local
remote_user: ec2-user
gather_facts: false
roles:
- ec2_apache
- apache
The 'ec2_apache' role provisions an EC2 instance and the first task within the apache/main.yml looks like this:
- name: confirm using the latest Apache server
become: yes
become_method: sudo
yum:
name: httpd
state: latest
However, I am getting the following error:
"module_stderr": "sudo: a password is required\n"
I did take a look at: How to switch a user per task or set of tasks? but it did not seem to resolve my problem.
Because the configuration of the Ec2 instance is in one role and the installation of Apache is in another, did I hork up the security in some way?
The issue you've got is that your playbook that runs both roles is targeting localhost so your Apache role is trying to run sudo yum install httpd locally rather than on the target EC2 instance.
As the ec2 module docs example shows you need to use the add_host module to add your new EC2 instance(s) to a group that you can then target with a further play.
So your playbook might look something like this:
# Configure and deploy Apache
- name: provision instance for Apache
hosts: localhost
connection: local
remote_user: ec2-user
gather_facts: false
roles:
- ec2_apache
- name: install Apache
hosts: launched
remote_user: ec2-user
roles:
- apache
And then, as per the example in the ec2 module docs, just do something like this in your ec2_apache role:
- name: Launch instance
ec2:
key_name: "{{ keypair }}"
group: "{{ security_group }}"
instance_type: "{{ instance_type }}"
image: "{{ image }}"
wait: true
region: "{{ region }}"
vpc_subnet_id: subnet-29e63245
assign_public_ip: yes
register: ec2
- name: Add new instance to host group
add_host: hostname={{ item.public_ip }} groupname=launched
with_items: ec2.instances
- name: Wait for SSH to come up
wait_for: host={{ item.public_dns_name }} port=22 delay=60 timeout=320 state=started
with_items: ec2.instances
As an aside you can see quickly that your ec2_apache role is actually pretty generic and you could turn this into a generic ec2_provision role that all sorts of other things could use, helping you re-use your code.
This is what I did to install apache.
Based on #ydaetskcoR suggestion, all I added was connection: local to fix the following problems.
Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive).", "unreachable": true}
See code below
---
- name: Install Apache and other packages
hosts: localhost
become: yes
connection: local
gather_facts: false
tasks:
- name: Install a list of packages with a list variable
yum:
name: "{{ packages }}"
state: latest
vars:
packages:
- httpd
- httpd-tools
- nginx
register: result
you also have to run your code as follows: -K stands for --ask-become-pass
ansible-playbook -i hosts.ini startapache.yml -K -vvv
Are you sure you are using ansible correctly and are you provindig a password for sudo on the remote host?
Just use --ask-become-pass when you execute the playbook. You should be prompted for the password.