I'm trying to create an small webapp infrastructure with ansible on Amazon AWS and I want to do all the process: launch instance, configure services, etc. but I can't find a proper tool or module to deal with that from ansible. Mainly EC2 Launch.
Thanks a lot.
This is the short answer of your question, if you want detail and fully automated role, please let me know. Thanks
Prerequisite:
Ansible
Python boto library
Set up the AWS access and secret keys in the environment settings
(best is inside the ~./boto)
To Create the EC2 Instance(s):
In order to create the EC2 Instance, please modified these parameters that you can find inside the "ec2_launch.yml" file under "vars":
region # where is want to launch the instance(s), USA, Australia, Ireland etc
count # Number of instance(s), you want to create
Once, you have mentioned these parameter, please run the following command:
ansible-playbook -i hosts ec2_launch.yml
Contents of hosts file:
[local]
localhost
[webserver]
Contents of ec2_launch.yml file:
---
- name: Provision an EC2 Instance
hosts: local
connection: local
gather_facts: False
tags: provisioning
# Necessary Variables for creating/provisioning the EC2 Instance
vars:
instance_type: t1.micro
security_group: webserver # Change the security group name here
image: ami-98aa1cf0 # Change the AMI, from which you want to launch the server
region: us-east-1 # Change the Region
keypair: ansible # Change the keypair name
count: 1
# Task that will be used to Launch/Create an EC2 Instance
tasks:
- name: Create a security group
local_action:
module: ec2_group
name: "{{ security_group }}"
description: Security Group for webserver Servers
region: "{{ region }}"
rules:
- proto: tcp
type: ssh
from_port: 22
to_port: 22
cidr_ip: 0.0.0.0/0
- proto: tcp
from_port: 80
to_port: 80
cidr_ip: 0.0.0.0/0
rules_egress:
- proto: all
type: all
cidr_ip: 0.0.0.0/0
- name: Launch the new EC2 Instance
local_action: ec2
group={{ security_group }}
instance_type={{ instance_type}}
image={{ image }}
wait=true
region={{ region }}
keypair={{ keypair }}
count={{count}}
register: ec2
- name: Add the newly created EC2 instance(s) to the local host group (located inside the directory)
local_action: lineinfile
dest="./hosts"
regexp={{ item.public_ip }}
insertafter="[webserver]" line={{ item.public_ip }}
with_items: "{{ ec2.instances }}"
- name: Wait for SSH to come up
local_action: wait_for
host={{ item.public_ip }}
port=22
state=started
with_items: "{{ ec2.instances }}"
- name: Add tag to Instance(s)
local_action: ec2_tag resource={{ item.id }} region={{ region }} state=present
with_items: "{{ ec2.instances }}"
args:
tags:
Name: webserver
As others have said, the cloud module contains just about all the AWS provisioning support you'd need. That said, Ansible's paradigm makes most sense once there's an existing SSH:able machine to target and connect to. The instantiation phase, by comparison, essentially asks you to target your local machine and calls AWS API endpoints from there.
Like you, I wanted a single-shot command with a graceful transition from EC2 instantiation into its configuration. There's suggestions on how to accomplish something like this in the documentation, but it relies on the the add_host module to tweak Ansible's idea of current host inventory, and even then I couldn't find a solution that didn't feel like i was working against rather than with the system.
In the end I opted for two distinct playbooks: a provision.yml that uses the ec2, ec2_group, ec2_vol, ec2_eip and route53 modules to ensure I have the "hardware" in place, and then configure.yml, more like a traditional Ansible site.yml, which is able to treat host inventory (static in my case, but dynamic will work well) as a given and do all that good declarative state transitioning.
Both playbooks are idempotent, but it's configure.yml that's meant to be rerun over and over in the long run.
The EC2 module was designed precisely for creating and destroying instances.
If you want the "best" way, it's hard to beat CloudFormation, which can be launched from Ansible.
Related
I have an Ansible configuration where I am creating an EC2 instance. After the instance is ready, I want to disable periodic apt updates and wait for current update process to finish. Whenever I add the config in yml file, it executes the command on my local system. What am I doing wrong?
yml file:
---
- name: Provision an EC2 Instance
hosts: localhost
connection: local
gather_facts: False
tags: provisioning
tasks:
- name: Create New security group with below given name
local_action:
module: ec2_group
name: "{{ security_group }}"
description: Security Group for Newly Created EC2 Instance
region: "{{ region }}"
rules:
- proto: tcp
from_port: 22
to_port: 22
cidr_ip: 0.0.0.0/0
- proto: tcp
from_port: 80
to_port: 80
cidr_ip: 0.0.0.0/0
rules_egress:
- proto: all
cidr_ip: 0.0.0.0/0
- name: Launch the new t2 micro EC2 Instance
local_action: ec2
group={{ security_group }}
instance_type={{ instance_type}}
image={{ image }}
wait=true
region={{ region }}
keypair={{ keypair }}
count={{count}}
register: ec2
Now, after this, I wait for ssh to finish and want to pass the following commands on the newly created Ec2 instance :
- name: Disable timers for unattended upgrade, so that none will be triggered by the `date -s` call.
raw: systemctl disable --now {{item}}
with_items:
- 'apt-daily.timer'
- 'apt-daily-upgrade.timer'
- name: Reload systemctl daemon to apply the new changes
raw: systemctl daemon-reload
- name: Purge autoupdate
raw: apt -y purge unattended-upgrades
- name: Update apt cache
raw: apt -y update
But adding them as raw is not working, or even adding them as command.
The first section of code you posted is provisioning a new EC2 instance by making calls to the AWS API from your local system:
- name: Provision an EC2 Instance
hosts: localhost
connection: local
gather_facts: False
...
- name: Create New security group with below given name
local_action:
module: ec2_group
Note the local_action section that specifies running an action locally. Also, your target is localhost.
If you then want to then configure the new system, you can add it to a host group and run some configuration steps. For example, add this following the Provision an EC2 Instance step to add the new instance's public IP to a hostgroup called ec2hosts:
- name: Add instance public IP to host group
add_host: hostname={{ item.public_ip }} groups=ec2hosts
loop: "{{ ec2.instances }}"
Now you can configure the host by targeting the host group:
- hosts: ec2hosts
name: configuration play
user: ec2-user
gather_facts: true
tasks:
- name: Disable timers for unattended upgrade, so that none will be triggered by the `date -s` call.
raw: systemctl disable --now {{item}}
with_items:
- 'apt-daily.timer'
- 'apt-daily-upgrade.timer'
- name: Reload systemctl daemon to apply the new changes
raw: systemctl daemon-reload
- name: Purge autoupdate
raw: apt -y purge unattended-upgrades
- name: Update apt cache
raw: apt -y update
To summarize, you first create the instance from your local system, wait for it to boot, add its IP address to a host group, then run additional configuration steps by running ansible against that host group. For this to work, make sure to use an SSH key pair for which the private key has been added to the SSH agent. Also, make sure to launch the EC2 instance in to a public subnet.
Refer to the Ansible Amazon Web Service Guide.
I have playbook to create ec2 like this
- name: Create an EC2 Instance
hosts: localmachine
connection: local
vars_files:
- vars/common.yml
roles:
- ec2
Now I also want that after the ec2 is created then I want to run another role inside that ec2 machine. I know how to wait for ec2 to be created but I don't know how to run role in newly created ec2.
This is how I wait for it
- name: Wait for instances to listen on port 22
wait_for:
state: started
host: "{{ ec2_info.instances[0].private_ip" }}"
port: 22
when: ec2_info|changed
but I want another task after that to run different roles inside that ec2
There is a detailed AWS guide with a section on provisioning. The short answer is that you should register the results of your provisioning into a variable (it looks like you've already chosen ec2_info for this) and then add those into another group:
- name: Add all instance public IPs to host group
add_host: hostname={{ item.public_ip }} groups=ec2hosts
with_items: ec2_info.instances
You can then assign roles to that group as you would normally.
I'm looking for a nice way to show or list security groups in Ansible
Currently I'm using the Ansible module ec2_group that silently changes the security group to match what is defined in Ansible but doesn't show what is changed.
changed: [localhost -> 127.0.0.1] => {"changed": true, "group_id":
"sg-8649adee"}
I'm worried that someone may add something in the web console which would get erased when Ansible's ec2_group task is executed. That is okay as long as I can get some information about the previous state (in the output), so in case of an erasing some important but 'undocumented' modification, it can be quickly restored.
Currently the only way I know is run this as local command module:
aws ec2 describe-security-groups [some pattern]
Is there a better way to do this, hopefully entirely within Ansible?
Ansible is a tool for performing changes on an environment to configure it so that it looks as has been defined. As such, nearly everything it does is purely for the purpose of enacting said changes and there are very few modules that don't make changes other than for fact gathering to enable other modules to make changes.
As such, I would say that if you are using Ansible to control your estate then you should back it entirely. If someone makes a change outside of Ansible then having Ansible change it back should be a good thing. In the event of some out of hours emergency where someone needs to make a manual change then that person should have some mechanism to prevent Ansible from running until the change is fed back into the automation code (at my company we use Jenkins to drive all of our automation so they can simply disable the relevant job(s)).
If this isn't an option for you then you could always shell out and describe the group before your change and then have a task check if the ec2_group module changed anything and if so output what the security group looked like before:
- hosts: localhost
connection: local
vars:
security_group_name: testing
tasks:
- name: describe ec2 security group before change
shell: "aws ec2 describe-security-groups --group-names {{ security_group_name }}"
register: before
changed_when: false
- name: create ec2 security group
ec2_group:
name: "{{ security_group_name }}"
description: "{{ security_group_name }}"
rules:
- proto: tcp
from_port: 22
to_port: 22
cidr_ip: 0.0.0.0/0
- proto: udp
from_port: 123
to_port: 123
cidr_ip: 10.0.0.0/8
rules_egress:
- proto: all
cidr_ip: 0.0.0.0/0
register: ec2_group
- name: security group changed
debug: var=before
when: ec2_group.changed
Another option might be to simply use CloudTrail to see what has been changed in your AWS account.
I'll just add one update:
The
--group-names YourGroupName
works for the default VPC only
For non-default VPC use filters, like:
aws ec2 --region YourRegion describe-security-groups --filters Name=group-name,Values=YourGroupName
http://docs.aws.amazon.com/cli/latest/reference/ec2/describe-security-groups.html
I am trying to stop/start the particular group of instances listed in the hosts file under group [target].
following playbook works fine to stop the instances.
---
- hosts: target
remote_user: ubuntu
tasks:
- name: Gather facts
action: ec2_facts
- name: Stop Instances
local_action:
module: ec2
region: "{{region}}"
instance_ids: "{{ansible_ec2_instance_id}}"
state: stopped
But when I am trying to start these instances, it's not working as the ec2_facts is not able to ssh into the instances (since they are stopped now) and get the instance-ids
---
- hosts: target
remote_user: ubuntu
tasks:
- name: start instances
local_action:
module: ec2
region: "{{region}}"
instance_ids: "{{ansible_ec2_instance_id}}"
state: running
I have already seen the documentation which make use of dynamic inventory file for hosts and the way of hard-coding the instance-ids. I want to start the instances whose IPs are listed in the target group of hosts file.
Got the solution, Following is the ansible-task that worked for me.
---
- name: Start instances
hosts: localhost
gather_facts: false
connection: local
vars:
instance_ids:
- 'i-XXXXXXXX'
region: ap-southeast-1
tasks:
- name: Start the feature instances
ec2:
instance_ids: '{{ instance_ids }}'
region: '{{ region }}'
state: running
wait: True
Here is the Blog post on How to start/stop ec2 instances with ansible
You have 2 options:
Option 1
Use AWS CLI to query the instance-id of a stopped instance using its IP or name. For example, to query the instance id for a given instance name:
shell: aws ec2 describe-instances --filters 'Name=tag:Name,Values={{inst_name}}' --output text --query 'Reservations[*].Instances[*].InstanceId'
register: inst_id
Option 2
Upgrade Ansible to version 2.0 (Over the Hills and Far Away) and use the new ec2_remote_facts module
- ec2_remote_facts:
filters:
instance-state-name: stopped
You should add gather_facts: False to prevent Ansible from trying to SSH into the hosts since they're not running:
- hosts: target
remote_user: ubuntu
gather_facts: false
If you need to gather facts after the instances have started up then you can use the setup module to explicitly gather facts after they have booted up.
Edit: I just realized that the issue is that you're trying to access the ansible_ec2_instance_id fact that you can't get because the instance is down. You might want to take a look at this custom module called ec2_lookup that will let you search fetch AWS instance IDs even when the instances are down. Using this you can get a list of the instances you're interested in and then start them up.
I know how to create an AWS instance using Ansible. Now what I want to achieve is to configure that instance as web server by installing nginx using the same playbook which created the instance.
The goal of the playbook will be:
Create an AWS instance.
Configure the instance as Web server by setting up the Nginx server.
Is it possible with ansible?
Read http://www.ansible.com/blog/ansible-ec2-tags It details how to spin up an ec2 instance (or multiple) and then run tasks against it (I.e install nginx).
I'f you want to jump straight to the example playbook https://github.com/chrismeyersfsu/playbook-ec2_properties/blob/master/new_group.yml
Bring up ec2 instance
Wait for ssh
add ec2 instance to Ansible dynamically created host group w/ associated ec2 pem file (so you can ssh to it)
Call an example play with a ping task to show everything works
Note: you would replace the ping task with your set of tasks to install nginx
#Bidyut How to reference ec2 ip address
look at Line 27 Note the use of register: ec2Then at Line 46 the ec2 ip address is "extracted" {{ ec2.results[item.0]['instances'][0]['public_ip'] }}. Note that the example calls register within a loop. If you are just creating one ec2 instance then the ec2 ip address reference would look like {{ ec2.results['instances'][0]['public_ip'] }}
Here is a working example that might help you.
---
- hosts: localhost
connection: local
gather_facts: no
tasks:
- name: Create the EC2 Instance
ec2:
region: us-east-1
group: sg-xxxxx # Replace your Security Group here
keypair: test-key # Replace Key here
instance_type: t2.mirco
image: ami-xxxxx # Replace AMI here
vpc_subnet_id: subnet-xxxxx # Replace Subnet here
assign_public_ip: yes
wait: yes
wait_timeout: 600
instance_tags:
Name: "My-EC2-Instance"
register: ec2
- name: Create SSH Group to login dynamically to EC2 Instance
add_host:
hostname: "{{ item.public_ip }}"
ansible_ssh_private_key_file: path/to/test-pair.pem
groupname: ec2_server
with_items: ec2.instances
- name: Wait for SSH to come up
wait_for:
host: "{{ item.public_ip }}"
port: 22
state: started
with_items: ec2.instances
- hosts: ec2_server
become: yes
# Use ec2_user if you are using CentOS/Amazon server
remote_user: ubuntu # for Ubuntu server
gather_facts: yes
roles:
- webserver
Yes, you can use a single playbook to launch an instance and install nginx. Use the ansible module add_host to add the ip of the just launched instance. Then write a play for the new host.
Launch an EC2 instance using ec2 module and register the instance
Use add_host module to add the new instance to the host inventory
Write a new play with host as the just registered host and call apt to install nginx
Try it and if you need code snippet, let me know.