Ansible dynamic inventory when connecting with AWS - amazon-web-services

I have downloaded ec2.py and ec2.ini files and placed them under
/etc/ansible/inventory
and i have changed the ansible.cfg to
inventory = /etc/ansible/inventory/ec2.py
i have environment variables set up as
export AWS_ACCESS_KEY_ID="XXXXXXXXXXXXXXX"
export AWS_SECRET_ACCESS_KEY="XXXXXXXXXXXXXXXXXXXXX"
export AWS_DEFAULT_REGION="us-east-1"
export EC2_INI_PATH="/etc/ansible/inventory/ec2.ini"
now when i run ansible -m ping all
i am getting
[WARNING]: provided hosts list is empty, only localhost is available
instead of pinging all instances in my account
when i run ./ec2.py --list i get
{
"_meta": {
"hostvars": {}
}
}
all my instances in aws are running

It seems you are not using the right inventory file.
Check ec2.py is executable (chmod +x)
Run ansible with -v option to check what config file it is really
using
ansible -v -m ping all
Check if you have ANSIBLE_INVENTORY or ANSIBLE_HOSTS environment
variables overriding expected config file. You can override inventory definition by setting
export ANSIBLE_INVENTORY=/etc/ansible/inventory/ec2.py
Remind that configuration files are processed in the following order
ANSIBLE_CONFIG (an environment variable)
ansible.cfg (in the current directory)
.ansible.cfg (in the home directory)
/etc/ansible/ansible.cfg
Prior to 1.5 the order was:
ansible.cfg (in the current directory)
ANSIBLE_CONFIG (an environment variable)
.ansible.cfg (in the home directory)
/etc/ansible/ansible.cfg

Related

Multistage dockerfile COPY from image - file not found

I am trying to COPY JAVA binaries from an already existing image over to new image using multistage dockerfile.
After the image is built, I do see all the files in the new image but when I execute JAVA, it gives me no such file or directory.
FROM quay.io/<private-repo>/node:12.8.0-slim
COPY --from=quay.io/<private-repo>/openjdk:8u212-jre-alpine /usr/lib/jvm/java-1.8-openjdk/ /usr/lib/jvm/java-8-openjdk-amd64/
# Setup JAVA_HOME, this is useful for docker commandline
ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64
RUN export JAVA_HOME
ENV PATH $PATH:$JAVA_HOME/bin
RUN export JAVA_HOME - will export the environment file only for the specific run instance. If you are logging in using docker exec and verifying the JAVA_HOME value. It will not exists.
Similarly ENV PATH $PATH:$JAVA_HOME/bin - will be used only on the run instance.
If you want to have these variables across multiple sessions append these entries to /etc/profile

How to install Ansible in Amazon Linux EC2?

I pick this Amazon Linux: Amazon Linux AMI 2017.09.1 (HVM), SSD Volume Type - ami-1a033c7a.
I installed Ansible using the command:
sudo pip install ansible,
it shows install completes.when I run ansible --version, it shows:
ansible 2.4.1.0
config file = None
configured module search path = [u'/home/ec2-
user/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/lib/python2.7/site-
packages/ansible
executable location = /usr/local/bin/ansible
python version = 2.7.12 (default, Nov 2 2017, 19:20:38) [GCC 4.8.5
20150623 (Red Hat 4.8.5-11)]
Why config file = None? Shouldn't it shows /etc/ansible/ansible.cfg? I do not see /etc/ansible/hosts, not even folder /etc/ansible. Did I install correctly, where is the folder /etc/ansible?
why config file = None?
Because at the time of running ansible --version no config file was found.
shouldn't it shows /etc/ansible/ansible.cfg?
No. It should show the ansible.cfg actually being used.
Per documentation, Ansible tries to find the config file in:
ANSIBLE_CONFIG (an environment variable)
ansible.cfg (in the current directory)
.ansible.cfg (in the home directory)
/etc/ansible/ansible.cfg
ansible --version will show you the exact path to the one being used.
Strictly speaking the last point is not always true, as package managers and virtual environment managers might cause the /etc directory to be located elsewhere.
did I install correctly
You didn't mention any error or warning during the installation and ansible --version returned a proper response.
There is no reason not to believe it's installed properly.
where is the folder /etc/ansible?
It's not existing on your system. There is no default inventory file, nor configuration file created by the installation package.
Create one.
Here I answer the question myself.
There are many ways to install ansible, and then you get difference default settings, depending on the OS. Many tutorials just assume the ansible_hosts and ansible.cfg already in /etc/ansible, which is not correct if you install ansible using pip.
In fact, if you install ansible using pip, then you will not see ansible.cfg and ansible_hosts in /etc/ansible. Even the folder /etc/ansible does not exist. but never mind, you can create these two files yourself as follows:
suppose you want to store ansible_hosts and ansible.cfg in /home/ec2-user, then you can:
echo <remote_host> /home/ec2-user/ansible_hosts
export ANSIBLE_INVENTORY=/home/ec2-user/ansible_hosts
wget https://raw.githubusercontent.com/ansible/ansible/devel/examples/ansible.cfg
mv ansible.cfg /home/ec2-user/
export ANSIBLE_CONFIG=/home/ec2-user/ansible.cfg
then if ansible --version, you will see
ansible 2.4.1.0
config file = /home/ec2-user/ansible.cfg
....
and if you test ansible ad-hoc command (my remote_host is ubuntu, so I use -u ubuntu, you can change it to be yours):
ansible all -m ping -u ubuntu
then you see ansible ping remote_host successfully.
This shows ansible does work.

How do I set the beanstalk .ebextensions .config "sources" key "target directory" to the current bundle directory

I'm working in a python 2.7 elastic beanstalk environment.
I'm trying to use the sources key in an .ebextensions .config file to copy a tgz archive to a directory in my application root -- /opt/python/current/app/utility. I'm doing this because the files in this folder are too big to include in my github repository.
However, it looks like the sources key is executed before the ondeck symbolic link is created to the current bundle directory so I can't reference /opt/python/ondeck/app when using the sources command because it creates the folder and then beanstalk errors out when trying to create the ondeck symbolic link.
Here are copies of the .ebextensions/utility.config files I have tried:
sources:
/opt/python/ondeck/app/utility: http://[bucket].s3.amazonaws.com/utility.tgz
Above successfully copies to /opt/python/ondec/app/utility but then beanstalk errors out becasue it can't create the symbolic link from /opt/python/bundle/x --> /opt/python/ondeck.
sources:
utility: http://[bucket].s3.amazonaws.com/utility.tgz
Above copies the folder to /utility right off the root in parallel with /etc.
You can use container_commands instead of sources as it runs after the application has been set up.
With container_commands you won't be able to use sources to automatically get your files and extract them so you will have to use commands such as wget or curl to get your files and untar them afterwards.
Example: curl http://[bucket].s3.amazonaws.com/utility.tgz | tar xz
In my environment (php) there is no transient ondeck directory and the current directory where my app is eventually deployed is recreated after commands are run.
Therefore, I needed to run a script post deploy. Searching revealed that I can put a script in /opt/elasticbeanstalk/hooks/appdeploy/post/ and it will run after deploy.
So I download/extract my files from S3 to a temporary directory in the simplest way by using sources. Then I create a file that will copy my files over after the deploy and put it in the post deploy hook directory .
sources:
/some/existing/directory: https://s3-us-west-2.amazonaws.com/my-bucket/vendor.zip
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/99_move_my_files_on_deploy.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
mv /some/existing/directory /var/app/current/where/the/files/belong

Elastic Beanstalk .ebextensions config file not getting deployed with git aws.push

I've linked a git branch to my Elastic Beanstalk environment and using git aws.push it deploys correctly.
I've now added a .extensions directory which contains a config script which should be creating a couple of directories. However, nothing appears to be happening.
I understand that the .extensions directory should be copied across to the ec2 instance as well but I'm not seeing it.
I've checked eb-tools.log and it's not mentioned in the upload.
Is there something additional that's required?
The script contains:
commands:
cache:
command: mkdir /tmp/cache
items:
command: mkdir /tmp/cache/items
chmod:
command: chmod -R 644 /tmp
You can find the run logs for this at /var/log/cfn-init.log.
In here I could see that the mkdir commands had worked initially but subsequently failed as the directory already existed.
Turns out that eb extensions run commands in alphabetical order so I had to change the commands to:
01command1:
02command2:
etc.
From this point on it worked fine.
Something else that was confusing me is that the .ebextensions directory in my local git repo was not appearing on the target instance directory. this is because once it's been run it will delete the directory.
Double check that your local script file has a .config extension. I was having a similar problem because my local file was called .ebextensions/01_stuff.yaml and it was fixed once I renamed it to .ebextensions/01_stuff.config.

How to set environment variable for root user at start-up?

I'm trying to add memory usage monitoring to the monitoring tab of an instance at console.aws.amazon.com. It's an instance running Amazon Linux AMI 2013.09.2 I have found the Amazon CloudWatch Monitoring Scripts for Linux and specifically mon-put-instance-data.pl that let's me collect memory stats and report it to CloudWatch as custom metrics.
To have this working I need to set the environment variable AWS_CREDENTIAL_FILE to point to a file containing my AWSAccessKeyId and AWSSecretKey. I do this by typing:
export AWS_CREDENTIAL_FILE=/home/ec2-user/aws-scripts-mon/awscreds.template
To avoid having to type this over and over again, I'm looking for a way to set the environment variable at startup. I have tried adding the code to these files:
/etc/rc.local file
/etc/profile
/home/ec2-user/.bash_profile
As adding the line of code in either of the files seems to work when I switch to root user, where should I put it? If I set the variable in /home/ec2-user/.bash_profile the variable is set for ec2-user but not for root. If i then sudo -E su it works, but I don't know if this is the best way to go about it?
Create a sh file and put the code in it. Then put this sh file in /etc/profile.d/ folder.
Note: create this sh file using the root user.
Once your instance is created, this sh file will automatically run and creates the environment variable for you and this environment variable will be accessible to all the users.