Script execution using ansible [duplicate] - amazon-web-services

I am using Ansible to deploy my project and I trying to check if an specified package is installed, but I have a problem with it task, here is the task:
- name: Check if python-apt is installed
command: dpkg -l | grep python-apt
register: python_apt_installed
ignore_errors: True
And here is the problem:
$ ansible-playbook -i hosts idempotent.yml
PLAY [lxc-host] ***************************************************************
GATHERING FACTS ***************************************************************
ok: [10.0.3.240]
TASK: [idempotent | Check if python-apt is installed] *************************
failed: [10.0.3.240] => {"changed": true, "cmd": ["dpkg", "-l", "|", "grep", "python-apt"], "delta": "0:00:00.015524", "end": "2014-07-10 14:41:35.207971", "rc": 2, "start": "2014-07-10 14:41:35.192447"}
stderr: dpkg-query: error: package name in specifier '|' is illegal: must start with an alphanumeric character
...ignoring
PLAY RECAP ********************************************************************
10.0.3.240 : ok=2 changed=1 unreachable=0 failed=0
Why is illegal this character '|' .

From the doc:
command - Executes a command on a remote node
The command module takes the command name followed by a list of
space-delimited arguments. The given command will be executed on all
selected nodes. It will not be processed through the shell, so
variables like $HOME and operations like "<", ">", "|", and "&" will
not work (use the shell module if you need these features).
shell - Executes a commands in nodes
The shell module takes the command name followed by a list of space-delimited arguments.
It is almost exactly like the command module but runs the command
through a shell (/bin/sh) on the remote node.
Therefore you have to use shell: dpkg -l | grep python-apt.

read about the command module in the Ansible documentation:
It will not be processed through the shell, so .. operations like "<", ">", "|", and "&" will not work
As it recommends, use the shell module:
- name: Check if python-apt is installed
shell: dpkg -l | grep python-apt
register: python_apt_installed
ignore_errors: True
For what it's worth, you can check/confirm the installation in a debian environment using the apt command:
- name: ensure python-apt is installed
apt: name=python-apt state=present

Related

Regex in Windows Batch to automate Docker run

I am trying to automate the process of sending my temporary Amazon AWS keys as environment variables to a Docker image using Windows. I have a file, credentials.txt that contains my AWS credentials (the 3 ids are always the same, but the string values change regularly). I am using Windows command prompt.
Input:
(includes 2 empty lines at end) credentials.txt:
[default]
aws_access_key_id = STR/+ing1
aws_secret_access_key = STR/+ing2
aws_session_token = STR/+ing3
Desired output:
I need to issue the following command in order to run a Docker image (substituting the strings with the actual strings):
docker run -e AWS_ACCESS_KEY_ID=STR/+ing1 -e AWS_SECRET_ACCESS_KEY=STR/+ing2 -e AWS_SESSION_TOKEN=STR/+ing3 my-aws-container
My idea is to try to use regex on credentials.txt to convert it to:
SET aws_access_key_id=STR/+ing1
SET aws_secret_access_key=STR/+ing2
SET aws_session_token=STR/+ing3
And then run:
docker run -e AWS_ACCESS_KEY_ID=%aws_access_key_id% -e AWS_SECRET_ACCESS_KEY=%aws_secret_access_key% -e AWS_SESSION_TOKEN=%aws_session_token% my-aws-container
Does anyone have any advice on how to achieve this?
You can parse your credentials.txt with a for /f loop to set the variables (effectively removing the spaces):
for /f "tokens=1,3" %%a in ('type credentials.txt ^| find "="') do set "%%a=%%b"
and then run the last code line from your question:
docker run -e AWS_ACCESS_KEY_ID=%aws_access_key_id% -e AWS_SECRET_ACCESS_KEY=%aws_secret_access_key% -e AWS_SESSION_TOKEN=%aws_session_token% my-aws-container
Note: the values should not contain spaces or commas.
I've had a go in python that seems to work. Someone else may have a better answer.
I create the python file:
docker_run.py
import re
import os
myfile = 'C:/fullpath/credentials'
with open(myfile,'r') as f:
mystr = f.read()
vals = re.findall('=[\s]*([^\n]+)',mystr)
keys = ['AWS_ACCESS_KEY_ID','AWS_SECRET_ACCESS_KEY','AWS_SESSION_TOKEN']
environment_vars = ''.join([' -e ' + k + '=' + v for k,v in zip(keys,vals)])
cmd = 'docker run'+environment_vars+' my-aws-container'
os.system(cmd)
Then from command prompt I run:
python docker_run.py
This succeeds in running docker
(note: I tried using exec() in the final line rather than os.system(), but got the error "SyntaxError: invalid syntax")

How to collect a grep and use it in a aws configset

In my aws Cloud Formation cfn configset I have a command to set an environment key to the name of the user group apache belongs to as it might be apache or www-data depending on the distro.
Something like this:
Metadata:
AWS::CloudFormation::Init:
configSets:
joomla:
- "set_permissions"
- "and_some_more..."
configure_cfn:
files:
/etc/cfn/hooks.d/cfn-auto-reloader.conf:
content: !Sub |
[cfn-auto-reloader-hook]
triggers=post.update
path=Resources.EC2.Metadata.AWS::CloudFormation::Init
action=/opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource EC2 --configsets joomla --region ${AWS::Region}
mode: "000400"
owner: root
group: root
.....
set_permissions:
commands:
01_01_get_WebServerGroup:
env:
#webserver group might be apache or www-data depending on the distro
WebServerGp:
command: "ps -ef | egrep '(httpd|apache2|apache)' | grep -v `whoami` | grep -v root | head -n1 | awk '{print $1}'"
However, when I launch this stack, the configsets process halts at this point and I get an an error in the cfn_init.log that looks like this:
File
"/usr/lib/python2.7/dist-packages/cfnbootstrap/command_tool.py", line
80, in apply
raise ToolError(u"%s does not specify the 'command' attribute, which is required" % name) ToolError: 01_01_get_WebServerGroup does
not specify the 'command' attribute, which is required
Is this the preferred method to catch and use a grep result in a configset command? Is there a better way? What can I do to address the error thrown in the cfn_init.log?
OK, I guess I can create parameter and mapping elements to capture the distro type on launch and then set the webserver group accordingly but I am really trying to understand how to set the env: key to a response from the cli.
The problem of your code is this line WebServerGp.
Line command is must be on the same level of env, under the commands name, in your case is 01_01_get_WebServerGroup. So, it has to be like this:
commands:
01_01_get_WebServerGroup:
env: ..
command: ..
If you want to use the result of grep, you can put them on variable and use it later.
You can specify more than one command under that command line using \n for executing the command.
Please check this code below.
command: "result=(ps ef | grep ...)\n echo $result\n ..."
If you have really long command, you can use the Fn::Join to be the value of command.

How do I use regex with Ansible?

I am having trouble getting a regex pattern to match in Ansible 2. Could someone help me understand what I am doing wrong? Thank you.
--- # Disable auto update for Ubuntu
- hosts: nonedgeLinux
become: yes
tasks:
- name: disable auto updates
replace:
dest: /etc/apt/apt.conf.d/50unattended-upgrades
regexp: '(?:[ \t]*\"\${distro_id}:\${distro_codename}-security\";)'
replace: '// "\${distro_id}:\${distro_codename}-security\";'
I used https://regex101.com/ to validate the regex against a copy/paste of the file contents being searched. This test reports the regexp pattern is correct. Each time it is run on a Ubuntu 16.04.2 box I get the following results:
root#sbx54:/data/scripts/ansible# ansible-playbook disableAutoUpdate.yml -vvvv
Using /data/scripts/ansible/ansible.cfg as config file
Loaded callback default of type stdout, v2.0
1 plays in disableAutoUpdate.yml
PLAY ****************************************************************************
TASK [setup] *******************************************************************
ESTABLISH LOCAL CONNECTION FOR USER: root
localhost EXEC ( umask 22 && mkdir -p "$( echo $HOME/.ansible/tmp/ansible-tmp-1492480514.73-9504514182168 )" && echo "$( echo $HOME/.ansible/tmp/ansible-tmp-1492480514.73-9504514182168 )" )
localhost PUT /tmp/tmpkYPc6g TO /root/.ansible/tmp/ansible-tmp-1492480514.73-9504514182168/setup
localhost EXEC LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 /usr/bin/python /root/.ansible/tmp/ansible-tmp-1492480514.73-9504514182168/setup; rm -rf "/root/.ansible/tmp/ansible-tmp-1492480514.73-9504514182168/" > /dev/null 2>&1
ok: [localhost]
TASK [disable auto updates] ****************************************************
task path: /data/scripts/ansible/disableAutoUpdate.yml:15
ESTABLISH LOCAL CONNECTION FOR USER: root
localhost EXEC ( umask 22 && mkdir -p "$( echo $HOME/.ansible/tmp/ansible-tmp-1492480515.71-278594852314124 )" && echo "$( echo $HOME/.ansible/tmp/ansible-tmp-1492480515.71-278594852314124 )" )
localhost PUT /tmp/tmpG3gchf TO /root/.ansible/tmp/ansible-tmp-1492480515.71-278594852314124/replace
localhost EXEC LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 /usr/bin/python /root/.ansible/tmp/ansible-tmp-1492480515.71-278594852314124/replace; rm -rf "/root/.ansible/tmp/ansible-tmp-1492480515.71-278594852314124/" > /dev/null 2>&1
ok: [localhost] => {"changed": false, "invocation": {"module_args": {"backup": false, "content": null, "delimiter": null, "dest": "/etc/apt/apt.conf.d/50unattended-upgrades", "directory_mode": null, "follow": false, "force": null, "group": null, "mode": null, "owner": null, "regexp": "(?:(?!\\/\\/).*[ \\t]*\\\"\\${distro_id}:\\${distro_codename}-security\\\";)/g", "remote_src": null, "replace": "replaced", "selevel": null, "serole": null, "setype": null, "seuser": null, "src": null, "validate": null}, "module_name": "replace"}, "msg": ""}
PLAY RECAP *********************************************************************
localhost : ok=2 changed=0 unreachable=0 failed=0
Here is a snip of the file being searched:
Unattended-Upgrade::Allowed-Origins {
"${distro_id}:${distro_codename}";
"${distro_id}:${distro_codename}-system";
// "${distro_id}:${distro_codename}-updates";
// "${distro_id}:${distro_codename}-proposed";
// "${distro_id}:${distro_codename}-backports";
};
# ansible --version
ansible 2.0.0.2
config file = /data/scripts/ansible/ansible.cfg
configured module search path = Default w/o overrides
One last note: I did have success in search and replace within this file when only looking for strings. When looking for the entire line is when this fails. Additionally I have tried escaping the special characters as well, e.g. [/][/] in place of \/\/\, etc. with no luck.
On top of the mistake in trying to match security with system, you don't need to escape the double quotes in either of the arguments or the dollar sign in the replace argument (otherwise the backslash will be inserted into the file):
- name: disable auto updates
replace:
dest: /etc/apt/apt.conf.d/50unattended-upgrades
regexp: '(?:[ \t]*"\${distro_id}:\${distro_codename}-system";)'
replace: '// "${distro_id}:${distro_codename}-system";'
And you can achieve the same result using lineinfile module (which makes code a bit more readable, imho):
- name: disable auto updates
lineinfile:
dest: /etc/apt/apt.conf.d/50unattended-upgrades
regexp: '"\${distro_id}:\${distro_codename}-system"'
line: '// "${distro_id}:${distro_codename}-system";'

Command cron_01_set_leader output: bash: /usr/local/bin/bundle: No such file or directory

After installing/configuring whenever-elasticbeanstalk gem, I'm seeing the following error in /var/log/cfn-init.log on my EC2 instance after running git aws.push from my local repo.
Iam using aws elastic benastalk with rails 4.
2014-10-21 08:08:37,602 [DEBUG] Running test for command cron_01_set_leader
2014-10-21 08:08:37,744 [DEBUG] Test command output:
2014-10-21 08:08:37,745 [DEBUG] Test for command cron_01_set_leader passed
2014-10-21 08:08:38,085 [ERROR] Command cron_01_set_leader (su -c "/usr/local/bin/bundle exec create_cron_leader --no-update" $EB_CONFIG_APP_USER) failed
2014-10-21 08:08:38,086 [DEBUG] Command cron_01_set_leader output: bash: /usr/local/bin/bundle: No such file or directory
Traceback (most recent call last):
I have added the whenever-elasticbeanstalk
Below is my cron.config file content..
Any idea ...what am i doing wrong?
files:
# Reload the on deployment
/opt/elasticbeanstalk/hooks/appdeploy/post/10_reload_cron.sh:
mode: "00700"
owner: root
group: root
content: |
#!/usr/bin/env bash
. /opt/elasticbeanstalk/containerfiles/envvars
cd $EB_CONFIG_APP_CURRENT
su -c "/usr/local/bin/bundle exec setup_cron" $EB_CONFIG_APP_USER
# Add Bundle to the PATH
"/etc/profile.d/bundle.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
export PATH=$PATH:/usr/local/bin
encoding: plain
container_commands:
cron_01_set_leader:
test: test ! -f /opt/elasticbeanstalk/containerfiles/.cron-setup-complete
leader_only: true
cwd: /var/app/ondeck
command: su -c "/usr/local/bin/bundle exec create_cron_leader --no-update" $EB_CONFIG_APP_USER
cron_02_write_cron_setup_complete_file:
cwd: /opt/elasticbeanstalk/containerfiles
command: touch .cron-setup-complete
Which solution stack are you using? Can you give the exact name, something like "64bit Amazon Linux 2014.03 v1.0.9 running Ruby 2.1 (Puma)".
I think you will need to replace "/usr/local/bin/bundle" with the actual version of bundle that is used for the solution stack.
Can you just try using "bundle" instaed of "/usr/local/bin/bundle"?

Packer's Puppet provisioning stalls

I'm very new to the this whole Packer/Vagrant,Puppet world. I'm trying to build my first VM using Packer and Puppet.
I can successfully build a virtualbox and I've included a shell script provisioner to install puppet. I've ssh'ed into the VM to verify that it works and puppet is installed.
Then I added an additional puppet-masterless provisioner that looks simply like this:
# java dependency
package { 'openjdk-7-jdk' :
ensure => present
}
When I run packer, it gets to this point and gets stuck:
==> virtualbox-iso: Provisioning with Puppet...
virtualbox-iso: Creating Puppet staging directory...
virtualbox-iso: Uploading manifests...
virtualbox-iso: Running Puppet: sudo -E puppet apply --verbose --modulepath='' --detailed-exitcodes /tmp/packer-puppet-masterless/manifests/ubuntu.pp
Any suggestions would be helpful. Even on how to debug it to see what's going on behind the scenes
I was having the same problem, and changed the execute_command to receive the password of the vagrant user.
"override": {
"virtualbox-iso": {
"execute_command": "echo 'vagrant' | {{.FacterVars}}{{if .Sudo}} sudo -S -E {{end}}puppet apply --verbose --modulepath='{{.ModulePath}}' {{if ne .HieraConfigPath \"\"}}--hiera_config='{{.HieraConfigPath}}' {{end}} {{if ne .ManifestDir \"\"}}--manifestdir='{{.ManifestDir}}' {{end}} --detailed-exitcodes {{.ManifestFile}}"
}
}
The whole block looks like this
{
"type": "puppet-masterless",
"manifest_file": "../puppet/manifests/base.pp",
"module_paths": [
"../puppet/modules/"
],
"override": {
"virtualbox-iso": {
"execute_command": "echo 'vagrant' | {{.FacterVars}}{{if .Sudo}} sudo -S -E {{end}}puppet apply --verbose --modulepath='{{.ModulePath}}' {{if ne .HieraConfigPath \"\"}}--hiera_config='{{.HieraConfigPath}}' {{end}} {{if ne .ManifestDir \"\"}}--manifestdir='{{.ManifestDir}}' {{end}} --detailed-exitcodes {{.ManifestFile}}"
}
}
}
Source: Found an example here https://github.com/AdoptOpenJDK/openjdk-virtual-images/blob/master/packer/openjdk-development/openjdk-development.json