I am encountering a strange problem with ansible, and I'm sure its just due to my lack of experience as I am relatively new to ansible (I've only been working with it for a couple weeks)
So, in short what I am trying to do is use the command module to run AWS CLI commands to list AWS access keys for a user then delete them from that user. The reason I am using CLI instead of iam module is because I believe there to be a bug with the IAM module in regard to removing access keys. Even when I specified state update and the access keys to remove and access key state remove it still would not remove access keys, or make them inactive when i set access key state to inactive.
The first task lists access keys for a given user and registers the output:
- name: List the access keys (if any) of the user(s) we just created
vars:
use_key: "{{ enable_access_keys }}"
command: "aws iam list-access-keys --user-name {{ item }}"
with_items:
- "{{ iam_user_name_list }}"
when: not use_key
register: list_key_output
^Keep in mind that iam_user_name_list only contains 1 user at the moment, which is why i access results the way I do. I know this needs to be changed in the future.
Since the stdout from list_key_output looks like this
"stdout": "{\n \"AccessKeyMetadata\": [\n {\n \"UserName\": \"other-guy\", \n \"Status\": \"Active\", \n \"CreateDate\": \"2017-06-29T18:45:04Z\", \n \"AccessKeyId\": \"removed\"\n }\n ]\n}",
I debug msg the stdout and register that to a variable test to give it proper json format without the slashes and newlines so I can use json_query to get the key from the stdout. I am using json query because AccessKeyId is not recognized as a key for the AccessKeyMetadata dictionary for whatever reason.
- name: list keys stdout
debug:
msg: "{{ list_key_output.results[0].stdout }}"
register: test
- name: test variable output
debug:
msg: "{{ test.msg.AccessKeyMetadata | json_query('[].AccessKeyId') }}"
At this point, I am successfully getting the access key from the stdout
ok: [127.0.0.1] => {
"changed": false,
"msg": [
"correct access key here"
]
}
Now, I feed the access key to the delete CLI command like so
- name: Remove any access keys our new console user might have
vars:
use_key: "{{ enable_access_keys }}"
command: "aws iam delete-access-key --access-key {{ test.msg.AccessKeyMetadata | json_query('[].AccessKeyId') }} --user-name other-guy"
when: not use_key
register: delete_key_output
This task fails due to an invalid access key being provided.
fatal: [127.0.0.1]: FAILED! => {"changed": true, "cmd": ["aws", "iam", "delete-access-key", "--access-key", "[u********************]", "--user-name", "other-guy"], "delta": "0:00:00.388902", "end": "2017-06-29 18:59:13.308230", "failed": true, "rc": 255, "start": "2017-06-29 18:59:12.919328", "stderr": "\nAn error occurred (ValidationError) when calling the DeleteAccessKey operation: The specified value for accessKeyId is invalid. It must contain only alphanumeric characters.", "stderr_lines": ["", "An error occurred (ValidationError) when calling the DeleteAccessKey operation: The specified value for accessKeyId is invalid. It must contain only alphanumeric characters."], "stdout": "", "stdout_lines": []}
As you can see, when I pass the access key to the command, [u is prepended to the front of the access key and ] is appended to the back of it.
Why is this happening? How can I achieve my goal without having the 3 characters added to the access key making it invalid? I don't understand why this happens because when I debug msg the access key the same way i provide it to the command, it only shows the access key without [u in front and ] behind.
Sorry for the long post but I felt I really had to describe the situation to be able to get help here. Thanks in advance for any answers!
To answer your exact question:
ok: [127.0.0.1] => {
"changed": false,
"msg": [
"correct access key here"
]
}
Note the [ and ] in the msg – this means that you print a list, that contain one element – correct access key here string.
When you try to convert list into string, you get it's Python interpretation [u'correct access key here'].
You need to get first element of a list:
{{ test.msg.AccessKeyMetadata | json_query('[].AccessKeyId') | first }}
P.S. but from my point of view, you are going in a wrong way. try to fix your issues with iam module.
Related
I'm experimenting with podman rootless.
Users in containers get assigned a subuid / subgid space from the host.
Files created or updated from a user in the container environment belong to the user id space,
that doesn't exist on the host.
That's where I'm currently stuck. I can calculate the subuid with ansible and ease access to the container owned files with ACL, but I can't get ansible to write out a jinja template and chown it to a user that doesn't exist on the host.
I also don't want to workaround by creating a dummy user with a matching UID on the host, since that would probably undermine the security advantages / the rootless concept.
Here the task:
- name: copy hass main config to storage
become: yes
template:
src: configuration.yaml.j2
dest: "{{ hass_data_dir }}/configuration.yaml"
owner: "{{ stat_container_base_dir }}.uid"
group: "{{ stat_container_base_dir }}.gid"
mode: 0640
and the error message when running the task.
TASK [server/smarthome/homeassistant/podman : copy hass main config to storage] ************************************************************************************************************************
fatal: [odroid]: FAILED! =>
changed: false
checksum: 20c59b4a12d4ebe52a3dd191a80a5091d8e6dc0c
gid: 0
group: root
mode: '0640'
msg: 'chown failed: failed to look up user {''changed'': False, ''stat'': {''exists'':
True, ''path'': ''/home/homeassistant/container'', ''mode'': ''0770'', ''isdir'':
True, ''ischr'': False, ''isblk'': False, ''isreg'': False, ''isfifo'': False,
''islnk'': False, ''issock'': False, ''uid'': 363147, ''gid'': 362143, ''size'':
4096, ''inode'': 4328211, ''dev'': 45826, ''nlink'': 3, ''atime'': 1669416005.068732,
I tried to find help in the modules documentation at: https://docs.ansible.com/ansible/latest/collections/ansible/builtin/template_module.html
My ansible version is: ansible [core 2.13.1]
As you can see in the error message, ansible is missing a user with UID 363147 on the host.
Is there any way to circumvent the test if a user exists in ansible.builtin.template and similar modules, that allow user assignment with owner: and group:?
The only workaround I found was using command, but with the need for templates, complexity will increase when I'd have to parse jinja templates without the ansible template module.
I would appreciate if I missed an existing option or would like to create a pull request for an option like:
ignore_usercheck: true or validate_user: false
Hope you can help me out here :)
After all this was only a misleading error message, not a missing feature in Ansible.
I tested with the debug module and found out, that the values of stat have to be accessed from inside the curly brackets.
- name: debug
debug:
msg: "{{ stat_container_base_dir.stat.uid }}"
What Ansible got, was the whole string content of stat, not just the UID.
User ID's that don't exist on the host can be assigned.
I am relatively new to Ansible and I am struggling to understand how to perform the following scenario:
I have an array with AWS security group names looking like this
['Security-Group-Name1', 'SecurityGroup-Name2', 'SecurityGroup-Name3']
However, what I want is to have an array of SecurityGroupIds. Using Ansible I have the ec2_group_info as an option to retrieve information about a security group. So far so good ...
Now comes my question. I need to loop through the above array using ec2_group_info, set the name of the security group I need and return the retrieved Id into a new array so in the end I have something like this.
['Security-Group-Id1', 'SecurityGroup-Id2', 'SecurityGroup-Id3']
I know I need to use a loop with sort of a dynamic index. But it is not really clear to me how to do this in Ansible.
I am aware of the latest loop section of Ansible Docs and I find them more than confusing...
https://docs.ansible.com/ansible/latest/user_guide/playbooks_loops.html
Edit:
This is the current code which works as needed:
- name: Installing pip if not existing on host
pip:
name: boto3
- name: Get SecurityGroupId information
ec2_group_info:
filters:
group_name: ['SG-One', 'SG-Two']
vpc_id: 'vpc-id'
register: my_groups
- set_fact:
my_group_ids: '{{ my_groups.security_groups | map(attribute="group_id") | list }}'
- debug:
msg: "{{ my_groups }}"
- debug:
msg: "{{ my_group_ids }}"
This is the outcome:
TASK [Gathering Facts] ***************************************************
ok: [localhost]
TASK [machine-provisioning : Installing pip if not existing on host] ************
ok: [localhost]
TASK [machine-provisioning : Get SecurityGroupId information] *************************
ok: [localhost]
TASK [machine-provisioning : set_fact] *********************************
ok: [localhost]
TASK [machine-provisioning : debug] ***********************************************
ok: [localhost] => {
"msg": [
"sg-00000000",
"sg-11111111"
]
}
On that linked page about loops, you'll observe the use of register:, which is how you'd capture the result of that ec2_group_info: lookup, then use the map jinja filter to extract map(attribute="group_id") from the resulting list of results; you have to feed the output of map into the list filter, because map and a few others are python generators, and thus need a terminal action to materialize their data. The set_fact: is how ansible does "assignment"
- ec2_group_info:
filters:
group_name: '{{ the_group_names }}'
vpc_id: '{{ my_vpc_id }}'
register: my_groups
- set_fact:
my_group_ids: '{{ my_groups.security_groups | map(attribute="group_id") | list }}'
yields:
ok: [localhost] => {"ansible_facts": {"my_group_ids": ["sg-0c5c277ed1edafb54", "sg-7597a123"]}, "changed": false}
I have the following step in a SSM document. The result of the call is a Json, so I wanted to parse it as a stringMap (which seems to be the correct type for it) instead of creating an output for each variable I want to reference
I've tried referencing this as both:
{{ GetLoadBalancerProperties.Description.Scheme }}
and
{{ GetLoadBalancerProperties.Description[\"LoadBalancerName\"] }}
In both cases I get an error saying the variable was never defined
{
"name": "GetLoadBalancerProperties",
"action": "aws:executeAwsApi",
"isCritical": true,
"maxAttempts": 1,
"onFailure": "step:deleteParseCloudFormationTemplate",
"inputs": {
"Service": "elb",
"Api": "describe-load-balancers",
"LoadBalancerNames": [
"{{ ResourceId }}"
]
},
"outputs": [
{
"Name": "Description",
"Selector": "$.LoadBalancerDescriptions[0]",
"Type": "StringMap"
}
]
}
This is the actual message:
Step fails when it is validating and resolving the step inputs. Failed to resolve input: GetLoadBalancerProperties.Description["LoadBalancerName"] to type String. GetLoadBalancerProperties.Description["LoadBalancerName"] is not defined in the Automation Document.. Please refer to Automation Service Troubleshooting Guide for more diagnosis details.
I believe the answer you were searching is in here:
https://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-plugins.html#top-level-properties-type
Just to name a few examples:
Map type is a Python dict, hence if your output is a dict you should use StringMap in the SSM Document.
While List type is same as Python list.
So if your output is a List of Dictionary the type you want to use is MapList.
In some cases it seems that you cannot. I was able to work around this issue, by using a Python script in the SSM document to output the right type, but otherwise I believe the SSM document is not flexible enough to cover all cases.
The script I used:
- name: myMainStep
action: aws:executeScript
inputs:
Runtime: python3.6
Handler: myMainStep
InputPayload:
param: "{{ myPreviousStep.myOutput }}"
Script: |-
def myMainStep(events,context):
myOutput = events['myOutput']
for tag in myOutput:
if tag["Key"] == "myKey":
return tag["Value"]
return "myDefaultValue"
outputs:
- Name: output
Selector: "$.Payload"
Type: String
You can find out what the myOutput should be in AWS web console > SSM > Automation > Your execution, if you have already executed your automation once > executeScript step > input parameters
I'm trying to parse ansible variables using python specified in an inventory file like below:
[webservers]
foo.example.com type=news
bar.example.com type=sports
[dbservers]
mongodb.local type=mongo region=us
mysql.local type=mysql region=eu
I want to be able to parse type=news for host foo.example.com in webservers and type=mongo region=us for host mongodb.local under dbservers. Any help with this is greatly appreciated
The play below
- name: List type=news hosts in the group webservers
debug:
msg: "{{ hostvars[item].inventory_hostname }}"
loop: "{{ groups['webservers'] }}"
when: hostvars[item].type == "news"
- name: List type=mongo and region=us hosts in the group dbservers
debug:
msg: "{{ hostvars[item].inventory_hostname }}"
loop: "{{ groups['dbservers'] }}"
when:
- hostvars[item].type == "mongo"
- hostvars[item].region == "us"
gives:
"msg": "foo.example.com"
"msg": "mongodb.local"
If the playbook will be run on the host:
foo.example.com
you can get "type = news" simply by specifying "{{type}}". If you want to use in "when" conditions, then simply indicating "type"
If the playbook will be run on the host:
mongodb.local
then the value for "type" in this case will automatically be = "mongo", and "region" will automatically be = "us"
The values of the variables, if they are defined in the hosts file as you specified, will automatically be determined on the specified hosts
Thus, the playbook can be executed on all hosts and if you get a value for "type", for example:
- debug:
msg: "{{type}}"
On each of the hosts you will get your unique values that are defined in the hosts file
I'm not sure that I understood the question correctly, but if it meant that on the foo.example.com host it was necessary to get a list of servers from the "webservers" group that have "type = news", then the answer is already given.
Rather than re-inventing the wheel, I suggest you have a look at how ansible itsef is parsing ini files to turn them into an inventory object
You could also easily get this info in json format with a very simple playbook (as suggested by #vladimirbotka), or rewrite your inventory in yaml which would be much easier to parse with any external tool
inventory.yaml
---
all:
children:
webservers:
hosts:
foo.example.com:
type: news
bar.example.com:
type: sports
dbservers:
hosts:
mongodb.local:
type: mongo
region: us
mysql.local:
type: mysql
region: eu
I have two instances on different VPCs which have the same private address.
ci-vpc:
172.18.50.180:
tags:
Environment: ci
Role: aRole
test-vpc:
172.18.50.180:
tags:
Environment: test
Role: web
I am running the following playbook:
- name: "print account specific variables"
hosts: "tag_Environment_ci:&tag_Role_web"
tasks:
- name: "print account specific variables for account {{ account }}"
debug:
msg:
- 'ec2_tag_Name': "{{ ec2_tag_Name }}"
'ec2_tag_Role': "{{ ec2_tag_Role }}"
'ec2_private_ip_address': "{{ ec2_private_ip_address }}"
'ec2_tag_Environment': "{{ ec2_tag_Environment }}"
Since I am asking for both role web and environment ci, none of these instances should be picked, but nevertheless the result that I am getting is:
ok: [172.18.50.180] => {
"changed": false,
"msg": [
{
"ec2_private_ip_address": "172.18.50.180",
"ec2_tag_Environment": "test",
"ec2_tag_Name": "test-web-1",
"ec2_tag_Role": "web"
}
]
}
Obviously this instance does not meet the requirements under hosts...
It seems like ec2.py searched for the Environment tag, found ci for 172.18.50.180, then searched separately for the role tag, found another one under 172.18.50.180, and just marked that instance as ok, even though these are two different instances on different vpcs.
I've tried changing vpc_destination_variable in ec2.ini to id but then I'm getting error when Ansible is trying to connect to these instances because it cannot connect to the id...
fatal: [i-XXX]: UNREACHABLE! => {
"changed": false,
"msg": "Failed to connect to the host via ssh: ssh: Could not resolve hostname i-XXX: Name or service not known\r\n", "unreachable": true
}
Is there another option that will work under vpc_destination_variable? Any known solution for such a collision?
tl;dr: This is exactly what hostname_variable in ec2.ini is for, as documented:
# This allows you to override the inventory_name with an ec2 variable, instead
# of using the destination_variable above. Addressing (aka ansible_ssh_host)
# will still use destination_variable. Tags should be written as 'tag_TAGNAME'.
Unfortunetely I've missed it and found it after looking around in ec2.py
Longer answer with additional options to hostnames
After finding out about hostname_variable, I had another problem that it can receive only one variable. In my case I had some instances with the same private ip on one hand, and some with the same tags on the other (AWS autoscaling groups, same tags on all hosts), so I needed a way to differentiate between them.
I've created a gist with this option. My change is in line 848. This allows you to use multiple comma separated variables in hostname_variable, e.g.:
hostname_variable = tag_Name,private_ip_address