Ansible using AWS interactions throws exception - amazon-web-services

I was trying to run a playbook to automate AWS but i encountered this exception which I can't understand. I have the following playbook:
- name: Just for testing
hosts: windows-server
gather_facts: no
vars:
aws_a_key: accesskey
aws_s_key: secretkey
tasks:
- name: Ping
win_ping:
- name: Create a file to C:\Temp\test.conf
win_file:
path: C:\Temp\test.conf
state: touch
- name: Create another file to C:\Temp\test2.conf
win_file:
path: C:\Temp\test2.conf
state: touch
- name: Gather S3 facts
aws_s3_bucket_facts:
aws_access_key: "{{ aws_a_key }}"
aws_secret_key: "{{ aws_s_key }}"
region: eu-central-1
When I try to execute it I get the following error:
PLAY [Just for testing] *******************************************************************************************************
TASK [Ping] *******************************************************************************************************************
ok: [WIN-40NQ43PHHA5]
TASK [Create a file to C:\Temp\test.conf] *************************************************************************************
changed: [WIN-40NQ43PHHA5]
TASK [Create another file to C:\Temp\test2.conf] ******************************************************************************
changed: [WIN-40NQ43PHHA5]
TASK [Gather S3 facts] ********************************************************************************************************
fatal: [WIN-40NQ43PHHA5]: FAILED! => {"changed": false, "module_stderr": "Exception calling \"Create\" with \"1\" argument(s): \"At line:4 char:21
+ def _ansiballz_main():
+ ~
An expression was expected after '('.
At line:12 char:27
+ except (AttributeError, OSError):
+ ~
Missing argument in parameter list.
At line:14 char:7
+ if scriptdir is not None:
+ ~
Missing '(' after 'if' in if statement.
At line:21 char:7
+ if sys.version_info < (3,):
+ ~
Missing '(' after 'if' in if statement.
At line:21 char:30
+ if sys.version_info < (3,):
+ ~
Missing expression after ','.
At line:21 char:25
+ if sys.version_info < (3,):
+ ~
The '<' operator is reserved for future use.
At line:23 char:32
+ MOD_DESC = ('.py', 'U', imp.PY_SOURCE)
+ ~
Missing expression after ','.
At line:23 char:33
+ MOD_DESC = ('.py', 'U', imp.PY_SOURCE)
+ ~~~~~~~~~~~~~
Unexpected token 'imp.PY_SOURCE' in expression or statement.
At line:23 char:32
+ MOD_DESC = ('.py', 'U', imp.PY_SOURCE)
+ ~
Missing closing ')' in expression.
At line:23 char:46
+ MOD_DESC = ('.py', 'U', imp.PY_SOURCE)
+ ~
Unexpected token ')' in expression or statement.
Not all parse errors were reported. Correct the reported errors and try again.
\"
At line:6 char:1
+ $exec_wrapper = [ScriptBlock]::Create($split_parts[0])
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : ParseException
The expression after '&' in a pipeline element produced an object that was not valid. It must result in a command
name, a script block, or a CommandInfo object.
At line:7 char:2
+ &$exec_wrapper
+ ~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : BadExpression
", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}
to retry, use: --limit #/Users/mpolgardi/ansible/test_playbook.retry
PLAY RECAP ********************************************************************************************************************
WIN-40NQ43PHHA5 : ok=3 changed=2 unreachable=0 failed=1
Can someone help me decode this error? I have no idea where to even look. Also is there a way for ansible to give me readable error messages?

According to the fine manual, use of aws_s3_bucket_facts requires python with both the boto and boto3 modules available to it.
What appears to be happening to you is that the windows machine is trying to execute that .py file as if it were PowerShell, and (of course) those two syntaxes are wholly incompatible.
Do you have ansible_python_interpreter set to a working binary? It is possible that switching gather_facts: to yes will surface any misconfiguration earlier, although I can't swear to it since I don't know if the winrm connection pre-empts fact gathering using just PowerShell, and wouldn't notice a bogus ansible_python_interpreter value.
While this isn't exactly what you asked, it's also quite a common behavior to use delegate_to: localhost and/or connection: local for those AWS-y tasks, since there is little to no value in having those tasks run on the target machine, given that the credentials are already present in the playbook (that is: it doesn't acquire credentials from anything on the target machine). That presumes, naturally, that you have a local python which has boto and boto3 installed in them, but that's presumably far more likely than convincing a random windows machine to work correctly.

Related

ansible playbook : Error to connect to a windows host (winrm) using aws secret manager

i try to connect my ansible host to a windows server using winrm
my ansible version :
ansible 2.10.8
config file = None
configured module search path = ['/home/ec2-user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /home/ec2-user/.local/lib/python3.6/site-packages/ansible
executable location = /home/ec2-user/.local/bin/ansible
python version = 3.6.8 (default, Dec 5 2019, 15:45:45) [GCC 8.3.1 20191121 (Red Hat 8.3.1-5)]
it work when the password is in vars
- hosts: win
vars:
ansible_user: "ansible"
ansible_password: "Itismypassword"
and it is not working with this configuration :
- hosts: win
vars:
ansible_user: "ansible"
ansible_password: "{{ lookup('amazon.aws.aws_secret', 'ansible_password', bypath=true) | regex_search ('ansible_password\\\":\\\"(.*)\\\"','\\1')}}"
i retrieve the password ( i used a regex to get only the password, i'm not sure that is the correct way to do that)
when i want to use it i get this error:
fatal: [win_server]: FAILED! => {
"msg": "Invalid type for configuration option plugin_type: connection plugin: winrm setting: remote_password : Invalid type provided for \"string\": ['Itismypassword']"
}
thanks for your help !
The error message is pretty clear: list[str] != str; while I don't have an AWS Secret Manager to test against, thankfully the error message makes knowing the fix pretty obvious: grab the | first of the resulting list
ansible_password: "{{
lookup('amazon.aws.aws_secret', 'ansible_password', bypath=true)
| regex_search ('ansible_password\\\":\\\"(.*)\\\"','\\1')
| first }}"
As for your aside, the answer is for sure not to use regex to extract those values; based on the regex you're using, I'm guessing the output of that aws_secret lookup is in fact JSON, and thus the correct behavior is to | from_json to turn it back into a dict (or list[dict]), then extract the key you want
something like this, but without knowing the actual shape this is likely not 100% correct:
ansible_password: >-
{{ lookup('amazon.aws.aws_secret', 'ansible_password', bypath=true)
| from_json | map(attribute='ansible_password') | first }}
thanks to mdaniel i progress in my understanding of ansible and aws secret manager: it's working with regex but i try to figure out how to do it in a proper way...
when i get secret from aws secret manager with this commands :
debug: msg="{{ lookup('amazon.aws.aws_secret', 'ansible_secret', bypath=true) | dict2items }} "
i have this answer :
"msg": [
{
"key": "ansible_secret",
"value": "{\"password\":\"itismypassword\",\"user\":\"ansible_user\"}"
}
]
i don't know how to get "itismypassword" from value field >< a jquery maybe ?

A playbook with two roles: running role B complains with role A's code which successfully ran

I am experiencing a strange behavior: when I run role B, it complains role A's code which I can successfully run! I have reproduced this to this minimal example:
$ cat playbooka.yml
- hosts:
- host_a
roles:
- role: rolea
tags:
- taga
- role: roleb
tags:
- tagb
I have tagged two roles because I want to selectively run role A or role B, they consist simple tasks as shown below in this minimal example:
$ cat roles/rolea/tasks/main.yml
- name: Get service_facts
service_facts:
- debug:
msg: '{{ ansible_facts.services["amazon-ssm-agent"]["state"] }}'
- when: ansible_facts.services["amazon-ssm-agent"]["state"] != "running"
meta: end_play
$ cat roles/roleb/tasks/main.yml
- debug:
msg: "I am roleb"
The preview confirms that I can run individual roles as specified by tags:
$ ansible-playbook playbooka.yml -t taga -D -C --list-hosts --list-tasks
playbook: playbooka.yml
play #1 (host_a): host_a TAGS: []
pattern: ['host_a']
hosts (1):
3.11.111.4
tasks:
rolea : Get service_facts TAGS: [taga]
debug TAGS: [taga]
$ ansible-playbook playbooka.yml -t tagb -D -C --list-hosts --list-tasks
playbook: playbooka.yml
play #1 (host_a): host_a TAGS: []
pattern: ['host_a']
hosts (1):
3.11.111.4
tasks:
debug TAGS: [tagb]
I can run role A OK:
$ ansible-playbook playbooka.yml -t taga -D -C
PLAY [host_a] *************************************************************************************************************************************************************************************************************************************
TASK [Gathering Facts] ****************************************************************************************************************************************************************************************************************************
ok: [3.11.111.4]
TASK [rolea : Get service_facts] ******************************************************************************************************************************************************************************************************************
ok: [3.11.111.4]
TASK [rolea : debug] ******************************************************************************************************************************************************************************************************************************
ok: [3.11.111.4] => {
"msg": "running"
}
PLAY RECAP ****************************************************************************************************************************************************************************************************************************************
3.11.111.4 : ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
But when I run role B, it complains the code in role A which I just successfully ran!
$ ansible-playbook playbooka.yml -t tagb -D -C
PLAY [host_a] *************************************************************************************************************************************************************************************************************************************
TASK [Gathering Facts] ****************************************************************************************************************************************************************************************************************************
ok: [3.11.111.4]
ERROR! The conditional check 'ansible_facts.services["amazon-ssm-agent"]["state"] != "running"' failed. The error was: error while evaluating conditional (ansible_facts.services["amazon-ssm-agent"]["state"] != "running"): 'dict object' has no attribute 'services'
The error appears to be in '<path>/roles/rolea/tasks/main.yml': line 9, column 3, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- when: ansible_facts.services["amazon-ssm-agent"]["state"] != "running"
^ here
We could be wrong, but this one looks like it might be an issue with
unbalanced quotes. If starting a value with a quote, make sure the
line ends with the same set of quotes. For instance this arbitrary
example:
foo: "bad" "wolf"
Could be written as:
foo: '"bad" "wolf"'
I have two questions:
Why role A's code should be involved at all?
Even it gets involved, ansible_facts has services, and the service is "running" as shown above by running role A.
PS: I am using the latest Ansible 2.10.2 and latest python 3.9.1 locally on a MacOS. The remote python can be either 2.7.12 or 3.5.2 (Ubuntu 16_04). I worked around the problem by testing if the dictionary has the services key:
ansible_facts.services is not defined or ansible_facts.services["amazon-ssm-agent"]["state"] != "running"
but it still surprises me that role B will interpret role A's code and interpreted it incorrectly. Is this a bug that I should report?
From the notes in meta module documentation:
Skipping meta tasks with tags is not supported before Ansible 2.11.
Since you run ansible 2.10, the when condition for your meta task in rolea is always evaluated, whatever tag you use. When you use -t tagb, ansible_facts.services["amazon-ssm-agent"] does not exist as you skipped service_facts, and you then get the error you reported.
You can either:
upgrade to ansible 2.11 (might be a little soon as I write this answer since it is not yet available over pip...)
rewrite your condition so that the meta task skips when the var does not exists e.g.
when:
- ansible_facts.services["amazon-ssm-agent"]["state"] is defined
- ansible_facts.services["amazon-ssm-agent"]["state"] != "running"
The second solution is still a good practice IMO in whatever condition (e.g. share your work with someone running an older version, running accidentally against a host without the agent installed....).
One other possibility in your specific case is to move the service_facts tasks to an other role higher in play order, or in the pre_tasks section of your playbook, and tag it always. In this case the task will always play and the fact will always exists, whatever tag you use.

Ansible replace seems to throw a parsing error?

I'm trying to use the Ansible replace module to change some text in a file on a Windows Server 2019 Standard target. I'm using Ansible 2.8.3 running on Python 2.7
- name: REPLACE | Replace baseline.local with FQDN in InternetSettings.xml
replace:
path: '/path/to/settings.xml'
regexp: 'baseline\.local'
replace: '{{ FQDN }}'
I don't think the issue is the path, although one of the directories on the path to the file has brackets '{}' in its name. Could that be it?
I've tried to do the same thing with win_lineinfile, and it didn't throw an error with the same path, but it'd be difficult in this case to replicate the functionality of replace, which is really what I need.
EDIT 2: It works when I copy the file over to my local machine and delegate to 127.0.0.1. I'm running ansible from a Windows Subsystem for Linux (WSL) installation. It also works when I copy the file to a remote linux system and run replace there, so it seems to be a Windows problem.?
EDIT: The stack trace of the error I'm getting:
"Exception calling "Create" with "1" argument(s): "At line:4 char:21
+ def _ansiballz_main():
+ ~
An expression was expected after '('.
At line:13 char:27
+ except (AttributeError, OSError):
+ ~
Missing argument in parameter list.
At line:15 char:7
+ if scriptdir is not None:
+ ~
Missing '(' after 'if' in if statement.
At line:22 char:7
+ if sys.version_info < (3,):
+ ~
Missing '(' after 'if' in if statement.
At line:22 char:30
+ if sys.version_info < (3,):
+ ~
Missing expression after ','.
At line:22 char:25
+ if sys.version_info < (3,):
+ ~
The '<' operator is reserved for future use.
At line:24 char:32
+
My teammates and I suspect that this issue is the result of attempting to use an ansible module not built for windows on a windows target - more particularly, a target that doesn't have python installed. We suspect that some part of the compiling of python on the ansible controller or subsequent execution of the python binary on the windows target is what's really causing the problem here. We've decided on the workaround of just using win_shell with powershell's 'replace' to do what we need to do.
Try changing your regexp string to double quotes:
regexp: "baseline.local"

Ansible until loop with regex in conditional statement

I need to check command stdout with a regex pattern in until loop. When I using regex_search to display debug msg all works fine:
- name: Checking if node is ready
shell: "kubectl --kubeconfig {{kubeconf}} get nodes"
register: k_output
- debug:
msg: "{{k_output.stdout | regex_search(node_hostname | string + '\\s+Ready')}}"
The message is
ok: [any_node_hostname] => {
"msg": "any_node_hostname Ready"
}
But if I trying to use that construction inside until conditional statement task fails with syntax error.
- set_fact:
regexp_pattern: "{{node_hostname | string + '\\s+Ready'}}"
- debug:
msg: "{{regexp_pattern}}"
- name: Checking if node is ready
shell: "kubectl --kubeconfig {{kubeconf}} get nodes"
register: k_output
until: "{{k_output.stdout | regex_search(regexp_pattern)}}" != ""
retries: 5
delay: 10
The same behaviour without set_fact when I just copying and pasting full string {{k_output.stdout | regex_search(node_hostname | string + '\\s+Ready')}} to until conditional statement.
So, how can I use regex_search or something that fits this case with until?
you have syntax error at until: statement : you must not quote the vars in expression, like in example here : Retrying a task until a condition is met
until: k_output.stdout | regex_search(regexp_pattern) != ""
I hope this will help

Azure VM Extensions

I'm executing the following:
Set-AzureRmVMExtension `
-VMName 'servername' `
-ResourceGroupName 'rgname' `
-Name 'JoinAD' `
-ExtensionType 'JsonADDomainExtension' `
-Publisher 'Microsoft.Compute' `
-TypeHandlerVersion '1.0' `
-Location 'West Europe' `
-Settings #{'Name' = 'domain.com'; 'OUPath' = 'OU=Server 2012 R2,OU=Servers,DC=domain,DC=com'; 'User' = 'domain.com\username'; 'Restart' = 'true'; 'Options' = 3} `
-ProtectedSettings #{'Password' = 'password'}
and get this error:
Set-AzureRmVMExtension : Long running operation failed with status
'Failed'. StartTime: 18.04.2016 18:03:30 EndTime: 18.04.2016 18:04:50
OperationID: 76825458-6c50-404d-bb1a-b27c722b1760 Status: Failed
ErrorCode: VMExtensionProvisioningError ErrorMessage: VM has reported
a failure when processing extension 'JoinAD'. Error message: "Join
completed for Domain 'ddomain.com'". At line:1 char:1
+ Set-AzureRmVMExtension `
+ ~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : CloseError: (:) [Set-AzureRmVMExtension], ComputeCloudException
+ FullyQualifiedErrorId : Microsoft.Azure.Commands.Compute.SetAzureVMExtensionCommand
What am I missing?
Kept having trouble with extension, therefore opted to perform the domain join using PowerShell Add-Computer without extensions.
One possibly cause might be that NSG configurations block connectivity to the internet and with that to services running in the Azure data center.