Ansible Regex Expression - regex

I'm looking to find all ports with a certain status on a Cisco router to shut them down. I was hoping to do this via std.out and using regex but have no idea on the regex syntax.
For example, output for the show command will look something like the below output.
Port Device State Call DN Dev State
---------- --------------- -------- ------------- ------- ----------- ----
0/1/0 DEV0001 IS IDLE 2344 ATT
0/1/1 DEV0002 IS IDLE 2567 ATT
0/1/2 DEV0002 IS IDLE 2567 DEL
What I'd like to do is to store the port numbers which has Dev State = ATT in a variable so I can shut them down. In Cisco I can filter the show command to say - show port | include ATT - this will only list the ports that contain the Dev State ATT but it wont show any of the Column heading in the output. From this output, I then need to loop through and store the port numbers. Hope this makes sense.
Appreciate any help. Thank you.
Ansible Script:
tasks:
- name: show port
ios_command:
commands:
- show port summary | incl ATT
register: config
- set_fact
myvalue: ""{{ config.stdout | regex_search(??) }}""
when config.stdout | length > 0
Output of Debug config:
"stdout_lines": [
[
"Total Devices: 4",
"Total Calls in Progress: 0",
"Total Call Legs in Use: 0",
"",
"Port Device Device Call Dev Directory Dev ",
"Identifier Name State State Type Number Cntl ",
"---------- --------------- -------- ------------- ------- ----------- ---- ",
"0/1/0 DEV0001 IS IDLE ALG 3880 DEL",
"0/1/1 DEV0002 IS IDLE ALG 3881 ATT",
"0/1/2 DEV0003 IS IDLE ALG ATT",
"0/1/3 DEV0004 IS IDLE ALG 3882 DEL"
]
]
} ]

There should be config.stdout_lines along with config.stdout
It's not necessary to test the list. The loop will be skipped if the list is empty.
Lines with data are selected by the pattern '^\d/\d/\d\s*(.*)$'
It's possible to customize the result by changing the when condition and changing the list of myitems added the list myvalues
The tasks below
- set_fact:
myvalues: "{{ myvalues|default([]) + [myitems.4] }}"
loop: "{{ config.stdout_lines|select('regex', myregex)|list }}"
vars:
myregex: '^\d/\d/\d\s*(.*)$'
myitems: "{{ item.split() }}"
when: myitems.5 == "ATT"
- debug:
var: myvalues
give
"myvalues": [
"2344",
"2567"
]
Generally, it's possible to create a list with the names of the columns and create ditionaries with selected data. For example
- set_fact:
cols: "{{ config.stdout_lines.0.split() }}"
- set_fact:
myvalues: "{{ myvalues|default([]) +
[dict(cols|zip(myitems))] }}"
loop: "{{ config.stdout_lines|select('regex', myregex)|list }}"
vars:
myregex: '^\d/\d/\d\s*(.*)$'
myitems: "{{ item.split() }}"
when: myitems.5 == "ATT"
- debug:
var: myvalues
gives
"myvalues": [
{
"Call": "IDLE",
"DN": "2344",
"Dev": "ATT",
"Device": "DEV0001",
"Port": "0/1/0",
"State": "IS"
},
{
"Call": "IDLE",
"DN": "2567",
"Dev": "ATT",
"Device": "DEV0002",
"Port": "0/1/1",
"State": "IS"
}
]
Then any combination of the data can be selected. For example
- set_fact:
myports: "{{ myvalues|json_query('[].Port') }}"
- debug:
var: myports
give the list of the Ports
"myports": [
"0/1/0",
"0/1/1"
]

Related

Create list with jijna2 regex_search - NoneType problem

I'm facing 2 problems:
Problem 1.
I'm trying to filter a list with jinja2 regex_search but I alse get None matches.
Problem 2.
Each elements of the new list seems to be list of one element (sigth!!!).
My code.
- name: Regex_Search Test
hosts: localhost
vars:
my_list:
- app-be-dev01-2
- app-be-dev02-2
- app-be-dev02-3
- app-be-dev03-2
- app-foo-2
- app-be-dev04-1
- app-be-dev04-2
tasks:
- name: Varsmng
set_fact:
customer_instances: >-
{% for instance in my_list -%} {{ customer_instances | default([]) + [ instance | string | regex_search('app-be-(.*)-([0-9]*)', '\1' ) ] }}
{%- endfor %}
- name: Debug
debug:
msg:
- "customer_instances: {{ customer_instances }}"
My output.
TASK [Varsmng] ****************************************************************************************************************************************
task path: /home/cin0633a/progetti/ansible/testenv/test.yml:19
ok: [localhost] => {
"ansible_facts": {
"customer_instances": "[[u'dev01']][[u'dev02']][[u'dev02']][[u'dev03']][None][[u'dev04']][[u'dev04']] "
},
"changed": false
As you can see, each element has a double square brackets. And can I avoid None values?
You get a list for each element because regex_search returns a list when you use the replace feature with capture groups in the expression.
$ ansible localhost -m debug -e toto=bla-bli-blo -a "msg={{ toto | regex_search('(bla).*') }}"
localhost | SUCCESS => {
"msg": "bla-bli-blo"
}
$ ansible localhost -m debug -e toto=bla-bli-blo -a "msg={{ toto | regex_search('(bla).*', '\\1') }}"
localhost | SUCCESS => {
"msg": [
"bla"
]
}
And you get None values because some items do not match your regex.
You can get your result with a better approach IMO using specific filters rather than a complex jinja2 template. The following playbook:
- name: Regex_Search Test
hosts: localhost
gather_facts: false
vars:
my_list:
- app-be-dev01-2
- app-be-dev02-2
- app-be-dev02-3
- app-be-dev03-2
- app-foo-2
- app-be-dev04-1
- app-be-dev04-2
searchreg: >-
app-be-(.*)-([0-9]*)
my_filtered_list: >-
{{
my_list |
select('regex', searchreg) |
map('regex_replace', searchreg, '\1')
}}
tasks:
- debug:
var: my_filtered_list
Gives:
PLAY [Regex_Search Test] *****************************************************************************
TASK [debug] *****************************************************************************
ok: [localhost] => {
"my_filtered_list": [
"dev01",
"dev02",
"dev02",
"dev03",
"dev04",
"dev04"
]
}
PLAY RECAP *****************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Note1: you can pipe the unique filter if you want unique environment results
Note2: if you are using ansible < 2.10, you will have to add the list filter at the end of all others to get the actual result.

Ansibe Jinja2 Filter

I want to apply filter to the following log file. But I keep missing something
task.yml
- ansible_loop_var: item
changed: false
failed: true
invocation:
module_args:
policy_package: Package1
version: null
wait_for_task: true
wait_for_task_timeout: 30
item: PackageItem1
msg: Task Verify policy operation with task id 01234567-89ab-cdef-a422-xxxxxxxxx
failed. Look at the logs for more details
- ansible_loop_var: item
changed: false
failed: true
invocation:
module_args:
policy_package: Package2
version: null
wait_for_task: true
wait_for_task_timeout: 30
item: PackageItem2
msg: Task Verify policy operation with task id 01234567-89ab-cdef-a6c4-xxxxxxxx
failed. Look at the logs for more details
Here is my playbook
filter.yml
---
- name: sftp-domain
hosts: check_point
connection: httpapi
gather_facts: False
vars_files:
- 'credentials/my_var.yml'
- 'credentials/login.yml'
tasks:
- name: set-details
set_fact:
filter: "{{ lookup('file', 'tmp/task.yml') }}"
- name: create list to loop through
set_fact:
new_list: "{{ filter | map(attribute='msg') | flatten }}"
- name: copy-file-to-log
local_action:
module: copy
content: "{{ new_list | to_nice_yaml }}"
dest: tmp/task2.yml
I get an error message saying
PLAY [sftp-domain] *******************************************************************************************************************************************
TASK [set-details] *******************************************************************************************************************************************
ok: [checkpoint]
TASK [create list to loop through] ***************************************************************************************************************************
fatal: [checkpoint]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'unicode object' has no attribute 'msg'\n\nThe error appears to be in '/home/tdeveu0/project/fwp_automation/filter.yml': line 15, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n - name: create list to loop through\n ^ here\n"}
Here is actually the result I want after applying filter
- msg: Task Verify policy operation with task id 01234567-89ab-cdef-a6c4-xxxxxxxx
failed. Look at the logs for more details
- msg: Task Verify policy operation with task id 01234567-89ab-cdef-a422-xxxxxxxxx
failed. Look at the logs for more details
I want to get only the list of all the 'msg'
Use the filter from_yaml
- name: set-details
set_fact:
filter: "{{ lookup('file', 'tmp/task.yml')|from_yaml }}"
Let's take a simplified file to show the problem, e.g.
shell> cat task.yml
- a: 1
b: 2
- a: 3
b: 4
When you read the file into the variable the result is AnsibleUnsafeText, not a list
- set_fact:
filter: "{{ lookup('file', 'task.yml') }}"
- debug:
msg: "{{ filter|type_debug }}"
- debug:
var: filter.0
gives
msg: AnsibleUnsafeText
filter.0: '-'
The first item of the text is the dash '-'.
Use filter from_yaml to get the list
- set_fact:
filter: "{{ lookup('file', 'task.yml')|from_yaml }}"
- debug:
msg: "{{ filter|type_debug }}"
- debug:
var: filter.0
gives
msg: list
filter.0:
a: 1
b: 2

Ansible: Unique Shortest Paths

Context
I have a list of files and directories to be deleted.
This is obtained from lines starting with the word "deleting" from an rsync stdout.
rsync stdout_lines: [
"building file list ... done",
"*deleting Lab02/Ex2/Doc1.txt",
"*deleting Lab02/Ex2/",
"*deleting Lab02/Ex1/Doc1.txt",
"*deleting Lab02/Ex1/",
"*deleting Lab02/",
".d..t...... ./",
"*deleting Lab01/Ex2/Doc1.txt",
"*deleting Lab01/Ex2/",
"*deleting Lab01/Ex1/Doc2.txt",
"*deleting Lab01/Ex1/Doc1.txt",
".d..t...... Lab01/",
".d..t...... Lab01/Ex1/",
"sent 350 bytes received 191 bytes 360.67 bytes/sec",
"total size is 614 speedup is 1.13 (DRY RUN)"
]
Formatted using:
'{{ sync_return.stdout_lines | select("regex", "^[*]deleting") | map("regex_replace", "^[*]deleting", "") | map("regex_replace", " ", "") | list }}'
A primitive example of the format of this list is as follows:
formatted list: [
"Lab02/Ex2/Doc1.txt",
"Lab02/Ex2/",
"Lab02/Ex1/Doc1.txt",
"Lab02/Ex1/",
"Lab02/",
"Lab01/Ex2/Doc1.txt",
"Lab01/Ex2/",
"Lab01/Ex1/Doc2.txt",
"Lab01/Ex1/Doc1.txt"
]
In an attempt to hasten the process of deleting (by reducing the number of elements to iterate over) - I separated the list into 2 sub-lists:
A list of directories. (elements of the main list that end in '/')
'{{ items_to_delete | select("regex", "/$") | list }}'
A list of file paths. (elements who's containing directory does not get deleted)
'{{ items_to_delete | reject("match", item) | list }}'
The sub-lists for the example above would be...
directories to delete: [
"Lab02/Ex2/",
"Lab02/Ex1/",
"Lab02/",
"Lab01/Ex2/"
]
files to delete: [
"Lab01/Ex1/Doc2.txt",
"Lab01/Ex1/Doc1.txt"
]
The Problem
Whilst the current solution works, it's not the best it could be. The dream is to have a solution where the "directories to delete" list only contains the highest level directories possible. i.e. Since we know the directory "Lab02/" is being deleted, "directories to delete" will NOT contain "Lab02/Ex2/" or "Lab02/Ex1/".
I believe my goal is somewhat similar to the os.path.commonprefix python function, however must be done for a variety of file paths within the list.
I'm relatively new to Ansible, so any guidance/help with this matter would be greatly appreciated.
I won't ask why you want to implement that, and I'll take it as an exercise.
Idea is, you can sort directories alphabetically, then while looping the paths, you strip any one that starts with the previous line.
You can write your filter like this (put in filter_plugins directory):
def common_paths(paths=[]):
sorted_paths = sorted(paths)
pfx = sorted_paths[0]
for path in sorted_paths[1:]:
if re.compile("^%s.*" % pfx).match(path):
sorted_paths.remove(path)
else:
pfx = path
return sorted_paths
class FilterModule(object):
def filters(self):
return { 'common_paths': common_paths }
then:
- name: Filter
set_fact:
bar: "{{ foo | common_paths }}"
Test locally with:
---
- hosts: localhost
tasks:
- name: Test data
set_fact:
foo:
- 'Lab01/'
- 'Lab01/Ex5/'
- 'Ex2/foo3/'
- 'Ex2/foo2/'
- 'Ex2/'
- 'Lab03/Ex5/e/'
- 'Lab02/y/z/Lab01/1/'
- 'Lab02/y/z/Lab01/3/'
- 'Lab01/Ex5/Lab02/'
- 'Lab03/Ex5/d/1'
- name: Filter
set_fact:
bar: "{{ foo | common_paths }}"
Output:
$ ansible-playbook common_paths.yml -vvv
ansible-playbook 2.10.4
PLAYBOOK: common_paths.yml *********************************************************************************
1 plays in common_paths.yml
PLAY [localhost] *********************************************************************************
TASK [Gathering Facts] *********************************************************************************
ok: [localhost]
TASK [Test data] ********************************************************************************* task path: /home/guido/Development/git/ansible-local/common_paths.yml:5
ok: [localhost] => {
"ansible_facts": {
"foo": [
"Lab01/",
"Lab01/Ex5/",
"Ex2/foo3/",
"Ex2/foo2/",
"Ex2/",
"Lab03/Ex5/e/",
"Lab02/y/z/Lab01/1/",
"Lab02/y/z/Lab01/3/",
"Lab01/Ex5/Lab02/",
"Lab03/Ex5/d/1/"
]
},
"changed": false
}
TASK [Filter] *********************************************************************************
task path: /home/guido/Development/git/ansible-local/common_paths.yml:19
ok: [localhost] => {
"ansible_facts": {
"bar": [
"Ex2/",
"Lab01/",
"Lab02/y/z/Lab01/1/",
"Lab02/y/z/Lab01/3/",
"Lab03/Ex5/d/1/",
"Lab03/Ex5/e/"
]
},
"changed": false
}
PLAY RECAP *********************************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

From a list of dictionaries, create one dictionary which members contain a list

Having the following list of dictionaries:
- set_fact:
inventory:
- dn: host1
lid: 0021-00
unit_id: 50
- dn: host2
lid: 1011-00
unit_id: 50
- dn: host2
lid: 1004-00
unit_id: 50
Iterating over that list I'd like to create a new
dictionary as like (dn becomes key and lid becomes list value):
- set_fact:
dn2lid:
host1:
- 0021-00
host2:
- 1011-00
- 1004-00
Your help is much appreciated.
The task below does the job
- set_fact:
dn2lid: "{{ dn2lid|default({})|combine({item.0: lids}) }}"
loop: "{{ inventory|groupby('dn') }}"
vars:
lids: "{{ item.1|map(attribute='lid')|list }}"
- debug:
var: dn2lid
gives
dn2lid:
host1:
- 0021-00
host2:
- 1011-00
- 1004-00

Ansible Regex Help needed: Unable to grab output and assign to a set_fact variable

I am trying to grab the "CRC Input Error" count and assign it to a variable "crc_count" however, the value of this variable is showing as an empty string instead ("crc_count": ""). What am I missing here?
Code:
- name: Checking CRC Errors on the Interfaces
nxos_command:
commands:
- show interface ethernet 1/1 counters detailed all
register: crc_output
- debug: var=crc_output.stdout_lines[0][50]
- set_fact: crc_count= "{{ crc_output.stdout_lines[0][50] | regex_search('(\d{0,20}$)') }}"
- debug:
msg: "{{ crc_count }}"
String output that I want to run regex on:
"crc_output.stdout_lines[0][50]": " 10. Input CRC Errors: = 4272"
Output I want to grab:
crc_count = "4272"
Playbook verbose output:
TASK [debug] **************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************
task path: /ansible/saurasar/nxos.crc.check.yml:19
ok: [ord12-pob-c1u1-dci-1.uspp1.oraclecloud.com] => {
"crc_output.stdout_lines[0][50]": " 10. Input CRC Errors: = 4272"
}
TASK [set_fact] ***********************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************
task path: /ansible/xxxx/nxos.crc.check.yml:20
ok: [cisco.nexus.switch] => {"ansible_facts": {"_raw_params": "\"4272\"", "crc_count": ""}, "changed": false}
TASK [debug] **************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************
task path: /ansible/xxxxx/nxos.crc.check.yml:22
ok: [cisco.nexus.switch] => {
"msg": ""
}
META: ran handlers
META: ran handlers
PLAY RECAP ****************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************
<cisco.nexus.switch> : ok=5 changed=0 unreachable=0 failed=0
Try as below
- set_fact:
crc_count: "{{ result.stdout | regex_replace('^.*Input CRC Errors: = (.*)$', '\\1') }}"
I tried with your sample and got as below
ok: [localhost] => {
"ansible_facts": {
"crc_count": "4272"
},
"changed": false
}