repo/
├─ cells/
│ ├─ cell1/
│ │ ├─ enabled-sites/
│ │ │ ├─ example2.conf
│ │ │ ├─ example1.conf
│ │ ├─ site.conf
├─ workgroups/
│ ├─ wg1/
│ │ ├─ enabled-sites/
│ │ │ ├─ example3.conf
│ │ │ ├─ example4.conf
│ │ │ ├─ example5.conf
│ │ ├─ site.conf
I'm trying to write a role which will template the first things it finds
All the above would exist on the ansible host not the remote
So in the above example if repo/cells/cell1/enabled-sites contains files they'll be templated to a remote machine
If they don't exist it would look for repo/cells/cell1/site.conf which would be templated
If that didn't exist it would look for files in repo/workgroups/wg1/enabled-sites
If that didn't exist if would look for ropo/workgroups/wg1/site.conf
Finally if none of those exist it would use the roles template site.conf
- name: Configure | Find first enabled-sites
find:
paths: "{{ item }}"
with_first_found:
- "{{ cvs_path }}/scripts/def/cells/{{ grouping.name }}/enabled-sites/"
- "{{ cvs_path }}/scripts/def/cells/{{ grouping.name }}/site.conf"
- "{{ cvs_path }}/scripts/def/workgroups/{{ grouping.workgroup }}/enabled-sites/"
- "{{ cvs_path }}/scripts/def/cells/{{ grouping.name }}/site.conf"
- "../templates/site.conf"
register: esites
delegate_to: localhost
run_once: true
- name: Configure | Template out found cell specific enabled-sites
template:
src: "{{ item.path }}"
dest: "{{ httpd_home }}/conf/enabled-sites/{{ item.path | basename }}"
with_items: "{{ esites.results | map(attribute='files') | list }}"
This only works for the directories, the find doesn't work on files
There seems to be another option within "with_first_found" loop.
Please have a look at bellow example
- name: Include tasks only if one of the files exist, otherwise skip the task
ansible.builtin.include_tasks:
file: "{{ item }}"
with_first_found:
- files:
- path/tasks.yaml
- path/other_tasks.yaml
There is another attribute as "paths" similar to "files" not sure can we combine them together.
https://docs.ansible.com/ansible/latest/collections/ansible/builtin/first_found_lookup.html
The best I've come up with is:
- name: Configure | Find first enabled-sites
find:
paths: "{{ item }}"
with_first_found:
- "{{ cvs_path }}/scripts/def/cells/{{ grouping.name }}/enabled-sites/"
- "{{ cvs_path }}/scripts/def/workgroups/{{ grouping.workgroup }}/enabled-sites/"
register: esites
delegate_to: localhost
run_once: true
ignore_errors: true
- name: Configure | Template out found cell specific enabled-sites
template:
src: "{{ item.path }}"
dest: "{{ httpd_home }}/conf/enabled-sites/{{ item.path | basename }}"
with_items: "{{ esites.results | map(attribute='files') | list }}"
when: esites.results is defined
- name: Configure | Template out single site.conf
template:
src: "{{ item }}"
dest: "{{ httpd_home }}/conf/enabled-sites/{{ grouping.environment }}{{ grouping.workgroup }}.conf"
with_first_found:
- "{{ cvs_path }}/scripts/def/cells/{{ grouping.name }}/site.conf"
- "{{ cvs_path }}/scripts/def/workgroups/{{ grouping.workgroup }}/site.conf"
- "site.conf"
when: esites.results is not defined
It doesn't quite get the ordering correct but gets the job done
Related
I have Ansible PHP role with defined versions and pools:
php_versions:
- 8.0
- 8.1
php_pools:
- name: wiki
version: 8.0
#...
- name: mail
version: 8.1
#...
It produces files /etc/php/8.0/fpm/pool.d/wiki.conf and /etc/php/8.1/fpm/pool.d/mail.conf plus /etc/php/*/fpm/pool.d/_default_pool.conf which role creates by default for every defined PHP version.
Everything works great except deleting and moving pools. If I need to switch the pool wiki from version 8.0 to version 8.1, it crashes. Because the old file /etc/php/8.0/fpm/pool.d/wiki.conf stay on the disk and the new file /etc/php/8.1/fpm/pool.d/wiki.conf will be created in different PHP directory - it crashes on the IP:port conflict (already used by old PHP version).
I need to delete all pools on the disk that are not defined in ansible except _default_pool.conf. Default pools must stay.
I tried:
# main.yml
- name: php versions
include_tasks: php.yml
loop: "{{ php_versions }}"
loop_control:
loop_var: php_version
# php.yml
- name: delete non-ansible pools
become: True
block:
- name: find all exiting php pool files
become: True
find:
paths: "/etc/php/{{ inner_item.php_version }}/fpm/pool.d/"
patterns: '\/etc\/php\/[0-9]\.[0-9]\/fpm\/pool\.d\/(.+(?<!_default_pool))\.conf$'
use_regex: True
register: existing_pool_files
- name: delete non-ansible pool files
become: True
file:
state: absent
path: "{{ item['path'] }}"
with_items:
- "{{ existing_pool_files['files'] | intersect(inner_item.name) }}"
notify: restart PHP
but it doesn't work.
I can delete all pool files on disk and recreate them. But it sounds stupid.
How can I fix it? After a few hours of debugging, I'm out of ideas :-(
For example, given the lists
php_versions: [8.0, 8.1]
php_pools:
- {version: 8.0, name: wiki}
- {version: 8.1, name: mail}
and the tree
shell> tree /etc/php
/etc/php
├── 7.4
│ └── fpm
│ └── pool.d
│ └── keep_this.conf
├── 8.0
│ └── fpm
│ └── pool.d
│ ├── _default_pool.conf
│ ├── mail.conf
│ ├── trash.conf
│ └── wiki.conf
└── 8.1
└── fpm
└── pool.d
├── _default_pool.conf
├── mail.conf
├── trash.conf
└── wiki.conf
Create the list of paths you want to maintain
present_paths_str: |
{% for ver in php_versions %}
- /etc/php/{{ ver }}/fpm/pool.d/
{% endfor %}
present_paths: "{{ present_paths_str|from_yaml }}"
gives
present_paths:
- /etc/php/8.0/fpm/pool.d/
- /etc/php/8.1/fpm/pool.d/
Create the list of present files
php_groups: "{{ dict(php_pools|groupby('version')) }}"
present_pool_files_str: |
{% for ver in php_versions %}
- /etc/php/{{ ver }}/fpm/pool.d/_default_pool.conf
{% for i in php_groups[ver] %}
- /etc/php/{{ ver }}/fpm/pool.d/{{ i.name }}.conf
{% endfor %}
{% endfor %}
present_pool_files: "{{ present_pool_files_str|from_yaml }}"
gives
present_pool_files:
- /etc/php/8.0/fpm/pool.d/_default_pool.conf
- /etc/php/8.0/fpm/pool.d/wiki.conf
- /etc/php/8.1/fpm/pool.d/_default_pool.conf
- /etc/php/8.1/fpm/pool.d/mail.conf
Declare variables
existing_pool_files: "{{ existing_pool.files|map(attribute='path')|list }}"
delete_pool_files: "{{ existing_pool_files|difference(present_pool_files) }}"
and find files
- name: find all exiting php pool files
become: True
find:
paths: "{{ present_paths }}"
patterns: '*.conf'
register: existing_pool
gives
existing_pool_files:
- /etc/php/8.0/fpm/pool.d/wiki.conf
- /etc/php/8.0/fpm/pool.d/_default_pool.conf
- /etc/php/8.0/fpm/pool.d/trash.conf
- /etc/php/8.0/fpm/pool.d/mail.conf
- /etc/php/8.1/fpm/pool.d/wiki.conf
- /etc/php/8.1/fpm/pool.d/_default_pool.conf
- /etc/php/8.1/fpm/pool.d/trash.conf
- /etc/php/8.1/fpm/pool.d/mail.conf
delete_pool_files:
- /etc/php/8.0/fpm/pool.d/trash.conf
- /etc/php/8.0/fpm/pool.d/mail.conf
- /etc/php/8.1/fpm/pool.d/wiki.conf
- /etc/php/8.1/fpm/pool.d/trash.conf
Delete the redundant files
- name: delete non-ansible pool files
become: True
file:
state: absent
path: "{{ item }}"
loop: "{{ delete_files }}"
notify: restart PHP
Example of a complete playbook for testing
- hosts: localhost
vars:
php_versions: [8.0, 8.1]
php_pools:
- {version: 8.0, name: wiki}
- {version: 8.1, name: mail}
present_paths_str: |
{% for ver in php_versions %}
- /etc/php/{{ ver }}/fpm/pool.d/
{% endfor %}
present_paths: "{{ present_paths_str|from_yaml }}"
php_groups: "{{ dict(php_pools|groupby('version')) }}"
present_pool_files_str: |
{% for ver in php_versions %}
- /etc/php/{{ ver }}/fpm/pool.d/_default_pool.conf
{% for i in php_groups[ver] %}
- /etc/php/{{ ver }}/fpm/pool.d/{{ i.name }}.conf
{% endfor %}
{% endfor %}
present_pool_files: "{{ present_pool_files_str|from_yaml }}"
existing_pool_files: "{{ existing_pool.files|map(attribute='path')|list }}"
delete_pool_files: "{{ existing_pool_files|difference(present_pool_files) }}"
tasks:
- debug:
var: php_groups
- debug:
var: present_paths
- debug:
var: present_pool_files
- name: find all exiting php pool files
become: True
find:
paths: "{{ present_paths }}"
patterns: '*.conf'
register: existing_pool
- debug:
var: existing_pool_files
- debug:
var: delete_pool_files
- name: delete non-ansible pool files
become: True
file:
state: absent
path: "{{ item }}"
loop: "{{ delete_files }}"
notify: restart PHP
I'm new to Ansible. I have many router config templates to generate but want to specify in the playbook which specific template should be generated. I'm showing only two template files here to keep things simple. I'm able to generate configs based on my setup, all works well. However, I don't know how to execute a single template file within roles in the site.yaml playbook. Here's my directory structure:
├── roles
│ ├── router
│ ├── tasks
│ │ └── main.yaml
│ ├── templates
│ │ ├── 4331-router.j2
│ │ ├── 881-router.j2
│ │ └── base.j2
│ └── vars
│ └── main.yaml
│
└── site.yaml
Here's how site.yaml playbook is constructed:
---
- name: Generate Router Configuration Files
hosts: localhost
roles:
- router
Here's the main.yaml in the tasks folder:
---
- name: Generate 4331 configuration files
template: src=4331-router.j2 dest=~/ansible/configs/{{item.hostname}}.txt
with_items: "{{ routers_4331 }}"
- name: Generate 881 configuration files
template: src=881-router.j2 dest=~/ansible/configs/{{item.hostname}}.txt
with_items: "{{ routers_881 }}"
When I run the playbook it generates all config templates. I want to be able to specify which config template to render, for example: routers_4331 or routers_881.
How can I specify this in the playbook?
i suppose you have a link between list of hostnames and file.j2 (same number for example)
- find:
path: "path of templates" # give the right folder of templates
file_type: file
patterns: '[0-9]+-.*?\.j2' #search files format xxxx-yyyy.j2
use_regex: yes
register: result
- set_fact:
hosting: "{{ hosting | d([]) + _dico }}"
loop: "{{ result.files | map(attribute='path') | list }}"
vars:
_src: "{{ item }}"
_num: "{{ (_src | basename).split('-') | first }}"
_grp: "{{ 'routers' ~ '_' ~ _num }}"
_hosts: "{{ lookup('vars', _grp) }}"
_dico: >-
{%- set ns = namespace() -%}
{%- set ns.l = [] -%}
{%- for h in _hosts -%}
{%- if h.update({'pathj2': _src}) -%}{%- endif -%}
{%- set ns.l = ns.l + [h] -%}
{%- endfor -%}
{{ ns.l }}
- name: Generate configuration files
template:
src: "{{item.pathj2}}"
dest: ~/ansible/configs/{{item.hostname}}.txt
loop: "{{ hosting }}"
the first task selects files j2 from folder templates (following the regex pattern)
the second task add the corresponding path of file j2 to the file containing the hostnames
Can somone explain to me why this doesn't work? I want to get a list of files within a directory and use it as an input for the loop.
---
tasks:
- set_fact:
capabilities: []
- name: find CE_Base capabilities
find:
paths: /opt/netsec/ansible/orchestration/capabilities/CE_BASE
patterns: '*.yml'
register: CE_BASE_capabilities
- name: debug_files
debug:
msg: "{{ item.path }}"
with_items: "{{ CE_BASE_capabilities.files }}"
- set_fact:
thispath: "{{ item.path }}"
capabilities: "{{ capabilities + [ thispath ] }}"
with_items: "{{ CE_BASE_capabilities.files }}"
- name: Include CE_BASE
include_tasks: /opt/netsec/ansible/orchestration/process_capabilities_CE_BASE.yml
loop: "{{ capabilities }}"
Edit:
This code is attempting to create a list called capabilties, which contatins a list of files in a particular directory.
When i ran this code without trying to get the files automatically, it looked like this.
- hosts: localhost
vars:
CE_BASE_capabilities:
- '/opt/netsec/ansible/orchestration/capabilities/CE_BASE/CE_BASE_1.yml'
- '/opt/netsec/ansible/orchestration/capabilities/CE_BASE/CE_BASE_2.yml'
- name: Include CE_BASE
include_tasks: /opt/netsec/ansible/orchestration/process_capabilities_CE_BASE.yml
loop: "{{ CE_BASE_capabilities }}"
Don't define thispath as a fact but as a local vars in the set_fact task. Beside that, you don't need to init capabilities if you use the default filter.
- vars:
thispath: "{{ item.path }}"
set_fact:
capabilities: "{{ capabilities | default([]) + [ thispath ] }}"
with_items: "{{ CE_BASE_capabilities.files }}"
Moreover, you don't even need to loop. You can extract the info directly from the existing result:
- set_fact:
capabilities: "{{ CE_BASE_capabilities.files | map(attribute='path') | list }}"
This is my first time using nested Helm charts and I'm trying to access a global value from the root values.yaml file. According to the documentation I should be able to use the syntax below in my secret.yaml file, however if I run helm template api --debug I get the following error:
Error: template: api/templates/secret.yaml:7:21: executing "api/templates/secret.yaml" at <.Values.global.sa_json>: nil pointer evaluating interface {}.sa_json
helm.go:84: [debug] template: api/templates/secret.yaml:7:21: executing "api/templates/secret.yaml" at <.Values.global.sa_json>: nil pointer evaluating interface {}.sa_json
/primaryChart/charts/api/templates/secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: {{ .Chart.Name }}-service-account-secret
type: Opaque
data:
sa_json: {{ .Values.global.sa_json }}
primaryChart/values.yaml
global:
sa_json: _b64_sa_credentials
Folder structure is as follows:
/primaryChart
|- values.yaml
|-- /charts
|-- /api
|-- /templates
|- secret.yaml
Having the following directory layout, .Values.global.sa_json will only be available if you call helm template api . from your main chart
/mnt/c/home/primaryChart> tree
.
├── Chart.yaml <-- your main chart
├── charts
│ └── api
│ ├── Chart.yaml <-- your subchart
│ ├── charts
│ ├── templates
│ │ └── secrets.yaml
│ └── values.yaml
├── templates
└── values.yaml <--- this is where your global.sa_json is defined
Your values file should be called values.yaml and not value.yaml, or use any other file with -f flag helm template api . -f value.yaml
/mnt/c/home/primaryChart> helm template api .
---
# Source: primaryChart/charts/api/templates/secrets.yaml
apiVersion: v1
kind: Secret
metadata:
name: api-service-account-secret
type: Opaque
data:
sa_json: _b64_sa_credentials
So I want to run taskcat tests with a parameter file but it doesn't seem to work, I have a setup like this but I keep getting errors:
├── .taskcat.yml
├── ci
│ ├── parameters.json
│ └── taskcat.yml
├── templates
└── sqs-yaml.template
-taskcat.yml is:
global:
qsname: sample-taskcat-project
regions:
- us-east-1
tests:
taskcat-yaml:
parameter_input: parameters.json
template_file: sqs.yml
regions:
- us-east-1
-.taskcat.yml is:
global:
qsname: sample-taskcat-project
regions:
- us-east-1
tests:
taskcat-yaml:
parameter_input: parameters.json
template_file: sqs.yml
regions:
- us-east-1
-parameters.json is:
[
{
"ParameterKey": "MyQueueName",
"ParameterValue": "TestQueue"
}
]
-sqs-yaml.template is:
AWSTemplateFormatVersion: '2010-09-09'
Description: Creates an SQS Queue.
Parameters:
MyQueueName:
Description: My Queue Name
Type: String
Resources:
MyQueue:
Type: AWS::SQS::Queue
Properties:
MyQueueName
Outputs:
MyQueueARN:
Value:
Ref: MyQueue
#Guzdo - Thanks for using taskcat. Github is the best support mechanism for taskcat.
That said, in v0.9.x, the entire configuration is now within the single config file. Looks like it's been auto-generated for you - have a look at '.taskcat.yml'
Here are a few examples:
project:
name: my-cfn-project
az_blacklist:
- use1-az1
build_submodules: false
lambda_source_path: functions/source
lambda_zip_path: functions/packages
owner: me#example.com
package_lambda: false
parameters:
KeyPairName: blah
project:
(...)
tests:
my-example-test:
(...)
parameters:
KeyPairName: blah
More comprehensive example config:
https://raw.githubusercontent.com/taskcat/taskcat/master/tests/data/config_full_example/.taskcat.yml