For motives of backwards compatibility (and to reduce the burden of having to update src code) follows:
I have an user input:
$values.yaml
output:
endpoint: MYENDPOINT
specialFile:
config1: |-
Some very nicely formated text.
Wow, such sample.
Much text.
[...]
config2: |-
# this is also text but i need to remove |- for interpretation
{{- $.Files.Get "configs/include.conf" | nindent 4 }}
That i want to replace with a blob of text:
$configs/include.conf
<match **>
#type http
endpoint_url {{ .Values.output.endpoint }}
serializer json
</match>
So that output replaces var on text and text replaces value on values.yaml. Is this possible?
I know this is something that is pretty close to impossible. I already saw someone replacing values.yaml with itself (didnt try myself) but is this possible?
Or can i overwrite the .Values.specialFile.configs2 value without writing directly into the .Values file and overwriting the variable somehow, maybe using templates?
This is possible by processing the included file as nested template with tpl:
{{ $endpoint := "MYENDPOINT" }}
output:
endpoint: {{ $endpoint }}
specialFile:
config1: |-
Some very nicely formated text.
Wow, such sample.
Much text.
[...]
config2: |-
# this is also text but i need to remove |- for interpretation
{{- tpl ($.Files.Get "configs/include.conf") $endpoint | nindent 4 }}
To avoid repetition, I put the endpoint in a variable and passed it to the loaded template. Therefore, it will be available as {{.}} there:
<match **>
#type http
endpoint_url {{ . }}
serializer json
</match>
Related
I am beginner in helm chart templating and I'm looking for best practise.
I would like to create a helm chart template enough generics to use same chart for all team project (backend, front-end,..)
On order to make it generics I can let developer to specify list of many cases in values.yml (volumes for deployment, network policy ingress egress etc...).
And i could keep the kubernetes template deployment, service etc.. Enough generics to never mention any specific keys.
So the developer could only modify values yml for add values for their application behavior.
The disadvantage is that the kubernetes generics template will not contain any logic about the application , and the generic template will be hard to maintain (because it will handle every possible case).
The advantage is that the developer doesn't need to understand helm because they will not modify the kubernetes template.
So you have any experience about that?
You can use _*.tpl files to define generic templates, They are located in ./templates/_*.tpl (. being the directory with global Chart.yaml and values.yaml).
Also by defaul in helm global values override local values. Solution to this can be found here - https://github.com/helm/helm/issues/5676
By using these 2 techniques in conjunction you can make generic templates and only use values.yaml to render what you want to render.
For example:
values.yaml:
global:
defaults:
switches:
volumesEnabled: false
ingressEnabled: false
ingress:
host: "generic-host.com"
volumes:
volumeName: "generic-volume-name"
subchart1:
defaultOverrides:
switches:
volumesEnabled: true
volumes:
volumeName: "not-so-generic-name"
subchart2:
defaultOverrides:
switches:
volumesEnabled: true
ingressEnabled: true
Then templates (java is just for grouping templates in one category, you can try to guess in which language my backend microservices are written :) )
./templates/java/_deployment.tpl:
{{- define "templates.java.deployment" }}
{{- $properties := merge .Values.defaultOverrides $.Values.global.defaults -}}
{{*/ generic deployment structure */}}
{{- if $properties.switches.volumesEnabled -}}
volume: {{ $properties.volumes.volumeName }}
{{- end }}
{{*/ generic deployment structure */}}
{{- end }}
./templates/java/_ingress.tpl:
{{- define "templates.java.ingress" }}
{{- $properties := merge .Values.defaultOverrides $.Values.global.defaults -}}
{{- if $properties.switches.ingressEnabled -}}
host: {{ $properties.ingress.host }}
{{*/ generic ingress structure */}}
{{- end }}
{{- end }}
And then subchart templates
./charts/subchart1/templates/deployment.yaml:
{{ include "templates.java.deployment" . }}
./charts/subchart1/templates/ingress.yaml:
{{ include "templates.java.ingress" . }}
subchart2 has exactly the same includes.
In the end we will have:
subchart1:
has deployment
volumeName is overriden from local values with "not-so-generic-name"
ingress is not rendered at all
subchart2:
has deployment
volumeName is default from global values
ingress host is default from global values
But I would say that it's a bad practice to generlize to much, because it will make your templates overly complex. In my case I found 2 distinct groups which have nearly identical manifests within them (basically frontend and backend) and made a set of _*.tpl files for each of them and settings default values for each group respectivelly in global values.yaml.
I have a setup in a flask app where I can use a form to upload a file into the static/img directory for my application, and the metadata is stored in a record in a database.
When someone makes a request to the appropriate page, I want to read the file from the server using the filepath saved in the database.
Here's an example of the record when it's pulled from the database:
(3, 'My Info', 'My Description', 'www.url.com', 'static\\img\\aws_nameservers.PNG')
What I'd like to do is combine the last item in the above record with the url_for function to render the image stored at that location.
What I have right now is a set method that gets the last part of the file path:
{% set filepath = workshop_info[4][7:] %}
This returns img\\aws_nameservers.PNG
Then inside an img tag I have the following:
<img src="{{ url_for('static', filename=filepath) }}">
Which gives me the following result:
<img src="/static/img%5Caws_nameservers.PNG">
It seems like a simple thing, but I can't figure out how to render the string correctly inside the jinja2 template.
Thank you.
If there is a better approach than what I'm attempting, I'm happy to get corrected.
You should use the forward slash (/) instead of backslash (\) at the file path. You can replace them either in the Python code or in the template using a filter, e.g.:
{% set filepath = "img\\aws_nameservers.PNG" | replace("\\", "/") %}
{{ url_for('static', filename=filepath) }}
# Outputs: /static/img/aws_nameservers.PNG
You can also use the pathlib module to convert between the Windows and Unix style paths.
I'm currently writing a small Ansible playbook whose job is to put in an additional domain in the search list in /etc/resolv.conf.
The second domain to add to the search list must contain part of the hostname of the target hosts. I'm getting the hostname of each of the target hosts during playbook execution using the magic variable {{ inventory_hostname }}.
I then need to extract characters 4 - 6 from the {{ inventory_hostname }} (say 'xyz') such that the second domain to add to the search list is xyz.foo.bar. In bash, this would be obtained with something like:
SERVER=$('hostname':3:3)
env=${SERVER:3:3}
... and the variable 'env' would be equal to 'xyz'.
The playbook works as long as 'xyz' is manually defined.
I am aware that Ansible has regular expression filters which can help with something like this, however I could not figure out a regular expression which does what I need.
For completeness sake, I have tried something like this in ansible:
{{ inventory_hostname|3:3 }}
Any help would be greatly appreciated.
It's almost the same, you can use "{{ inventory_hostname[3:6] }}" to select characters 3 to 6.
For example this task
- debug:
msg: "{{ inventory_hostname[3:6] }}"
Will output
ok: [localhost] => {
"msg": "alh"
}
Summary
I have a Jinja2 template which I'm running with Ansible.
I would like my template to load another file, as a template (i.e. evaluating {{ var }}), then I'll filter that, and then paste the result in to the top level template.
I think I'm almost there, I just need to find a Jinja2 filter which takes in a string and parses it as a template.
MWE
In this example lets assume the filter I want to apply is just to make the file uppercase.
(Obviously this case is so simple I could do it in one template file. But my real use case is more complex.)
Top level template main.yaml.j2:
---
something:
blah:
x: {{ y }}
{%- set names = [ 'John', 'Amy' ] %}
z: >
{{ lookup('file', './other-file.j2') | upper | indent(4*2) }}
other-file.j2:
{%- for name in names %}
Hello {{ name }}
{%- endfor %}
Running it with this Ansible playbook:
---
- hosts: localhost
connection: local
tasks:
- name: generate template
template:
src: "main.yaml.j2"
dest: "output.yaml.j2"
trim_blocks: False
register: templating
vars:
y: 5
Desired output
---
something:
blah:
x: 5
z: >
HELLO JOHN
HELLO AMY
Actual Output
---
something:
blah:
x: 5
z: >
{%- FOR NAME IN NAMES %}
HELLO {{ NAME }}
{%- ENDFOR %}
Best Guess
I think I'm almost there.
I just need a filter which applies a Jinja2 template to text.
i.e. something like:
{{ lookup('file', './other-file.j2') | template | upper | indent(4*2) }}
(But template is not a real filter. Maybe there's another name?)
What else I've tried
{{ include './other-file.j2' | upper | indent(4*2) }}
doesn't work.
fatal: [127.0.0.1]: FAILED! => {"changed": false, "msg": "AnsibleError: template error while templating string: expected token 'end of print statement', got 'string'. String: ---\nsomething:\n blah:\n x: {{ y }}\n {%- set names = [ 'John', 'Amy' ] %}\n z: >\n {{ include './other-file.j2' | upper | indent(4*2) }}"}
{% include './other-file.j2' | upper | indent(4*2) %}
"TemplateNotFound: ./OTHER-FILE.J2"
doesn't work.
Use Case
For context, my use case is that I have a Jinja2 template generating AWS CloudFormation templates.
I'm trying to do it all in YAML, not JSON.
(Because YAML can have comments, and you don't have to worry about whether the last item in a list has a trailing comma, and it's generally easier to read and write and debug.)
Some CloudFormation resources need literal JSON pasted into the YAML file. (e.g. CloudWatch Dashboard bodies).
So I want to have another file in YAML, which Jinja2 converts to json, and pastes into my overall YAML template.
I want this dashboard to be generated with a for loop, and to pass in variables.
I would like to have a separate
Instead of file plugin
lookup('file', './other-file.j2')
use template plugin
lookup('template', './other-file.j2')
Note that the scope of the variable {% set names = ['John', 'Amy'] %} is the template main.yaml.j2. If this variable is used in the template other-file.j2 the command lookup('template', './other-file.j2') will crash with the error:
"AnsibleUndefinedVariable: 'names' is undefined"
Solution
Declare the variable in the scope of the playbook. For example
- template:
src: "main.j2"
dest: "output.txt"
vars:
names: ['John', 'Amy']
main.j2
{{ lookup('template', './other-file.j2') }}
other-file.j2
{% for name in names %}
Hello {{ name }}
{% endfor %}
give
shell> cat output.txt
Hello John
Hello Amy
Im new at helm. Im building a splunk helm chart with numerous conf files. I currently use something like this in a configmap ..
apiVersion: v1
kind: ConfigMap
metadata:
name: splunk-master-configmap
data:
indexes.conf: |
# global settings
# Inheritable by all indexes: no hot/warm bucket can exceed 1 TB.
# Individual indexes can override this setting.
homePath.maxDataSizeMB = 1000000
but I would prefer to have the conf files in a seperate folder e.g. configs/helloworld.conf and have come accross "tpl" but am struggling to understand how to implement it. - can anyone advise best practices. On a side note splunk has orders of presidences >> so there may be many indexes.conf files used in various locations. does anyone have any thoughts on how best to implement this?!??!
Cheers.
If the content of the files is static then you could create a files directory in your chart at the same level as the templates directory (not inside it) and reference them like:
kind: ConfigMap
metadata:
name: splunk-master-configmap
data:
{{ (.Files.Glob "files/indexes.conf").AsConfig | indent 2 }}
{{ (.Files.Glob "files/otherfile.conf").AsConfig | indent 2 }}
# ... and so on
Where this would break down is if you want to be able to reference the values of variables inside the files so that the content is controlled from the values.yaml. If you want to expose each value individually then there's an example in the helm documentation using range. But I think a good fit or your case is what the stable/mysql chart does. It has a ConfigMap that takes values as strings:
{{- if .Values.configurationFiles }}
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ template "mysql.fullname" . }}-configuration
data:
{{- range $key, $val := .Values.configurationFiles }}
{{ $key }}: |-
{{ $val | indent 4}}
{{- end }}
{{- end -}}
And the values.yaml allows both the files and their content to be set and overridden by the user of the chart:
# Custom mysql configuration files used to override default mysql settings
configurationFiles:
# mysql.cnf: |-
# [mysqld]
# skip-name-resolve
# ssl-ca=/ssl/ca.pem
# ssl-cert=/ssl/server-cert.pem
# ssl-key=/ssl/server-key.pem
It comments out that content and leaves it to the user of the chart to set but you could have defaults in the values.yaml.
You would only need tpl if you needed further flexibility. The stable/keycloak chart lets the user of the chart create their own configmap and point it into the keycloak deployment via tpl. But I think your case is probably closest to the mysql one.
Edit: the tpl function can also be used to take the content of files loaded with Files.Get and effectively make that content part of the template - see How do I load multiple templated config files into a helm chart? if you're interested in this