How to read a YAML file into a Hash - crystal-lang

I have a requirement to read a YAML config file from disk into memory as a Hash in Crystal. (I am aware of this discussion on the topic in the mailing list, although I find it a bit confusing.)
The YAML file I am trying to read is a fairly simple config file:
---
:excludes:
- Stage
- Class
- Anchor
- Notify
- Node
- /::/
:only_include: []
:md5sums: false
:class_name: null
:catalog_file: null
:output_file: spec/classes/init_spec.rb
:compile_test: true
:setup: {}
A code example showing how to read a YAML config file from disk into a Hash using Crystal 0.25 would be very helpful.

Here's the best way to do it.
require "yaml"
class Config
include YAML::Serializable
#[YAML::Field(key: ":excludes")]
property excludes : Array(String)
#[YAML::Field(key: ":only_include")]
property only_include : Array(String)
#[YAML::Field(key: ":md5sums")]
property md5sums : Bool
#[YAML::Field(key: ":class_name")]
property class_name : String?
#[YAML::Field(key: ":catalog_file")]
property catalog_file : String?
#[YAML::Field(key: ":output_file")]
property output_file : String
#[YAML::Field(key: ":compile_test")]
property compile_test : Bool
#[YAML::Field(key: ":setup")]
property setup : Hash(String, String)
end
yaml = <<-YAML
:excludes:
- Stage
- Class
- Anchor
- Notify
- Node
- /::/
:only_include: []
:md5sums: false
:class_name: null
:catalog_file: null
:output_file: spec/classes/init_spec.rb
:compile_test: true
:setup: {}
YAML
p Config.from_yaml(yaml)
Just note that YAML in Crystal currently does not support creating a Regex from YAML, so you'll probably want to do that after you parse it (it's a bit tricky with a converter).

I'm not sure, if this straightforward example will help you, but maybe it will give some hints:
yaml = File.open("conf.yml") { |file| YAML.parse(file) }
yaml.class # => YAML::Any
hash = yaml.as_h # As you can see below, it gives a Hash, but with YAML::Any underneath
hash.class # => Hash(YAML::Any, YAML::Any)
yaml[":excludes"].as_a.first # => "Stage"
yaml[":md5sums"] == false # => true
yaml[":output_file"].as_s + "-FOO" # => "spec/classes/init_spec.rb-FOO"
yaml[":setup"].as_h.class # => Hash(YAML::Any, YAML::Any)
So, you need to keep in mind that YAML was the source of the data and not forget to convert it into required type with a .as_* method.

Related

How to assign a value to a variable in if condition in yaml files?

I'm trying to assign the value inif condition and use it as runtime in yaml file.
1)Based on if condition,assign values for basicvalue
2)Use that basicvalue in the placeholder.
Variables:
- ${{if eq(${env},'envn1')}}:
basicvalue : 'Hello basic1'
- ${{if eq(${env},'envn2')}}:
basicvalue : 'Hello basic2'
notifications:
- name: "myteam"
settings:
Url:'{{basicvalue}}'
Placeholder vale of url is not getting replaced and I'm doubting whether my if condition is working..
Any suggestions to achieve this?

Plop.js Template files adding a root directory in the destination folder

I'm running into an issue unsure if it's me.
I'm using Plop.js within a project to quickly code generate boilerplate (bp) for new packages.
I've got 4 different template folders that plop maps from when a user enters the generate command.
User is prompted with a list of package types to create:
choices: ['react', 'node', 'browser', 'isomorphic'],
Based on the users response to the prompts, plop chooses the folder to pull the template files from.
Template folder structure looks like this:
plop-template/
- react/
- node/
- browser/
- isomorphic/
The templateFiles: property is correctly identifying and creating the bp based on the user response.
templateFiles: 'plop_templates/{{project-type}}/**/*/',
the issue i'm running into is the project-type is being added to the file destination path
So what i would like to occur is the
/project-name/ ... boiler plate created
But what is happening is:
/project-name/project-type/ ... boiler plate created
So, is it possible to remove the /project-type/ from the destination path?
Plopfile.js (v. "plop": "3.1.1"):
const findEtensionFile = require("../lib/file-extention-locator");
module.exports = function (plop) {
plop.getDestBasePath;
plop.setGenerator("component", {
prompts: [
{
type: "input",
name: "project-name",
message: "What's the name your project? ",
},
{
type: "list",
name: "project-type",
message: "Project Type:",
choices: ["react", "node", "browser", "isomorphic"],
},
],
actions: function (data) {
var actions = [];
actions.push({
type: "addMany",
globOptions: { dot: true },
destination: "../../../{{project-name}}",
base: "/",
templateFiles: "plop_templates/{{project-type}}/**/*/",
});
return actions;
},
});
};
What I've tried:
filter: property ... This can be used to modify the file contents, seems like that only affects the
base: property (string).. documentation seems to indicate this is the route where I can filter out but can't find a value that doesn't break BP creation.
Any help would be greatly appreciated.
Discovered the error was mine.
Seems like the base: value must match the entire value for templateFiles.
In this case:
base: 'plop_templates/{{project-type}}',
templateFiles: "plop_templates/{{project-type}}/**/*/",
Even though the /plop_templates/ folder wasn't being created in the path.

regex ansible lineinfile subsection

I am fairly new to ansible and have been solving the following problem with a shell script, but the proper way I believe is to use the lineinfile module, just not sure how to accomplish this.
Let's say I have a file with the following text in it.
<cpu>
<alarm>
active = yes
</alarm>
</cpu>
<disk>
<alarm>
active = yes
<fixed>
<#>
active = yes
description = File system /
<inode_error>
active = yes
threshold = 2
message = InodeError
</inode_error>
</#>
<#boot>
active = yes
description = File system /boot
percent = yes
<error>
active = yes
threshold = 5
message = DiskError
</error>
</#boot>
</fixed>
</alarm>
</disk>
I want to make sure the following section is set correctly.
<disk><alarm><fixed><#boot><error>"threshold = 2"</error></#boot></fixed></alarm></disk>
is there a way to only (modify/make sure exists) that line, normally this file is much larger with many more sections, but I erased some so the question is readable.
Update: Modifying this as it is not valid XML and the XML module will not parse the file correctly.
Thanks!
lineinfile scans file per line, so you can't define complex multiline regexp for context definition.
replace module support multiline regexp.
If you have threshold = X in the file and want to be sure it is set to specific value, you can use this regexp:
- replace:
path: ./test.txt
regexp: '(<disk>[\s\S]*<alarm>[\s\S]*<#boot>[\s\S]*<error>[\s\S]*)(threshold\s*=\s*\d+)([\s\S]*?<\/error>)'
replace: '\1threshold = 2\3'
It searches for line threshold\s*=\s*\d+ inside <disk>...<alarm>...<#boot>...<error>....
This code is idempotent – so if threshold = 2, then nothing is done.
But if there is no threshold = X string, it will fail. You should construct more complex regular expression for that case.
you could use the lineinfile module (http://docs.ansible.com/ansible/latest/lineinfile_module.html) where you could write a regex to modify/add the line and use the validate function to run a command to ensure that the xml file has the proper syntax.
If you are on Ansible 2.4 you can use the xml module (https://docs.ansible.com/ansible/2.4/xml_module.html) and use the attribute parameter to check if the xpath in xml file is set, like that:
- name: Read attribute value
xml:
path: /foo/bar.xml
xpath: /business/website/validxhtml
content: attribute
attribute: validatedon
register: xmlresp
regards

Logstash config-file don't catch my logs, but debugger did

So, I'm a little bit new at the elk-stack, and I’m having an issue with further experiment with the tools. I'm using a linux machine.
First of all, here's my config-file :
input {
file {
type => "openerp"
path => "/home/jvc/Documents/log/openerp-cron.log.2014-11-20.txt"
start_position => "beginning"
codec => multiline{
pattern => "^%{TIMESTAMP_ISO8601} "
negate => true
what => previous
}
}
}
filter{
if [type]=="openerp"{
date{
match => ["timestamp","yyyy-MM-dd HH:mm:ss,SSS"]
}
grok{
patterns_dir => "./patterns"
match => { "message" => "%{ODOOLOG}" }
}
}
}
output{
file{
path => "/home/jvc/Bureau/testretour.txt"
}
}
I have some patterns too :
REQUESTTIMESTAMP %{MONTHDAY}/%{MONTH}/%{YEAR} %{TIME}
REQUEST %{IPORHOST:client} %{USER:ident} %{USER:auth} [%{REQUESTTIMESTAMP:request_timestamp}] "%{WORD:request_type} %{URIPATHPARAM:request} HTTP/%{NUMBER:httpversion}" %{NUMBER:response} -
ODOOMISC %{GREEDYDATA}
ODOOLOG %{TIMESTAMP_ISO8601:timestamp} %{POSINT:pid} %{LOGLEVEL:level} (?:%{USERNAME:user}|\?) %{PROG:module}: (?:%{REQUEST}|%{ODOOMISC:misc})
Some examples of the logs :
2014-11-21 08:00:16,715 17798 DEBUG noe openerp.addons.base.ir.ir_cron: cron.object.execute('noe', 1, '*', u'crossovered.budget.lines', u'computeblank')
2014-11-21 08:00:17,172 17798 WARNING noe openerp.osv.orm.browse_record.noe_utils.synchro_date: Field 'conform' does not exist in object 'browse_record(noe_utils.synchro_date, 13)'
2014-11-21 08:00:17,172 17798 ERROR noe openerp.sql_db: Programming error: can't adapt type 'browse_record', in query SELECT id
FROM crossovered_budget_lines
WHERE is_blank='t'
AND general_budget_id in %s
AND date_from <= %s AND date_to >= %s
2014-11-21 08:00:17,173 17798 ERROR noe openerp.addons.base.ir.ir_cron: Call of self.pool.get('crossovered.budget.lines').computeblank(cr, uid, *()) failed in Job 10
I'm having trouble with this config. For some reason that I can't find, this produces nothing.
What I have tried - done :
-First of all, I tested my grok, and multiline pattern in some grok debugger I have find on the web. All of them matches my logs.
-Before using the codec for multiline, i used the multiline filter. This one worked, but seems to be deprecated. So it's not a solution.
-I know that logstash keep in mind what he had read or not with the "sincedb" files : I delete these before every test, but you know what happens.
-I tried to run logstash with the -verbose, but nothing wrong is displayed.
-I don't really know if I must write the ".txt" at the end of my paths. But anyway, none of them works.
Have I missed something ? Thank you in advance for helping hands.
So, with more test i succeeded. I copied the content of one of my logs file and pasted it in another file : It works.
But, there is now another question : if deleting the "sincedb" file doesn't work, how can i "empty" the cache of logstash ?

puppet matching multiple agent hostnames to params.pp data structure

I have a module called appserver in my puppet modules.
In that module manifests I have a params.pp file which is inherited by init.pp file.
In params.pp file I have the following data structure.
$servers = {
appserver-mgr => { axis2 => {subDomain => 'mgt',},
carbon => {subDomain => 'mgt',},
serverOptions => '-Dsetup',
server_home => $carbon_home, },
appserver-wkr => { axis2 => {subDomain => 'worker', members => ['appserver-mgr2-ip']},
carbon => {subDomain => 'worker',},
serverOptions => '-DworkerNode=true',
server_home => $carbon_home, },
}
In my init.pp file I'm filling my templates as follows using the said data structure.
define fill_templates($axis2, $carbon, $clustering, $serverOptions, $server_home) {
$ipAdd = $::ipaddress
$hostName = $::hostname
if $hostName == "${name}" {
notify {"host name match found for $hostName for $ipAdd":}
file { "${server_home}/repository/conf/axis2/axis2.xml":
ensure => file,
content => template('appserver/axis2.xml.erb'),
}
->
file { "${server_home}/repository/conf/carbon.xml":
ensure => file,
content => template('appserver/carbon.xml.erb'),
}
->
file { "${server_home}/repository/conf/tomcat/catalina-server.xml":
ensure => file,
content => template('appserver/catalina-server.xml.erb'),
}
}
}
As per the current method, if a matching node is found (say appserver-mgr) the respective data structure values are retrieved and applied to the templates.
Currently these scripts are working as expected.
Now I want to change it as follows.
I have a cluster containing following nodes.
appserver-mgr-1
appserver-mgr-2
appserver-mgr-3
appserver-wkr-1
appserver-wkr-2
appserver-wkr-3
appserver-wkr-4
appserver-wkr-5
By using the same data structure in params.pp file, how can I apply the appserver-mgr configuration to *.mgr nodes 1-3 and appserver-wkr configuration to *.wkr nodes 1-5?
Can I use regular expressions for this task?
I'm quite sure that it would be possible to bend the Puppet DSL to do what you need here. However, the far better approach to this issue is Hiera.
node /appserver-mgr/ {
$node_class = 'appserver'
$node_subclass = 'manager'
}
node /appserver-wrk/ {
$node_class = 'appserver'
$node_subclass = 'worker'
}
Use the node_class and node_subclass variables in your Hierarchy.
# /etc/puppet/hiera.yaml
---
:backends:
- yaml
:yaml:
:datadir: /etc/puppet/hieradata
:hierarchy:
- "%{::clientcert}"
- "class-%{node_class}-%{node_subclass}"
- "class-%{node_class}"
- common
Now you can define your data right in the YAML for Hiera, instead of params.pp.
# /etc/puppet/hieradata/class-appserver-manager.yaml
servers:
axis2:
subDomain: mgt
carbon:
subDomain: mgt
serverOptions: -Dsetup
server_home: %{carbon_home}
and for the worker:
# /etc/puppet/hieradata/class-appserver-worker.yaml
servers:
axis2:
subDomain: worker
members:
- appserver-mgr2-ip
carbon:
subDomain: worker
serverOptions: -DworkerNode=true
server_home: %{carbon_home}
In your params class, the following then suffices:
$servers = hiera('servers')
Or you don't even bother with the params class, and just replace the uses of the $servers variable with hiera calls. But doing just one call in a params style class is a good practice.
Note: Using the variable value %{carbon_home} from Hiera is somewhat dangerous, you might want to hardcode the actual value in the YAML there.