Test Kitchen, store credentials - test-kitchen

With Test Kitchen, in the yaml configs... where is the best place to store globally used attributes that apply to multiple platforms and multiple suites?
To use my .kitchen.yml as an example:
---
provisioner:
name: chef_solo
platforms:
- name: centos-6.5
driver:
name: vagrant
- name: amazon
driver:
name: ec2
image_id: ami-ed8e9284
flavor_id: t2.medium
aws_ssh_key_id: <snip>
ssh_key: <snip>
availability_zone: us-east-1a
subnet_id: subnet-<snip>
require_chef_omnibus: true
iam_profile_name: <snip>
ebs_delete_on_termination: true
security_group_ids: sg-<snip>
# area in question (does not work here)
attributes:
teamcity:
server: 'build.example.com'
port: 80
username: 'example'
password: 'example'
# end area in question
suites:
- name: resin4
run_list:
- recipe[example_server::resin4]
- recipe[example_server::deploy_all_artifacts]
- name: deploy
run_list:
- recipe[example_server::deploy_all_artifacts]
- name: default
run_list:
- recipe[example_server::elasticsearch]
- recipe[example_server::resin4]
- recipe[example_server::deploy_all_artifacts]
I know there are other kitchen files, such as ~/kitchen/config.yml and .kitchen.local.yml but I've been unable to find a away to get attributes to apply to all platforms and suites. Is copy and pasting attributes to platforms the best way?

Is there a reason to specify these attributes in kitchen's yaml and not recipe[example_server::deploy_all_artifacts]? If necessary you could set overrides in kitchen.
Also, this post might be helpful: Access Attributes Across Recipes

Related

Mounting AWS Secrets Manager on Kubernetes/Helm chart

I have created an apps cluster deployment on AWS EKS that is deployed using Helm. For proper operation of my app, I need to set env variables, which are secrets stored in AWS Secrets manager. Referencing a tutorial, I set up my values in values.yaml file someway like this
secretsData:
secretName: aws-secrets
providerName: aws
objectName: CodeBuild
Now I have created a secrets provider class as AWS recommends: secret-provider.yaml
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
name: aws-secret-provider-class
spec:
provider: {{ .Values.secretsData.providerName }}
parameters:
objects: |
- objectName: "{{ .Values.secretsData.objectName }}"
objectType: "secretsmanager"
jmesPath:
- path: SP1_DB_HOST
objectAlias: SP1_DB_HOST
- path: SP1_DB_USER
objectAlias: SP1_DB_USER
- path: SP1_DB_PASSWORD
objectAlias: SP1_DB_PASSWORD
- path: SP1_DB_PATH
objectAlias: SP1_DB_PATH
secretObjects:
- secretName: {{ .Values.secretsData.secretName }}
type: Opaque
data:
- objectName: SP1_DB_HOST
key: SP1_DB_HOST
- objectName: SP1_DB_USER
key: SP1_DB_USER
- objectName: SP1_DB_PASSWORD
key: SP1_DB_PASSWORD
- objectName: SP1_DB_PATH
key: SP1_DB_PATH
I mount this secret object in my deployment.yaml, the relevant section of the file looks like this:
volumeMounts:
- name: secrets-store-volume
mountPath: "/mnt/secrets"
readOnly: true
env:
- name: SP1_DB_HOST
valueFrom:
secretKeyRef:
name: {{ .Values.secretsData.secretName }}
key: SP1_DB_HOST
- name: SP1_DB_PORT
valueFrom:
secretKeyRef:
name: {{ .Values.secretsData.secretName }}
key: SP1_DB_PORT
further down in same deployment file, I define secrets-store-volume as :
volumes:
- name: secrets-store-volume
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: aws-secret-provider-class
All drivers are installed into cluster and permissions are set accordingly
with helm install mydeployment helm-folder/ --dry-run I can see all the files and values are populated as expected. Then with helm install mydeployment helm-folder/ I install the deployment into my cluster but with kubectl get all I can see the pod is stuck at Pending with warning Error: 'aws-secrets' not found and eventually gets timeout. In AWS CloudTrail log, I can see that the cluster made request to access the secret and there was no error fetching it. How can I solve this or maybe further debug it? Thank you for your time and efforts.
Error: 'aws-secrets' not found - looks like CSI Driver isn't creating kubernetes secret that you're using to reference values
Since yaml files looks correctly, I would say it's probably CSI Driver configuration Sync as Kubernetes secret - syncSecret.enabled (which is false by default)
So make sure that secrets-store-csi-driver runs with this flag set to true, for example:
helm upgrade --install csi-secrets-store \
--namespace kube-system secrets-store-csi-driver/secrets-store-csi-driver \
--set grpcSupportedProviders="aws" --set syncSecret.enabled="true"

Run Ansible task only on one server - AWS

I have my ansible task running in all my api_servers which i would restrict it to run only on one IP (one of the api_server)
I have added run_once: true but it didnt helps.
Kindly advise.
EDIT :
Will the below work? I have 10 instances of app_servers running, I want the task to run only on one app_server
run_once: true
when:
- inventory_hostname == groups['app_servers'][0]
Where my inventory file is like
[app_servers]
prod_app_[1:4]
I would write my playbook like that:
---
# Run on all api_servers
- hosts: api_servers
tasks:
- name: do something on all api_servers
#...
# Run only on one api_server e.q. api_server_01
- hosts: api_server_01
tasks:
- name: Gather data from api_server_01
#...
The other option would be to work with when: or to run the playbook with the --limit option
---
- hosts: all
tasks:
- name: do something only when on api_server_01
#...
when: inventory_hostname == "api_server_01"
EDIT:
Here you will see all the option in one example:
---
- hosts: all
tasks:
- debug: msg="run_once"
run_once: true
- debug: msg=all
- debug: msg="run on the first of the group"
when: inventory_hostname == groups['app_servers'][0]
# Option with separated hosts, this one will be faster if gather_facts is not tuned.
- hosts: app_servers[0]
tasks:
- debug: msg="run on the first of the group"
(Since I can not comment, I have to answer.)
What about delegate_to? Where you delegate the task to a specific host.
hosts: all
tasks:
- name: Install vim on specific host
package:
name: vim
state: present
delegate_to: staging_machine
Or
As #user2599522 mentioned: --limit is also an option to use:
You can also limit the hosts you target on a particular run with the --limit flag. (Patterns and ansible-playbook flags)

GCP Deployment Manager "ResourceErrorCode":"400" while database user creation

I am experimenting with deployment manager and each time I try to deploy an SQL instance with a DB on it and 2 users; some of the tasks are failing. Most of the time they are the users:
conf.yaml:
resources:
- name: mycloudsql
type: gcp-types/sqladmin-v1beta4:instances
properties:
name: mycloudsql-01
backendType: SECOND_GEN
instanceType: CLOUD_SQL_INSTANCE
databaseVersion: MYSQL_5_7
region: europe-west6
settings:
tier: db-f1-micro
locationPreference:
zone: europe-west6-a
activationPolicy: ALWAYS
dataDiskSizeGb: 10
- name: mydjangodb
type: gcp-types/sqladmin-v1beta4:databases
properties:
name: django-db-01
instance: $(ref.mycloudsql.name)
charset: utf8
- name: sqlroot
type: gcp-types/sqladmin-v1beta4:users
properties:
name: root
host: "%"
instance: $(ref.mycloudsql.name)
password: root
- name: sqluser
type: gcp-types/sqladmin-v1beta4:users
properties:
name: user
instance: $(ref.mycloudsql.name)
password: user
Error:
PS C:\Users\user\Desktop\Python\GCP> gcloud --project=sound-catalyst-263911 deployment-manager deployments create dm-sql-test-11 --config conf.yaml
The fingerprint of the deployment is TZ_wYom9Q64Hno6X0bpv9g==
Waiting for create [operation-1589869946223-5a5fa71623bc9-1912fcb9-bc59aafc]...failed.
ERROR: (gcloud.deployment-manager.deployments.create) Error in Operation [operation-1589869946223-5a5fa71623bc9-1912fcb9-bc59aafc]: errors:
- code: RESOURCE_ERROR
location: /deployments/dm-sql-test-11/resources/sqluser
message: '{"ResourceType":"gcp-types/sqladmin-v1beta4:users","ResourceErrorCode":"400","ResourceErrorMessage":{"code":400,"message":"Precondition
check failed.","status":"FAILED_PRECONDITION","statusMessage":"Bad Request","requestPath":"https://www.googleapis.com/sql/v1beta4/projects/sound-catalyst-263911/instances/mycloudsql-01/users","httpMethod":"POST"}}'
- code: RESOURCE_ERROR
location: /deployments/dm-sql-test-11/resources/sqlroot
message: '{"ResourceType":"gcp-types/sqladmin-v1beta4:users","ResourceErrorCode":"400","ResourceErrorMessage":{"code":400,"message":"Precondition
check failed.","status":"FAILED_PRECONDITION","statusMessage":"Bad Request","requestPath":"https://www.googleapis.com/sql/v1beta4/projects/sound-catalyst-263911/instances/mycloudsql-01/users","httpMethod":"POST"}}'
Console View:
It doesn`t say what that precondition failing is or am I missing something?
It seems the installation of database is not completed by the time the Deployment Manager starts to create users despite the reference notation is used in the YAML code to take care of dependencies. That is why you receive the "FAILED_PRECONDITION" error.
As a workaround you can split the deployment into two parts:
Create a CloudSQL instance and a database;
Create users.
This does not look elegant, but it works.
Alternatively, you can consider using Terraform. Fortunately, Cloud Shell instance is provided with Terraform pre-installed. There are sample Terraform code for Cloud SQL out there, for example this one:
CloudSQL deployment with Terraform

Deploy node-pool in different subnetwork in same yaml file

I am creating a yaml config to deploy a gke cluster with multi-node-pool. I like to be able to create a new cluster and put each node-pool in a different subnetwork. Can this be done.
I have tried putting the subnetwork in different part of the properties under the second node-pool but it errors out. Below is the following error.
message: '{"ResourceType":"gcp-types/container-v1:projects.locations.clusters.nodePools","ResourceErrorCode":"400","ResourceErrorMessage":{"code":400,"message":"Invalid
JSON payload received. Unknown name \"subnetwork\": Cannot find field.","status":"INVALID_ARGUMENT","details":[{"#type":"type.googleapis.com/google.rpc.BadRequest","fieldViolations":[{"description":"Invalid
JSON payload received. Unknown name \"subnetwork\": Cannot find field."}]}],"statusMessage":"Bad
The current code for the both node-pools. first node is creates but second one error out.
resources:
- name: myclus
type: gcp-types/container-v1:projects.locations.clusters
properties:
parent: projects/[PROJECT_ID]/locations/[ZONE/REGION]
cluster:
name: my-clus
zone: us-east4
subnetwork: dev-web ### leave this field blank if using the default network
initialClusterVersion: "1.13"
nodePools:
- name: my-clus-pool1
initialNodeCount: 1
config:
machineType: n1-standard-1
imageType: cos
oauthScopes:
- https://www.googleapis.com/auth/cloud-platform
preemptible: true
- name: my-clus
type: gcp-types/container-v1:projects.locations.clusters.nodePools
properties:
parent: projects/[PROJECT_ID]/locations/[ZONE/REGION]/clusters/$(ref.myclus.name)
subnetwork: dev-web ### leave this field blank if using the default
nodePool:
name: my-clus-pool2
initialNodeCount: 1
version: "1.13"
config:
machineType: n1-standard-1
imageType: cos
oauthScopes:
- https://www.googleapis.com/auth/cloud-platform
preemptible: true
I like the expected out come to have 2 node-pools in 2 different subnetworks.
I found out that this is actually not a limitation of Deployment Manager but a limitation of GKE.
We can’t assign a different subnet to different node pools, the network and subnets are defined at the cluster level. There is no “Subnetwork” field in the node pool API.
Here is a link you can refer to for more information.

Error during bosh deploy: "Arguments are not correct, details: 'expected string value for member 1 of key values of member 1 of option filters"

I try to deploy simple example. I didn't change anything except warden.yml file. So, I tried to deploy it to AWS and use elastic IP, so I can access the server with specific IP.
When I deploy it I receive:
Director task 67
Deprecation: Ignoring cloud config. Manifest contains 'networks' section.
Started preparing deployment > Preparing deployment. Done (00:00:00)
Started preparing package compilation > Finding packages to compile. Done (00:00:00)
Started creating missing vms > webapp/3a8acd3a-77a8-4bad-8de4-fb544d70f76d (0). Failed: Unknown CPI error 'InvalidCall' with message 'Arguments are not correct, details: 'expected string value for member 1 of key values of member 1 of option filters'' in 'create_vm' CPI method (00:00:05)
Error 100: Unknown CPI error 'InvalidCall' with message 'Arguments are not correct, details: 'expected string value for member 1 of key values of member 1 of option filters'' in 'create_vm' CPI method
What is the reason of this error?
warden.yml
name: webapp-warden
director_uuid: <%= `bosh status --uuid` %>
releases:
- name: simple-bosh-release
version: latest
compilation:
workers: 1
network: webapp-network
reuse_compilation_vms: true
cloud_properties:
instance_type: t2.medium
availability_zone: us-west-2a
update:
canaries: 1
canary_watch_time: 30000-240000
update_watch_time: 30000-600000
max_in_flight: 3
resource_pools:
- name: common-resource-pool
network: webapp-network
size: 1
stemcell:
name: bosh-aws-xen-ubuntu-trusty-go_agent
version: latest
cloud_properties:
instance_type: t2.medium
availability_zone: us-west-2a
networks:
- name: webapp-network
type: vip
cloud_properties:
security_groups:
- default
# cloud_properties:
# subnet: subnet-87d256ce
- name: default
type: dynamic
cloud_properties:
security_groups:
- default
jobs:
- name: webapp
template: webapp
instances: 1
resource_pool: common-resource-pool
networks:
- name: webapp-network
static_ips:
- 52.40.58.163
- name: default
default: [dns, gateway]
properties:
webapp:
admin: foo#bar.com
servername: 52.40.58.163
This error is coming because there is an missing properties in network configuration. Provides the subnet id and try it. it will work.