I am setting up EMR cluster using CloudFormation template. Even though, I am passing TerminationProtected: false cluster is still created with termination protection turned on. Is this possible that the global company policy is overwriting this value? Here's a snippet of my YAML template
Type: AWS::EMR::Cluster
Properties:
Name: !Sub ${AWS::StackName}-cluster
Applications:
- Name: Hadoop
- Name: ZooKeeper
- Name: Spark
- Name: Ganglia
Instances:
MasterInstanceGroup:
...
CoreInstanceGroup:
...
Ec2SubnetId: ...
Ec2KeyName: ...
TerminationProtected: False
AdditionalMasterSecurityGroups:
- ...
BootstrapActions:
- ...
JobFlowRole : ...
LogUri: ...
ServiceRole : ...
ReleaseLabel: ...
VisibleToAllUsers: ...
Configurations:
- Classification: hdfs-site
ConfigurationProperties:
dfs.replication: '2'
Related
I'm trying to create a cloudFormation stack with MSK Configuration and associating MSK Configuration with MSK Cluster. Creation of AWS::MSK::Configuration returns only ARN while I need ARN and Revision number to associate MSK Configuration to MSK Cluster. Is there any way to achieve this? Currently I'm hard-coding it to 1 which means it will work only for creating stack.
...
MSKConfiguration:
Type: AWS::MSK::Configuration
Properties:
Name: aws-msk-configuration
ServerProperties: |
auto.create.topics.enable = true
zookeeper.connection.timeout.ms = 1000
log.roll.ms = 604800000
MSKCluster:
Type: AWS::MSK::Cluster
Properties:
ClusterName: !Ref ClusterName
ClientAuthentication: !If
- UsingIamAuthentication
- Sasl:
Iam:
Enabled: true
- Sasl:
Scram:
Enabled: true
ConfigurationInfo:
Arn: !GetAtt MSKConfiguration.Arn
Revision: 1
...
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-msk-configuration.html
You can only get the latest version if you define a custom resource. Since you program the full logic of the resource, you can do what you want, including automatically setting up latest version for MKS.
I have created an apps cluster deployment on AWS EKS that is deployed using Helm. For proper operation of my app, I need to set env variables, which are secrets stored in AWS Secrets manager. Referencing a tutorial, I set up my values in values.yaml file someway like this
secretsData:
secretName: aws-secrets
providerName: aws
objectName: CodeBuild
Now I have created a secrets provider class as AWS recommends: secret-provider.yaml
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
name: aws-secret-provider-class
spec:
provider: {{ .Values.secretsData.providerName }}
parameters:
objects: |
- objectName: "{{ .Values.secretsData.objectName }}"
objectType: "secretsmanager"
jmesPath:
- path: SP1_DB_HOST
objectAlias: SP1_DB_HOST
- path: SP1_DB_USER
objectAlias: SP1_DB_USER
- path: SP1_DB_PASSWORD
objectAlias: SP1_DB_PASSWORD
- path: SP1_DB_PATH
objectAlias: SP1_DB_PATH
secretObjects:
- secretName: {{ .Values.secretsData.secretName }}
type: Opaque
data:
- objectName: SP1_DB_HOST
key: SP1_DB_HOST
- objectName: SP1_DB_USER
key: SP1_DB_USER
- objectName: SP1_DB_PASSWORD
key: SP1_DB_PASSWORD
- objectName: SP1_DB_PATH
key: SP1_DB_PATH
I mount this secret object in my deployment.yaml, the relevant section of the file looks like this:
volumeMounts:
- name: secrets-store-volume
mountPath: "/mnt/secrets"
readOnly: true
env:
- name: SP1_DB_HOST
valueFrom:
secretKeyRef:
name: {{ .Values.secretsData.secretName }}
key: SP1_DB_HOST
- name: SP1_DB_PORT
valueFrom:
secretKeyRef:
name: {{ .Values.secretsData.secretName }}
key: SP1_DB_PORT
further down in same deployment file, I define secrets-store-volume as :
volumes:
- name: secrets-store-volume
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: aws-secret-provider-class
All drivers are installed into cluster and permissions are set accordingly
with helm install mydeployment helm-folder/ --dry-run I can see all the files and values are populated as expected. Then with helm install mydeployment helm-folder/ I install the deployment into my cluster but with kubectl get all I can see the pod is stuck at Pending with warning Error: 'aws-secrets' not found and eventually gets timeout. In AWS CloudTrail log, I can see that the cluster made request to access the secret and there was no error fetching it. How can I solve this or maybe further debug it? Thank you for your time and efforts.
Error: 'aws-secrets' not found - looks like CSI Driver isn't creating kubernetes secret that you're using to reference values
Since yaml files looks correctly, I would say it's probably CSI Driver configuration Sync as Kubernetes secret - syncSecret.enabled (which is false by default)
So make sure that secrets-store-csi-driver runs with this flag set to true, for example:
helm upgrade --install csi-secrets-store \
--namespace kube-system secrets-store-csi-driver/secrets-store-csi-driver \
--set grpcSupportedProviders="aws" --set syncSecret.enabled="true"
I Have a parameter PreferredMaintenanceWindow setup within AWS::RDS::DBCluster in my cloudformation template and it was all good.
Recently I had to add the parameter EnableCloudwatchLogsExport to the template, but when I tried to deploy the stack it failed with the following message:
You currently can't modify MaintenanceWindow with Aurora Serverless.
(Service: AmazonRDS; Status Code: 400; Error Code:
InvalidParameterCombination; Request ID:
bf40fd7e-e599-481a-ac23-b3c68e9794a1)
This is when I didn't even touch that parameter, did anyone faced such issue or any workaround which I can use with CloudFormation
PS: I am aware that I can setup the EnableCloudwatchLogsExport from the CLI but I don't want drift and I would really want it to do it via cloudformatio.
Update:
RDSServerlessDB:
Type: "AWS::RDS::DBCluster"
Properties:
AvailabilityZones:
Fn::GetAZs: !Ref 'AWS::Region'
BackupRetentionPeriod: 3 # Number of days
DatabaseName: !Join ["_", [!Join ["_", !Split [ "-", !Sub '${MasterStack}']], 'fol']]
DBClusterParameterGroupName: "default.aurora5.6"
DBSubnetGroupName: !Ref RDSDBSubnetGroup
EnableCloudwatchLogsExports:
- error
- general
- slowquery
- audit
Engine: "aurora"
EngineMode: "serverless"
# EngineVersion: "5.6.10a"
MasterUsername: "username"
MasterUserPassword: "{{resolve:ssm-secure:/password:1}}"
# Port: 3306
# PreferredMaintenanceWindow: "sun:01:00-sun:03:00"
ScalingConfiguration:
AutoPause: True
MaxCapacity: 32
MinCapacity: 2
SecondsUntilAutoPause: 300
Tags:
- Key: owner
Value: !Ref StackOwner
- Key: task
Value: !Ref Task
VpcSecurityGroupIds:
- Ref: RDSSecurityGroup
It's a bug.
As a work-around, you can temporarily remove (or comment out) PreferredBackupWindow and PreferredMaintenanceWindow from the template. In my testing, the CF update succeeds and both values are left intact on the instance configuration.
There is actually some debate as to what the expected behavior is. The bug report comments say these settings are not supported in serverless mode, but the AWS Documentation and AWS Console don't seem to support that assertion.
Apparently this is a limitation (among many others) of Aurora Serverless v1: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless.html#aurora-serverless.limitations (it's the last in the list of limitations).
I tried setting it with the AWS CLI v1 and with the AWS Java SDK v2:
software.amazon.awssdk.services.rds.model.ScalingConfiguration scalingConfiguration = software.amazon.awssdk.services.rds.model.ScalingConfiguration.builder().minCapacity(1).build()
final ModifyDbClusterRequest request = ModifyDbClusterRequest.builder().dbClusterIdentifier(clusterIdentifier).preferredBackupWindow("18:00-19:00").scalingConfiguration(scalingConfiguration).build()
final ModifyDbClusterResponse response = rdsClient.modifyDBCluster(request)
It returns the same error regardless.
I have a deployment comprising a managed instance group and two instance templates (A and B). The deployment was initially created with the instance group referencing instance template A.
I tried updating the sourceImage in instance template B using deployment manager (gcloud beta deployment-manager deployments update my-deployment --template ...), but got the following error:
ERROR: (gcloud.beta.deployment-manager.deployments.update) Error in
Operation [operation-1538798895713-57787898f4ae9-8b478716-0bb72a09]:
errors:
- code: NO_METHOD_TO_UPDATE_FIELD
message: No method found to update field 'properties' on
resource 'fwp-app-preprod-instance-template-a' of type
'compute.v1.instanceTemplate'. The resource may need to be
recreated with the new field.
I should make it clear that the only change I made from the original deployment is the instance template's sourceImage.
Is it possible to perform an update of an instance template via deployment manager so that it references an updated sourceImage?
The error states clearly that the resource (instance template) may need to be recreated, and I'm happy for deployment manager to do that. But I have no idea how to instruct/force deployment manager to take that action.
I don't doubt it can be done outside of deployment manager, but I want to avoid configuration drift.
My app.jinja.schema:
imports:
- path: instance-group.jinja
- path: instance-template.jinja
My app.jinja:
resources:
- name: instance-template-a
type: instance-template.jinja
properties:
name: {{ env["deployment"] }}-instance-template-a
machineType: g1-small
sourceImage: "projects/my-project/global/images/my-image"
diskSizeGb: '30'
- name: instance-template-b
type: instance-template.jinja
properties:
name: {{ env["deployment"] }}-instance-template-b
machineType: g1-small
sourceImage: "projects/my-project/global/images/my-image"
diskSizeGb: '30'
- name: fwp-instance-group
type: instance-group.jinja
My instance-group.jinja:
resources:
- name: 'instance-group-{{ env["deployment"] }}'
type: compute.v1.regionInstanceGroupManager
properties:
baseInstanceName: ig-instance-{{ env["deployment"] }}
instanceTemplate: '$(ref.{{ env["deployment"] }}-instance-template-a.selfLink)'
targetSize: 1
region: australia-southeast1
- name: 'autoscaler-{{ env["deployment"] }}'
type: compute.v1.regionAutoscalers
properties:
autoscalingPolicy:
coolDownPeriodSec: 60
loadBalancingUtilization:
utilizationTarget: 0.9
maxNumReplicas: 10
minNumReplicas: 2
target: $(ref.instance-group-{{ env["deployment"] }}.selfLink)
region: australia-southeast1
And my instance-template.jinja
resources:
- name: {{ properties["name"] }}
type: compute.v1.instanceTemplate
properties:
name: {{ properties["name"] }}
description: ''
properties:
machineType: {{ properties["machineType"] }}
tags:
items:
- no-ip
- web-server
- http-server
- https-server
disks:
- type: 'PERSISTENT'
boot: true
mode: 'READ_WRITE'
autoDelete: true
deviceName: instance-device
initializeParams:
sourceImage: {{ properties["sourceImage"] }}
diskType: 'pd-standard'
diskSizeGb: {{ properties["diskSizeGb"] }}
canIpForward: false
networkInterfaces:
- network: projects/my-project/global/networks/vpc-fwp-nonprod
subnetwork: projects/my-project/regions/australia-southeast1/subnetworks/subnet-private-fwp-nonprod
aliasIpRanges: []
labels: { environment: {{ env["deployment"] }}, tenancy: "fwp-nonprod" }
scheduling:
preemptible: false
onHostMaintenance: MIGRATE
automaticRestart: true
nodeAffinities: []
serviceAccounts:
- email: some-service-account#developer.gserviceaccount.com
scopes:
- https://www.googleapis.com/auth/cloud-platform
To recap the comments:
The DM config includes an instance template for the managed instance group. The change of source image is attempting to change the image used in the template.
Unfortunately, instance templates are immutable once created
"So it is not possible to update an existing instance template or change an instance template after it has been created."
This explains the error message returned. The proper way to change the image you want to use for a Managed Instance Group is to create a new template and perform a rolling update on the group and using the new instance template.
I'm trying to create an unmanaged instanceGroup with several VM's in it via Deployment Manager Configuration (YAML file).
I can easily find docs about addInstances via Google API, but couldn't find docs about how to do this in a YAML file:
instances
instanceGroups
What properties should be included in instances/instanceGroup resource to make it work?
The YAML below will create a compute engine instance, create an unmanaged instance group, and add the instance to the group.
resources:
- name: instance-1
type: compute.v1.instance
properties:
zone: australia-southeast1-a
machineType: zones/australia-southeast1-a/machineTypes/n1-standard-1
disks:
- deviceName: boot
type: PERSISTENT
diskType: zones/australia-southeast1-a/diskTypes/pd-ssd
boot: true
autoDelete: true
initializeParams:
sourceImage: projects/debian-cloud/global/images/debian-9-stretch-v20180716
networkInterfaces:
- network: global/networks/default
accessConfigs:
- name: External NAT
type: ONE_TO_ONE_NAT
- name: ig-1
type: compute.v1.instanceGroup
properties:
zone: australia-southeast1-a
network: global/networks/default
- name: ig-1-members
action: gcp-types/compute-v1:compute.instanceGroups.addInstances
properties:
project: YOUR_PROJECT_ID
zone: australia-southeast1-a
instanceGroup: ig-1
instances: [ instance: $(ref.instance-1.selfLink) ]
There is no possibility right now, to do it with gcloud deployment manager.
This was tested and it seemed that while Google Deployment Manager was able to complete without issue having the following snippet:
{
"instances": [
{
"instance": string
}
]
}
it did not add the instances specified, but created the IGM.
However Terraform seems to be able to do it https://www.terraform.io/docs/providers/google/r/compute_instance_group.html
I think #mcourtney answer is correct.
I just had this scenario and i used python template with yaml config to add instances to an un-managed instance group.
Here is the snippet of resource definition in my python template :
{
'name': name + '-ig-members',
'action': 'gcp-types/compute-v1:compute.instanceGroups.addInstances',
'properties': {
'project': '<YOUR PROJECT ID>',
'zone' : context.properties['zone'], // Defined in config yaml
'instanceGroup': '<YOUR Instance Group name ( not url )>',
"instances": [
{
"instance": 'projects/<PROJECT ID>/zones/<YOUR ZONE>/instances/<INSTANCE NAME>'
}
]
}
}
Reference API is documented here :
https://cloud.google.com/compute/docs/reference/rest/beta/instanceGroups/addInstances
This is just an example. you can abstract all the hard coded things to either yaml configuration or variables at the top of python template.