Update AWS Athena workgroup using CloudFormation template - amazon-web-services

I have 2 templates those I have taken from the AWS::Athena::WorkGroup - AWS CloudFormation documentation.
The first template athena_create.yaml works as expected. The second template needs to modify the workgroup created in the first template. But I get an error:
MyCustomWorkGroup already exists in stack
arn:aws:cloudformation:us-east-1:XXX:stack/a1/7cc670a0-8d19-11ea-872c-12217e59f19f
Here is the code. create template works correctly.
athena_create.yaml
Resources:
MyAthenaWorkGroup:
Type: AWS::Athena::WorkGroup
Properties:
Name: MyCustomWorkGroup
Description: My WorkGroup
State: ENABLED
Tags:
- Key: "key1"
Value: "value1"
- Key: "key2"
Value: "value2"
WorkGroupConfiguration:
BytesScannedCutoffPerQuery: 200000000
EnforceWorkGroupConfiguration: false
PublishCloudWatchMetricsEnabled: false
RequesterPaysEnabled: true
ResultConfiguration:
OutputLocation: s3://path/to/my/bucket/
athena_update.yaml
Resources:
MyAthenaWorkGroup:
Type: AWS::Athena::WorkGroup
Properties:
Name: MyCustomWorkGroup
Description: My WorkGroup Updated
State: DISABLED
Tags:
- Key: "key1"
Value: "value1"
- Key: "key2"
Value: "value2"
WorkGroupConfigurationUpdates:
BytesScannedCutoffPerQuery: 10000000
EnforceWorkGroupConfiguration: true
PublishCloudWatchMetricsEnabled: true
RequesterPaysEnabled: false
ResultConfigurationUpdates:
EncryptionConfiguration:
EncryptionOption: SSE_S3
OutputLocation: s3://path/to/my/bucket/updated/
The update template mentioned above does not work as expected.

The reason for the error is that the two templates were used to create two independent stacks. This didn't work because they two Athena WorkGroups of same Name: MyCustomWorkGroup.
The correct way to perform create and update the MyCustomWorkGroup is as follows:
Create a stack using athena_create.yaml file.
Once the stack is created, use its Update option to upload athena_update.yaml which is going to update the stack.

Related

Template validation failed when using SAM syntax instead of CloudFormation syntax for step function

I have the following step function in my AWS SAM template, it was defined using the syntax in the documentation. I'm using intrinsic functions to get some pseudoparameters but something is going wrong with them.
SfdcOrderEventsStepFunction:
Type: AWS::Serverless::StateMachine
Properties:
DefinitionSubstitutions:
Region: !Ref "AWS::Region"
AccountId: !Ref "AWS::AccountId"
EventBusPublishTarget: "order-events"
DefinitionUri: sfn-definition.asl.yml
Events:
EventBridgeEvent:
Type: EventBridgeRule
Properties:
EventBusName: sfdc-events
Pattern:
# TODO: Update pattern when the salesforce service is ready
source:
- prefix: salesforce.com
detail-type:
- Order
detail:
Payload__c:
attributes:
type:
- order
InputPath: $.detail
Name: sfdc-order-events
Role: !Sub 'arn:aws:iam::${AWS::AccountId}:role/stepfunction_execution_role'
Tracing:
Enabled: true
when I try to deploy it shows me the following error:
Resource template validation failed for resource
SfdcOrderEventsStepFunction as the template has invalid properties.
Please refer to the resource documentation to fix the template.
Properties validation failed for resource SfdcOrderEventsStepFunction
with message:
#/De finitionSubstitutions/ AccountId: 2 subschemas matched instead of one
#/DefinitionSubstitutions/AccountId: expected type: Boolean, found: String
At the end it deploys without problems. The step function and all of its components run as expected and without errors, but I wanted to know if there if something I can do to fix the template.

Pathid required (but not always) in CloudFormationProvisionedProduct

I am trying to build a CloudFormationProvisionedProduct. From the cli, no issue:
aws servicecatalog provision-product --product-id prod-123456abcd --provisioning-artifact-name 1.49 \
--provisioned-product-name whatisgoingon\
--provisioning-parameters '[{"Key":"Name","Value":"RStudioIngress-CFCLI"},{"Key":"Host","Value":"rstudio-cloudformedcli"},{"Key":"OriginHostname","Value":"rstudio-cf.alb.***"},{"Key":"ForwardOriginalHostHeader","Value":"true"},{"Key":"AuthType","Value":"SSO"},{"Key":"AccessByApimGw","Value":"false"},{"Key":"ApimEnabled","Value":"false"},{"Key":"AppRoles","Value":"[]"},{"Key":"GroupsOnly","Value":""}]'
This works perfectly fine.
When I run the same via cloudformation:
Ingress:
Properties:
#PathId: lpv2-***
ProductId: prod-123456abcd
ProvisionedProductName: whatisgoingon
ProvisioningArtifactName: "1.49"
ProvisioningParameters:
- Key: Name
Value: RStudioIngress-CFCLI
- Key: Host
Value: rstudio-cloudformedcli
- Key: OriginHostname
Value: rstudio-cf.alb.***
- Key: ForwardOriginalHostHeader
Value: "true"
- Key: AuthType
Value: SSO
- Key: AccessByApimGw
Value: "false"
- Key: ApimEnabled
Value: "false"
- Key: AppRoles
Value: "[]"
- Key: GroupsOnly
Value: ""
Type: AWS::ServiceCatalog::CloudFormationProvisionedProduct
I get: No launch paths found for resource. If I find the relevant path (aws servicecatalog list-launch-paths --product-id prod-123456abcd) and give it via PathId in the couldformation template, I get Invalid PathId. There is only one path anyway, so it should not be required.
Question:
Why is there a difference between the cli and cloudformation call, and how can I fix it?

How to get the ARN of an SSM Document in CloudFormation?

I have a CloudFormation template that creates an AWS::Events::Rule and an AWS::SSM::Document. I need to provide a list of Targets for the SSM::Rule, but each target expects an ARN:
mySSMDocument:
Type: AWS::SSM::Document
Properties:
DocumentType: 'Command'
Content:
schemaVersion: '2.2'
description: "Code that will be run on EC2"
mainSteps:
- action: "aws:runShellScript"
name: runShellScript
inputs:
runCommand:
- 'Some command to execute'
myEventRule:
Type: AWS::Events::Rule
Properties:
Description: "A description for the Rule."
EventPattern:
source:
- "aws.autoscaling"
detail-type:
- "EC2 Instance-terminate Lifecycle Action"
detail:
AutoScalingGroupName:
- !Ref 'someAutoScalingGroupInThisTemplate'
RoleArn: 'some role ARN'
State: "ENABLED"
Targets:
- Id: "some-unique-id"
Arn: <-- This is the value that I need to fill in.
RunCommandParameters:
RunCommandTargets:
- Key: "tag: Name"
Values:
- 'The name of the EC2 machine'
I think that I need to replace the <-- This is the value that I need to fill in. with the ARN of mySSMDocument, but I don't see any way to retrieve this value from within the template itself. The documentation does not specify any GetAtt functionality on SSM::Document that allows to get the ARN. Anyone know how to solve this issue?
This is ARN pattern of Document
arn:${Partition}:ssm:${Region}:${Account}:document/${DocumentName}
example:
arn:aws:ssm:us-east-2:12345678912:document/demoooo
You can use Ref function to get name of document, then Sub to create final ARN
refer: https://docs.aws.amazon.com/IAM/latest/UserGuide/list_awssystemsmanager.html#awssystemsmanager-resources-for-iam-policies
!Sub arn:${AWS::Partition}:ssm:${AWS::Region}:${AWS::AccountId}:document/${mySSMDocument}
You can produce the ARN format for AWS::SSM::Document using the return Value for AWS::SSM::Document, the Pseudo Parameters for Partition, Region, and AccountId, and the Sub intrinsic function

CloudFormation pass tags

I want to have an option to pass to my CFT tags so I could tag S3 bucket that I am creating and that it will not be mandatory - to have default value. How ever I cannot find any parameter type that will do the trick
In order to set a default value to cloudformation parameters, you need to use the Default attribute when you define the parameter.
The following template snippet creates S3 Bucket and add the relevant tag(single one!) based on user input :
AWSTemplateFormatVersion: 2010-09-09
Description: Create S3 Bucket with relevant tags
Parameters:
S3TagKey:
Description: The S3 Tag key
Type: String
Default: Your Default key
S3TagValue:
Description: The S3 Tag Value
Type: String
Default: Your Default Value
Resources:
ExampleS3:
Type: 'AWS::S3::Bucket'
Properties:
BucketName: s3-name
Tags:
- Key: !Ref S3TagKey
Value: !Ref S3TagValue
PublicAccessBlockConfiguration:
BlockPublicAcls: true
BlockPublicPolicy: true
IgnorePublicAcls: true
RestrictPublicBuckets: true
When you click on create stack you will be asked to provide the key-value pair values, As you can see the default values are set already. In case you want to change the tags, just update the relevant field.
Results :

Use a Stackdriver resource group's ID in a GCP Deployment Manager configuration

I'm trying to create a Stackdriver alert policy with a Deployment Manager configuration. The same configuration first creates a resource group and a notification channel and then a policy based on those:
resources:
- name: test-group
type: gcp-types/monitoring-v3:projects.groups
properties:
displayName: A test group
filter: >-
resource.metadata.cloud_account="aproject-id" AND
resource.type="gce_instance" AND
resource.metadata.tag."managed"="yes"
- name: test-email-notification
type: gcp-types/monitoring-v3:projects.notificationChannels
properties:
displayName: A test email channel
type: email
labels:
email_address: incidents#example.com
- name: test-alert-policy
type: gcp-types/monitoring-v3:projects.alertPolicies
properties:
enabled: true
displayName: A test alert policy
documentation:
mimeType: text/markdown
content: "Test incident"
notificationChannels:
- $(ref.test-email-notification.name)
combiner: OR
conditions:
- conditionAbsent:
aggregations:
- alignmentPeriod: 60s
perSeriesAligner: ALIGN_RATE
duration: 300s
filter: metric.type="compute.googleapis.com/instance/uptime" group.id="$(ref.test-group.id)"
trigger:
count: 1
displayName: The instance is down
The policy's only condition has a filter based on the resource group, i.e. only the members of the group could trigger this alert.
I'm trying to use a reference to the group's ID, but it doesn't work - "The reference 'id' is invalid, reason: The field 'id' does not exists on the reference schema.
Also when I try to use $(ref.test-group.selfLink) I get The reference 'selfLink' is invalid, reason: The field 'selfLink' does not exists on the reference schema.
I could get the group's name (e.g. "projects/aproject-id/groups/3691870619975147604") but the filters only accept group IDs (e.g. only the "3691870619975147604" part):
'{"ResourceType":"gcp-types/monitoring-v3:projects.alertPolicies","ResourceErrorCode":"400","ResourceErrorMessage":{"code":400,"message":"Field
alert_policy.conditions[0].condition_absent.filter had an invalid value of \"metric.type=\"compute.googleapis.com/instance/uptime\"
group.id=\"projects/aproject-id/groups/3691870619975147604\"\":
must specify a restriction on \"resource.type\" in the filter; see \"https://cloud.google.com/monitoring/api/resources\"
for a list of available resource types.","status":"INVALID_ARGUMENT","statusMessage":"Bad
Request","requestPath":"https://monitoring.googleapis.com/v3/projects/aproject-id/alertPolicies","httpMethod":"POST"}}'
Try replacing your alert policy with the following:
- name: test-alert-policy
type: gcp-types/monitoring-v3:projects.alertPolicies
properties:
enabled: true
displayName: A test alert policy
documentation:
mimeType: text/markdown
content: "Test incident"
notificationChannels:
- $(ref.test-email-notification.name)
combiner: OR
conditions:
- conditionAbsent:
aggregations:
- alignmentPeriod: 60s
perSeriesAligner: ALIGN_RATE
duration: 300s
filter: metric.type="compute.googleapis.com/instance/uptime" $(ref.test-group.filter)
trigger:
count: 1
displayName: The instance is down
metadata:
dependsOn:
- test-group
This adds 1) an explicit dependency to test-group using a dependsOn clause and 2) $(ref.test-group.filter) to the metric filter so that it, while not strictly linked to test-group, ends up containing all the same resources as test-group.
As Deployment Manager resources are ran in parallel its necessary to use dependsOn to ensure test-group is instantiated before attempting to create test-alert-policy; apparently Deployment Manager isn't quite smart enough to reason this just by the references.