Here is my cloud formation template which passes event patter to the sub stack which in fact creates the rule depending on the event data.
cloudwatchRule:
Type: AWS::CloudFormation::Stack
Properties:
TemplateURL: !Sub "${s3Path}/cw-rule.yml"
Parameters:
eventPattern: !Join
- ' '
- - '{"source":["aws.iam"],"detail-type":["AWS API Call via CloudTrail"],"detail":{"eventSource":["iam.amazonaws.com"],"eventName":['
- Fn::Split:
- ','
- !Sub ${ssmParamWhichContainsEventNames}
- ']}}'
ruleState: "ENABLED"
#The value of ssmParamWhichContainsEventNames is of format #"CreateServiceSpecificCredential,DeactivateMFADevice"
When I run this I get the following error
Template error: every Fn::Join object requires two parameters, (1) a string delimiter and (2) a list of strings to be joined or a function that returns a list of strings (such as Fn::GetAZs) to be joined.. Rollback requested by user.
I have tried various techniques to format the order of !Join !Split !Sub
I have also tried using Fn::Join (full function format) but it keeps failing.
eventName in the eventPattern parameter expects the input in following format.
"eventName":["event1","event2","event3","event4"]
My SSM variable has event names in the format "event1,event2,event3..." To make it compatible with eventName and make the cloudwatch rule run, I'll have to transform "event1,event2,event3..." to '"event1","event2","event3"...'
One option is that I convert the SSM to my acceptable format but this is the thing I want to avoid for some reason.
Can anyone help me figure out the way to transform the "CreateServiceSpecificCredential,DeactivateMFADevice" to ' "CreateServiceSpecificCredential","DeactivateMFADevice" ' (each value enclosed within double quotes and whole string enclosed within single quotes
I keep feeling that I'm not correctly writing the intrinsic functions in the above code in the correct order.
In case somebody is still looking for how to make it happen. Here is the cloudformation template piece that split comma separated string and make it into an array of individual elements to be injected into the "eventName" attribute:
cloudwatchRule:
Type: AWS::CloudFormation::Stack
Properties:
TemplateURL: !Sub "${s3Path}/cw-rule.yml"
Parameters:
eventPattern: !Sub
- |
{"source":["aws.iam"],"detail-type":["AWS API Call via CloudTrail"],"detail":{"eventSource":["iam.amazonaws.com"],"eventName":[
"${eventNames}"
]}}
- eventNames: !Join
- '","'
- !Split
- ','
- !Ref ssmParamWhichContainsEventNames
ruleState: "ENABLED"
Related
I use Terraform to launch a Cloudformation stack to create Glue Databrew resources that don't exist yet on Terraform.
The thing is that I've a variable in Terraform that corresponds to the list of my data sources and in order to create the databrew resources associated to this data, I loop over my list to create one instance of my Cloudformation template for each data source.
Inside this template, I've a resource that I want to be different per data source. It correspond to the AWS::DataBrew::Ruleset resource.
It looks like this :
DataBrewDataQualityRuleset:
Type: AWS::DataBrew::Ruleset
Properties:
Name: !Ref RuleSetName
Description: Data Quality ruleset
Rules:
- Name: Check columns for missing values
Disabled: false
CheckExpression: AGG(MISSING_VALUES_PERCENTAGE) == :val1
SubstitutionMap:
- ValueReference: ":val1"
Value: '0'
ColumnSelectors:
- Regex: ".*"
- Name: Check two
Disabled: false
CheckExpression: :col IN :list
SubstitutionMap:
- ValueReference: ":col"
Value: "`group`"
- ValueReference: ":list"
Value: "[\"Value1\", \"Value2\"]"
TargetArn: !Sub SomeArn
What I want to do is, extract the Rules part of the component and create one file where I will put all my rules per data sources. In fact having something like below :
DataBrewDataQualityRuleset:
Type: AWS::DataBrew::Ruleset
Properties:
Name: !Ref RuleSetName
Description: Data Quality ruleset
Rules: !Ref Rules
TargetArn: !Sub SomeArn
And in my terraform, my Rules parameter would be my actual set of rules for one particular data source.
I've thought about having one YAML file from which I would loop on terraform but I'm not sure it's doable and if cloudformation would accept YAML as parameter type.
Below you'll also find my terraform component :
resource "aws_cloudformation_stack" "databrew_jobs" {
for_each = var.data_sources
name = "datachecks-${each.value.stack_name}"
parameters = {
Bucket = "test_bucket"
DataSetKey = "raw/${each.value.job_name}"
DataSetName = "dataset-${each.value.stack_name}"
RuleSetName = "ruleset-${each.value.stack_name}"
JobName = "profile-job-${each.value.stack_name}"
DataSourceName = "${each.value.stack_name}"
JobResultKey = "databrew-results/${each.value.job_name}"
RoleArn = iam_role_test.arn
}
template_body = file("${path.module}/databrew-job.yaml")
}
Do you have any idea how could I achieve this ?
Thanks in advance !
I'm trying to create a metric filter in a CloudWatch template which includes a colon:
e.g.
TotalLocationFound:
Type: AWS::Logs::MetricFilter
Properties:
FilterPattern: "abc_found: True"
LogGroupName: "/aws/lambda/blah"
MetricTransformations:
-
MetricValue: "1"
MetricNamespace: "ProductionClient"
MetricName: "TotalAbcFound"
It seems to take issue with the filter pattern. I can use that same pattern from the console but when I deploy using CloudWatch command line I get this error:
Invalid metric filter pattern (Service: AWSLogs; Status Code: 400; Error Code: InvalidParameterException
Playing with it seems to point to the issue being the :
Thanks
You should try with quotation in the filter pattern. From docs:
Metric filter terms that include characters other than alphanumeric or underscore must be placed inside double quotes ("").
This the FilterPattern could be:
FilterPattern: '"abc_found: True"'
You may try different ways of escaping double quotes in CloudFormation if this does not work as expected.
As mentioned already, if you want to include characters other than alphanumeric or underscore in your FilterPattern, it must be placed inside double quotes. If you are creating this resource via CloudFormation, then you will also need to escape the enclosing quotations as well.
For example, to achieve a FilterPattern of:
[::Application.Services.Unhealthy::]
In YAML CloudFormation the FilterPattern property should be:
FilterPattern: "\"[::Application.Services.Unhealthy::]\""
Full YAML CloudFormation for this resource would look something like this:
UnHealthyMetricFilter:
Type: AWS::Logs::MetricFilter
Properties:
FilterPattern: "\"[::Application.Services.Unhealthy::]\""
LogGroupName: /aws/lambda/my-lambda
MetricTransformations:
-
MetricValue: 1
MetricNamespace: "health/dev"
MetricName: "Unhealthy"
So I have my CloudFormation template defined to include a Parameters section with several parameters including
Parameters:
DefaultLimit:
Type: Number
I also have a GraphQL API defined in which I am using AppSync PIPELINE resolvers to run multiple operations in sequence.
QueryResolver:
Type: AWS::AppSync::Resolver
DependsOn: AppSyncSchema
Properties:
ApiId: !GetAtt [AppSyncAPI, ApiId]
TypeName: Query
FieldName: getData
Kind: PIPELINE
PipelineConfig:
Functions:
- !GetAtt AuthFunction.FunctionId
- !GetAtt ScanDataFunction.FunctionId
RequestMappingTemplate: |
{
# Inject value of DefaultLimit in $context object
}
ResponseMappingTemplate: "$util.toJson($context.result)"
That all works as expected, except for injecting CFN parameter values in mapping templates.
The issue I am having is this -- I would like to pass the value of DefaultLimit to the before RequestMappingTemplate so that the value is available to the ScanDataFunction. The goal is for that value be used as the default limit value when the second function does, say a DynamoDB scan operation, and returns paginated results.
My current approach is to hardcode a default value of 20 for limit in the request mapping template of the ScanDataFunction. I am using a DynamoDB resolver for this operation. Instead, I would like to inject the parameter value because it would give me the flexibility to set different default values for different deployment environments.
Any help with this would be appreciated.
The | character in YAML starts a block and what you enter indented after that is all treated as text.
CloudFormation isn't going to process any of that. The solution I have generally seen is to use the Join intrinsic function. It ends up looking pretty bad can be difficult to maintain so I recommend using it sparingly. Below is a rough possible example:
Parameters:
DefaultLimit:
Type: Number
Resourece:
QueryResolver:
Type: AWS::AppSync::Resolver
DependsOn: AppSyncSchema
Properties:
ApiId: !GetAtt [AppSyncAPI, ApiId]
TypeName: Query
FieldName: getData
Kind: PIPELINE
PipelineConfig:
Functions:
- !GetAtt AuthFunction.FunctionId
- !GetAtt ScanDataFunction.FunctionId
RequestMappingTemplate:
Fn::Join:
- ""
- - "Line 1 of the template\n"
- "Line 2 of the template, DefaultList="
- Ref: DefaultLimit
- "\nLine 3 of the template"
ResponseMappingTemplate: "$util.toJson($context.result)"
Untested code warning
When I'm creating CloudFormation template with the use of interactive Parameters, I can define the type of List<> to be able to select multiple values, for example:
SubnetIds:
Type: List<AWS::EC2::Subnet::Id>
Description: Select multiple subnets from selected VPC.
Default: "????"
or:
SecurityGroups:
Type: List<AWS::EC2::SecurityGroup::Id>
Description: Select security groups.
Default: "???"
The question is how do I pre-set default value with multiple selections? if default takes only the string instead of a list, and string with commas between multiple values also doesn't help
Any ideas? please, hint me
I recently run in the same issue.
The answer is simple - there should be no spaces in your comma-separated list.
So it would look like:
SecurityGroups:
Type: List<AWS::EC2::SecurityGroup::Id>
Description: Select security groups.
Default: "sg-11111111,sg-22222222"
And this way the values would be preselected in your template.
P.S. Do not try CommaDelimitedList or so - it won't work in the way you want. The string values would be selected, but not the actual security groups.
Source: https://forums.aws.amazon.com/thread.jspa?threadID=165144
We are automating a lambda via SAM to run on a Schedule Event. We use YAML but we are unable to work out how to use !Sub to make the Input be dynamic.
If you read the sam documentation it says that Input needs to be a JSON formatted string
The following code works for us:
Events:
Event1:
Type: Schedule
Properties:
Schedule: rate(1 minute)
Input: >-
{
"sqsUrl": "https://sqs.12344.url",
"snsArn": "arn:val"
}
But we need to insert dynamic params into the Input like so:
Events:
Event1:
Type: Schedule
Properties:
Schedule: rate(1 minute)
Input: >-
{
"sqsUrl": "https://sqs.${AWS::AccountId}.url",
"snsArn": "arn:val"
}
We have tried to do this in multiple ways, using a !Sub but the deployment always fails saying that it needs to be valid JSON.
What is the correct way to make this JSON string use variables?
Thanks,
Mark
So, you should wrap all Input value (in your case this is json-string and of course it should be wrapped with some quotes) with the !Sub function.
Then, it will look like:
Input:
Fn::Sub: '{"sqsUrl": "https://sqs.${AWS::AccountId}.url","snsArn": "arn:val"}'
I've used something like:
!Sub |
{
"sqsUrl": "https://sqs.${AWS::AccountId}.url",
"snsArn": "arn:val"
}
the | (and >- among others) define the way yaml handles the line breaks in the string.