I am creating an AWS CloudWatch dashboard using CloudFormation and trying to use Fn::FindInMap in the DashboardBody body. However, the value is not getting replaced successfully. Can someone please help?
Below is my code:
Mappings:
EnvSource:
Dev:
"SMSFailed": sns/us-east-1/XXX/DirectPublishToPhoneNumber/Failure
Resources:
MonitoringDashboard:
Type: "AWS::CloudWatch::Dashboard"
Properties:
DashboardName:
Monitoring-Test-Dashboard
DashboardBody:
!Sub "{\"widgets\":[{\"height\":6,\"width\":12,\"y\":1,\"x\":12,\"type\":\"log\",\"properties\":{\"query\":\"SOURCE 'Fn::FindInMap' : [ 'ABC', 'DomainParameters', 'DomainName'] | fields #timestamp, #message, delivery.providerResponse, status\\n| filter #message like //\\n|stats count(status) as ErrorCount by delivery.providerResponse\\n| sort #timestamp asc\",\"region\":\"us-east-1\",\"title\":\"SMS Failed\",\"view\":\"table\"}}]}"
Output:
You can use a list form of Sub:
Resources:
MonitoringDashboard:
Type: "AWS::CloudWatch::Dashboard"
Properties:
DashboardName:
Monitoring-Test-Dashboard
DashboardBody:
!Sub
- "{\"widgets\":[{\"height\":6,\"width\":12,\"y\":1,\"x\":12,\"type\":\"log\",\"properties\":{\"query\":\"SOURCE ${FindInMap} | fields #timestamp, #message, delivery.providerResponse, status\\n| filter #message like //\\n|stats count(status) as ErrorCount by delivery.providerResponse\\n| sort #timestamp asc\",\"region\":\"us-east-1\",\"title\":\"SMS Failed\",\"view\":\"table\"}}]}"
- DomainName: !FindInMap [ 'ABC', 'DomainParameters', 'DomainName']
Related
I'm implementing substacks in my AWS serverless app, I would like to know if there is a way to extract the from my template of appsync resolvers the content that is in the field RequestMappingTemplate, and my ResponseMappingTemplate
currently the resolver resource looks like this
AppSyncResolver:
Type: "AWS::AppSync::Resolver"
Properties:
TypeName: "Query"
FieldName: "my_resolver"
DataSourceName: !GetAtt AppSyncDataSource.Name
RequestMappingTemplate: |
#set($SCHEMA = $ctx.args.schema)
#set($limit = $ctx.args.limit)
#set($table = ".customers")
#set($customers = $SCHEMA$table_ordendscto_lote)
{
"version": "2018-05-29",
"statements": [
"SELECT * FROM $customers LIMIT $limit ;"
]
}
ResponseMappingTemplate: |
## Raise a GraphQL field error in case of a datasource invocation error
#if($ctx.error)
$utils.error($ctx.error.message, $ctx.error.type)
#end
$utils.toJson($utils.rds.toJsonObject($ctx.result)[0])
Kind: "UNIT"
ApiId: !GetAtt AppSyncGraphQLApi.ApiId
and I wanted something like this
AppSyncResolver:
Type: "AWS::AppSync::Resolver"
Properties:
TypeName: "Query"
FieldName: "my_resolver"
DataSourceName: !GetAtt AppSyncDataSource.Name
RequestMappingTemplate: src/resolvers/my-resolver
ResponseMappingTemplate: src/resolvers/my-resolver-response
Kind: "UNIT"
ApiId: !GetAtt AppSyncGraphQLApi.ApiId
is it possible?
I have the following resource in my SAM template.yml:
MetricsRule:
Type: AWS::Events::Rule
Properties:
Name: MetricsRule
Description: Puts metrics to the CloudWatch log group
EventBusName: !FindInMap [LambdaConfig, !Ref stage, eventBusName]
EventPattern:
detail:
status:
- GENERATED
State: !FindInMap [LambdaConfig, !Ref stage, eventEnabled]
Targets:
- Id: LogGroupTarget
Arn: !Sub "arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:${LogGroup}"
InputTransformer:
InputPathsMap:
customer-id: $.detail.customerId
destination-type: $.detail.destinationType
provider-id: $.detail.providerId
InputTemplate: '"cusId = <customer-id> - <destination-type> for <provider-id>"'
If I try to deploy this stack it fails with the exception Input for target LogGroupTarget is not a valid JSON text.
UPD:
If I put InputTemplate as '{"message":"cusId = <customer-id> - <destination-type> for <provider-id>"}' it will work. But, in the documentation, examples and even in the placeholder of the field in UI it stands that Input Template: A string containing placeholders which will be filled with values defined in Input Paths e.g. "The state of Instance <instance> is <state>"
Is there any option to specify InputTemplate as a string?
Thanks.
I hope author found answer.
But just to have answer here too.
This should work:
InputTemplate: |
"cusId = <customer-id> - <destination-type> for <provider-id>"
More information is here: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-events-rule-inputtransformer.html
I am trying to create a GLUE table from S3 csv file , below is my CF template
#######################################################################################################
# AWS RESOURCE CHILD STACKS CONFIGURATION DETAILS
#######################################################################################################
Resources:
TempDataMasterTable:
Type: AWS::Glue::Table
Properties:
DatabaseName: "temp_db"
CatalogId: !Ref AWS::AccountId
TableInput:
Name: "temp_table"
Description: "Master table"
TableType: EXTERNAL_TABLE
Parameters: { "classification" : "csv", "compressionType" : "none", "typeOfData": "file" }
StorageDescriptor:
Location: s3://temp-location/temp-folder/
InputFormat: ''
OutputFormat: ''
SerdeInfo:
Parameters:
serialization.format: ','
field.delim: ','
Columns:
- {"Name": "name", "Type": "string"}
- {"Name": "lastname", "Type": "string"}
When I go to Athena console , I can see the table has been created but on trying to query the table , I get the error - " HIVE_UNKNOWN_ERROR: Unable to create input format on Athena" so clearly I am doing something wrong while giving the input or output format in the CF template. Could you please advise what should be the format in the CF template. Also the location here -"s3://temp-location/temp-folder/" has files for each day , so format filename-date. Could you please advise what's wrong with the CF template that Athena can not read this table ?
You should pass org.apache.hadoop.mapred.TextInputFormat for input and org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat for output. This template also needs to have SerdeInfo which contains the library that will help you in reading the data from S3.Below is a sample template which worked for me in Athena.Refer to this for more examples.
AWSTemplateFormatVersion: 2010-09-09
Description: Glue Table
Resources:
MyGlueTable:
Type: AWS::Glue::Table
Properties:
DatabaseName: ddddd
CatalogId: !Ref AWS::AccountId
TableInput:
Name: my_glue_table
TableType: EXTERNAL_TABLE
StorageDescriptor:
Columns:
- {"Name": "name", "Type": "string"}
- {"Name": "lastname", "Type": "string"}
Compressed: false
InputFormat: org.apache.hadoop.mapred.TextInputFormat
Location: s3://<>
NumberOfBuckets: -1
OutputFormat: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
SerdeInfo:
SerializationLibrary: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
SortColumns: []
StoredAsSubDirectories: false
Finally found the solution to the problem , below were the parameters that were changed and fixed the issue
InputFormat: 'org.apache.hadoop.mapred.TextInputFormat'
OutputFormat: 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
SerdeInfo:
Parameters:
serialization.format: ','
field.delim: ','
SerializationLibrary: 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
To answer this more broadly - a better solution to figure out these cryptic parameters is to create a test crawler and crawl your S3 data with it by 'Run Crawler'. Once that is done you can go to Athena and see your new table created by the Crawler. If you open the table properties you can see the following among other config details
and then use these properties in your cloud formation template . This is how I fixed the issue and can be used for other formats as well
I have a CloudFormation template that creates an AWS::Events::Rule and an AWS::SSM::Document. I need to provide a list of Targets for the SSM::Rule, but each target expects an ARN:
mySSMDocument:
Type: AWS::SSM::Document
Properties:
DocumentType: 'Command'
Content:
schemaVersion: '2.2'
description: "Code that will be run on EC2"
mainSteps:
- action: "aws:runShellScript"
name: runShellScript
inputs:
runCommand:
- 'Some command to execute'
myEventRule:
Type: AWS::Events::Rule
Properties:
Description: "A description for the Rule."
EventPattern:
source:
- "aws.autoscaling"
detail-type:
- "EC2 Instance-terminate Lifecycle Action"
detail:
AutoScalingGroupName:
- !Ref 'someAutoScalingGroupInThisTemplate'
RoleArn: 'some role ARN'
State: "ENABLED"
Targets:
- Id: "some-unique-id"
Arn: <-- This is the value that I need to fill in.
RunCommandParameters:
RunCommandTargets:
- Key: "tag: Name"
Values:
- 'The name of the EC2 machine'
I think that I need to replace the <-- This is the value that I need to fill in. with the ARN of mySSMDocument, but I don't see any way to retrieve this value from within the template itself. The documentation does not specify any GetAtt functionality on SSM::Document that allows to get the ARN. Anyone know how to solve this issue?
This is ARN pattern of Document
arn:${Partition}:ssm:${Region}:${Account}:document/${DocumentName}
example:
arn:aws:ssm:us-east-2:12345678912:document/demoooo
You can use Ref function to get name of document, then Sub to create final ARN
refer: https://docs.aws.amazon.com/IAM/latest/UserGuide/list_awssystemsmanager.html#awssystemsmanager-resources-for-iam-policies
!Sub arn:${AWS::Partition}:ssm:${AWS::Region}:${AWS::AccountId}:document/${mySSMDocument}
You can produce the ARN format for AWS::SSM::Document using the return Value for AWS::SSM::Document, the Pseudo Parameters for Partition, Region, and AccountId, and the Sub intrinsic function
I want to create a crawler resource from CFN (Cloudformation).
Here is my code:
Type: AWS::Glue::Crawler
Properties:
Name: !Ref GlueCrawlerName
Role: !GetAtt crawlerRole.Arn
Description: AWS Glue crawler to crawl DLG data
DatabaseName: !Ref GlueDatabaseName
Targets:
S3Targets:
- Path:
!Join
- ''
- - 's3://'
- !Ref s3bucket
- '/'
- !Ref GlueTableName
SchemaChangePolicy:
UpdateBehavior: UPDATE_IN_DATABASE
DeleteBehavior: DEPRECATE_IN_DATABASE
Schedule:
ScheduleExpression: cron(0 1 * * ? 2019)
Everything is alright as expected, only 'Create a single schema for each S3 path' is false. Which property is for this to set to true?
Do you need one table per subfolder or only one table at the root level of the s3 path?
for single root level table, append following in your CFN:
Configuration: "{\"Version\":1.0,\"Grouping\":{\"TableGroupingPolicy\":\"CombineCompatibleSchemas\"}}"
Maybe it will be helpful. As per AWS documentation:
Set the Configuration field with a string representation of the
following JSON object in the crawler API:
{
"Version": 1.0,
"Grouping": {
"TableGroupingPolicy": "CombineCompatibleSchemas" }
}