I'm new to AWS services in general, so this may be a simple where I've gone down a wrong path.
I have an application that consists of some AWS lambda functions, with an API Gateway to access the routes.
I'm trying to make it so I can deploy the application across a dev, staging and prod environment.
In adding a unique domain to the API gateway I end up with a gateway config in my template.yaml like so:
Resources:
ApiGatewayApi:
Type: AWS::Serverless::HttpApi
Properties:
StageName: "example-stage-name"
CorsConfiguration:
AllowOrigins:
- !Sub "${CorsOrigins}"
Domain:
DomainName: !Sub "${APIDomain}"
CertificateArn: !Sub "${APIDomainARN}"
EndpointConfiguration: REGIONAL
Route53:
HostedZoneId: !Sub "${APIDomainZoneID}"
I'm using Parameters for the domain values so they can be different for each environment (i.e. application-dev.example.com, application-staging.example.com, application.example.com).
In my samconfig.toml I end up with a long list of parameter overrides to configure this along with the other parameters I have in my app.
version = 0.1
[dev]
[dev.deploy]
[dev.deploy.parameters]
stack_name = "application-dev"
s3_bucket = "application-dev.example.com"
s3_prefix = "dev-env"
region = "XX-YYYY-Z"
capabilities = "CAPABILITY_IAM"
parameter_overrides = "MY_ENV=dev APIDomain=application-dev.example.com APIDomainARN=arn:aws:acm:XX-YYYY-Z:123456789123:certificate/12345678-1234-1234-1234-123456789123 APIDomainZoneID=ABCDEGHIJ CorsOrigins=https://other.example.com"
Is a long list of parameters the expected approach here? Is there a better or more common approach?
Related
I'm trying to create a cloudFormation stack with MSK Configuration and associating MSK Configuration with MSK Cluster. Creation of AWS::MSK::Configuration returns only ARN while I need ARN and Revision number to associate MSK Configuration to MSK Cluster. Is there any way to achieve this? Currently I'm hard-coding it to 1 which means it will work only for creating stack.
...
MSKConfiguration:
Type: AWS::MSK::Configuration
Properties:
Name: aws-msk-configuration
ServerProperties: |
auto.create.topics.enable = true
zookeeper.connection.timeout.ms = 1000
log.roll.ms = 604800000
MSKCluster:
Type: AWS::MSK::Cluster
Properties:
ClusterName: !Ref ClusterName
ClientAuthentication: !If
- UsingIamAuthentication
- Sasl:
Iam:
Enabled: true
- Sasl:
Scram:
Enabled: true
ConfigurationInfo:
Arn: !GetAtt MSKConfiguration.Arn
Revision: 1
...
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-msk-configuration.html
You can only get the latest version if you define a custom resource. Since you program the full logic of the resource, you can do what you want, including automatically setting up latest version for MKS.
I have a AWS SAM template that creates a API Gateway hooked into a Step Function.
This is all working fine, but I need to add a Integration Response Mapping Template to the response back from Step Functions.
I cant see that this is possible with SAM templates?
I found the relevant Cloud Formation template for it: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-apitgateway-method-integration-integrationresponse.html
But It looks like I would have to create the whole AWS::ApiGateway::Method / Integration / IntegrationResponses chain - and then I'm not sure how you reference that from the other parts of the SAM template.
I read that it can be done with openAPI / Swagger definition - is that the only way? Or is there a cleaner way to simply add this template?
This is watered down version of what I have just to demonstrate ...
Transform: AWS::Serverless-2016-10-31
Description: My SAM Template
Resources:
MyAPIGateway:
Type: AWS::Serverless::Api
Properties:
Name: my-api
StageName: beta
Auth:
ApiKeyRequired: true
UsagePlan:
CreateUsagePlan: PER_API
UsagePlanName: my-usage-plan
Quota:
Limit: 1000
Period: DAY
Throttle:
BurstLimit: 1000
RateLimit: 1000
MyStateMachine:
Type: AWS::Serverless::StateMachine
Properties:
Name: my-state-machine
DefinitionUri: statemachines/my-state-machine.asl.json
Events:
MyEvent:
Type: Api
Properties:
Path: /myApiMethod
Method: post
RestApiId: !Ref MyAPIGateway
# TODO: how to we define this Integration Response Template ?
# IntegrationResponse:
# Template:
# application/json: |
# ## parse arn:aws:states:REGION:ACCOUNT:execution:STATE_MACHINE:EXECUTION_NAME
# ## to get just the name at the end
# #set($executionArn = $input.json('$.executionArn'))
# #set($arnTokens = $executionArn.split(':'))
# #set($lastIndex = $arnTokens.size() - 1)
# #set($executionId = $arnTokens[$lastIndex].replace('"',''))
# {
# "execution_id" : "$executionId",
# "request_id" : "$context.requestId",
# "request_start_time" : "$context.requestTimeEpoch"
# }
Right now you're using AWS SAM events in your state machine to construct the API for you, which is a very easy way to easily construct the API. However, certain aspects of the API cannot be constructed this way.
You can still use AWS SAM however to construct the API with all the advanced features when you use the DefinitionBody attribute of the AWS::Serverless::Api (or the DefinitionUri). This allows you to specify the API using the OpenAPI specification with the OpenAPI extensions.
You still need to define the event in the StateMachine though, since this will also ensure that the correct permissions are configured for your API to call your other services. If you don't specify the event, you'll have to fix the permissions yourself.
We are trying create a AWS::WAFv2::IPSet in our SAM template.
WhitelistedIPAddressesIPSet:
Type: AWS::WAFv2::IPSet
Properties:
Description: 'Merchant IPs'
Scope: REGIONAL
IPAddressVersion: IPV4
Addresses: [0.0.0.0/32, 0.0.10.0/32]
The creation of the IP sets is done successfully.
Once creating the AWS::WAFv2::WebACLAssociation.
WAFApiAssociation:
Type: AWS::WAFv2::WebACLAssociation
DependsOn:
- ApiGateway
- WAFWebAcl
Properties:
ResourceArn: !Sub 'arn:aws:apigateway:${AWS::Region}::/restapis/${ApiGateway}/stages/${EnvType}'
WebACLArn: !GetAtt WAFWebAcl.Arn
The CloudFormation failes and does a rollback. Error displayed is as follows:
Resource handler returned
ion message: "AWS WAF couldn?t
perform the operation
because your resource
doesn?t exist. (Service:
Wafv2, Status Code: 400,
Request ID: e337720a-e32c-
4c29-acde-1896855405c9,
Extended Request ID:
null)" (RequestToken: f24d
0488-3016-4030-3a3b-bbb246
66f130, HandlerErrorCode:
NotFound)
We tried different formatting the SAM template of the IP set, to see if that causes the issues, without any success.
Anyone that could share some helpful insights to this issue?
A) You don't need DependsOn if your resource already directly depends on those other resources. In this case it does, so you can remove this property.
B) You'll need to share your whole stack here, not just what is shared because there is likely a problem with your APIGW configuration. Because that failed to be created, it's possible you get this subsequent problem showing up.
Creating the APIGW isn't enough, you need to make sure to actually attach the WAF after the APIGW stage was created and not just the APIGW. In this case replace the ResourceArn with one that references the APIGW Stage. (And further you might need to wait for the stage deployment to finish.)
This is the APIGW template Warren Parad
CDEAPI:
Type: AWS::Serverless::Api
Properties:
# Domain:
# DomainName: !Ref CDEAPIDomainName
# SecurityPolicy: TLS_1_2
# CertificateArn: !Sub 'arn:aws:acm:us-east-1:${AWS::AccountId}:certificate/${CDEAPICertificateArn}'
# EndpointConfiguration: EDGE
# Route53:
# HostedZoneId: !Ref CDEAPIHostedZoneId
AccessLogSetting:
DestinationArn: !GetAtt CDEAPIAccessLogGroup.Arn
Format: >-
{ "requestId":"$context.requestId",
"ip":"$context.identity.sourceIp",
"caller":"$context.identity.caller",
"user":"$context.identity.user",
"userAgent":"$context.identity.userAgent",
"userArn":"$context.identity.userArn",
"requestTime":"$context.requestTime",
"requestTimeEpoch":"$context.requestTimeEpoch",
"httpMethod":"$context.httpMethod",
"resourcePath":"$context.resourcePath",
"path":"$context.path",
"status":"$context.status",
"protocol":"$context.protocol",
"responseLength":"$context.responseLength",
"responseLatency":"$context.responseLatency",
"authorizerLatency":"$context.authorizer.integrationLatency",
"integrationLatency":"$context.integrationLatency",
"integrationStatus":"$context.integrationStatus",
"xrayTraceId":"$context.xrayTraceId",
"errorMessage":"$context.error.message",
"domainName":"$context.domainName",
"domainPrefix":"$context.domainPrefix",
"tokenScopes":"$context.authorizer.claims.scope",
"tokenIat":"$context.authorizer.claims.iat",
"tokenExp":"$context.authorizer.claims.exp",
"cognitoIdentityId":"$context.identity.cognitoIdentityId",
"awsEndpointRequestId":"$context.awsEndpointRequestId",
"arn":"$context.identity.userArn",
"account":"$context.identity.accountId",
"claims-sub":"$context.authorizer.claims.sub",
"waf-error":"$context.waf.error",
"waf-status":"$context.waf.status",
"waf-latency":"$context.waf.latency",
"waf-response":"$context.waf.wafResponseCode",
"authenticate-error":"$context.authenticate.error",
"authenticate-status":"$context.authenticate.status",
"authenticate-latency":"$context.authenticate.latency",
"integration-error":"$context.integration.error",
"integration-status":"$context.integration.status",
"integration-latency":"$context.integration.latency",
"integration-requestId":"$context.integration.requestId",
"integration-integrationStatus":"$context.integration.integrationStatus",
"response-latency":"$context.responseLatency" }
StageName: !Ref EnvType
Auth:
DefaultAuthorizer: CognitoAuthorizer
AddDefaultAuthorizerToCorsPreflight: false
Authorizers:
CognitoAuthorizer:
AuthType: COGNITO_USER_POOLS
UserPoolArn: !Sub 'arn:aws:cognito-idp:${AWS::Region}:${AWS::AccountId}:userpool/${CognitoUserPoolArn}'
I want to read the URL of my database from parameter store in my CloudFormation template. This is easy enough for a single URL, but I can't figure out how to change the URL with different environments.
I have four environments (development, integration, pre-production and production) and their details are stored on four different paths in Parameter Store:
/database/dev/url
/database/int/url
/database/ppe/url
/database/prod/url
I now want to pick the correct Database URL when deploying via CloudFormation. How can I do this?
Parameters:
Environment:
Type: String
Default: dev
AllowedValues:
- dev
- int
- ppe
- prod
DatabaseUrl:
Type: 'AWS::SSM::Parameter::Value<String>'
# Obviously the '+' operator here won't work - so what do I do?
Default: '/database/' + Environment + '/url'
This feature isn't as neat as one would wish. You have to actually pass name/path of each parameter that you want to look up from the parameter store.
Template:
AWSTemplateFormatVersion: 2010-09-09
Description: Example
Parameters:
BucketNameSuffix:
Type: AWS::SSM::Parameter::Value<String>
Default: /example/dev/BucketNameSuffix
Resources:
Bucket:
Type: AWS::S3::Bucket
Properties:
BucketName: !Sub parameter-store-example-${BucketNameSuffix}
If you don't pass any parameters to the template, BucketNameSuffix will be populated with the value stored at /example/dev/BucketNameSuffix. If you want to use, say, a prod value (pointed to by /example/prod/BucketNameSuffix), then you should specify value for parameter BucketNameSuffix, but instead of passing the actual value you should pass the alternative name of the parameter to use, so you'd pass /example/prod/BucketNameSuffix.
aws cloudformation update-stack --stack-name example-dev \
--template-body file://./example-stack.yml
aws cloudformation update-stack --stack-name example-prod \
--template-body file://./example-stack.yml \
--parameters ParameterKey=BucketNameSuffix,ParameterValue=/example/prod/BucketNameSuffix
A not so great AWS blog post on this: https://aws.amazon.com/blogs/mt/integrating-aws-cloudformation-with-aws-systems-manager-parameter-store/
Because passing million meaningless parameters seems stupid, I might actually generate an environment-specific template and set the right Default: in the generated template so for prod environment Default would be /example/prod/BucketNameSuffix and then I can update the prod stack without passing any parameters.
You can populate CloudFormation templates with parameters stored in AWS Systems Manager Parameter Store using Dynamic References
In this contrived example we make two lookups using resolve:ssm and replace the environment using !Join and !Sub
AWSTemplateFormatVersion: 2010-09-09
Parameters:
Environment:
Type: String
Default: prod
AllowedValues:
- prod
- staging
Resources:
TaskDefinition:
Type: AWS::ECS::TaskDefinition
Properties:
ContainerDefinitions:
Image: !Join
- ''
- - 'docker.io/bitnami/redis#'
- !Sub '{{resolve:ssm:/app/${Environment}/digest-sha/redis}}'
Environment:
- Name: DB_HOST
Value: !Sub '{{resolve:ssm:/app/${Environment}/db-host}}'
You can make use of Fn::Join here.
Here is some pseudo code.
You will have to take Environment as an parameter, which you are already doing.
Creating the required string in the resources where DatabaseUrl is required.
Resources :
Instance :
Type : 'AWS::Some::Resource'
Properties :
DatabaseURL : !Join [ "", [ "/database/", !Ref "Environment" , "/url ] ]
Hope this helps.
Note: You cannot assign value to a Parameter dynamically using some computation logic. All the values for defined parameters should be given as Input.
I like Fn::Sub, which is much cleaner and easy to read.
!Sub "/database/${Environment}/url"
I am stuck into the same problem, below are my findings:
We can compose values and descriptions while writing into the SSM Parameter Store from CloudFormation like below :
LambdaARN:
Type: AWS::SSM::Parameter
Properties:
Type: String
Description: !Sub "Lambda ARN from ${AWS::StackName}"
Name: !Sub "/secure/${InstallationId}/${AWS::StackName}/lambda-function-arn"
Value: !GetAtt LambdaFunction.Arn
We can not compose values/defaults to looking for in SSM Parameter Store. Like below:
Parameters:
...
LambdaARN:
Type: Type: AWS::SSM::Parameter::Value<String>
Value: !Sub "/secure/${InstallationId}/teststack/lambda-function-arn"
This is not allowed as per AWS Documentation[1]. Both(Sub/Join) Functions won't work. Following is the error which I was getting:
An error occurred (ValidationError) when calling the CreateChangeSet
operation: Template format error: Every Default member must be a
string.
Composing and passing values while creating stacks can be done like this:
Parameters:
...
LambdaARN:
Type: Type: AWS::SSM::Parameter::Value<String>
....
$ aws cloudformation deploy --template-file cfn.yml --stack-name mystack --parameter-overrides 'LambdaARN=/secure/devtest/teststack/lambda_function-arn'
If you add your custom tags while putting the values in the Parameter Store, it will overwrite the default tags added by CFN.
Default Tags:
- aws:cloudformation:stack-id
- aws:cloudformation:stack-name
- aws:cloudformation:logical-id
Every time we update the values in parameter store, it creates a new version, which is beneficial when we are using DynamicResolvers, this can serve as a solution to the problem in the question like
{{ resolve:ssm:/my/value:1}}
The last field is the version. Where different versions can point to different environments.
We are using versions with the parameters, and adding the labels to them[2], this can't be done via CFN[3], only possible way via CLI or AWS Console. This is AWS's way of handling multiple environments.
[1] https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference.html
[2] https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-paramstore-labels.html
[3] https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ssm-parameter.html
I have a cloudformation template to build my api using the API Gateway.
I don't know how to:
Enable cloudwatch logs for the stage in the cloudformation template
Assign the stage to a Custom Domain Name in the cloudformation template.
Is either of these possible in a json cloudformation template?
Cloudwatch logs:
Yes you can enable cloudwatch logs in cloudformation:
Configure CloudTrail log file delivery to CloudWatch Logs.
Create a AWS CloudFormation stack by using the template.
the cloudwatch entry should be something simalar to this:
"SecurityGroupChangesAlarm": {
"Type": "AWS::CloudWatch::Alarm",
"Properties": {
"AlarmName" : "CloudTrailSecurityGroupChanges",
"AlarmDescription" : "Alarms when an API call is made to create, update or delete a Security Group.",
"AlarmActions" : [{ "Ref" : "AlarmNotificationTopic" }],
"MetricName" : "SecurityGroupEventCount",
"Namespace" : "CloudTrailMetrics",
"ComparisonOperator" : "GreaterThanOrEqualToThreshold",
"EvaluationPeriods" : "1",
"Period" : "300",
"Statistic" : "Sum",
"Threshold" : "1"
}
},
Check the aws official doc everything is detailed there.
Custom Domain Name:
the custom domain name is not defined in the cloudformation template. It should be created separately as specified in aws doc:
Sign in to the API Gateway console at https://console.aws.amazon.com/apigateway.
Choose Custom Domain Names from the main navigation pane.
Choose Create in the secondary navigation pane.
In Create Custom Domain Name
setup DNS using Amazon Route 53
Update Jul 5 2017: The AWS::ApiGateway::DomainName resource is now available, so a Custom Resource is no longer needed for this part.
Original post Dec 24 2016:
Enable cloudwatch logs for the stage in the cloudformation template
To enable CloudWatch logs for an ApiGateway Stage using CloudFormation for every method call to your API, you need to set the DataTraceEnabled property to true for all methods in your AWS::ApiGateway::Stage resource.
As noted in the Set Up a Stage section of the documentation, you will also need to associate your API Gateway account with the proper IAM permissions to push data to CloudWatch Logs. For this purpose, you will also need to create an AWS::ApiGateway::Account resource that references an IAM role containing the AmazonAPIGatewayPushToCloudWatchLogs managed policy, as described in the documentation example:
CloudWatchRole:
Type: "AWS::IAM::Role"
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service:
- "apigateway.amazonaws.com"
Action: "sts:AssumeRole"
Path: "/"
ManagedPolicyArns:
- "arn:aws:iam::aws:policy/service-role/AmazonAPIGatewayPushToCloudWatchLogs"
Account:
Type: "AWS::ApiGateway::Account"
Properties:
CloudWatchRoleArn:
"Fn::GetAtt":
- CloudWatchRole
- Arn
Assign the stage to a Custom Domain Name in the cloudformation template
Unfortunately, CloudFormation does not provide an official resource corresponding to the DomainName APIGateway REST API. Fortunately, Carl Nordenfelt's unofficial API Gateway for CloudFormation project does provide Custom::ApiDomainName. Here's the example provided in the documentation:
TestApiDomainName:
Type: Custom::ApiDomainName
Properties:
ServiceToken: {Lambda_Function_ARN}
domainName: example.com
certificateName: testCertificate
certificateBody": "-----BEGIN CERTIFICATE-----line1 line2 ... -----END CERTIFICATE-----"
certificateChain: "-----BEGIN CERTIFICATE-----line1 line2 ... -----END CERTIFICATE-----"
certificatePrivateKey: "-----BEGIN RSA PRIVATE KEY-----line1 line2 ... -----END RSA PRIVATE KEY-----"
Also note that once the domain name has been created, you should create a Route53 alias record that points to !GetAtt TestApiDomainName.distributionDomainName and the static CloudFront hosted zone ID (Z2FDTNDATAQYW2), for example:
myDNSRecord:
Type: AWS::Route53::RecordSet
Properties:
HostedZoneName:
!Ref HostedZone
Name:
!Ref DomainName
Type: A
AliasTarget:
DNSName: !GetAtt TestApiDomainName.distributionDomainName
HostedZoneId: Z2FDTNDATAQYW2