I'm creating a AWS::Timestream::Database service and I want to use one of our KMS keys that's been created externally of the CloudFormation template.
I want to use the alias of the key, but I can't find any good way to get the reference of the key into the template.
I've found some references to use !Sub to be able to get the ARN, but how to get ID from there?
I have used the following code in my S3 cross account replication CF template for collecting the KMS alias. You can try the same approach.
ReplicaKmsKeyID: !Join ['',['arn:aws:kms:', !Ref BucketRegion, ':', !Ref AWS::AccountId, ':alias/aws/s3']]
#ReplicaKmsKeyID: arn:aws:kms:us-west-2:111111111111:alias/aws/s3
You can use alias with your kms id like arn:<partition>:kms:<region>:<account-id>:alias/<alias-name>
AWSTemplateFormatVersion: 2010-09-09
Resources:
DB:
Type: AWS::Timestream::Database
Properties:
DatabaseName: <DB_NAME>
KmsKeyId: arn:aws:kms:<REGION>:<ACCOUNT_ID>:alias/<ALIAS>
Related
I have a CFN template where in I am creating 2 s3 buckets for the image resizing using CloudFront.
the issue is that I want to use an already existing bucket from s3 for these functions.
but I get an error that s3 already exists when I provide the resource ARN and other data.
how can I resolve this?
I tried giving the details ARN name etc and tried deploying but it doesn't work
Something like this would help you:
AWSTemplateFormatVersion: '2010-09-09'
Description: 'CFN template example for referencing existing S3 bucket to lambda'
Parameters:
myS3Bucket:
Type: String
Description: Provide the S3 bucket you want to referece into your lambda.
Resources:
MyLambdaFunction:
Type: AWS::Lambda::Function
Properties:
Description: A lambda function
Handler: index.handler
Runtime: python3.7
Environment:
Variables:
S3_BUCKET: !Ref myS3Bucket
I'm trying to define some common resources (specifically, a couple of IAM roles) that will be shared between two environments, via a nested stack. The first environment to use the nested stack's resources creates ok, but the second one fails when trying to run the nested stack. Am I not understanding something about how nested stacks work, or am I just doing something wrong?
My nested stack is defined as:
AWSTemplateFormatVersion: '2010-09-09'
Description: Defines common resources shared between environments.
Parameters:
ParentStage:
Type: String
Description: The Stage or environment name.
Default: ""
ParentVpcId:
Type: "AWS::EC2::VPC::Id"
Description: VpcId of your existing Virtual Private Cloud (VPC)
Default: ""
Resources:
LambdaFunctionSG:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Identifies Lamdba functions to VPC resources
GroupName: BBA-KTDO-SG-LambdaFunction
VpcId: !Ref ParentVpcId
RdsAccessSG:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Allows Lambda functions access to RDS instances
GroupName: BBA-KTDO-SG-RDS
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 3306
ToPort: 3306
SourceSecurityGroupId: !Ref LambdaFunctionSG
VpcId: !Ref ParentVpcId
I've uploaded that YAML file to an S3 bucket, and I'm then trying to use it in two separate stack files (i.e. app_dev.yaml and app_prod.yaml) as:
Resources:
CommonResources:
Type: 'AWS::CloudFormation::Stack'
Properties:
TemplateURL: "https://my-buildbucket.s3-eu-west-1.amazonaws.com/common/common.yaml"
Parameters:
ParentStage: !Ref Stage
ParentVpcId: !Ref VpcId
And referring to its outputs as (e.g):
VpcConfig:
SecurityGroupIds:
- !GetAtt [ CommonResources, Outputs.LambdaFunctionSGId ]
The first environment creates fine, including the nested resources. When I try to run the second environment, it fails with error:
Embedded stack
arn:aws:cloudformation:eu-west-1:238165151424:stack/my-prod-stack-CommonResources-L94ZCIP0UD9W/f9d06dd0-994d-11eb-9802-02554f144c21
was not successfully created: The following resource(s) failed to
create: [LambdaExecuteRole, LambdaFunctionSG].
Is it not possible to share a single resource definition between two separate stacks like this, or have I just missed something in the implementation?
As #jasonwadsworth mentioned that's correct names of the stacks are always amended with a random string at the end AWS::CloudFormation::Stack check the return values. Use GetAtt to get the name of the stack and construct the output. How do I pass values between nested stacks within the same parent stack in AWS CloudFormation?
Plus use aws cloudformation package command for packaging the nested stacks, no need to manually upload them to s3 bucket.
someting like
aws cloudformation package \
--template-file /path_to_template/template.json \
--s3-bucket bucket-name \
--output-template-file packaged-template.json
Take a look on the cloudformation output exports as well in case you are curious Difference between an Output & an Export
I am trying to deploy a serverless project which has s3 bucket creation cloudformation in the serverless.yml file, but the problem is when I tried to deploy, it says the s3 bucket already exists and failing the deployment.
I know s3 bucket name should be globally unique, and I am damn sure it is a unique name that I am using, even if changed to something else, it still says the same.
the cloudformation stack it says the s3 bucket exists is actually the newly created stack, not sure how to fix this issue. can anyone help me out with this issue and tell me how to fix the deployment issue and the cause for the issue :).
Thanks in advance.
The issue I had was, for one of the lambdas I had the above-mentioned bucket as the event source, so when some bucket is added as event source it actually creating that bucket as well, therefore when it runs the actual creation related cloudformation it is saying the bucket already exists.
So I fixed it by only keeping the event source and removed the actual declaration of that bucket.
If you add existing: true to the S3 config in your serverless.yml file it won't try to create the S3 bucket like the below:-
funcName:
handler: handler
events:
- s3:
bucket: 'my-bucket-name'
events: s3:ObjectCreated:*
existing: true
rules:
- suffix: .pdf
- prefix: documents
Anything involving CloudFormation (or any other infrastructure-in-code) is fussy, and the error messages can mislead, meaning there are a ton of things that can cause this problem (see issues on GitHub like this one).
But in my experience, the most common causes of these kind of problems are are not the pre-existing bucket, but problems with AWS credentials, permissions, or region that give misleading error messages. To fix these, or at least rule them out:
Make sure your serveless.yml is set to the region you already deployed the stack in. Example:
custom:
stage: dev
region: us-east-2
Override any latent credentials from, for example, ~/.aws/credentials, by explicitly setting your credentials in the shell you'll use to deploy. Example from the Serverless docs:
export AWS_ACCESS_KEY_ID=<your access key here>
export AWS_SECRET_ACCESS_KEY=<your access secret here.
Make sure those AWS credentials have the roles and permissions they need.
But, as I mentioned, CloudFormation is fussy. There may be other problems to solve, but try these first. You may try them and still be beating your head against the wall, but it'll more likely be the right wall. Hope this helps.
Try to use Conditional statements and pass them as a Parameter to create the bucket or not
AWSTemplateFormatVersion: 2010-09-09
Parameters:
EnvType:
Description: Environment type.
Default: test
Type: String
AllowedValues:
- prod
- test
ConstraintDescription: must specify prod or test.
Conditions:
CreateProdResources: !Equals
- !Ref EnvType
- prod
Resources:
EC2Instance:
Type: 'AWS::EC2::Instance'
Properties:
ImageId: ami-0ff8a91507f77f867
MountPoint:
Type: 'AWS::EC2::VolumeAttachment'
Condition: CreateProdResources
Properties:
InstanceId: !Ref EC2Instance
VolumeId: !Ref NewVolume
Device: /dev/sdh
NewVolume:
Type: 'AWS::EC2::Volume'
Condition: CreateProdResources
Properties:
Size: 100
AvailabilityZone: !GetAtt
- EC2Instance
- AvailabilityZone
Follow the sample condition flow to decide wheather to create a resource or not.
See this for more details
When deploying, the BucketName must be unique across all regions. So if anyone has already created a bucket with "local-bucket-dev," it will throw
An error occurred: AttachmentsBucket - local-bucket-dev already
exists.
Try to just the BucketName to be unique.
I hope that helps.
has somebody a hint how I have to transform !Ref und !Sub from CloudFormation into serverless.yml.
resources:
Resources:
AthenaCreateDatabaseQuery:
Type: 'AWS::Athena::NamedQuery'
Properties:
Description: Run this query to initialize the Athena database
QueryString: "CREATE DATABASE IF NOT EXISTS $(self:custom.etlDatabase};"
Database: ${self:custom.etlDataBase}
In Cloudformation the QueryString Property starts with !Sub and
the Database Property with !Ref.
Thanks Christian
!Sub isn't currently supported natively by serverless.com (see this issue on GitHub) but you can use the following plugin https://gitlab.com/kabo/serverless-cf-vars
Whenever you want the cloudformation template to have a string that contains ${}, simply use #{} instead, and it will get transformed into correct ${} (with Fn::Sub inserted for you) in the cloudformation template before deployment.
Or use a custom variable syntax as suggested here. Both require the use of Fn::Sub and Fn::Ref in yaml instead of the short form !Sub and !Ref
I am trying to create an AWS cloudformation stack using a yaml template.
The goal is to create a sns topic for some notifications.
I want to output the topic arn, to be able to subscribe multiple functions to that topic by just specifying the topic arn.
However I am getting an error when I try to create the stack from the aws console:
"Template validation error: Template error: resource NotificationsTopic does not support attribute type Arn in Fn::GetAtt"
I have done exactly the same for s3 buckets, dynamodb tables, and all working good, but for some reason, with SNS topic I cannot get the ARN.
I want to avoid hardcoding the topic arn in all functions that are subscribed. Because if one day the the ARN topic changes, I'll need to change all functions, instead I want to import the topic arn in all functions and use it. This way I will have to modify nothing if for any reason I have a new arn topic in the future.
This is the template:
Parameters:
stage:
Type: String
Default: dev
AllowedValues:
- dev
- int
- uat
- prod
Resources:
NotificationsTopic:
Type: AWS::SNS::Topic
Properties:
DisplayName: !Sub 'notifications-${stage}'
Subscription:
- SNS Subscription
TopicName: !Sub 'notifications-${stage}'
Outputs:
NotificationsTopicArn:
Description: The notifications topic Arn.
Value: !GetAtt NotificationsTopic.Arn
Export:
Name: !Sub '${AWS::StackName}-NotificationsTopicArn'
NotificationsTopicName:
Description: Notifications topic name.
Value: !Sub 'notifications-${stage}'
Export:
Name: !Sub '${AWS::StackName}-NotificationsTopicName'
Not all resources are the same. Always check the documentation for the particular resource. It has the "Return Values" section and you can easily verify that SNS topic has ARN as a Ref value, so you don't have to use GetAtt function
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-sns-topic.html
Edit: Thanks for the comment which points out that not every resource provides its ARN. A notable example is the Autoscaling group. Sure, the key thing in my answer was "check the documentation for each resource", this is an example that not every resource has every attribute.
Having said that, ARN missing for the ASG output is a really strange thing. It cannot be also constructed easily, because the ARN also contains GroupId which is a random hash. There is probably some effort to solve this at least for the use-case of ECS Capacity Providers https://github.com/aws-cloudformation/aws-cloudformation-coverage-roadmap/issues/548 and https://github.com/aws/containers-roadmap/issues/631#issuecomment-648377011 but I think that is is an significant enough issue that it should be mentioned here.
For resources that don't directly return ARN, I found a workaround which consists of building the ARN myself.
For instance, to get the ARN of my codepipeline:
!Join [ ':', [ "arn:aws:codepipeline", !Ref AWS::Region, !Ref AWS::AccountId, !Ref StackDeletePipeline ] ]