I am creating a an AppSync cloudformation and a dashboard and I notice that the ApiId export is finished creation much after.
I create my output like so in a appsync file :
Outputs:
GraphQlApiIdOutput:
Description: Main GraphQl Api ID.
Value:
Fn::GetAtt:
- GraphQlApi
- ApiId
Export:
Name: GraphQlApiIdOutput
In another dashboard file:
CloudwatchDashboard:
Type: AWS::CloudWatch::Dashboard
Properties:
DashboardBody:
{
'Fn::Sub': ['{
...
,{
ApiId:
!ImportValue GraphQlApiIdOutput}]
However, I get the error: No export named GraphQlApiIdOutput found. Rollback requested by user.
The only way I can get around this is by removing the import value first so that the stack creation succeeds and then add in the import value. I notice that there is a waitcondition but none of the resources are external. I also notice that creation policy is only supported for AWS::AutoScaling::AutoScalingGroup, AWS::EC2::Instance, and AWS::CloudFormation::WaitCondition.
Is there a way to wait for resource creation in Cloudformation so that the template does not try to use an output value before it is created in the same stack?
When you have this in your code:
!ImportValue GraphQlApiIdOutput
it means that stack from Template 1 must be created beforehand.
So normally what you do is the following:
Deploy Template 1
When this succeeds, then you deploy your another template file.
The DependsOn is for the resources within same stack, not across different stacks. The same goes for WaitCondition as you wont be able to reference conditions in another template file from Template 1 before the Template 1 stack has been successful created.
Yes, use the DependsOn Attribute:
With the DependsOn attribute you can specify that the creation of a specific resource follows another. When you add a DependsOn attribute to a resource, that resource is created only after the creation of the resource specified in the DependsOn attribute.
The attribute points to another resource in the same stack. If Resource A DependsOn Resource B, then Resource A will only be created once Resource B has finished creation.
Related
while configuring resource configuration , is there any way I can use so that serverless wont create throw any error if resource is already present.
eg. don't throw this error if following resource is already present.
Error : An error occurred: PaymentQueue - dev_payment_cron_queue already exists in stack
resources:
Resources:
PaymentQueue:
Type: "AWS::SQS::Queue"
Properties:
QueueName: ${self:provider.stage}_payment_cron_queue
VisibilityTimeout: 40
Sadly, there is no such way with pure CloudFormation (CFN), as this is not how CFN (or Terraform as a matter of fact) was designed to work. From CFN perspective, a given resource exists and is managed by CFN, or it does not exist at all. There is no middle ground.
If your resource already exist, you have to import it to CFN so that it gets managed by CFN. Alternatively, you have to create custom resource in the form of a lambda function. The function would perform any action you want based on the existing resources, including checking if it exists or not.
Is there a way to create a resource if it doesn't exist and use an existing resource if it does?
resources:
- name: "my-topic"
type: gcp-types/pubsub-v1:projects.topics
properties:
topic: "this-exists-already"
- name: "my-other-resource"
type: 'gcp-types/cloudfunctions-v1:projects.locations.functions'
properties:
functionName: "function"
environmentVariables:
# get a ref to new or already existing topic
my-toptic: "$(ref.my-topic.name)"
Per #kolban's link I think I want to use abandon here. Can I selectively "abandon" a specific resource so I can, for example, attach an accessControl policy to an existing bucket but then NOT delete that bucket if the deployment is deleted?
ABANDON - This removes any references to the resource from the
deployment but does not delete the underlying resource. For example,
abandoning an instance means that it is removed from a deployment but
the instance still exists for you to use.
Edit
Maybe I should use an "action" to assign an acl instead of a resource? Is this the right way and are there examples of this? So DM would essentially just execute an api call to apply an acl out-of-band. That would mean it would leave the acl behind if the deployment is deleted but I'm okay with that.
It looks like I want to do something like this but instead of applying an acl to a specific file I want to set it on the bucket (with an action) https://github.com/GoogleCloudPlatform/deploymentmanager-samples/blob/master/community/storage-bucket-acl/storagebucket-acl.jinja#L29.
If we read this section of the Deployment Manager documentation:
https://cloud.google.com/deployment-manager/docs/deployments/updating-deployments#policies_for_adding_resources
We read about the concept of "create or acquire". The way I read this is that if a resource your configuration says should be created then the default appears to be that if it already exists, it will not cause an error and will be "acquired" for this deployment which I take to mean that it will be as though it had been created.
I have a lambda which has a log group, say LG-1, for which retention is set to Never Expire (default). I need to change this Never Expire to 1 month. I am doing this using CloudFormation. As the log group already exists, when I am trying to deploy my lambda again with the changes in template as :
LambdaFunctionLogGroup:
Type: 'AWS::Logs::LogGroup'
DependsOn: MyLambda
Properties:
RetentionInDays: 30
LogGroupName: !Join
- ''
- - /aws/lambda/
- !Ref MyLambda
the update is failing with error :
[LogGroup Name] already exists.
One possible solution is to delete the log group and then again create it with new changes as shown above which works perfectly well.
But I need to do it without deleting the log group as it will result in the deletion of all the previous logs that I have.
Is there any workaround which is possible ?
#ttulka answered:
".. it is impossible to manipulate resources from CF which already exist out of the stack."
But actually the problem is more general than that and applies to resources created inside of the stack. It has to do with AWS CloudFormation resource "Replacement policy". For some resources the way CloudFormation "updates" the resource is to create a new resource, then delete the old resource (this is called the "Replacement" update policy). This means there is a period of time where you've got two resources of the same type with many of the same properties existing at the same time. But if a certain resource property has to be unique, the two resource can't exist at the same time if they have the same value for this property, so ... CloudFormation blows up.
AWS::Logs::LogGroup.LogGroupName property is one such property. AWS::CloudWatch::Alarm.AlarmName is another example.
A work around is to unset the name so that a random name is used, perform an update, then set the name back to it's predictable fixed value and update again.
Rant: It's an annoying problem that really shouldn't exist. I.e. AWS CF should be smart enough to not have to use this weird clunky resource replacement implementation. But ... that's AWS CF for you ...
I think it is impossible to manipulate resources from CF which already exist out of the stack.
One workaround would be to change the name of the Lambda like my-lambda-v2 to keep the old log group together with the new one.
After one month you can delete the old one.
Use customresource Backed lambda within your cloudformation template. The custom resource would be triggered automatically the first time and update your retention policy of the existing log group. If you need it you custom resource lambda to be triggered every time, then use a templating engine like jinja2.
import boto3
client = boto3.client('logs')
response = client.put_retention_policy(
logGroupName='string',
retentionInDays=123
)
You can basically make your CF template do (almost) anything you want using Custom Resource
More information (Boto3, you can find corresponding SDK for the language you use) - https://boto3.amazonaws.com/v1/documentation/api/1.9.42/reference/services/logs.html#CloudWatchLogs.Client.put_retention_policy
EDIT: Within the CloudFormation Template, it would look something like the following:
LogRetentionSetFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: src
Handler: set_retention_period.handler
Role: !GetAtt LambdaRole.Arn
DeploymentPreference:
Type: AllAtOnce
PermissionForLogRetentionSetup:
Type: AWS::Lambda::Permission
Properties:
Action: lambda:invokeFunction
FunctionName:
Fn::GetAtt: [ LogRetentionSetFunction, Arn ]
Principal: lambda.amazonaws.com
InvokeLambdaFunctionToSetLogRetention:
DependsOn: [PermissionForLogRetentionSetup]
Type: Custom::SetLogRetention
Properties:
ServiceToken: !GetAtt LogRetentionSetFunction.Arn
StackName: !Ref AWS::StackName
AnyVariable: "Choose whatever you want to send"
Tags:
'owner': !Ref owner
'task': !Ref task
The lambda function would have the code which sets up the log retention as per the code which I already specified before.
For more information, please google "custom resource backed lambda". Also to get you a head start I have added the ink below:
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-custom-resources.html
I have one CFT that creates an EBS volume. Then I have a second CFT that attaches the volume to the instance. I use a Custom Resource which runs a lambda to find the EbsVolumeId and the InstanceId
The 2 CFT's are part of a jenkins pipeline and its possible that the playbook that uses the CFT to create the EBS volume can be skipped and thus there is no EbsVolumeId to reference.
So in the CFT that attaches the EBS volume, I tried adding a Condition
Resources:
MountPoint:
Type: AWS::EC2::VolumeAttachment
Condition: AttachEBS
Properties:
InstanceId: !GetAtt SCSHelper.InstanceId
VolumeId: !GetAtt SCSHelper.EbsVolumeId
The custom resource looks like
SCSHelper:
Type: Custom::SCSHelper
So I have to define the condition before the resources:
Conditions:
AttachEBS: !Not [!Equals http:// !GetAtt SCSHelper.EbsVolumeId , None ]
The problem is the Conditions is failing with:
An error occurred (ValidationError) when calling the CreateStack operation: Template format error: Unresolved dependencies https://forums.aws.amazon.com/. Cannot reference resources in the Conditions block of the template An error occurred (ValidationError) when calling the CreateStack operation: Template format error: Unresolved dependencies https://forums.aws.amazon.com/.
So it appears that the SCSHelper.EbsVolumeId attribute is not available for the condition to use.
Is there a way to make this work or is there a better way to conditionally run the CFT that attaches the EBS volume?
Thanks in advance...
The important section of the Conditions documentation is this
Parameters
Define the input values that you want to evaluate in your conditions. Conditions will result in true or false based on values from these input parameters.
Conditions in CloudFormation can be based on Parameters alone. They can't take in-flight resource values into account, as these values aren't known at compile time.
Is there a way to access auto-generated URLs for deployed resources before the deployment is finished? (like db host, lambda function URL, etc.)
I can access them after the deployment is finished, but sometimes I need to access them while building my stack. (E.g. use them in other resources).
What is a good solution to handle this use-case? I was thinking about outputting them into the SSM parameter store from CloudFormation template, but I'm not sure if this is even possible.
Thanks for any suggestion or guidance!
If "use them in other resources" means another serverless service or another CloudFormation stack, then use CloudFormation Outputs to export the values you are interested in. Then use CloudFormation ImportValue function to reference that value in another stack.
See https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/outputs-section-structure.html and https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-importvalue.html
Within Serverless Framework, you can access a CloudFormation Output value using https://serverless.com/framework/docs/providers/aws/guide/variables/#reference-cloudformation-outputs
If you want to use the autogenerated value within the same stack, then just use CloudFormation GetAtt function. See https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-getatt.html.
For example, I have a CloudFormation stack that outputs the URL for an ElasticSearch cluster.
Resources:
Search:
Type: AWS::Elasticsearch::Domain
Properties: <redacted>
Outputs:
SearchUrl:
Value: !GetAtt Search.DomainEndpoint
Export:
Name: myapp:search-url
Assuming that the CloudFormation stack name is "mystack", then in my Serverless service, I can reference the SearchUrl by:
custom:
searchUrl: ${cf:mystack.SearchUrl}
To add to bwinant's answer, ${cf:<stack name>.<output name>} does not work if you want to reference a variable in another stack which is located in another region. There is a plugin to achieve this called serverless-plugin-cloudformation-cross-region-variables. You can use it like so
plugins:
- serverless-plugin-cloudformation-cross-region-variables
custom:
myVariable: ${cfcr:ca-central-1:my-other-stack:MyVariable}