I created and deployed a S3 resource (bucket) using cloudformation.
After that i deployed a version without that resource.
Then i deployed a version with the resource.
Since the bucket exists, it gives me an error that it cannot deploy.
This has happened to me before, in past times I deleted the resource and deployed again.
I'm looking for a way to use the resource for future deployments. It is the exact same resource, this is the yaml :
Bucket:
Type: AWS::S3::Bucket
Properties:
BucketName: !Sub "myBucketName"
Is there anything I can add to the resource, a policy, a unique ID, anything so that i could use the existing resource?
Thanks!
To "use the existing resource" in a CFN, you have to import it. Also its a bad practice to keep modify resources created by CFN outside of CFN. This leads to drift and number of issues, one of which you are experiencing.
Related
I've created a bucket without a DeletionPolicy and want to add it. Updated our configuration (in our serverless.yml), and we now see the DeletionPolicy: retain in the cloud-formation template.
However, based on this blurb:
One quirk in the update template workflow is that DeletionPolicy
cannot be updated by itself but must accompany some other change that
"add, modify or delete properties" of an existing resource. A fun fact
about the AWS::S3::Bucket resource: it cannot be updated after
creation. Good news: this does not apply to the DeletionPolicy
attribute. Bad news: CFN won't pick up the changes unless another
property of the "immutable" S3 bucket is updated.
I've searched for quite a while, and I cannot determine how to query the S3 bucket and determine if the DeletionPolicy is actually set or not. I don't see where this is exposed in the AWS console, nor do I see in the AWS cli where this can be queried. It doesn't appear to be in the Bucket Policy as far as I can tell.
How do I validate the DeletionPolicy is actually set?
I think the linked article (from 1/12/2014) is outdated. I just did the following:
Deployed a stack containing
MyBucket:
Type: AWS::S3::Bucket
.
.
.
Verified it deployed successfully, then deployed
MyBucket:
Type: AWS::S3::Bucket
DeletionPolicy: Retain
.
.
.
Verified it deployed successfully, then deleted my stack, and verified MyBucket is still present in S3.
Im creating API gateway stage using cloudformation.
ApiDeployment:
Type: AWS::ApiGateway::Deployment
Properties:
RestApiId: !Ref ExampleRestApi
StageName: dev
Here is the problem, Whenever I create a new API, I just need to deploy the stage using AWS console. is there any way that I can automate the deploy process so that no further console action is required.
When you define a Deployment resource like this, CloudFormation will create the deployment only on the first run. On the second run it will observe that the resource already exists and the CloudFormation definition did not change, so it won't create another deployment. To work around that, you can add something like a UUID/timestamp placeholder to the resource ID and replace it everytime before doing the CloudFormation update:
ApiDeployment#TIMESTAMP#:
Type: AWS::ApiGateway::Deployment
Properties:
RestApiId: !Ref ExampleRestApi
StageName: dev
This way you are still able to see your deployment history in the API Gateway console.
If you don't want to manipulate your template like this, you can also add a Lambda-backed Custom Resource to your CloudFormation stack. Using an AWS SDK, you can have the Lambda function automatically creating new deployments for you when the API was updated.
I've found berenbums response to be mostly correct, but there are a few things I don't like.
The proposed method of creating a resource like ApiDeployment#TIMESTAMP# doesn't keep the deployment history. This makes sense, since the old ApiDeployment#TIMESTAMP# element is being deleted and a new one is being created every time.
Using ApiDeployment#TIMESTAMP# creates a new deployment every time the template is deployed, which might be undesirable if the template is being deployed to create/update other resources.
Also, using ApiDeployment#TIMESTAMP# didn't work well when adding the StageDescription property. A potential solution is to add a static APIGwDeployment resource for the initial deployment (with StageDescription) and ApiDeployment#TIMESTAMP# for the updates.
The fundamental issue though, is that creating a new api gw deployment is not well suited for cloudformation (beyond the initial deployment). I think after the initial deployment, it's better to do an AWS API invocation to update the deployment (see https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-deployments.html).
In my particular case I created a small Ansible module to invoke aws apigateway create-deployment which updates an existing stage in one operation.
I'm attempting to achieve the following through CloudFormation.
From a stack created in EU region I want to create (and verify) a public certificate against Route53 in US-EAST-1 due to using Cloudfront. Aiming to have zero actions performed in the console or AWS CLI.
The new CloudFormation support for ACM was a little sketchy last week but seems to be working now.
Certifcate
Resources:
Certificate:
Type: AWS::CertificateManager::Certificate
Properties:
DomainName: !Sub "${Env}.domain.cloud"
ValidationMethod: DNS
DomainValidationOptions:
-
DomainName: !Sub "${Env}.domain.cloud"
HostedZoneId: !Ref HostedZoneId
All I need to do is use Cloudformation to deploy this into the US-EAST-1 region from stack in a different region. Everything else is ready for this.
I thought that using Codepipeline's cross region support would be great so I started to look into [this documentation][1] after getting setting things up in my template I met the following error message...
An error occurred while validating the artifact bucket {...} The bucket named is not located in the `us-east-1` AWS region.
To me this makes no sense as it seems that you already need at least a couple of resources to exist in target region for it to work. Cart before the horse kind of behavior. To test this I create an artifact bucket in the target region by hand and things worked fine, but requires using CLI or the console when I'm aiming for a CloudFormation based solution.
Note: I'm running out of time to write this so I'll update it when I can in a few hours time. any help before I can do that would be great though
Sadly, that's required for cross-region CodePipeline. From docs:
When you create or edit a pipeline, you must have an artifact bucket in the pipeline Region and then you must have one artifact bucket per Region where you plan to execute an action.
If you want to fully automate this through CloudFormation, you either have to use custom resource to create buckets in all the regions in advance or look at stack sets to deploy one template bucket in multiple regions.
p.s.
Your link does not work, thus I'm not sure if you refer to the same documentation page.
Is there a way to create a resource if it doesn't exist and use an existing resource if it does?
resources:
- name: "my-topic"
type: gcp-types/pubsub-v1:projects.topics
properties:
topic: "this-exists-already"
- name: "my-other-resource"
type: 'gcp-types/cloudfunctions-v1:projects.locations.functions'
properties:
functionName: "function"
environmentVariables:
# get a ref to new or already existing topic
my-toptic: "$(ref.my-topic.name)"
Per #kolban's link I think I want to use abandon here. Can I selectively "abandon" a specific resource so I can, for example, attach an accessControl policy to an existing bucket but then NOT delete that bucket if the deployment is deleted?
ABANDON - This removes any references to the resource from the
deployment but does not delete the underlying resource. For example,
abandoning an instance means that it is removed from a deployment but
the instance still exists for you to use.
Edit
Maybe I should use an "action" to assign an acl instead of a resource? Is this the right way and are there examples of this? So DM would essentially just execute an api call to apply an acl out-of-band. That would mean it would leave the acl behind if the deployment is deleted but I'm okay with that.
It looks like I want to do something like this but instead of applying an acl to a specific file I want to set it on the bucket (with an action) https://github.com/GoogleCloudPlatform/deploymentmanager-samples/blob/master/community/storage-bucket-acl/storagebucket-acl.jinja#L29.
If we read this section of the Deployment Manager documentation:
https://cloud.google.com/deployment-manager/docs/deployments/updating-deployments#policies_for_adding_resources
We read about the concept of "create or acquire". The way I read this is that if a resource your configuration says should be created then the default appears to be that if it already exists, it will not cause an error and will be "acquired" for this deployment which I take to mean that it will be as though it had been created.
We have already created some infrastructure manually and with terraform, including some s3 buckets. In the future I would like to use pure CloudFormation to define the infrastructure as code.
So I created a CloudFormation yaml definition which references an existing bucket:
AWSTemplateFormatVersion: '2010-09-09'
Resources:
TheBucket:
Type: AWS::S3::Bucket
Properties:
BucketName: my-existing-bucket-name
When I try to apply it, execution fails, with CloudFormation stack event:
The following resource(s) failed to update: [TheBucket].
12:33:47 UTC+0200 UPDATE_FAILED AWS::S3::Bucket TheBucket
my-existing-bucket-name already exists
How can I start managing existing resources with CloudFormation without recreating them? Or is it impossible by design?
You will need to create a new bucket and sync the data from the old bucket to the new bucket. I have not seen a way to use an modify an existing S3 bucket.
The resources section of a cloud formation template defines which resources should be created by cloud formation. Try refering to the existing resources by defining them as parameters instead.
You should be able to import it by using the "Import resources into stack" option:
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/resource-import-existing-stack.html
As the documentation explains, you should add a "DeletionPolicy": "Retain" attribute to the already existing resources in your stack.