Is there a way to access auto-generated URLs for deployed resources before the deployment is finished? (like db host, lambda function URL, etc.)
I can access them after the deployment is finished, but sometimes I need to access them while building my stack. (E.g. use them in other resources).
What is a good solution to handle this use-case? I was thinking about outputting them into the SSM parameter store from CloudFormation template, but I'm not sure if this is even possible.
Thanks for any suggestion or guidance!
If "use them in other resources" means another serverless service or another CloudFormation stack, then use CloudFormation Outputs to export the values you are interested in. Then use CloudFormation ImportValue function to reference that value in another stack.
See https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/outputs-section-structure.html and https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-importvalue.html
Within Serverless Framework, you can access a CloudFormation Output value using https://serverless.com/framework/docs/providers/aws/guide/variables/#reference-cloudformation-outputs
If you want to use the autogenerated value within the same stack, then just use CloudFormation GetAtt function. See https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-getatt.html.
For example, I have a CloudFormation stack that outputs the URL for an ElasticSearch cluster.
Resources:
Search:
Type: AWS::Elasticsearch::Domain
Properties: <redacted>
Outputs:
SearchUrl:
Value: !GetAtt Search.DomainEndpoint
Export:
Name: myapp:search-url
Assuming that the CloudFormation stack name is "mystack", then in my Serverless service, I can reference the SearchUrl by:
custom:
searchUrl: ${cf:mystack.SearchUrl}
To add to bwinant's answer, ${cf:<stack name>.<output name>} does not work if you want to reference a variable in another stack which is located in another region. There is a plugin to achieve this called serverless-plugin-cloudformation-cross-region-variables. You can use it like so
plugins:
- serverless-plugin-cloudformation-cross-region-variables
custom:
myVariable: ${cfcr:ca-central-1:my-other-stack:MyVariable}
Related
I have a UI5 web application which has no access to AWS Parameter store. However, it is deployed by CloudFormation, by an account which has this access - I already collect some values from SSM during deployment and use them as parameters for resources which are being deployed by template.yml file.
However, now I need some of these parameters not only during deployment, but also during actual run of the app. Does someone familiar with AWS and UI5 know, how can I store these values during deployment so I can use them later, during the run of the web application? Thank you.
If you can edit the CloudFormation template then you can add resources of type AWS:SSM:Parameter to the template which will create SSM Parameters as part of the deployment.
Example:
Resources:
BasicParameter:
Type: AWS::SSM::Parameter
Properties:
Name: /my/basic/parameter
Type: String
Value: my-basic-parameter-value
Description: A description of the parameter
Im creating API gateway stage using cloudformation.
ApiDeployment:
Type: AWS::ApiGateway::Deployment
Properties:
RestApiId: !Ref ExampleRestApi
StageName: dev
Here is the problem, Whenever I create a new API, I just need to deploy the stage using AWS console. is there any way that I can automate the deploy process so that no further console action is required.
When you define a Deployment resource like this, CloudFormation will create the deployment only on the first run. On the second run it will observe that the resource already exists and the CloudFormation definition did not change, so it won't create another deployment. To work around that, you can add something like a UUID/timestamp placeholder to the resource ID and replace it everytime before doing the CloudFormation update:
ApiDeployment#TIMESTAMP#:
Type: AWS::ApiGateway::Deployment
Properties:
RestApiId: !Ref ExampleRestApi
StageName: dev
This way you are still able to see your deployment history in the API Gateway console.
If you don't want to manipulate your template like this, you can also add a Lambda-backed Custom Resource to your CloudFormation stack. Using an AWS SDK, you can have the Lambda function automatically creating new deployments for you when the API was updated.
I've found berenbums response to be mostly correct, but there are a few things I don't like.
The proposed method of creating a resource like ApiDeployment#TIMESTAMP# doesn't keep the deployment history. This makes sense, since the old ApiDeployment#TIMESTAMP# element is being deleted and a new one is being created every time.
Using ApiDeployment#TIMESTAMP# creates a new deployment every time the template is deployed, which might be undesirable if the template is being deployed to create/update other resources.
Also, using ApiDeployment#TIMESTAMP# didn't work well when adding the StageDescription property. A potential solution is to add a static APIGwDeployment resource for the initial deployment (with StageDescription) and ApiDeployment#TIMESTAMP# for the updates.
The fundamental issue though, is that creating a new api gw deployment is not well suited for cloudformation (beyond the initial deployment). I think after the initial deployment, it's better to do an AWS API invocation to update the deployment (see https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-deployments.html).
In my particular case I created a small Ansible module to invoke aws apigateway create-deployment which updates an existing stage in one operation.
I am using Terraform for most of my infrastructure, but at the same time I'm using the serverless framework to define some Lambda functions. Serverless uses CloudFormation under the hood where I need access to some ARNs for resources created by Terraform.
My idea was to create a CloudFormation stack in Terraform and export all of the value that I need, but it complains that it cannot create a stack without any resources. I don't want to define any resources in CloudFormation, only the outputs, so I though maybe there is a way to define some dummy resource, but I couldn't find any.
Is there a way to work around this issue? If not, I'm also open to other suggestions for getting parameters passed from Terraform to CloudFormation.
You can use AWS::CloudFormation::WaitConditionHandle for this. Example:
Resources:
NullResource:
Type: AWS::CloudFormation::WaitConditionHandle
The Resource section is required, but you can create non-resource type of resource.
For example, minimalist template with only a non-resource would be:
Conditions:
Never:
!Equals [ "A", "B" ]
Resources:
NonResource:
Type: Custom::NonResource
Condition: Never
Outputs:
MyOutput:
Value: some-value
You can use create AWS SSM parameter using Terraform and reference them in your serverless framework. That would do the job easily.
https://www.serverless.com/blog/definitive-guide-terraform-serverless/
I am using serverless framework -
https://serverless.com/framework/docs/providers/aws/guide/serverless.yml/
Before I deploy the serverless stack, there are some manual steps, which I need to perform -
Creating S3 buckets
Creating Cognito User Pools, App clients, etc.
3.....
The ARNs of these AWS resources which are created in the above steps, are configured as environment variables in the serverless.yml file.
Apart from this, I want to avoid the possible problem of reaching the AWS cloudformation limit of 200 resources in one stack.
What is the best way/tools to split this stack into two parts?
Are there any examples, in which output of one stack is used as environment variables in the another stack?
Another option, I am thinking is to use the Cloudformation template, which Serverless framework creates and then use it inside a nested CF stack.
Any better options/tools?
Yes. This is very much possible.
Assuming you are using deploying from the same AWS account and Region
Instead of manually creating resources, use serverless to deploy these resources on AWS and use:
resources:
Outputs:
BucketName:
Value:
Ref: S3BucketResource
Export:
Name: VariableNameToImport
You can directly import these variable names in your main serverless.yml file and set them to ENVIRONMENT variables like:
environment:
S3BucketName:
'Fn::ImportValue': VariableNameToImport
OPTION 2 (Easier approach)
Or you can simply use plugin: serverless-plugin-split-stacks
I would like to perform the following operations in order with CloudFormation.
Start up an EC2 instance.
Give it privileges to access the full internet using security group A.
Download particular versions of Java and Python
Remove its internet privileges by removing security group A and adding a security group B.
I observe that there is a DependsOn attribute for specifying the order in which to create resources, but I was unable to find a feature that would allow me to update the security groups on the same EC2 instance twice over the course of creating a stack.
Is this possible with CloudFormation?
Not in CloudFormation natively, but you could launch the EC2 instance with a configured userdata script that itself downloads Java/Python and the awscli, as necessary, and then uses the awscli to switch security groups for the current EC2 instance.
However, if all you need is Java and Python pre-loaded then why not simply create an AMI with them already installed and launch from that AMI?
The best way out is to utilise a Cloudformation custom resource here. You can create a lambda function that does exactly what you need. This lambda function can then be called as a custom resource function in the cloud formation template.
You can pass your new security group ID and instance ID to the lambda function and code the lambda function to use AWS SDK and do the modifications that you need.
I have leveraged it to post an update to my web server about the progress of the cloud formation template. Below is the sample code of the template.
EC2InstanceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
Path: /
Roles: [!Ref 'EC2Role']
MarkInstanceProfileComplete:
Type: 'Custom::EC2InstanceProfileDone'
Version: '1.0'
DependsOn: EC2InstanceProfile
Properties:
ServiceToken: !Ref CustomResourceArn
HostURL: !Ref Host
LoginType: !Ref LoginType
SecretId: !Ref SecretId
WorkspaceId: !Ref WorkspaceId
Event: 2
Total: 3
Here the resource MarkInstanceProfileComplete is a custom resource that calls a Lambda function. It takes the event count and total count as input and processes them to calculate percentage progress. Based on that it sends out a request to my web server. For all we care, this Lambda function can do potentially anything you want it to do.