I am trying to convert a SAM project to a cloudformation template in order to call
cloudformation.createStack()
to create multiple stacks when a lambda is invoked. So far I can upload the SAM project with
sam build
sam package
But the size of the S3 is to big and I am getting errors. What are the steps to correctly upload the cloudformation template?
These pre-reqs need to be met before continuing:
Install the SAM CLI.
Create an Amazon S3 bucket to store the serverless code artifacts that the SAM template generates. At a minimum, you will need permission to put objects into the bucket.
The permissions applied to your IAM identity must include iam:ListPolicies.
4.You must have AWS credentials configured either via the AWS CLI or in your shell's environment via the AWS_* environment variables.
5.Git installed.
6.Python 3.x installed.
(Optional) Install Python's virtualenvwrapper.
Source Link:- https://www.packetmischief.ca/2020/12/30/converting-from-aws-sam-to-cloudformation/
Related
I'm trying to deploy an AWS reference solution using AWS Cloud9, that demonstrates a sample SaaS system. The repo for the solution is here:
https://github.com/aws-samples/aws-saas-factory-ref-solution-serverless-saas/blob/main/DOCUMENTATION.md
There's a deployment file that can be run in order to bootstrap the example system in AWS - it's /aws-saas-factory-ref-solution-serverless-saas/deployment.sh within the repo.
Part of the deployment script builds bootstrap resources using the file /aws-saas-factory-ref-solution-serverless-saas/server/bootstrap-template.yaml, which is a CloudFormation template. The script then deploys bootstrap resources based on the configuration file /aws-saas-factory-ref-solution-serverless-saas/server/samconfig-bootstrap.toml
The relevant parts of ./deployment.sh that perform the build and bootstrap are as follows. The build and deployment utilise the AWS 'Serverless Application Model' (sam):
sam build -t bootstrap-template.yaml --use-container --region=$REGION
sam deploy --config-file samconfig-bootstrap.toml --region=$REGION --parameter-overrides AdminEmailParameter=$1
When I run the ./deployment.sh on a Cloud 9 instance on EC2 (in Singapore region), I get the following error during the common resource bootstrapping stage:
[11034] Failed to execute script main
sam deploy --config-file samconfig-bootstrap.toml --region=ap-southeast-1 --parameter-overrides AdminEmailParameter=chris.halcrow#elenium.com
Error: Template file not found at /home/ec2-user/environment/aws-saas-factory-ref-solution-serverless-saas/server/template.yml
I'm not seeing anything when I search the code for template.yml, so I'm not sure what's attempting to reference this. The file isn't present in the solution.
The guide here describes how to quickly set up an AWS Cloud9 environment running on an EC2 instance, into which the solution can be cloned from its Github repo in order to run the ./deployment.sh script - https://catalog.us-east-1.prod.workshops.aws/workshops/b0c6ad36-0a4b-45d8-856b-8a64f0ac76bb/en-US/pre-requisites/12-own-aws-account
After looking at the build-spec file (buildspec.yaml files are used to provide meta-data for CI/CD in AWS CodeBuild), it seems that the AWS team is testing the deployment using CI/CD Pipelines and forgot to update the shell script.
You on the other hand don't need to worry about the the build-spec file.
Solution
Go and just change template.yml to tenant-template.yaml
I found out that template.yml is an artifact that's normally produced during the build process - see SAM deploy command reference
sam deploy
By default when you use this command, the AWS SAM CLI assumes that
your current working directory is your project's root directory. The
AWS SAM CLI first tries to locate a template file built using the sam
build command, located in the .aws-sam subfolder, and named
template.yaml.
It seems that the template.yml wasn't being created due to insufficient memory size of the EC2 instance that Cloud9 was running in.
The solution for this was provided by #ujwalbukka in the following git issue, within the repo for the reference solution:
https://github.com/aws-samples/aws-saas-factory-ref-solution-serverless-saas/issues/19
To get the SaaS reference solution to run in Cloud9, I cloned an additional repo inside Cloud9 (the repo for the AWS workshop for the SaaS reference solution). I cloned this using git clone https://github.com/aws-samples/aws-serverless-saas-workshop.git
After cloning that repo, I ran through the steps here to run scripts that exist in the workshop repo, that increase the Cloud9 EC2 instance size and install the prerequisites for running the reference solution.
I am hoping for some clarification around some terms I have been seeing on the web as it pertains to AWS and specifically lambdas. For starters, I would like the know how the commands sam build/deploy work versus setting up a CodeBuild job. Do I need a CodeBuild job to run those commands? What files specifically does the sam deploy command look for? Does it look for serverless.yml or template.yml or both? What is a sam.yml file or are they antiquated?
I have an app with a CodeBuild pipeline for a lambda, but I am expanding my repo to contain multiple lambdas and thinking about putting a serverless.yml file in each lambda directory, but I don't want to create a CodeBuild job and buildspec for each one. I assume sam deploy searches for all template.yml and serverless.yml files and constructs your stack as a whole (and updates only what needs to be updated?)
App is in Node if curious using API Gateway. Any insight would be appreciated.
I will try to give brief answers:
What does sam deploy do: It will zip the code and create cloudformation yaml file into .aws-sam folder and run cloudformation deploy.
Do we need CodeBuild to run same deploy: We still need some server to run sam deploy or build with node installed, which could be a local machine or remote server or a CodeBuild environment.
Do we need multiple templates? All Lambdas can be created in single template. But there is limit of 150 resources in cloudformation. if we have too many functions and APIs in single template, we will easily hit that limit. Each api might get converted into multiple cloud-formation resources. ex: 1 lambda function can be iam roles, cloudwatch logs, api routes, methods, integration, event source, etc.
Does sam deploy always looks for template.yaml By default yes, but can be easily overridden by passing --template-file sam deploy --template-file template-x.yml
Only changed resources are updated? Cloudformation update-stack updates only the resources that are changed.
We can plan terraform before applying the deployment, similarly do we have something in serverless?
You can run serverless package to create the CloudFormation json files.
The files will be created under the .serverless directory.
I am trying to create a pipeline using terraform to create a codepipeline in aws to automatically deploy a lambda function.
i have already created 2 stages to get the code from github and build the artifact using codebuild and store the artifact to S3.
But i can't seem to find a terraform configuration for the codedeploy to deploy the artifact from s3 to lambda. I do see there is deployment setting from the console where i can specify the detail of the deployment.
I am using amazon web services cli. I use a makefile to to build my lambda project and upload it to aws lambda. I am on a windows machine and using powershell to call make.
I try to delete my lambda function with the following lines
AWS_PATH = /cygdrive/c/Users/TestBox/AppData/Roaming/Python/Scripts/aws
AWS_WIN_PATH = $(shell cygpath -aw ${AWS_PATH})
AWS_REGION = eu-west-2
lambda_delete:
$(AWS_WIN_PATH) lambda delete-function --function-name LambdaTest --region $(AWS_REGION) --debug
I get this error..
NoCredentialsError: Unable to locate credentials
Unable to locate credentials. You can configure credentials by running "aws configure".
Running aws configure list prints out a valid default profile.
I think the problem is because i am using gnu make installed by cygwin on a windows machine. Using powershell to call make.
So the path to credentials looks like this "cygdrive/c/users/testbox/.aws/credentials" instead "c:\users\testbox.aws\credentials", when ~/.aws/credentials is evaluated by aws. I think :)
I had the same problem with the path to aws itself and had to use $(shell cygpath -aw ${AWS_PATH}) to convert it to a path windows python could use.
Is there any way to pass the credentials directly to the lambda delete-function or indirectly through a path to a file? I cant seem to think of a way because the code that searches for the credentials is internal to botocore.
Is there a way around this that you know off?
Alternative solution, consider using AWS SAM templates
Use AWS SAM templates to deploy your Lambda functions and AWS resources using CloudFormation.
Edit your SAM template and define your AWS resources. For example, define Lambda functions/path to your code.
aws cloudformation package to package and upload your local code to S3.
aws cloudformation deploy to provision and update AWS resources with the updated code on S3.
This would work in CMD/Powershell without the make hassle. You will also have the benefit of having your resources versioned as code and you won't need to worry about tracking or adding new AWS APIs in your make file.
More complex serverless frameworks for reference:
AWS Chalice https://github.com/aws/chalice
Django/Flask + Lambda https://github.com/Miserlou/Zappa
Cross cloud serverless solution https://github.com/serverless/serverless