I'm attempting to setup AWS Codepipeline with a Cloudformation SAM stack that deploys to a Lambda and am doing this via the Pipeline console. The pipeline passes the clone from Codecommit process and the build process with Codebuild, however fails on the deploy with Cloudformation.
I get the following error message:
Action execution failed
Invalid TemplatePath: MyAppBuild::samTemplate.yml
I've been following the documentation (http://docs.aws.amazon.com/lambda/latest/dg/automating-deployment.html) pretty closely and in field 16 of the codepipeline setup form (Template File) I have been putting samTemplate.yml.
I also have samTemplate.yml in the root of my repo (which is also the root of the project).
I've matched the Codebuild output artifact name with the Cloudformation input artifact name and they match exactly.
Am I missing something here? How do I get the Cloudformation deploy process to recognize the sam template?
EDIT 1 I've switched to using Codestar instead of directly using CodePipeline. Nothing special in my buildspec.yml, but it's below if anyone's interested.
version: 0.2
phases:
install:
commands:
- echo "install"
pre_build:
commands:
- echo "pre_build"
build:
commands:
- aws cloudformation package --template template.yml --s3-bucket $S3_BUCKET --output-template template-export.json
artifacts:
type: zip
files:
- template-export.json
One thing to keep in mind for anyone new to CodeBuild, the zip file that gets created as a result is of the root level directory of your build operations and IS the zip file that gets transferred to final deployment (in my case, Lambda).
You need to add aws cloudformation package command in you buildspec.yml.
aws cloudformation package --debug --template-file <YourSamTemplate.yml> --s3-bucket <YourbucketName> --output-template-file <YourOutputSamTemplate.yml>
where YourSamTemplate.yml is the name of samTemplate.yml in your project root.
and YourOutputSamTemplate.yml is the new name which you eant to give your output file after cloud formation package command is done.
Then in the next stage you need to define your input-artifact as output of your aws codebuild stage and then use this input-artifact to map your template.
build-output::YourOutputSamTemplate.yml
I believe the output of the build stage is what needs to be passed into the CloudFormation action.
In the tutorial the build output is called NewSamTemplate.yaml
So try updating your TemplatePath to MyAppBuild::NewSamTemplate.yml
In my case, the error was in the filename. Instead of template.yml the actual file was template.yaml
Related
When i push changes in AWS CodeCommit Repo, I want to make JAR file with mvn install command for that Java Code and upload it to AWS Lambda function. Location of that Jar file should be inside src/main/target. Can anyone suggest buildspec.yaml file?
Assuming that you're using AWS SAM (Serverless Application Model), this is as simple as calling a single command in the post_build section of your buildspec.yaml. Example:
version: 0.2
phases:
install:
runtime-versions:
java: corretto8
pre_build:
commands:
- mvn clean
build:
commands:
- mvn install
post_build:
commands:
- sam deploy --stack-name lambda-java --no-confirm-changeset
artifacts:
files:
- target/lambda-java.jar
discard-paths: no
Please note though that you'll also have to set up a mechanism that kicks off the build process when you push any changes to your repository. The easiest way doing this is using AWS CodePipeline, as that nicely integrates with CodeCommit. Simply create a new pipeline, choose your existing CodeCommit repository where the Java-based Lambda is stored, and select CodeBuild as the build provider (skip the deploy stage).
Also note that your CodeBuild service role will have to have the appropriate permissions to deploy the Lambda function. As SAM is leveraged, this includes permissions to upload to S3 and update the corresponding CloudFormation stack (see stack-name parameter above).
From here on, whenever you push any changes to your repo, CodePipeline will trigger a build using CodeCommit, which will then deploy a new version of your Lambda via the sam deploy command in your buildspec.yaml.
I have created a working CodePipeline for my AWS SAM application.
It is using only Source and Build phases with the following buildspec.yaml file
version: 0.2
phases:
install:
runtime-versions:
python: 3.7
commands:
- pip install --user aws-sam-cli
- USER_BASE_PATH=$(python -m site --user-base)
- export PATH=$PATH:$USER_BASE_PATH/bin
build:
commands:
- sam build
post_build:
commands:
sam package --s3-bucket deploy-bucket --output-template-file deployment.yaml
# finally:
# sam deploy --template-file deployment.yaml --stack-name MyStackSAM--region us-east-1 --capabilities CAPABILITY_IAM
As you can see I have commented out the last two lines as I want to move that action to a Deploy stage in CodePipeline
My Deploy step looks like this:
My CloudFormationPipelineServiceRole has full admin permission at this point, never the less, I'm still getting the following error as the result of executing this stage.
Action execution failed
Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: XXXXFFFFFXXXX; S3 Extended Request ID: XXXXFFFFFXXXXFFFFFXXXXX=)
I am stuck as to why I'm getting this error. Any help would be greatly appreciated.
First, sam package expects source template file that needs to be passed via --template-file flag. I don't see that template file anywhere in your code. Which template file are you trying to package?
Second, you are not uploading the necessary artifacts to the s3 bucket. The only thing that you are uploading is zipped code but as you can see from the command that you have commented out:
sam deploy --template-file deployment.yaml --stack-name MyStackSAM--region us-east-1 --capabilities CAPABILITY_IAM
you also need this file deployment.yaml but you didn't specify that in your code. There is no way for CodeBuild to guess which artifacts you want to preserve.
You will need to add additional artifacts section to the bottom of your buildspec file and specify those artifacts.
artifacts:
type: zip
files:
- template.yaml # (where do you have this file?)
- outputtemplate.yaml # (deployment.yaml in your case)
Note that the artifacts section needs to be on the same level as version and phases
version: 0.2
phases:
...
artifacts:
...
I'm trying to hook my GitHub repo with S3 so every time there's a commit, AWS CodePipeline will deploy the ./<path>/public folder to a specified S3 bucket.
So far in my pipeline, the Source works (hooked to GitHub and picks up new commits) but the Deploy failed because: Action execution failed
BundleType must be either YAML or JSON.
This is how I set them up:
CodePipeline
Action name: Source
Action provider: GitHub
Repository: account/repo
Branch: master
GitHub webhooks
CodeDeploy
Compute type: AWS Lambda
Service role: myRole
Deployment settings: CodeDeployDefault.LambdaAllAtOnce
IAM Role: myRole
AWS Service
Choose the service that will use this role: Lambda / CodeDeploy
Select your use case: CodeDeploy
Policies: AWSCodeDeployRole
I understand that there must be a buildspec.yml file in the root folder. I've tried using a few files I could find but they don't seem to work. What did I do wrong or how should I edit the buildspec file to do what I want?
Update
Thanks to #Milan Cermak. I understand I need to do:
CodePipeline:
Stage 1: Source: hook with GitHub repo. This one is working.
Stage 2: Build: use CodeBuild to grab only the wanted folder using a buildspec.yml file in the root folder of the repo.
Stage 3: Deploy: use
Action Provider: S3
Input Artifacts: OutputArtifacts (result of stage 2).
Bucket: the bucket that hosts the static website.
CodePipeline works. However, the output contains only files (.html) not folders nested inside the public folder.
I've checked this and figured how to remove path of a nested folder with discard-paths: yes but I'm unable to get all the sub-folders inside the ./<path>/public folder. Any suggestion?
CodeBuild use buildspec, but CodeDeploy use appspec.
Is there any appspec file?
You shouldn't use CodeDeploy, as that's a service for automation of deployments of applications, but rather CodeBuild, which executes commands and prepares the deployment artifact for further use in the pipeline.
These commands are in thebuildspec.yml file (typically in the root directory of the repo, but it's configurable). For your use case, it won't be too complicated, as you're not compiling anything or running tests, etc.
Try this as a starting point:
version: 0.2
phases:
build:
commands:
- ls
artifacts:
files:
- public/*
The phases section is required, that's why it's included (at least, thanks to the ls command, you'll see what files are present in the CodeBuild environment), but it's not interesting for your case. What is interesting is the artifacts section. That's where you define what is the output of the CodeBuild phase, i.e. what gets passed further to the next step in the pipeline.
Depending on how you want to have the files structured (for example, do you want to have the public directory also in the artifact or do you only want to have the files themselves, without the parent dir), you might want to use other configuration that's possible in the artifacts section. See the buildspec reference for details.
Remember to use the output artifact of the CodeBuild step as the input artifact of the Deploy to S3 step.
Buildspec is for CodeBuild as t_yamo pointed out.
You are using CodeDeploy which uses an appspec.yml file, which looks something like this for my config.
version: 0.0
os: linux
files:
- source: /
destination: /path/to/destination
hooks:
BeforeInstall:
- location: /UnzipResourceBundle.sh
ApplicationStart:
- location: /RestartServer.sh
timeout: 3600
UnzipResourceBundle.sh is just a bash script which can be used to do any number of things.
#!/bin/bash
// Do something
You can find a sample for the AppSpec.yml file from Amazon Documentation here - https://docs.aws.amazon.com/codedeploy/latest/userguide/reference-appspec-file-example.html#appspec-file-example-lambda
CodePipeline recently announced a deploy to S3 action: https://aws.amazon.com/about-aws/whats-new/2019/01/aws-codepipeline-now-supports-deploying-to-amazon-s3/
A simple buildspec like:
version: 0.2
phases:
install:
commands:
- (cd lambda/src; npm install)
- aws cloudformation package --template-file lambda/sam.yml --s3-bucket skynet-lambda --output-template-file SkynetLambdaPackaged.yml
artifacts:
type: zip
files:
- SkynetLambdaPackaged.yml
Works fine when I have one action in my build stage. But what if I want to have more build actions for example: I want to build my api server and frontend files in parallel. How do I model this?
UPDATE
In CodePipeline I can create actions that run in parallel like below, how is this modeled in buildspec? Or isit impossible?
You can use two different CodeBuild projects from the same source as two separate parallel actions in your CodePipeline.
For this to happen, you can use two buildspec files in your source.
e.g.
buildspec-frontend.yml
phases:
install:
commands:
- (cd frontend/src; npm run build)
- aws s3 sync frontend/dist s3://<insert s3 bucket url here>/ --delete
buildspec-backend.yml
phases:
install:
commands:
- (cd lambda/src; npm install)
- aws cloudformation package --template-file lambda/sam.yml --s3-bucket skynet-lambda --output-template-file SkynetLambdaPackaged.yml
Then, create a frontend CodeBuild project that uses the frontend buildspec. Repeat for the backend.
Then, when you go to your Build stage in your CodePipeline, use the two CodeBuild projects as parallel actions.
Update: The information below is now irrelevant since I misunderstood the question.
If your frontend can be deployed to s3, just add its deployment commands where you put your api deployment commands.
e.g.
phases:
install:
commands:
- (cd lambda/src; npm install)
- aws cloudformation package --template-file lambda/sam.yml --s3-bucket skynet-lambda --output-template-file SkynetLambdaPackaged.yml
- (cd frontend/src; npm run build)
- aws s3 sync frontend/dist s3://<insert s3 bucket url here>/ --delete
If your frontend is not on s3, just replace those lines with your own frontend deployment commands.
CodeBuild executes those commands in sequence. If you really need to run them in parallel, there are many ways to do it.
My preference is to put the commands in a Makefile and call them from your buildspec.yml (e.g. make --jobs 2 backend frontend).
enter image description here
From my understanding, if you just have one source, you cannot have two buildspec file because you can only name that file as buildspec. Maybe you can try "Insert build commands" option
I have a pipeline in AWS with Codestar, CodeBuild, CloudFormation, etc.
I am trying to figure out how to get information from the CloudFormation step returned to the CodeBuild step. Let me break it down:
I have a buildspec.yml for CodeBuild
# buildspec.yml
...
phases:
...
build:
commands:
- aws cloudformation package --region $REGION --template template.yml --s3-bucket $S3_BUCKET --output-template $OUTPUT_TEMPLATE
The above kicks off a CloudFormation build using our template.yml
# template.yml
...
S3BucketAssets:
Type: AWS::S3::Bucket
...
At this point, it creates a unique name for an S3 bucket. Awesome. Now, for step 2 in my buildspec.yml for CodeBuild, I want to push items to the S3 bucket just created in the CloudFormation template. BUT, I don't know how to get the dynamically created name of the S3 bucket from the CloudFormation template. I want something similar to:
# buildspec.yml
...
phases:
...
build:
commands:
# this is the same step as shown above
- aws cloudformation package --region $REGION --template template.yml --s3-bucket $S3_BUCKET --output-template $OUTPUT_TEMPLATE
# this is the new step
- aws s3 sync dist_files/ s3://{NAME_OF_THE_NEW_S3_BUCKET}
How can I accomplish getting the dynamically named S3 bucket so that I can push to it?
I am aware that within a CloudFormation template, you can reference the S3 bucket name with something like !GetAtt [ClientWebAssets, WebsiteURL]. But, I do not know how to get that information out of the cloudformation template and back into the codebuild template.
You could move to using CodePipeline. Stage one would be deploying the application via cloudformation with an output artifact being the stack creation output
http://docs.aws.amazon.com/codepipeline/latest/userguide/reference-pipeline-structure.html#action-requirements