Deploying Serverless Transforms with AWS CodePipelines - amazon-web-services

I have a pipeline set up per this CloudFormation Template.
When I try to deploy a template that is using AWS SAM, I get an error in the pipeline
Action execution failed
CreateStack cannot be used with templates containing Transforms. (Service: AmazonCloudFormation; Status Code: 400; Error Code: ValidationError; Request ID: 167007a4-7672-11e8-8f67-67e79ae9de20)
which is notabily complain about my Action Mode,
Configuration:
ActionMode: CREATE_UPDATE
I can use a Pipeline Code Build stage that uses AWS CLI cloudformation package like this,
version: 0.1
phases:
install:
commands:
- npm install time
- aws cloudformation package --template-file samTemplate.yaml --s3-bucket bucket-name
--output-template-file outputSamTemplate.yaml
artifacts:
type: zip
files:
- samTemplate.yaml
- outputSamTemplate.yaml
But I would rather use something prebuilt. How do I deploy Serverless Transform Cloudformation Templates with CodePipelines? Can I do it without using the AWS CLI to package and deploy the template?

Spinning up a CodeStar Python project gave me the answer. They notably have two Cloudformation actions in their deploy stage that do CHANGE_SET_REPLACE and CHANGE_SET_EXECUTE.
Removing extraneous info from the CF template, you can see the structure of the actions in,
Resources:
...
ProjectPipeline:
Type: 'AWS::CodePipeline::Pipeline'
Properties:
Stages:
-
Name: Deploy
Actions:
- Name: GenerateChangeSet
ActionTypeId:
Provider: CloudFormation
Configuration:
ActionMode: CHANGE_SET_REPLACE
- Name: ExecuteChangeSet
ActionTypeId:
Provider: CloudFormation
Configuration:
ActionMode: CHANGE_SET_EXECUTE
Below is the full template resource for the pipeline. Using the same buildspec.yml as above, their CodePipeline template looks like,
Resources:
...
ProjectPipeline:
Type: 'AWS::CodePipeline::Pipeline'
Description: Creating a deployment pipeline for your project in AWS CodePipeline
Properties:
Name: pipeline-pipeline
ArtifactStore:
Type: S3
Location:
Ref: PipelineArtifacts
RoleArn: !GetAtt [PipelineRole, Arn]
Stages:
-
Name: Source
Actions:
-
Name: CheckoutSourceTemplate
ActionTypeId:
Category: Source
Owner: AWS
Version: 1
Provider: CodeCommit
Configuration:
PollForSourceChanges: True
RepositoryName: !GetAtt [PipelineRepo, Name]
BranchName: master
OutputArtifacts:
- Name: TemplateSource
RunOrder: 1
-
Name: Build
Actions:
- ActionTypeId:
Owner: AWS
Category: Build
Version: 1
Provider: CodeBuild
Configuration:
ProjectName: !Ref ProjectId
InputArtifacts:
- Name: TemplateSource
OutputArtifacts:
- Name: BuildTemplate
RunOrder: 1
-
Name: Deploy
Actions:
- Name: GenerateChangeSet
ActionTypeId:
Owner: AWS
Category: Deploy
Version: 1
Provider: CloudFormation
Configuration:
ActionMode: CHANGE_SET_REPLACE
RoleArn: !GetAtt [PipelineRole, Arn]
StackName: project-stack
Capabilities: CAPABILITY_IAM
TemplatePath: BuildTemplate::outputSamTemplate.yaml
ChangeSetName: pipeline-changeset
InputArtifacts:
- Name: BuildTemplate
RunOrder: 1
- Name: ExecuteChangeSet
ActionTypeId:
Owner: AWS
Category: Deploy
Version: 1
Provider: CloudFormation
Configuration:
ActionMode: CHANGE_SET_EXECUTE
ChangeSetName: pipeline-changeset
StackName: project-stack
RunOrder: 2

Related

How can I skip a codepipeline stage?

I am using AWS codepipeline as CI/CD pipeline tool. There are many stages in my yml file. The cloudformation reference for codepipeline is: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-codepipeline-pipeline.html
...
Stages:
- Name: ...
Actions: ...
...
There are a list of stages in the pipeline but not all of them are required to run for every branch. How can I skip some of them based on branch name?
I tried below configuration which is passed by parameters via aws cloudformation deploy --parameter-overrides ... command. It works but if I change the parameter in the deploy command, it shows No changes to deploy.. It seems that the template doesn't update the parameter change.
Parameters:
ShouldRunTest:
Type: String
Default: false
Conditions:
ShouldRunTest: !Equals [!Ref ShouldRunTest, "true"]
...
- !If
- ShouldRunTest
- Name: test
Actions:
- Name: Deploy
ActionTypeId:
Category: Deploy
Owner: AWS
Provider: CloudFormation
Version: 1
InputArtifacts:
- Name: SourceOutput
Configuration:
Capabilities: CAPABILITY_NAMED_IAM
ActionMode: REPLACE_ON_FAILURE
RoleArn: !GetAtt CodePipelineServiceRole.Arn
StackName: codeBuild
RunOrder: 1
- !Ref "AWS::NoValue"
You can skip entire AWS CodePipeline actions or stages using conditions in your AWS CloudFormation template.
For example:
Conditions:
HaveCodeDeployStage:
!Not [ !Equals [!Ref CodeDeployAppName, ''] ]
Resources:
MyCodePipepline:
Type: AWS::CodePipeline::Pipeline
Properties:
#...
Stages:
- !If
- HaveCodeDeployStage
- Name: DeployStage
Actions:
- ActionTypeId:
Category: Deploy
Owner: AWS
Provider: CodeDeployToECS
Version: 1
#....
- !Ref "AWS::NoValue"
# ...

Pass AWS CodePipeline variable to actions

I'm trying to pass the CodePipeline variable #{codepipeline PipelineExecutionId} to both the codeBuild action and then a deploy action.
I understand this variable is readily available to all actions as explained in AWS docs.
I'm however having a difficulty with the syntax as the parameters is not passed thru into the actions.
I've been using the following code:
For the build actions:
- Name: "Build-Docker-Container"
Actions:
- Name: "Build-Docker-Container"
ActionTypeId:
Category: Build
Owner: AWS
Provider: CodeBuild
Version: "1"
Configuration:
ProjectName: !Sub ${ProjectName}-build-${BranchName}
EnvironmentVariables:
- Name: IMAGE_TAG
Type: PLAINTEXT
Value: "#{codepipeline.PipelineExecutionId}"
InputArtifacts:
- Name: !Ref ProjectName
RunOrder: 3
and for the deploy action:
- Name: "Deploy-Services"
Actions:
- Name: "Deploy-Services"
ActionTypeId:
Category: Deploy
Owner: AWS
Provider: CloudFormation
Version: "1"
Configuration:
ActionMode: CREATE_UPDATE
StackName: !Sub "${ProjectName}-services-${BranchName}"
TemplatePath: !Sub "${ProjectName}::aws/03-services.yml"
Capabilities: "CAPABILITY_NAMED_IAM"
RoleArn: !GetAtt DeployRole.Arn
ParameterOverrides: !Sub |
{
"ProjectName": "${ProjectName}",
"ExecutionId": "#{codepipeline.PipelineExecutionId}"
}
InputArtifacts:
- Name: !Ref ProjectName
- Name: InfrastructureOutput
RunOrder: 4
UPDATE The code was actually good; I just needed to update the CloudFormation pipeline stack to apply it (I thought the github webhook would trigger this, but it only update the actions inside the pipeline)
I can confirm that the syntax you've used in the deploy action is correct:
ParameterOverrides: !Sub |
{
"ProjectName": "${ProjectName}",
"ExecutionId": "#{codepipeline.PipelineExecutionId}"
}
I verified that on my Pipeline with CloudFormation provider. I can also confirm that this works as expected.
You can check in console if you edit the CFN action's in question, that the parameters are correctly set:
For now I can't verify build action, but by the look of it, it also seems fine.

Can AWS CodePipeline deploy with tags like the aws-cli?

I'm using AWS CodePipeline to deploy Cloudformation stacks. Previously I've used the aws-cli to create stacks with the aws create-stack --tags flag to propogate tags to the resources created. Is this possible with CodePipeline using the builtin CloudFormation provider? Here's an example of a deploy step I use:
- Name: DeployToDev
Actions:
- Name: CreateChangeSetDev
ActionTypeId:
Category: Deploy
Owner: AWS
Version: 1
Provider: CloudFormation
Configuration:
ChangeSetName: !Sub ${PipelineName}
ActionMode: CHANGE_SET_REPLACE
StackName: !Sub ${PipelineName}
Capabilities: CAPABILITY_NAMED_IAM
TemplatePath: BuildOutput::outputtemplate.yaml
TemplateConfiguration: !If [HasStackConfig,
!Sub 'SourceOutput::${StackConfigDev}',
!Ref "AWS::NoValue"]
RoleArn: !Sub arn:aws:iam::123456789012:role/${CloudFormationDeploymentRole}
InputArtifacts:
- Name: BuildOutput
- Name: SourceOutput
RunOrder: 1
RoleArn: !Sub arn:aws:iam::123456789012:role/CICD-CF-Role-${AWS::Region}
- Name: DeployChangeSetDev
ActionTypeId:
Category: Deploy
Owner: AWS
Version: 1
Provider: CloudFormation
Configuration:
ChangeSetName: !Sub ${PipelineName}
ActionMode: CHANGE_SET_EXECUTE
StackName: !Sub ${PipelineName}
RoleArn: !Sub arn:aws:iam::123456789012:role/${CloudFormationDeploymentRole}
InputArtifacts:
- Name: BuildOutput
RunOrder: 2
RoleArn: !Sub arn:aws:iam::123456789012:role/CICD-CF-Role-${AWS::Region}
Under your Configuration add tags like this
StackTags:
application-name: xxxx
environment: xxxx
owner: xxx
or try this if it doesn't work.
Tags:
- application-name: xxxx
- environment: xxxx
- owner: xxx

Cloudformation nested stacks without rollback root stack

I currently have a "master.yaml" template that runs "service-a.yaml" and "service-b.yaml" then "service-c.yaml" which relies on outputs from service-a and service-b.
Is there a way to break this nested stack into multiple nested stacks? That way when something deep inside "service-c" fails it doesn't cause a rollback all the way up the chain? I want to kick off A+B in parallel and then C when they are finished in an automated fashion.
I could have a master.yaml which builds "service-a" and "service-b" then manually kick off "service-c" when they're done but I would like to automate this somehow?
You can create a stack with Codebuild project and Codepipeline (Basically performing CI/CD) to trigger one stack after the other and thus each stack would fail and roll back separately.
For example the cloudformation template would have a Codebuld project as follows
CodeBuildProject:
Type: AWS::CodeBuild::Project
Properties:
Artifacts:
Type: CODEPIPELINE
Environment:
ComputeType: BUILD_GENERAL1_LARGE
Image: aws/codebuild/python:3.6.5
Type: LINUX_CONTAINER
EnvironmentVariables:
- Name: bucket
Value: !Ref ArtifactStoreBucket
Type: PLAINTEXT
- Name: prefix
Value: build
Type: PLAINTEXT
Name: !Ref AWS::StackName
ServiceRole: !Ref CodeBuildRole
Source:
Type: CODEPIPELINE
BuildSpec: stack/buildspec.yaml
Tags:
- Key: owner
Value: !Ref StackOwner
- Key: task
Value: !Ref RepositoryName
In the buildspec.yaml file, you can package the cloudfromation templates as follows:
- aws cloudformation package --template-file master.yaml
--s3-bucket $bucket --s3-prefix $prefix
--output-template-file master-template.yaml
- aws cloudformation package --template-file service-a.yaml
--s3-bucket $bucket --s3-prefix $prefix
--output-template-file service-a-template.yaml
And finally, a codepipeline stage which links all together. For example in the below-provided snippet, you can have source code triggered by codecommit. So every push to the repository would build your pipeline automatically.
Pipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
ArtifactStore:
Location: !Ref ArtifactStoreBucket
Type: S3
DisableInboundStageTransitions: []
Name: !Sub "${AWS::StackName}"
RoleArn: !GetAtt [PipelineRole, Arn]
Stages:
# Stage 1 - CodeUpdate Stage
- Name: CodeUpdate
Actions:
- Name: SourceCodeUpdate
ActionTypeId:
Category: Source
Owner: AWS
Version: '1'
Provider: CodeCommit
OutputArtifacts:
- Name: SourceCode
Configuration:
PollForSourceChanges: 'false'
RepositoryName: !Ref RepositoryName
BranchName: !Ref BranchName
RunOrder: '1'
# Stage 2 - Build Stage
- Name: Build
Actions:
- Name: UpdateLambda
ActionTypeId:
Category: Build
Owner: AWS
Version: '1'
Provider: CodeBuild
InputArtifacts:
- Name: SourceCode
OutputArtifacts:
- Name: BuildArtifact
Configuration:
ProjectName: !Ref 'CodeBuildProject'
RunOrder: '1'
# Stage 3 - Build master stack
- Name: MasterSetup
Actions:
- Name: CreateMasterChangeset
ActionTypeId:
Category: Deploy
Owner: AWS
Version: '1'
Provider: CloudFormation
InputArtifacts:
- Name: BuildArtifact
Configuration:
ActionMode: CHANGE_SET_REPLACE
StackName: !Sub "${AWS::StackName}-master"
ChangeSetName: !Sub "${AWS::StackName}-master-update"
RoleArn: !GetAtt [CFNRole, Arn]
TemplatePath: BuildArtifact::master-template.yaml
Capabilities: CAPABILITY_IAM
ParameterOverrides: !Sub
- |
{
"MasterStack": "${w}",
"StackOwner": "${x}",
"Task": "${y}"
}
- {
w: !Sub '${AWS::StackName}',
x: !Sub '${StackOwner}',
y: !Sub '${RepositoryName}'
}
RunOrder: '1'
- Name: ExecuteMasterChangeset
ActionTypeId:
Category: Deploy
Owner: AWS
Version: '1'
Provider: CloudFormation
Configuration:
ActionMode: CHANGE_SET_EXECUTE
StackName: !Sub "${AWS::StackName}-master"
ChangeSetName: !Sub "${AWS::StackName}-master-update"
RunOrder: '2'
# Stage 4 - Build service-a stack
- Name: ServiceASetup
Actions:
- Name: CreateServiceAChangeset
ActionTypeId:
Category: Deploy
Owner: AWS
Version: '1'
Provider: CloudFormation
InputArtifacts:
- Name: BuildArtifact
Configuration:
ActionMode: CHANGE_SET_REPLACE
StackName: !Sub "${AWS::StackName}-service-a"
ChangeSetName: !Sub "${AWS::StackName}-service-a-update"
RoleArn: !GetAtt [CFNRole, Arn]
TemplatePath: BuildArtifact::service-a-template.yaml
Capabilities: CAPABILITY_IAM
ParameterOverrides: !Sub
- |
{
"MasterStack": "${w}",
"StackOwner": "${x}",
"Task": "${y}"
}
- {
w: !Sub '${AWS::StackName}',
x: !Sub '${StackOwner}',
y: !Sub '${RepositoryName}'
}
RunOrder: '1'
- Name: ExecuteServiceAChangeset
ActionTypeId:
Category: Deploy
Owner: AWS
Version: '1'
Provider: CloudFormation
Configuration:
ActionMode: CHANGE_SET_EXECUTE
StackName: !Sub "${AWS::StackName}-service-a"
ChangeSetName: !Sub "${AWS::StackName}-service-a-update"
RunOrder: '2'
If you want to have stacks executing in parallel, you can add more than 1 stack in each stage.
Obviously you need to setup the roles and buckets yourself and this should give you basic idea how to get started.
For more information, you can read up more about codepipeline as follows:
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-cd-pipeline.html
https://docs.aws.amazon.com/codepipeline/latest/userguide/reference-pipeline-structure.html

AWS CodePipeline Add Github Source from CloudFormation Template

I'm working off of the Cloudformation stack from this tutorial:
https://aws.amazon.com/blogs/compute/continuous-deployment-for-serverless-applications/
It creates a pipeline with a CodeCommit repository as a source. I'd like to switch this to a Github repository. Here's the code that is defining this resource:
Pipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
ArtifactStore:
Location: !Ref BuildArtifactsBucket
Type: S3
Name: !Sub ${ServiceName}_pipeline
RoleArn: !GetAtt PipelineExecutionRole.Arn
Stages:
- Name: Source
Actions:
- Name: CodeCommitRepo
ActionTypeId:
Category: Source
Owner: AWS
Provider: CodeCommit
Version: 1
Configuration:
RepositoryName: !Sub '${ServiceName}_repo'
BranchName: master
OutputArtifacts:
- Name: SourceZip
RunOrder: 1
How is GitHub defined as a resource and how is the authentication handled for a private repository?
For github you need to replace provider with github for example
Pipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
ArtifactStore:
Location: !Ref BuildArtifactsBucket
Type: S3
Name: !Sub ${ServiceName}_pipeline
RoleArn: !GetAtt PipelineExecutionRole.Arn
Stages:
- Name: Source
Actions:
- Name: GithubRepo
ActionTypeId:
Category: Source
Owner: ThirdParty
Provider: GitHub
Version: 1
Configuration:
"Owner": "MyGitHubAccountName",
"Repo": "MyGitHubRepositoryName",
"PollForSourceChanges": "false",
"Branch": "master",
"OAuthToken": "****"
OutputArtifacts:
- Name: SourceZip
RunOrder: 1
For more information click on
code pipeline thirdparty source provider
Here, is how to get github personal token and insert it to your code pipeline
github personal token intergration into code pipeline