Pass AWS CodePipeline variable to actions - amazon-web-services

I'm trying to pass the CodePipeline variable #{codepipeline PipelineExecutionId} to both the codeBuild action and then a deploy action.
I understand this variable is readily available to all actions as explained in AWS docs.
I'm however having a difficulty with the syntax as the parameters is not passed thru into the actions.
I've been using the following code:
For the build actions:
- Name: "Build-Docker-Container"
Actions:
- Name: "Build-Docker-Container"
ActionTypeId:
Category: Build
Owner: AWS
Provider: CodeBuild
Version: "1"
Configuration:
ProjectName: !Sub ${ProjectName}-build-${BranchName}
EnvironmentVariables:
- Name: IMAGE_TAG
Type: PLAINTEXT
Value: "#{codepipeline.PipelineExecutionId}"
InputArtifacts:
- Name: !Ref ProjectName
RunOrder: 3
and for the deploy action:
- Name: "Deploy-Services"
Actions:
- Name: "Deploy-Services"
ActionTypeId:
Category: Deploy
Owner: AWS
Provider: CloudFormation
Version: "1"
Configuration:
ActionMode: CREATE_UPDATE
StackName: !Sub "${ProjectName}-services-${BranchName}"
TemplatePath: !Sub "${ProjectName}::aws/03-services.yml"
Capabilities: "CAPABILITY_NAMED_IAM"
RoleArn: !GetAtt DeployRole.Arn
ParameterOverrides: !Sub |
{
"ProjectName": "${ProjectName}",
"ExecutionId": "#{codepipeline.PipelineExecutionId}"
}
InputArtifacts:
- Name: !Ref ProjectName
- Name: InfrastructureOutput
RunOrder: 4
UPDATE The code was actually good; I just needed to update the CloudFormation pipeline stack to apply it (I thought the github webhook would trigger this, but it only update the actions inside the pipeline)

I can confirm that the syntax you've used in the deploy action is correct:
ParameterOverrides: !Sub |
{
"ProjectName": "${ProjectName}",
"ExecutionId": "#{codepipeline.PipelineExecutionId}"
}
I verified that on my Pipeline with CloudFormation provider. I can also confirm that this works as expected.
You can check in console if you edit the CFN action's in question, that the parameters are correctly set:
For now I can't verify build action, but by the look of it, it also seems fine.

Related

How can I skip a codepipeline stage?

I am using AWS codepipeline as CI/CD pipeline tool. There are many stages in my yml file. The cloudformation reference for codepipeline is: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-codepipeline-pipeline.html
...
Stages:
- Name: ...
Actions: ...
...
There are a list of stages in the pipeline but not all of them are required to run for every branch. How can I skip some of them based on branch name?
I tried below configuration which is passed by parameters via aws cloudformation deploy --parameter-overrides ... command. It works but if I change the parameter in the deploy command, it shows No changes to deploy.. It seems that the template doesn't update the parameter change.
Parameters:
ShouldRunTest:
Type: String
Default: false
Conditions:
ShouldRunTest: !Equals [!Ref ShouldRunTest, "true"]
...
- !If
- ShouldRunTest
- Name: test
Actions:
- Name: Deploy
ActionTypeId:
Category: Deploy
Owner: AWS
Provider: CloudFormation
Version: 1
InputArtifacts:
- Name: SourceOutput
Configuration:
Capabilities: CAPABILITY_NAMED_IAM
ActionMode: REPLACE_ON_FAILURE
RoleArn: !GetAtt CodePipelineServiceRole.Arn
StackName: codeBuild
RunOrder: 1
- !Ref "AWS::NoValue"
You can skip entire AWS CodePipeline actions or stages using conditions in your AWS CloudFormation template.
For example:
Conditions:
HaveCodeDeployStage:
!Not [ !Equals [!Ref CodeDeployAppName, ''] ]
Resources:
MyCodePipepline:
Type: AWS::CodePipeline::Pipeline
Properties:
#...
Stages:
- !If
- HaveCodeDeployStage
- Name: DeployStage
Actions:
- ActionTypeId:
Category: Deploy
Owner: AWS
Provider: CodeDeployToECS
Version: 1
#....
- !Ref "AWS::NoValue"
# ...

AWS CodePipeline Add Github Source from CloudFormation Template

I'm working off of the Cloudformation stack from this tutorial:
https://aws.amazon.com/blogs/compute/continuous-deployment-for-serverless-applications/
It creates a pipeline with a CodeCommit repository as a source. I'd like to switch this to a Github repository. Here's the code that is defining this resource:
Pipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
ArtifactStore:
Location: !Ref BuildArtifactsBucket
Type: S3
Name: !Sub ${ServiceName}_pipeline
RoleArn: !GetAtt PipelineExecutionRole.Arn
Stages:
- Name: Source
Actions:
- Name: CodeCommitRepo
ActionTypeId:
Category: Source
Owner: AWS
Provider: CodeCommit
Version: 1
Configuration:
RepositoryName: !Sub '${ServiceName}_repo'
BranchName: master
OutputArtifacts:
- Name: SourceZip
RunOrder: 1
How is GitHub defined as a resource and how is the authentication handled for a private repository?
For github you need to replace provider with github for example
Pipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
ArtifactStore:
Location: !Ref BuildArtifactsBucket
Type: S3
Name: !Sub ${ServiceName}_pipeline
RoleArn: !GetAtt PipelineExecutionRole.Arn
Stages:
- Name: Source
Actions:
- Name: GithubRepo
ActionTypeId:
Category: Source
Owner: ThirdParty
Provider: GitHub
Version: 1
Configuration:
"Owner": "MyGitHubAccountName",
"Repo": "MyGitHubRepositoryName",
"PollForSourceChanges": "false",
"Branch": "master",
"OAuthToken": "****"
OutputArtifacts:
- Name: SourceZip
RunOrder: 1
For more information click on
code pipeline thirdparty source provider
Here, is how to get github personal token and insert it to your code pipeline
github personal token intergration into code pipeline

Conditionally create CodePipeline actions based on CloudFormation conditions - not working

Conditionally create CodePipeline actions based on CloudFormation conditions
As per the above link fn::if works within aws codepipeline but unfortunately its not working for me
Below is my code:
- !If
- testCondition
- Name: SwitchEnvironment
ActionTypeId:
Category: Build
Owner: AWS
Provider: CodeBuild
Version: 1
Configuration:
ProjectName: !Ref SwitchDeployment
InputArtifacts:
- Name: Source
OutputArtifacts:
- Name: SwitchDeployment
RunOrder: 1
- !Ref AWS::NoValue
If I set this condition false, the cloudformation says "Property Actions cannot be empty".
I ran into the same error message when i put the !IF statement inside the Actions section. According to AWS documentation (link to AWS docs) a minimum of 1 action is required in a pipeline stage. So if condition evaluates to false there will be 0 actions and leads to that error.
The following worked for me (adapted to your example):
- !If
- testCondition
- Name: SwitchEnvironment
Actions:
- Name: NameOfYourConditionalAction
ActionTypeId:
Category: Build
Owner: AWS
Provider: CodeBuild
Version: 1
Configuration:
ProjectName: !Ref SwitchDeployment
InputArtifacts:
- Name: Source
OutputArtifacts:
- Name: SwitchDeployment
RunOrder: 1
- !Ref AWS::NoValue

How to set Branch Filter option in AWS CodeBuild cloudformation template?

If using a Github repository as a source in a CodeBuild project, the Branch Filter option allows to run builds only for branches, whose name is matching a certain regular expression.
AWS Management Console
In the AWS Management Console you can configure the branch filter upon creating or editing a CodeBuild project:
AWS CLI
For awscli exists an option --update-webhook (documented here)
$ aws codebuild update-webhook --project-name myproject --branch-filter ^master$
CloudFormation
In CodeBuild cloudformation template exists an option Triggers > Webhook (documented here), but this option is just a boolean for simple enabling/disabling the github webhook.
Resources:
MyCodeBuildProject:
Type: AWS::CodeBuild::Project
Properties:
Name: myproject
...
Triggers:
Webhook: true
So my question is, how to directly define a branch filter in a cloudformation template, without subsequently having to execute an awscli command or use the AWS Management Console?
You can try using AWS CodePipeline
Stages:
-
Name: "Source"
Actions:
-
Name: "Checkout"
ActionTypeId:
Category: "Source"
Owner: "ThirdParty"
Provider: "GitHub"
Version: "1"
Configuration:
Owner: !Ref "UsernameOrOrg"
Repo: !Ref "ProjectName"
Branch: "master"
OAuthToken: !Ref "GitHubOAuthToken"
OutputArtifacts:
-
Name: "checkout"
-
Name: "Build"
Actions:
-
Name: "Build"
ActionTypeId:
Category: "Build"
Owner: "AWS"
Provider: "CodeBuild"
Version: "1"
Configuration:
ProjectName: !Ref "BuildProject"
InputArtifacts:
-
Name: "checkout"
Then you just need to define your CodeBuild project with CodePipeline integration:
BuildProject:
Type: "AWS::CodeBuild::Project"
Properties:
...
Artifacts:
Type: "CODEPIPELINE"
Source:
Type: "CODEPIPELINE"
Here is a minimal example using triggers and webhook filters, filter group pattern can also be something like ^refs/heads/.*:
AWSTemplateFormatVersion: "2010-09-09"
Description: "CodeBuild project and IAM role"
Parameters:
Image:
Type: String
Description: "Name of the docker image."
Default: "my-image"
Resources:
CodeBuildRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
Effect: Allow
Principal:
Service: codebuild.amazonaws.com
Action: sts:AssumeRole
Policies:
- PolicyName: "CodeBuild-Service-Policy"
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: "Allow"
Action:
- "ecr:BatchCheckLayerAvailability"
- "ecr:CompleteLayerUpload"
- "ecr:DescribeImages"
- "ecr:GetAuthorizationToken"
- "ecr:InitiateLayerUpload"
- "ecr:ListImages"
- "ecr:PutImage"
- "ecr:UploadLayerPart"
- "logs:*"
Resource: "*"
CodeBuildProject:
Type: AWS::CodeBuild::Project
Properties:
Artifacts:
Type: NO_ARTIFACTS
Environment:
ComputeType: "BUILD_GENERAL1_SMALL"
Image: "aws/codebuild/docker:18.09.0"
Type: LINUX_CONTAINER
ServiceRole: !GetAtt CodeBuildRole.Arn
Source:
Type: GITHUB
Location: "https://github.com/ORG/REPO.git"
BuildSpec: "codebuild/create_docker_image.yml"
Triggers:
Webhook: true
FilterGroups:
- - Type: EVENT
Pattern: PUSH
- Type: HEAD_REF
Pattern: master
See also:
https://docs.amazonaws.cn/en_us/codebuild/latest/userguide/sample-bitbucket-pull-request.html#sample-bitbucket-pull-request-filter-webhook-events-cfn
Set source version in your template and branch will be selected automatically by cloud formation
Docs: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-codebuild-project.html#cfn-codebuild-project-sourceversion
"main" is the name of my branch, so
SourceVersion: refs/heads/main

Deploying Serverless Transforms with AWS CodePipelines

I have a pipeline set up per this CloudFormation Template.
When I try to deploy a template that is using AWS SAM, I get an error in the pipeline
Action execution failed
CreateStack cannot be used with templates containing Transforms. (Service: AmazonCloudFormation; Status Code: 400; Error Code: ValidationError; Request ID: 167007a4-7672-11e8-8f67-67e79ae9de20)
which is notabily complain about my Action Mode,
Configuration:
ActionMode: CREATE_UPDATE
I can use a Pipeline Code Build stage that uses AWS CLI cloudformation package like this,
version: 0.1
phases:
install:
commands:
- npm install time
- aws cloudformation package --template-file samTemplate.yaml --s3-bucket bucket-name
--output-template-file outputSamTemplate.yaml
artifacts:
type: zip
files:
- samTemplate.yaml
- outputSamTemplate.yaml
But I would rather use something prebuilt. How do I deploy Serverless Transform Cloudformation Templates with CodePipelines? Can I do it without using the AWS CLI to package and deploy the template?
Spinning up a CodeStar Python project gave me the answer. They notably have two Cloudformation actions in their deploy stage that do CHANGE_SET_REPLACE and CHANGE_SET_EXECUTE.
Removing extraneous info from the CF template, you can see the structure of the actions in,
Resources:
...
ProjectPipeline:
Type: 'AWS::CodePipeline::Pipeline'
Properties:
Stages:
-
Name: Deploy
Actions:
- Name: GenerateChangeSet
ActionTypeId:
Provider: CloudFormation
Configuration:
ActionMode: CHANGE_SET_REPLACE
- Name: ExecuteChangeSet
ActionTypeId:
Provider: CloudFormation
Configuration:
ActionMode: CHANGE_SET_EXECUTE
Below is the full template resource for the pipeline. Using the same buildspec.yml as above, their CodePipeline template looks like,
Resources:
...
ProjectPipeline:
Type: 'AWS::CodePipeline::Pipeline'
Description: Creating a deployment pipeline for your project in AWS CodePipeline
Properties:
Name: pipeline-pipeline
ArtifactStore:
Type: S3
Location:
Ref: PipelineArtifacts
RoleArn: !GetAtt [PipelineRole, Arn]
Stages:
-
Name: Source
Actions:
-
Name: CheckoutSourceTemplate
ActionTypeId:
Category: Source
Owner: AWS
Version: 1
Provider: CodeCommit
Configuration:
PollForSourceChanges: True
RepositoryName: !GetAtt [PipelineRepo, Name]
BranchName: master
OutputArtifacts:
- Name: TemplateSource
RunOrder: 1
-
Name: Build
Actions:
- ActionTypeId:
Owner: AWS
Category: Build
Version: 1
Provider: CodeBuild
Configuration:
ProjectName: !Ref ProjectId
InputArtifacts:
- Name: TemplateSource
OutputArtifacts:
- Name: BuildTemplate
RunOrder: 1
-
Name: Deploy
Actions:
- Name: GenerateChangeSet
ActionTypeId:
Owner: AWS
Category: Deploy
Version: 1
Provider: CloudFormation
Configuration:
ActionMode: CHANGE_SET_REPLACE
RoleArn: !GetAtt [PipelineRole, Arn]
StackName: project-stack
Capabilities: CAPABILITY_IAM
TemplatePath: BuildTemplate::outputSamTemplate.yaml
ChangeSetName: pipeline-changeset
InputArtifacts:
- Name: BuildTemplate
RunOrder: 1
- Name: ExecuteChangeSet
ActionTypeId:
Owner: AWS
Category: Deploy
Version: 1
Provider: CloudFormation
Configuration:
ActionMode: CHANGE_SET_EXECUTE
ChangeSetName: pipeline-changeset
StackName: project-stack
RunOrder: 2