I've been trying to add a simple environment variable to a YAML cloudformation template. I pass in to the template as a parameter when creating whether it is "production" or "staging", and want to pass that on in the "Build" section of my codepipline.
- Name: "Build"
Actions:
- Name: "Build"
ActionTypeId:
Category: "Build"
Owner: "AWS"
Provider: "CodeBuild"
Version: "1"
Configuration:
ProjectName: !Ref CodeBuildProject
# EnvironmentVariables:
# - Name: "CURRENT_ENVIRONMENT"
# Type: PLAINTEXT
# Value: !Ref CodeEnvironment
# EnvironmentVariables: !Sub |
# "{\"name\":\"CURRENT_ENVIRONMENT\", \"type\": \"PLAINTEXT\" \"value\": \"${CodeEnvironment}\"}"
EnvironmentVariables: "{\"name\":\"CURRENT_ENVIRONMENT\", \"type\": \"PLAINTEXT\" \"value\": \"${CodeEnvironment}\"}"
InputArtifacts:
- Name: "SourceArtifact"
OutputArtifacts:
- Name: "secondary_artifact_name_1"
- Name: "secondary_artifact_name_2"
Region: !Ref AWS::Region
Namespace: "BuildVariables"
RunOrder: 1
`
I've tried a few different ways of passing in "EnvironmentVariables", but keep getting this error:
The configuration for the action 'Build' configurationKey 'EnvironmentVariables' does not match the expected format. The expected format is JSON array adhering to the following format: [{"name": "string", "type": "string", "value": "string"}] (Service: AWSCodePipeline; Status Code: 400; Error Code: InvalidActionDeclarationException; Request ID: fe7b8f6b-5410-48d2-b18f-f9377d1898cb; Proxy: null)
I've seen ways to do this in a json template, but having trouble doing it in a yaml template
The EnvironmentVariables should be JSON array, not a plain map. So it should be:
EnvironmentVariables: "[{\"name\":\"CURRENT_ENVIRONMENT\", \"type\": \"PLAINTEXT\" \"value\": \"${CodeEnvironment}\"}]"
Related
I have 2 CloudFormation and CodeBuild CodePipeline stages:
- Actions:
- ActionTypeId:
Category: "Deploy"
Owner: "AWS"
Provider: "CloudFormation"
Version: "1"
Configuration:
ActionMode: "CREATE_UPDATE"
Capabilities: "CAPABILITY_AUTO_EXPAND,CAPABILITY_NAMED_IAM,CAPABILITY_IAM"
RoleArn: !GetAtt CodePipelineServiceRole.Arn
StackName: !Ref CFNStackName
TemplatePath: !Sub "BuildArtifact::${ArtifactName}"
TemplateConfiguration: BuildArtifact::CFTemplateConfig.json
ParameterOverrides: !Sub '{"Env": "${Env}"}'
Name: "CloudFormation-step"
Region: !Sub ${AWS::Region}
InputArtifacts:
- Name: BuildArtifact
RunOrder: 1
Name: "Deploy"
- Actions:
- ActionTypeId:
Category: "Test"
Owner: "AWS"
Provider: "CodeBuild"
Version: "1"
Configuration:
ProjectName: !Ref CodeBuildTest
InputArtifacts:
- Name: SourceArtifact
Name: "Test"
Region: !Sub ${AWS::Region}
RunOrder:
The CloudFormation stage is creating an ALB. I would like to pass that ALB's ARN to the CodeBuild stage. How do I pass the ALB ARN to the CodeBuild buildspec.yml file?
Assuming that your CFN template returns the ALB ARN in its outputs, e.g. called AlbArn, then you can provide a namespace for it and use the namespace later to get the value.
For example (may need some further adjustments):
- Actions:
- ActionTypeId:
Category: "Deploy"
Owner: "AWS"
Provider: "CloudFormation"
Version: "1"
Configuration:
ActionMode: "CREATE_UPDATE"
Capabilities: "CAPABILITY_AUTO_EXPAND,CAPABILITY_NAMED_IAM,CAPABILITY_IAM"
RoleArn: !GetAtt CodePipelineServiceRole.Arn
StackName: !Ref CFNStackName
TemplatePath: !Sub "BuildArtifact::${ArtifactName}"
TemplateConfiguration: BuildArtifact::CFTemplateConfig.json
ParameterOverrides: !Sub '{"Env": "${Env}"}'
Name: "CloudFormation-step"
Region: !Sub ${AWS::Region}
InputArtifacts:
- Name: BuildArtifact
Namespace: CloudFromationDeployNamespace # <--- namespace
RunOrder: 1
Name: "Deploy"
- Actions:
- ActionTypeId:
Category: "Test"
Owner: "AWS"
Provider: "CodeBuild"
Version: "1"
Configuration:
ProjectName: !Ref CodeBuildTest
EnvironmentVariables: | # <- pass the AlbArn as ENV variable
{
"ALB_NAME": "#{CloudFromationDeployNamespace.AlbArn}"
}
InputArtifacts:
- Name: SourceArtifact
Name: "Test"
Region: !Sub ${AWS::Region}
RunOrder:
I'm trying to pass the CodePipeline variable #{codepipeline PipelineExecutionId} to both the codeBuild action and then a deploy action.
I understand this variable is readily available to all actions as explained in AWS docs.
I'm however having a difficulty with the syntax as the parameters is not passed thru into the actions.
I've been using the following code:
For the build actions:
- Name: "Build-Docker-Container"
Actions:
- Name: "Build-Docker-Container"
ActionTypeId:
Category: Build
Owner: AWS
Provider: CodeBuild
Version: "1"
Configuration:
ProjectName: !Sub ${ProjectName}-build-${BranchName}
EnvironmentVariables:
- Name: IMAGE_TAG
Type: PLAINTEXT
Value: "#{codepipeline.PipelineExecutionId}"
InputArtifacts:
- Name: !Ref ProjectName
RunOrder: 3
and for the deploy action:
- Name: "Deploy-Services"
Actions:
- Name: "Deploy-Services"
ActionTypeId:
Category: Deploy
Owner: AWS
Provider: CloudFormation
Version: "1"
Configuration:
ActionMode: CREATE_UPDATE
StackName: !Sub "${ProjectName}-services-${BranchName}"
TemplatePath: !Sub "${ProjectName}::aws/03-services.yml"
Capabilities: "CAPABILITY_NAMED_IAM"
RoleArn: !GetAtt DeployRole.Arn
ParameterOverrides: !Sub |
{
"ProjectName": "${ProjectName}",
"ExecutionId": "#{codepipeline.PipelineExecutionId}"
}
InputArtifacts:
- Name: !Ref ProjectName
- Name: InfrastructureOutput
RunOrder: 4
UPDATE The code was actually good; I just needed to update the CloudFormation pipeline stack to apply it (I thought the github webhook would trigger this, but it only update the actions inside the pipeline)
I can confirm that the syntax you've used in the deploy action is correct:
ParameterOverrides: !Sub |
{
"ProjectName": "${ProjectName}",
"ExecutionId": "#{codepipeline.PipelineExecutionId}"
}
I verified that on my Pipeline with CloudFormation provider. I can also confirm that this works as expected.
You can check in console if you edit the CFN action's in question, that the parameters are correctly set:
For now I can't verify build action, but by the look of it, it also seems fine.
I'm working off of the Cloudformation stack from this tutorial:
https://aws.amazon.com/blogs/compute/continuous-deployment-for-serverless-applications/
It creates a pipeline with a CodeCommit repository as a source. I'd like to switch this to a Github repository. Here's the code that is defining this resource:
Pipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
ArtifactStore:
Location: !Ref BuildArtifactsBucket
Type: S3
Name: !Sub ${ServiceName}_pipeline
RoleArn: !GetAtt PipelineExecutionRole.Arn
Stages:
- Name: Source
Actions:
- Name: CodeCommitRepo
ActionTypeId:
Category: Source
Owner: AWS
Provider: CodeCommit
Version: 1
Configuration:
RepositoryName: !Sub '${ServiceName}_repo'
BranchName: master
OutputArtifacts:
- Name: SourceZip
RunOrder: 1
How is GitHub defined as a resource and how is the authentication handled for a private repository?
For github you need to replace provider with github for example
Pipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
ArtifactStore:
Location: !Ref BuildArtifactsBucket
Type: S3
Name: !Sub ${ServiceName}_pipeline
RoleArn: !GetAtt PipelineExecutionRole.Arn
Stages:
- Name: Source
Actions:
- Name: GithubRepo
ActionTypeId:
Category: Source
Owner: ThirdParty
Provider: GitHub
Version: 1
Configuration:
"Owner": "MyGitHubAccountName",
"Repo": "MyGitHubRepositoryName",
"PollForSourceChanges": "false",
"Branch": "master",
"OAuthToken": "****"
OutputArtifacts:
- Name: SourceZip
RunOrder: 1
For more information click on
code pipeline thirdparty source provider
Here, is how to get github personal token and insert it to your code pipeline
github personal token intergration into code pipeline
If using a Github repository as a source in a CodeBuild project, the Branch Filter option allows to run builds only for branches, whose name is matching a certain regular expression.
AWS Management Console
In the AWS Management Console you can configure the branch filter upon creating or editing a CodeBuild project:
AWS CLI
For awscli exists an option --update-webhook (documented here)
$ aws codebuild update-webhook --project-name myproject --branch-filter ^master$
CloudFormation
In CodeBuild cloudformation template exists an option Triggers > Webhook (documented here), but this option is just a boolean for simple enabling/disabling the github webhook.
Resources:
MyCodeBuildProject:
Type: AWS::CodeBuild::Project
Properties:
Name: myproject
...
Triggers:
Webhook: true
So my question is, how to directly define a branch filter in a cloudformation template, without subsequently having to execute an awscli command or use the AWS Management Console?
You can try using AWS CodePipeline
Stages:
-
Name: "Source"
Actions:
-
Name: "Checkout"
ActionTypeId:
Category: "Source"
Owner: "ThirdParty"
Provider: "GitHub"
Version: "1"
Configuration:
Owner: !Ref "UsernameOrOrg"
Repo: !Ref "ProjectName"
Branch: "master"
OAuthToken: !Ref "GitHubOAuthToken"
OutputArtifacts:
-
Name: "checkout"
-
Name: "Build"
Actions:
-
Name: "Build"
ActionTypeId:
Category: "Build"
Owner: "AWS"
Provider: "CodeBuild"
Version: "1"
Configuration:
ProjectName: !Ref "BuildProject"
InputArtifacts:
-
Name: "checkout"
Then you just need to define your CodeBuild project with CodePipeline integration:
BuildProject:
Type: "AWS::CodeBuild::Project"
Properties:
...
Artifacts:
Type: "CODEPIPELINE"
Source:
Type: "CODEPIPELINE"
Here is a minimal example using triggers and webhook filters, filter group pattern can also be something like ^refs/heads/.*:
AWSTemplateFormatVersion: "2010-09-09"
Description: "CodeBuild project and IAM role"
Parameters:
Image:
Type: String
Description: "Name of the docker image."
Default: "my-image"
Resources:
CodeBuildRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
Effect: Allow
Principal:
Service: codebuild.amazonaws.com
Action: sts:AssumeRole
Policies:
- PolicyName: "CodeBuild-Service-Policy"
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: "Allow"
Action:
- "ecr:BatchCheckLayerAvailability"
- "ecr:CompleteLayerUpload"
- "ecr:DescribeImages"
- "ecr:GetAuthorizationToken"
- "ecr:InitiateLayerUpload"
- "ecr:ListImages"
- "ecr:PutImage"
- "ecr:UploadLayerPart"
- "logs:*"
Resource: "*"
CodeBuildProject:
Type: AWS::CodeBuild::Project
Properties:
Artifacts:
Type: NO_ARTIFACTS
Environment:
ComputeType: "BUILD_GENERAL1_SMALL"
Image: "aws/codebuild/docker:18.09.0"
Type: LINUX_CONTAINER
ServiceRole: !GetAtt CodeBuildRole.Arn
Source:
Type: GITHUB
Location: "https://github.com/ORG/REPO.git"
BuildSpec: "codebuild/create_docker_image.yml"
Triggers:
Webhook: true
FilterGroups:
- - Type: EVENT
Pattern: PUSH
- Type: HEAD_REF
Pattern: master
See also:
https://docs.amazonaws.cn/en_us/codebuild/latest/userguide/sample-bitbucket-pull-request.html#sample-bitbucket-pull-request-filter-webhook-events-cfn
Set source version in your template and branch will be selected automatically by cloud formation
Docs: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-codebuild-project.html#cfn-codebuild-project-sourceversion
"main" is the name of my branch, so
SourceVersion: refs/heads/main
I have a CloudFormation stack template which contains a DataPipeline resource having an RdsDatabase object:
AWSTemplateFormatVersion: '2010-09-09'
Resources:
ProUsageReportsPipelineStg:
Type: AWS::DataPipeline::Pipeline
Properties:
Name: my-db
PipelineObjects:
- id: ProAccountDB
type: RdsDatabase
region: us-west-2
username: username
"*password": password
rdsInstanceId: mydb
When I try to create this stack, I get the following error:
Encountered unsupported property *password
However, according to the documentation that is the place to pass the password.
You were quite close. The correct syntax would be something like this:
AWSTemplateFormatVersion: '2010-09-09'
Resources:
ProUsageReportsPipelineStg:
Type: AWS::DataPipeline::Pipeline
Properties:
Name: my-db
PipelineObjects:
-
Id: ProAccountDB
Name: "My Pro Account database"
Fields:
-
Key: "type"
StringValue: "RdsDatabase"
-
Key: "region"
StringValue: "us-west-2"
-
Key: "username"
StringValue: "username"
-
Key: "*password"
StringValue: "password"
-
Key: "rdsInstanceId"
StringValue: "mydb"
You can also check this example in the AWS docs for reference.