I am very new to Cloudformation and I have a (maybe) stupid question. Here goes.
I want to describe my pipelines with cloudformation templates, which i commit to a repository (just for cloudformation templates).
Then i create a pipeline, that deploys the templates, and with that create the pipelines for the different apps.
This is currently working, but I have one issue: I can't connect to an existing repository. I can only figure out how to create a new repository within the stack, and use that repository in the pipeline. Se below template.
I have seen examples of people connection to github, doing something like this, and I would like to know if this is possible also with Codecommit. The issue is of course, that if you delete your cloudformation stack, you also delete your repository. And if you have an existing repository with your app, it gets gritty.
So is this possible, or have I misunderstood something (remember, I am new to this).
AWSTemplateFormatVersion: 2010-09-09
Resources:
CodePipeline:
Type: 'AWS::CodePipeline::Pipeline'
Properties:
RoleArn: !GetAtt CodePipeLineRole.Arn
ArtifactStore:
Location: !Ref PipelineBucket
Type: S3
Stages:
-
Name: Source
Actions:
- Name: CheckoutSourceTemplate
ActionTypeId:
Category: Source
Owner: AWS
Version: 1
Provider: CodeCommit
Configuration:
PollForSourceChanges: true
RepositoryName: !GetAtt
- PipelineRepo
- Name
BranchName: master
OutputArtifacts:
- Name: MyApp
RunOrder: 1
-
Name: Build
Actions:
-
Name: BuildAction
ActionTypeId:
Category: Build
Owner: AWS
Version: 1
Provider: CodeBuild
InputArtifacts:
-
Name: MyApp
OutputArtifacts:
-
Name: MyAppBuild
Configuration:
ProjectName: !Ref CodeBuild
PipelineRepo:
Type: 'AWS::CodeCommit::Repository'
Properties:
RepositoryName: evenz-react-app
RepositoryDescription: Pipeline repository
CodeBuildRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: Allow
Principal:
Service:
- "codebuild.amazonaws.com"
Action:
- "sts:AssumeRole"
Path: /service-role/
Policies:
- PolicyName: root
PolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: Allow
Action:
- "s3:GetObject"
- "s3:GetObjectVersion"
- "s3:GetBucketVersioning"
- "s3:PutObject"
Resource:
- !GetAtt PipelineBucket.Arn
- !Join ['', [!GetAtt PipelineBucket.Arn, "/*"]]
-
Effect: Allow
Action:
- "s3:GetObject"
- "s3:GetObjectVersion"
- "s3:GetBucketVersioning"
- "s3:PutObject"
- "s3:PutObjectAcl"
Resource:
- !GetAtt DeployBucket.Arn
- !Join ['', [!GetAtt DeployBucket.Arn, "/*"]]
-
Effect: Allow
Action:
- "logs:CreateLogGroup"
- "logs:CreateLogStream"
- "logs:PutLogEvents"
- "cloudfront:CreateInvalidation"
Resource:
- "*"
CodePipeLineRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: Allow
Principal:
Service:
- "codepipeline.amazonaws.com"
Action:
- "sts:AssumeRole"
Policies:
- PolicyName: root
PolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: Allow
Action:
- "s3:GetObject"
- "s3:GetObjectVersion"
- "s3:GetBucketVersioning"
- "s3:PutObject"
Resource:
- !GetAtt PipelineBucket.Arn
- !Join ['', [!GetAtt PipelineBucket.Arn, "/*"]]
-
Effect: Allow
Action:
- "codebuild:BatchGetBuilds"
- "codebuild:StartBuild"
Resource: "*"
-
Effect: Allow
Action:
- "codecommit:GetRepository"
- "codecommit:ListRepositories"
- "codecommit:GetBranch"
- "codecommit:GetCommit"
- "codecommit:UploadArchive"
- "codecommit:GetUploadArchiveStatus"
Resource: "*"
CodeBuild:
Type: 'AWS::CodeBuild::Project'
Properties:
Name: !Sub ${AWS::StackName}-CodeBuild
ServiceRole: !GetAtt CodeBuildRole.Arn
Artifacts:
Type: CODEPIPELINE
Name: MyProject
Source:
Type: CODEPIPELINE
Environment:
ComputeType: BUILD_GENERAL1_SMALL
Type: LINUX_CONTAINER
Image: "aws/codebuild/nodejs:8.11.0"
Source:
Type: CODEPIPELINE
BuildSpec: !Sub |
version: 0.1
phases:
pre_build:
commands:
- echo Installing source NPM dependencies...
- npm install
build:
commands:
- echo Build started on `date`
- npm run build
post_build:
commands:
- aws s3 cp --recursive --acl public-read ./build s3://${DeployBucket}/
- aws s3 cp --acl public-read --cache-control="max-age=0, no-cache, no-store, must-revalidate" ./build/service-worker.js s3://${DeployBucket}/
- aws s3 cp --acl public-read --cache-control="max-age=0, no-cache, no-store, must-revalidate" ./build/index.html s3://${DeployBucket}/
- aws cloudfront create-invalidation --distribution-id ${Distribution} --paths /index.html /service-worker.js
artifacts:
files:
- '**/*'
base-directory: build
PipelineBucket:
Type: 'AWS::S3::Bucket'
Properties: {}
DeployBucket:
Type: 'AWS::S3::Bucket'
Properties:
WebsiteConfiguration:
IndexDocument: index.html
Distribution:
Type: "AWS::CloudFront::Distribution"
Properties:
DistributionConfig:
Origins:
-
DomainName: !GetAtt DeployBucket.DomainName
Id: !Ref DeployBucket
S3OriginConfig:
OriginAccessIdentity: ''
DefaultRootObject: index.html
Enabled: true
DefaultCacheBehavior:
MinTTL: 86400
MaxTTL: 31536000
ForwardedValues:
QueryString: true
TargetOriginId: !Ref DeployBucket
ViewerProtocolPolicy: "redirect-to-https"
UPDATE:
Thanks to the answer from Marcin below, I changed the reference to the repository to a parameter instead of AWS::CodeCommit::Repository, which works perfectly like I was looking for. The full template now looks like this:
AWSTemplateFormatVersion: 2010-09-09
Parameters:
PipelineRepo:
Type: String
Default: evenz-react-app
Description: "Codecommit repo name"
Resources:
CodePipeline:
Type: 'AWS::CodePipeline::Pipeline'
Properties:
RoleArn: !GetAtt CodePipeLineRole.Arn
ArtifactStore:
Location: !Ref PipelineBucket
Type: S3
Stages:
-
Name: Source
Actions:
- Name: CheckoutSourceTemplate
ActionTypeId:
Category: Source
Owner: AWS
Version: 1
Provider: CodeCommit
Configuration:
PollForSourceChanges: true
RepositoryName: !Ref PipelineRepo
BranchName: master
OutputArtifacts:
- Name: MyApp
RunOrder: 1
-
Name: Build
Actions:
-
Name: BuildAction
ActionTypeId:
Category: Build
Owner: AWS
Version: 1
Provider: CodeBuild
InputArtifacts:
-
Name: MyApp
OutputArtifacts:
-
Name: MyAppBuild
Configuration:
ProjectName: !Ref CodeBuild
CodeBuildRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: Allow
Principal:
Service:
- "codebuild.amazonaws.com"
Action:
- "sts:AssumeRole"
Path: /service-role/
Policies:
- PolicyName: root
PolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: Allow
Action:
- "s3:GetObject"
- "s3:GetObjectVersion"
- "s3:GetBucketVersioning"
- "s3:PutObject"
Resource:
- !GetAtt PipelineBucket.Arn
- !Join ['', [!GetAtt PipelineBucket.Arn, "/*"]]
-
Effect: Allow
Action:
- "s3:GetObject"
- "s3:GetObjectVersion"
- "s3:GetBucketVersioning"
- "s3:PutObject"
- "s3:PutObjectAcl"
Resource:
- !GetAtt DeployBucket.Arn
- !Join ['', [!GetAtt DeployBucket.Arn, "/*"]]
-
Effect: Allow
Action:
- "logs:CreateLogGroup"
- "logs:CreateLogStream"
- "logs:PutLogEvents"
- "cloudfront:CreateInvalidation"
Resource:
- "*"
CodePipeLineRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: Allow
Principal:
Service:
- "codepipeline.amazonaws.com"
Action:
- "sts:AssumeRole"
Policies:
- PolicyName: root
PolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: Allow
Action:
- "s3:GetObject"
- "s3:GetObjectVersion"
- "s3:GetBucketVersioning"
- "s3:PutObject"
Resource:
- !GetAtt PipelineBucket.Arn
- !Join ['', [!GetAtt PipelineBucket.Arn, "/*"]]
-
Effect: Allow
Action:
- "codebuild:BatchGetBuilds"
- "codebuild:StartBuild"
Resource: "*"
-
Effect: Allow
Action:
- "codecommit:GetRepository"
- "codecommit:ListRepositories"
- "codecommit:GetBranch"
- "codecommit:GetCommit"
- "codecommit:UploadArchive"
- "codecommit:GetUploadArchiveStatus"
Resource: "*"
CodeBuild:
Type: 'AWS::CodeBuild::Project'
Properties:
Name: !Sub ${AWS::StackName}-CodeBuild
ServiceRole: !GetAtt CodeBuildRole.Arn
Artifacts:
Type: CODEPIPELINE
Name: MyProject
Source:
Type: CODEPIPELINE
Environment:
ComputeType: BUILD_GENERAL1_SMALL
Type: LINUX_CONTAINER
Image: "aws/codebuild/nodejs:8.11.0"
Source:
Type: CODEPIPELINE
BuildSpec: !Sub |
version: 0.1
phases:
pre_build:
commands:
- echo Installing source NPM dependencies...
- npm install
build:
commands:
- echo Build started on `date`
- npm run build
post_build:
commands:
- aws s3 cp --recursive --acl public-read ./build s3://${DeployBucket}/
- aws s3 cp --acl public-read --cache-control="max-age=0, no-cache, no-store, must-revalidate" ./build/service-worker.js s3://${DeployBucket}/
- aws s3 cp --acl public-read --cache-control="max-age=0, no-cache, no-store, must-revalidate" ./build/index.html s3://${DeployBucket}/
- aws cloudfront create-invalidation --distribution-id ${Distribution} --paths /index.html /service-worker.js
artifacts:
files:
- '**/*'
base-directory: build
PipelineBucket:
Type: 'AWS::S3::Bucket'
Properties: {}
DeployBucket:
Type: 'AWS::S3::Bucket'
Properties:
WebsiteConfiguration:
IndexDocument: index.html
Distribution:
Type: "AWS::CloudFront::Distribution"
Properties:
DistributionConfig:
Origins:
-
DomainName: !GetAtt DeployBucket.DomainName
Id: !Ref DeployBucket
S3OriginConfig:
OriginAccessIdentity: ''
DefaultRootObject: index.html
Enabled: true
DefaultCacheBehavior:
MinTTL: 86400
MaxTTL: 31536000
ForwardedValues:
QueryString: true
TargetOriginId: !Ref DeployBucket
ViewerProtocolPolicy: "redirect-to-https"
Maybe I misunderstood the issue, but I think that the separation of your PipelineRepo into its own template and stack would be helpful.
You could create a simple template for only the PipelineRepo which exports the RepoName e.g.
Resources:
PipelineRepo:
Type: 'AWS::CodeCommit::Repository'
Properties:
RepositoryName: evenz-react-app
RepositoryDescription: Pipeline repository
Outputs:
RepoName:
Value: !GetAtt PipelineRepo.Name
Export:
Name: RepoName
Then you would use ImportValue to import it:
Configuration:
PollForSourceChanges: true
RepositoryName: !ImportValue RepoName
Subsequently, the lifecycle of your PipelineRepo is not tided with the stack of your CodePipeline. CodePipeline stack can be deleted and re-created at anytime, without affecting the PipelineRepo.
p.s.
You may hear about importing existing resources into CloudFormation. Normally, this could be considered in a similar case to yours, but AWS::CodeCommit::Repository is not supported for such imports.
Related
I am trying to create a cloud formation stack to deploy a react app from an S3 bucket. The file is below. When I go to the cloud front URL, I get the following error:
ERROR
Failed to contact the origin.
Here is the file
AWSTemplateFormatVersion: 2010-09-09
Parameters:
ProjectSource:
Type: String
Default: "https://github.com/..."
Description: "Source control URL (e.g. Github)"
GithubOwner:
Type: String
Default: (myaccount)
GithubRepo:
Type: String
Default: (myrepo)
GithubOAuthToken:
Type: String
Default: "(my token)"
Description: "Github personal access token"
GithubBranch:
Type: String
Default: master
Description: "e.g. master or main"
Resources:
CodePipeline:
Type: 'AWS::CodePipeline::Pipeline'
Properties:
RoleArn: !GetAtt CodePipeLineRole.Arn
ArtifactStore:
Location: !Ref PipelineBucket
Type: S3
Stages:
-
Name: Source
Actions:
-
Name: SourceAction
ActionTypeId:
Category: Source
Owner: ThirdParty
Provider: GitHub
Version: 1
OutputArtifacts:
-
Name: NameForOutput
Configuration:
Owner: !Ref GithubOwner
Repo: !Ref GithubRepo
Branch: !Ref GithubBranch
OAuthToken: !Ref GithubOAuthToken
-
Name: Build
Actions:
-
Name: BuildAction
ActionTypeId:
Category: Build
Owner: AWS
Version: 1
Provider: CodeBuild
InputArtifacts:
-
Name: (NameHere)
OutputArtifacts:
-
Name: (NameBuild)
Configuration:
ProjectName: !Ref CodeBuild
CodeBuildRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: Allow
Principal:
Service:
- "codebuild.amazonaws.com"
Action:
- "sts:AssumeRole"
Path: /service-role/
Policies:
- PolicyName: root
PolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: Allow
Action:
- "s3:GetObject"
- "s3:GetObjectVersion"
- "s3:GetBucketVersioning"
- "s3:PutObject"
Resource:
- !GetAtt PipelineBucket.Arn
- !Join ['', [!GetAtt PipelineBucket.Arn, "/*"]]
-
Effect: Allow
Action:
- "s3:GetObject"
- "s3:GetObjectVersion"
- "s3:GetBucketVersioning"
- "s3:PutObject"
- "s3:PutObjectAcl"
Resource:
- !GetAtt DeployBucket.Arn
- !Join ['', [!GetAtt DeployBucket.Arn, "/*"]]
-
Effect: Allow
Action:
- "logs:CreateLogGroup"
- "logs:CreateLogStream"
- "logs:PutLogEvents"
- "cloudfront:CreateInvalidation"
Resource:
- "*"
CodePipeLineRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: Allow
Principal:
Service:
- "codepipeline.amazonaws.com"
Action:
- "sts:AssumeRole"
Policies:
- PolicyName: root
PolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: Allow
Action:
- "s3:GetObject"
- "s3:GetObjectVersion"
- "s3:GetBucketVersioning"
- "s3:PutObject"
Resource:
- !GetAtt PipelineBucket.Arn
- !Join ['', [!GetAtt PipelineBucket.Arn, "/*"]]
-
Effect: Allow
Action:
- "codebuild:BatchGetBuilds"
- "codebuild:StartBuild"
Resource: "*"
CodeBuild:
Type: 'AWS::CodeBuild::Project'
Properties:
Name: !Sub ${AWS::StackName}-CodeBuild
ServiceRole: !GetAtt CodeBuildRole.Arn
Artifacts:
Type: CODEPIPELINE
Name: MyProject
Source:
Type: CODEPIPELINE
Environment:
ComputeType: BUILD_GENERAL1_SMALL
Type: LINUX_CONTAINER
Image: "aws/codebuild/nodejs:10.14.1"
Source:
Type: CODEPIPELINE
BuildSpec: !Sub |
version: 0.1
phases:
pre_build:
commands:
- echo Installing source NPM dependencies...
- npm install
build:
commands:
- echo Build started on `date`
- npm run build
post_build:
on-failure: CONTINUE
commands:
- aws s3 cp --recursive --acl public-read ./build s3://${DeployBucket}/
#uncomment if you have service-worker.js
#- aws s3 cp --acl public-read --cache-control="max-age=0, no-cache, no-store, must-revalidate" ./build/service-worker.js s3://${DeployBucket}/
- aws s3 cp --acl public-read --cache-control="max-age=0, no-cache, no-store, must-revalidate" ./build/index.html s3://${DeployBucket}/
- aws cloudfront create-invalidation --distribution-id ${Distribution} --paths /index.html /service-worker.js
artifacts:
files:
- '**/*'
base-directory: build
PipelineBucket:
Type: 'AWS::S3::Bucket'
Properties: {}
DeployBucket:
Type: 'AWS::S3::Bucket'
Properties:
WebsiteConfiguration:
IndexDocument: index.html
Distribution:
Type: "AWS::CloudFront::Distribution"
Properties:
DistributionConfig:
Origins:
-
DomainName: !GetAtt DeployBucket.DomainName
Id: !Ref DeployBucket
S3OriginConfig:
OriginAccessIdentity: ''
DefaultRootObject: index.html
Enabled: true
DefaultCacheBehavior:
MinTTL: 86400
MaxTTL: 31536000
ForwardedValues:
QueryString: true
TargetOriginId: !Ref DeployBucket
ViewerProtocolPolicy: "redirect-to-https"
Your S3 bucket does not have any policy to allow Cloudfront Distribution to access files. You need to give permission to Cloudfront with CloudFrontOriginAccessIdentity.
AWSTemplateFormatVersion: 2010-09-09
Parameters:
ProjectSource:
Type: String
Default: "https://github.com/..."
Description: "Source control URL (e.g. Github)"
GithubOwner:
Type: String
Default: (myaccount)
GithubRepo:
Type: String
Default: (myrepo)
GithubOAuthToken:
Type: String
Default: "(my token)"
Description: "Github personal access token"
GithubBranch:
Type: String
Default: master
Description: "e.g. master or main"
Resources:
CodePipeline:
Type: 'AWS::CodePipeline::Pipeline'
Properties:
RoleArn: !GetAtt CodePipeLineRole.Arn
ArtifactStore:
Location: !Ref PipelineBucket
Type: S3
Stages:
-
Name: Source
Actions:
-
Name: SourceAction
ActionTypeId:
Category: Source
Owner: ThirdParty
Provider: GitHub
Version: 1
OutputArtifacts:
-
Name: NameForOutput
Configuration:
Owner: !Ref GithubOwner
Repo: !Ref GithubRepo
Branch: !Ref GithubBranch
OAuthToken: !Ref GithubOAuthToken
-
Name: Build
Actions:
-
Name: BuildAction
ActionTypeId:
Category: Build
Owner: AWS
Version: 1
Provider: CodeBuild
InputArtifacts:
-
Name: (NameHere)
OutputArtifacts:
-
Name: (NameBuild)
Configuration:
ProjectName: !Ref CodeBuild
CodeBuildRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: Allow
Principal:
Service:
- "codebuild.amazonaws.com"
Action:
- "sts:AssumeRole"
Path: /service-role/
Policies:
- PolicyName: root
PolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: Allow
Action:
- "s3:GetObject"
- "s3:GetObjectVersion"
- "s3:GetBucketVersioning"
- "s3:PutObject"
Resource:
- !GetAtt PipelineBucket.Arn
- !Join ['', [!GetAtt PipelineBucket.Arn, "/*"]]
-
Effect: Allow
Action:
- "s3:GetObject"
- "s3:GetObjectVersion"
- "s3:GetBucketVersioning"
- "s3:PutObject"
- "s3:PutObjectAcl"
Resource:
- !GetAtt DeployBucket.Arn
- !Join ['', [!GetAtt DeployBucket.Arn, "/*"]]
-
Effect: Allow
Action:
- "logs:CreateLogGroup"
- "logs:CreateLogStream"
- "logs:PutLogEvents"
- "cloudfront:CreateInvalidation"
Resource:
- "*"
CodePipeLineRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: Allow
Principal:
Service:
- "codepipeline.amazonaws.com"
Action:
- "sts:AssumeRole"
Policies:
- PolicyName: root
PolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: Allow
Action:
- "s3:GetObject"
- "s3:GetObjectVersion"
- "s3:GetBucketVersioning"
- "s3:PutObject"
Resource:
- !GetAtt PipelineBucket.Arn
- !Join ['', [!GetAtt PipelineBucket.Arn, "/*"]]
-
Effect: Allow
Action:
- "codebuild:BatchGetBuilds"
- "codebuild:StartBuild"
Resource: "*"
CodeBuild:
Type: 'AWS::CodeBuild::Project'
Properties:
Name: !Sub ${AWS::StackName}-CodeBuild
ServiceRole: !GetAtt CodeBuildRole.Arn
Artifacts:
Type: CODEPIPELINE
Name: MyProject
Source:
Type: CODEPIPELINE
Environment:
ComputeType: BUILD_GENERAL1_SMALL
Type: LINUX_CONTAINER
Image: "aws/codebuild/nodejs:10.14.1"
Source:
Type: CODEPIPELINE
BuildSpec: !Sub |
version: 0.1
phases:
pre_build:
commands:
- echo Installing source NPM dependencies...
- npm install
build:
commands:
- echo Build started on `date`
- npm run build
post_build:
on-failure: CONTINUE
commands:
- aws s3 cp --recursive --acl public-read ./build s3://${DeployBucket}/
#uncomment if you have service-worker.js
#- aws s3 cp --acl public-read --cache-control="max-age=0, no-cache, no-store, must-revalidate" ./build/service-worker.js s3://${DeployBucket}/
- aws s3 cp --acl public-read --cache-control="max-age=0, no-cache, no-store, must-revalidate" ./build/index.html s3://${DeployBucket}/
- aws cloudfront create-invalidation --distribution-id ${Distribution} --paths /index.html /service-worker.js
artifacts:
files:
- '**/*'
base-directory: build
PipelineBucket:
Type: 'AWS::S3::Bucket'
Properties: {}
DeployBucket:
Type: 'AWS::S3::Bucket'
Properties: {}
Distribution:
Type: "AWS::CloudFront::Distribution"
Properties:
DistributionConfig:
Origins:
-
DomainName: !GetAtt DeployBucket.RegionalDomainName
Id: !Ref DeployBucket
S3OriginConfig:
OriginAccessIdentity: !Sub 'origin-access-identity/cloudfront/${CloudFrontOriginAccessIdentity}'
DefaultRootObject: index.html
Enabled: true
DefaultCacheBehavior:
MinTTL: 86400
MaxTTL: 31536000
ForwardedValues:
QueryString: true
TargetOriginId: !Ref DeployBucket
ViewerProtocolPolicy: "redirect-to-https"
CloudFrontOriginAccessIdentity:
Type: AWS::CloudFront::CloudFrontOriginAccessIdentity
Properties:
CloudFrontOriginAccessIdentityConfig:
Comment: 'CloudFront OAI'
BucketPolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket: !Ref DeployBucket
PolicyDocument:
Statement:
- Action:
- s3:GetObject
Effect: Allow
Resource: !Join ['', [!GetAtt DeployBucket.Arn, '/*']]
Principal:
AWS: !Sub 'arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity ${CloudFrontOriginAccessIdentity}'
I'm trying to write a template that configures the whole ecs fargate server and its code pipeline.
There is no problem in all other configurations, but the image is empty because it is right after creating the ecr in cloudformation, and the create ecs service refers to the empty image and the process does not end.
So I want to push the server image to ecr with code build and then ecs service create to work, but I don't know how.
Could it be possible to trigger code build or code pipeline inside cloudformation?
If not, is there any way to do docker build & push?
Yes it can be done, I used it before to perform jobs like a database restore as part of stack creation (don't ask). What you need:
A custom resource lambda which kicks off the codebuild job. It will receive the codebuild project to start via a property passed to it in the cloudformation resource definition (assuming that the codebuild is determined at deploy time; else feel free to make the lambda know about which codebuild to run in whatever way makes the most sense to you).
The endpoint to call when the custom resource is complete. A custom resource will stay in CREATE_IN_PROGRESS state until the endpoint is called. It's been a while since I've used custom resources so don't remember where it comes from but I think it's found in the event that the custom resource lambda is invoked with.
Your codebuild job needs that endpoint and needs to be able to send a (GET or POST?) request to it, on both success and failure cases (you pass different params signifying success or failure).
So the overall sequence of steps is:
Define/reference the custom resource in your template, passing in whatever properties the lambda needs.
Deploy stack, custom resource lambda is invoked.
The custom resource status goes into CREATE_IN_PROGRESS
Lambda kicks off codebuild, passing in custom resource endpoint as a param or env var, and returns.
Codebuild starts doing its work.
Until the endpoint is invoked, the custom resource will remain as CREATE_IN_PROGRESS, and the stack create/update process will wait for it, even if it takes hours.
When codebuild has finished its work, it uses curl or similar to invoke that endpoint to signal it's complete.
Custom resource status goes to CREATE_COMPLETE (assuming you've invoked the endpoint with params saying it was successful).
Stack creation completes (or moves on to any resources that were dependent on the custom resource).
Yes, it is possible to trigger the Fargate deployment after pushing the image rather than when the CloudFormation template is run.
The trick is to set the DesiredCount property of AWS::ECS::Service to zero:
Service:
Type: AWS::ECS::Service
Properties:
Cluster: !Ref Cluster
DesiredCount: 0
LaunchType: FARGATE
NetworkConfiguration:
AwsvpcConfiguration:
SecurityGroups:
- !Ref SecG
Subnets: !Ref Subs
ServiceName: !Select [4, !Split ['-', !Select [2, !Split ['/', !Ref AWS::StackId]]]]
TaskDefinition: !Ref TaskDefinition
That said, you can also choose to create a repo with an initial commit that will trigger the build as soon as the template is done executing. This requires you to upload the zipped source code to an S3 bucket and configure the CodeCommit repository like so:
Repo:
Type: AWS::CodeCommit::Repository
Properties:
Code:
BranchName: main
S3:
Bucket: some-bucket
Key: code.zip
RepositoryName: !Select [4, !Split ['-', !Select [2, !Split ['/', !Ref AWS::StackId]]]]
RepositoryDescription: Repository
Triggers:
- Name: Trigger
CustomData: The Code Repository
DestinationArn: !Ref Topic
Branches:
- main
Events: [all]
Note that the some-bucket S3 bucket needs to contain the zipped .Dockerfile and any source code without any .git directory included.
You can see my implementation of this system and the rest of the stack below:
AWSTemplateFormatVersion: '2010-09-09'
Description: CloudFormation Stack to Trigger CodeBuild via CodePipeline
Parameters:
SecG:
Description: Single security group
Type: AWS::EC2::SecurityGroup::Id
Subs:
Description: Comma separated subnet IDs
Type: List<AWS::EC2::Subnet::Id>
ImagesFile:
Type: String
Default: images.json
Resources:
ArtifactBucket:
Type: AWS::S3::Bucket
DeletionPolicy: Retain
Properties:
PublicAccessBlockConfiguration:
BlockPublicAcls: true
BlockPublicPolicy: true
IgnorePublicAcls: true
RestrictPublicBuckets: true
Tags:
- Key: UseWithCodeDeploy
Value: true
CodeBuildServiceRole:
Type: AWS::IAM::Role
Properties:
Path: /
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service: codebuild.amazonaws.com
Action: sts:AssumeRole
Policies:
- PolicyName: !Sub 'ssm-${AWS::Region}-${AWS::StackName}'
PolicyDocument:
Version: '2012-10-17'
Statement:
-
Effect: Allow
Action:
- ssm:GetParameters
- secretsmanager:GetSecretValue
Resource: '*'
- PolicyName: !Sub 'logs-${AWS::Region}-${AWS::StackName}'
PolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: Allow
Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
Resource: '*'
- PolicyName: !Sub 'ecr-${AWS::Region}-${AWS::StackName}'
PolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: Allow
Action:
- ecr:BatchCheckLayerAvailability
- ecr:CompleteLayerUpload
- ecr:GetAuthorizationToken
- ecr:InitiateLayerUpload
- ecr:PutImage
- ecr:UploadLayerPart
- lightsail:*
Resource: '*'
- PolicyName: !Sub bkt-${ArtifactBucket}-${AWS::Region}
PolicyDocument:
Version: '2012-10-17'
Statement:
-
Effect: Allow
Action:
- s3:ListBucket
- s3:GetBucketLocation
- s3:ListBucketVersions
- s3:GetBucketVersioning
Resource:
- !Sub arn:aws:s3:::${ArtifactBucket}
- arn:aws:s3:::some-bucket
- PolicyName: !Sub obj-${ArtifactBucket}-${AWS::Region}
PolicyDocument:
Version: '2012-10-17'
Statement:
-
Effect: Allow
Action:
- s3:GetObject
- s3:PutObject
- s3:GetObjectAcl
- s3:PutObjectAcl
- s3:GetObjectTagging
- s3:PutObjectTagging
- s3:GetObjectVersion
- s3:GetObjectVersionAcl
- s3:PutObjectVersionAcl
Resource:
- !Sub arn:aws:s3:::${ArtifactBucket}/*
- arn:aws:s3:::some-bucket/*
CodeDeployServiceRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Sid: '1'
Effect: Allow
Principal:
Service:
- codedeploy.us-east-1.amazonaws.com
- codedeploy.eu-west-1.amazonaws.com
Action: sts:AssumeRole
Path: /
ManagedPolicyArns:
- arn:aws:iam::aws:policy/AWSCodeDeployRoleForECS
- arn:aws:iam::aws:policy/service-role/AWSCodeDeployRole
- arn:aws:iam::aws:policy/service-role/AWSCodeDeployRoleForLambda
CodeDeployRolePolicies:
Type: AWS::IAM::Policy
Properties:
PolicyName: !Sub 'CDPolicy-${AWS::Region}-${AWS::StackName}'
PolicyDocument:
Statement:
- Effect: Allow
Resource:
- '*'
Action:
- ec2:Describe*
- Effect: Allow
Resource:
- '*'
Action:
- autoscaling:CompleteLifecycleAction
- autoscaling:DeleteLifecycleHook
- autoscaling:DescribeLifecycleHooks
- autoscaling:DescribeAutoScalingGroups
- autoscaling:PutLifecycleHook
- autoscaling:RecordLifecycleActionHeartbeat
Roles:
- !Ref CodeDeployServiceRole
CodePipelineServiceRole:
Type: AWS::IAM::Role
Properties:
Path: /
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service: codepipeline.amazonaws.com
Action: sts:AssumeRole
Policies:
- PolicyName: !Sub 'root-${AWS::Region}-${AWS::StackName}'
PolicyDocument:
Version: '2012-10-17'
Statement:
- Resource:
- !Sub 'arn:aws:s3:::${ArtifactBucket}/*'
- !Sub 'arn:aws:s3:::${ArtifactBucket}'
Effect: Allow
Action:
- s3:PutObject
- s3:GetObject
- s3:GetObjectVersion
- s3:GetBucketAcl
- s3:GetBucketLocation
- Resource: "*"
Effect: Allow
Action:
- ecs:*
- Resource: "*"
Effect: Allow
Action:
- iam:PassRole
Condition:
StringLike:
iam:PassedToService:
- ecs-tasks.amazonaws.com
- Resource: !GetAtt Build.Arn
Effect: Allow
Action:
- codebuild:BatchGetBuilds
- codebuild:StartBuild
- codebuild:BatchGetBuildBatches
- codebuild:StartBuildBatch
- Resource: !GetAtt Repo.Arn
Effect: Allow
Action:
- codecommit:CancelUploadArchive
- codecommit:GetBranch
- codecommit:GetCommit
- codecommit:GetRepository
- codecommit:GetUploadArchiveStatus
- codecommit:UploadArchive
AmazonCloudWatchEventRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
-
Effect: Allow
Principal:
Service:
- events.amazonaws.com
Action: sts:AssumeRole
Path: /
Policies:
-
PolicyName: cwe-pipeline-execution
PolicyDocument:
Version: '2012-10-17'
Statement:
-
Effect: Allow
Action: codepipeline:StartPipelineExecution
Resource: !Sub arn:aws:codepipeline:${AWS::Region}:${AWS::AccountId}:${Pipeline}
AmazonCloudWatchEventRule:
Type: AWS::Events::Rule
Properties:
EventPattern:
source:
- aws.codecommit
detail-type:
- CodeCommit Repository State Change
resources:
- !GetAtt Repo.Arn
detail:
event:
- referenceCreated
- referenceUpdated
referenceType:
- branch
referenceName:
- main
Targets:
-
Arn: !Sub arn:aws:codepipeline:${AWS::Region}:${AWS::AccountId}:${Pipeline}
RoleArn: !GetAtt AmazonCloudWatchEventRole.Arn
Id: codepipeline-Pipeline
Topic:
Type: AWS::SNS::Topic
Properties:
Subscription:
- Endpoint: user#example.com
Protocol: email
TopicPolicy:
Type: AWS::SNS::TopicPolicy
Properties:
PolicyDocument:
Version: '2012-10-17'
Statement:
-
Sid: AllowPublish
Effect: Allow
Principal:
Service:
- 'codestar-notifications.amazonaws.com'
Action:
- 'SNS:Publish'
Resource:
- !Ref Topic
Topics:
- !Ref Topic
Repo:
Type: AWS::CodeCommit::Repository
Properties:
Code:
BranchName: main
S3:
Bucket: some-bucket
Key: code.zip
RepositoryName: !Select [4, !Split ['-', !Select [2, !Split ['/', !Ref AWS::StackId]]]]
RepositoryDescription: Repository
Triggers:
- Name: Trigger
CustomData: The Code Repository
DestinationArn: !Ref Topic
Branches:
- main
Events: [all]
RepoUser:
Type: AWS::IAM::User
Properties:
Path: '/'
ManagedPolicyArns:
- arn:aws:iam::aws:policy/AWSCodeCommitPowerUser
RepoUserKey:
Type: AWS::IAM::AccessKey
Properties:
UserName:
!Ref RepoUser
Registry:
Type: AWS::ECR::Repository
Properties:
RepositoryName: !Select [4, !Split ['-', !Select [2, !Split ['/', !Ref AWS::StackId]]]]
RepositoryPolicyText:
Version: '2012-10-17'
Statement:
- Sid: AllowPushPull
Effect: Allow
Principal:
AWS:
- !GetAtt CodeDeployServiceRole.Arn
Action:
- ecr:GetDownloadUrlForLayer
- ecr:BatchGetImage
- ecr:BatchCheckLayerAvailability
- ecr:PutImage
- ecr:InitiateLayerUpload
- ecr:UploadLayerPart
- ecr:CompleteLayerUpload
Build:
Type: AWS::CodeBuild::Project
Properties:
Artifacts:
Type: CODEPIPELINE
Source:
Type: CODEPIPELINE
BuildSpec: !Sub |
version: 0.2
phases:
pre_build:
commands:
- echo "[`date`] PRE_BUILD"
- echo "Logging in to Amazon ECR..."
- aws ecr get-login-password --region $REGION | docker login --username AWS --password-stdin $ACCOUNT.dkr.ecr.$REGION.amazonaws.com
- IMAGE_URI="$ACCOUNT.dkr.ecr.$REGION.amazonaws.com/$REPO:$TAG"
build:
commands:
- echo "[`date`] BUILD"
- echo "Building Docker Image..."
- docker build -t $REPO:$TAG .
- docker tag $REPO:$TAG $IMAGE_URI
post_build:
commands:
- echo "[`date`] POST_BUILD"
- echo "Pushing Docker Image..."
- docker push $IMAGE_URI
- echo Writing image definitions file...
- printf '[{"name":"svc","imageUri":"%s"}]' $IMAGE_URI > $FILE
artifacts:
files: $FILE
Environment:
ComputeType: BUILD_GENERAL1_SMALL
Image: aws/codebuild/standard:6.0
Type: LINUX_CONTAINER
EnvironmentVariables:
- Name: REGION
Type: PLAINTEXT
Value: !Ref AWS::Region
- Name: ACCOUNT
Type: PLAINTEXT
Value: !Ref AWS::AccountId
- Name: TAG
Type: PLAINTEXT
Value: latest
- Name: REPO
Type: PLAINTEXT
Value: !Ref Registry
- Name: FILE
Type: PLAINTEXT
Value: !Ref ImagesFile
PrivilegedMode: true
Name: !Ref AWS::StackName
ServiceRole: !GetAtt CodeBuildServiceRole.Arn
Pipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
RoleArn: !GetAtt CodePipelineServiceRole.Arn
ArtifactStore:
Type: S3
Location: !Ref ArtifactBucket
Stages:
- Name: Source
Actions:
- Name: Site
ActionTypeId:
Category: Source
Owner: AWS
Version: '1'
Provider: CodeCommit
Configuration:
RepositoryName: !GetAtt Repo.Name
BranchName: main
PollForSourceChanges: 'false'
InputArtifacts: []
OutputArtifacts:
- Name: SourceArtifact
RunOrder: 1
- Name: Build
Actions:
- Name: Docker
ActionTypeId:
Category: Build
Owner: AWS
Version: '1'
Provider: CodeBuild
Configuration:
ProjectName: !Ref Build
InputArtifacts:
- Name: SourceArtifact
OutputArtifacts:
- Name: BuildArtifact
RunOrder: 1
- Name: Deploy
Actions:
- Name: Fargate
ActionTypeId:
Category: Deploy
Owner: AWS
Version: '1'
Provider: ECS
Configuration:
ClusterName: !Ref Cluster
FileName: !Ref ImagesFile
ServiceName: !GetAtt Service.Name
InputArtifacts:
- Name: BuildArtifact
RunOrder: 1
Cluster:
Type: AWS::ECS::Cluster
Properties:
ClusterName: !Select [4, !Split ['-', !Select [2, !Split ['/', !Ref AWS::StackId]]]]
FargateTaskExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- ecs-tasks.amazonaws.com
Action:
- sts:AssumeRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
TaskRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- ecs-tasks.amazonaws.com
Action:
- sts:AssumeRole
TaskDefinition:
Type: AWS::ECS::TaskDefinition
Properties:
ContainerDefinitions:
-
Name: svc
Image: !Sub ${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/${Registry}:latest
PortMappings:
- ContainerPort: 8080
Cpu: 256
ExecutionRoleArn: !Ref FargateTaskExecutionRole
Memory: 512
NetworkMode: awsvpc
RequiresCompatibilities:
- FARGATE
RuntimePlatform:
CpuArchitecture: ARM64
OperatingSystemFamily: LINUX
TaskRoleArn: !Ref TaskRole
Service:
Type: AWS::ECS::Service
Properties:
Cluster: !Ref Cluster
DesiredCount: 0
LaunchType: FARGATE
NetworkConfiguration:
AwsvpcConfiguration:
SecurityGroups:
- !Ref SecG
Subnets: !Ref Subs
ServiceName: !Select [4, !Split ['-', !Select [2, !Split ['/', !Ref AWS::StackId]]]]
TaskDefinition: !Ref TaskDefinition
Outputs:
ArtifactBucketName:
Description: ArtifactBucket S3 Bucket Name
Value: !Ref ArtifactBucket
ArtifactBucketSecureUrl:
Description: ArtifactBucket S3 Bucket Domain Name
Value: !Sub 'https://${ArtifactBucket.DomainName}'
ClusterName:
Value: !Ref Cluster
ServiceName:
Value: !GetAtt Service.Name
RepoUserAccessKey:
Description: S3 User Access Key
Value: !Ref RepoUserKey
RepoUserSecretKey:
Description: S3 User Secret Key
Value: !GetAtt RepoUserKey.SecretAccessKey
BuildArn:
Description: CodeBuild URL
Value: !GetAtt Build.Arn
RepoArn:
Description: CodeCommit Repository ARN
Value: !GetAtt Repo.Arn
RepoName:
Description: CodeCommit Repository NAme
Value: !GetAtt Repo.Name
RepoCloneUrlHttp:
Description: CodeCommit HTTP Clone URL
Value: !GetAtt Repo.CloneUrlHttp
RepoCloneUrlSsh:
Description: CodeCommit SSH Clone URL
Value: !GetAtt Repo.CloneUrlSsh
PipelineUrl:
Description: CodePipeline URL
Value: !Sub https://console.aws.amazon.com/codepipeline/home?region=${AWS::Region}#/view/${Pipeline}
RegistryUri:
Description: ECR Repository URI
Value: !GetAtt Registry.RepositoryUri
TopicArn:
Description: CodeCommit Notification SNS Topic ARN
Value: !Ref Topic
Hope this helps!
I have the following configuration in serverless.yml functioning well:
service: fx-crawler-av
frameworkVersion: '2'
provider:
name: aws
runtime: python3.8
lambdaHashingVersion: 20201221
region: eu-west-1
environment:
S3BucketName: !Ref 'S3BucketFXDataStorage'
AlphaVantageAPIKey: ${ssm:AlphaVantageAPIKey}
package:
exclude:
- requirements.txt
- package.json
- package-lock.json
- node_modules/**
- .serverless/**
- .env/**
functions:
fetch_api:
handler: handler.fetch_api
iamRoleStatements:
- Effect: "Allow"
Action:
- "s3:PutObject"
Resource:
Fn::Join:
- ""
- - "arn:aws:s3:::"
- "Ref": "S3BucketFXDataStorage"
- "/*"
set_ccy_pair_scope:
handler: handler.set_ccy_pair_scope
iamRoleStatements:
- Effect: "Allow"
Action:
- "s3:GetObject"
Resource:
Fn::Join:
- ""
- - "arn:aws:s3:::"
- "Ref": "S3BucketFXDataStorage"
- "/*"
- Effect: "Allow"
Action:
- "s3:ListBucket"
Resource:
Fn::Join:
- ""
- - "arn:aws:s3:::"
- "Ref": "S3BucketFXDataStorage"
get_ccy_list:
handler: handler.get_ccy_list
iamRoleStatements:
- Effect: "Allow"
Action:
- "s3:GetObject"
Resource:
Fn::Join:
- ""
- - "arn:aws:s3:::"
- "Ref": "S3BucketFXDataStorage"
- "/*"
append_parquet_file:
handler: handler.append_parquet_file
iamRoleStatements:
- Effect: "Allow"
Action:
- "s3:GetObject"
- "s3:PutObject"
Resource:
Fn::Join:
- ""
- - "arn:aws:s3:::"
- "Ref": "S3BucketFXDataStorage"
- "/*"
- Effect: "Allow"
Action:
- "s3:ListBucket"
Resource:
Fn::Join:
- ""
- - "arn:aws:s3:::"
- "Ref": "S3BucketFXDataStorage"
resources:
Resources:
S3BucketFXDataStorage:
Type: 'AWS::S3::Bucket'
Properties:
BucketName: !Join
- ''
- - !Ref 'AWS::AccountId'
- '-fx-data-storage'
stepFunctions:
stateMachines:
FXCrawler:
events:
- schedule: cron(35 21 * * ? *)
definition:
Comment: "State machine for crawling AlphaVantage API for FX rates"
StartAt: GetCurrencyList
States:
GetCurrencyList:
Type: Task
Resource:
Fn::GetAtt: [get_ccy_list, Arn]
Next: SetCurrencyPairScope
SetCurrencyPairScope:
Type: Task
Resource:
Fn::GetAtt: [set_ccy_pair_scope, Arn]
Next: mapped_task
mapped_task:
Type: Map
ItemsPath: '$.ccy_pair_scope'
MaxConcurrency: 1
Iterator:
StartAt: FetchAPI
States:
FetchAPI:
Type: Task
Resource:
Fn::GetAtt: [fetch_api, Arn]
Next: WaitForAPI
WaitForAPI:
Type: Wait
Seconds: 60
End: true
End: true
plugins:
- serverless-python-requirements
- serverless-iam-roles-per-function
- serverless-step-functions
custom:
pythonRequirements:
slim: true
dockerizePip: true
useDownloadCache: true
Now I would like to add an S3 event trigger to my append_parquet_file function, e.g.:
append_parquet_file:
handler: handler.append_parquet_file
events:
- s3:
bucket: !Ref 'S3BucketFXDataStorage'
event: s3:ObjectCreated:*
rules:
- suffix: .json
But unfortunately this does not seem to work..
I get a warning during deployment Serverless: Configuration warning at 'functions.append_parquet_file.events[0].s3.bucket': should be string
The deployment fails:
Serverless Error ---------------------------------------
[object Object] - Bucket name must conform to pattern (?!^(\d{1,3}\.){3}\d{1,3}$)(^(([a-z0-9]|[a-z0-9][a-z0-9-]*[a-z0-9])\.)*([a-z0-9]|[a-z0-9][a-z0-9-]*[a-z0-9])$). Please check provider.s3.[object Object] and/or s3 events of function "append_parquet_file".
I tried multiple ways to create and inject the bucket name as variable but did not succeed. Any idea what can go wrong?
You don't need to create bucket resouces, according to the docs, you can specify the bucket and add properties you need and bucket will create and configure automatically to setup the trigger
functions:
resize:
handler: resize.handler
events:
- s3: bucketOne
provider:
s3:
bucketOne:
name: my-custom-bucket-name
# Eventual additional properties in camel case
https://www.serverless.com/framework/docs/providers/aws/events/s3#custom-bucket-configuration
You should reference your bucket as follows:
bucket: ${self:resources.Resources.S3BucketFXDataStorage.Properties.BucketName}
All you need is this: (remove the double quotes)
Resource: !Ref S3BucketFXDataStorage
event: s3:ObjectCreated:*
existing: true
and you have another problem in the policies, try this instead to reference you bucket in the policies:
Resource: !Join ["", [!GetAtt S3BucketFXDataStorage.Arn, "/*" ]]
We have pipelines that pull the code from a CodeCommit repository and builds and deploys the code.
Between the Source stage and the Build stage, there is a manual approval stage and for random reason this stage sometimes gets skipped and the Pipeline continues directly to the Build stage without getting approved or rejected. Some other times even when it's approved, the manual approval stage would remain pending and waiting for approval. Some other times the Approval stage gets triggered simultaneously with the Source as soon as any code is pushed.
Weirdly enough this only happens to two pipelines that were created using a CloudFormation Template.
In a year or working with aws CodePipeline I never experienced such a thing.
Example of Approval stage getting skipped
Example of Approval stage getting triggered at the same time as the Source stage:
The CloudFormation Template:
AWSTemplateFormatVersion: 2010-09-09
Description: >-
Pipeline for React mobile WebApps. Creates a CodePipeline, along with a Deployment S3 that hosts the static and a CodeBuild Project to build and package the project
Parameters:
RepositoryName:
Type: String
Description: Name of repository to build from
RepositoryBranch:
Type: String
Description: The branch to pull from and build
Default: master
AllowedValues:
- master
- staging
ApiURL:
Type: String
Description: domain of the api to be used by the web app
SocketURL:
Type: String
Description: url of the socket to be used by the web app
SentryURL:
Type: String
Description: url of sentry
CloudfrontURL:
Type: String
Description: url of storage
Resources:
CodeBuildProject:
Type: AWS::CodeBuild::Project
DependsOn: CodeBuildRole
Properties:
Name: !Join
- '-'
- - !Ref RepositoryName
- !Ref RepositoryBranch
- build
- project
ServiceRole: !GetAtt CodeBuildRole.Arn
Environment:
Type: LINUX_CONTAINER
ComputeType: BUILD_GENERAL1_SMALL
Image: aws/codebuild/nodejs:8.11.0
EnvironmentVariables:
- Name: S3_URL
Value: !Join
- '-'
- - !Ref RepositoryName
- !Ref RepositoryBranch
- Name: ENV
Value: !Ref RepositoryBranch
- Name: API_DOMAIN
Value: !Ref ApiURL
- Name: SOCKET_URL
Value: !Ref SocketURL
- Name: SENTRY_URL
Value: !Ref SentryURL
- Name: AWS_CLOUDFRONT_URL
Value: !Ref CloudfrontURL
Source:
Type: CODEPIPELINE
BuildSpec: !Sub |
version: 0.2
phases:
install:
commands:
- echo "installing dependencies"
- npm install
pre_build:
commands:
- echo "building static files"
- npm run build
- echo "static files finished building"
build:
commands:
- echo "build phase started"
- aws s3 sync ./dist s3://${DeploymentBucket}/ --cache-control max-age=0
- echo "build complete"
Artifacts:
Type: CODEPIPELINE
CodeBuildRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service:
- codebuild.amazonaws.com
Action:
- sts:AssumeRole
Path: "/"
Policies:
- PolicyName: codebuild-service
PolicyDocument:
Statement:
- Effect: Allow
Action: "*"
Resource: "*"
Version: '2012-10-17'
CodePipeline:
Type: 'AWS::CodePipeline::Pipeline'
DependsOn:
- CodePipelineTrustRole
Properties:
Stages:
- Name: Source
Actions:
- InputArtifacts: []
Name: Source
ActionTypeId:
Category: Source
Owner: AWS
Version: '1'
Provider: CodeCommit
OutputArtifacts:
- Name: MyApp
Configuration:
PollForSourceChanges: false
BranchName: !Ref RepositoryBranch
RepositoryName: !Ref RepositoryName
RunOrder: 1
- Name: Approval
Actions:
- InputArtifacts: []
Name: Approval
ActionTypeId:
Category: Approval
Owner: AWS
Version: '1'
Provider: Manual
OutputArtifacts: []
Configuration:
RunOrder: 1
- Name: BuildAndDeploy
Actions:
- InputArtifacts:
- Name: MyApp
Name: CodeBuild
ActionTypeId:
Category: Build
Owner: AWS
Version: '1'
Provider: CodeBuild
OutputArtifacts:
- Name: MyAppBuild
Configuration:
ProjectName: !Ref CodeBuildProject
RunOrder: 1
- Name: Invalidation
Actions:
- InputArtifacts: []
Name: Invalidate-CloudFront
ActionTypeId:
Category: Invoke
Owner: AWS
Version: '1'
Provider: Lambda
Configuration:
FunctionName: "Cloudfront-Invalidator"
UserParameters: !Sub '{"S3Bucket": "${DeploymentBucket}"}'
OutputArtifacts: []
RunOrder: 1
ArtifactStore:
Type: S3
Location: pipeline-store-bucket
RoleArn: !GetAtt
- CodePipelineTrustRole
- Arn
Name: !Join
- '-'
- - !Ref RepositoryName
- !Ref RepositoryBranch
- pipeline
CodePipelineTrustRole:
Type: 'AWS::IAM::Role'
Description: Creates service role in IAM for AWS CodePipeline
Properties:
AssumeRolePolicyDocument:
Statement:
- Action: 'sts:AssumeRole'
Effect: Allow
Principal:
Service:
- codepipeline.amazonaws.com
Sid: 1
Path: /
Policies:
- PolicyDocument:
Statement:
- Action:
- 's3:GetObject'
- 's3:GetObjectVersion'
- 's3:GetBucketVersioning'
- 's3:PutObject'
Effect: Allow
Resource:
- !Join
- ''
- - 'arn:aws:s3:::'
- 'pipeline-store-bucket'
- '/*'
- !Join
- ''
- - 'arn:aws:s3:::'
- 'pipeline-store-bucket'
- Action:
- 'codecommit:CancelUploadArchive'
- 'codecommit:GetBranch'
- 'codecommit:GetCommit'
- 'codecommit:GetUploadArchiveStatus'
- 'codecommit:UploadArchive'
Effect: Allow
Resource:
- !Join
- ':'
- - arn
- aws
- codecommit
- !Ref 'AWS::Region'
- !Ref 'AWS::AccountId'
- !Ref RepositoryName
- Action:
- 'codebuild:StartBuild'
- 'codebuild:BatchGetBuilds'
- 'codebuild:StopBuild'
Effect: Allow
Resource: '*'
- Action:
- 'cloudformation:DescribeStacks'
- 'cloudformation:DescribeChangeSet'
- 'cloudformation:CreateChangeSet'
- 'cloudformation:DeleteChangeSet'
- 'cloudformation:ExecuteChangeSet'
Effect: Allow
Resource: '*'
- Action:
- 'sns:Publish'
Effect: Allow
Resource: '*'
- Action:
- 'lambda:*'
- 'cloudwatch:*'
- 'events:*'
- 'codepipeline:PutJobSuccessResult'
- 'codepipeline:PutJobFailureResult'
Effect: Allow
Resource: '*'
PolicyName: CodePipelineTrustPolicy
RoleName: !Join
- '-'
- - !Ref AWS::StackName
- CodePipeline
- Role
SourceEvent:
Type: 'AWS::Events::Rule'
Properties:
Description: >-
Rule for Amazon CloudWatch Events to detect changes to the source
repository and trigger pipeline execution
EventPattern:
detail:
event:
- referenceCreated
- referenceUpdated
referenceName:
- !Ref RepositoryBranch
referenceType:
- branch
detail-type:
- CodeCommit Repository State Change
resources:
- !Join
- ':'
- - arn
- aws
- codecommit
- !Ref 'AWS::Region'
- !Ref 'AWS::AccountId'
- !Ref RepositoryName
source:
- aws.codecommit
Name: !Join
- '-'
- - !Ref RepositoryName
- !Ref RepositoryBranch
- SourceEvent
State: ENABLED
Targets:
- Arn: !Join
- ':'
- - arn
- aws
- codepipeline
- !Ref 'AWS::Region'
- !Ref 'AWS::AccountId'
- !Join
- '-'
- - !Ref RepositoryName
- !Ref RepositoryBranch
- pipeline
Id: ProjectPipelineTarget
RoleArn: !GetAtt SourceEventRole.Arn
SourceEventRole:
Type: 'AWS::IAM::Role'
Description: >-
IAM role to allow Amazon CloudWatch Events to trigger AWS CodePipeline
execution
Properties:
AssumeRolePolicyDocument:
Statement:
- Action: 'sts:AssumeRole'
Effect: Allow
Principal:
Service:
- events.amazonaws.com
Sid: 1
Policies:
- PolicyDocument:
Statement:
- Action:
- 'codepipeline:StartPipelineExecution'
Effect: Allow
Resource:
- !Join
- ':'
- - arn
- aws
- codepipeline
- !Ref 'AWS::Region'
- !Ref 'AWS::AccountId'
- !Join
- '-'
- - !Ref RepositoryName
- !Ref RepositoryBranch
- pipeline
PolicyName: CodeStarWorkerCloudWatchEventPolicy
RoleName: !Join
- '-'
- - CodePipeline
- !Ref RepositoryName
- !Ref RepositoryBranch
- CloudWatchEventRule
DeploymentBucket:
Type: 'AWS::S3::Bucket'
Description: >-
S3 Bucket to host the website built from the CodeCommit repository
Properties:
AccessControl: PublicRead
WebsiteConfiguration:
IndexDocument: index.html
ErrorDocument: index.html
BucketName: !Join
- '-'
- - !Ref 'RepositoryName'
- !Ref 'RepositoryBranch'
DeletionPolicy: Delete
DeploymentBucketPolicy:
Type: AWS::S3::BucketPolicy
Description: >-
Policy for the web hosting deployment S3 bucket
Properties:
Bucket: !Ref DeploymentBucket
PolicyDocument:
Statement:
- Sid: PublicReadForGetBucketObjectsxw
Effect: Allow
Principal: '*'
Action: s3:GetObject
Resource: !Join ['', ['arn:aws:s3:::', !Ref 'DeploymentBucket', /*]]
CloudFrontDistribution:
Type: AWS::CloudFront::Distribution
Description: A CloudFront Distribution for the website hosting S3 buckets
DependsOn: DeploymentBucket
Properties:
DistributionConfig:
Origins:
- DomainName: !Join
- '.'
- - !Ref DeploymentBucket
- s3
- amazonaws
- com
Id: !Join
- '-'
- - S3
- !Ref DeploymentBucket
CustomOriginConfig:
HTTPPort: 80
HTTPSPort: 443
OriginProtocolPolicy: https-only
Enabled: true
DefaultCacheBehavior:
AllowedMethods:
- GET
- HEAD
- OPTIONS
- PUT
- POST
- PATCH
- DELETE
ForwardedValues:
QueryString: 'false'
Cookies:
Forward: none
TargetOriginId: !Join
- '-'
- - S3
- !Ref DeploymentBucket
ViewerProtocolPolicy: redirect-to-https
IPV6Enabled: true
DefaultRootObject: index.html
Outputs:
PipelineURL:
Value: !Sub https://console.aws.amazon.com/codepipeline/home?region=${AWS::Region}#/view/${CodePipeline}
Description: URL for the CodePipeline of this stack
SiteUrl:
Value: !GetAtt [DeploymentBucket, WebsiteURL]
Description: URL for the S3 Website
CodeBuildUrl:
Value: !Sub https://eu-west-1.console.aws.amazon.com/codebuild/home?${AWS::Region}#/projects/${CodeBuildProject}/view
Description: URL for the CodeBuild Project of this stack
RepositoryUrl:
Value: !Sub https://eu-west-1.console.aws.amazon.com/codesuite/codecommit/repositories/${RepositoryName}/browse?region=eu-west-1
Description: URL for the repository of this stack
Each CodePipeline stage can have a different version in it so that you can eg. test a new version in beta in parallel with the previous version being deployed to prod. All actions within a stage run the same set of versions (artifacts) though.
CodePipeline will not de-duplicate based on source commit ID because often it's desirable to re-release the same source commit (eg. because the build will pull updated dependencies, or because you want to reset things to a clean state after eg. a rollback).
This means there can be different versions with the same source commit ID in different stages. You go to the "View history" page to verify this.
Example of Approval stage getting skipped
In this print screen I suspect there's 2 changes. C1 was previously approved and is now deployed in "BuildAndDeploy". C2 is the change showing Succeeded in "Source" and waiting for approval in "Approval".
Example of Approval stage getting triggered at the same time as the Source stage:
This is also a case of 2 version. There's one version in Source and another in Approval.
I'm trying to use a CloudFormation template to define CodeBuild and CodePipeline to automate the deployment of a static website hosted in an S3 bucket. To give credit where credit's due, I'm largely following the template from https://dzone.com/articles/continuous-delivery-to-s3-via-codepipeline-and-cod.
The problem I can't resolve is that after I add an environment variable for the Hugo version I'd like to use to create the static site files, I get an error from the AWS console that reads: "Template validation error: Template format error: Unresolved resource dependencies [HUGO_VERSION] in the Resources block of the template."
Why isn't it accepting the HUGO_VERSION environment variable that I define under environment_variables? This is version 0.1 of the format, so it's a little different than the current 0.2, but I've read the following link: https://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html#build-spec-ref-syntax
The thing that really confuses me is that if I remove the lines with ${HUGO_VERSION}, the template is accepted just fine - and then inspection of the CloudWatch logs after building shows (because of the printenv command) HUGO_VERSION=0.49! What gives?
Originally, the template looks like this.
---
AWSTemplateFormatVersion: '2010-09-09'
Description: Pipeline using CodePipeline and CodeBuild for continuous delivery of a single-page application to S3
Parameters:
SiteBucketName:
Type: String
Description: Name of bucket to create to host the website
GitHubUser:
Type: String
Description: GitHub User
Default: "stelligent"
GitHubRepo:
Type: String
Description: GitHub Repo to pull from. Only the Name. not the URL
Default: "devops-essentials"
GitHubBranch:
Type: String
Description: GitHub Branch
Default: "master"
GitHubToken:
NoEcho: true
Type: String
Description: Secret. It might look something like 9b189a1654643522561f7b3ebd44a1531a4287af OAuthToken with access to Repo. Go to https://github.com/settings/tokens
BuildType:
Type: String
Default: "LINUX_CONTAINER"
Description: The build container type to use for building the app
BuildComputeType:
Type: String
Default: "BUILD_GENERAL1_SMALL"
Description: The build compute type to use for building the app
BuildImage:
Type: String
Default: "aws/codebuild/ubuntu-base:14.04"
Description: The build image to use for building the app
Metadata:
AWS::CloudFormation::Interface:
ParameterGroups:
- Label:
default: "Site Configuration"
Parameters:
- SiteBucketName
- Label:
default: "GitHub Configuration"
Parameters:
- GitHubToken
- GitHubUser
- GitHubRepo
- GitHubBranch
- Label:
default: "Build Configuration"
Parameters:
- BuildType
- BuildComputeType
- BuildImage
ParameterLabels:
SiteBucketName:
default: Name of S3 Bucket to create for website hosting
GitHubToken:
default: GitHub OAuth2 Token
GitHubUser:
default: GitHub User/Org Name
GitHubRepo:
default: GitHub Repository Name
GitHubBranch:
default: GitHub Branch Name
BuildType:
default: CodeBuild type
BuildComputeType:
default: CodeBuild instance type
BuildImage:
default: CodeBuild image
Resources:
CodeBuildRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service:
- codebuild.amazonaws.com
Action:
- sts:AssumeRole
Path: "/"
Policies:
- PolicyName: codebuild-service
PolicyDocument:
Statement:
- Effect: Allow
Action: "*"
Resource: "*"
Version: '2012-10-17'
CodePipelineRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service:
- codepipeline.amazonaws.com
Action:
- sts:AssumeRole
Path: "/"
Policies:
- PolicyName: codepipeline-service
PolicyDocument:
Statement:
- Action:
- codebuild:*
Resource: "*"
Effect: Allow
- Action:
- s3:GetObject
- s3:GetObjectVersion
- s3:GetBucketVersioning
Resource: "*"
Effect: Allow
- Action:
- s3:PutObject
Resource:
- arn:aws:s3:::codepipeline*
Effect: Allow
- Action:
- s3:*
- cloudformation:*
- iam:PassRole
Resource: "*"
Effect: Allow
Version: '2012-10-17'
SiteBucket:
Type: AWS::S3::Bucket
DeletionPolicy: Delete
Properties:
AccessControl: PublicRead
BucketName: !Ref SiteBucketName
WebsiteConfiguration:
IndexDocument: index.html
PipelineBucket:
Type: AWS::S3::Bucket
DeletionPolicy: Delete
CodeBuildDeploySite:
Type: AWS::CodeBuild::Project
DependsOn: CodeBuildRole
Properties:
Name: !Sub ${AWS::StackName}-DeploySite
Description: Deploy site to S3
ServiceRole: !GetAtt CodeBuildRole.Arn
Artifacts:
Type: CODEPIPELINE
Environment:
Type: !Ref BuildType
ComputeType: !Ref BuildComputeType
Image: !Sub ${BuildImage}
Source:
Type: CODEPIPELINE
BuildSpec: !Sub |
version: 0.1
phases:
post_build:
commands:
- aws s3 cp --recursive --acl public-read ./samples s3://${SiteBucketName}/samples
- aws s3 cp --recursive --acl public-read ./html s3://${SiteBucketName}/
artifacts:
type: zip
files:
- ./html/index.html
TimeoutInMinutes: 10
Pipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
RoleArn: !GetAtt CodePipelineRole.Arn
Stages:
- Name: Source
Actions:
- InputArtifacts: []
Name: Source
ActionTypeId:
Category: Source
Owner: ThirdParty
Version: '1'
Provider: GitHub
OutputArtifacts:
- Name: SourceArtifacts
Configuration:
Owner: !Ref GitHubUser
Repo: !Ref GitHubRepo
Branch: !Ref GitHubBranch
OAuthToken: !Ref GitHubToken
RunOrder: 1
- Name: Deploy
Actions:
- Name: Artifact
ActionTypeId:
Category: Build
Owner: AWS
Version: '1'
Provider: CodeBuild
InputArtifacts:
- Name: SourceArtifacts
OutputArtifacts:
- Name: DeploymentArtifacts
Configuration:
ProjectName: !Ref CodeBuildDeploySite
RunOrder: 1
ArtifactStore:
Type: S3
Location: !Ref PipelineBucket
Outputs:
PipelineUrl:
Value: !Sub https://console.aws.amazon.com/codepipeline/home?region=${AWS::Region}#/view/${Pipeline}
Description: CodePipeline URL
SiteUrl:
Value: !GetAtt [SiteBucket, WebsiteURL]
Description: S3 Website URL
Now after I try to add an environment variable to use Hugo in the pipeline, the template looks like this.
---
AWSTemplateFormatVersion: '2010-09-09'
Description: Pipeline using CodePipeline and CodeBuild for continuous delivery of a single-page application to S3
Parameters:
SiteBucketName:
Type: String
Description: Name of bucket to create to host the website
GitHubUser:
Type: String
Description: GitHub User
Default: "stelligent"
GitHubRepo:
Type: String
Description: GitHub Repo to pull from. Only the Name. not the URL
Default: "devops-essentials"
GitHubBranch:
Type: String
Description: GitHub Branch
Default: "master"
GitHubToken:
NoEcho: true
Type: String
Description: Secret. It might look something like 9b189a1654643522561f7b3ebd44a1531a4287af OAuthToken with access to Repo. Go to https://github.com/settings/tokens
BuildType:
Type: String
Default: "LINUX_CONTAINER"
Description: The build container type to use for building the app
BuildComputeType:
Type: String
Default: "BUILD_GENERAL1_SMALL"
Description: The build compute type to use for building the app
BuildImage:
Type: String
Default: "aws/codebuild/ubuntu-base:14.04"
Description: The build image to use for building the app
Metadata:
AWS::CloudFormation::Interface:
ParameterGroups:
- Label:
default: "Site Configuration"
Parameters:
- SiteBucketName
- Label:
default: "GitHub Configuration"
Parameters:
- GitHubToken
- GitHubUser
- GitHubRepo
- GitHubBranch
- Label:
default: "Build Configuration"
Parameters:
- BuildType
- BuildComputeType
- BuildImage
ParameterLabels:
SiteBucketName:
default: Name of S3 Bucket to create for website hosting
GitHubToken:
default: GitHub OAuth2 Token
GitHubUser:
default: GitHub User/Org Name
GitHubRepo:
default: GitHub Repository Name
GitHubBranch:
default: GitHub Branch Name
BuildType:
default: CodeBuild type
BuildComputeType:
default: CodeBuild instance type
BuildImage:
default: CodeBuild image
Resources:
CodeBuildRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service:
- codebuild.amazonaws.com
Action:
- sts:AssumeRole
Path: "/"
Policies:
- PolicyName: codebuild-service
PolicyDocument:
Statement:
- Effect: Allow
Action: "*"
Resource: "*"
Version: '2012-10-17'
CodePipelineRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service:
- codepipeline.amazonaws.com
Action:
- sts:AssumeRole
Path: "/"
Policies:
- PolicyName: codepipeline-service
PolicyDocument:
Statement:
- Action:
- codebuild:*
Resource: "*"
Effect: Allow
- Action:
- s3:GetObject
- s3:GetObjectVersion
- s3:GetBucketVersioning
Resource: "*"
Effect: Allow
- Action:
- s3:PutObject
Resource:
- arn:aws:s3:::codepipeline*
Effect: Allow
- Action:
- s3:*
- cloudformation:*
- iam:PassRole
Resource: "*"
Effect: Allow
Version: '2012-10-17'
SiteBucket:
Type: AWS::S3::Bucket
DeletionPolicy: Delete
Properties:
AccessControl: PublicRead
BucketName: !Ref SiteBucketName
WebsiteConfiguration:
IndexDocument: index.html
PipelineBucket:
Type: AWS::S3::Bucket
DeletionPolicy: Delete
CodeBuildDeploySite:
Type: AWS::CodeBuild::Project
DependsOn: CodeBuildRole
Properties:
Name: !Sub ${AWS::StackName}-DeploySite
Description: Deploy site to S3
ServiceRole: !GetAtt CodeBuildRole.Arn
Artifacts:
Type: CODEPIPELINE
Environment:
Type: !Ref BuildType
ComputeType: !Ref BuildComputeType
Image: !Sub ${BuildImage}
Source:
Type: CODEPIPELINE
BuildSpec: !Sub |
version: 0.1
environment_variables:
plaintext:
AWS_DEFAULT_REGION: "US-WEST-2"
HUGO_VERSION: "0.49"
phases:
install:
commands:
- printenv
- echo "Install step..."
- curl -Ls https://github.com/gohugoio/hugo/releases/download/v${HUGO_VERSION}/hugo_${HUGO_VERSION}_Linux-64bit.tar.gz -o /tmp/hugo.tar.gz
- tar xf /tmp/hugo.tar.gz -C /tmp
- mv /tmp/hugo_${HUGO_VERSION}_linux_amd64/hugo_${HUGO_VERSION}_linux_amd64 /usr/bin/hugo
- rm -rf /tmp/hugo*
build:
commands:
- hugo
post_build:
commands:
- aws s3 cp --recursive --acl public-read ./public s3://${SiteBucketName}
artifacts:
type: zip
files:
- ./html/index.html
TimeoutInMinutes: 10
Pipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
RoleArn: !GetAtt CodePipelineRole.Arn
Stages:
- Name: Source
Actions:
- InputArtifacts: []
Name: Source
ActionTypeId:
Category: Source
Owner: ThirdParty
Version: '1'
Provider: GitHub
OutputArtifacts:
- Name: SourceArtifacts
Configuration:
Owner: !Ref GitHubUser
Repo: !Ref GitHubRepo
Branch: !Ref GitHubBranch
OAuthToken: !Ref GitHubToken
RunOrder: 1
- Name: Deploy
Actions:
- Name: Artifact
ActionTypeId:
Category: Build
Owner: AWS
Version: '1'
Provider: CodeBuild
InputArtifacts:
- Name: SourceArtifacts
OutputArtifacts:
- Name: DeploymentArtifacts
Configuration:
ProjectName: !Ref CodeBuildDeploySite
RunOrder: 1
ArtifactStore:
Type: S3
Location: !Ref PipelineBucket
Outputs:
PipelineUrl:
Value: !Sub https://console.aws.amazon.com/codepipeline/home?region=${AWS::Region}#/view/${Pipeline}
Description: CodePipeline URL
SiteUrl:
Value: !GetAtt [SiteBucket, WebsiteURL]
Description: S3 Website URL
EDIT 10/20
Still haven't solved this. I tried to follow the advice given below by matsev, but I'm still getting the same validation error. For completeness, the latest template I'm trying is
AWSTemplateFormatVersion: '2010-09-09'
Description: Pipeline using CodePipeline and CodeBuild for continuous delivery of a single-page application to S3
Parameters:
SiteBucketName:
Type: String
Description: Name of bucket to create to host the website
GitHubUser:
Type: String
Description: GitHub User
Default: "stelligent"
GitHubRepo:
Type: String
Description: GitHub Repo to pull from. Only the Name. not the URL
Default: "devops-essentials"
GitHubBranch:
Type: String
Description: GitHub Branch
Default: "master"
GitHubToken:
NoEcho: true
Type: String
Description: Secret. It might look something like 9b189a1654643522561f7b3ebd44a1531a4287af OAuthToken with access to Repo. Go to https://github.com/settings/tokens
BuildType:
Type: String
Default: "LINUX_CONTAINER"
Description: The build container type to use for building the app
BuildComputeType:
Type: String
Default: "BUILD_GENERAL1_SMALL"
Description: The build compute type to use for building the app
BuildImage:
Type: String
Default: "aws/codebuild/ubuntu-base:14.04"
Description: The build image to use for building the app
Metadata:
AWS::CloudFormation::Interface:
ParameterGroups:
- Label:
default: "Site Configuration"
Parameters:
- SiteBucketName
- Label:
default: "GitHub Configuration"
Parameters:
- GitHubToken
- GitHubUser
- GitHubRepo
- GitHubBranch
- Label:
default: "Build Configuration"
Parameters:
- BuildType
- BuildComputeType
- BuildImage
ParameterLabels:
SiteBucketName:
default: Name of S3 Bucket to create for website hosting
GitHubToken:
default: GitHub OAuth2 Token
GitHubUser:
default: GitHub User/Org Name
GitHubRepo:
default: GitHub Repository Name
GitHubBranch:
default: GitHub Branch Name
BuildType:
default: CodeBuild type
BuildComputeType:
default: CodeBuild instance type
BuildImage:
default: CodeBuild image
Resources:
CodeBuildRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service:
- codebuild.amazonaws.com
Action:
- sts:AssumeRole
Path: "/"
Policies:
- PolicyName: codebuild-service
PolicyDocument:
Statement:
- Effect: Allow
Action: "*"
Resource: "*"
Version: '2012-10-17'
CodePipelineRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service:
- codepipeline.amazonaws.com
Action:
- sts:AssumeRole
Path: "/"
Policies:
- PolicyName: codepipeline-service
PolicyDocument:
Statement:
- Action:
- codebuild:*
Resource: "*"
Effect: Allow
- Action:
- s3:GetObject
- s3:GetObjectVersion
- s3:GetBucketVersioning
Resource: "*"
Effect: Allow
- Action:
- s3:PutObject
Resource:
- arn:aws:s3:::codepipeline*
Effect: Allow
- Action:
- s3:*
- cloudformation:*
- iam:PassRole
Resource: "*"
Effect: Allow
Version: '2012-10-17'
SiteBucket:
Type: AWS::S3::Bucket
DeletionPolicy: Delete
Properties:
AccessControl: PublicRead
BucketName: !Ref SiteBucketName
WebsiteConfiguration:
IndexDocument: index.html
PipelineBucket:
Type: AWS::S3::Bucket
DeletionPolicy: Delete
CodeBuildDeploySite:
Type: AWS::CodeBuild::Project
DependsOn: CodeBuildRole
Properties:
Name: !Sub ${AWS::StackName}-DeploySite
Description: Deploy site to S3
ServiceRole: !GetAtt CodeBuildRole.Arn
Artifacts:
Type: CODEPIPELINE
Environment:
Type: !Ref BuildType
ComputeType: !Ref BuildComputeType
Image: !Sub ${BuildImage}
EnvironmentVariables:
- Name: HUGO_VERSION
Value: '0.49'
Type: PLAINTEXT
Source:
Type: CODEPIPELINE
BuildSpec: !Sub |
version: 0.2
env:
variables:
AWS_DEFAULT_REGION: "US-WEST-2"
phases:
install:
commands:
- printenv
- curl -Ls https://github.com/gohugoio/hugo/releases/download/v${HUGO_VERSION}/hugo_${HUGO_VERSION}_Linux-64bit.tar.gz -o /tmp/hugo.tar.gz
- tar xf /tmp/hugo.tar.gz -C /tmp
- mv /tmp/hugo_${HUGO_VERSION}_linux_amd64/hugo_${HUGO_VERSION}_linux_amd64 /usr/bin/hugo
- rm -rf /tmp/hugo*
build:
commands:
- hugo
post_build:
commands:
- aws s3 cp --recursive --acl public-read ./samples s3://${SiteBucketName}/samples
- aws s3 cp --recursive --acl public-read ./html s3://${SiteBucketName}/
artifacts:
type: zip
files:
- ./html/index.html
TimeoutInMinutes: 10
Pipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
RoleArn: !GetAtt CodePipelineRole.Arn
Stages:
- Name: Source
Actions:
- InputArtifacts: []
Name: Source
ActionTypeId:
Category: Source
Owner: ThirdParty
Version: '1'
Provider: GitHub
OutputArtifacts:
- Name: SourceArtifacts
Configuration:
Owner: !Ref GitHubUser
Repo: !Ref GitHubRepo
Branch: !Ref GitHubBranch
OAuthToken: !Ref GitHubToken
RunOrder: 1
- Name: Deploy
Actions:
- Name: Artifact
ActionTypeId:
Category: Build
Owner: AWS
Version: '1'
Provider: CodeBuild
InputArtifacts:
- Name: SourceArtifacts
OutputArtifacts:
- Name: DeploymentArtifacts
Configuration:
ProjectName: !Ref CodeBuildDeploySite
RunOrder: 1
ArtifactStore:
Type: S3
Location: !Ref PipelineBucket
Outputs:
PipelineUrl:
Value: !Sub https://console.aws.amazon.com/codepipeline/home?region=${AWS::Region}#/view/${Pipeline}
Description: CodePipeline URL
SiteUrl:
Value: !GetAtt [SiteBucket, WebsiteURL]
Description: S3 Website URL
Please check the Environment property of the AWS::CodeBuild::Project in your CloudFormation template. Specifically the EnvironmentVariables allows you to specify environment variables, e.g.
CodeBuildDeploySite:
Type: AWS::CodeBuild::Project
DependsOn: CodeBuildRole
Properties:
Name: !Sub ${AWS::StackName}-DeploySite
Description: Deploy site to S3
ServiceRole: !GetAtt CodeBuildRole.Arn
Artifacts:
Type: CODEPIPELINE
Environment:
Type: !Ref BuildType
ComputeType: !Ref BuildComputeType
Image: !Sub ${BuildImage}
EnvironmentVariables:
- Name: HUGO_VERSION
Value: '0.49'
Type: PLAINTEXT
# More properties...
Now in you can reference the HUGO_VERSION as an environment variable in your buildspec.yml file, e.g.
pre_build:
commands:
- echo HUGO_VERSION $HUGO_VERSION
I believe the following happens:
CloudFormation attempts to resolve ${HUGO_VERSION} as a Parameter of Cloudformation Template as it is within the "!Sub" function.
From AWS documentation on Sub function
To write a dollar sign and curly braces (${}) literally, add an exclamation point (!) after the open curly brace, such as ${!Literal}. AWS CloudFormation resolves this text as ${Literal}.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-sub.html#w2ab1c21c28c59b7
Therefore your build spec phase should be similar to:
phases:
install:
commands:
- printenv
- echo "Install step..."
- curl -Ls https://github.com/gohugoio/hugo/releases/download/v${!HUGO_VERSION}/hugo_${!HUGO_VERSION}_Linux-64bit.tar.gz -o /tmp/hugo.tar.gz
- tar xf /tmp/hugo.tar.gz -C /tmp
- mv /tmp/hugo_${!HUGO_VERSION}_linux_amd64/hugo_${!HUGO_VERSION}_linux_amd64 /usr/bin/hugo
- rm -rf /tmp/hugo*
Hope this helps!