The install commands in the Build Step of the AWS CodePipeline do not update when there are changes made in the AWS CDK Code (TypeScript) and are pushed to the repository. The Buildspec section under the Build details of the project has the same configuration as when it was created.
Is there a way to fix it? We've made some changes to the BuildStep CDK but does not take effect on the AWS CodeBuild configuration details. I'm only new to AWS CodeBuild and CodePipeline. Any answer/suggestion would be a great help.
Sample Code
const pipeline = new CodePipeline(this, 'SamplePipeline', {
pipelineName: 'SamplePipeline',
synth: new CodeBuildStep('BuildSynthStep', {
input: source,
buildEnvironment: {
buildImage: codebuild.LinuxBuildImage.STANDARD_5_0
},
installCommands: [
'install_command_1',
'install_command_2',
...
'install_command_n'
],
commands: [
'command_2',
...
'command_n'
],
}
)
});
Artifact Provider: Amazon S3
The self-mutation of a CDK Pipeline is only applied, when you change something on an application stage (precisely CDK Stage) or other phases after the synth codebuild job.
If you have something running before, e.g. unit tests, then you won't get into the self-update job.
So, what are your options now?
Well, changes according to a pipeline itself are mostly done manually.
So you have to re-run a cdk deploy PipelineStack on your local machine with your changes committed to the source branch aside.
Related
I uses CDK to create and deploy pipeline(with AWS CodePipeline https://aws.amazon.com/codepipeline/), and today Source stage stopped working, complaining about insufficient permission, which turns out to be that the Github token is expired. Error:
"Could not access the GitHub repository: "pandaWebsite". The access token might be invalid or has been revoked. Edit the pipeline to reconnect with GitHub."
So I re-genereate the Github token, and updated it in AWS Secrets Manager. And click "Retry" button in pipeline, and it still failed. Eventually I have to run cdk destroy to destroy the pipeline, and run cdk deploy to re-deploy the pipeline, and then it works.
My question is, why I have to destroy and re-deploy the pipeline? I was expecting that once I updated the token in Secrets Manager, it should just work.
More context
AWS Secrets Manager is where I stored the Github token, and my CDK code fetch from it. See code here:
// Add Source stage to fetch code from GitHub repository.
private addSourceStage(
pipeline: codepipeline.Pipeline,
sourceCode: codepipeline.Artifact
) {
pipeline.addStage({
stageName: "Source",
actions: [
new codepipeline_actions.GitHubSourceAction({
actionName: "Checkout",
owner: "yangliu",
repo: "pandaWebsite",
branch: "main",
// read the value from Secrets Manager
oauthToken: CDK.SecretValue.secretsManager(
"github-token"
),
output: sourceCode,
trigger: codepipeline_actions.GitHubTrigger.WEBHOOK,
}),
],
});
}
I have a CDK Pipeline stack that synths and deploys some infrastructure. After the infrastructure is created, I want to build a frontend react app that knows the URL to the newly constructed API Gateway. Once the app is built, I want to move the built files to a newly created S3 bucket.
I have the first two steps working no problem. I use a CfnOutput to get the API URL and the bucket name. I then use envFromCfnOutputs in my shell step to build the react app with the right env variable set up.
I can't figure out how to move my files to a s3 bucket. I've tried for days to figure out something using s3deploy, but run into various permission issues. I thought I could try to just use the aws cli and move the files manually, but I don't know how to give the CLI command permission to add and delete objects. To make things a bit more complicated, My infrastructure is deployed to a separate account from where my pipeline lives.
Any idea how I can use the CLI or another thought on how I can move the built files to a bucket?
// set up pipeline
const pipeline = new CodePipeline(this, id, {
crossAccountKeys: true,
pipelineName: id,
synth: mySynthStep
});
// add a stage with all my constructs
const pipelineStage = pipelineAddStage(myStage)
// create a shellstep that builds and moves the frontend assets
const frontend = new ShellStep('FrontendBuild', {
input: source,
commands: [
'npm install -g aws-cli',
'cd frontend',
'npm ci',
'VITE_API_BASE_URL="$AWS_API_BASE_URL" npm run build',
'aws s3 sync ./dist/ s3://$AWS_FRONTEND_BUCKET_NAME/ --delete'
],
envFromCfnOutputs: {
AWS_API_BASE_URL: myStage.apiURL,
AWS_FRONTEND_BUCKET_NAME: myStage.bucketName
}
})
// add my step as a poststep to my stage.
pipelineStage.addPost(frontendApp);
I want to give this a shot and also suggest a solution for cross account pipelines.
You figured out the first half of how to build the webapp, this works by passing the output of the cloudformation to the environment of a shell action building the app with the correct outputs (e.g. API endpoint url).
You now could add permissions to a CodeBuildStep and attach a policy there to allow the step to do call certain actions. That should work if your pipeline and your bucket are in the same account (and also cross account with a lot more fiddling). But there is a problem with scoping those permissions:
The Pipeline and the Bucket are created in an order where first the Pipeline is created or self-updated, so you do not know the Bucket Name or anything else at this point. It then deploys the resources to its own account or to another account. So you need to assign a name which is known beforehand. This is a general problem and broadens if you e.g. also need to create a Cloudfront Invalidation and so on.
My approach is the following (in my case for a cross account deployment):
Create a Role alongside the resources which allows the role to do things (e.g. ReadWrite S3 bucket, create Cloudfront Invalidation, ...) with a predefined name and allow a matching principal to assume that role (In my case an Account principal)
Code snippet
const deploymentRole = new IAM.Role(this, "DeploymentRole", {
roleName: "WebappDeploymentRole",
assumedBy: new IAM.AccountPrincipal(pipelineAccountId),
});
// Grant permissions
bucket.grantReadWrite(deploymentRole);
2. Create a `CodeBuildStep` which has permissions to assume that role (by a pre-defined name)
Code snippet
new CodeBuildStep("Deploy Webapp", {
rolePolicyStatements: [
new PolicyStatement({
actions: ["sts:AssumeRole"],
resources: [
`arn:aws:iam::${devStage.account}:role/${webappDeploymentRoleName}`,
],
effect: Effect.ALLOW,
}),
],
...
}
3. In the `commands` i do call `aws sts assume-role` with the predefined role name and save the credentials to the environment for following calls to use
Code snippet
envFromCfnOutputs: {
bucketName: devStage.webappBucketName,
cloudfrontDistributionID: devStage.webbappCloudfrontDistributionId,
},
commands: [
"yarn run build-webapp",
// Assume role, see https://stackoverflow.com/questions/63241009/aws-sts-assume-role-in-one-command
`export $(printf "AWS_ACCESS_KEY_ID=%s AWS_SECRET_ACCESS_KEY=%s AWS_SESSION_TOKEN=%s" $(aws sts assume-role --role-arn arn:aws:iam::${devStage.account}:role/${webappDeploymentRoleName} --role-session-name WebappDeploySession --query "Credentials.[AccessKeyId,SecretAccessKey,SessionToken]" --output text))`,
`aws s3 sync ${webappPath}/build s3://$bucketName`,
`aws cloudfront create-invalidation --distribution-id $cloudfrontDistributionID --paths \"/*\"`,
],
4. I do call other aws cli actions like `aws s3 sync ...` with the credentials from step 3. which are now correctly scoped to the actions needed
The ShellStep is likely running under the IAM permissions/role of the Pipeline. Add additional permissions to the Pipeline's role and that should trickle down the AWS CLI call.
You'll also need to probably call buildPipeline before you try to do this:
pipeline.buildPipeline();
pipeline.pipeline.addToRolePolicy(...)
I'm using CDK and am trying to define a CodePipeline that triggers a Lambda Deployment. It doesn't seem as if there are any CDK constructs to achieve this.
I could only find CodeDeployEcsDeployAction and CodeDeployServerDeployAction.
The problems are:
CodeDeployEcsDeployAction: asks for an ECS task definition
CodeDeployServerDeployAction: only accepts an artifact input for properties. However, it won't pick up the appspec file I have defined in the artifact and there are no properties to define the path
(There is no way to do artifact.atPath('appspec.json'))
(Submitted issue 20782 just in case this is an actual request)
Here's my Lambda CodeDeploy setup
const application = new codedeploy.LambdaApplication(
this,
'CodeDeployLambdaApplication',
{
applicationName: 'LambdaApplication',
},
);
const lambdaDeploymentGroup = new codedeploy.LambdaDeploymentGroup(
this,
'AllAtOnceDeployment',
{
application,
alias,
deploymentConfig: codedeploy.LambdaDeploymentConfig.ALL_AT_ONCE,
},
);
You need to use CodeDeploy and Lambda Action:
https://docs.aws.amazon.com/cdk/api/v1/docs/aws-codedeploy-readme.html for the general docs, specifically:
https://docs.aws.amazon.com/cdk/api/v1/docs/aws-codedeploy-readme.html#lambda-applications
Alternatively, if your program is not that complex and does not need all the additional overhead that CodeDeploy brings (having to synth the template and pass it to CodeDeploy and all) then just use a codebuild, that takes a source from your git repo and runs the command cdk deploy your stack
I am creating a aws-amplify app via aws-cdk and everything works fine except it doesn't start a build automatically. If I do a git commit (I have enabled continuous deploys) it will build and run just fine. But on a new aws account with a cdk deploy I have to start the first commit manually...
I had this issue as well while working on the deployment of an amplify hosted application.
After going through the AWS CLI documentation for Amplify, I found the start-job command. This command allows you to start an amplify job for a specific branch.
You can then create an AwsCustomResource that makes an SDK call to start-job.
This is what I ended up with.
const build_trigger = new customResource.AwsCustomResource(this, 'triggerAppBuild', {
policy: customResource.AwsCustomResourcePolicy.fromSdkCalls({
resources: customResource.AwsCustomResourcePolicy.ANY_RESOURCE
}),
onCreate: {
service: 'Amplify',
action: 'startJob',
physicalResourceId: customResource.PhysicalResourceId.of('app-build-trigger'),
parameters: {
appId: amplifyApp.appId,
branchName: master.branchName,
jobType: 'RELEASE',
jobReason: 'Auto Start build',
}
},
});
Note that you should replace amplifyApp.apiId and master.branchName with your own app ID and branch name. I am using the amplify-alpha, so you may need to get the App ID and branch name another way.
Amplify app builds are triggered by push events. Use a CDK Custom Resource or its simpler cousin Trigger to generate a push event during the CDK stack creation.
For a github repo, for instance, your Trigger construct's lambda would call the Github webhook test API, which will trigger the hook with the latest push to the current repository. Both Custom Resources and Triggers can be configured to run only on stack creation. Remember to give your lambda any necessary repo credentials (e.g. set environment variables via a secret).
I am using AWS CDK to deploy codepipeline and codebuild. What I am current doing is to create codebuild in one cloudformation stack and reference the codebuild in the codepipeline in a different cf stack.
Below is my code, I create a codebuild action like:
const action = new actions.CodeBuildAction({
actionName: "MockEventBridge",
type: actions.CodeBuildActionType.BUILD,
input: input,
project: new codebuild.PipelineProject(this, name, {
projectName: mockName,
environment: {
computeType: codebuild.ComputeType.SMALL,
buildImage
privileged: true,
},
role,
buildSpec: codebuild.BuildSpec.fromSourceFilename(
"cicd/buildspec/mockEventbridge.yaml"
),
}),
runOrder: 1,
})
...
const stages = {
stageName, actions: [action]
}
once build the action, I use below code to build codepipeline.
new codepipeline.Pipeline(this, name, {
pipelineName: this.projectName,
role,
stages,
artifactBucket
});
The problem is that both the codebuild project and codepipeline are built into one stack. If I build the codebuild project in a separate cf stack, how can I reference this stack in codepipeline?
when look at the api reference https://docs.aws.amazon.com/cdk/api/v1/docs/#aws-cdk_aws-codepipeline.Pipeline.html, I can't find a way to reference the codebuild arn in codepipeline instance.
Use the codebuild.Project.fromProjectArn static method to import an external Project resource using its ARN. It returns an IProject, which is what your pipeline's actions.CodeBuildAction props expect.
You can use the export value to export the resource Codebuild created in another Stack.
The exported CodeBuild from the first Stack can be imported in the new Stack of CodePipeline.
You can see this page for more info https://lzygo1995.medium.com/how-to-export-and-import-stack-output-values-in-cdk-ff3e066ca6fc