Right now we have few branches in aws code commit one of the repository like master , develop , release and main branches. I don't want any developer commit the code into main branch but i want that branch will be there only because we have code on that. How we will restrict that main branch
You can do so via IAM policy restrictions on the User. Following documents provide the details on the steps required:
https://docs.aws.amazon.com/codecommit/latest/userguide/how-to-conditional-branch.html
https://aws.amazon.com/blogs/devops/refining-access-to-branches-in-aws-codecommit/
Related
I have a number of services in a single GitHub repository, each service has its own CodePipeline on AWS managed through Terraform. Instead of triggering all of the pipelines on commit, I'd like to know how I can trigger each service's pipeline if its directory had any changes on commit, without having to split the services each into its own repository.
I don't think that there's a conditional source stage support per folder at code pipeline as we speak. Just finished checking this documentation about sources in CodePipeline. It does not seem to contain a folder-level filtering.
You could try this CDK-based template solution which showcases a mono-repository, which is composed of multiple services, have different CI/CD pipelines for each service. The solution detects which top level directory the modification happened and triggers the AWS CodePipeline configured to that directory.
This is sad but they might add it in the future. I've also wanted Quality gates, images from readme files in code-commit but these features seem too hard to implement haha.
It ended up being simpler than I had anticipated, there are github actions that do exactly what I needed.
This action checks whether a path had a change, and this action triggers a specific pipeline.
Background
I want to create the following CI/CD flow in AWS and Github, for a react app using Amplify:
A single main branch, with short-lived feature branches and PRs into main.
Each PR triggers its own test environment in Amplify, with its own temporary subdomain, which gets torn down when the PR is merged, as described here.
Merging into main does not automatically trigger a deploy to production.
Instead, there is a separate mechanism (a web page, or amplify command, or even triggers based on git tags) for manually selecting a commit from main to deploy to production.
Questions
It's not clear to me if...
Support for this flow is already built into Amplify (based on the docs I've read, I think the answer is "no", but I'm not sure).
Support for this flow is already built into AWS CodePipeline, or if it can be configured there.
There is another AWS tool that solves this.
I'm looking for answers to those questions, or specific references in the docs which address them.
The answers for Amplify are Yes, Yes, Yes, Partially.
(1) A single main branch, with short-lived feature branches and PRs into main.
Yes. Feature branch deploys. Can define which branch patterns, such as feature*/, you wish to auto-deploy.
(2) Each PR triggers its own test environment in Amplify, with its own temporary subdomain,
Yes. Web Previews for PRs. "A web preview deploys every pull request made to your GitHub repository to a unique preview URL which is completely different from the URL your main site uses."
(3) Merging into main does not automatically trigger a deploy to production.
Yes. Disable automatic builds on main.
(4) Instead, there is a separate mechanism ... for manually selecting a commit from main to deploy to production.
Partially (HEAD only?). Call the StartJob API to manually trigger a build from, say, Lambda. The job type RELEASE starts a new job with the latest change from the specified branch. I am not sure if jobType: MANUAL with a commitId starts a job from an arbitrary commit hash.
Another workaround for 3+4 is to skip the build for an arbitrary commit. Amplify will skip building if [skip-cd] appears at the end of a commit message.
In my experience, I don't think there is any easy way to meet your requirement.
If you are using Gitlab, you can try Gitlab Review Apps to achieve that (I tried before with some scripts)
Support for this flow is already built into Amplify (based on the docs I've read, I think the answer is "no", but I'm not sure).
Check below links, if this help:
https://www.youtube.com/watch?v=QV2WS535nyI
https://dev.to/rajandmr/deploying-react-app-using-aws-amplify-with-ci-cd-pipeline-setup-3lid
Support for this flow is already built into AWS CodePipeline, or if it can be configured there.
For this, you need to create a full your own pipeline. Yes, you can configure your pipeline.
There is another AWS tool that solves this.
If you are okay with Jenkins, then Jenkins will help you to achieve this.
You can deploy Jenkins docker in AWS EC2 and create your pipeline. You can also use the parameterised option for selecting your environment and git branch.
CDK pipelines seems to only work, by default, with one branch. Am I missing something or is there a way to:
have a dev branch to deploy to the Dev account/ env
test branch deploy to Test account/env.
jons-cool-feature-branch to X account/env etc
Ideally we do not want to have to push everything to the master branch to deploy to dev / test, so that we can keep the master branch clean, tidy, and stable.
I have thought about having multiple pipelines, one for dev, one for test, and one for master, this would solve the issue, but doesn’t feel like the cleanest solution.
Are there any recommended patterns?
The AWS-prescribed best practice is to use trunk-based development.
Thus, a single pipeline cannot use multiple branches for deploying to different environments cleanly.
You should look into creating a single pipeline that would in turn create environment-specific pipelines.
Here is a relevant issue in the CDK repo:
https://github.com/aws/aws-cdk/issues/9461
Solution
Building on what #gshpychka said https://stackoverflow.com/a/69812428/12907894
A pipeline that deploys pipleines. I found lots of overcomplicated solutions online, but in the end it turned out to be quite simple.
Just adding extra pipelines, for each branch we wished to deploy.
A core pipeline that builds the branch pipelines.
Only variables that need to change between any of this:
Account ID
env name
branch name
Account Pattern
Build
Dev
Staging
Prod
core-pipeline
branch
master
Webhook -> null (so it doesn't fire on each build)
Deploys:
master-pipeline -> build account
staging-pipeline -> build account
master-pipeline
branch
master
deploys
app stack -> prod account
staging-pipeline
branch
Staging
deploys
app stack -> staging account
Codepipeline cannot branch. It is not designed to do so.
A solution is to have a multi stage pipeline that has manual approval steps in the middle if you absolutely must have multiple environments and a single pipeline.
That is
Source (Dev branch) -> Build/Deploy -> Manual Approval step -> Make use of of a Codebuild or a lambda to move your now tested code (still in the artifact chain) to your test branch for you (ie make use of a git server api to initiate the merge based on the commit message from the initial commit that started the chain -> Another Build./Deploy to your test env (can even do cross account deployment here) -> Manual Approval step -> Repeat as many times as you want until you deploy to Production.
However.... this is entirely a hack. You're better off with multiple pipelines. I would use the CDK to be able to dynamically adjust the cloudformation template for the pipeline itself to handle Dev/Prod and then simply deploy it twice, linking one to the source of Dev and one to the source of Main.
My team has been running into issues with our CodePipeline where features were pushed out into production when they shouldn't have been due to our Docker image patching. A little background on our architecture: Our pipeline has two sources, one for the source code and one for the Docker image builder. Docker builds via CodeBuild and is deployed to dev, test, and then prod environments with manual approval steps in between.
Our Docker image receives monthly patching which triggers the pipeline to execute and is what caused the features to be pushed out. We redesigned our git branching strategy so that our master branch will only contain stable releases, but I could still see this issue potentially occurring again if a specific release date is specified. Is there a way to push out the image patching without pushing out the latest commit?
Can CodePipeline Use a Specific Commit
This is an often requested feature but unfortunately CodePipeline will always bring the latest commit from the selected branch in the Source action.
CodePipeline tied to a single git branch is more of a feature of CodePipeline as the design is more inclined towards Trunk based development [0]. Also, as per the designers of this service, CodePipeline is designed for post-merge/release validation. That is, once your change is ready to be released to production and is merged into your master/main branch, CodePipeline takes over and automatically tests and releases the final merged set of changes. CodePipeline has a lot of features like stage locking, superseding versions, etc. which don't work well for the case where you want to test a change in isolation before it's merged (e.g. feature branch testing or pull request testing.) Therefore there currently isn't a recommended way to do this in CodePipeline.
[0] https://trunkbaseddevelopment.com/
Having said that, there is a way to hack this with S3 Source action in pipeline instead of GitHub/CodeCommit source action. Essentially your pipeline's S3 source action is tied to S3 bucket/key. You can then upload a zip of any specific commit to this S3 bucket/key and trigger the pipeline.
I am new to Code Commit and know a little about git.
I have root access to the repo
Basically, I want to maintain code Commit branch (master).
I want to implement trigger, (if possible) such as:
All developers need to create their own branch from "master" and when they want to push to "master" branch, a new Pull request gets created.
Not sure if possible (create gating),
when I approve the pull request, it goes to master branch.
I hope I am clear
A Million thanks
There’s no way to add IAM permissions on branches. It’s one of the BIG pains of CodeCommit that makes it hard to implement - it drastically reduce security compared to all the other solutions on the marker
Edit:
CodeCommit now supports branch level permissions. You can read more about it here
https://aws.amazon.com/about-aws/whats-new/2018/05/aws-codecommit-supports-branch-level-permissions/
or here
https://aws.amazon.com/blogs/devops/refining-access-to-branches-in-aws-codecommit/
There is currently no branch level permissions on a repository that would prevent someone from pushing specifically to master, or any other branch.
A number of customers are interested in this feature, your request will help us prioritize our work.